Convert an ASCII file containing coordinates to a 3d model

(BlenderNoobey) #1

Hello,

I have some data in the form of a space separated ASCII file containing coordinate data obtained from an atomic force microscope that I would like to visualise using blender.

Basically the coordinates are height values for a 2D matrix (256 x256). With these values I would like to create a surface made up of 256 x 256 points.

I have programmed in java2 and C++ before but have never done anything in python. Where best should I start with this project? Could someone point me towards some good tutorials, I guess this kind of thing has been done before.

Kind Regards,

Zac

0 Likes

(Bao2) #2

First learn very quick some tutorial about python. The best place www.python.org
Then try to understand the scripts like add_mesh_torus.py that is in blender scripts/addon/ folder.
Then use blender console autocomplete feature (you enter bpy. and autocomplete shows you all the childs and so)
Then good luck.

Also don’t forget google “read geometry from file to blender in python” and so…

0 Likes

(RickyBlender) #3

here is an example for pairs XY in text file with one pair per line

Sequential numbers X Y 2 per line .

format is

x1 y1

x2 y2

etc

import bpy
from math import pi,sin,cos

textfile1 = ‘data.txt’

data1 = open(textfile1,‘r’).read().split()
data1= [float(i) for i in data1]

for i in range(0,len(data1),2):
print (data1[i])

salutations

0 Likes

(batFINGER) #4

Howdy,

that kinda reminds me of some of the terrain formats producing surface maps from height data, stored as far as i know in a similar format. There are some scripts for 2.4something that produce these surfaces.

Doing this yourself would be quite simple. As long as you know the row order of the file, add a subdivided plane (256x256) and loop thru and change the z value…(assuming z is up) from your file

Out of interest what is the dimension of the square scanned?

0 Likes

(zeffii) #5

how about posting the whole ascii file? would be great for debugging

0 Likes

(zeauro) #6

Did you try to import your file in blender 2.49 with the Milkshape3D ASCII import ?

0 Likes

(ValterVB) #7

Try this, change name and dimensions. No new line in the file.

Ciao
VB

import bpy 
from mathutils import *

def createFaces(vertIdx1, vertIdx2, closed=False, flipped=False):
    faces = []

    if not vertIdx1 or not vertIdx2:
        return None

    if len(vertIdx1) < 2 and len(vertIdx2) < 2:
        return None

    fan = False
    if (len(vertIdx1) != len(vertIdx2)):
        if (len(vertIdx1) == 1 and len(vertIdx2) > 1):
            fan = True
        else:
            return None

    total = len(vertIdx2)

    if closed:
        # Bridge the start with the end.
        if flipped:
            face = [
                vertIdx1[0],
                vertIdx2[0],
                vertIdx2[total - 1]]
            if not fan:
                face.append(vertIdx1[total - 1])
            faces.append(face)

        else:
            face = [vertIdx2[0], vertIdx1[0]]
            if not fan:
                face.append(vertIdx1[total - 1])
            face.append(vertIdx2[total - 1])
            faces.append(face)

    # Bridge the rest of the faces.
    for num in range(total - 1):
        if flipped:
            if fan:
                face = [vertIdx2[num], vertIdx1[0], vertIdx2[num + 1]]
            else:
                face = [vertIdx2[num], vertIdx1[num],
                    vertIdx1[num + 1], vertIdx2[num + 1]]
            faces.append(face)
        else:
            if fan:
                face = [vertIdx1[0], vertIdx2[num], vertIdx2[num + 1]]
            else:
                face = [vertIdx1[num], vertIdx2[num],
                    vertIdx2[num + 1], vertIdx1[num + 1]]
            faces.append(face)

    return faces

def Disegna():
    ############################################################
    # CHANGE HERE PATH AND FILE NAME
    ############################################################
    FileName="D:/Temp/100 Continuo.csv"

    Lines=10 #Change here
    Points=10 #Change here
    Verts=[]
    FirstRow=[]
    SecondRow=[]
    Faces=[]
    FaceTemp=[]
    
    f=open(FileName,'rb')
    DatiLetti=f.read() 
    Matrice = [float(s) for s in DatiLetti.split()] 
    f.close
    
    for y in range(Lines):
        for x in range(Points):
            z=Matrice[y*Points+x]
            Verts.append(Vector((float(x),float(y),float(z))))
    
    for Vvertici in range(Lines-1):
        for Hvertici in range(Points):
            FirstRow.append(Vvertici*Points+Hvertici)
            SecondRow.append((Vvertici+1)*Points+Hvertici)
        FaceTemp=createFaces(FirstRow,SecondRow,closed=False,flipped=False)
        Faces.extend(FaceTemp)
        FirstRow=[]
        SecondRow=[]
        
    mesh = bpy.data.meshes.new("MyMesh")
    mesh.from_pydata(Verts, [], Faces)
    mesh.update()
    ob_new = bpy.data.objects.new("MyObject", mesh)
    ob_new.data = mesh
    scene = bpy.context.scene
    scene.objects.link(ob_new)
    scene.objects.active = ob_new
    ob_new.select = True
        
        
Disegna()

0 Likes

(BlenderNoobey) #8

Wow,

Thanks to everyone who has replied to this thread. I’m slowly working my way through all the advice starting with learning the basics of python scripting.

I will in due course post up an example txt file of the space separated ASCII data. The atomic force microscope (AFM) is capable of measuring surfaces on the nanoscale (~1nm divided by 256 x 256 points).

I am currently doing lower resolution scans around 20 x 20 micrometer. I have a few nice scans of a silicon wafer surface with etched circular pits which I will upload soon.

Thanks for all your interest. I hope one day to be able to contribute usefully.

PS: I tried copy pasting ValterVB’s code but get the following error:

“No module named mathutils”

?

0 Likes

(BlenderNoobey) #9

For some reason I do not have permission to upload attachments and the ASCII file data is a little to large to be posted within the reply box.

So here is a link to example data on rapidshare:

rapidshare.com/files/438749023/zac14dec10.txt

The above line is an http but for some reason the forum won’t allow links to be posted.

Either way I tried.

0 Likes

(ValterVB) #10

“No module named mathutils”

Strange, I try it also with the original Blender 2.55 Beta on Blender.org and it Work :confused:

I have downloaded your file, I must change something in my example (more late).
I have a question:
First line of your file:

Width: 20.00 µm

Height: 20.00 µm

Value units: m

I must divided by 1000000 the value in file (for consistency with Width and Height)?

Ciao
VB

0 Likes

(zeffii) #11

ValterVB has written a nice importer :slight_smile:

0 Likes

(ValterVB) #12

Is correct?
http://lh4.ggpht.com/_uGKL7dhNVTY/TRJicBxdBDI/AAAAAAAAAqQ/VL2ALDnGMHY/s800/Result.PNG

The code isn’t complete because you must delete the first 4 lines from the file by hand. After the import you must set manually the dimension on 20 x 20.

Ciao
VB

import bpy 
from mathutils import *

def createFaces(vertIdx1, vertIdx2, closed=False, flipped=False):
    faces = []

    if not vertIdx1 or not vertIdx2:
        return None

    if len(vertIdx1) < 2 and len(vertIdx2) < 2:
        return None

    fan = False
    if (len(vertIdx1) != len(vertIdx2)):
        if (len(vertIdx1) == 1 and len(vertIdx2) > 1):
            fan = True
        else:
            return None

    total = len(vertIdx2)

    if closed:
        # Bridge the start with the end.
        if flipped:
            face = [
                vertIdx1[0],
                vertIdx2[0],
                vertIdx2[total - 1]]
            if not fan:
                face.append(vertIdx1[total - 1])
            faces.append(face)

        else:
            face = [vertIdx2[0], vertIdx1[0]]
            if not fan:
                face.append(vertIdx1[total - 1])
            face.append(vertIdx2[total - 1])
            faces.append(face)

    # Bridge the rest of the faces.
    for num in range(total - 1):
        if flipped:
            if fan:
                face = [vertIdx2[num], vertIdx1[0], vertIdx2[num + 1]]
            else:
                face = [vertIdx2[num], vertIdx1[num],
                    vertIdx1[num + 1], vertIdx2[num + 1]]
            faces.append(face)
        else:
            if fan:
                face = [vertIdx1[0], vertIdx2[num], vertIdx2[num + 1]]
            else:
                face = [vertIdx1[num], vertIdx2[num],
                    vertIdx2[num + 1], vertIdx1[num + 1]]
            faces.append(face)

    return faces

def Disegna():
    ############################################################
    # CHANGE HERE PATH AND FILE NAME
    ############################################################
    FileName="D:/Temp/zac14dec10.txt"

    Lines=256 #Change here
    Points=256 #Change here
    Verts=[]
    FirstRow=[]
    SecondRow=[]
    Faces=[]
    FaceTemp=[]
    Matrice=[]
    a=0
    for text in open(FileName,"r"):
        temp1=text.lstrip()
        temp=temp1.split()
        for f in temp:
            Matrice.append(float(f)*1000000)

    for y in range(Lines):
        for x in range(Points):
            z=Matrice[y*Points+x]
            Verts.append(Vector((float(x),float(y),float(z))))
    
    for Vvertici in range(Lines-1):
        for Hvertici in range(Points):
            FirstRow.append(Vvertici*Points+Hvertici)
            SecondRow.append((Vvertici+1)*Points+Hvertici)
        FaceTemp=createFaces(FirstRow,SecondRow,closed=False,flipped=True)
        Faces.extend(FaceTemp)
        FirstRow=[]
        SecondRow=[]
        
    mesh = bpy.data.meshes.new("MyMesh")
    mesh.from_pydata(Verts, [], Faces)
    mesh.update()
    ob_new = bpy.data.objects.new("MyObject", mesh)
    ob_new.data = mesh
    scene = bpy.context.scene
    scene.objects.link(ob_new)
    scene.objects.active = ob_new
    ob_new.select = True
       
        
Disegna()

0 Likes

(PKHG) #13

the script of ValterVB works nicely, I changed it a little bit
Matrice = [10000000*float(s) for s in DatiLetti.split()]
And got more or less the same picture on the 256x256 data-file :wink:

0 Likes

(zeffii) #14

i came up with this, kind of a learning exercise :

import bpy

# Constants, for testing.
ZOOM_RES = 1e4
XY_SPREAD = 2e-3

# assumes a square matrix of Z values
def CreateMeshUsingMatrix(VertIndices, Verts):

    Faces = []
    dims = len(VertIndices)

    # going to use current row and next row all the way down
    for row in range(dims-1):
        #now take row and row+1
        for col in range(dims-1):
            #we generate faces clockwise from 4 Verts
            val1 = VertIndices[row][col]
            val2 = VertIndices[row][col+1]
            val3 = VertIndices[row+1][col+1]
            val4 = VertIndices[row+1][col]
            face = [val1, val2, val3, val4]
            Faces.append(face)
        

    ## TAKEN FROM USER    ValterVB 
    # create new mesh structure
    mesh = bpy.data.meshes.new("Relief")
    mesh.from_pydata(Verts, [], Faces)
    mesh.update()
    # create an object from this mesh
    new_object = bpy.data.objects.new("Ascii Data", mesh)
    new_object.data = mesh
    # adding object called "Ascii Data" to the scene
    scene = bpy.context.scene
    scene.objects.link(new_object)
    # deals with selecting
    scene.objects.active = new_object
    new_object.select = True
    ## / TAKEN FROM USER


# i am assuming the input data is 256*256
def startASCIIimport():
    
    VertIndices = []
    heightMatrix = []

    # Deals with opening the file and taking data line by line.
    filename = "zac14dec10.txt"
    f = open(filename, 'rU')
    for m in f:
        if m[0] != "#":
            xCol = m.split()
            floatVals = []
            for val in xCol:
                floatVals.append(float(val)*ZOOM_RES)
            heightMatrix.append(floatVals)
    # We have all important data in a usable structure now, close the file
    f.close()

    # this supposes that X, Y separation are going to be the same.
    xy_val = []
    for i in range(256):
        xy_val.append(i*XY_SPREAD)

    # Generates the (x,y,height) matrix, no need for Vector(...) 
    yVal = 0
    vertNum = 0
    rawVertCollection = []
    for height_x in heightMatrix:
        xVal = 0
        vertRow = []
        for item in height_x:
            t_vertice = (xy_val[xVal], -xy_val[yVal], heightMatrix[yVal][xVal])
            rawVertCollection.append(t_vertice)
            
            vertRow.append(vertNum)
            xVal+=1
            vertNum+=1
        yVal+=1
        VertIndices.append(vertRow)

    # done here, lets make a mesh! 
    CreateMeshUsingMatrix(VertIndices, rawVertCollection)

startASCIIimport()

but my image is flipped, dunno which of the two is right.

http://img262.imageshack.us/img262/9024/fdcrq.png

0 Likes

(zeffii) #15

ok, i made a little Importer for File->Import

http://www.digitalaphasia.com/code/Python/blenderPY/io_import_mesh_afm_ascii.rar
unpack the rar into …/scripts/addons/ so you have a folder inside addons called io_import_mesh_afm_ascii. inside that folder you should have 2 files EMC_12_FORIMPO.py and init.py.

you can enable the addon from preferences-> Import/Export called Import AFM mesh
even if you never use this, it was fun to code :slight_smile:

0 Likes

(BlenderNoobey) #16

Brilliant,

Yes that image is 100% correct. The indentation you see is a pitt in a silicon wafer that was made by etching the surface with hydrofluroic acid.

Its a really great job you guys have done and very useful. I will apply myself to understanding the script but you have achieved in hours what would have taken me weeks. If I get any more interesting images I will post up the ascii files here for you.

I am also as a hobby project going to attempt to create a macroscale imaging device (perhaps using sound) rigged to an arduino board to create similar files for surface profiling and maybe fast model creation.

I’ll let you know if I get it done.

Kind Regards,

Zac

0 Likes

(blenderfanz) #17

hi,

've come accross this excellent work while searching on a similar requirement. But I couldn’t see the addon installed if I unzip to the path mentioned. However, if I place each of .py files in same path I could see it installed under preferences->Import(New tab created on left side of preferences window) but couldn’t enable it (not able to select checkbox…)

I am using blender 2.59. Does the script needs to modified to be compatible with this latest version of blender…?

Also, I need to import a file containing x,y,z coordinates to build a mesh out of it in blender. I understand there is need to write python script to achieve the same (have no idea about python)…

Any help is appreciated.

Thanks,

0 Likes

(zeffii) #18

hey, i updated this a while back but didn’t release a public version when the api was still updating a lot. But this version works nicely as an addon.

here: https://dl.dropbox.com/u/3397495/phys/io_import_mesh_afm_ascii.rar
Located in Import/Export: AFM Mesh

this is the textfile i tested it against:
https://dl.dropbox.com/u/3397495/phys/zac14dec10.txt

Along with the script in the following link you can derive a color map from the zheights of the mesh: https://gist.github.com/2942269

https://dl.dropbox.com/u/3397495/phys/interesting_maybe.png

0 Likes

(zeffii) #19

updated the zip to include a ‘Make Color Map’ option in the import window. (may be slow)

0 Likes