[script help] How to speed up Mesh Creation?

Hello,
I recently came across some mathematical artwork that just made me want to duplicate and explore it, so I just started with python scripting in blender.
Here is the article: http://www.sciencenews.org/articles/20080216/mathtrek.asp

I was trying to simulate the Sierpinski Triangle/Fathauer crystal model. Basically take a cube, create a 1/2-size cube on each of its faces and repeat.

I have never done any python before, so please if you have any comments on my usage of python as well let me know, this is only my second day using it.

Basically, I need to generate the vertex, faces and materials for 11 recursions of the algorithm, which is basically 5^^11+5^^10+5^^9, etc squares big. That is 61,035,156 squares, 488,281,248 vertex’s, 366,210,936 faces.

Even if I cut down the recursions, its still a lot.

At lower recursions its about 60 squares a second, at higher recursions that falls dramatically to like 6 squares a second once its been running awhile. At 6 squares a second it would basically take a month or more to generate a higher-Recursion model.

I have a quad-core CPU and other backup computers availible, and since the algorithm is recursive, i can build “levels” of the final model separately and merge together later. I have thought about just firing up 4 copies of blender(for cpu) and generating different levels but that seems like a kludge. Is there any information on threads in Blender Python scripts?

My other thought was maybe it would be faster to learn the format for a .blend or .dxf file and generate the verts, faces, materials in text?

Is there an api command to just take the created Mesh object and write it to a file rather than loading it into a scene for display/saving as I am currently doing?

Anyway, If you run the script should see what it does(Might i suggest keeping recursion <5 to start for sub-minute generation), any comments on how to improve speed or quality of code would be appreciated !!!

Change the “TOTAL_RUN” variable to modify level of recursion.

Basically each call to the recursive algorthm Rectangle takes a starting vertex(x,y,z), a length and a direction and generates a rectangle based off that information.
Z+ is direction 0. X- is 1, Y+ is 2, X+ is 3, Y- is 4, Z- is 5.


import Blender
import time
from Blender import *

TOTAL_RUN = 3

def Rectangle(x,y,z, length, counter , last_direction,me):
    #print "Entered Rectangle with ",x,y,z,length,counter
    #me = Mesh.New('myMesh2')
    coords=[ [x,y,z],
        [x,y,z+length], 
        [x,y+length,z], 
        [x,y+length,z+length],
        [x+length,y,z],
        [x+length,y,z+length],
        [x+length,y+length,z],
        [x+length,y+length,z+length] ]
    
          
    me.verts.extend(coords)          # add vertices to mesh
    total_verts = len(me.verts) - 8
    total_faces = len(me.faces)
    faces = [ [1+total_verts,3+total_verts,7+total_verts,5+total_verts], [2+total_verts,3+total_verts,7+total_verts,6+total_verts], [4+total_verts,5+total_verts,7+total_verts,6+total_verts], [0+total_verts,2+total_verts,6+total_verts,4+total_verts], [0+total_verts,1+total_verts,5+total_verts,4+total_verts], [0+total_verts,1+total_verts,3+total_verts,2+total_verts] ]
    me.faces.extend(faces)           # add faces to the mesh (also adds edges)
    
    for here in range(total_faces,total_faces+6):
        me.faces[here].mat = counter
    

    if (counter &lt; TOTAL_RUN): 
      for direction in range(0,6):
        if direction == 0:
            xloc = x+length/4
            yloc = y+length/4
            zloc = z+length
            if last_direction == 5: continue
        elif direction == 1:
            xloc = x-length/2
            yloc = y+length/4
            zloc = z+length/4
            if last_direction == 3: continue
        elif direction == 2:
            xloc = x+length/4
            yloc = y+length
            zloc = z+length/4
            if last_direction == 4: continue
        elif direction == 3:
            xloc = x+length
            yloc = y+length/4
            zloc = z+length/4
            if last_direction == 1: continue
        elif direction == 4:
            xloc = x+length/4
            yloc = y-length/2
            zloc = z+length/4
            if last_direction == 2: continue
        elif direction == 5:
            xloc = x+length/4
            yloc = y+length/4
            zloc = z-length/2
            if last_direction == 0: continue
        Rectangle(xloc,yloc,zloc,length/2,counter+1,direction,me)
    Blender.Redraw()

def totalSquares(count):
    if count == 0:
      return 1
    elif (count &gt; 0 and count &lt; 20):
      return pow(5,count) + totalSquares(count-1)
    return 0
    
    
print "
New Run!"
print "--------"
print "For",TOTAL_RUN,"Recursions:"
total = totalSquares(TOTAL_RUN)
print "Squares:",total
est_time = total/60
print "Estimated Time:",round(est_time,2),"sec,",round(est_time/60,2),"min,",round(est_time/3600,2),"hours,",round(est_time/(3600*24),2),"days."
start_time = time.time()
editmode = Window.EditMode() 
if editmode: Window.EditMode(0)
me = Mesh.New('myMesh2')
scn = Scene.GetCurrent()
ob = scn.objects.new(me, 'myObj2')

newmat = Material.New()
me.materials += [newmat]
mat = me.materials[0]
mat.setName('Red')
mat.R = 1.0
mat.G = 0.0
mat.B = 0.0

newmat = Material.New()
me.materials += [newmat]
mat = me.materials[1]
mat.setName('Orange')
mat.R = 1.0
mat.G = 0.5
mat.B = 0.0

newmat = Material.New()
me.materials += [newmat]
mat = me.materials[2]
mat.setName('Yellow')
mat.R = 1.0
mat.G = 1.0
mat.B = 0.0

newmat = Material.New()
me.materials += [newmat]
mat = me.materials[3]
mat.setName('Turq')
mat.R = 0.0
mat.G = 1.0
mat.B = 0.5

newmat = Material.New()
me.materials += [newmat]
mat = me.materials[4]
mat.setName('LBlue')
mat.R = 0.0
mat.G = 0.75
mat.B = 1.0

newmat = Material.New()
me.materials += [newmat]
mat = me.materials[5]
mat.setName('Blue')
mat.R = 0.75
mat.G = 0.75
mat.B = 1.0

# DO recursion
Rectangle(0,0,0,1.,0,0,me)

if editmode: Window.EditMode(1)  # optional, just being nice
finish_time = time.time()
print "Run Time: ", round(finish_time - start_time,2),"seconds"
print "Rate:", round(total / (finish_time - start_time),2),"squares per second"
print "--------"

Sweet idea! I like the clean recursion method. Modified your code slightly. It will now do 6 recursions in 1.55 seconds! 7 recursions in 12.3 seconds. I expect there’s still a lot of optimization to do, but this is a start.
The trick I used is to make a list of the geometry and then create the mesh at the very end. Adding verticies to a mesh is time consuming, so adding to the mesh at each point takes much longer than adding them all at the end. I also switched the redraw to the end, since there’s no geometry to see until it is created. Not as much fun as watching it grow one branch at a time, but much faster.

import Blender
import time
from Blender import *
TOTAL_RUN = 5
def Rectangle(x,y,z, length, counter , last_direction, coords, faces, facemats):
    #print "Entered Rectangle with ",x,y,z,length,counter
    #me = Mesh.New('myMesh2')
    coords += [ [x,y,z],
        [x,y,z+length], 
        [x,y+length,z], 
        [x,y+length,z+length],
        [x+length,y,z],
        [x+length,y,z+length],
        [x+length,y+length,z],
        [x+length,y+length,z+length] ]
 
 
 
    total_verts = len(coords) - 8
    total_faces = len(faces)
    faces += [ [1+total_verts,3+total_verts,7+total_verts,5+total_verts], [2+total_verts,3+total_verts,7+total_verts,6+total_verts], [4+total_verts,5+total_verts,7+total_verts,6+total_verts], [0+total_verts,2+total_verts,6+total_verts,4+total_verts], [0+total_verts,1+total_verts,5+total_verts,4+total_verts], [0+total_verts,1+total_verts,3+total_verts,2+total_verts] ]
 
 
    for here in range(total_faces,total_faces+6):
 facemats += [(here,counter)]
        #me.faces[here].mat = counter
 
    if (counter &lt; TOTAL_RUN): 
      for direction in range(0,6):
        if direction == 0:
            xloc = x+length/4
            yloc = y+length/4
            zloc = z+length
            if last_direction == 5: continue
        elif direction == 1:
            xloc = x-length/2
            yloc = y+length/4
            zloc = z+length/4
            if last_direction == 3: continue
        elif direction == 2:
            xloc = x+length/4
            yloc = y+length
            zloc = z+length/4
            if last_direction == 4: continue
        elif direction == 3:
            xloc = x+length
            yloc = y+length/4
            zloc = z+length/4
            if last_direction == 1: continue
        elif direction == 4:
            xloc = x+length/4
            yloc = y-length/2
            zloc = z+length/4
            if last_direction == 2: continue
        elif direction == 5:
            xloc = x+length/4
            yloc = y+length/4
            zloc = z-length/2
            if last_direction == 0: continue
        Rectangle(xloc,yloc,zloc,length/2,counter+1,direction,coords,faces,facemats)
def totalSquares(count):
    if count == 0:
      return 1
    elif (count &gt; 0 and count &lt; 20):
      return pow(5,count) + totalSquares(count-1)
    return 0
 
 
print "
New Run!"
print "--------"
print "For",TOTAL_RUN,"Recursions:"
total = totalSquares(TOTAL_RUN)
print "Squares:",total
est_time = total/60
print "Estimated Time:",round(est_time,2),"sec,",round(est_time/60,2),"min,",round(est_time/3600,2),"hours,",round(est_time/(3600*24),2),"days."
start_time = time.time()
editmode = Window.EditMode() 
if editmode: Window.EditMode(0)
me = Mesh.New('myMesh2')
scn = Scene.GetCurrent()
ob = scn.objects.new(me, 'myObj2')
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[0]
mat.setName('Red')
mat.R = 1.0
mat.G = 0.0
mat.B = 0.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[1]
mat.setName('Orange')
mat.R = 1.0
mat.G = 0.5
mat.B = 0.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[2]
mat.setName('Yellow')
mat.R = 1.0
mat.G = 1.0
mat.B = 0.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[3]
mat.setName('Turq')
mat.R = 0.0
mat.G = 1.0
mat.B = 0.5
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[4]
mat.setName('LBlue')
mat.R = 0.0
mat.G = 0.75
mat.B = 1.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[5]
mat.setName('Blue')
mat.R = 0.75
mat.G = 0.75
mat.B = 1.0
# DO recursion
coords = []
faces = []
facemats = []
Rectangle(0,0,0,1.,0,0,coords, faces, facemats)
me.verts.extend(coords)
me.faces.extend(faces)
for here in facemats:
        me.faces[here[0]].mat = here[1]
 
if editmode: Window.EditMode(1)  # optional, just being nice
finish_time = time.time()
Blender.Redraw()
print "Run Time: ", round(finish_time - start_time,2),"seconds"
print "Rate:", round(total / (finish_time - start_time),2),"squares per second"
print "--------"

Basically, I need to generate the vertex, faces and materials for 11 recursions of the algorithm, which is basically 5^^11+5^^10+5^^9, etc squares big. That is 61,035,156 squares, 488,281,248 vertex’s, 366,210,936 faces.

You aren’t going to be able to do that unless you have quite an insane amount of ram.
Back of the envelope calculations point to you needing about 35 gigs. That’s just in blender, not considering the use in python or in the calculations.

Wow, thanks! Talk about speed improvements! 6squares / sec to 14,000 squares / sec @ 7 recursions

I seem to run into a calloc error when trying 8 recursions, as i’m guessing the memory management for the arrays is handled in python and not in blender, but I think by sectioning off the recursions and generating in sections might help this, will add code to start midway or just one direction in a bit. As its reflective, can just generate one side and rotate into the other five directions rather than generate them, but havent tested the overhead of doing this and joining resulting meshes, etc.

Thanks a ton, it definitely gives me something to go on, and the syntax of adding an array of arrays to the main array of arrays is very helpful, as is generating the 2dimen array for materials and faces… I’m definitely still learning how python handles its data structures, and to pay more attention to my indenting.

Will keep this thread updated with any further changes, and I’m definitely wanting to see even more optimization if its possible!

Edit:
@IanC Thanks for your calculations, yes I suspected hardware limitations would probably cut me short, but hopefully some techniques for sectioning off recursions, or duplicating them once created, etc might help to get some extra distance out of memory.

Fiddled around some more. Replace the material code with this code if you’d like to have the textures generated automatically, progressing from red, through green, to blue and scaled to the number of recursions.

for idx in range(TOTAL_RUN+1):
 newmat = Material.New()
 me.materials += [newmat]
 newmat.setName(str(idx))
 if float(idx)/TOTAL_RUN &gt; 0.5:newmat.R = 0.
 else: newmat.R = 1 - float(idx)*2/TOTAL_RUN
 newmat.G = 1 - abs(float(idx)*2/TOTAL_RUN - 1)
 if float(idx)/TOTAL_RUN &lt; 0.5:newmat.B = 0.
 else: newmat.B = (float(idx))*2/TOTAL_RUN - 1

Minor updates to script (now centers the generated mesh over obj center), simple statements to determine what level to start at, end at, as well as if you want to lock into one direction(currently Z+ only) and how many levels you want to lock into for.

Got me up to 8 levels, will play around more to get 9. I believe the blender memory management for handling them as objects is much better than the python memory management for handling them as list, so i think generating in smaller batches as vertex counts go up and then simply link-duplicating them might be the way to go.

Actually that gives me an idea, that really only one square mesh needs to be generated, the rest could be linked duplicates with different size and loc modifiers. That should really cut down on memory requirements(but might remove ability for diff materials or increase other overhead which might mean only useful at higher levels where colors tend to blend together or the vertex count dwarfs any object overhead).

Will keep playing, but in mean time I have a really terrible pic(havent even begun to work on a proper render) and the script, thanks!

And as for material code, yes i do appreciate it and had considered, but currently enjoy the exact control over levels that manually specifying provides.


import Blender
import time
from Blender import *
START_RUN = 0 # Recursion level, starts at 1
TOTAL_RUN = 7 # Recursion Level
LOCK_DIRECTION = 0 #is flag, 0 for off, 1 for lock
LOCK_END = 0 #should be recursion level(0 for just first)

def Rectangle(x,y,z, length, counter , last_direction, coords, faces, facemats):
    #print "Entered Rectangle with ",x,y,z,length,counter
    #me = Mesh.New('myMesh2')
    if START_RUN &lt;= counter: coords += [ [x,y,z],
        [x,y,z+length], 
        [x,y+length,z], 
        [x,y+length,z+length],
        [x+length,y,z],
        [x+length,y,z+length],
        [x+length,y+length,z],
        [x+length,y+length,z+length] ]
 
 
 
    total_verts = len(coords) - 8
    total_faces = len(faces)
    if START_RUN &lt;= counter: faces += [ [1+total_verts,3+total_verts,7+total_verts,5+total_verts], [2+total_verts,3+total_verts,7+total_verts,6+total_verts], [4+total_verts,5+total_verts,7+total_verts,6+total_verts], [0+total_verts,2+total_verts,6+total_verts,4+total_verts], [0+total_verts,1+total_verts,5+total_verts,4+total_verts], [0+total_verts,1+total_verts,3+total_verts,2+total_verts] ]
 
 
    for here in range(total_faces,total_faces+6):
         if START_RUN &lt;= counter: facemats += [(here,counter)]
 
    if (counter &lt; TOTAL_RUN): 
      for direction in range(0,6):
        if direction == 0:
            xloc = x+length/4
            yloc = y+length/4
            zloc = z+length
            if last_direction == 5: continue
        elif direction == 1:
            xloc = x-length/2
            yloc = y+length/4
            zloc = z+length/4
            if last_direction == 3: continue
        elif direction == 2:
            xloc = x+length/4
            yloc = y+length
            zloc = z+length/4
            if last_direction == 4: continue
        elif direction == 3:
            xloc = x+length
            yloc = y+length/4
            zloc = z+length/4
            if last_direction == 1: continue
        elif direction == 4:
            xloc = x+length/4
            yloc = y-length/2
            zloc = z+length/4
            if last_direction == 2: continue
        elif direction == 5:
            xloc = x+length/4
            yloc = y+length/4
            zloc = z-length/2
            if last_direction == 0: continue
        if LOCK_DIRECTION == 1 and counter &lt;= LOCK_END and direction != 0: continue
        Rectangle(xloc,yloc,zloc,length/2,counter+1,direction,coords,faces,facemats)
def totalSquares(count):
    if count == 0:
      return 1
    elif (count &gt; 0 and count &lt; 20):
      return pow(5,count) + totalSquares(count-1)
    return 0
 
 
print "
New Run!"
print "--------"
print "For",TOTAL_RUN,"Recursions:"
total = totalSquares(TOTAL_RUN)
print "Squares:",total
est_time = total/60
print "Estimated Time:",round(est_time,2),"sec,",round(est_time/60,2),"min,",round(est_time/3600,2),"hours,",round(est_time/(3600*24),2),"days."
start_time = time.time()
editmode = Window.EditMode() 
if editmode: Window.EditMode(0)
me = Mesh.New('myMesh2')
scn = Scene.GetCurrent()
ob = scn.objects.new(me, 'myObj2')
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[0]
mat.setName('Red')
mat.R = 1.0
mat.G = 0.0
mat.B = 0.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[1]
mat.setName('Orange')
mat.R = 1.0
mat.G = 0.5
mat.B = 0.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[2]
mat.setName('Yellow')
mat.R = 1.0
mat.G = 1.0
mat.B = 0.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[3]
mat.setName('Turq')
mat.R = 0.0
mat.G = 1.0
mat.B = 0.5
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[4]
mat.setName('LBlue')
mat.R = 0.0
mat.G = 0.75
mat.B = 1.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[5]
mat.setName('Blue')
mat.R = 0.75
mat.G = 0.75
mat.B = 1.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[6]
mat.setName('Blue2')
mat.R = 0.75
mat.G = 0.75
mat.B = 1.0
newmat = Material.New()
me.materials += [newmat]
mat = me.materials[7]
mat.setName('White')
mat.R = 1.0
mat.G = 1.0
mat.B = 1.0
# DO recursion
coords = []
faces = []
facemats = []
Rectangle(-.5,-.5,-.5,1.,0,0,coords, faces, facemats)
me.verts.extend(coords)
me.faces.extend(faces)
for here in facemats:
        me.faces[here[0]].mat = here[1]
 
if editmode: Window.EditMode(1)  # optional, just being nice
finish_time = time.time()
Blender.Redraw()
print "Run Time: ", round(finish_time - start_time,2),"seconds"
if start_time != finish_time:
  print "Rate:", round(total / (finish_time - start_time),2),"squares per second"
print "--------"

http://img180.imageshack.us/img180/851/crystalxf8.jpg