How to use external values as input for color ramp shader node?

[Derek_Jacob]

All,

I’ve recently started dabbling in Blender to do some scientific visualization, like ParaView but 1000x prettier

At the moment I’m trying to tackle adding a color ramp to a 3D surface. As of now, I’ve been able to figure out how to use the surface’s vertex geometry as the input for coloring. My current node setup looks something like this:

This separates the z-values from the geometry and uses them as input for the color map (after normalizing them in the map range node).

I want to do this exact same process, but instead of using the z-values, I want to use arbitrary values.

If I have a surface with n points, I will have an array with n values. These may represent temperature, salinity, velocity etc.

The node setup would look similar but have some kind of vector/array input into the map range node, removing the geometry + separate xyz nodes.

How could I go about doing this? Both GUI and python based solutions welcome! Would like to learn it first by GUI but will eventually be coded.

I’ve seen a similar question asked here but the solution seems to suggest that the process would have to be done for each value individually, as opposed to normalizing the array of values to 0…1 and using that? Similar Question: here

PS - if I’m way off and there’s a totally different and better way of doing this, please share!

[Photox (Tim Gregory)]

Do you want to hand paint the vertices, or do you already have separate data you want to link to them?

[Derek_Jacob]

Definitely don’t want to hand paint, I would like to automate this.

I have an array of values equal in length to number of points in the surface, or number of faces on the surface, whichever is easier to use.

Thanks, any suggestions appreciated, I’m totally lost. Can’t find anything

[Photox (Tim Gregory)]

explain exactly what your external data set is. does it contain both location and property (temperature) data,? if so it’s not too tricky to loop through it with a python script and move the verts to the location and then assign a vertex color based on the other property, however a bunch of verts is basically a point cloud, it cannot be rendered, verts and edges cannot be rendered, only faces.

[Derek_Jacob]

Right, sorry I was unclear on that.

the external data set will be coming as a point cloud with XYZ+other data associated with each point.

I’ve already written a script to take in the points, and generate a surface…the gist of it is here (using random dummy data):

def make_surface():
    x = np.random.rand(100) * 100
    y = np.random.rand(100) * 100
    z = np.random.rand(100) * 10
    c = np.random.rand(100)
    pts = np.array([x,y,z,c]).T
    for i,pt in enumerate(pts):
        v = [Vector((pt[0], pt[1], pt[2]))]
        m = bpy.data.meshes.new(name='genpoint'+str(i))
        m.from_pydata(v, [], [])
        object_data_add(bpy.context,m)
    bpy.ops.object.select_all(action='DESELECT')
    for ob in bpy.context.scene.objects:
        if 'genpoint' in ob.name:
            ob.select_set(True)
    bpy.ops.object.join()
    bpy.context.selected_objects[0].name = 'points'
    bpy.ops.tesselation.delaunay()
    bpy.context.selected_objects[0].name = 'surface'

This is the geometry I used in the above node setup. For context, this is like coastal modelling output. The xyz is the model mesh + bathymetry of the modelled area. Then, for each point I also have model output (temp, salinity, etc…).

Color ramping the actual geometry is not an issue, I make the surface and use the XYZ info as above in the node setup.

For other renders, I would essentially make the surface flat (z=0, or a flat water surface) then color it by the different scalar values instead of z.

Any tips appreciated.

[Photox (Tim Gregory)]

The most straightforward method would be to use vertex colors. if you switch to vertex paint mode, a map (Array really) is created called ‘col’ you can rename it in the context panel
(green triangle) usually you paint by hand in blender, and then use it within the material nodes by using an attribute node, and typing in ‘col’ or your custom name. in your case you would assign the rgb data to the vertex color using python. You can still render the points without scaling the z axis to 0 though,

[Derek_Jacob]

Thanks, I’m extremely new.

I will give this ago, but if you are able to put together a step by step with a couple screenshots, even just the buttons I need to click that would be appreciated.

As a newbie, Blender can be quite daunting and I still don’t know where everything is or how to use it.

You’v given me the framework and with enough googling I should be able to figure it out, but any further advice is appreciated.

[Secrop]

Yup, as Photox sugested, vertex colors are an option. There’s a small problem, though, about storing values into vertex color layers: they are sent to the render engine with sRGB transformations, so the values won’t end up linear in the shading process.
My personal solution, is a custom build of blender, where cycles stores the vertex colors without sRGB, so with the ‘Attribute’ node, you get exactly what you put in each vertex layer.

btw, vertex colors are stored in loops, not in vertices. so it happens for a vertex to have so many colors as many loops it belongs.

[gorion103 (Zuorion)]

wait, that conversion to srgb in cycles vertex color is done on purpose and there is no option to disable that?

i though that usage vertex color as actual color is only fraction of real usecase

[Derek_Jacob]

can you explain the implications of what this means for me? literally been in this space for a few weeks, really don’t understand waht this would do

[Derek_Jacob]

About to give it a go with the suggestions here, and some code I was able to track down… will come back with some more info/updates

[gorion103 (Zuorion)]

oh, i misunderstood your post

[Secrop]

@gorion103

wait, that conversion to srgb in cycles vertex color is done on purpose and there is no option to disable that?

I’d like that it would chose between converting and not converting based on the output connections (vector vs color)… It’s possible to implement some check when storing the data in the engine data, or even duplicate the data as sRGB, or even convert the values to sRGB for each sample… But as a simple solution, that diff is the shortest path.

@Derek_Jacob

can you explain the implications of what this means for me?

Using vertex colors allows you to add information to each vertex in each poly (loop), and it’s accessible when sampling some point on that poly.

For example, i have a small script that stores the angles between a face and their neighbours (which I can later manipulate with osl), and it goes like this:

def storeData():
    ob = bpy.context.active_object
    if ob.mode=='OBJECT':
        bpy.ops.object.editmode_toggle()
    else:
        return
    bm = bmesh.from_edit_mesh(ob.data)
    bmesh.ops.triangulate(bm, faces=bm.faces)
    edgedata=[]
    for edge in bm.edges:
        edgedata.append(edge.calc_face_angle()/pi)
    del bm
    bpy.ops.object.editmode_toggle()
    
    col=ob.data.vertex_colors.new(name='EdgeData')
    for p in ob.data.polygons:
        for lp in p.loop_indices:
            for cl in p.loop_indices:
                col.data[cl].color[lp-p.loop_start]=edgedata[ob.data.loops[lp].edge_index]

On the shader part, i just use the ‘Attribute’ node in conjunction with the ‘SeparateXYZ’ to get that angle for each edge of the triangulated mesh.

You can use it for other kinds of values.

[Derek_Jacob]

this is great, and pretty much exactly what I was hoping for…one problem now though (I now appreciate your comment on attributes being stored with sRGB transformation, because now my surface ends up looking like this without linear shading:

is there any way for me to smooth this or get around it? I know you have a personal solution, but I looked at it and have no idea what that is and is way out of my knowledge zone lol.

is there a simpler way or some kind of post-coloring smoothing I can do?

or maybe you could walk me through how to do your solution

[Derek_Jacob]

PS I also posted a question on stack exchange (ended up making an acc on here and posting because there seems to be quite a bit of traffic, figured why not)

they’re the same question, perhaps some information in both that can help someone answer this

https://blender.stackexchange.com/questions/174943/how-to-use-external-values-as-input-for-color-ramp-shader-node/175030#175030

[Secrop]

Seams you got a good answer in SE!

Here’s another alternative for that nested loop:

colors_layers = {'L1':[(r0, g0, b0, 1.0), (r1, g1, b1, 1.0), ... (rn, gn, bn, 1.0)], 'L2': [(....)]}
for layer in mesh.vertex_colors:
    for index, loop in enumerate(layer.data):
         loop.color = colors_layers[layer.name][mesh.loops[index].vertex_index]

[Photox (Tim Gregory)]

Do you have smooth shading enabled (it is flat by default) in the Object menu (top left) click ‘shade smooth’

[Derek_Jacob]

All,

Thanks everyone for the help.

I was able to figure it out from a combination of suggestions here and on my stack post, see answer here.

Essentially, I have generated colors based on external data (same length as n vertices in surface) then added them as vertex color layers to the surface object. I then rigged up a node setup to use the vertex colors as the color input to my main shader node.

I’m still playing with various settings to make it look the way I want, but below is the sample code node setup, and what the result looks like.

Code:

import bpy
import bmesh

obj = bpy.context.selected_objects[0]  # Get the currently selected object

vertex_colors = obj.data.vertex_colors
while vertex_colors:
    vertex_colors.remove(vertex_colors[0])  # Remove all vertex color layers form the mesh data

bm = bmesh.new()  # Create new bmesh object
bm.from_mesh(obj.data)  # Init the bmesh with the current mesh

verts_count = len(bm.verts)  # Get the number of vertices in the mesh 
# Not obligatory, I just use this to populate the vertex color layers)

# This dictionary will map each vertex color layer name to the data for each vertex
data_layers = {
    'Temperature':  # vertex color layer name 1
        [
        (1, 0, 0, 1),  # Color of vertex index 0
        (0, 1, 0, 1),  # Color of vertex index 1
        (0, 0, 1, 1),  # Color of vertex index 2
        (1, 1, 1, 1),  # Color of vertex index 3...
        ],    
    'Velocity':  # vertex color layer name 2
        [(i / verts_count, i / verts_count, i / verts_count, 1) for i in range(verts_count)],
    'Salinity':  # vertex color layer name 3
        [(i / verts_count, 1 - (i / verts_count), 0.5, 1) for i in range(verts_count)],        
    }

color_layers = {}
for layer_name in data_layers:
    # Create the vertex color layers and store them in a dictionary for easy access by name
    color_layers[layer_name] = bm.loops.layers.color.new(layer_name)

for layer_name, layer in color_layers.items():
    # Loop over each vertex color layer
    for v in bm.verts:
        # Loop over every vertex
        for loop in v.link_loops:
            # Loop over every loop of this vertex (the corners of the faces in which this vertex exists)  
            try:
                loop[layer] = data_layers[layer_name][v.index]
                # Get the value from the init dictionary
            except IndexError:
                loop[layer] = (1, 0, 1, 1)
                # Set a placeholder (magenta) value if the mapping array is not long enough

bm.to_mesh(obj.data)  # Give the data back to the actual mesh

Node setup:

Example of colored 3D surface:

Same data, but with flat surface:

Once I develop this a bit further I’ll share it publicly.

[Derek_Jacob]

Thanks, will probably use this instead of what I currently have.

[Derek_Jacob]

lol jeez…I’m new to 3d graphics/rendering in general but even more so for Blender. Did not know about that. I’ve since developed my code since I made the render you were referring to so can’t test to see if it would have solved it…probably would, but this suggestion at least makes my shading look alot better now.