Blender 2.67 Pynodes

Nope, I haven’t dug into PyNodes at all yet beyond skimming that wiki page, just making guesses here mostly.

https://twitter.com/tonroosendaal : it was an error, Pynodes are a target for 2.67 :slight_smile:
I downloaded and build the latest svn of pynodes, I found no way to do what is shown in the video.

Edit : Lukas did a lot of things, but I see none of them in 2.66 despite most of them are 2+ yeras old things working on 2.5x :
http://vimeo.com/14355450 (pynodes ?)
http://vimeo.com/13495148 (particle nodes ?)
Or am I wrong ? If yes, where are they ?

That’s not pynodes in the videos. It’s just some of Lukas previous experiments.

AFAIK pynodes is not equivalent to XSI’s ICE, but it’s a step in the right direction to getting there. Pynodes does not add any features/functionality that blender users can use, but it enables other developers to more easily code custom nodes etc.

Lukas posted this on the maillist in august 2011. Maybe not 100% accurate now, but it answers some of the questions in this thread:

I’ve been doing some work on the node system in my branch(es), part of which is also related to “pynodes” [1]. There is at this time no feature comparable to 2.4x python script nodes, the changes are more fundamental than “simply” adding a node type for script links.

The core change i made is that node types, as well as tree and socket types, are defined as dynamically registered information structs,which can be done from C code as well as python scripts (hence"pynodes"). It works much in the same way as operator, menu and panel types are currently defined from python scripts. The idea is that this will allow external engines (e.g. renderers) to define nodes outside of the blender core code and add a few simple python script addons to make them available in the editor, instead of having to modify the DNA files and other core stuff.

Note that this is not directly tied to a particular “execution” system, such as a renderer (be it BI or Cycles), the compositor orcough texture nodes. In order to get this functionality there will need to be an extended node base class that supports the execution mechanism, so python scripts can implement the appropriate function.

However, i’m currently busy working on particle nodes [2] (which are also based on the pynodes branch), so i won’t implement this any time soon. Beside that there is lots of other things todo before this code is anywhere near trunk ready, currently all previous node types are disabled even! If somebody would want to continue working on this code i’d be glad to give advice and any help i can :slight_smile:

To clear up a couple things…

First: The current implementation of pynodes (what will go into trunk) doesn’t have an execute() method or anything like that one could override to get them to do useful things. Their main goal is to ‘look pretty’ and be used by other code that walks the node tree and turns a node type + links into something useful.

Second: It would be fairly easy to add an execute() method to them – if someone was into that sort of thing – using the operators C/py-binding code as a guide. Stupid easy actually, I had that working when I did a pynodes experiment a while back but managed to lose that work during The Makefile Incident that took out my /home directory.

Third: (And this doesn’t really clear up anything) I have this crazy idea on how to do an implementation of execute() that involves using libjit (and my python wrappers around said library) where the code to be called gets just-in-time compiled and runs at regular ol’ compiled code speed. Based on my OSL comp node experiment (which didn’t quite work out since OSL is a pita to work with) I think this wouldn’t be too hard of a project to do on the comp nodes – the other node types would probably be kind of difficult since they don’t use C++ as a back end like the Composite Node Execution Engine thingie.

Anyhoo, fun times ahead…

ok, so if I understand it right:

  1. we won’t see particle nodes until pynodes are out ?
  2. Does someone know how to use pynodes ? I compiled it without problem from Lukas GIT but i don’t see anything new in the node editor. Where is it ?

You gotta use it through python.

Essentially PyNodes is the ability to script an interface. It is used to build / parse between generic relationships and code. This could be C++, python or anything that has access to a builder/parser function.
HIVE is a nodal logic system in Python. It’s entirely separated and encapsulated within itself. You can add Blender Game hooks with custom HIVEs which means that the entire codebase can be ported between any Python scripted environment, as long as it has the custom HIVE. PyNodes will give us the ability to use a built-in GUI and create nodes in HIVE. This is because HIVE is built on run-time. So, I suppose, it would walk the node tree and use the relations to create its own tree. This would mean it wouldn’t need an “execute” method for each node. Primarily because adding an execute method would introduce a dependency on scripting in that way, and thus PyNodes is no longer an interface interpreter but also an executer of logic.

From the Sunday meeting minutes.

  • Pynode project: Lukas Toenne worked on new preview code, needs work still. It’s still on target for 2.67.

There’s still a little work to do still, but Lukas apparently wants to make sure it’s fully usable before he commits it to trunk (hence the preview code)

Just a recap of resources–
PyNodes is now in trunk as of Monday the 18th build 55373 and therefor in the builder downloads.
http://builder.blender.org/download/

“Merge of the PyNodes branch (aka “custom nodes”) into trunk.”
http://projects.blender.org/scm/viewvc.php?view=rev&root=bf-blender&revision=55373
–this contains some additional notes and the following 3 links

http://projects.blender.org/scm/viewvc.php/trunk/blender/release/scripts/templates_py/custom_nodes.py?view=markup&root=bf-blender
–actual node example, note this is also in the text editor python templates as Custom Nodes in builds 55373 and higher
–it adds Add>Input>Custom Group Node and Custom Node
–NOTE: link above does not, but in 55494 the internal text editor template version fails on
AttributeError: expected NodeSocket, MyCustomSocket class to have an “draw_color” attribute

http://wiki.blender.org/index.php/User:Phonybone/Python_Nodes
–lukas’ blender wiki page with some information broken down in segments


–dated but usable original code blender post

NOTE: there is also a working copy of the example at
https://github.com/nicholasbishop/blender/blob/5bd0fd85ab42fac884c84adb55692dfb58388574/release/scripts/templates_py/custom_nodes.py

This seems to be the best thread on BA so far on PyNodes…if anybody has a better one please post it.
-= thx =-

It begins for LuxBlend:


Very slowly, but it begins. (just the UI, it doesn’t do anything atm)

Exciting.

One thing I thought of recently, I think the pynodes could be of use not just for render addons, but for a number of mesh generation addons as well.

Think of the old ANT terrain tool becoming node based and a lot more like Terragen or Vue as a result, a stair or path tool where you can create branches and junctions and control the parameters of each of them independently, a building creator where you create packs of rules and control how you put them together ect…

I’m sure once 2.67 comes out, we will start to see a nice little selection of addons that make use of Pynodes :smiley:

The way the node graph is set up actually works better for generic tools than material editors. It’s cleverly disguised here, but the node editor in that screenshot actually is not connected to materials, and has nothing to do with material datablocks. You don’t add to or take over an existing node editor, you make your own additional node editor. You’d have to rig up some method of manually assigning node trees to a material. There may be some method of populating the material node editor instead of rolling your own, we need to look into it still.

Make sure you let Lukas Tonne know about your inquiries, there is still time for him to make some changes before the feature freeze goes into effect and he won’t be able to do anything if the render plugin developers don’t approach him.

In the template, we can see how to creste and display a custom node, ok.
But how we do to do anything with, there an execute or invoke def to call??

But how we do to do anything with, there an execute or invoke def to call??

As far as I see it, there is no execution context. You’re editing a graph, it’s up to you how you translate it into actual commands.

Hey everyone, thanks for the interest.

LiquidApe pointed me to this thread and asked me to clarify a few things about “PyNodes”, so i’ll try to answer some questions here.

What can you do with PyNodes?

It lets you define data structures and graphical interfaces for nodes in the node editor. For this it uses the same techniques already known from other bpy extensions, such as subclasses of bpy.types.* and storage properties using bpy.props.* . You can not only define nodes this way, but also entirely new node tree systems (actually extending shader or compo nodes doesn’t really work yet, see below).

Besides being possible without recompiling Blender, defining nodes in python is extremely easy compared to the terribly bloated procedure in C. While the current node systems still use the existing C code i hope that we can start migrating nodes over to python, to reduce the amount of coding needed for simple additions. The majority of work can then be invested in the actual functionality, not boilerplate changes to 7 different C files.

What can you not (yet) do with PyNodes?

While it is possible to define nodes that can be added to existing compositor or shader node setups, these nodes will not have any effect at this point. Renderer and compositor code still has to be written in C/C++ and compiled in advance and can be pretty complicated.

On the UI/python level a few features are also still missing. In particular only very basic data types can be stored in nodes defined from python. Pointers to data blocks (such as materials) and complex data such as color ramps or curves are not supported by bpy.props yet. But this is a general scripting issue and not limited to pynodes.

Ok, so how are nodes “executed” then?

While the older node systems such as Blender Internal render nodes or the (utterly broken) texture node system had callbacks (like the requested “execute()” function) to manipulate data, this approach has some serious drawbacks (performance, instancing, cached data). Both Cycles and the new compositor use a largely separate system for processing their data - and this is basically how pynodes should be used as well (as several people have pointed out above).

The PyNodes define a UI layer and basic data storage and are used as a kind of “source code” for whatever data processing system uses them. There is usually a converter that maps each type of node to a particular code function, which itself is stored in a much more optimized structure (a “shader virtual machine” in cycles, or the operations graph in the compositor). I don’t have a working example of a fully functional python-based system of this type yet, but will try to make a basic example soon.

Unfortunately neither the compositor nor the cycles SVM currently allow easy runtime extension. Their individual node functions are still mapped in precompiled C++ code, so it’s not possible to just add a new node and make cycles or compositor use them as-is. One idea is to make “meta-nodes” or auto-generated group nodes which are in turn converted into nodes that these systems understand, but it’s a bit of a workaround.

What’s next?

There are a couple of smaller improvements to the node editor coming up. Several features that are currently written in C could benefit from being ported to python, such as the “Add” menu, toolbar and many operators.

Now that pynodes are available i also hope that external renderers can start integrating into Blender more easily (this was a major motivation for starting this project). Many modern renderers already use modular systems comparable to cycles, but defining materials in Blender was not offering their full potential (disclaimer: i love cycles, but competition is good too :p). Here is a really nice example of pynodes usage for an external renderer: Matt Ebb’s 3delight exporter

Working on the Cycles OSL backend last year made me curious about the possibilities of domain-specific languages (DSL) and JIT compilation (see UncleEntity’s comment above). IMHO this would be a great solution to the problems with extensibility and performance outlined above. While it’s possible to use python scripting for some purposes (simple mesh generation), it’s not a suitable language for processing large data sets (despite potential gains with numpy). So i would like to investigate this potential … no further comment at this point :RocknRoll:

believe it or not…trueSpace 7 already had these features in their nodes system…you could control most anything with nodes and codeable node bricks…unfortunately people always saw truespace as purely hobbyist…regardless of it having nodeable logic and Vray renderer etc…sorry for off topic. this thread really takes me back… :slight_smile:
But ia m really looking forward to this feature…and the game engine project as well…hive??? what was that called?

Would be great if we can extend this functionality not only for the render engines but for all the modules in Blender, I was thinking in write a code for having a new extra nodes in the compositor to bring some functionality from After Effects into Blender but now I just realized it’s not possible yet, so hopefully someway. Also to manage mesh and object information with pynodes would be great so we can build our own non-destructional modifiers in a similar way as Houdini does.

An update on LuxBlend: http://www.luxrender.net/forum/viewtopic.php?f=11&t=7931&start=40#p94466

Anyone else working on integrating pynodes? I’d love to see what other people have come up with so far.

I have one I am working on for easy settings while in compositor for an addon. The bpy addon makes doing render test strips/thumbnail renders easier and faster without changing render settings for final output. You still have to click the panel button to execute/render, but the settings are easily adjusted in the compositor while you are working there. I will post it after I get back from vacation.