I was wondering what the current situation is regarding the development of our own custum geometry nodes. Doing some searches around these forums gives some very different results depending on the Blender version used – which is good in itself, I suppose, but can be quite confusing to a beginner like myself.
So what is the situation for Blender 3.5? Can we develop geometry nodes from within Python only, or do we still need to do some stuff in C as well (and possibly recompile Blender from source), as I believe used to be the case? Where can I find the official documentation on this? Any basic tutorials I could check?
Blender is not only open source but also the discussion about the developement is open… and starting from Bender.org → get-involved… you maybe have a look here:
Thanks, I saw that post/blog. It’s almost a year old though, and covers Blender 3.1, so I’m not sure if this is still up to date. It also still involves doing some stuff in C/++ and recompiling Blender from source – which I had hoped would no longer be necessary at this point, TBH.
Still, it’s definitely a nice resource, so thanks for pointing us to it here.
@guy_lateur custom nodes in blender (compiled) C++ or blender python?
A custom geometry node with python can only be a UI after-touch I’m afraid
So long story short, most likely you won’t be able to share your node around if made in C++, and nodes that are shareable (python) are not powerful enough to be significant. There’s a hard wall here…
Not the answer I was hoping for, but thank you for making it clear. I was afraid that the “good stuff” still required C++ development, and I don’t think I’m willing to go that far ATM. I think I read somewhere that it should be possible to do everything in Python in the future, but I don’t know if that is still the case, and if it is, then in what time frame this is planned.
Nice thread you’ve got going there, I’ll be sure to give that a thorough read.
Yes and the problem is that python api has deliberate walls to incite people to contribute, while simultaneously it’s very hard to get your ideas accepted in main build, and there’s basically no alternative to the canonical distributions.
You could create your own branch of course but let’s be real, it will eventually die. (+ there’s the blender copyright problem)
Really? Can you provide any references to back this up? Because that’s quite a claim to make, IMHO.
I agree that making and maintaining your own branch of the code is a very big step, which a lot of ‘casual’ addon developers (myself included) are not going to take. At the current rate new Blender versions are coming out, it would indeed be quite the task to keep it alive, even if you’re doing it just for yourself. Furthermore, in my case, Blender is a tool in my workflow, not a goal of my efforts.
Also, I would think you can also contribute by making stuff in Python. If the devs like what you’re doing, they can always incorporate it in the C++ code if they want. So deliberately limiting the capabilities of the Python API seems like a strange way to go about it if you want people to contribute, if you ask me. But then again, what do I know…
Why it’s intentionally limited? by whose decision? that I do not know, and i doubt these “intentional limitation” intensions will ever be expressed clearly.
Once you go deep enough the blender rabbit hole… i can tell you it’s a bit fuzzy down there.
I mean, in 10+ years of existence, it’s not the first time someone wanted/proposed to have modifiers as plugins for example (in our case, nodes). the main excuse was “we don’t want blender to be unstable because of plugins” AFAIK, but that excuse is weak. I don’t understand what’s the big deal about it THB, maybe a blender veteran could shine a light on this topic.
Hehe, I bet it is… Thanks for elaborating on this.
That’s true. I myself have asked about being able to develop custom modifiers before. I understood it wasn’t possible at that time and moved on. Never really gave it much thought about why this might be the case. At least with geometry nodes, the possibilities in easily creating custom modifications have already been expanded a great deal.
Anyway, I can imagine development resources are limited, and decisions must be made and priorities assigned. I’m still assuming the devs have good reasons for implementing/allowing some features and not others (yet).
That would be cool indeed. Actually, maybe they already have. As you may know, there’s this weekly stream called Blender Today Live, which sometimes discusses this kind of behind-the-scenes info. Link to the latest episode below, for those interested. It’s about an hour long, every week, so that’s a lot of info to digest.
Anyway, I don’t want to come across as complaining about (the current state of) Blender. It’s a great package and eco-system, and I’m very grateful to be able to use it totally free of charge. But there will always be room for improvement, of course…
As stated in the doc, the limitations are intentional. it’s not about resources. There were plenty of opportunities to merge patches or work with enthusiastic developers who wanted to explored these kind of powerful C++ plugins, the latest I have in mind is Élie Michel, with his mesh effect project, and his C++ node implementation attempt, his work is very impressive. I’m sure there were older attempt.
Once I’ve saw a post by a veteran developer stating that "opening the door" to these kinds of concepts would be a “mistake”… There’s a strong pushback, it seems. don’t quote me on that.
Anyway, I don’t want to come across as complaining about (the current state of) Blender. It’s a great package and eco-system, and I’m very grateful to be able to use it totally free of charge. But there will always be room for improvement, of course…
Same!
Just highly interested in this topic
If someone is reading this and knows the answer, please let us know
I’m not sure about what you’re referring to…
Problem with python is that it’s much slower than what C++ node could do. So even if it was possible to make a python node that operates on geometry it’s very unlikely to be implemented in blender for that reason.
What might append at some point is a script node that basically do the same thing as actual node but in a scripted form. Since sometime some code can be simpler to operate than a nodetree.
That’s what I think houdini is doing with the VEX language.
But here again, python might makes things slow, so maybe a dedicated language would be needed. But take what I say with a grain of salt since I’m not 100% sure of what I’m saying here !
I think the problem is the API… it has to be very stable if you incorporate directly in C/C++ and changes are afterward… hard to do… the world players in the 3D market had much more money to do so from the start…
In contrast: if the python code does something wrong… then the blender core can just stop/kill the python interpreater thread…
Pro: blender doesn’t die because of an addon
Contra: it’s slower…
But then again… python can be fast…
Back on topic: especially geometry nodes have to interact with other nodes and if they break up the intercommunication in a node network… than it may be better that third party developers has to have a deep lock into the blender source code to make it right…
There where already complains about the buggyness of blender and than the reason was some third party addon…
But i’m not as much into this as you
… so the first think i do: i liked you on devtalk…
Problem is performance, most operation in python are single threaded, and probably some conversion are needed to access data stored in memory and to put it back.
For addons it’s ok, but something like a modifier can be run for every frame of an animation, or used on very big meshes.
I agree with developers that it’s not a good idea to put python code in these key areas knowing that it will be impossible to optimize them, unless it’s completely rewritten in C++.
Python is great for prototyping, you can get very good performance to some extend but it will always be limited compared to C/C++.
When 2.8 was out mesh performance was an issue, generally to be able to get some speedup it’s mounts of refactor to allow blender to work differently, but eventually if we keep it like that we can get rid of many limitations that slow down blender by nature.
In that case you don’t want to put some python code in the middle of that that would completely ruin these efforts Even if for sure it would open a lot of possibilities…
TBH, I don’t feel the ‘Python is too slow’ argument alone is good enough. There are plenty of addons out there (I’ve even written some myself ), even (mesh-) modifying ones, that are written in Python and are more than fast enough in a lot of cases. Sure, some of them could be optimized by doing them in C++, and there may indeed be situations when they will be (too) slow (large meshes, complicated nodetrees), but I’d rather have something slow than have nothing at all. And if it’s prohibitively slow in your case, then that’s too bad; you don’t have to use it. That does not undo other optimization efforts made previously/elsewhere.
It just feels strange to me that the devs should a priori make this decision, because IMO they’d unnecessarily be throwing away a lot of potential development (even if they’re only prototypes or proofs of concepts), just because it might be too slow in some cases.
Also, let’s not forget the ‘higher-levelness’ of the Python API. I can only speak for myself, but I’m pretty sure I wouldn’t have written nearly as many addons (or sometimes attempts there at ) if I’d had to do all of that in C++. It makes the development of custom functionality way more accessible, and it’s easier to share and maintain your efforts.
Actually, you could argue that node based development is kind of the next step in this evolution in a way. It makes adding custom functionality even more accessible to users who have little to no programming experience – and have no desire to become proficient in it; not in Python, let alone in C++. It only stands to reason that we (or rather the devs) should try to make this system as accessible and extensible as possible. I believe that providing a Python API for it could be an important part of that. Yes, users will probably pay a price for this, performance wise, but if it’s fast enough to get the job done, then why worry about that?
When I was younger, people often told me to make sure it works first, and worry about optimization later. That’s actually pretty good advice, if you ask me…
I know software comparisons aren’t popular, but I’ll relate this to how this was in C4D years ago. Developers had 2 choices: they could write in C++, or they could use C4D’s proprietary scripting language (called COFFEE. Sort of similar to Maxscript, but harder to write.)
A developer could write a shader in either language. If they chose to code in C++, it rendered quite fast. If they coded in COFFEE, it might be dreadfully slow. In either case, Maxon didn’t lock down part of the API just because some use cases weren’t fast. A volume fog shader in COFFEE - forget it, you’ll be here all night waiting for a frame. But a COFFEE shader that did something simple - say, a grid - was perfectly acceptable on speed.
As Guy stated, “And if it’s prohibitively slow in your case, then that’s too bad; you don’t have to use it.”
Blender offers a very different twist on this. You can code in Python or C++, but both of them lack key API access depending on your goal. If you want full access, you have to write for the main branch - and then hope that you’ve not wasted several months of your life when the reply is “this is not the blender way, update request denied”.
I’m not saying that BF should accept every code change that is offered to main, of course not. But it feels like contributing developers are somehow between a rock and another rock at times.
Cycles, Luxcore, process heavy amount of data and are plugins implemented via python.
From python you can pass memory pointers to compiled C++ modules
It’s already possible to create bpy operators that fully process a mesh within a C++ module with no speed cost what-so-ever. Just not from a node or modifier
There where already complains about the buggyness of blender and than the reason was some third party addon…
This argument is about control
If you don’t want to take the risk of running into extra bugs provoked by plugins, then don’t use them. Don’t start imposing what you think is best to others…
Same goes for developers, ultimately they could impose their policies on users. Keep controls over users’ behavior, that’s typically done in Adsk software tho…
I’ve read every reply in this topic, I’m well aware of your objections to the Blender developer’s methodology It doesn’t change the fact that there’s a very simple solution to all this. Whether you like the solution is entirely up to you of course
Blender Today LIVE #225 - YouTube
Even if for sure it would open a lot of possibilities…