Patches vs Plugins

General Forums Blender and CG Discussions
1

…future thinking 2.6 then stable until 3.0!

ok,
i’ve been thinking about this for a while.
I don’t like the current dev system of patches.
I know it works. it’s useful. it’s needed. it makes things work!
so i’m not saying get rid of it.
maybe change a few things…

the current situation:

From an User Perspective,
As I see it, (really basic)
Patches are chunks of code that replace or add to Blender code, for adding new features.
You can only use patches if:
>you are committed enough to download from graphicall a “Patched Build”
>you are building Blender yourself.
Or you won’t get to try them out.
Plugins are either Python Scripts (I am only partly talking about them here),
or Textures/Sequencer most are precompiled code, (I presume C/C++ code).
so it’s plug in & most times they work…

*note I am not a Blender Dev.

What i would like to see is:
The current system be replaced by a 2 tiered/level system.

Highest level:
Writing Patches for Blender!
If you are a known, approved or requested developer,
you can write patches to Blender with the team.

Currently if you have a patch that requires approval from the Blender Team,
but this may be rejected.
Sorry if you put in a lot of work, but it does not fit into Blender’s design/code/well enough.
I know that sounds harsh, but there were no promises & you can run a branch because your feature is still cool, but it clashed.

Ouch, says the prospective dev, running away in frustration.
(or they put their cool stuff on graphicall for those that know.)

That brings me to proposed Second Level.
Using the Python/Plugin Api, (yet to be written?)
Instead of dev’s writing then submitting patches;

Why can’t they just write a plugin like other programs?
If the plugin is especially cool or useful, it can be released with or integrated into Blender.
If not, it can be cataloged & made available to the general public via Blender.org or the Blenderwiki.

That way:
Potential devs can write cool plugins for everyone to use.
not just sequence & texture, but the full spectrum.

This requires a big change.
In both the way Blender handles plugins, (the api would have to be extended?)
and the way devs handle potential devs/development.

In closing…
The benefits are huge.
For the Average user, who will on occasion, want something more out of Blender.
Bonus, just install your favorite plugin.
For Blender Devs, well, it would be good for them too. :slight_smile:
For potential Devs, compile your patch as a plugin where possible.
Then you can submit you plugin, if there’s a problem it’s ok.
You still have your plugin, it still works & I’ll be using it.

I hope that was understandable…:eek:

2

This should be posted on some Blender dev mailing list or something…
Because what do you expect from us - users? Yeah, we want cool plugins!?

3

didn’t read your post(too tired right now), but yeh a propper (C-)plugin API would be suh-weet. :slight_smile:

4

popski,
just some open discussion.
Of course we want cool plugins/features/scripts.
In some cases it would be better, i think, to have plugins rather than patches.
There’s going to be a flood of plugins & patches soon.
how to handle them?
how to make the most out of Blender?
as much as i hate to say it, character studio was a plugin for max?
it would be nice to port *topmod to blender as a plugin.
there’s no way anyone would agree to having it patched or released.
but as a plugin it would be very favorable.

*example only to show the difference of a possible plugin against smaller scripts.

5

i think i see what you mean and would be great for 2.5

i’ll give one example for this the Patch for Bas relief nodes

looks very nice but it’s a patch and i don’t how to build blender

if this was a plugin i could load it up on Vista and i supposed would work fine
and easy to use and test

but right now this is not a plugin and i cannot use it and that’s furstrating caue i did find any built version using this new nodes system!

hope people makes more plugin instead of patch

happy 2.5

6

New node system?? I want to do some work with nodes which was probably going to turn into a patch, how would you suggest it could work as a plugin? Would that simply mean that perhaps when the python api has been completely written then python scripts are loaded as soon as you start a blender session?

Or would you perhaps need a c api and write all plugins in c?

7

mandoragon,
as the py api & plugin api? grows,
it should be more than possible to use a script to call from a .dll, (windows) or other os formats.
so python & c could be used together for plugins.
i have seen this done in blender 2.4x, so it should be possibly.

In Theory. :slight_smile:

RickyBlender,
yeah, that’s what I meant to say. :wink:

8

Yeah, I hinted about this in the Python forum. I wanted to make a modifier (ie plugin) in Python for Blender. I don’t want to have to compile Blender to do this. And you are right, if someone wants to use it they can and they don’t have to either. Either way it would not affect the foundations goals because it is completely external. Plugins are the way of the professional world crashy though they may be. Consider SinSati and Real Flow. A lot of visual FX production is based completely on plugins for applications. Who cares if it is Maya or Max, it just needs to be Real Flow.

I still can’t get Blender to compile, anyway. I have looked at how modifiers are constructed in the source code. I just don’t have an example or even know if it is possible to do it in python.

Patches are useless to people like me. Like the Kai branch, there is a patch for the z-twist bug, but I can’t make use of it. I need an EXE not some patch code.

9

well there is another advantage also
ounce it is made as a plugin it can loaded into any blender OS version

so more available to eveybody on any system!

now i’m not a specialiste in Programation but i’m certain devs know about this
and would be able to find a simple solution hopefully

happy 2.5 with plugins

10

the problem is with a plugin system…

  • someone has to write it (maintain it, keep API compatibility, document it etc), nobody likes doing this work as far as I know.

  • plugins will have compatibility issues, 32, 64bit compiles will be needed for at least 3 major OS’s - so 6 binary targets if you want to support mac/win/linux, excluding problems different versions of OS’s may have as well as libraryes the plugins may also depend on.

  • Existing binary plugins for textures and the sequencer were not used much (to my knowledge), admittedly they were not that powerful either.

Im experimenting with adding external python modules to blender, The idea is we can have blender build with python modules which are written in C/C++ and dont link to blender, but python can pass arrays to/from these modules.

Example:

  • Renderman integration, written in python with mesh export done in a C module that can be distributed with blender.

  • OBJ Parser able to import huge meshes by reading the array’s in C++ and passing to blender so python never needs to touch each vertex/face.

The workflow would be to write in python and to move the slow parts into their own modules.

If people compile their own modules that are not apart of blender distribution, they could distribute the compiled module with a python script much like you might distribute a plugin.

We could even have compositor nodes written in python, where python only passes the image buffer to its own C module. python modifiers could be done this way too, for deformations at least would be simple.

To be clear, C modules would never be a necessity, you could always prototype in pure python, but tasks python is typically to slow with, would eventually want to be moved into C/C++ for production.

While technically this is like a patch, it would not have the problem that the patch gets out of date with blenders code since it would be a standalone C/C++ module.

While in some ways this isnt as nice as a well written API, I think its fairly flexible more likely to get done.

11

Is there any guide to the blender code as the blender.org docs are very out of date??

12

there is also in the nodes system a special node called Pynode i think where yo can make your own nodes if needed but not certain how easy it is to work with !

also for python script you could use a compiled version instead of the text vesion which run as an interpreter while the pre compile one run much faster

but in any case the idea with 2.5 i think is to have more flexibility and userfiendliness
for eveybody to enjoy and work with blender

so hope it can be implemented in 2.5

Thanks & happy 2.5

13

I plan to make some blender coding video tutorials for total beginners, but it keeps getting put off.

even so, Im not sure more docs actually help much, if you want to develop something like blender you have to get used to reading other peoples code (in most cases), so rather then trying to be spoon fed knowledge, accept that you need to have the initiative to find out how things work on your own.
This may sound extreme, so start small and write something easy, work up to something more complicated.

@RickyBlender, compiled python scripts are not faster in any way that matters, and psyco isnt that much help either, until unladen swallow bares fruit (yet to be seen) I expect C/C++ modules will be the only significant speedup.

Other trick could be to use Python/OpenCL module, not sure what it would be useful for, compositing perhaps?
Numpy (when its ported to py3.x) is also a great option for people who dont want to write C/C++ modules, but wasnt to manipulate arrays fast. This could be included with blender too though it only helps in some cases.

14

Reading other peoples code is fine for me to do, but trying to find my way around the blender code is going through a maze, the svn respository is just folders and files. It would be great if there was some kind of index or contents where you could search for a particular part of the code (eg node editor) and be diverted to the particular sections of code.

I need to look at some of the blender code for my university dissertation although it is more likely that most of the work I will be doing will be through python scripts.

15

1- i tough compiled PY vesion where supposed to be much faster !
i got to do some testing on this later on this January

2- Numpy

well are we going to be able to use Numpy in 2.5

if i remember well this is the one with the huge math librairy
hundreths of math functions

which i find super for peoples in need for these math functions already pre define in Numpy
including lot’s of matrix real or complex

mind you this is more python lib functions then using API functions anyway!

hope we can use it in 2.5

Thanks

16

I should have mentioned as well as reading, you need to learn to search the codebase.
probably people assume blender devs know the entire codebase or something? - not at all, its crutiual to be able to quickly run searches.

mostly use a simple grep wrapper, http://wiki.blender.org/index.php/User:Ideasman42#crep
though there are some similar tools specifically for this I didnt find them quick enough to use.

@RickyBlender, theres a limit to how much you can speedup python, even with compilation. Since internally the PyObjects have to be reference counted. have their types inspected, callbacks run etc. 1+1 in python is doing a lot of work behind the scenes, even if its compiled.

17

I definetly think a good module system would make blender highly customizable and open up integration of third party products into the blender pipeline more easily but as already mentioned it has to be well supported or else it will fail miserable.

Another drawback I see with a plugin/module system where modules are written by third party developers (meaning devs without commit rights) is maintainance. It is highly likely that there will be a lot of plugins out there that will not be maintained and break from version to version bringing instability to the whole of blender. So unless the API for the plugin system is designed with loose coupling in mind and with good error protection against memory corruption I do not see how this system can produce as stable blender as we are used to seeing.

18

Is crep used when browsing the svn in the terminal?

19

@musk. an api can be kept stable but that means if you want to expose a lot of useful functions, they cant change either, or have to be kept from being removed or working in slightly different ways between versions of blender for spesific functions like a texture plugin a stable api isnt so hard, but to expose modifiers say, this is less likely to be stable.

Or we could do what Linux do and just give a binary api that isnt ever called stable (in practice it may not change all that much) and expect devs to update between versions.

@mandoragon, yes, I don’t even ‘cd’ into directories, mostly I just grep from the same place I commit and checkout from. you can narrow down results with grep…
eg.
crep node
…too many hits
crep node | grep mute
… 6 hits

… or exclude depsgraph
crep node | grep -v depsgraph

20

Instead of the ‘plugin vs patches’ choice I’ve been looking into tacking a jit compiler onto blender…mostly for a new dynamic node system but it could be used for much more.

Apparently llvm (somehow) calls dlsym on the main process so your extension modules have full use of all built-in functions without having to go through an exported API to get at them. If you call some function it doesn’t know about it can automagically link to it.

I’ve been seriously looking into libjit since it uses a standard (elf) binary format so all you have are big/little endian and not windows/bsd/linux/etc linkage issues.

Those two things together deal with a lot of the problems brought up so far…there would still be api stability issues but it wouldn’t be any different from the rna wrapped c functions available to python.

It might be easier to just add a button to the text editor that calls something like ccons instead of my original plan though.