This is true… However, i do think there can be room for improvement…
i am far from an experienced programmer but everytime i download source code for an OSS project, i get overwhelmed by the amount of files i have to look through before i could understand anything…
i went to a presentation by a developer from Mozilla, and he said they had the same problem too. if a feature they wanted to develop takes longer than expected, the managers would put more developers into the project, which actually decreases efficiency because the new developers now need to battle their way through the jungle of existing code before they become productive…
Mozilla have been developing a system where browsing code wouldn’t be as much a hassle as it is now… i don’t know the intricate technicalities of it, but he showed how you could put the code in a browser and all functions and global variables would become click-able links that take you to the definition of the function or the variable. with such system, it’d be much easier to navigate through code instead of having 5 text editors open to be able to connect the dots…
these tools are under development still, and to my understanding they’ll be open source… The point i’m trying to make is just being aware of the need to make things easier for new comers is a good first step…
do you mean a bit like how songbird has feathers? Its based on how firefox has extensions. Although it would not work for every user, would it be forseeable in the future to connect Blender to the internet?
@dyf, agree things can be improved and agree that its often overwhelming.
But I think to look over many files isnt a good approach, first try understand one function in one file, something your interested in (or want to improve).
@Uncle Entity, these projects look interesting. re: adding in a JIT spesific tasks for nodes, ok so we get faster shading, but this adds say 5-20mb (rough guess) to blenders download and probably only speeds up node-compiling or shading.
Even tho extending blender with python isnt the best way to do plugins IMHO, its not a bad one-size-fits-all way to extend blender and has the advantage that scripts can always access other data via rna as well as special functions we expose especially for fast data access/manipulation.
Libjit is pretty lightweight, less than 2mb total I believe.
I was reading an article about someone using a jit compiler to get 4x speedups in image processing by on-the-fly compiling the functions to hardwire values and get rid of loops and such. With ~12.5 million pixels on a 4k video frame I’m thinking the renderer is going to be spending a lot of time in the composite nodes.
Uncle Entity, what’s your take on libjit’s OS X compatibility issues? Last fall, I chatted a bit with folks on libjit’s mailing list re: compiling on OS X (re: ruby & computer vision); it sounds like it would take a pretty serious developer commitment to make that happen, though.
No clue, it built just fine on my linux box a little while ago so I could figure out how big the lib turned out to be. Well, after the usual battle with autoconf/automake.
patches are patches… source code. nothing more nothing less than that.
If a patch affects a platform specific file that deals with some platform specific thing, then the patch is platform specific. If not then the patch is not.
hi,
thanks for all the informative, clever & useful comments.
ideasman42, especially, thanks for your contribution.
.
I wanted to open this up for discussion as there’s several 3rd party scripts & plugins people are designing.
so this is a good thing, to try to formulate a plan.
To get user/coder input as to whats wanted, what’s needed & how to best implement it, is what it’s all about.
Uncle Entity, IIRC there are some issues with memory management being handled slightly differently on OS X in general (both Intel and PPC). My thread on their listserv may be found at http://lists.gnu.org/archive/html/dotgnu-libjit/2009-12/msg00001.html. I’m not a low-level programmer so I found a different solution for my project; looks like it might be possible to make it run smoothly on OS X, but with a ton of work porting.
:o :o I’m sure there are so many beautyfull Plugins and Patchs around for Blender… I downloaded some but never came to get them visible as tools… Could there be some “installer” or something to have them pop right in the UI. This would be so nice for the non-geeks and non-scripters-programmers…
And this post precisely demonstrates why I think ‘proper’ plug-in’s would be good.
as far as I’m concerned it might just be a .bplug file that’s really just a zip archive,
including (all) the .py file(s of the pack).
this way it would be easy to deploy your (multi-file) scripts.
Especially sweet would be if blender’s installation would just do the file association,
so you just dl the plug-in to your desktop(or where ever),
blender copies the .bplug to the plug-in folder and continues happily.
I was just about to suggest something like that. It doesn’t matter if the plug-ins are not actual plugins the way they are traditionally done in other platforms. the issue is bundling these in way that users can just download a file put it somewhere Blender can recognize and be done with it.
Well, basically my point being that at they very least they should be as easy to use as firefox add-on’s(example) are,
although a c-api would be awesome still for it’s reason,
I can understand that there’s an extra overhead to developing and maintaining it, so not a killer…
I think there is a bit of a disconnect in people’s understanding of the hazards that come from with having the source code openly available.
You can’t go mucking around in the code base and expect to be presented with a nice big glossy button titled “Just Make It Work” that magically probes your mind and alters the code to match the mental paradigm you’ve dreamed up.
If you expect developers to put in the extra effort to making code open-source and easily available to you, you’re going to have to meet them halfway and learn basic things such as dealing with build systems, applying patches, compilers, code management systems, doing back-traces for bugs, etc.
A plug-in API to cover every single possible scenario that third-party developers might come up with would basically result a 1 to 1 duplication of the entire code base. That’s silly.
There are always ways to improve things. (I think the auto-generated python API is brilliant) But eventually you are going to have to deal with the code directly if you want to play with experimental features.
OK, lots of funky jit stuff is being worked on, but recently I was reading the PEP for Unladen-swallow which has average 1.8x or so speedup on many real world tests.
I also tested array access with C compared to python and got around 800x speedup, so IMHO JIT compiling interpreted languages is cool but can only get us so far. OR - may be very useful in certain situations (like a JIT shading language for eg).
In saying all that. One thing that I dare suggest is this…
External C/Python extensions in blender! - I added this to blenders build system using CMake on my system as a test. Can commit to SVN soon.
Anyone can write a C/Python extension, this isnt a patch.
No Plugin interface… or… Python IS the plugin interface. A script would need to access the C modules.
Last and most interestingly :)…
We allow these modules to use ALL BLENDERS FUNCTIONS… thats right. KD-tree’s, Octree, hash table… the whole kit-and kaboodle. This actually doesnt involve any work from devs… Heres an example I tested of a C++/Python module that calls a blender internal function.
#include <Python.h>
#include <stdio.h>
#include <stdlib.h>
extern "C" {
#include "BLI_path_util.h" /* THIS IS TO USE A BLENDER INTERNAL FUNCTION */
}
static PyObject *test_module_parse(PyObject *self, PyObject *args)
{
char foo[12] = "Hello##";
BLI_convertstringframe(foo, 10); /* <--- RUN THE EXTERNAL BLENDER FUNCTION! */
printf("%s
", foo);
Py_RETURN_NONE;
}
static struct PyMethodDef test_module_methods[] = {
{"testme", (PyCFunction)test_module_parse, METH_VARARGS, ""},
{NULL, NULL, 0, NULL}
};
static PyModuleDef test_module_module = {
PyModuleDef_HEAD_INIT,
"test_module",
"Example module that creates an extension type.",
-1,
test_module_methods,
NULL, NULL, NULL, NULL
};
PyMODINIT_FUNC
PyInit_test_module(void)
{
PyObject* m = PyModule_Create(&test_module_module);
if (m == NULL)
return NULL;
return m;
}
Using this is as simple as…
import test_module
test_module.testme()
The main disadvantage is this means extensions are allowed to do anything
abuse blender’s internal state or whatever, we have to trust the plugin authors dont be idiots.
There is no stable plugin api, they just use blender functions the same way blender does. In one way its bad but it also makes plugins incredibly capable.
…Developers may complain their plug-ins break between releases, but practically the chances of breakage isn’t so high and probably only mean changing a few lines of code.
Does python do some sort of automagic linking thing to get at the blender internal symbols or do you have to get it to link another way like through your makefiles? I’m kind of guessing that BLI_path_util.h has something to do with it.
RE: the JIT shading language, been working on a parser based on the OSL grammar that should be able to spit out an AST in a day or two, from there it shouldn’t be to hard to get it plugged into blender…once I figure out what brecht’s up to with his shader rewrite.
The real trick will be getting the UI button elements from the shader function declaration but that’s mainly because I don’t understand how the new UI system works.
Might be able to get a prototype shader node up and running in about a week.
Is there any reason people may want to download and install plugins for Blender when scripts put into the scripts folder do a lot of the same thing in Blender 2.5? Because 2.5 has extensive python access, even more so than 2.49 and will be even more so once python defined spaces is in.
Plus wouldn’t this slow down Blender development (in a way of features getting into SVN itself) because people might rather make users download 50 plugins rather than coding them as patches and this possibly seeing them with every Blender build?
