Blender Cycles - Technical (developer) question about various types of Cycles Shader

NOTE: Since this forum does not allow me to upload more then 3 images, please take a look over here (for all related images)
- https://cgcookie.com/t/974-blender-cycles-technical-developer-question-about-various-types-of-cycles-shader-data-streams

In short… I’m busy to fully understand Cycles and “how stuff really works under the hood”:

Why do I do this?

  • To understand which type-of-datastream/shader-socket is generated by any Cycles Node output.
  • To understand which type-of-datastream/shader-socket can be accepted by a given Cycles Node input.
  • What type of data a stream acutally holds (bits, bytes, floats, vectors, pixels, shader-data)

Where am I getting stuck and why?
So far the official

  • Blender Wiki
  • Blender online end-user manual
  • Developer-level documentation over Cycles
    all fail to mention the different type of data which is contained in/ carried the Cycles Shader sockets.

So I went a step further and downloaded the entire Blender (2.79a) Source code package and started looking for the exact definition of those Cycles Shader data types … and found this information (os far):



Fig 6 - Blender Shader sockets information, located in various source code-files

So now my question to you all is:
Where must I look (further) to find the exact documenation / definitions of the Shader sockets, when it is not documented
in any official Blender (developer, manual) source and its own source-code-files?

Thanks in advance for your hints, tips, and tricks to get me closer to demistifing the Cycles Sockets data_types.

The main website for documentation on Blender source can be found here, which is the official Blender wiki

https://wiki.blender.org/index.php/Dev:Source/Architecture

The ultimate documentation is of course reading and understanding the code.

I am not familiar with the Cycles internal architecture but judging from the fact that it leverages CUDA, some familiarity with low level gpu hardware will be required to gain a deep understanding.

Unfortunately when it comes to understanding the source you are pretty much alone because the only who can really help you are blender developers, especially ones that work on the are that interest you.

In case of cycles that person is Brecht (he is the coder that created Cycles), he does not even work for Blender Institute anymore because he left to go work on Arnold (another amazing render engine), he is supposed to be hired back for several months to work on 2.8 if I am not mistaken. But even so I doubt he will have the time to anser your every question on Cycles. Essentially your best place will be the BF mailing list which is what devs use to communicate and of course #blendercoders in irc if you are lucky enough to bump on Brecht.

But basically, your only reliable way of learning is taking your head and burying it inside the code. Graphics coding is one of the most difficult forms of coding and as such, there is very little handholding.

Have fun :wink:

1 Like

@Kilon: Thanks for the link to the documention on the Blender source.
I’ve already gone through that material (which to my opinion could be vastly improved, but that’s another point). I was not able to find my answers there :frowning:

The ultimate documentation is of course reading and understanding the code.

Agreed, That is the main reason behind dropping the “Blender source code” in the first place, since it is far from complete. Source-code never lies, so that is why i’ve alrady been busy decipher Cycles internal modules and architceture, straight from the source_code itself.

I am not familiar with the Cycles internal architecture but judging from the fact that it leverages CUDA, some familiarity with low level gpu hardware will be required to gain a deep understanding.

That is no problem for me, since there are tons of well documented stuff on low-level GPU_hardware and CUDA.

Unfortunately when it comes to understanding the source you are pretty much alone because the only who can really help you are blender developers, especially ones that work on the are that interest you.

Agreed… and I’ve already found out that that the original creator of Cycles no longer is a part of the active Blender development team.

Essentially your best place will be the BF mailing list which is what devs use to communicate and of course #blendercoders in irc if you are lucky enough to bump on Brecht.

Thanks for the tip on the mailing (had not found that entry yet). I also will give the #blendercoders chatroom a try. I hope that there are still Blender development members left, who are able to point me in the right direction.

But basically, your only reliable way of learning is taking your head and burying it inside the code.

That is just the way I’m loving it… learn from soembody elses code-examples. BUT without a proper explanation and mind_frame it gets a little thoughter to decypher somebody else design and thought-patterns they had when writing their open-sourced code

Graphics coding is one of the most difficult forms of coding and as such, there is very little handholding.

Nope, since I already in 3D programming about 32 years ago (in 1986). I designed and created a 3d wireframe display program the universities DEC PDP-11 in USCD Pascal, when I was still studying to get my Master degree in Computer Science and Industrial automation, back in my university-school days. You can see some examples below how I got started:


Fig. 1 - It all started with this listing to convert a 3d space_coordinate into 2d screen_coordinate
Written in MS- Dos Basic and for a Hewlett Packard programmable calculator (back in the 1980’s)


Fig. 2 - I rewrote it into ~1600 lines of USCD Pascal on a Dec PDP-11 16-bits minicomputer
This was back in 1986 - about 10-12 years before Blender got designed and created!


Fig. 3 - This was the computer equipment back then in the 1980’s
The DEC PDP-11 16 bits mini-computer and a DEC GT40 dedicated vector graphics display terminal

To be continued in the next posting…

1 Like

Here you can see examples of my first “vector-graphics-on-screen” to “printed paper output”


Fig.4 - My very first 3D Perspective view printed out in 2D on a dot-matrix printer
(which looks remarkable to the Default Scene in Blender :slight_smile:


Fig. 5 - Grumman X-29 Prototype
3D view printed out via a plotter


Fig. 6 - McDonald Douglas F-15 “Eagle”
Upper row : Front-view, 3D view,
Bottom row :Top-view , Right-view in orthogonal projection
(and yes this looks like the Quad_View in the3D viewport in Blender too)These are scanned printed/plotted paper examples of how I got started in 3D programming back in the 1980’s.

So in short:
I already know my 3D Graphics stuff for a long time, way before

  • Blender came into existence,
  • Personal Computers where invented,
  • Windows ™ with its native DirectX ™ graphics programming environment
  • Linux™ with its native OpenGL™, OpenCL™ and CUDA™ graphics programming environments

And PC’s where got equipped with GPU’s that became more powerful then their build-in CPU’s.

And now… switching back to dissolving Blender Cycles mysteries :slight_smile:

1 Like

Always great to meet people from the distant past of computer technology , I started coding in 1988 too, but of course nothing remotely close to 3d coding which is an are that I have ventured very recently (still fist fighting with OpenGL)

Back to your original question.

One thing to keep in mind is that Blender, even though written in C, it uses its own OOP system. Taking into account the code diagram I am betting on the central data types

they can be found in makesdna, makesrna, blenderkernel while of course directly what interests you should be located in nodes. I advise reading and studying the whole section of “General Blender APIs” , as you can imagine there is a lot of dependency on these libraries. What you should looking for should be one of the headers of those library most likely the node library.

It is also very likely is located in the general section because those variables or to be precise, enum values should be default values for all types of nodes, including compositor nodes which are not related to cycles.

I also strongly advise making a debug build of blender.

Blender debug build like all C debug builds are coming included with symbol tables which provides extra information for the debugger. Most like GDB is the most well supported debugger but you may have some luck with LLVM debugger if Blender is compilable through this compiler tool. Then its a matter of inserting a breakpoint on the right section, again most likely on nodes library, you may also have some luck with the render pipeline where it should be where cycles render files are located. You can then use the debugger to directly inspect the memory and find exactly the type of any library that interests you.

I am not so sure if Blender supports Visual Studio builds, it used to support only MingW but I am pretty certain it does, if it does I highly advise downloading the community edition of Visual Studio and using the in IDE debugger which is pretty good , as well.

I have some little experience with VS which acquired when I was studying the arcane art of shared memory for a project of mine to make C++ libraries accessible from a programming language (Pharo) that only supported C libraries. I accomplished my goal and to a great extend I had to thank the Visual Studio debugger for that.

On the matter of the mailing lists (highly advisable to join them) you can find a list here

https://lists.blender.org/mailman/listinfo/

Obviously the one it interests you is the BF-cycles mailing list, but others can be helpful tool for more general dependencies questions

Of couse if you run into dead ends and you don’t get answers from the code and the experts you can always ask me here for more clarification. I am no C experienced coder and definetly not familiar with cycles internal. But I am familiar a bit with Blender internal data structure and opengl binding which I studied to gain a more deep understanding of the bgl python module (it gives full access to blender’s opengl context which I use to creat a custom GUI API which is independent from the existing Blender GUI API)

Good luck and keep asking questions :slight_smile:

@indigowarrior9, nice code there!!! I also wrote a renderer back in those times, first in BASIC (damm slow) and after in assembler. Thought my machine was a 48k Spectrum with 3khz 8bit processor… drawing a sphere was back then a huge challenge :slight_smile:

Hello Kilon
Likewise… It feels like walking among the experienced ICT giants once again :wink: and that feels great fo me to be able to communicate with a person who goes as far back as I have done.

REGARDING BLENDER SOURCE_CODE INVESTIGATION:
I’ve installed Git today (09-march-2018) and cloned the master repository, to further investigate it deeper and deeper on my local harddrive.
I think this will provide me my answers much faster then endless surfing on the Internet.

Thanks for the link to the diagram, which I already encountered myself a couple of weeks ago. BUT immediately I find myself asking
“What is its real and actual value” when is is being posted on a section od the Blender Development Wiki webpage which has not been updated since “Ages”?

The more I dig deeper into the Blender Wiki / Developer “document” pages, the more sad I start to feel:

  • What a giant holes in (complete and up-to-this-date) “developer grade documentation”.
  • What a serious lack of overview, structural architectural diagram images / pictures.
  • Little to no User-Interaction/ Interface-diagram, no UML and Use case material of the entire package / individual modules.
  • Little or no graphics who show you the inner workings of Blender and her modules… so sad!
    I have even using the Google site-search function to to fully scrape the Blender Developer Wiki… but again…don’t get me started how sad the state of "Blender (developer /enduser) documentatie really is. </break… since I can go one for …>

That is the main reason why I’ve downloaded the source code and am currently busy to start designing (and writing) my own Blender documentation tools, which makes - by the way - a nice refreshing course in C and C++ back from the good old days.

they can be found in makesdna, makesrna, blenderkernel

Thanks for the reference to the best place to get started, really appreciated!

I advise reading and studying the whole section of “General Blender APIs” , as you can imagine there is a lot of dependency on these libraries. What you should looking for should be one of the headers of those library most likely the node library.

Thanks again for pointing me in the right direction.

I also strongly advise making a debug build of blender.

Thanks for the tips, Where can I learn how this works?

Of couse if you run into dead ends and you don’t get answers from the code and the experts you can always ask me here for more clarification.

That seems to become the default / the norm on this part of my Blender journey… experience ;-} since this project seriously lacks good documentation from the top-down. But hé I like a challenge so now and then, and being forced to create my own documentation,
by figuring how Blender really works from its own source_code, might become my cup of tea in the upcoming 2,3 months.

I am no C experienced coder and definitely not familiar with cycles internal. But I am familiar a bit with Blender internal data structure and opengl binding which I studied to gain a more deep understanding of the bgl python module (it gives full access to blender’s opengl context which I use to creat a custom GUI API which is independent from the existing Blender GUI API)

That makes (now) 2 of us, since my C-skills are now a little rusty (from 20-25 year ago), but I still have the hacker-mindset, so i’m sure will reboot that long_term memory section of my brain into active duty again :slight_smile: And with the help of the right C-tutorial and code-snippet websites I’m sure i get quickly up-to-speed…

QUESTION: Is it correct to think that you have succeed to create your own GUI into Blender already?
Sounds interesting, and quit a deep dive into the Python belly of the Blender-beast too.

Good luck and keep asking questions

I have a few… right here and now… are you ready?:
1 - DO YOU KNOW ANY (FREE) SOURCE-CODE AUTOMATED ANALYSES AND DOCUMENTATION TOOLS?
I’ve found installation / project fragments of “Doxygen” setup file in the \doc\doygen folder. I wonder why this was put into there, since … no docs foun about it (yet).

2 - DO YOU KNOW ANY (FREE) SOFTWARE, WHICH CAN GENERATED SOURCECODE / HEADERFILE (INTER)DEPENDANCY TREES FROM GITHUB REPOSITORY
Off course I can do this task all by hand, (or start writing my own toolset for it) but going through about:
~ 1.250 C source files
~ 1.200 CPP source files
~ 1.600 Python files
~ 3.000 Header files
In total about 10.000+ files might take a day, week. month or x… so I was wondering if there are any good automated tools which can swallow up the Blender master-source code package and assist me a little bit to get an overview quick?

I already found a nice list of possible packages:

3 - DOES BLENDER HAS SOME SORT OF GLOBAL DATA-DICTIONARY somewhere? anywhere?
Where I can quickly find names, scopes, data-types of all variables used inside the source-code? Right now there are a few fragments of the “Blender Developer Wiki” but not of them paint the complete global overview picture:

Thanks for the tips so far Kilon. much appreciated!.

Wishing you a heppy weekend

Hé Secrops! nice to see/meet you here too. Thanks for the compliment.

48k Spectrum with 3khz 8bit processor…(yes i still can remember those days) . and writing a renderer in Basic and later Assembler… <silence> I can only say respect! for your skills back then already…
I never got any deeper then Texas Instruments TI99/4 Basic (and a little peek/poking) in that time frame.

I only got down to the assembler level-of-things when programming Motorola 6802, 6809 embedded micro-controllers in hex and assembler during my school studies. But I think I will pick this skill up again somewhere this year…

Q: So you (too) are an really old-school computer programmer, just as Kilon here?
Q: What do you do with Blender if I may ask?

Frankly not having any documentation is the norm in the coding world, especially for open source. The fact that Blender has outdate docs is a luxury.

I am studying opengl and I am not sad , I am horrified how such an extremely important API have such terrible and so minimum documentation. 2018 and there is still only a handful of good opengl tutorials around.

QUESTION: Is it correct to think that you have succeed to create your own GUI into Blender already?
Sounds interesting, and quit a deep dive into the Python belly of the Blender-beast too.

Yes Morpheas is working great , still very limited but get the job done as a secondary inpendent Blender GUI API. I made a first in September

I was about to make a new one because I have enhanced it quite a lot since then (I use it for an addon I am making) but I decided just to be safe and port the entire library from OpenGL 1 to OpenGL 3.3 which is the minimum that Blender 2.8 will work, so to make it also Blender 2.8 compatible although we are still very far from a stable Blender 2.8 release (probably around October). I am battling right now with OpenGL 3 tutorials and documentation because its almost completely diffirent architercture from OpenGL 1 but it will get there.

I have a few… right here and now… are you ready?:
1 - DO YOU KNOW ANY (FREE) SOURCE-CODE AUTOMATED ANALYSES AND DOCUMENTATION TOOLS?
I’ve found installation / project fragments of “Doxygen” setup file in the \doc\doygen folder. I wonder why this was put into there, since … no docs foun about it (yet).

2 - DO YOU KNOW ANY (FREE) SOFTWARE, WHICH CAN GENERATED SOURCECODE / HEADERFILE (INTER)DEPENDANCY TREES FROM GITHUB REPOSITORY
Off course I can do this task all by hand, (or start writing my own toolset for it) but going through about:
~ 1.250 C source files
~ 1.200 CPP source files
~ 1.600 Python files
~ 3.000 Header files
In total about 10.000+ files might take a day, week. month or x… so I was wondering if there are any good automated tools which can swallow up the Blender master-source code package and assist me a little bit to get an overview quick?

I already found a nice list of possible packages:

No , this is not my style of coding.

My style of coding is based on a technique known as “live coding” meaning coding an app while it executes. I have learned this technique from Smalltalk that I was introduced to when I was learning Lisp (Common Lisp). It’s basically real time coding where you ask the program “questions” and it replies which is fairly easy for someone like me that mainly codes in Python , so I have made a live coding library for python even though it does not support it by default like Smalltalk does. It basically reloads code without stopping execution and keeping live data intact.

I have made a similar library for C++ when I was coding in Pharo (Smalltalk). The python implementation is based on module reloading and exchanging references of instances to old classes (classes store both class and instance methods) with reference to new ones.

C/C++ is similar instead of reloading modules I reload DLLs and instead of echanging references I exchange pointers. Both implementation use exception handling in order not to allow the code to exit in case of error , instead live coding allows me to correct the error and carry on like nothing happened. Live coding also reduced C++ horrible compiling time down to almost 0 seconds, mainly because everything is broken down to small DLLs that take very littile time to compile, instead of having to recompile the entire code base.

So even in the case of C I can easily inspect objects, change references, play around with code and experiment on the fly without any penalties which makes code analysis not that necessary for me.

There are of course ton of techniques I learned while coding in Pharo, Smalltalk after all is even older than C and the entire language is based on live coding (in Lisp live coding is optional, in Smalltalk is mandatory and main feature). When ones can interact with code so fast and has powerful reflection abilities (which C does not have but Python does) its very easy to understand code compared to traditional dead code methods that can take a ton of time.

3 - DOES BLENDER HAS SOME SORT OF GLOBAL DATA-DICTIONARY somewhere? anywhere?
Where I can quickly find names, scopes, data-types of all variables used inside the source-code? Right now there are a few fragments of the “Blender Developer Wiki” but not of them paint the complete global overview picture:

Thanks for the tips so far Kilon. much appreciated!.

Wishing you a heppy weekend

Blender is based on two types BPY-DNA , which is a dictionary at least in Python and BPY-RNA , I think DNA is what it uses for most data, RNA I think is mainly for animation and modification of parameters. But yeah pretty much everything in blender is classic dictionary type of data. The specifics obviously are extremely complicate because Blender may looks small but its actually a huge app, over 1 million lines of code so data heavily depends on the blender context which is basically the internal Blender windows.

The good news is that the code has a lot of comments which is very rare for any code , so I don’t think you will have an issue understanding it but will take you some time but hey, it takes me forever to even figure out OpenGL 3 , so who am I to judge ? :smiley:

I advise to read the code but dont read it all only the areas that interest you. I have worked with BGL and I rarely had to get out of the folder that contained to look at dependencies but of course it happens sometimes when its something basic. In any case I dont think you will have an issue understanding it, just give it time.

A1: Not really. At that time, I was more comfortable making eletronic stuff with my father. Programming was something that I learned while using a HP-65 (was pretty old already when i start messing with it). Jumping to the Z80 was kind of easy, and there was lots of magazines with assembler code (mostly for poking games)… When I got my first 80386 I was so afraid of making stuff in it (a machine that costed me too much) that I dropped programming and started using programs that were already in the market… Specially the 3DConstructionKit which already had a game engine inside!!

A2: Blender is more or less my tool for everything. I use it mostly for illutrations, but it’s also my playground for testing ideas and stuff. I even use blender as a calculator, since I’ve it always opened.

I would start looking to the bpy.data; Though it’s a python API, it represents quite accuratelly the inside structure of the a blender file… Then, knowing what each editor does with that data is also important (operators, context polls). The deep loop stays in the Creator module (events, signals, etc), so understading how it works is also a nice thing to know. For Cycles i can’t say much as I’m still digesting the code, but as it’s mainly an independent module, there’s not much that links it to blender except for the nodes interface, and how the blend data is passed to Cycles.

Hello Everyone.
Sorry for the delay in my reaction time, since I’ve allowed myself to take a deep-dive in C again to refresh my old ANSI / Turbo C skills.
I’ve found some nice visually supported tutorials websites, which make re-learning C a fresh breeze in relation to the “before internet era” and high-school books.

Hello Kilon

Frankly not having any documentation is the norm in the coding world, especially for open source. The fact that Blender has outdate docs is a luxury. I am studying opengl and I am not sad , I am horrified how such an extremely important API have such terrible and so minimum documentation. 2018 and there is still only a handful of good opengl tutorials around.

Agreed, Since i’ve first browsed the OpenGL section of the internet too, in order to find out how Blender internally feeds its 3Dd models to the underlying visualization hardware pipeline.



Blender internal data flow - from 3D model to screen

It sometimes feels - to me - as if certain “software developers” deliberately only publish a PART of the whole API to the rest of the world, while feeding the complete API + documentation only to their preferred business partners and paying customers… I sense this (hiding everything behind a simlpy rotten documented API pattern) almost everywhere in the ict-industry.

No , this is not my style of coding. <cut> It basically reloads code without stopping execution and keeping live data intact.

What a great way of learning to code, especially since i’ve never heared of this principle before.

Q: What type of development environment / IDE do you use for it? Does is look and act something like-wise as and (in-circuit hardware) emulator / software-debugger / virtual machine which shows all under-the-hood information in real-time? like memory content, hardare register contents?

C/C++ is similar instead of reloading modules I reload DLLs and instead of echanging references I exchange pointers. Both implementation use exception handling in order not to allow the code to exit in case of error, instead live coding allows me to correct the error and carry on like nothing happened. Live coding also reduced C++ horrible compiling time down to almost 0 seconds, mainly because everything is broken down to small DLLs that take very littile time to compile, instead of having to recompile the entire code base.

This sounds pretty sophisticated Kilon, it feels almost like how real-time operating system (like Qnx, Minix) does the trick to stay alive under every circumstance. At the other side, … a whole lot of little DLL’s… isnt this creating a bucket-load of memory traffic overhead, disk i/o traffic and memory-fragmentation?

So even in the case of C I can easily inspect objects, change references, play around with code and experiment on the fly without any penalties which makes code analysis not that necessary for me.

Affirmative… you’ve basically rewritten the operating system rules to fit your own needs. Great!

There are of course ton of techniques I learned while coding in Pharo<cut> … dead code methods that can take a ton of time.

That I what I think too… since one can learn quicker, dynamically in real-time when one can see what goes right of crashes and why. This method of learning perfectly fits into the human-beings natural perception and learning abilities.

Blender is based on two types BPY-DNA , which is a dictionary at least in Python and BPY-RNA , I think DNA is what it uses for most data, RNA I think is mainly for animation and modification of parameters. But yeah pretty much everything in blender is classic dictionary type of data. The specifics obviously are extremely complicate because Blender may looks small but its actually a huge app,
over 1 million lines of code so data heavily depends on the blender context which is basically the internal Blender windows.

Yeap, from what i’ve read and documented already, Blender is a pretty amazing piece of coding Kilon, that is one of the main reason why I want to study it from the inside out. To learn from an actual real-life example and learn the tricks of the trade which are not mentioned in the “C-coding schoolbooks” of today.

The good news is that the code has a lot of comments which is very rare for any code, so I don’t think you will have an issue understanding it but will take you some time but hey, it takes me forever to even figure out OpenGL 3 , so who am I to judge ?

Before I react indepth on that, I’d first want to know the right context and want to read your #definition of <“a lot of comments”>
My first C-exercise was a little analyses program (input: c,cpp,py source-file, output:statistics-file) which simply counted:

  • the total number of characters in a blender (.c .cpp. py) sourcecode file
  • the total number of lines / source-code file
  • the total number of comment lines in between the source-code
    You do not want to know the exact numbers Kilon since the most “commentlines” in almost any Blender source-file can to be found in the header section, containing the GPL license text, eating a woppy 25 comment-lines at the beginning. In between the actual Blender (.c .cpp. py) source-code, one finds very few lines of explanatary comments on what this piece-of-code does ;-).

I advise to read the code but dont read it all only the areas that interest you. I have worked with BGL and I rarely had to get out of the folder that contained to look at dependencies but of course it happens sometimes when its something basic. In any case I dont think you will have an issue understanding it, just give it time.

Thanks for this tip Kilon.
I do have the intention to create a complete internal overview of Blender, (to make it 100& transparant for myself and others) so guess I have to digest it piece by piece then. <burp!>

I would start looking to the bpy.data; Though it’s a python API, it represents quite accurately the inside structure of the a blender file… Then, knowing what each editor does with that data is also important (operators, context polls).

Thanks Secrops for this reference.
Q: Do you know the AT MIND website already http://www.atmind.nl/ and its free tools to peek into the .blend files?

The deep loop stays in the Creator module (events, signals, etc), so understanding how it works is also a nice thing to know.

That is - for me - THE thing to understand, since it is Blender’s core routine, That is where I always what to start digging deeper into stuff.
since sooner of later I will end up there anyway :slight_smile:

For Cycles I can’t say much as I’m still digesting the code, but as it’s mainly an independent module, there’s not much that links it to blender except for the nodes interface, and how the blend data is passed to Cycles.

… and it seriously lacks overall descriptive / background documentation what “Cycles” technically IS ( a “BVH based” Renderer who fires rays from its camera-viewpoint to get things done) and not a old-school Rasterizing Renderer like Blender Internals.

Q: What type of development environment / IDE do you use for it? Does is look and act something like-wise as and (in-circuit hardware) emulator / software-debugger / virtual machine which shows all under-the-hood information in real-time? like memory content, hardare register contents?

It does not depends on an enviroment or IDE, it affects code execution. |I use PyCharm but essentially anything can be used. No its nowhere near as sophisticated as VM is pretty simple. A simple library that one command can reload a python module find instances referencing classes with same as the ones inside the module and replaced the references to the old classes when references to new ones. This way the object methods are replaced with an updated code and the process is automatic, variables are not affected in any way so you never lose your live data.

https://github.com/kilon/pylivecoding

You basically change your code save it and you see the applications executing the new code automatically.

This sounds pretty sophisticated Kilon, it feels almost like how real-time operating system (like Qnx, Minix) does the trick to stay alive under every circumstance. At the other side, … a whole lot of little DLL’s… isnt this creating a bucket-load of memory traffic overhead, disk i/o traffic and memory-fragmentation?

Not really , live coding is a whole coding philosophy that entire books have been written about. It’s pretty popular in game coding and DLL’s themselves is one aspect of the ide of live coding. I learned live coding in Lisp and Smalltalk but how to implement it in C and Python I was inspired by this video

https://www.youtube.com/watch?v=WMSBRk5WG58

Even though the video is quite long he barely scratch the surface and he goes to a bit more detail in the following two videos.

https://www.youtube.com/watch?v=oijEnriqqcs&t=20s
https://www.youtube.com/watch?v=xrUSrVvB21c&t=929s

So as you can see, this guys is hard focused on performance and use live coding. Now you may ask why live coding is so good performance wise even though it’s basically dynamic reloading of code which in theory sounds like a very bad idea if you want to get performance.

Fortunately, live coding is more than technique and a coding philosophy/ideology it’s also a necessity for fast debugging and code experimentation as well a plethora of coding scenarios. Not only all OSes are created with live coding support (see dlls) but live coding is even supported directly by the hardware. For example, the hardware allows you to manipulate the actual mechanism of machine code execution, while the OS allows you to changed a part of memory purely for data to being executable even if that part is shared memory between applications. Exceptions which an extremely important part of live coding because they capture errors without allowing the application to exit, crash or stop executing while at the same time reloading code dynamically is the very mechanism that DLLs are implemented which also have equivalents in pretty much any OS. Exceptions are supported by languages but also supported by OSes and even hardware.

Why live coding is not more heavily used ?

Mainly live coding is not always necessary, there are cases where you want immediate feedback to changes you make to the code but in many cases you may not care so much when you know what you are doing. Even in Smalltalk I was not relying on live coding all the time and the entire language, IDE, VM and libraries are built on top of its live coding mechanism.

In the end live coding is about controlling the execution of the code during run time, as you can imagine it can infinite amount of usages and is another tool for your toolbox if you choose to use it.

Live coding is actually very popular in the game creation world , Unreal for example supports live coding for C++ code from it’s own editor and there are many similar tools around.

It’s also a great way to reduce compile times to zero. Even myriads of DLLs can be turned into static libraries that become one during compilation if you so choose. Blender, for example, has limited reliance on DLLs but most software , including 3d apps and games, do rely a lot more as it makes it easier to update games, without having to redownload the whole game which is what we do to get the latest update of Blender.

I really advising watching those videos in your time, probably in the span of weeks / months to get an idea how useful it can be.

It’s not hard to implement and will change the way you code forever.

Hi.

Since this is a general development topic and since it involves actual work in the source code and the technical details of it. I think the BF’s new developer forum might be of interest.

I’m not a Blender developer by any means, but I wanted to point this out.

@Ace Dragon: Thanks for the reference to the Blender Devtalk website. I will check it out soon.

1986? “Bah! Humbug! These kids today …” :yes:

Well do I remember those days – but I certainly don’t miss them.

People who watch the original Star Wars movie don’t realize that the only place where a computer actually appears is in a Tektronix display in Luke’s fighter console, and in the creation of the Death Star diagram that is projected in the Rebel’s conference-room as they plan the subsequent attack. It was all done with motion-controlled photography and multiple exposure, because that was the only way to do it then. (And just look at what they did!)

I was fortunate enough to see it – in a single-screen movie theater that was four broad aisles wide – when the “crawl” simply said, STAR WARS. (No “episode.”) And long before George Lucas utterly butchered it.