Blender Physics History, and my (our?) dream for a brighter future

General Forums Blender and CG Discussions
1

WARNING: Wall of text ahead.

Physics simulations in blender did go (and are still going) through a lot of ups and downs throughout the years, tracing all the way back to version 2.40 (the oldest one with “proper” release notes I could find).

2.40:

  • New Fluid simulations using “El’Beem” or LBM (Lattice-Boltzmann Method)
  • BGE (Blender Game Engine), Particles and Cloth were all already there (pre 2.40)

2.41: …
2.42:

  • New Physics Engine
  • Fluid sim + Particles + BGE improvements

2.43: Fluid sim + BGE improvements
2.44: BGE improvements
2.45: …
2.46:

  • New Particles System
  • Fluid sim + BGE improvements

2.47: …
2.48: Fluid sim improvements
2.49: BGE improvements

2.50-2.59: … (Couldn’t find a “proper” release note for what happened during this period)

[EDIT]

  • New Smoke simulation system
  • New Particles System

[/EDIT]

2.60: BGE improvements
2.61: …
2.62: BGE improvements
2.63: …
2.64: BGE improvements
2.65:

  • New Fire type Simulations
  • other Fluid sim improvements

2.66:

  • New Fluid simulations using SPH (Smoothed Particles Hydrodynamics)
  • New RBD (Rigid Body Dynamics) in the viewport (previously only available in BGE)
  • BGE improvements

2.67: …
2.68: Fluid sim + Particles improvements
2.69: …

2.70: …
2.71: BGE improvements
2.72: BGE improvements
2.73: …
2.74: Particles (Hair) improvements
2.75: BGE improvements
2.76: BGE improvements
2.77: BGE improvements
2.78: …
2.79: …

during 2.80 dev period: Bye Bye BGE…

2.80: …
2.81: …
2.82:

  • New “Mantaflow” Fluid (Fire,Smoke/Liquid FLIP “Fluid Implicit Particle”) simulation system
  • Cloth improvements

2.83: Fluid + Cloth + Hair improvements

2.90: Cloth improvements
2.91: Fluid + RBD improvements
2.92: Fluid improvements

  • 37 Releases (2.40-2.92 “excluding 2.5x”)
  • 13 of them had zero physics improvements
  • 10 releases had only BGE improvements (most of 2.7x) which ended up removed…
  • Particles have been reworked once (2.46) and are being reworked again (particle nodes)
  • Fluids have been reworked twice, from LBM to SPH to FLIP/APIC

The current state of blender physics as of “01/22/2021” with 2.91 as the latest stable release:

  • The Particle system is starting to show it’s age (implemented 12 years ago)
  • The Fluid system have a few nasty bugs and is limited in what it can do compared to many new techniques
  • Bullet engine is lacking GPU acceleration and soft bodies are a pain to work with
  • Cloth is crying in the corner with nobody to play with
  • The only way to fracture things is an old addon (cell fracture) stuck on version 0.2 with a slow algorithm and disruptive workflow (the fracture modifier branch have quite an amazing set of tools that could help, but there seems to be a few road blocks that needs to be cleared first “like right now ? please ?” the devs should play well together for the better future of Blender !)

Basically nothing is working “properly” with 3/4 engines doing their own thing with little communication between them.

Isn’t it time to get things done the “right” way ?

What I’m hoping for, is a Multi-Physics solver that let all aspects of physics to play nice with each other.

There are many white papers about all kinds of techniques for two way coupling for fluid-rigid interaction, just like Cycles/EEVEE, some of them are geared toward physically based methods, while others are suited for real time simulations with less emphasize on accuracy, to name a few:

(Latest iteration 2019)

Paper: https://animation.rwth-aachen.de/media/papers/63/2019-TOG-StrongCoupling.pdf

They rendered most of those simulations in Blender, they even recorded an AR simulation with the blender rocket sitting there in the corner :slight_smile:

1366×768 180 KB

Same people with their first iteration of the technique back in 2012

One more technique leveraging NVIDIA’s (Now Open source) Physx for real time Fluid-Rigid interaction (SIGGRAPH 2014)

Paper: http://mmacklin.com/uppfrta_preprint.pdf

With such techniques, not only it would open up all kinds of interactions, but it will also help the dev maintain ONE single API/Framework/Engine instead multiple conflicting ones that are limited on their own and provide more work then needed. (or so it seems, correct me if I’m wrong).

Blender did come a long way, I can’t deny the huge efforts, but it seems to me that some parts are being “neglected” in favor of others (lack of man power I guess?), blender have now a nice amount of funding (in hope for more) to afford small teams per module.

  • How about we start by updating those module roadmaps please ?

Thanks In advance for anything new physics related in the horizon.

Voidium.

4 Likes
2

Most of the series was considered alpha,beta releases. 2.57 was officially first stable release of series.
(But personally, I consider that was 2.59 because many animation bugs were not solved until this release.)
So, release logs of all releases of the series were merged together.
https://archive.blender.org/development/release-logs/blender-259/index.html
During 2.5x, smoke simulation was integrated for Sintel movie. (But it was not really used for the movie, because artists did not really know how to use it efficiently. They did not have time to play with it.
They succeeded to make a Tornado with it for Cosmos Laundraumat, years later.)
During 2.5x, particles were refactored to match new UI.
Almost every aspect of particle system has been improved during those 3 years of development.
http://jahkaparticles.blogspot.com/

And system that is supposed to replace it is currently in development.
A work was done on Simulation nodes and is in stand-by during development of Geometry Nodes.

No, Mantaflow replaced LBM.
SPH mention in 2.66 release logs are relative to improvements of Fluid physic type of Emitter particles.
That solution is still available with particle system in 2.91.
Mantaflow’s fluid modifier replaced old Elbeem fluid modifier and old smoke modifier.

It is sure that is not GPU driven.
It was merged too soon without enough documentation and examples.
That may explain problem and bugs.
But that is a recent change about a framework that is still in development and will continue to evolve. During its early state Manftaflow was supporting CUDA.
Currently, you have to choose between Gas or Liquid domain type.
But there was the promise that after a while, a domain could handle both simulations, at same time.

There was an hope to have it handling rigid bodies, softbodies and, maybe, cloth at same time.
And maybe, one day, see it having a GPU support when it was chosen for rigid bodies simulation.
That day did not happen.
Cloth simulation was kept apart because bullet results were judged not satisfying enough.

There was a claim that fracture modifier will be included in 2.8x.
But during 2.8x development, that target was abandoned. The idea of a modifier became less and less attractive to its developers who wanted to develop a whole set of tools.

No. Blender does not have funding to afford small teams per module.
It has money for around twenty developers. Maybe thirty in some years.
There are officially 13 modules. What would currently correspond to around 1.5 _ 2 developers per module. That is too small to be called a small team.
In reality, things are not happening that way.
Blender is a successful project because many volunteers or people employed by external companies and opensource projects are contributing to Blender code.
Developers are organized to collect and review those external contributions.
If you take a look to how module are handled, you will understand that several developers are contributing to many modules.
https://wiki.blender.org/wiki/Modules
That is a question of experience with Blender code. Things are not always obvious and they need old members to remember purpose of old code.
They also need to reduce risks of a module becoming orphan in case a developer wanted to quit the project.
Current situation is that newer developers with less experience can be affected to only one module (but already in one module, there are hundreds or thousands of tasks to do).
And developers with more experience are trying to assure a back-up on several crucial modules.
And several names are corresponding to volunteers or people not employed by blender foundation.

So, the team is too small to really be split.
But that is not only a lack of developers that is responsible for current situation.

There is a desire to unify physics together since 2013. It was a target for Cosmos Laundromat movie.
But, first, that requires a way to unify physics cache, an unified UI to expose physics solvers and free opensource solvers.

  • So, developers unified cache of cloth, softbodies and particles with point cache system.
    But that can not handle geometry from fluids and rigid bodies. So, they focused on Alembic and OpenVDB support.
  • About the UI, they developed a custom nodes python API used by add-ons like animation nodes.
    And they did several dozen of theoretical design documents before starting concrete tries of future “everything nodes” UI.
  • They took a look to free opensource solutions compatible solutions that seemed to be solid. There was none doing everything at that period. There is a often a gap between one convincing demo video and everyday practice. They chose to follow path of several solutions delivering correct results instead of one and to rely on humans with a desire to support a physics field. Bullet Rigid bodies and Mantaflow Fluids are the fruit of GSOCs where it was possible to find mentors and students. They chose to continue with work done by their GSOC students and enthusiasts like Luca Rood is about Cloth simulation.

Of course, with all the GPU stuff happening, that is frustrating. We passed to almost everything covered with fracture modifier in 2.79. To almost everything broken by 2.8 refactoring. Current situation will not stay as is. If they follow their roadmap, developers will fix bugs and things will become more reliable.
That is not the perfect moment to abandon short-term targets to long-term ones.
Physics are really a complicated field. A multi-physics solution would probably imply to have several experienced physics developers dedicated to it, at same time. Not just one enthusiast. That implies something that is widely recognized as good. And it looks like speed versus quality is still a debate for some cases.
IMO, “Everything Nodes” UI will not be ready to handle such thing before several years.
During those years, a FOSS GPU solution will probably come up.
I don’t think we have to convince any developer that is the way to go.
They have heard about Embergen, other GPU driven fluid particles solutions or Marvelous designer.

But things that are involving humans don’t magically happen at lightspeed. Big projects like that are staying projects during years before being integrated.
If researchers can show a work that took them years to develop, in one video ; that still takes years to developers to become familiar with, and still years of user feedback to make it officially supported in an official release.
So, finding a way to combine several good ideas into one unified solution : that is work for a decade of years.

5 Likes
3

Thanks for the explanations, always good to know more about behind the scenes.

What led me to think otherwise was their 2019 highlights.

766×146 12.2 KB

Along with the recent 2021 big project statement

763×247 18.2 KB

I know little about how organizations work, but when I read such statement, It seems like they had enough devs to cover all modules and now they are tackling the management side of things.

But based on the modules link mentioned above, there are some which are left in the dust

838×444 30.2 KB

Speaking of which, It is always sad to see projects like last year’s Volumetric Softbody GSoC by Mattoverby, end up rejected with no plans from the person behind it to continue the work later on.

https://wiki.blender.org/wiki/User:Mattoverby/GSoC2020/mattoverby_final_report


example by: Everton Schneider

It looked too promising, but…yeah roadblocks.

2 Likes
4

I’m gonna guess it has to do with further expansion and bringing some structure to the development side. While I’m sure it’s necessary and I find the widespread sprint workflow efficient I can’t help but wonder if scrum and the likes will have a negative impact on some of the developers morale.

That aside, thanks for writing up a bit of history! Interesting to see a bigger picture, helps to set expectations as well. I think it’s mostly about that - knowing what can be done and what can be expected in the near future.

While I find long term plans like everything nodes interesting, they are anything but helpful from a production side. I perfectly understand the limitations of a small team thinly spread across so many fields, that doesn’t change the fact though that the rest of the industry is not in a standstill, and people/companies who can afford it and need to get a job done have to choose the most practical solutions.

Long story short, I do like Blender a lot. For some things it just falls short. OpenVDB and USD support can at least help to create some sort of a bridge. Looking forward to some cool new additions/features on the physics side, and hopefully we’ll see the word GPU appear in a lot of them.

(And yes, the softbody GSOC branch not making it was a shame. Even something broken and incomplete is better than what we have currently (for which I struggle to find words), just put it under experimental or create a switch for the adventurous…)

5

Physics is such a hard field. Takes ages to develop and ages to review… And blender wants a general system that is pipeline proof and polished. The system blender uses is just old but somehow complete to work but most of the time not good or at least not easy to dial in the settings.

We just got a dedicated physics developer for mantaflow and thats a great start. It takes time but plans are made with a unified physics simulation and thats also why the fracture modifier is waiting to be merged. Some changes with the rigid body system but that modifier is a beast.

To take is short: I wish more devs want to work on physics but the current state can only get better and for some things it works great.

1 Like
6

There is somebody in charge of each module.
Whatever bug you report is about, somebody will try to fix it if it is about an high priority problem.
There are experienced developers that have their name mentioned almost everywhere (Sergey, Campbell…) in the list.
That also means that those guys are sharing their time between many fields. And for each field, they have less time to dedicate to it than a developer that supports less of them.
And now, with the recent increase of developers to help them, they have to pass more time to communicate with more people.
Names mentioned in module list are not always corresponding to an employee of blender institute working full time on Blender Development. Some people may be have another employer, may not work in Amsterdam but in another country.
So, communication requires an organization.
And there is no proportional linear growth between increase of developers and tasks accomplished, to expect in present and near future.

A transition is necessary to transfer workload and knowledge from old experienced developers to new ones.
And to do that, they need to reorganize themselves and check what works and what does not.
That is what the management side is about.

7

I found this “relic” link of 2007 where blender team made a “Physics Sprint” to implement hair for Big Buck Bunny.

So the idea of Physics Sprint 2.0 is totally possible as long as they get enough brains interested in the topic.

2 Likes