FLIP Fluids Addon: A liquid fluid simulation tool for Blender

Just a general question about viscosity. Do the numbers in the addon have any actual relevance to real world numbers, like IOR is for transparency (or at uneven wrong there?) or are they just arbitrary numbers in terms of 1 is low, 10 is high, kind of a way?

The viscosity value does not correspond to a real-life quantity. Documentation here: Domain World Settings: Viscosity

The value of viscosity to use in this simulation. A higher viscosity value relates to a ‘thicker’ fluid.

NOTE: The visual ‘thickness’ of the fluid is dependant on the size of the simulation domain. For example, a viscosity value may appear ‘thick’ in a small domain size, but if the same value is used in a larger domain, the fluid may appear visually ‘thin’.

NOTE: The viscosity value does not correspond to any real-life physical quantity. It is just a number that specifies the amount of viscosity.

TIP: The viscosity method is highly accurate for high viscosity fluids that buckle and coil and this level of accuracy will increase simulation time. For very low viscosity or thin fluids, enabling the viscosity solver can be overkill. A trick to bypass the solver for very low viscosity fluids is to instead increase the PIC/FLIP Ratio setting in the Advanced Panel. This will lower the accuracy of the simulation which naturally results in a more viscous looking fluid. Try comparing the default value of 0.05 with a value of 1.0 to get an idea for how this changes the behaviour of the fluid.

2 Likes

I love how 70% of your replies on this thread consists of RTFM :sweat_smile:

1 Like

A question about thicker fluids.

If I up the viscosity, do I also need to (or should I) increase the friction on an obstacle so that it slides off it in a gooey type of way?

Due to they way that the viscosity simulation works, thicker fluids will often always stick to the obstacle even if the friction is set to 0.0. Increasing friction above 0.0 will result in the fluid sticking more tightly against the obstacle.

1 Like

A question about many obstacles. In the attached image you can see that I’m making a big cylinder from many smaller cylinders. 60 to be exact. The big cylinder is rotating, by having the small cylinders using a Follow Path constraint on a circular path. This is taking a very long time to simulate/bake for each frame. Is the slowness entirely due to the large number of obstacles or is it also due to having Export Animated Mesh turned on for each small cylinder?

In the next iteration I’m going to also have the small cylinders rotate individually on their centers, like a conveyor belt, so the obvious solutions (at least the ones I can think of) to not using Follow Path won’t work but before I investigate other ways of making them rotate together I wanted to know if the Export Animated Mesh is part of the problem.

Nitram_2000, you could also try increasing the surface tension.

As a side note, in your preferences do not set the Temporary Files to a folder/directory where you also have your blend files. I had mine set to d:/blender and then I was making separate folders in that for each .blend file that I was working on/using. Recently blender got some exception and before I realized what was going on I watched it delete all of those folders. Luckily I’m just a hobbyist so nothing crucial was deleted.

Anyhow, the point of the above is that I lost the blend file for this animation. I think I had both viscosity and surface tension turned on but I don’t remember the values.

1 Like

That’s a good question! I am currently away from my development system and will need to take a look at the code before a definite answer. I’ll be able to get back to you within the next few days or January 6th at the latest.

Short answer: it is likely caused by the large number of obstacles, but I’m sure there can be some simple optimizations made within the simulator to make scenes with many small obstacles faster.

If you still have the simulation cache available, would you be able to post a screen shot of the FLIP Fluid Stats panel set to Frame Info for a typical frame? The timing stats will give me an idea on how the simulator is performing and what calculations are taking a long time.

Here’s the screen shot. Looks like it’s the many collision objects, to me at least. The domain is 5 meters all three sides, 320 resolution, viscosity 5, surface tension 25. It’s taking about 45 minutes per frame; heh.

In the Advanced Settings, Frame Substeps, Max, I’ve set it to 100 so that it never clamps/clips what’s needed for Surface Tension. For my current experiment of Surface Tension at 25, domain at 5 meters cube, the estimated substeps is 38. When is clamping the Frame Substeps to a maximum value useful, would it be for example if I’m not using surface tension? I was wondering if it would be helpful to have a checkbox in the Surface Tension panel to let it automatically override (increase) the setting for max frame substeps.

i found a very small but kinda funny bug, i have a 3D scan called Laocoon and His Sons and whenever i name a fluid object that name by pasting it in the simulation fails and can not find the object with that name, retyping the name manually instead of copy pasting it or using a different name fixes it.
http://pasteall.org/blend/index.php?id=52731

It’s “Laocoön and His Sons”; notice the umlaut over the last o character. I’m guessing that that’s the problem.

1 Like

Thanks for letting me know about this! It looks like this issue is related to this bug: NotADirectoryError when exporting objects that contain special filesystem characters in the name. And is caused by a side effect of the Windows filesystem that I was not aware of.

The addon stores the mesh files according to the name of the object. The problem is that there is a trailing space at the end of the the object name: [Laocoon and His Sons ]. Windows will let you create this file without any errors, but as soon as it is saved, the filename is updated to remove the trailing space. This is causing the addon to not be able to locate the file again after creation.

Anyways, having the addon create files based on the exact name of the object was a bad design decision. Should probably get around to fixing this soon!

1 Like

Thanks for attaching the screenshot! 61% of the frame time spent on updating the Simulation Objects category is quite high. I’ll run some tests to see how this can be optimized.

There is an explanation for why the max substeps value is needed in this documentation topic: What are substeps, and how do the min, max, and CFL parameters relate to each other?

The Max Substeps parameter sets the limit for how many substeps the simulator is allowed to take during a frame. If some fluid particles are moving fast enough so that the required number of substeps exceeds this parameter, the simulator will remove these particles from the simulation. The reason for having this parameter is to prevent simulation ‘blow ups’ in extreme cases. There is a chance that a particle may become unstable and take on an extremely high velocity. This could cause the simulator to require a very large number of substeps during a frame, blowing up the baking time. There is also a chance this particle could affect the data of other particles leading to a chain reaction. Having a limit for the number of substeps will eliminate this unstable particle before it can cause any trouble.

I think it would be. Something similar to this is on my todo list for when I get back to regular development starting on Monday.

Is there a reason why my spray and foam are going outside of my tubular obstacle?

I had to scale the res up to 250 even before I could get the fluid to stay inside, using the debug mode from the help video, but it seems that the whitewater stuff acts a little different.

I’m using the “On Surface” method as I won’t render the tube itself.

I think I would need to see a .blend file to understand what could be causing this.

If the tube is not being rendered, something that could help the simulator run better is modelling the tube with thicker walls. This will also reduce the resolution requirement to resolve the thin walls and also helps to prevent leaks.

Let me clean up the file and I’ll send it to you through the Blender Market messaging system as per your help page. This one can’t be shared publicly unfortunately.

Alright, I’m back from holidays and have just had a chance to check this out and run some tests! This might end up being a long explanation (it was)!

TL;DR: The simulator chooses between two different ways to multithread the calculations for handling obstacle objects. For this scene, the simulator chose a less than ideal scheme.

The slowdown will be caused by the large number of objects rather than if the object have the Export Animated Mesh option enabled. The animated mesh option will only affect the time it takes to export the mesh from Blender for use in the fluid engine. From the point of view of the simulator, it only sees a sequence of meshes and does not know whether the object was keyframe exported or animated exported.

How are animated/dynamic obstacles handled in the simulator?

For obstacles to interact with the fluid and to be used in the physics calculations, their geometry needs to be converted to another form. The objects are converted to volumetric data called a Signed Distance Field (SDF for short). A SDF is basically a grid of data that tells us whether a point is inside or outside of the object volume, and also stores the distance to the nearest point on the volume surface. You may have heard about SDFs as one of the data structures in OpenVDB.

Updating the SDF can be quite an expensive operation within the engine. Every time an object moves in the simulator, the SDF must be updated. Even during substeps, we need to update the SDF. The meshes are interpolated within the frame between substeps. So if you have 38 substeps in a frame, the SDF will be updated 38 times if there are moving objects and all that time really adds up.

Since the SDF update is so expensive, we have some optimizations to reduce the amount of data that needs to be updated. If objects are not moving, they only need to be computed once. The engine uses separate grids to store the static obstacles and dynamic obstacles, and also a master grid that merges and stores all of the SDF information. So if you have a few static obstacles and a few dynamic obstacles, the static and dynamic data will be stored on the separate static and dynamic grids. This avoids needing to re-compute the static data.

Animated obstacles can also start/stop their motion. Another optimization is that the engine automatically detects if a dynamic object is actually moving/changing during the frame. If a dynamic object is static during part of the animation, it will be moved and stored on the static grid until it starts moving and needs to be updated again. This is another way that we avoid re-computing SDF.

Additional optimizations for dynamic objects

And then there are more optimizations within the simulator specifically for handling dynamic objects. Due to the previous optimizations of separating static/dynamic objects, the majority of the time spent updating the SDF concerns only the dynamic objects in the simulation. There is some further optimization for how we multithread dynamic SDF calculations.

Inside the engine, the meshes may not be represented exactly as they are in the Blender scene. The engine actually separates all of the mesh geometry into separate pieces if it can (we’ll call each these mesh islands). If the geometry of a Blender object can be separated into individual pieces, the engine will do this and treat each piece/island as a separate object. For example, if you have a single Blender object that is made up of 60 cylinders, the engine will separate this object into 60 mesh islands that will be each handled separately in the engine. This optimization is to create smaller object volumes when possible and smaller volumes are quicker to compute than a single large volume containing many pieces.


How are the SDF calculations for dynamic mesh islands multithreaded?

There are two different multithreading schemes in the engine for how all of the mesh islands are converted into an SDF. For the rest of the explanations, I’ll just assume there are 8 threads available.

Scheme 1: Calculate each mesh island one by one and assign 8 threads to work on each island

Here is some general pseudocode for how this works:

for each mesh island:
    1. Create a tight fitting grid around the island.
    2. Divide the grid into subgrids (typically smaller chunks
       of 8 x 8 x 8 voxels). 
    3. Put each subgrid into a queue for processing
    4. Launch 8 threads. each thread will:
        4.1. pull a subgrid off the queue
        4.2. calculate the SDF for this small region
        4.3. merge the subgrid SDF into the mesh island grid
        4.4. repeat until processing queue is empty
    5. Finally, merge the mesh island grid with the dynamic SDF grid

Pros:

  • Memory efficient. Memory only needs to be reserved to compute a single mesh island at a time.
  • Fast for large obstacles that cover a lot of grid space

Cons:

  • Can be slow for small obstacles that do not cover much grid space. For example, for thin narrow cylinders, having many threads work on a single object can be overkill and also add a lot overhead

This multithreading scheme is what is being used in your scene and is the reason for why the SDF update is running slowly.

Scheme 2: Calculate 8 mesh islands at a time and assign 1 thread to work on each island

Here is some general pseudocode for how this works:

1. Put each mesh island into a queue for processing
2. Launch 8 threads. Each thread will:
    2.1. pull a mesh island off the queue
    2.2. create a tight fitting grid around the island
    2.3. calculate the SDF for the entire island grid in the single thread
    2.4. merge the mesh island grid with the dynamic SDF grid
    2.5. repeat until the processing queue is empty

Pros:

  • Very fast if the mesh islands are small and do not cover much grid space

Cons:

  • Uses more memory than scheme 1 since now 8 mesh island grids need to be reserved at once. If all mesh islands are large and cover a lot of grid space, this scheme has the potential to use a very large amount of memory. If all mesh islands are small however, the increase in memory can be negligible.
  • If there are not a large number of objects and the objects vary in size, there may be wasted computing power. For example if there are 7 small objects and 1 large object, the 7 threads will complete quickly and the simulator will be waiting on a single thread to complete the single large object.

When is each multithreading scheme used in the simulator?

In general, scheme 1 is used most of the time due to the assumption that most use cases for the simulator would involve a small number of obstacles that are on the larger side.

Scheme 2 is mostly used for simulations that involve the Blender Fracture Modifier branch. Early on in development we decided that we wanted to support the Fracture Modifier within the FLIP Fluids simulator. Fracture modifier simulations could contain hundreds or thousands of small obstacle pieces, which would be too slow to process under scheme 1. Scheme 2 was designed around adding fracture modifier support.

There is one thing to know about the fracture modifier that determines how the scheme 2 optimization is triggered: Fracture modifier simulations are entirely contained in a single Blender object. Hundreds of small individually moving pieces are contained in the object. This fact is what determines whether to use Scheme 1 or 2.

Whether to use scheme 1 or 2 depends on the number of mesh islands contained in a single Blender object. The engine is not limited to use a single scheme in an SDF update and will choose which to use on a per-Blender-object basis.

Specifics:

  • Scheme 1 - Will be used if the number of mesh islands in a single Blender object is less than or equal to 25.
  • Scheme 2 - Will be used if the number of mesh islands in a single Blender object is greater than 25.

Since each of the 60 cylinders are separate Blender objects in your scene, scheme 1 will be chosen by the engine. Scheme 2 would actually be optimal for this scene.

Testing and timing results

I have recreated a scene with a similar setup that you had described. Since this optimization testing mainly concerns the SDF update, I removed surface tension and viscosity features to leave out irrelevant computations. The timing to update the SDF was measured for single steps over a few frames on an Intel i7-7700 @3.60 GHz CPU.

cylinder_setup

Scheme 1: 48 seconds per substep
Scheme 2: 8 seconds per substep

This shows that scheme 2 would be a much better choice for multithreading this type of scene.

Forcing your scene to use scheme 2

A way to force the simulator to choose scheme 2 would be to merge all of the cylinder objects into a single Blender object. As far as I know there is not a simple way to do this in Blender when each mesh island needs to move independently of eachother. Scripting is the only way I could think of doing this.

This example script will merge all objects whos name starts with Cylinder. The mesh will be merged into a single object named MeshCache and will update its geometry every frame:

# This script will merge all objects that contain 
# the prefix below into a single procedural object 
OBJECT_PREFIX = "Cylinder"

# Objects will be merged and stored in an object 
# named below. This object can be set as a FLIP Fluid 
# object for use in the simulator.
MERGED_OBJECT_NAME = "MeshCache"

import bpy


def initialize_mesh_cache_object():
    global MERGED_OBJECT_NAME
    if bpy.data.objects.get(MERGED_OBJECT_NAME) is None:
        mesh_cache_data = bpy.data.meshes.new("mesh_cache_data")
        mesh_cache_data.from_pydata([], [], [])
        mesh_cache_object = bpy.data.objects.new(MERGED_OBJECT_NAME, mesh_cache_data)
        bpy.context.scene.collection.objects.link(mesh_cache_object)
        

def frame_change_post(scene):
    global OBJECT_PREFIX
    global MERGED_OBJECT_NAME
    
    initialize_mesh_cache_object()
    
    object_list = []
    for obj in bpy.data.objects:
        if obj.name.startswith(OBJECT_PREFIX):
            object_list.append(obj)
            
    vertex_tuples = []
    triangle_tuples = []
    index_offset = 0
    depsgraph = bpy.context.evaluated_depsgraph_get()
    for obj in object_list:
        obj_eval = obj.evaluated_get(depsgraph)
        new_mesh = obj_eval.to_mesh()
        
        for mv in new_mesh.vertices:
            v = obj.matrix_world @ mv.co
            vertex_tuples.append((v.x, v.y, v.z))
        for t in new_mesh.polygons:
            triangle_batch = []
            for idx in t.vertices:
                triangle_batch.append(idx + index_offset)
            triangle_tuples.append(tuple(triangle_batch))
        
        index_offset += len(new_mesh.vertices)
        
    mesh_cache_object = bpy.data.objects.get(MERGED_OBJECT_NAME)
    mesh_cache_object.data.clear_geometry()
    mesh_cache_object.data.from_pydata(vertex_tuples, [], triangle_tuples)
     

initialize_mesh_cache_object()
bpy.app.handlers.frame_change_post.insert(0, frame_change_post)

The MeshCache object can then be set as a FLIP Fluid obstacle with the Export Animated Mesh option enabled. Since the simulator will detect this as an object with more than 25 mesh islands, scheme 2 will be used.

An example .blend file including the script in case you would like to take a look at the setup: many_objects_fracture_optimization.blend (2.2 MB)

You will need to press the Run Script button in the text editor window to activate the script. This script is only supported in Blender 2.81 or later.

How can we optimize the SDF update in the future to better handle this type of scene?

  • Smarter triggers for choosing between scheme 1 and 2. Maybe base this choice on grid coverage of mesh islands?
  • Maybe we could use a mixure of scheme 1 and 2. For example, compute 4 mesh islands at a time and assign 2 threads to each mesh island.
  • Does the SDF really need to be updated on every substep? Maybe we can get away with updating the SDF less often when there are many substeps during a frame?
  • Do we need to calculate the entire SDF grid? Maybe we could only compute parts of the SDF that are in close proximity of the fluid?
  • Add options to select an object primitive rather than a general mesh. The SDF is expensive to compute because it handles general meshes of any shape. Adding primitives with known shapes are very quick to compute (Cubes, Spheres, Cylinders, Cones, etc.).

There are a lot of ways we can make the simulator faster. Optimization is one of my favourite parts of development. It just takes time to develop! Some optimizations can take a lot longer to develop than others.


Anyway, this explanation ended up being a lot longer than I had expected and did not plan to go this much into detail how the internals of the engine works!

1 Like

Dern, you da man with that detailed explanation!

I’ll look at your script; so that’s making each cylinder rotate on its own, around its own center, even though they’re all part of the same object? I was wondering about that, if it’s possible.

After I’d had it running for a few days I slapped my head and realized that since I don’t yet have the individual cylinders rotating around their centers that I could join them all and make it one object, so I did that, but it’s still slow as molasses in winter.

What the script is doing is merging all of the geometry of the cylinders into a single object. It does this by running a script on every frame change. When a frame changes, the scripts will gather all of the geometry of the cylinders and load it into the MeshCache object. In this way, each objects can move independently of each other and the motion of all objects can be captured in the combined mesh.

A FLIP simulation using a large number of substeps will take a very long time to compute. After around 6 - 10 substeps, the speed benefits of the FLIP simulation method will be lost and can result in a much slower simulation compared to other methods such as SPH.

SPH methods often require a very large number of substeps per frame for stability. FLIP generally requires a relatively low number of substeps to remain stable. Many scenarios ideal for FLIP require around 1 to 4 substeps while an SPH simulator may require 20+. However, a FLIP substep often takes longer to compute than an SPH substep. So there is a point where FLIP loses it’s speed advantages when many substeps are required.

The FLIP Fluids simulator was designed to work well with small to large scale simulations. Such as the range from a glass of water to large beach waves. This simulator and many other computer graphics simulators are designed for this scale range and may not perform well for micro-scale simulations such as on the scale of individual droplets. At very high surface tension values, simulations become more like a micro-scale simulation of droplets, which can result in very long simulation times, instability, and inaccurate results.

These scenarios can still be done in the simulator, they just take a lot of patience and can be more difficult to setup. They’re still fun to experiment with and the results you’ve had with with your high surface tension simulations have been certainly interesting to watch!