(More) Accurate 3DGS rendering

Edit: See the latest version below! The instructions that follow in this post are now out of date.

Original post continues:
Hello!

Been playing around with Gaussian splatting, and felt the need for a better rendering option than the default plugin - don’t get me wrong, the plugin is great for import/export, simple visualization and editing, but as others have pointed out it isn’t fully accurate.

So I put together an alternate geo/shader nodes setup that allows both fast, point-based previs and accurate rendering! It’s still a work in progress, here is what I have so far (in order, 3DGS reference paper image, my mesh rendering, my point rendering):

It’s not perfect and I intend to keep on working on it, but here is a good first pass. To use it, you need the Reshot AI addon to import the .ply file, then set the geo nodes modifier to the modifier in the following blend file:
V1Pub.blend (1.6 MB)

Feedback/comments/suggestions appreciated!

Notes:
The 3DGS train was done on my machine from the MipNerf360 dataset at 7k iterations.
There is a technically correct transparent point shader hidden in the GaussianPoint material, but it doesn’t practically work because blender has a maximum of 1024 transparent bounces (which don’t cut it when you are rendering millions of points). To play with it unplug the dot product node (BSDF transparency ← multiply) and plug in the exponent node that is left unconnected.

8 Likes

Looks pretty cool already ! Keep up the good work !

New version!
Still some issues with accurate opacity, but in mesh mode splats render with proper Gaussian falloff!


Hope people find this useful, am still planning on improving on this a bit.

Blend file containing modifiers:
V2Pub.blend (1.7 MB)

1 Like

In the meantime, random cool NPR result:

Latest version!

As ever here is the blend file (now with the nodes setup marked as an asset for easier use!): V3Pub.blend (1.8 MB)

Still a couple of opacity glitches, but managed to fix some and improve the color to more closely match the original.

3 Likes

Hi there, thanks for sharing. Looks like a very promising work!
I’ve tried to apply this your geometry node on one of my .ply import but it doesn’t seem to work on my side. You have to apply your asset on top of the geometry node created by the plugin right ?

I’m running blender 4.0 and my gaussian splat was trained in Postshot and export directly to Blender. A short tutorial on how to set this up would be nice :slight_smile:

Hello and welcome to blender artists!
For the asset, no, it’s meant to replace the geometry node created by the plugin, like this:


As for a tutorial I’m planning on adding a guide to the first post so that it’s visible for everyone.

1 Like

Thanks!
The render as points works very well, however once I uncheck the option to render the splats I only get these glitches. Increasing the subdivision level only slightly improves the result. Any idea of what could be going wrong?

Is the splat rotated by any chance? The way the addon is importing the scale means that it rotation doesn’t currently affect the transparency shader, only the geometry.

Also, just checked and the addon rotates the object by default, will update guide so it mentions resetting the rotation.

Finally, make sure that you turn up the transparent samples.

Yes it was! Resetting the rotation doesn’t have much effect, but increasing the transparency samples indeed improve things a bit, although it still isn’t on par with your results.

The performance is also very low. I’m on a 3090 with an i9900K and Blender is barely running. Is that normal?

Would you mind sharing the example file from your previous posts?

1 Like

Performance wise I’m on a 3060 and my experience is largely similar, the viewport stops being interactive and more like rendering out a preview; adjusting any setting appart from reducing the view percentage or enabling point rendering is normally a bad idea. I think it’s simply the fact that the mesh preview generates millions of (instanced) meshes with a face count in the tens of millions.

I’m sorry about the quality, I must admit that my results with custom data (instead of their datasets) have been a bit mixed, I largely put it down to my inexperience with photoscanning.

As for sharing the file, sure! Unfortunately I’m away from my main machine at the moment, I’ll try to upload it as soon as possible.

File: https://file.io/C2QWE5ypUKqq

I have spent more than 5 hours trying to get Gaussian splats to work in blender. Could you please help me, the quality is horrible. I tried following this tutorial https://www.youtube.com/watch?v=yKz7OfomyCo&t=87s but again the quality is trash. Ive been trying to get your Node tree to work but it doesn’t work when I switch away from point cloud.

Hello!
Unfortunately the only recommendations I have is to double-check all the settings: rotation (which looks good from the screen-shot), max transparent bounces, view percentages (preferably as high as your computer can cope). I’m aware that at the moment the setup isn’t perfect, I am considering putting together a completely new addon to simplify the process and make it more accurate at the same time.
Also, I would double-check polycam’s preview screen to see the quality of the training, I’ve found that it can be very touch-and-go which datasets work and which don’t. If polycam’s preview doesn’t look good, try looking into how to photoscan scenes and objects, I must admit I’m still learning myself in that regard.
Hopefully that helps!

1 Like

Hey, Thank you for the quick reply, the quality in Luma Ai is 20x better than what it is in blender. Im am new to the nodes in blender, which is why I was struggling for so long. Would you mind getting in touch with me on instagram? Here is a private note with my handle: https://privnote.com/6x4Jhja6#VEDat03wY Thank you so much!

I love this result! I know this isn’t your intended final result but I can I ask how you got to this NPR look? I’m using your V3 Node group at the minute.

Thank you so much for your hard work.

I simply unplugged the opacity input from the principled BSDF in the shader nodes. Make sure to select the “mesh” material not the points one.
Glad you are finding it useful!

Ah simple! Thank you so much. Finding it very useful! Admittedly its slow (Blender issue not you of course) but the best implementation I’ve seen so far, so very much appreciated.

1 Like

This looks very promising! Unfortunately, as soon as I drag and drop “3dgaussian” node tree from the asset browser to the point cloud (after deleting the original reshot geometry node modifier), it just disappears from the viewport, and opacity slider does nothing. I’m obviously doing something wrong, since many people here commented that it worked for them. I’m using Blender 4.0 and V3pub.blend as an asset. What could I be doing wrong?

UPDATE
Apparently, “Blender 4.0” was the reason it wasn’t working for me. Works on 4.1

1 Like