The Blender AI Assistant

Coding Released Scripts and Themes
,
1

We are releasing an ultimate Blender AI Assistant addon called Vibe4D

It adds a custom UI with AI chat in the left

image
image6144×3336 1.79 MB

The AI has tools to read, see and manipulate scene. You can ask it to do anything, and it will try it best to achieve the result.
(It’s the very first version, so there are some limitations and bugs. Contact us to report any issues, missing features or to give any suggestion, please!)

Some examples of how you can use it:

1. Create a shader

shader

2. Get suggestions for render

AI can give you feedback on your render. Experiment with prompting to get different points of view.

render

3. Get a guide, get help with a problem

Can explain the problem and try to solve it.

chairs

4. Automate a mundane task

organize

5. Experiment!

There’s no limited set of capabilities, you can try different tasks and setups, and see what the AI can do.

fractal

How to try it out for free:

  1. Subscribe to a free tier on Gumroad. You will receive a .zip and your personal license key.
  2. Install Vibe4D as a regular addon (make sure you have Blender 4.4 or similar)
  3. Enter your license key in the new window and hit the button.
  4. Done!

Feel free to suggest features, report bugs and UI issues.

Links:

Discord
Website
Twitter
GitHub

2 Likes
2

Well, this seems fairly useless.

How can i improve this image?

Sure. Let me quote some generic tips that I pulled off the internet, which may or may not have any relation to your image.

These suggestions are often given in many other times people want feedback, so no doubt they apply to your image as well.

Please sort my scene.

Sure. I have no idea how objects and collections work, but I have sorted everything according to the type of object. This may completely break your scene, but it is now truly sorted.

11 Likes
3

What kind of data is uploaded to analyse the scene or for any other purposes?
To where is it uploaded and what exactly happens with the data there?

2 Likes
4

Thanks for your opinion.

Not here to convert you from your AI scepticism (kind of a religious topic nowadays). Though I would like to say, that it’s not “just pulling off generic tips from internet”.
It’s giving you a configurable outside perspective. Can be helpful, especially to new users, who can get feedback on their work only on forums, where they may get just passive aggressive sarcastic comments about their work, which is probably worse than even most generic tips.

Regarding the scene organization.
Can it break your scene? Yes, but magic “Ctrl+Z” can help (if you find bug, when it doesn’t help, please tell us, and we will fix it).
Will it just sort everything by the type of object? By default, probably. But you can specify criterias. Important to note the limitations, that it can’t read you mind, and also can’t see every single mesh. Also we noticed, that it struggles on big scenes, but it depends on how you prompt it. If many people tried it and gave their opinion, maybe the product naturally became better.
Sarcasm aside, maybe you could provide your opinion on how do you imagine “perfect” automatic organization of scene? Or is it even needed? Or maybe you have another ideas of useful automatizations?

Noticed you didn’t write anything about two other examples, like making shaders/geo nodes setups and getting guides. If you care, you could provide your feedback on that too.

Kind regards.

5

It can capture screenshots of viewport, and can access rendered images. Also it can query objects, nodes, render settings and other data in JSON format when requires that data to operate the scene.
Whole scene, meshes, textures are not loaded anywhere.
All chat history is saved on OpenAI side.

1 Like
6

Heh, not here to argue about the usefulness of the addon - you do you… but honestly, I don’t think you can get any more generic than “god rays + AO + surface imperfections”. Especially since the example scene doesn’t seem to even need any of that :person_shrugging:

To be fair, an addon like that could be potentially useful for getting simple instructions off the internet… but only if it provides “source links” for user to investigate further (and in the more-than-likely scenario where it hallucinates, especially since Blender changes all the time).

5 Likes
7

I think such a tool could be very useful. Unfortunately as long as it transfers data to somewhere i can not use it for legal reasons.
There would have to be an ai that can do this running locally.

I think it could be useful for organzing scenes. Of course not by prompting “organize my scene” but in a more granular way. Also probably short python snippets and perhaps some GN and math logic.

5 Likes
8

I agree that rude feedback is something beginners shouldn’t have to deal with. But users should just avoid places that skew towards that (like reddit, for example).

Telling a user “add some god rays, use warm colors, put cracks on the walls” for an image that doesn’t need god rays, is already using warm colors, and is a clean interior bedroom that doesn’t call for chipped paint and cracked walls - how is that helpful?

Noticed you didn’t write anything about two other examples, like making shaders/geo nodes setups and getting guides. If you care, you could provide your feedback on that to

Making shaders: I’ll go with the assumption that the shader tree is actually correct. This is slightly more helpful than a user just downloading a collection of shaders, and picking one they like. I also think the more involved the shader, the more likely the AI is going to fail at doing it correctly. User will wonder why things don’t work, because AI presented it as a correct solution.

As to the rock example: AI is literally just giving a google result. In this case, it’s actually a correct result. You picked a question that had a low likelyhood of failing. But it’s not analysing anything about the context of the question. The answer is also wrong depending on the shaders the user actually has in the scene. It’s easy to make shaders in Blender than you cannot export via FBX, is your AI aware of that? Does it know the difference?

AI is well-known for getting the answers completely wrong, and is generally unaware of version changes in Blender. I’ll not go on more about that, it’s already well-discussed in the AI thread here.

5 Likes
9

Updated examples to use less generic prompts.

Yes, AI sometimes messes up what’s available, how it’s called, etc in that or another version. (For example, default GPT-4 cannot output working geo nodes code for Blender 4+, because can’t figure out node sockets added into API in that version) It’s resolvable by uploading knowledge from full changelogs by version from blender.org and API docs, which we are currectly working on.

10

Thanks for feedback.

Local models require more time to be implemented, because they are simply less smart that the proprietary ones, and need more careful fine tuning. But we will add them eventually.

Anyway, If you ever have the opportunity to try it on a non-private scene or test scene, would be very interesting to hear your thoughts. Not on just the AI answers, but on the UI intergration into Blender too.

1 Like
11

Don’t get me wrong, I’d love an AI assistant in Blender so I can save tons of time and focus more on making art, but all these assistants I’ve tested so far are worthless, and I completely agree with thorn’s first feedback.

I was hooked first when I saw that it can organize the scene, but those LLM’s can only guess what objects are based on their names. If you have a scene with object,object.001,object.002… it will just randomly assign names and put it in collections. Even if you’d have all objects correctly named, it would not know how to categorize collections in a scene only you can interpret.

It’s like Blender when they released hair curves. It looked and worked like magic sure, on a simple plane. Try it on something more complex like suzanne, and it was barely functional.

Not to mention that you have to pay for it, share your data, and be token limited for 50 or 100 objects because the memory is too small. It’s a nice gadget to play with for a few minutes, not to help you in production.

And contrary to what you say, it is indeed generic. It’s not AI, it’s an LLM.

3 Likes
12

Current iterations of LLMs have excelled at helping me create custom addons and scripts - tiny modules with extreme specificity to my workflow, save me a ton of time and headache, all with me knowing barely a lick of python.

image
image298×552 18 KB

However, this effect can only be achieved if you know exactly what you want, understand blender’s data structure, and know to read the ever-changing api docs so you can debug their hallucinations.

For beginners, which I assume to be the main target audience, the chance of them being able to effectively prompt LLMs to do anything beyond what they can already do by googling is probably not very high. Because LLMs’ training data only consists of solved problems already documented online - any task quasi-novel, semi-complex, the LLM is bound to fail.

More likely, beginners will be constantly frustrated by LLMs faulty code, their own lack of understanding of blender and the disillusion that there is no magic (yet). And this is the strongest suit LLMs have currently in terms of what they can bring to blender.

Consulting LLMs for artistic advice is such a faulty premise that I can’t imagine any amount of architectural work one can do on top of api calls, no matter how multi-modal they are, how many trillions of parameters they have, will overcome the fact that they are statistical text prediction machines at the end of the day, based on my own understanding and definition of what art is.

I’m sure I’ll be wrong one day, but unfortunately, not today.

I’m sure you already have inklings of this, if these models truly can change the game, that would have already happened with their web apps. Wrapping them inside blender, while does smooth out some friction, and you can feed them docs and add more blender specific system prompts, can’t actually extend their “intelligence” by much. Plus, there are already a lot of projects trying the same thing. Seeing that none of them made any splash, I’d probably find a more unique direction.

To leave on a more positive note: your presentation, UI, and UX game is very good, especially compared to your competitors. You’re clearly an talented developer. I wish you the best of luck.

5 Likes
14

Very unfortunate and shameful great tools such as the add on you made cannot be accepted with clarity and open mindness. Like seriously it may be due to exhaustion of AI being implemented into older tools or another factor at play, but to be so quick to downplay and dismiss a tool that can be clearly be integrated into your workflow to make things easier is hilarious and ignorant. You do not need AI to make your scene and with the power of free will you do not need to let AI take your spot of creativity (AI wont replace you), but the image analysis to text providing feedback to enhance your scene is a no brainer of great intent mixed with great execution. To the original creator of this great job well done.

1 Like
15

I have decades of experience as a professional designer and 3D artist. Unlike an AI chat bot, which has none.

So tell me: which of my criticisms were ignorant?

5 Likes
16

I have tried to use Vibe4D but was not working properly for me. I discovered blender-mcp and later forked it to create 3d-agent.com

Hope you guys find it usefull!

17

Not impressed so far with AI integration attempts. Although it is going in the right direction.

I am not sure about the answer as to how. But Blender will be replaced by AI in 5 years unless the devs or 3rd party saves it from the same paril that the 3D Animation industry has already suffered.

And I know this is not what most people seem to want to hear.

18

Well, do you have a portfolio of the tool because TBH all I see is models that students with 3 days into blender can do ?
When I put that next to the user feedback comments it definitely look like a scam since I can’t see anyone using this with an actual application besides the fun part of seeing how the AI will react to the prompt.

As for the tool described here, I’m getting the similar vibes. It really revolves around basics problems. I can see some tiny application for learning purpose, or even solving 1 or 2 “once in a lifetime” real production issues. But in any case definitely not something that can be useful on a regular basis.

The metal material is a good example : it looks like something you’ll do after a 1 day of training in material. And it’s far from something actually useful in practice. What if I need stuff like rust, scratches, stains ? How to art-direct that because ok maybe the AI can do something but then I’ll have to dig into it’s work to art-direct it. Or then I’ll have to keep prompting until I got the result I want.

So the time I could have saved in practice it’s just going to be the same but instead of actually doing the thing in blender right away, it’s going to be spending some time hoping the AI will find something I can work with. Best thing in this case is to buy a material pack if the goal is to save time and have some generic material provided…

My point is I don’t see those tools helping you to either get faster results if results there is, or either making you better at CG beside the very basic stuff.

They look cool on paper but in practice it’s not there yet and it’s unclear if that can ever be, but maybe eventually …
That’s why people need to create false user feedback to make people believe someone is actually making an useful use case of those tools…

4 Likes
19

Yeah, the examples are really far from regular every day tasks one might find useful. The 5th example with geometry nodes actually fails miserably and is actually giving the wrong advice since nobody in their right mind would do this with duplicated nodes. It’s just the wrong way to do it :laughing:

2 Likes
20

Appreciate the honest take!

For 3D-Agent specifically, the value isn’t the visual complexity of what it produces – it’s that it builds inside Blender with clean quad topology, proper UVs, and organized hierarchy in minutes. The real use case is rapid prototyping, blocking out, base meshes you can actually work with – not replacing a senior artist’s hero asset.

Our Discord is public if you want to check the feedback yourself… but honestly though, I don’t think you’re the target audience, and that’s fine. Sounds like you’ve been deep in Blender for years, you’ve built your workflows, and you’re probably more capable than the AI is right now. For someone at your level, I get why this doesn’t click.

But things are moving fast. Look at AI video generation 3 years ago… it was horrible, and now look where it is. This space is going to look very different in 2-3 years. We’re building for that trajectory, not just where things are today

1 Like
21

I don’t think you’re the target audience, and that’s fine.

You’re on a server populated by people who have spent years developing their skills as modellers, animators, and artists - and you’re evagelizing tech that aims to replace all of that with automated slop.

Read the room, dude.