I’ve just tried deepseek and gemini but they both repeatedly churn out errors. They make the same errors in a circle, correct one error and it makes one it made before etc…
I’ve never found either of those models to work well for anything. I’ve had fairly good experience with ChatGPT, so long as you know just a little bit of Python and can help guide it a bit. I’ve also tried Claude which seems to do pretty well.
2 Likes
The best thing you can do is learn python, because these things will always have errors and hallucinations.
4 Likes
There is a qwen ai model dedicated for programming. but yes you need to know the blender api and python to understand the code and know what you need to fix manually or tell the AI to fix correctly with proper programming context that you give it.
no. I use copilot sometimes… they all suck.
Ahh, my ongoing quest for an AI that can write good Blender Python code.
Don’t know about the full blown commercial ones, since my primary interest has been in something I can run locally.
With that in mind, I’ve tried a few and so far, Qwen2.5-Coder-14B-Instruct-Q5_K_S that I’m using now has performed about the best.
Claude is kinda good in code related tasks. GPT-o3 Mini worth a try. Learning Python (the best :D)
10 Likes
Full disclosure - I work on LLM tech and services in Microsoft. I’ve been using copilot extensively to write code initially for scripts, but they will turn in to an add on in the near future. In my experience, the question is really - can we be productive with these tools? They will certainly make errors (especially in terms of not really understanding the version of SDKs and APIs). But if you understand how to work with them you can get great results.
My general cycle is to describe the task, observe the output, break it down myself and then iterate on the modules. I then also run the code and report back to the LLM any errors or incorrect behaviour.
Most of the code I’ve been writing like this is for mesh generation, UV unwrapping and add on UI creation.
When Microsoft’s copilot gets mixed up, I will then go to ChatGPT for an alternative. I’ve also tried a few queries in to DeepSeek but don’t yet have an opinion.
3 Likes
this sounds less like advice and more like an LLM-ai advert, you’re not a double agent are you?
1 Like
Not necessarily. AI is advancing quickly, and today’s models aren’t the peak of what AI will achieve. They’re just the best we have right now. But companies like OpenAI, xAI, Anthropic, and others are already developing even more advanced reasoning models that’ll likely launch in the near future
1 Like
Which is one of the biggest problems with pretty much every LLM, especially when it comes to Blender. The training data is in basically every case, out of date compared to the Blender Python API.
If one is lucky, you get around mid 3.X release, while nothing seems to know about 4.X
This is why I’d like the Blender Foundation to train and release an open source LLM specific to Blender. It wouldn’t even need to be 0-day release accurate, just the yearly LTS API, etc would be enough.
Things is I feel that’s only half the issue, the other half is the training data and likely some/much of it is rubbish, with mixed up and/or contradicting ‘facts’, so the AI uses that to calculate errors with what it ‘thinks’ is absolute accuracy.
1 Like
And the sprinkles on the cupcake is that the chatbot doesn’t tell you this. It fakes the answer, then presents it as valid.
It’s like hiring a Russian translator for your office, without knowing that this person doesn’t actually speak Russian. They just make a lot of weird vowel sounds, and you assume they’re correctly translating your words to the other listener.
6 Likes
I actually have a little text file with some initial tests and questions for any local LLM that I try. My first two are:
What was the last release of Blenders Python API that you were trained on?
And
What is the most recent release of the Blender API that you are aware of?
Now you would think those should give the same answer, especially for any LLM that doesn’t have or do internet access, which is all of them that I test locally.
However, no, a lot of the time one gets a difference answer. Then, on top of that, much of the time, at least one of those will have an answer that includes the release date of said Blender version that it ‘knows’. That date is ALWAYS wrong. At best by a few months, at worse, by a year or more.
Then I have my first coding question, what should be a pretty simple one.
Can you write the python code for Blender that adds a sphere to the current scene, sets it to smooth shading and adds a subdivision modifier at 2 levels.
Turns out not many get that right first go, but after feeding the error back in, can usually produce something that works.
Tried that on my local Deepseek model, got it wrong and then after feeding the error back, still got it wrong.
On the other hand, my local Qwen Coder, gets it right first time.
Regarding the correctness of the output, and the ignorance of the model with respect to the correctness - the nice thing about generating code (rather than, say, doing your taxes) is that you have a simple way to test if it is correct as well as a report (error message) as to the problem.
Regarding the idea that models won’t always produce errors - I don’t think this is really true. All software (every single program you are currently interacting with) has errors (bugs) in it. Software engineering is a process that includes maintenance (it doesn’t end when the code ships). These models have trained on examples of that code.
That being said, I do think it will be possible (even now with fine tuning) to create a model that will be able to produce working code on the first shot. In fact, I would identify this as a key milestone: does the code run. Currently, much of it doesn’t (which is why one needs to report errors and iterate).
I have the opposite problem. it gives me code that’s too new
@ontogenic: The fatal(!) flaw with all of these “AI models” is that “the programmer” never actually knows, nor can (s)he know, the full description of the problem to be solved. Furthermore, as business conditions change and the system must (quickly!) evolve to change with them, these parameters cannot be “algorithmically described.”
Doing so would pre-suppose that you already know “the” answer. Which you don’t. And, there is always more than one possible answer, which must be weighed. (But: the relevant factors cannot be “quantified,” because doing so would pre-suppose that you already know “the” answer. Or, am I repeating repeating myself?)
To borrow an entirely-unrelated example: “If there were a (!) ‘better mousetrap,’ then we surely would have already calculated an algorithm to determine it.” And, all of the mice in our houses would already be dead.
I don’t think you’re right. I ask ai to do things all the time that I already know the answer too and they still get them wrong. like simply not using the @ operator for a simple matrix multiplication then it goes ahead and uses it anyway.
100% this. LLMs don’t even have the concept of “confidence” right now, which they desperately need. Deep Research is an interesting concept where it uses an agent to go out and collect information from the web and then summarize but the web is getting filled with LLM AI trash now so good luck with that. Garbage in, garbage out all the way down.
4 Likes
I personally think that we all must remind ourselves of: “GIGO.” (Garbage In = Garbage Out)
Those of us who are old enough to remember 
… remember this very same thing before: it was called, “Y2K.” And, for us practioners at the time, it was: “a royal pain in the ass!”
At the time, “familiar computer technology concepts,” which all of us [practitioners …] “fully understood,” had suddenly been "stock-market (!)-ized!"
(This was the time in which your only business objective was to: “go public.”)
I was savvy enough to grab many “oh my god the sky is falling!” books off the shelves at that time, and since then I have profitably sold all-but-one of them.
And, yes: exactly the same “gambit” is now being played here. Here the story is not that "your world is ending," but that it will "magically be replaced."
Do not be displaced by these fictions. Just hold on and wait it out . . . “Speculators” have a very short institutional memory, after which it will all be gone.
3 Likes