Blender on Spatial Computing (WIP)

Hello everyone!

This is my first post here.

Although I haven’t contributed to the official Blender source code yet, I have always wanted to explore what Blender could become on spatial computing devices.

Recently, I started experimenting with customizing the Blender for iPad branch to run natively on Apple Vision Pro. My current goal is not simply to launch Blender, but to explore what a spatial-native Blender workflow could look like.

Recently, I succeeded in achieving seamless real-time editing inside an immersive space. Experiencing Blender in this environment made me realize how much potential it has, so I wanted to share this work in progress with the community.

(Video)

https://youtu.be/49vzoLBswfc


Some thoughts

The following is entirely my personal opinion.

If Blender eventually evolves into a true spatial computing application, I believe it could become much more than a traditional desktop DCC tool.

I’ve been working in the XR industry for almost ten years, and one phrase I’ve heard repeatedly is:

“XR still lacks a killer application.”

Perhaps Blender itself could become one of those applications—not simply by displaying Blender on a headset, but by enabling artists to create directly inside the space they are designing.

For my prototype, I use Blender’s default unit system (meters) as the scale of the immersive environment.

This allows artists to work with objects at their actual size, almost like sketching while looking directly at a real-world subject instead of observing it through a monitor.


UI

As the Blender for iPad project has already pointed out, the current Blender interface is not well suited for Vision Pro.

However, I personally believe that maintaining compatibility with the desktop workflow is just as important as redesigning the interface.

Because projects remain standard .blend files, creators can move seamlessly between desktop Blender and immersive creation without changing their pipeline.

Rather than replacing desktop Blender, I imagine spatial computing becoming another creative workspace within the same ecosystem.


I’ll continue improving this prototype and would love to share more progress as development continues.

I’d really appreciate any thoughts or feedback from the community!

4 Likes

This looks interesting ! however is the way we interact with blender going to change ? currently it’s like operating blender on a giant tablet, with the result behind, where ideally you’ll have only the object in front of you and all operators around…

I’m not sure what form it would take obviously…

Did you try to reach the team working on VR and tablet officially ? maybe you can work together this way you’ll have more chance to have your work included right away …

2 Likes

I sense a lack of purpose here. 3d graphics is a thing. Spacial computing?.. What is that? I haven’t heard of it. I don’t need my space computed. At least I don’t think I do… Do I? What is it for? A toy app on a phone?.. Cool gimmick?.. Doesn’t sound serious. Sounds like a made up term with the sole purpose of it sounding cool :man_shrugging: Same as

That’s very vague. What is it exactly? What is “more” specifically? More in what tangible ways? More of what? :man_shrugging: Who knows?..

I think XR lacks purpose more. Making a killer application sounds like mixed priorities in that case.

3 Likes

Everyone has ideas. Show us what you’re doing with your idea and it becomes something more than yet another cool concept tossed into the void

4 Likes

Yeah I thought this was related to NASA first, but the demo video is pretty self explanatory !
I’m not sure XR or VR is the future of 3D but it’s a great thing to have around, just exploring your scenes with VR makes a whole difference of perception of your work, it’s definitely interesting event though for now Desktop seems the best way to interact with a 3D software …

2 Likes

Yeah, sure. It’s cool. Entertaining maybe. Maybe a good way to get a new perspective in a creative process. I do think that’s true.

It’s still very painful to watch that video. :laughing: in the context where I need to perform thousands of those operations every day and orders of magnitude faster. :laughing: and the user in the video struggles to do them with any meaningful precision, speed, comfort, convenience or even first or second attempt… Why would I want to keep my hands lifted all day long?.. That would be so tiring… :grimacing: Looks like torture. Maybe it’s wrong kind of demonstration. Maybe exploring a cool project and getting that difference of perception of your work would be a better demonstration… Now it provokes very negative emotions to see that. At least for me. :grimacing:

2 Likes

yes I agree that this doesn’t look like something useful currently, but it’s enough to start a conversation I think …
Anyway, if this was working perfectly would there be need for development and UI design ?

When VR came out I was very skeptical about it’s usage as 3D artist, with tablet there is the same kind of vibes, but this is also because we look at it as a replacement of our current interaction tools.

Obviously spending 8h per day on a VR headset sounds un-healthy, however, working on a character and be able to have it in your room at 1:1 scale sounds awesome !
Presenting an architecture project as if you’re there looks great too ! And probably for your work this might help to refine your design as you really experience your idea…

Now to be able to make small edits like ‘taking notes’ or putting placeholder stuff right in that mode sounds already like a good first step and then you can move on to simpler and faster tools to polish that…

Note that I don’t make a lot of difference from VR and XR as they seems similar to me, at least in the way it’s demonstrated, even though I understand that one is completely immersive the other shows the reality … Anyway, it’s deeply connected as tools for one should probably be the same as the other…

1 Like

It seems like an amazing idea… But in reality it’s just not practical. It’s like a gimmick. One time thing to tell friends about, and forget about in a few days. VR/XR is not new and it just never became anything. It’s not even successful as entertainment, which is the most logical use for it… It really seems like an amazing idea to use it in architecture and interior design and seems to makes sense. In practice however it’s possible for a very long time now and nobody is really using it. Maybe as a marketing trick to attract more interest. :man_shrugging: It’s just not practical.

I don’t really have a problem with it. It’s just presented here as “the next big thing :collision: . :laughing: Which it’s not. It’s old at this point and it failed already. If there was interest in it, we would see some popularity already, but it’s a niche thing nobody really cares about. At least that’s what I think. It doesn’t mean that niche things cannot be nice…

Maybe it should be presented more accurately. Something fun to make life more interesting. It’s cool to have a foosball table in the office… We don’t call it rotational kinetic multi-user collaboration platform that’s gonna change everything forever for creative professionals. :laughing:

2 Likes

Computers in Space, dur.

4 Likes

I NEED one then. Sorry. I was wrong before.

3 Likes

Yeah, while I’m as skeptical as you for that to be the next best thing, we can’t blame people for being motivated about it to a point to implement stuff in blender…

Blender feature stereo cameras which was supposed to be the next best thing a while ago and while it’s something I might be using once in a while it’s great to have dedicated tools when it’s needed.

VR/XR stuff is a niche thing but if tools are there and people make something good out of it once in a while then IMO it’s enough to consider it… where to draw the line of something useful once in a while that should be there to something that is just cluttering blender : IDK but to me this falls more in the good to have stuff. Given the design evolve so it’s better that what is presented here …

1 Like

check out oculus medium if you want inspiration. It was a legendary 3d VR (spatial computing) sculpting software. It’s now owned by adobe unfortunately and goes by some other name. I sculpted my logo in it:

2 Likes

A 3D DCC is IMHO one of the few truly viable use cases for XR. People don’t realize how difficult it is to make a 2D user interface for 3D workflow, or the distortions the current solutions introduce. I’ve actually done some research into various advanced curve systems that attempt to correct for these distortions, but they’re fairly difficult to get right and often numerically unstable.

For example, take a guitar string. You can bend it into a variety of pretty shapes quite easily. Good luck replicating that on almost all graphics software today. Humans can learn to draw similar curves in the air in 3D space fairly easily. It’s especially valuable if you have any kind of haptic feedback device; as anyone who has ever done sculpting or woodworking in the real world knows, humans can feel the curvature continuity of real-world objects with their hands and they have this wonderful ability to make that curvature beautiful without knowing all the boring math people like me study.

I actually do agree that XR is 99% useless, but that 1% includes things like 3D DCC software.

I don’t see a problem. I can bend anything into any shape I want in Blender quite easily. Not a very surprising skill here… This is a Blender forum… We do 3d here. Bending things into variety of pretty shapes is sort of very much included in that. :man_shrugging: Not a problem at all. I wouldn’t imagine awkwardly holding my hands in the air and struggling to hit the right spots is going to be of much help with that. I think most people working with 3d would agree that it absolutely cannot compete with mouse/pen and keyboard for the task of bending stuff into a variety of pretty shapes easily. :man_shrugging: Am I wrong? :laughing:

Who here struggles with bending things into variety of pretty shapes? :laughing:

If you really want I can go into precise mathematical detail when I get to a keyboard (on my phone). The guitar string was an analogy. It is true that you will struggle to make the kind of curves you get with guitar strings in Blender, but you can get close in Inkscape (with its spiros) and likely could do it freehand in XR.

Oh, I see. OK. Sorry. I guess you know better. I guess I was wrong then. My mistake. Thanks for telling me as I would have never thought of that otherwise. Now I know. I’ll just have to keep that in mind from now on.

:laughing:

Joking aside, I really have no idea what you are talking about. Maybe show an image or something?..

Even though you are acting in 100% bad faith and I am not near a computer where I can create a proper illustration for you, I have decided to create an example with AI.

Here is an ai generated interactive curve demo, two links in case one doesn’t work

https://codepen.io/joeedh/pen/YPNRNgQ

I seem to be acting in bad faith because what you say is based solely on your lack of knowledge and skills to use the tools. I am not trying to be unfriendly here and I am sorry - I know it sounds harsh. But it is what it is. There is absolutely no trouble to create curves like that in Blender or in any other 3d DCC tools. You can ask about it in the support section and you will probably be given multiple ways how to make stuff like this absolutely effortlessly. This is off-topic. Sorry…

I wrote like half the modelling tools in Blender. I’ve done extensive work on this problem in my career and I even hold a patent from that work. I gave you an example and even a demo.

I shall report you.

2 Likes