Use object as a sculpt/3d cursor controller (leap motion)

Hello ,
I would like to get sculpting working with the leap motion. I can technically already move an object in 3d space. If I try to sculpt with the object, I get a improper context error since the mouse is not always in the 3d viewport. Is there a way around this? Is there a way for an object to be used as a sculpt brush?
Any suggestions would be appreciated.

Haven’t heard much out of Leap in a few years. Seems like they’ve been making good progress and are aiming to get built-in to a future generation of VR and AR headsets, and have gotten a lot more funding to help make this happen. Not sure there’s much support for non-VR apps currently though.

I just ordered one from Amazon which should be here later today and I will see what I can figure out. It would be super cool to have good Blender integration, but it’s not clear whether it’s technically viable or not, and what the state of full cross platform support is.

Can I assume you’re using the old V2 Leap software, or what it your hardware setup?

I think I’m going to be using v3 leap software sdk. I just use what comes out of the box.
It seems like a C++ SDK solution would be more portable cross platform, but there’s also a python setup as well.

I made a data-sender application for sending VR headset data to a blender camera - that used c++ and piped over the data through zeromq and a tcp socket to python. It looks like the python sdk will be easier to implement - it may require pip for blender to install the package though :slight_smile:

I suppose we could work on the project until we get a repeatable error with context then go bother blender developers on #blendercoders, the main website, or the mailing list for a fix.

The latest V4 SDK that has a lot of tracking improvements apparently has eliminated all the official language binding except for the C API, so I’m not sure anyone has built a new Python API yet. They assume people will build thin language-specific API layers on top of C, and they provide C# it sounds like right now. V3 may still be viable, but I’d be inclined to use the latest in order to get the associated improvements. I looked through the V4 API doc and it seems like desktop (non-VR) apps are still supported, though clearly they’re putting ALL their development efforts into VR and AR now.

I’m slightly lazy /slightly not seeing v4 sdk, could you link the C api? I can do the tcp socket stream to python from C, it shouldn’t be too hard - the performance will need some threading from my current solution.

Ah, this orion one? I think it is windows only…I suppose I could build out this one and then port to older versions when I’d like to move to other systems. Thanks for the link

Yeah, they call it Beta and as you say I think it’s Windows only at the moment, but they have plans for MacOS and I think Linux for it, but it has been like a year I think since 4.0.0 came out, so bet on the future at your peril :slight_smile:

I’ll mess around with it on Windows. Some of their examples look similar to what you describe where there’s a thread receiving the tracking updates and storing them in some shared state where the main UI thread picks the data up and uses it.

Got my Leap going. I installed V4/Orion to try it out, and there are a small number of apps that still work with it in desktop mode, though as expected almost everything is VR now and I don’t have a supported VR device right now (so no dissecting cute kitties unfortunately).

I am currently pondering whether I might be able to use Blender’s existing NDOF device support to build an adapter to use it for navigation in the 3d view.

Hey, that would be cool.
I was just thinking about using it to move a camera around, NDOF api I’m unfamiliar with.
Most of them use Unity, I know Unreal Engine has support for the Leap Motion

It would be cool to make something like that :slight_smile:
1 Like

I am trying to find any kind of support for the leap motion device and cant find a single thing that works anymore. All i would like to do with it is use as a rig set up that i can control bones with and am extremely surprised there are no simple add-ons for it like in the other (more expensive) programs. Joe, the man who made the dog of wisdom, used a leap motion to animate and I want to do the same but its like blender was left out and everyone who has attempted to do something with it gave up, very saddening. (joes bts video)

It’s still sort of a background project here, though the Leap is in a box somewhere after moving recently.

One issue is that the Leap people just don’t care about desktop applications it seems, as they have repurposed themselves pretty much exclusively as a head-mounted-display technology with the goal of getting built-in to every HMD.