Leap frames into Mac Blender via Client/Server

Yes it has been done already. I downloaded the files and set them up in Mac Blender and it works! Of course it’s not going to be as fast as using Leap Motion files directly in Blender via the Leap.py and LeapPython.so and so on.

[ATTACH=CONFIG]253405[/ATTACH]

Anyway, here is the website to get the files:

In this page we present the software we developed to edit 3D scenes and characters though the use of hand tracking devices.

developed a Blender addon to enable the use of a Leap Motion to edit position and orientation of objects in space using hands movement.

Download

The addon can be downloaded here: http://www.dfki.de/~fanu01/leap_modal_controller/LeapModalController-v1_0.py

For a quick test, a Blender test scene can be found here: http://www.dfki.de/~fanu01/leap_modal_controller/LeapModalController-v1_0.blend. In the scene you will find the script (click on the “Run Script” button to initialise it) as well as a character exported from MakeHuman and a Monkey head to play around. Created and tested on Mac with Blender version 2.66.1 (or 2.66a).

Installation

Prerequisite. You need the “websocket-client” module (Home page https://pypi.python.org/pypi/websocket-client/). Tested with version 0.11.0.
Download directly the version for python 3 HERE: https://github.com/liris/websocket-client/blob/py3/websocket.py.
You must “install” it in the Python 3 distribution embedded in Blender.
On Mac :
Right click on the “Blender” app icon;
Select “Show Package Contents”;
Follow the path “Contents/MacOS/2.66/python/lib/python3.3/”;
On WIndows:
Explore the path similar to “C:\Program Files\Blender Foundation\Blender\2.66\python\lib\”
Copy the file “websocket.py” into this last “python3.3″ or “lib” folder.
Restart Blender, if it was already open.
Install the LeapModalController-vX_Y.py file using the standard Blender addon installation procedure (File -> User Preferences -> Addons -> “Install from File…”). Or copy paste the script text into a script buffer and execute it.
Usage

Once initialized in Blender, the script will override the behavior of keys G, R, T, F and H. F and H and functions tailored for MakeHuman characters.

G – Select an object or a PoseBone, press G to “grab”, and use your fingertip to move the object in the scene. Press G again to confirm or ESC to cancel.

R – Select an object or a PoseBone, press R to “rotate”, and use your hand palm to rotate the object on place. Press R again to confirm or ESC to cancel.

T – Select an object or a PoseBone, press T to “translate and rotate” at the same time, and use your hand palm to translate and rotate the object in the scene. Press T again to confirm or ESC to cancel.

F – Select a MakeHuman hand IK controller, press F, and perform “circles” in space to adjust the position of the elbow, which will rotate around the line between the shoulder and the wrist. Press F again to confirm or ESC to cancel.

H – Select a MakeHuman armature, press H to move both “hands” of the character as you would move them in first person perspective. Press H again to confirm or ESC to cancel. Works best with a camera positioned in a first person view. If auto-recording is enabled, hands controllers location and rotation is written in the timeline.

shift + H – Same as H, but in “mirror” mode. Character hands will move as you would see in a mirror. Works best with a camera framing the character frontally.

Known Issues

Using Leap SDK 0.8 we experienced problems in connecting directly to the Leap daemon through web sockets. Repeatedly connecting and disconnecting form the device cause the Leap software to hang.

To overcome this problem I developed a little python script that creates a long-term websocket connection to the Leap daemon process and forwards information on localhost through UDP packets.

The script can be downloaded here: http://www.dfki.de/~fanu01/leap_modal_controller/LeapForwarder.zip. It has been tested on Mac and python 2.7. Should run on windows flawlessly.

The Leap Modal Controller script can be configured to listen for UDP packets (instead of calling the Leap daemon) by setting to True the USE_UDP_SOCKET boolean flag at the beginning of the script:

USE_UDP_SOCKET = True

This problem should be solved from SDK 0.8.1, but we have not tested it yet.

Credits
Leap Modal Controller has been developed by Fabrizio Nunnari, in collaboration with Alexis Heloir for the Sign Language Synthesis and Interaction research group at DFKI / MMCI, Saarbrücken, Germany.

Share this:

Kind of strange to use sockets in an addon when they could just add an exception for leap to their gpl header (or change their license to something like BSD) and call functions directly out of the leap py-module but, hey, whatever works I suppose.

I guess this means I have to ‘waste’ my day off messing around with this silly leap now…

I have no trouble using Leap software inside Blender (Mac) but the C++ dynamic module doesn’t work (LeapPython.so). I have not been able to compile it so it works from Blender. Sometimes I think it does work, but cannot get the right setup with the other Leap files or they are the wrong ones that doesn’t match up. Information is not being passed around correctly. I get errors that a item cannot be found.