Smartphone remote

This deserves a lot more views.
I can see the potential in this.
Question: So do I tell blender to record the timeline, and I move my camera and voilá? What about dropping frames? Right now we saw a monkey, but if I switch my 3d viewport to wireframe, the content could be recorded much faster?
Also is there any new update plans for 2020?
This is very impressive @slumber!

1 Like

The addon is not running on blender 2.90.1

Hi @DavidRivera,

The addon is not running on blender 2.90.1

Thanks for the bug report, I will investigate !

So do I tell blender to record the timeline, and I move my camera and voilá?

Actually, the recording is triggered from the smartphone. But it can also be done manually with the blender auto keyframe option :slight_smile:

What about dropping frames?

Since the application is sending frames at the ArCore tracking frequency with TCP (which depends on the smartphone camera frame rate), no frame should be dropped during the data transmission process. The drop may occur when blender apply those data in the scene because the frame is received from a thread. A solution could be to lock the camera frame application.

Right now we saw a monkey, but if I switch my 3d viewport to wireframe, the content could be recorded much faster ?

Scene optimization will surely help to improve recording performance.

Also is there any new update plans for 2020?

About the plan update there is 2 ongoing features:

  • On one side I started to prototype a new version based on a WebXR backend, it seems to be promising and will ease a LOT the development process :slight_smile:
  • I also started to experiment blender viewport streaming to the smartphone, It will to stream the evee viewport to the smartphone application with the tracking. we will be able to get real lighting information into the smartphone \o/

The compatibility issues with blender 2.90.1 shall be fixed,
I updated the zip on the 0.0.3 release page:

2 Likes

All right! What about the APK?

You have to take the same version than the addon so you must install the apk v0.0.3 from the release page.

1 Like

Thank you!! I am so interested in doing this AR core tracking since Jan this year. I will get back once I get it working. Great addon!

1 Like

I installed the addon. I get this error: Traceback (most recent call last):
File “C:\Program Files (x86)\Steam\steamapps\common\Blender\2.90\scripts\modules\addon_utils.py”, line 351, in enable
mod = import(module_name)
ModuleNotFoundError: No module named ‘smartphoneremote-v0’

Also, this activated without a problem:

What should I configure to active it?

I don’t know what I pressed but the timeline on my Blender started to run when I pushed (a button) in the APK interface. Tracking is rock solid.
I cannot figure out how to correctly work with this because the description (and the choice for icon buttons) is not very descriptive.
Please help with a step by step.
My android didn’t trigger anything when I pointed to the QR code in Blender with front camera mode.

Tracking is the most rock solid I’ve ever seen in 20 YEARS of 3d experience matchmoving. Man! This is a dream come true. I would totally love to know how to control Blender camera with the phone. I keep getting a message “cannot stream to camera”.
I hit the Refresh button to setup a new scene but it will not let me.
I set up a tracker (null reference) in the AR scene, and it scales and tracks solidly.
What are the next steps?

Hi @DavidRivera :wave:
About the installation, blender need to be launched as admin during the installation (needed for dependencies installation)
From your screenshot it seems that you have 2 times the blender remote installed :thinking:
It may cause conflicts, if the add-on remove doesn’t work, you can go to %appdata%/Blender Foundation/Blender/2.90/scripts/addons and remove the folder smartphoneremote manually.

1 Like

I will do a step by step today :slight_smile:

1 Like
  1. Make sure that your smartphone and your computer are connected to the same wireless network.
  2. Make sure that you have a least a camera in the scene.
  3. Start blender
  4. Start the remote addon

5.You shoud get a screen like that:

  1. Start the smartphone remote on android

  1. Try to scan the QR code.

  1. (Fail): If the connection state button(Highlighted in green) becomes red, it means that the connection failed. In this case you should try a direct connection by clicking on the connection state button, confirm disconnect.

Find your IP from a command line (ipconfig on windows for instance)
Then click again on the connection state button which should be blue and enter manually your connection address in the format IP:PORT (ex: 154.154.2.32:5555) then hit yes.

If the connection status button becomes green you are good to go to step 8. Else you should try on another network or double check your firewall/antivirus settings.

  1. Once connected the connection state button becomes green, everything is good to start !

  1. If a white gird isn’t already present, follow the instruction at the bottom of the screen to make one appear

.

  1. Once you have the gird, you can choose the scene origin by touching it, feel free to zoom in and zoom out with your fingers to scale it. I will scale and move the blender world reference.

  2. (:warning: To start here you need to have a camera in your blender scene !)
    Start the camera tracking transmission by hitting the camera button.

At this point you should be able to see the blender camera moving :slight_smile:

  1. (recording) To start to record you need to hit the play button at the bottom

  1. (Update the scene on the smartphone) If you made some change on your blender scene, you can update it from the phone by hitting the refresh button

Feel free to ask any questions and to propose any improvements (ux, ui or anything else )

1 Like

Impressive!! Thank you for such details. I am so willing to do a video tutorial, but please help me out with these inquiries: what (or where) does it record to? How does the user have feedback of “how long” can they record?
Also:
If you switch that play (record) button from this (resting state):
not-record

to this when it is active:
record

It will make a lot more sense for the whole planet

1 Like

When you hit the record button, It records key frames in the blender camera action. A progress bar is showing how many frames are available to record.

If the camera is selected you will see key frames appear during the record.

I agree with you about the play record button !
I will try to change it with your icons soon ^^

1 Like

@slumber. Thank you for your reply to the feedback about this addon. This is mint-perfect to work with, the visual communication language is second nature to me, I’ve been a beta tester for a lot of different applications and UI designer as well for more than 20 years.

One little thing that can help the user (without writing more text in the screen) is to use the same camera icon like in Blender. When it is without the connection:
camera off

And once it has connected:
camera-on

Regarding the camera bridge connection, I got more ideas but I don’t want to go and do things without knowing if they are easy changes for you, or if you’d like to implement them. DM if you’d like me to collaborate more for your UI.

I am aware of the coding part, so this is why I am looking for a short-quick to implement things.
Let me know.
Cheers!

1 Like

Oh my! That video!! It’s awesome to see this working! I want to make a walk through video about it.
Also: I develop on Spark AR, and with today’s news about Facebook joining the development fund, plus a twitter from this user give me hope for how deep the integration between Blender and FB could be regarding creation tools.
I see your addon as part of that combination.

1 Like