Smartphone remote

Oh dear I see the connection button is red even though it can read the QR. I used a hotspot from the phone but perhaps there is firewall issues stopping the access to the port? I am using a work PC and they allow limited access

Hi @3pointEdit :wave:

Also I gather that the camera cannot stream until a connection is established, is that right?

Exactly :slight_smile:

Hmmm, the connection button in the lower right corner is red now.

What version of the multiuser apk and addon are you using ?

Do I need to somehow authorize the camera stream?

When the connection status becomes green it means that the remote is ready to stream, then you should be able to push the “camera” button to start the stream

Do both devices, the phone and the PC have to be on wifi?

Both devices must be on the same network

I used a hotspot from the phone but perhaps there is firewall issues stopping the access to the port

You are right about the firewall, it may be in cause :thinking:

Yes I have tried another network at home but still with a managed firewall. Sadly still broken. I’ll try to get permissions sorted. What should I ask for, as I don’t think our security would want to turn off network firewall completly. I got the latest APK from your website today.

On windows 10 at work, is there a specific port that I should ask to open?

Sorry for the answer delay :confused:
on Windows 10 you have to open 3 ports:

  • The service port (configured from the side panel)
    image
  • The service port + 1 (here it would be 61521) (used for data transmission)
  • The service port + 2 (here it would be 61522) (used for TTL)
1 Like

Thank you for those tips. I have the help of our local IT person at work. But it seems that the add-on is failing now. I removed it via preferences but it still thinks it’s installed… so I can’t start it up, also I got a slightly different alert when starting the first time which included a Socket Operation

.

Oh dear it was because the laptop was not connected to a network. So it was operator error :crazy_face: however it might be worth having it fail gracefully to remind the user that the app needs access to a network to run :wink:

Ok I was able to perform further tests in studio. Using my Samsung A70 phone with the android app, I was able to establish connection over a local wifi host to the laptop. But the addon doesn’t recognize if the network is unavailable and will just throw an exception.

I noticed that if you terminate the session while the phone is active the addon will hang with a zmqerror. I gather this is due to the port getting stuck open. I had to reboot the laptop to clear it.

I was able to establish a connection between addon and phone. I could load a cube into the phone and it tracked very well, once the phone app had built some mesh on the sparse cloud. BUT once I started transmitting the phone data back to Blender (which is what I need), it appeared to suffer a gimbal lock. The camera started spinning around the center of the scene. It did begin tracking normally, but soon the wobble got very bad and looked like a feedback loop. Here is an example, I was able to record while using the app on the phone.

Firstly, thanks a lot for all those feedbacks ! :bowing_man:
I hope that we get it working correctly on your setup :crossed_fingers:

however it might be worth having it fail gracefully to remind the user that the app needs access to a network to run

I totally agree with you, I will add that
I should avoid a lot of pain during the add-on setup :+1:

About the glitch you had during the camera tracking transmission, the result in blender should be the same as the one displayed in the smartphone :thinking:
I never experience this but few things might be in cause :

  1. I might be due to the multi-window instance of blender, are you able to reproduce the bug with only one window open ? (I tried on my side, but it worked as expected :thinking: )
  2. It could be a zombie process, If the smartphone remove ended badly, blender might not be close correctly. In this case you can test 2 things:
    1. Go to the process manager and kill every blender instances
    2. If it doesn’t solve the problem, maybe try a computer reboot : /
  3. It could come from the wireless network connection, are you using a 2.4 or 5(better) Ghz wifi ?

Thank you so much for the help. I will try with one window (although that will ruin my use case).

There is only one instance of blender open, although I noticed that those errors suggested existing open traffic to the phone. I did restart a number of times to fix this issue.

The phone shows a completely stable version of the scene but how large a scene could I expect to sync to the phone? So far I have only tested with a cube.

Also is there a way to set the center of the virtual scene in the real world or must it always be where the QR code scans?

Thinking some more about my issue, is it possible that there are multiple data streams coming from The phone app? I recall the operator of the phone performing a long press on the camera icon, could this have initiated more than one send stream?

Turns out that you must run only a single window for blender, which I compensated for. Otherwise the extra floating window fight for dominance.

I was projecting the 3D scene up onto the LCD wall in the studio. I found that if I got to close to the wall the phone would start trying to track the virtual set instead!

I like that I can drag and scale the 3D scene on the phone that is very helpful for positioning the virtual camera.
On the phone is there any way to get a wireframe overlay instead of solid view? As it blocks my view of the real set when I want to calibrate.

Thank you again for this great tool. It seems to have lots of potential for low cost access to VR/AR

1 Like

Sorry for the late answer I had works up to the neck ^^

The phone shows a completely stable version of the scene but how large a scene could I expect to sync to the phone?

Good question :slight_smile:
I never did a stress test but the scene should always be optimized to have good performances on the phone :wink:
If I manage to set up the viewport streaming we will be able to have the viewport rendering directly on the phone :rocket:

On the phone is there any way to get a wireframe overlay instead of solid view?

Not yet but it should not be too hard to add to :thinking:

It’s awesome that you were able to make it work \o/
Did the tracking was stable enough for your needs ?

Yes the tracking is great. But you don’t realize how unsteady your hands are until they are magnified up onto a room sized screen! :smiley:
I have since used some constraints on the camera so that it always points forwards in the scene, this assumes that the IRL camera will mostly point to the center of the LCD wall. Also I constrained the movement to a large plane in the scene (shrinkwrap) that represents the LCD wall. This inhibits wild perspective changes on the LCD wall.

I am having trouble setting a common origin, but that could be because I am not driving Blender during the test. I should have someone else operate the AR/camera rig while I recenter the virtual set around the virtual camera.

Here is the latest test.

1 Like

Awesome ! Thanks a lot for sharing your experiments :bowing_man:
About the origin setup you have two ways to adjust it, but I think that you already tried them

  • Before connecting to blender(scanning the qr code) you should be able to track the ground then touch it, a gizmo should appear, and you shall be able to translate it and scale it from the smartphone (rotations are not ready yet :confused: )
  • After connecting to the session if you create a simple object at the blender origin you should still be able to translate and scale the scene (still on the smartphone) to fit it to the real scene.