Problem With Integrating Keyed 2D Clip In a 3D Environment (Motion Issues)

Tutorials, Tips and Tricks
, , , ,
1

Hello guys,

I’m still not very experienced in Blender and I’m experiencing some difficulties integrating a keyed out in after effects and imported as a plane clip that I would like to integrate into a 3d environment, using a rock model that I imported as well. The idea is to place the girl sitting at the edge of the rock.

I tracked the camera in After Effects using Mocha Pro, then imported the camera into Blender.

Then I started following these tutorials from Unrealab, mostly the first one, as that’s the raw green screen footage I’m using, but I’m also referencing the 2nd one, as it has some more details about the settings:

Tutorial 1

Tutorial 2

I get to the point where I’ve aligned the plane’s location and rotation properties to my imported camera.

The mismatching moment is when they move the plane from Edit mode (to keep the origin point the same to my understanding) along the Z axis as they say, but for me, I have to move the plane on the Y axis to move it in front of the camera.

They only move it along one axis until it perfectly matches the view frame of the camera, whilst in my case, I have to play around along all axises until I manage to make it fit let’s say 97,98%, because it’s moving away from the camera viewport when I move it to the front.

The other confusing moment is when in the tutorial they say that they scale the plane until it reaches the necessary location, and when they scale it, it also moves back along with the scale. But when I scale it, it just scales by staying in one place, it doesn’t move further.

Still, I don’t know if all this is crucial to placing the talent on the rock edge. I just moved it and scaled it to visually seem that the girl is sitting there and it should move along with the camera motion. The big problem is that, as you see, when I hit play, the plane with the talent moves around way too much. It seems that it moves precisely with the imported camera when you look at it, but somehow, there seems to be a desynchronization between the talent and the rocks. It visually looks like as if there are 2 cameras, the clip plane following the movement of the first, and the rocks following the movement of the other. I don’t think this could be fixed with some keyframes of the plane/girl position, the mismatch is just way too much.

Here are my screen recordings from the project so you can see the issue (sorry I couldn’t make it all in 1 video):

Screen Rec 1

Screen Rec 2

Screen Rec 3 - Final Position

So what am I doing wrong? What is the correct workflow to achieve this? Thank you to anyone who decides to help.

2

Hi Nick, have you attempted to do the tracking and keying inside of blender? Or another matchmove software? It would save you having to import the plane, camera and footage from after effects.

3

Hi and thanks for joining the thread.

I haven’t tried doing the tracking and keying in Blender. I’ve done a few motion tracks in Blender until now and have never tried green screen keying here. Since I’m an AE user, I thought for my future workflows, I would do operations like tracking and keying in AE first, then move the results in Blender. For the camera tracking transfer I used the AE2Blend plugin, which seemed to pretty easily create a new camera with the corresponding keyframes of the track.

if nobody has an idea how I can solve the problem using this workflow, I guess I would have to do everything in Blender as a last resort.

4

Hi Nick,

You have to move the plane in the ‘LOCAL’ space: select the plane, press ‘G’ and press ‘Z’ twice to go from the Global to the Local Z axis. This should do the trick (I hope)

5

Hey, thanks for the hint. I was able to move it like that in front of the camera until it fit the frame. However, I still can’t scale it the way they did in the tutorial. But I don’t think that’s what’s causing the main problem anyway.

Today I tried a few more times to track only the portion of the original green screen clip and import the camera and plane from AE to Blender, but again with no luck.

I also tested the motion tracking straight in Blender, then I set up a tracking scene. I went back to the Layout tab to see the results. The camera was strangely positioned and orientated, but it was just above the plane that was created. As for the green screen keying, from what I understand, even if I were to key it in Blender, I would have to export the result ,then import it as a separate file into the scene all the same. So I decided to stick with the keyed clip from AE. This time, even the local Space didn’t help for keeping the plane along the X and Y when I was moving it in Z, so I had to adjust it. For the sake of testing, I turned back the visibility of the rock model and positioned it just below the camera view. just to test things. The result was again the same, the clip plane and the rock model don’t move in synchronization with each other. Here’s the screen rec:

Screen Rec 4