Camera tracking down a long path going fast?

I am creating a stop motion project and I want to add CG elements in the background of my scene. The scene is a robot riding a hoverboard over a rocky planet. The problem is that he is traveling at scale 40mph. The trick is that he is not truly moving, only the planet tiles are moving underneath him. I have added colored tracking markers to the surface however because of the speed he is flying the markers are only on camera for three to 4 frames at the most.There are three planet tiles in each frame and each tile has 4 colored markers. In the first frame there is at least 8 but in the second frame a few are already gone and are replaced by more on the opposite end. Most tutorials I have scene uses a camera that stays in close to the initial position. The background is a green screen and the planet is mostly flat. I could manually create the camera for part of the sequence however my camera is on a dolly/jib crane and I will have some rotation in other parts of the project. I do not think auto detect features will help due the planet surface is made from foam and painted cat litter.
My initial thought is that each planet tile must contain 8 markers then when the go out of frame delete the track data from that point.
Hints, and Suggestions welcome.

I assume the background is moving at constant speed? Why not just try to match the animation speed (maybe even by doing some computataion cm/seconds) instead of trying some fancy motion tracking ??

Thanks for the suggestion and that is my fall back plan. The entire sequence is 20 seconds long and the camera is on a dolly/crane so it will move from behind the character to the side to a 3/4 view all while the ground is moving and then slowing when he reaches his destination. The camera while it will rotate in X and Y it will not be moving up and down so I have that as my advantage if I need to create a camera move manually.

Ah sorry i was also assuming just a simple moving to the right (ground… so hovering to the left)… Maybe it helps to have simpler markers?? Maybe simple round white pinheads on black background? You also must have only 8 (or so) markers in total in one frame so also maybe you don’t need 8 marker per tile… :thinking: or you may also make your stop motion with double/trible frame rate for the tiles but normal for yyou hand made animation so you get more frames to camera track and additionally just deleting ever second (or additional third) image… → disadvantage: more RAM needed for a smooth tracking… or just wating a litle longer…

If you’re willing to use CG elements, and the character is going to be moving along quite fast, why not just green screen the whole thing - including the floor! It will be so blurry that your beautifully crafted scene will barely be visible anyway. So if the whole scene were CG, it would be much easier to figure out the speed.
But really interested to see the finished project!

Thanks for your suggestions OkiDoki and Peter. I have decided to just use a digital model of the set and animate the digital camera by hand. I have all the measurements I need and the hoverboard and robot were both made in Blender so I will have them as a digital model. The plan is to line up the camera rotation with the digital hoverboard similar to what Okidoki mentioned earlier.
I considered that option big time. I am a very novice animator and the thought of animating a camera, a planet, a robot and flying meteors in each frame sets me up for failure and filming them separately then combining in post was considered.
Animating the digital camera may take some time but it has to be quicker then painting out several thousand track markers.

The finished project is for a “Jam” over at the website stopmotionanimation and each person will submit a 20 second film based on the subject science fiction. The website is run by Anthony Scott (professional animator, animation director) and this will be the fourth jam. Rules are no toys, no graphic violence, no nudity and he lists some other rules and format requirements. This is a hand made film so no computer characters. I use Blender for all my modeling, 3D printing, and compositing. This will be my first VFX attempt as I am looking to add smoke for the hoverboard and set extensions to the planet (may end up being just a sky replacement and not a modeled set). I have been testing camera tracking and thought I would throw it out here in case i could do something to help my job go easier. Did I mention there is no money, no prize and very few people see the finished projects? Finished films are due August 21, 2022 and Scott will post them after he has edited them all together.
Here is my movie set with my green screens and the camera crane. The film lighting is not on yet. More green screen than I need but its so cheap might as well hang it up. The camera crane is a mix of a dolly tutorial I found online and mini jib crane files that I bought and printed.