You probably have seen some behind the scenes reveals where there are crew and talent alike huddled around huge green screens that will eventually be turned into some amazing visual effect. Have you paid attention to the tracking markers that are on that high-budget shot though?
Matthew Merkovich has.
Matthew is a highly skilled and seasoned 3D tracking artist who has worked on a ton of feature films and top-tier television. From tracking shots for such films as Minority Report, Equilibrium, Final destination 2, to TV shows such as Star Trek Voyager, Terra Nova, and Grey’s Anatomy. In between, are almost endless commercials and ad work — all with rock-solid tracking.
tracking software has advanced considerably, setting up for tracking shot hasn’t
Tracking software has really come a long way in a short amount of time. There have been huge advancements in technology, that is attainable to pretty much anyone. As an example, SynthEyes is an amazing software for camera, object, geometry and planar tracking — and it is completely affordable.
The thing is, with the fast-paced advancements in tracking, people are still laying out markers as they did on day 1. If you have ever seen a green screen littered with orange, pink or dark green tape in a variety of shapes and sizes, this is where the discussion is centered.
Matthew’s campaign’s message is that that elaborate tracking makers are not needed any longer. I spoke with Matthew in the hopes of clarifying, and spreading the word for his mission to tone down tracking markers on set.
Where did the idea of using orange and different marker types such as crosses and patterns originate?
That is a fantastic question, and not just because I ask it all the time. I once got pulled onto a movie by the producers to help fix a VFX budget gone out of control and I asked the owner of the VFX company contracted to do the work why there were these huge hot pink Xs all over the blue screen footage she supervised. She rolled her eyes at me and said, “That is an industry standard best practice and I’ve been doing this with [name of renowned VFX supervisor redacted] for years!” This was a movie with a 20 million dollar budget and Academy Award winning stars.
When I ask younger people why they are placing these big plus signs on their low budget pictures, I hear, “Isn’t that what you are supposed to do? Our VFX guy said that’s what we were supposed to do.” My conclusion is that they have no good reason for these big +s or Xs.
I’ve researched this historically as well. I cannot find the big-orange-X-on-green-screen patient zero. Having personally done VFX since my first credited shot on the TV show Unsolved Mysteries way back when, I’ve seen solutions to specific problems turn into generic solutions many times. I can imagine somebody in the 1990s came up with the big garish colored X to solve a specific tracking problem and it just stuck. 3D tracking software didn’t even exist back then, but that’s when I began seeing this phenomenon, so I could conclude this was to solve a 2D tracking problem.
Later, in the early to mid 2000s, I began seeing shots with black circles with white triangles and all manner of crazy geometric shapes. Just googling “tracking marker” gave me this as the first search result…
in 2004, I was at Asylum VFX and had many discussions about these weird new patterned markers with a coworker, math Phd Dr. Mathew Lamb. He said these features best “stimulated” the Boujou auto tracking algorithm. In 2016, placing markers like this in generic footage is no longer really a thing and I’m personally glad to see that Boujou has mostly gone the way of the dodo, though some people still use it. I can only guess that is a sunk costs issue. I still occasionally see these kinds of markers though. I’ve never needed them, ever.
People have no idea that tracking a knuckle on a c-stand is just as easy as tracking the tennis ball they cut a hole in to mount on that very same c-stand.
I guess that is what my one man campaign is all about.
You are also quite vocal about taking camera data while on set. Why do you think this practice has become irrelevant?
I just got lens notes on a show I am working on now. Everything came from a RED Dragon shooting a 16mm Primo V lens. When I tracked the three shots, the calculated lenses were 16.534, 16.536, and 16.321. I get results this accurate every single time, with any lens you can throw at me. These are calculated lenses! I did not give my tracking software, SynthEyes, any clue from the camera notes.
I’m working on some new tutorials about this right now, going into the subjects of lens breathe and camera and lens manufacturing error tolerances to explain why the focal length printed on the lens barrel is merely a very close guideline. There is also the factor of human error. Is that camera report accurate? Did the camera operator bump in on that zoom ring and not tell anyone? How do you even take a focal length note on a 70-200mm lens?
Bottom line, my calculated lenses are going to be more physically accurate than forcing a potentially wrong lens into a math problem.
What are some of the best practices for preparing to track green screen shots?
I was overseeing nearly all the VFX on a TV show a while back and I did a slapdash guide for production on how they should place tracking markers. The video was really only intended for them, but I am revising it to be more thorough. Until I finish that video, here it is:
And here are my notes from that page with a few small revisions:
As you can see, there is very little reason to add tracking markers most of the time. But I thought I should add a few important points for when you do need them.
- Always DOTS.
- Never X’s made with colored tape, or checkerboard patterns, or triangles, etc. Just circular dots.
- Preferably small white dots on a dark background. If your track is in a white environment or a white object, black dots will work, but will disappear during whip pans. (I don’t make the rules of light physics; I just have to live by them.)
- The dots should be just big enough to be visible. If you are shooting a lot of coverage, close medium and wide, let the wide shot be the guide for how small they should be. Make them just barely visible in the wide. I really don’t overthink this. I rip off half inch squares from a tape roll and call it a day.
- You typically need to see at least nine dots on screen at any point to get a good track/match- move. The dots can leave and re-enter frame during camera moves, but you still need to see at least nine dots at any given time. That just means you need more dots off camera that come on when you pan and tilt. Nine is a bare minimum. On some tracks, the software will detect scores or hundreds of features to lock onto. The more trackers, the more accurate the 3D track.
- And move the camera, if that’s what you want! “We only moved the camera a little to make it easier to track.” All you succeeded in doing is making your shot more boring. >Obligatory winky face goes here.< Big moves or little moves are no different in difficulty. However, fast moves with lots of motion blur are harder.
- Crazy shapes are useful when shooting crazy, whip panning, hand held shots. The markers are going to become big motion blur streaks so having many crazy shaped patters helps disambiguate the tracking markers…I suppose. Frankly, I would still prefer plain old dots on a green screen.
- Circular dots. CIRCULAR DOTS! CIRCULAR DOTS!!!
- And if you need to throw up a lot of tracking markers fast, for example on a green or blue screen, get a cheap laser with diffraction grating. Blue lasers are great for blue screen. Likewise, green lasers for green screen. Too bright? Unscrew the cap and jam some ND filter in there. Boom! A field of tiny dots to track. They should be just bright enough to be visible and you won’t need to spend a lot of time doing roto or paint to clean up your edges. Of course the lasers won’t work if there is any atmo on set. Smoke machine? Forget I said anything! Break out your digi-green tape roll!
I’ll add that the reason you shoot green screen is to pull keys. Obvious, right? No one seems to remember this though when placing tracking markers. They place these big orange Xs and then a blond actor’s hair crosses that marker and your two hour comp turns into days of hair reconstruction, roto, and headaches.
What do you recommend someone do when they see someone going overboard with markers?
Yell at them! 🙂