How to track a subject in a video with SAM 3.1

SAM 3.1 returns a mask: a black-and-white video the same length as your clip, white where your subject is and black everywhere else. Nothing about the picture changes. What changes is that the editor now knows which pixels are the subject, so a grade, a blur or a background swap can be aimed at those pixels and nowhere else.

You mark the subject once, on one frame. The model finds it, holds on to it as it moves, turns, is partly hidden and comes back, and writes a mask for every frame. Meta released SAM 3.1 in March 2026, and Morphic added it on November 4, 2026.

Prefer the chat? Ask Copilot, for example: track the cyclist in this clip.

The three ways of pointing at a subject do different jobs, and the good results come from combining them.

Clicks are the most precise. One click inside the subject is usually enough; add an Alt-click on anything the model wrongly swallowed, such as the chair a person is sitting on, and the mask tightens. A box is faster and better for a subject with thin parts, like a bicycle, where a single click can land on background showing through the frame. A description is how you resolve ambiguity: in a two-shot, a click tells the model where, and "the woman on the left" tells it which.

Meta is specific about what the text side understands. Short noun phrases are what it was trained on. Long compositional phrases are not.

When the words run out, go back to clicking. A click says exactly which thing you mean, with no vocabulary involved.

If you came here for the job rather than the model, AI rotoscoping covers the same work from the task side.

The subjects below were each marked once, on the first frame, and tracked forward. A white chef's jacket in a dark kitchen, a pale dress crossing shafts of hard daylight, and a dog running on wet sand are all cases where the subject reads clearly against what is behind it, which is the condition that matters most.

A tracked mask behaves like any other mask on the clip, which is the point of returning one rather than a finished effect.

• Grade inside it. Warm up a face, lift a product off a dim background, or drop the saturation on everything else.
• Invert it and the mask covers everything except the subject, which is the setup for a background change or a blurred backdrop.
• Feather the edge so the cut reads as a soft boundary instead of a sticker, and use expansion to grow or shrink the shape a little.
• Combine it with a shape. Add a rectangle to include something the model missed, subtract one to punch a hole, or intersect to keep only the overlap.
• Stack up to sixteen masks on one clip, one per subject, and treat each one separately.

The mask is lined up against the clip's source frames rather than its place on the timeline, so trimming the clip, splitting it or changing its speed does not call for a new run.

Tracking a shape you drew

If a rectangle or ellipse is already roughly on your subject, you do not have to redraw it every few frames. Open the mask's AI tracking option, say what to track, and the model finds that subject inside the shape and moves the shape along with it. Follow set to Position moves it; Position and Scale also resizes it as the subject comes closer or goes further away. It tracks from the playhead frame and writes ordinary keyframes, so you can adjust any of them afterwards.

Cutting objects out of a still

On the Canvas, Auto-segment splits an image into separate layers, and Separate as layer lifts out the one object you point at. Same model, same idea: a mask you can act on rather than a flattened picture.

• Mark the subject where it first appears if you are running forward. Frames before the one you marked stay empty until you run backward too.
• Clicks or a box on a video, not both, and at most sixteen clicks.
• Both directions is two runs and costs twice as much as one. Use it when the subject is clearest somewhere in the middle of the clip.
• The playhead has to be on the clip, and the mask has to be over the picture, or tracking will not start.
• Meta notes the model is weaker on very fine-grained or unusual concepts than on everyday ones. If a description keeps missing, a click will not.
• Check the mask over the whole clip, not just your marked frame. Motion blur, heavy occlusion and a subject that leaves and re-enters are where edges tend to soften.
• A subject that reads clearly against its background is what holds. In our tests a dancer in a translucent skirt against a near-black studio was lost after the first frame, twice, from a box and from clicks, while a yellow coat in grey fog held across every frame.
• Expect the mask to answer the question you asked. Tracking the cyclist in a night street gave us the rider and not the bicycle, which is correct and may not be what you wanted. Add a box or a second click to take in the rest.