Rotoscoping is the job nobody in visual effects wants. It's the frame-by-frame tracing of an actor's silhouette out of a background plate, done by hand, for hours, so that a compositor can later drop in a new sky or remove a boom mic shadow. It is also the single task where machine learning has made the deepest, least controversial inroads into film production. Not because studios decided AI should replace artists, but because roto is exactly the kind of repetitive, well-defined, low-creative-judgment task that neural networks are good at accelerating.
That distinction — between AI doing the grinding parts of a shot and AI generating the shot — is the one most conversations about "AI in Hollywood" collapse. This post is about untangling it: what machine learning actually does inside a real VFX and post-production pipeline today, how it got there, and where the line currently sits between automation and generation.
What "AI in the pipeline" actually means
A VFX pipeline is a sequence of discrete tasks that turn raw footage into a finished shot. Roughly, in order:
- Ingest and organization — sorting footage, syncing timecode, building proxies.
- Tracking — determining camera motion and object motion in 3D space so digital elements can be locked to the plate.
- Rotoscoping and matting — isolating subjects from backgrounds, frame by frame.
- Cleanup and paint — removing rigs, wires, markers, unwanted objects.
- Compositing — layering CG, plates, and effects into a final image.
- Color and finishing — grading, noise management, format conversion.
- Editorial and dialogue tools — cutting, ADR, de-aging, dubbing, visual continuity.
Machine learning has been adopted unevenly across this list. It's heavily embedded in steps 2 through 4 and 6 through 7, where the task is pattern recognition or interpolation on existing pixels. It's far more contested in step 5 and in any task that involves generating new imagery rather than manipulating captured imagery — because that's where questions about authorship, likeness, and job displacement get sharp.
The two categories that matter
It helps to split "AI in VFX" into two buckets that get conflated in press coverage:
- Assistive ML — models trained to do a narrow, previously manual task faster: segment a person from a background, track a point across frames, fill in a hole left by wire removal, denoise a plate. The artist still directs the shot and fixes the output.
- Generative AI — models that synthesize new pixels, faces, voices, or video from a prompt or a small reference set: full de-aging synthesis, voice cloning for dubbing, text-to-video for pre-vis or even final pixels.
Most of what's shipped in production pipelines to date falls in the first bucket. The second bucket gets far more attention and generates far more anxiety, but it's used more selectively — often for pre-visualization, temp comps, or narrow, heavily supervised tasks like de-aging a single actor in a handful of scenes.
Where ML actually earns its keep today
Rotoscoping and matting
Traditional roto is manual spline-drawing around a subject on every frame, tightened frame-over-frame for motion blur and hair detail. ML-assisted roto tools use segmentation models (many derived from research like Segment Anything and video object segmentation networks) to propagate a rough mask across dozens or hundreds of frames from a handful of manually corrected keyframes. The artist still cleans up edges — hair, motion blur, semi-transparent objects remain weak points — but the bulk propagation work that used to eat days now takes hours.
This is the least controversial AI use case in the industry because it doesn't change what gets shown on screen. It changes how long it takes an artist to get to the same result.
Match-moving and camera tracking
Determining where a camera was in 3D space during a shot, so CG elements can be locked to it, has traditionally relied on tracking manually placed markers or naturally occurring high-contrast features. ML-based trackers are better at holding onto features through occlusion, lighting changes, and motion blur than classical optical-flow trackers, which reduces the amount of manual re-tracking needed when a shot goes wrong.
Cleanup: wire removal, rig removal, object removal
Removing a stunt wire, a green rig, or a modern building from a period piece is inpainting: filling a masked region with plausible pixels drawn from surrounding frames. Generative inpainting models — trained specifically for temporal consistency across video, not just single images — are now standard in cleanup workflows. The gain here is speed and consistency: a human painting frame-by-frame can introduce flicker; a model conditioned across a frame window tends to hold texture and lighting steadier.
Upscaling, restoration, and format conversion
Frame rate conversion, resolution upscaling for re-releases, and film restoration (descratching, degrain, colorization of archival footage) are pattern-completion problems ML handles well, and this is one of the oldest production uses of neural nets in post — well predating the current generative AI wave.
De-aging and face work
De-aging has moved from purely geometric/textural CG work (rebuilding a younger face in 3D) toward ML-assisted approaches that use trained models on an actor's own younger-era footage to guide texture and shape changes, often layered under traditional compositing rather than replacing it outright. This remains one of the most labor-intensive and heavily supervised uses of ML in the pipeline — a far cry from a one-click filter.
Dialogue and audio: ADR, dubbing, voice matching
Automated dialogue replacement and multilingual dubbing increasingly use voice synthesis to match an actor's own voice across looped lines or foreign-language versions, plus visual dubbing tools that adjust mouth shapes to match translated audio. This is one of the fastest-growing applications precisely because streaming platforms need same-title releases in dozens of languages simultaneously, and manual ADR/dubbing at that scale is a real bottleneck.
Why this matters now
Netflix disclosed that generative AI workflows touched roughly 300 titles in 2026, and the company was specific about where: mostly rotoscoping, cleanup, and tracking — the assistive-ML bucket, not full shot generation. That disclosure matters less for the number itself and more for what it confirms about where the industry actually is: adoption is concentrated in the unglamorous, high-labor, low-creative-judgment tasks, at a scale that shows this is now routine infrastructure rather than an experiment on a handful of prestige productions.
That's a meaningfully different story than the one that dominated public discussion after the 2023 SAG-AFTRA and WGA strikes, where the central fear was generative AI replacing performers and writers outright. Three years on, the actual deployment pattern looks more like traditional software automation applied to a craft that has always adopted new tools — non-linear editing, digital compositing, motion capture — when they demonstrably save time without degrading the final image. The difference this time is that the underlying models are also capable of the more disruptive use case (full synthesis), which is why labor agreements now explicitly carve out consent and compensation terms for AI use in ways earlier tooling transitions never required.
It's also worth noting what the Netflix figure implies about scale versus visibility. Three hundred titles touched by generative-AI workflows in a single year, concentrated in roto, cleanup, and tracking, means this is no longer a handful of experimental shots on a flagship production — it's routine tooling applied across a large slice of an active content pipeline. Yet almost none of that usage is visible to an audience watching the finished product, because the whole point of assistive tooling is that the final image looks the same as it would have looked with a slower, manual process. That gap between adoption scale and audience visibility is a big part of why public perception of "AI in Hollywood" still lags well behind actual pipeline practice.
For studios and vendors, a disclosure at that scale also signals something practical: the tooling has crossed from pilot projects into standard pipeline steps with established QC processes, which is a prerequisite for any vendor selling into that pipeline to take seriously.
Practical implications for studios, vendors, and builders
For production and post houses
- Budget planning shifts, not disappears. Roto and cleanup line items shrink per-shot, but the freed capacity typically gets redirected to more shots per schedule or more iteration rounds, not fewer artists — at least at current adoption levels. Whether that holds as tools mature is an open question, not a settled one.
- QC becomes the bottleneck. When a model propagates a mask or fills a hole across 200 frames, an artist still needs to review every frame for flicker, edge bleed, and temporal drift. Tool vendors that under-invest in review/diff interfaces create hidden labor costs that erase the speed gain.
- Data and rights matter before the tool does. Any studio using de-aging or voice-matching tools trained partly on an actor's own footage needs a contractual basis for that use, independent of how good the model is. This is now a pre-production legal step, not a post-production technical afterthought.
For streaming platforms specifically
Streamers face a distribution problem traditional theatrical studios historically didn't: a single title often needs to ship in dozens of languages and formats simultaneously, on a release calendar set by subscriber engagement rather than a single global premiere date. That pressure is a large part of why dubbing and voice-matching tools have moved faster into production use than, say, generative shot creation — the business case is about localization throughput, not spectacle. The same logic applies to cleanup and roto at catalog scale: a platform commissioning hundreds of original titles a year has a much stronger incentive to standardize AI-assisted pipeline steps across vendors than a single studio releasing a handful of tentpole films.
For tool builders and vendors
| Pipeline stage | ML maturity | Where the value is | Where humans still lead |
|---|---|---|---|
| Rotoscoping | High | Mask propagation across frames | Hair, motion blur, transparency edges |
| Tracking | High | Feature persistence through occlusion | Recovering from total tracking failure |
| Cleanup/paint | High | Temporal-consistent inpainting | Complex reflections, unusual materials |
| Upscaling/restoration | High | Resolution and frame-rate conversion | Judgment calls on stylistic intent |
| De-aging/face work | Medium | Guiding texture under CG structure | Full shot, performance-preserving results |
| Dubbing/voice | Medium-high | Voice matching, lip-sync adjustment | Emotional nuance across languages |
| Full shot generation | Low-medium | Pre-vis, temp comps, concept work | Final-pixel delivery at feature quality |
- Build for the review workflow, not just the inference. The highest-value products in this space aren't the models themselves — they're the diffing, flagging, and correction interfaces that let a human supervise thousands of AI-touched frames efficiently.
- Temporal consistency is the hard problem. Single-frame image models are commoditized; the differentiator in video VFX tooling is holding texture, color, and geometry steady across a frame sequence without flicker.
- Interoperability with existing pipelines beats standalone tools. Post houses run on entrenched software (Nuke, Flame, Resolve, Maya). Tools that plug into those environments as a node or plugin get adopted; standalone web apps mostly don't, because they break the version-controlled, render-farm-integrated workflow the rest of the pipeline depends on.
Limitations and open questions
- Generative shot creation still isn't reliable at feature-film finishing quality. Text-to-video and image-to-video models are increasingly used for pre-visualization, animatics, and pitch material, but full generative shots that go to final delivery without heavy manual compositing remain rare outside of specific, forgiving use cases (background elements, crowd augmentation in wide shots).
- Consistency across a shot, and across a whole film, is unsolved at the edges. A model that nails 190 of 200 frames still requires an artist to find and fix the other 10 — and finding them is itself labor.
- Rights and consent frameworks are still being negotiated in practice, not just on paper. Union agreements set baseline terms for AI use and likeness rights, but the actual mechanics — how a de-aging model trained on an actor's earlier footage is licensed, who owns the resulting model weights, what happens when an actor's likeness is reused across projects — are still being worked out contract by contract.
- Attribution and credit norms haven't caught up. There's no industry-standard way to disclose "this shot used AI-assisted roto" to audiences or even within crew credits, which makes it hard to have a grounded public conversation about scale versus the more alarmist narrative.
- Smaller shops face a tooling gap. Enterprise-grade AI-assisted pipelines are easiest to justify at studio scale; independent and mid-size post houses often rely on the AI features bundled into general-purpose software rather than custom-trained, shot-specific models, which narrows their gains relative to large studios with in-house ML teams.
What to watch next
- Whether disclosure becomes standard practice. Netflix naming a specific title count is unusual; if other major studios and streamers start reporting similar figures, it suggests the industry is moving toward transparency as a norm rather than a one-off.
- How far de-aging and face-replacement tools move from "guided by a model" to "generated by a model." The gap between those two is currently where most of the remaining manual VFX labor lives.
- Whether generative pre-vis tools start bleeding into final-pixel work in specific shot types — background plates, crowd extensions, matte paintings — where quality bars are lower and the payoff for full automation is higher.
- Union contract renewals and how AI clauses evolve as more concrete production data (like Netflix's disclosure) becomes available to negotiate against, replacing speculation with actual usage patterns.
- Whether smaller studios get access to comparable tooling, or whether AI-assisted pipelines become another axis of consolidation favoring large, well-capitalized studios and streamers.
FAQ
Is AI replacing VFX artists?
Not in the way most headlines suggest. Current large-scale deployment, based on disclosures like Netflix's 2026 figure, is concentrated in assistive tasks — rotoscoping, cleanup, tracking — that speed up existing artist workflows rather than replace the creative judgment involved in compositing and shot finishing. Whether that balance holds as generative tools mature is genuinely unresolved.
What's the difference between AI-assisted VFX and fully AI-generated VFX?
AI-assisted VFX uses models to accelerate a specific manual task — like propagating a rotoscoping mask across frames — while an artist still directs and corrects the result. Fully AI-generated VFX means a model synthesizes new footage or shot elements from a prompt or reference material with minimal manual compositing afterward; this is far less common in finished feature and series work today.
Which VFX tasks are most automated by AI right now?
Rotoscoping and matting, camera and object tracking, wire/rig cleanup, and video upscaling or restoration are the most mature use cases. De-aging, voice dubbing, and face work use ML heavily but still require significant manual supervision. Full generative shot creation remains the least mature for final-pixel delivery.
How does AI-assisted rotoscoping actually work?
An artist manually corrects a mask on a handful of keyframes; a segmentation model then propagates that mask across the remaining frames in the shot, tracking the subject's edges through motion. The artist reviews the propagated frames and manually fixes weak spots — typically hair, motion blur, and semi-transparent areas — before the mask goes to compositing.
Are studios required to disclose AI use in VFX to audiences?
There's no universal industry standard for on-screen or credit disclosure of AI-assisted VFX work today. Some studios have started sharing usage figures publicly, as Netflix did for 2026, but formal disclosure requirements are still evolving and vary by studio, union agreement, and jurisdiction.
What do actor and writer union agreements say about AI in post-production?
Post-2023-strike agreements from SAG-AFTRA and the WGA established baseline consent and compensation requirements for uses of a performer's or writer's likeness or work in AI systems, including de-aging and synthetic voice work. The detailed mechanics of licensing and reuse are still being negotiated on a project-by-project basis rather than fully standardized.
Will AI make VFX cheaper for independent filmmakers?
Assistive tools embedded in mainstream editing and compositing software are gradually lowering the cost of tasks like cleanup and basic tracking for smaller productions. Full custom-trained pipelines and the most advanced de-aging or generative tools remain more accessible to large studios with dedicated ML teams and budgets, so the cost gap between independent and studio-scale VFX hasn't closed evenly across all task types.
If you're evaluating where AI genuinely fits into a media or post-production pipeline versus where it's still overhyped, Woyce Technologies works with teams building and adopting these tools hands-on.