Most of the AI video models that made headlines this year — Veo, Sora, Kling — are closed. You send a prompt to an API, you get back a clip, and you have no idea what's happening in between beyond what the provider's blog post tells you. MiniMax H3 breaks that pattern in a specific way: it's a 33-billion-parameter omni-modal video-and-audio generation model released with open weights, a published architecture, and inference recipes for SGLang, vLLM, diffusers, and ComfyUI. You can read exactly how it works, and if you have the GPUs, you can run it yourself.
That distinction — open weights on a model in the same capability class as the frontier closed labs — is the actual news here, more than any single benchmark number. This piece explains what H3 does, how its three-stage architecture is put together, and where it fits if you're deciding between a hosted API and something you control end to end.
What MiniMax H3 Actually Does
H3 takes text, images, video clips, and audio as input and produces video with native stereo audio as output — up to 2K resolution, 4 to 15 seconds long, at 24 FPS, in a wide range of aspect ratios (21:9 down to 9:16). It supports 11 languages for dialogue generation and handles a broad set of input combinations:
| Mode | What you give it | What comes out |
|---|---|---|
| Text-to-video | A text prompt, nothing else | Generated video + audio from scratch |
| First/last-frame | Text plus one or two reference images | Video that starts and/or ends on those exact frames |
| Omni-reference | Up to 9 images, 3 video clips, and 3 audio clips, mixed | Video that draws style, subjects, or sound from all of them |
That last mode is the one that separates H3 from a typical text-to-video tool. Most generators take a single prompt and maybe one reference image. H3's omni-reference mode can be handed a product photo, a clip of camera motion you like, and a piece of reference audio, and asked to synthesize a new video that's consistent with all three — which is closer to how a creative brief actually gets handed to a human video editor than how most AI video tools are used today.
The Three-Module Pipeline
H3 isn't one monolithic model — it's three components that run in sequence, and understanding the split explains both its capabilities and what's not open-sourced yet.
H3-Context-IR: Turning a Messy Prompt Into Structured Instructions
Before any pixels get generated, H3-Context-IR parses the input — text, images, video, audio, in whatever combination was supplied — and works out how the pieces relate to each other and to the intended output. It handles instruction parsing, cross-modal association (which image goes with which part of the prompt), temporal understanding, and fills in underspecified details without drifting from what the user actually asked for. The output is a structured "Context Intermediate Representation" that the generation model consumes directly.
This module is not part of the open-source release — MiniMax runs it as a hosted API, because it depends on a multi-stage workflow across several internal models. What is open is the prompting guidance that documents how to replicate its behavior, so teams that want a fully self-hosted pipeline can build an equivalent preprocessing layer rather than depending on the hosted service.
H3-Base: The Model You Actually Download
This is the open-weight core: the H3-Omni-Transformer, a 33B-parameter dense transformer (roughly 13B of those parameters sit in AdaLN-related branches, which can be precomputed and cached for inference-only deployment). It's a single-stream design — the attention and feedforward layers carry no modality-specific structure at all. Text, image, and audio each get their own encoder or VAE, get packed into one unified sequence using RoPE-based positional encoding across time and space, and then flow through the same transformer blocks. Modality only re-enters through the AdaLN branches and the input/output layers.
That single-stream choice matters architecturally. A lot of multimodal systems bolt separate expert pathways onto a shared backbone per modality; H3 instead keeps the diffusion transformer itself modality-agnostic and pushes all the modality-specific work to the edges. It's a bet on generalization: the same core reasoning capacity gets applied to visual and audio generation instead of being split across specialized subnetworks.
Two separate VAEs handle compression:
- H3-VisualVAE — a temporally causal video autoencoder with 16× spatial and 4× temporal compression, 24 latent channels. Visual latents get patchified further before hitting the transformer, for an effective 32× spatial / 4× temporal downsample overall.
- H3-AudioVAE — compresses 32kHz stereo audio into latent tokens at 40Hz per channel, processing left and right independently with a shared encoder/decoder before recombining them into stereo output.
The transformer jointly predicts video and audio latents in the same forward pass, which is what produces synchronized sound rather than sound bolted on afterward — the same native-audio principle behind Veo 3-class models, applied here inside an openly published architecture.
H3-Regenerate-2K: Upscaling by Re-Generating, Not Interpolating
Rather than running a conventional super-resolution model over the 768p output, H3 feeds the low-res result back into itself, alongside the original multimodal context, and regenerates at 2K in-context. The reasoning: a traditional upscaler has to guess fine detail — small text, precise textures — from pixels alone. Feeding the original context back in lets the model recover detail it already had access to the first time, rather than hallucinating it from a blurry input. Like H3-Context-IR, this module isn't open-sourced yet; MiniMax provides an API to validate 2K output while it's still being prepared for release.
Put together, the practical picture is: the generation core is open and self-hostable, the orchestration layers around it currently are not. That's a meaningfully different position than a fully closed model, and a meaningfully different position than a fully open one — worth being precise about if you're evaluating it for a real deployment.
How to Actually Run It
H3 ships as two task-specific checkpoints rather than one model that does everything:
| Checkpoint | Tasks | Input |
|---|---|---|
| H3 Base FL2VA | Text-to-video, first/last-frame-to-video | Text, plus optional first/last frame images |
| H3 Base Ref2VA | Omni-reference generation | Text plus mixed image/video/audio references |
Both are distributed as self-contained repositories (model_index.json, processor, tokenizer, text encoder, transformer, visual VAE, audio VAE) and can be pulled with a scoped hf download, loaded directly through diffusers' ModularPipeline.from_pretrained, or served with SGLang or vLLM. MiniMax's own reference deployment for either checkpoint targets 4 GPUs with tensor/sequence parallelism (--num-gpus 4 --ulysses-degree 4) — a meaningful but not exotic hardware bar, closer to what a mid-sized team already running open-weight LLM inference would have on hand than to a hyperscaler-only requirement.
For teams that want the full 2K pipeline end to end without standing up H3-Context-IR themselves, MiniMax exposes it alongside the base model as a combined API workflow — locally hosted H3-Base for generation, hosted Context-IR and Regenerate-2K for the preprocessing and upscaling steps. That hybrid setup is arguably the most realistic way most teams will run this in the near term: self-host the expensive, capability-defining part; call an API for the two pieces that aren't open yet.
Why an Open-Weight Model in This Class Is Notable
Native-audio video generation at usable resolution has mostly been the domain of closed frontier labs over the past year, largely because the training and serving cost is high enough that only a handful of organizations have built it. H3 doesn't change the training cost — building a model like this from scratch still isn't something most teams will do — but it changes who can run one at this capability tier.
That matters for a few concrete reasons:
- No per-call pricing risk. A hosted API's per-second video pricing is fine for prototyping and painful at production video volume. Self-hosting shifts the cost structure to fixed GPU spend, which is a better fit once usage is predictable and high.
- Data doesn't leave your infrastructure. Reference images, brand assets, or proprietary footage used in the omni-reference mode never have to be sent to a third-party API if the generation step runs on hardware you control.
- You can fine-tune it. MiniMax released full model weights specifically to support further development, not just inference — a closed API gives you prompt engineering as your only lever; open weights give you fine-tuning.
- It's inspectable. When output looks wrong, you can trace it back through a documented architecture instead of guessing at a black box, which matters for any team that needs to explain or debug production behavior.
None of that makes H3 strictly better than the closed alternatives — Veo and Sora-class models still have the advantage of being fully managed, with the orchestration layer (H3's equivalent of Context-IR) baked in rather than partially external. The honest framing is that H3 opens up a third option between "call a closed API" and "train your own multimodal video model from scratch," and that option didn't really exist at this capability level a year ago.
Where the Rough Edges Are
- The full 2K pipeline isn't fully self-hostable yet. H3-Context-IR and H3-Regenerate-2K remain hosted-only for now, so a genuinely offline deployment currently tops out at the 768p H3-Base output, or requires building an equivalent preprocessing system from the published prompting guidance.
- Hardware requirements are real. A 33B-parameter multimodal transformer serving both video and audio latents needs multi-GPU deployment for practical throughput — this isn't something you run on a single consumer card.
- Sparse attention isn't out yet. MiniMax notes the model was trained with native sparse-attention support for longer sequences, but the initial open-source release only ships full-attention inference; the more efficient path is coming in a future update.
- Safety filtering sits in the hosted layer. Automated content moderation on inputs and enhanced prompts is described as part of the hosted Context-IR workflow — teams building a fully self-hosted pipeline will need to think through their own moderation layer rather than inheriting one automatically.
Practical Takeaway
If you're building a product that needs occasional AI video generation — marketing clips, prototyping, low-volume content — a closed API is still the path of least resistance; you don't want to run multi-GPU inference for a handful of clips a week. If video generation is becoming a real part of your product's cost structure, or you need to keep reference assets off third-party infrastructure, or you want to fine-tune generation behavior for a specific brand or use case, H3 is the first model in its capability class where self-hosting that decision is actually on the table rather than theoretical.
Teams evaluating whether to self-host an open-weight generative model like H3 versus building against a hosted AI video generation API can get hands-on architecture and deployment help from Woyce Technologies.
FAQ
What is MiniMax H3?
MiniMax H3 is an open-weight, omni-modal generative AI system that produces video with synchronized stereo audio from text, image, video, and audio inputs, at resolutions up to 2K and durations up to 15 seconds.
Is MiniMax H3 fully open source?
The core generation model, H3-Base (the H3-Omni-Transformer plus its VAEs), is released with open weights and can be self-hosted. Two supporting modules — H3-Context-IR for input preprocessing and H3-Regenerate-2K for upscaling — are currently hosted-only via API and not yet open-sourced.
What hardware do you need to run MiniMax H3?
MiniMax's reference deployment uses 4 GPUs with sequence parallelism for either checkpoint. It's a multi-GPU workload, not something that runs practically on a single consumer graphics card.
How is MiniMax H3 different from Veo or Sora?
The underlying techniques — diffusion transformers with native joint audio-video generation — are similar in spirit. The practical difference is access: Veo and Sora are closed APIs, while H3's core model can be downloaded, inspected, fine-tuned, and run on your own infrastructure.
What is H3-Context-IR?
It's the preprocessing stage that interprets and structures multimodal input — text, images, video, audio — before generation, resolving how the different inputs relate to each other and to the intended output. It runs as a hosted API rather than as part of the open-source release.
Can MiniMax H3 generate video from a reference image and audio clip?
Yes — its omni-reference mode (the Ref2VA checkpoint) accepts up to 9 images, 3 video clips, and 3 audio clips in combination, and generates new video that draws on all of them, which goes well beyond a typical single-image-reference workflow.