OmniHuman 2: a performance from your photo and your audio

What it is

OmniHuman 2 takes one photo of a person and a piece of audio, and makes the person perform it. The mouth, the face, the body and the hands move on the sound, in one generation. There is no second lip sync pass on top.

The prompt gives the direction: per its documentation it is for "precise control of the scene, movements and camera". Say where they look, what they do with their hands and how the camera moves.

One run takes audio from 4 seconds up to 5 minutes, so a whole verse can be one continuous take, the way a performance is filmed on a real set.

What it is good at

The music drives the take

Timing comes from your audio, so the performance lands on the words and the beat you gave it.

A whole verse in one take

Up to five minutes per run. Cut away in your edit and come back, and the take is still on the music.

You direct it

Gestures, eye line and camera move go in the prompt, the same way you brief a performer.

Where it struggles

Audio has to be at least 4 seconds long, in wav or mp3, at most 15 MB.
It needs a person it can see. Start from a photo where the face and hands are clear.
The price follows the length of your audio. The Studio measures it and shows the amount before you run.

How to prompt it

Describe the performance and the camera, not the mouth. The sync comes from the audio. Spend the prompt on what the person does and how the shot moves.

An example that shows what that means

He raps straight into the lens with drill energy, points at the camera on the strong words and nods on every beat. Slow handheld push in.

What we offer here

Straight from the catalogue, so this list never goes stale. The price is the provider’s published rate with our markup on top, and it is on the button before you run anything.

OmniHuman 2 · performance from audioVideofrom 1,050 cr

Questions

Is this lip sync?

More than that. Lip sync redraws the mouth on a video you already have. OmniHuman makes the whole take from your photo and your audio, so the face, the body and the hands move on the music too.

Should I use the full track or only the vocal?

Try a piece of each on one shot. The vocal gives the mouth the cleanest signal, and the full mix gives the body the beat.

How do I build a music video with it?

One take per section, each from its own photo and the part of the song that belongs to it. Put each take in your edit at the second its audio starts, and lay your original track under the whole edit.