OrtaHuman: How Digital Humans Work on Ortavox
September 29, 2026 · 6 min read · Ortavox Team
How Ortavox adds a real-time face to a voice agent: the avatar preparation pipeline, GPU rendering workers, how rendered video reaches the browser through LiveKit, where digital humans make sense, and the latency and quality trade-offs you should expect.
OrtaHuman is the avatar layer of the Ortavox platform, currently in beta. It takes an agent you have already built, with its prompt, knowledge, tools and voice, and renders a face that speaks the agent's audio in real time inside a browser call. This article explains what that layer is made of, where it fits, and what it costs you in latency and operational complexity compared with a voice-only agent.
The principle: the agent does not change
The core design decision is that the conversation engine stays exactly as it is. The Ortavox core still runs STT, the LLM, tools, memory, turn-taking and interruptions. The avatar worker is a consumer of the agent's synthesised speech: it receives the TTS audio, computes the lip motion and facial rendering, and publishes synchronised audio and video into the LiveKit room. The core does not publish a second copy of the voice; the only audio the user hears is the one that comes out of the renderer, in sync with the face.
This matches the avatar model in the LiveKit Agents SDK: an avatar session joins the room as a separate participant that publishes on behalf of the agent, and the agent's audio output is redirected to it over a data stream instead of to the room. Front-ends identify the avatar participant by its publish-on-behalf attribute, so the same React hooks that render a voice assistant render a digital human.
Two pipelines: preparation and rendering
A digital human on Ortavox is two separate GPU workloads with different shapes, and it helps to keep them apart.
Preparation
Preparation turns approved source material into a reusable avatar. From the workspace you create a catalog entry, upload a source video or photo to private object storage through a signed URL, and submit a build. The API records the job and publishes it to a queue; a preparation worker on a GPU claims it under a lease, downloads the source, runs the face-preparation pipeline and uploads an immutable, versioned asset bundle and manifest. Jobs move through queued, processing and succeeded or failed; a version only becomes ready once its assets are complete and validated.
Versions are immutable. A call that starts on version 3 of an avatar keeps using version 3 even if you publish version 4 during the call. Deleting an avatar cancels pending builds and revokes worker leases, but does not pull the rug from under an active render.
Rendering
Rendering is the real-time part. When a web call is started with the avatar mode, the API verifies that the caller owns the avatar, that the requested version is ready, and passes only the provider, avatar ID and version ID into the dispatch. No storage tokens or GPU endpoints are visible to the browser. The core worker then asks the avatar gateway for a rendering slot; the gateway reserves a worker from the registry, starts the renderer for that version, and the renderer joins the room. The core waits for the avatar's video track to appear before it plays the greeting, so users do not hear a voice over a frozen frame.
Render session state | Meaning |
|---|---|
allocating | A GPU slot is being reserved for this call |
loading | The worker is fetching and loading the avatar version from cache or storage |
ready | Audio and video tracks are published; the agent may speak |
active | The conversation is running |
stopping / ended | The call ended; the slot is released only after confirmed cleanup |
failed | Start-up failed; the call fails rather than silently falling back to audio only |
That last row is a deliberate choice. A customer who asked for a face and got only a voice would file a bug; a call that fails fast with a clear reason is easier to monitor and retry.
Which renderers and providers
OrtaHuman today ships with Ortavox's own self-hosted renderer, built on an open lip-sync pipeline (MuseTalk via the LiveTalking project) running in Python workers on NVIDIA GPUs. The control plane around it, the worker registry, heartbeats, capacity reservations and gateway, is written to a provider-independent contract, so a hosted third-party avatar API could be registered as another provider without changing how calls are dispatched. We do not list third-party avatar vendors as supported until they are wired and tested; if you need a specific one, ask.
Self-hosting the renderer is also what lets enterprise deployments keep avatar video, and the source material used to build an avatar, inside their own network. Avatar assets live in private object storage under the owner's prefix, with signed, short-lived URLs for previews, and workers hold scoped credentials rather than master keys.
Where digital humans fit
- Web support and onboarding: a face on the help page for product questions and guided set-up, where a visual presence changes how long people stay in the conversation.
- Sales and pre-sales: explaining a product and routing to the right next step, using the same tools and knowledge base as the voice agent.
- Kiosks and reception: hotels, clinics and venues where a screen already exists and a face is more approachable than a speaker.
- Internal assistants: turning connected knowledge into a conversation for staff, with the avatar as a familiar entry point.
What it is not for: telephony. Phone calls have no video, so the avatar mode is reserved for web calls, and a phone call never consumes a GPU slot. Keep the same agent for both; the avatar is a presentation option on the call, not a different agent.
Latency and quality trade-offs
Adding a face adds work between the TTS output and the user's ear, because audio and video must leave the renderer together. The practical consequences:
- First-response latency rises relative to audio-only. The renderer needs audio to compute mouth shapes, so the first frames trail the first audio bytes. Fast TTS matters more with an avatar, not less.
- Cold starts are the dominant cost. Loading an avatar version into a GPU process takes seconds when it is not cached; warm slots and version prefetch are how you hide it. Expect an allocating and loading phase at the start of a call, which the UI should show rather than hide.
- Interruptions must flush video as well as audio. When the user barges in, the renderer discards queued frames so the face stops talking when the voice does.
- Concurrency is a GPU budget. Each rendering session occupies a slot; how many slots a GPU can serve depends on measured VRAM, frames per second and p95 latency under simultaneous speech. Capacity for a launch should be planned from a benchmark on your actual avatars, not from a number on a vendor page.
- Source material drives quality. A short, well-lit, front-facing video with the mouth clearly visible prepares into a far better avatar than a long clip with head turns. The workspace is built around preparing from approved source material for exactly this reason.
We do not publish an end-to-end latency figure for OrtaHuman yet. The beta exists to measure it on real customer avatars and real networks before we commit to a number.
Starting a call
A web call with an avatar is the normal web-call request with a mode and an avatar reference. The response is the usual room and token; the client renders the avatar participant's video track alongside the transcript.
POST /v1/calls/web-call
{
"agent_id": "6d1c...",
"mode": "avatar",
"avatar": {
"avatar_id": "c5d6...",
"version_id": "8a5e..."
}
}From the console the flow is shorter: pick a library avatar or prepare your own, connect it to an agent, and press Call. The session, transcript and latency breakdown appear in the same call view as a voice-only call.
Sources
- Ortavox avatar architecture, API contract and core avatar documentation (internal)
- OrtaHuman product page — https://ortavox.ai/ortahuman
- LiveKit Agents: Virtual avatar models overview — https://docs.livekit.io/agents/models/avatar/
- LiveTalking documentation — https://doc.livetalking.ai/en/docs/