Ultravox v0.5
Speech-to-speech model — ~100ms latency, no ASR/TTS pipeline needed
Comparative Scores
Architecture
Critical reference for Phase 1 MVP architecture decision: cascading (ASR+LLM+TTS) vs end-to-end (Ultravox). End-to-end eliminates latency accumulation but loses controllability. GamiWays Phase 1 should benchmark both approaches. CC-BY-NC-4.0 limits commercial self-hosting.
Analysis
Ultravox v0.5 is an end-to-end speech-to-speech model that eliminates the cascading ASR+LLM+TTS pipeline. ~100ms response latency vs 800ms–2s for cascading systems. Turn reliability: 300/300 vs 296/300 for GPT-4o Realtime. Median response latency: 0.864s vs 1.536s for GPT-4o. Best for latency-critical voice agents.
Strengths
- ~100ms response latency
- Eliminates cascading pipeline overhead
- 300/300 turn reliability
- 0.864s median latency vs 1.536s GPT-4o
- End-to-end speech understanding
Weaknesses
- No voice cloning
- CC-BY-NC-4.0 — non-commercial self-hosting only
- Less controllable than cascading systems
- No lip-sync data
Voice Capabilities
No voice cloning. Fixed voice output.
Vocal behaviour is chiefly guided by the system prompt; an external voice can strengthen control.
How: `systemPrompt`, `voice`, `temperature`, `externalVoice` and `voiceOverrides`.
Validate: Direction does not produce deterministic prosody; document response through scene tests.
~100ms response latency. Full-duplex capable. Eliminates ASR+LLM+TTS pipeline latency accumulation.
No native lip-sync data.
Pricing
~$0.05/min API. Self-hosted: GPU cost only.
Sovereignty & Compliance
Weights available under CC-BY-NC-4.0. Non-commercial self-hosting.
Data residency: Self-hosted: fully local.
Ultravox v0.5 — Strategic Positioning
Beyond technical specs: where does this tool sit in the ecosystem, what are the risks and strategic implications for GamiWays?
Ultravox eliminates the STT→LLM→TTS pipeline latency with a single multimodal LLM — open-source, self-hostable, created by the inventor of WebRTC. The architecture is the moat.
A. Strategic Positioning
Target customer: Developer / Enterprise — real-time voice agents, multimodal LLM
Open-source multimodal LLM (speech+text, no separate ASR) by Fixie AI — real-time voice agents without the STT→LLM→TTS pipeline latency.
B. Competitive Moat
- Multimodal LLM architecture — processes speech directly without separate ASR step
- WebRTC creator (Justin Uberti) involvement — real-time communication expertise
- Open-source + managed service hybrid — flexibility for enterprise deployment
Vulnerability: Open-source nature could lead to rapid commoditization. Small team (Fixie AI) — long-term maintenance risk.
E. Strategic Questions for GamiWays
Sovereignty fit
Fully self-hostable on Swiss/EU infrastructure. Open-source license. Multimodal architecture reduces data processing surface.
Build vs. Buy
Build (integrate open-source) for Phase 2 sovereignty. Use managed service for Phase 1 speed. Multimodal architecture reduces pipeline complexity.
Lock-in risk
Open-source model — zero vendor lock-in. Managed service creates soft dependency but self-hosted alternative always available.
Roadmap alignment
Good for voice agent use cases. Multimodal LLM approach aligns with GamiWays's need for natural real-time interaction.
This sheet's verification
Ultravox AIEWF eval, Feb 2026
Update note: Ultravox v0.5 released Feb 2026. MIT license. End-to-end S2S, no separate TTS. Llama 3.1 backbone. Self-hosted or Fixie.ai API. AIEWF eval: 0.864s median latency.
This date covers provider-specific prices, capabilities and notes. Comparative benchmarks follow the synchronization and methodology shown above.