Technology
Experience, voice, characters, orchestration, sources, media and observation remain distinct responsibilities.
The stack separates interface, voice, characters, orchestration, sources and observation.
The Où est Ava ? orchestrator belongs to this specific prototype; it is not a generic engine.
In Où est Ava ?, the application retains control of the microphone, transcription, RAG and dialogue. Depending on the output service, final text is converted to speech or sent to the avatar renderer as text or audio. Device conditions are described in Experience.
Prototype execution architecture
From speech to delivery: the path that must answer now, then what prepares and measures the next turn.
A web experience, an editing and orchestration application, AI services and content sources connected through clearly separated flows.
Web experience
AI services · listening
Editing & orchestration
Narrative memory
Orchestration
AI services · response
Beside the audible path
Triggered toward the experience, separately from the avatar.
Latency and traces, without delaying the response.
Diagram focus
Conversation engine
This engine runs the live turn. It requests useful context, calls the response model and sends final text to the output chain.
Software stack
The prototype services, with their logo and role. The four avatar rendering services are included in this list.
Three connected layers: where we write (Notion), how the experience is orchestrated (API, RAG, GM), which external services listen and speak (Deepgram, OpenRouter, TTS…). Click a component for detail and links.
Creation / editing tools
Application / orchestration
Application / orchestration
Où est Ava ? prototype orchestrator
Technical conductor — chains STT, RAG, GM, LLM, TTS per turn
This is not the generic Core. It is ava-proto1's specific orchestrator: query rewriting, character-scoped retrieval, generation, validation and sentence-level synthesis. The relational Game Master runs after the turn, not as blocking approval before each line. The generic Core is developed separately; it is not this implementation.
What this component does
- Edge functions: proxy-llm, proxy-stt, proxy-tts, query-rag, rewrite-query…
- Aggregates per-segment latencies for admin and PostHog.
- Persists sessions, conversation_log, voice_turn_events.
In the turn flow
Incoming turn: STT → rewrite-query → RAG → Max LLM → validator → TTS → post-turn GM (async)
Links to other components
The layer in one sentence
Layer visible to users and the team: post-film experience in the browser, API chaining AI services, control back-office, and narrative memory feeding Max.
Infrastructure / services
Tap an item to show its detail below.
Three connected layers: where we write (Notion), how the experience is orchestrated (API, RAG, GM), which external services listen and speak (Deepgram, OpenRouter, TTS…). Click a component for detail and links.
Creation / editing tools
Application / orchestration
Infrastructure / services
Application / orchestration
Où est Ava ? prototype orchestrator
Technical conductor — chains STT, RAG, GM, LLM, TTS per turn
This is not the generic Core. It is ava-proto1's specific orchestrator: query rewriting, character-scoped retrieval, generation, validation and sentence-level synthesis. The relational Game Master runs after the turn, not as blocking approval before each line. The generic Core is developed separately; it is not this implementation.
What this component does
- Edge functions: proxy-llm, proxy-stt, proxy-tts, query-rag, rewrite-query…
- Aggregates per-segment latencies for admin and PostHog.
- Persists sessions, conversation_log, voice_turn_events.
In the turn flow
Incoming turn: STT → rewrite-query → RAG → Max LLM → validator → TTS → post-turn GM (async)
Links to other components
The layer in one sentence
Layer visible to users and the team: post-film experience in the browser, API chaining AI services, control back-office, and narrative memory feeding Max.
Game Master & narrative RAG
Three angles — why a GM, its concrete role, and narrative RAG risks.
An AI character cannot simply be a "well-prompted" LLM. It must respect what it knows, what it ignores, what it hides, its emotional state and the rhythm of the experience. The Game Master acts as an invisible director.
The difficulty is not just making Max speak. The difficulty is preventing him from knowing too much, answering too well, or leaving his world.
Main technical challenges
Seven practical questions — browse the list for the checks associated with each challenge.
Seven practical questions — browse each challenge to see what to observe or check.
Latency
Critical challenge — measure the wait until first audio
Questions and checks
- Measure transcription time
- Distinguish first text from the complete response
- Measure first audio for each output mode
- Observe waiting with visual feedback
Tap an item to show its detail below.
Seven practical questions — browse each challenge to see what to observe or check.
Latency
Critical challenge — measure the wait until first audio
Questions and checks
- Measure transcription time
- Distinguish first text from the complete response
- Measure first audio for each output mode
- Observe waiting with visual feedback