LiveKit and Pipecat

Umay (speech to text) and Ses (text to speech) answer the OpenAI audio APIs, so the OpenAI plugins of LiveKit Agents and Pipecat work with them. Change the address and the key; nothing else is needed.

	Speech to text (Umay)	Text to speech (Ses)
Address	Europe (Italy): `https://stt.eu.mihu.ai/v1` Türkiye: `https://stt.tr.mihu.ai/v1`	Europe (Italy): `https://tts.eu.mihu.ai/v1` Türkiye: `https://tts.tr.mihu.ai/v1`
Live path	`WS /v1/realtime`: words while the caller speaks, final text at the end of each turn	`POST /v1/audio/speech`: audio streams while it is generated
Model name	`umay`	`mihu-ses-v1`
Audio	24 kHz 16-bit PCM in, as the plugins send it	24 kHz 16-bit PCM out, as the plugins expect it

On your own servers, use your server’s address instead (see On-premises). Tested with livekit-agents 1.8.12 and pipecat-ai 1.16.0.

LiveKit Agents

# pip install "livekit-agents[openai,silero]"
import os
from livekit.agents import AgentSession
from livekit.plugins import openai, silero

stt = openai.STT(
    base_url="https://stt.eu.mihu.ai/v1",
    api_key=os.environ["MIHU_STT_KEY"],
    model="umay",
    language="tr",
    use_realtime=True,                     # live over WS /v1/realtime
)

tts = openai.TTS(
    base_url="https://tts.eu.mihu.ai/v1",
    api_key=os.environ["MIHU_TTS_KEY"],
    model="mihu-ses-v1",
    voice="alp",                           # a Ses voice id or your custom voice id
    response_format="pcm",                 # no encoder in between: lowest delay
)

session = AgentSession(stt=stt, llm=..., tts=tts, vad=silero.VAD.load())

STT: with use_realtime=True the plugin opens /v1/realtime?intent=transcription. Umay ends a turn after about 0.8 s of silence; the turn_detection setting the plugin sends is accepted but does not change that. Without use_realtime, LiveKit cuts the audio with its VAD and sends each utterance as a file to /v1/audio/transcriptions; that also works, with more delay.
TTS: the plugin sends each sentence as its own request and plays it while it streams. When the caller interrupts, LiveKit closes the request and Ses stops synthesis.
Language of the voice: the TTS plugin does not send language; the language comes from the voice. Pick a voice of the language you speak: alp for Turkish, emily for English (see Voices).

Pipecat

# pip install "pipecat-ai[openai,silero]"
import os
from pipecat.audio.vad.silero import SileroVADAnalyzer
from pipecat.audio.vad.vad_analyzer import VADParams
from pipecat.services.openai.stt import OpenAIRealtimeSTTService
from pipecat.services.openai.tts import OpenAITTSService
import pipecat.services.openai.tts as openai_tts
from pipecat.transcriptions.language import Language

stt = OpenAIRealtimeSTTService(
    api_key=os.environ["MIHU_STT_KEY"],
    base_url="wss://stt.eu.mihu.ai/v1/realtime",
    settings=OpenAIRealtimeSTTService.Settings(model="umay", language=Language.TR),
)

# Pipecat accepts only OpenAI voice names; add the Ses voice (or custom voice id) you use to its list
openai_tts.VALID_VOICES["alp"] = "alp"

tts = OpenAITTSService(
    api_key=os.environ["MIHU_TTS_KEY"],
    base_url="https://tts.eu.mihu.ai/v1",
    settings=OpenAITTSService.Settings(model="mihu-ses-v1", voice="alp"),
)

vad = SileroVADAnalyzer(params=VADParams(stop_secs=0.8))   # give it to your transport or VAD processor

# pipeline: transport.input() -> stt -> user aggregator -> llm -> tts -> transport.output()

STT: Pipecat sends input_audio_buffer.commit when its VAD hears the caller stop, and Umay gives the final text of that turn at once. With Silero’s default 0.2 s, a short pause inside a sentence (“Adresiniz Bornova, …”) already ends the turn and the sentence comes back in two pieces; set SileroVADAnalyzer(params=VADParams(stop_secs=0.8)) to keep sentences whole.
TTS: Pipecat asks for pcm at 24 kHz, which is what Ses sends.
Voice names: Pipecat refuses voice names that are not OpenAI’s. The line that adds alp to VALID_VOICES lets you use any Ses voice or custom voice id. Without it, an OpenAI name works too: Ses answers with the voice of the same gender in the default language (onyx → a Turkish male voice).
For file-based recognition instead of live, OpenAISTTService(base_url="https://stt.eu.mihu.ai/v1", ...) sends each utterance to /v1/audio/transcriptions.

Names and places

Both plugins send a prompt (LiveKit: openai.STT(prompt=...), Pipecat: Settings(prompt=...)). Umay turns it into hints for Turkish: a list such as "Bornova, Karşıyaka, Ali Yılmaz" makes those names easier to recognise. See OpenAI-compatible API.

Getting Started

API Reference

Transcription

Voice

Integrate and deploy

LiveKit Agents

Pipecat

Names and places