Text To Speech | Techlist.io

cloudflareApr 15, 20263 min readCurated summary

Add voice to your agent

Cloudflare’s experimental `@cloudflare/voice` package adds real-time voice to existing Agents SDK applications without requiring a separate voice framework. Voice interactions use the same Durable Object, WebSocket connection, tools, and SQLite-backed history as text interactions. The package provides ready-made STT and TTS integrations while keeping provider interfaces open for alternative speech, telephony, and transport systems. ## Voice Support for Existing Agents - `withVoice(Agent)` enables full conversational voice agents. - `withVoiceInput(Agent)` supports speech-to-text-only features such as dictation and voice search. - React applications can use `useVoiceAgent` and `useVoiceInput`. - Framework-independent clients can use `VoiceClient`. - Built-in Workers AI providers include: - Deepgram Flux for continuous speech-to-text - Deepgram Nova 3 for speech-to-text - Deepgram Aura for text-to-speech - Developers can get started without external API keys. ## Minimal Server and Client Setup - A voice agent extends a class created with `withVoice(Agent)`. - The server configures a transcriber and TTS provider, then implements `onTurn()`. - `onTurn()` receives the user’s transcript and returns the agent’s response. - React clients can display: - Connection status - Interim and finalized transcripts - Conversation messages - Start, end, and mute controls - Non-React applications can connect through `@cloudflare/voice/client`. ## How the Voice Pipeline Works - The browser captures 16 kHz mono PCM microphone audio. - Audio streams over the agent’s existing WebSocket connection. - A continuous STT session remains active for the duration of the call. - The speech-to-text model detects completed utterances and produces stable transcripts. - Each transcript is passed to `onTurn()` for application or LLM logic. - The response is synthesized into audio and streamed back to the client. - Streamed responses can be sentence-chunked so audio begins playing before the full response is complete. - User and agent messages are persisted in the Durable Object’s SQLite database, surviving reconnections and deployments. ## Extensible Provider Architecture - The package is designed not to lock developers into one fixed voice stack. - Small provider interfaces allow speech, telephony, and transport providers to build integrations. - Developers can mix and match components based on their application’s requirements. - Voice therefore becomes another interaction mode for the same stateful agent rather than a separate application architecture. Cloudflare’s approach is best suited to developers who already use the Agents SDK and want to add conversational voice while preserving existing state, tools, persistence, and connection patterns. Since the package is experimental, teams should evaluate provider support and API stability before relying on it in production.

Read original

Text To Speech Database Design React Sqlite+3

kakaoFeb 11, 20262 min readCurated summary

Introducing the new Kanana-o

Kanana-o is Kakao’s new Korean-focused omni-modal AI model, designed to understand and generate text, images, and audio naturally. Kakao is opening a closed beta for the Kanana-1.5-o-9.8b-2602 model to gather feedback from developers and partners before commercial release. The service emphasizes practical experimentation rather than large-scale traffic handling. ## Model Capabilities - Supports simultaneous processing of multiple modalities, including text, images, and audio. - Specializes in: - Deep understanding of Korean language, culture, and user intent. - Natural Korean speech with expressive intonation, pacing, and emotion. - Flexible applications such as podcast narration, multi-turn conversations, and multi-speaker text-to-speech. - Balances text-generation speed with audio-processing speed to produce more natural spoken responses. ## API Beta Service - **Service:** Kanana-o API Beta - **Model:** Kanana-1.5-o-9.8b-2602 - **Beta period:** February 27–May 27, 2026 - **Access:** Selected testers receive a fixed number of daily API uses during the beta. - The closed beta is intended for meaningful developer testing and feedback, not high-volume production workloads. ## Application and Selection - Applicants should visit [omni.kanana.ai](https://omni.kanana.ai/), sign in with a Kakao account, and submit information about: - Their organization or affiliation - Intended purpose - Expected technical scenarios - Selected applicants will receive invitations and API documentation through KakaoTalk notifications starting February 27. - Kakao is seeking developers, students, startups, and researchers with concrete implementation plans. - Specific proposals—such as building a visual shopping assistant for people with visual impairments—are favored over general interest in trying AI. Developers interested in exploring Korean-language, audio, and vision applications can apply for the beta with a clearly defined use case and prototype plan.

Read original

Text To Speech AI API Kanana O+1

googleNov 18, 2025Original article

Real-time speech-to-speech translation (opens in new tab)

Google DeepMind and Google Core ML have developed an innovative end-to-end speech-to-speech translation (S2ST) model that enables real-time, voice-preserved communication with only a two-second delay. By replacing traditional cascaded pipelines with a streaming architecture trained on time-synchronized data, the system overcomes long-standing issues of high latency and accumulated errors. This advancement represents a significant shift toward natural, fluid cross-language dialogue that retains the original speaker's personality. ## Limitations of Cascaded S2ST Traditional real-time translation systems typically rely on a cascaded chain of three distinct AI models: Automatic Speech Recognition (ASR), Automatic Speech Translation (AST), and Text-to-Speech (TTS). This approach suffers from several critical drawbacks: * **High Latency:** Processing through three separate stages results in a 4–5 second delay, forcing users into unnatural, turn-based interactions. * **Error Propagation:** Inaccuracies in the initial transcription or translation phase accumulate, often leading to garbled or incorrect final audio output. * **Loss of Identity:** General-purpose TTS engines generate generic voices, stripping the communication of the original speaker’s unique vocal characteristics. ## Time-Synced Data Acquisition Pipeline To train an end-to-end model capable of low-latency output, researchers created a scalable pipeline that transforms raw audio into a specialized time-synchronized dataset. * **Alignment Multi-mapping:** The process uses forced alignment algorithms to map source audio to source text, source text to translated text, and finally, translated text to generated speech. * **Voice Preservation:** A custom TTS engine generates the target language audio while intentionally preserving the vocal characteristics of the original speaker. * **Strict Validation:** Automated filters discard any segments where alignments fail or where the translated audio cannot meet specific real-time delay requirements. * **Data Augmentation:** The training set is further refined using techniques such as sample rate reduction, denoising, and reverberation to ensure the model performs well in real-world environments. ## End-to-End Streaming Architecture The model’s architecture is designed for continuous audio streams, leveraging the AudioLM framework and fundamental transformer blocks to make real-time decisions. * **Streaming Encoder:** This component summarizes source audio data by focusing on the preceding 10-second window of input. * **Streaming Decoder:** This module predicts translated audio autoregressively, utilizing compressed encoder states and previous predictions to maintain flow. * **RVQ Audio Tokens:** The system represents audio as a 2D set of Residual Vector Quantization (RVQ) tokens, where the X-axis represents time and the Y-axis represents audio quality/fidelity. * **SpectroStream Integration:** By using SpectroStream codec technology, the model manages hierarchical audio representations, allowing it to prioritize the sequential output of audio segments for immediate playback. This technology effectively bridges the gap between high-quality translation and real-time responsiveness. For developers and researchers in the field, the transition from modular cascaded systems to end-to-end streaming architectures—supported by rigorous time-aligned datasets—is the recommended path for achieving truly seamless human-to-human cross-language communication.

Text To Speech AI Machine Learning Transformer+5