christos-antono/NPCTalk: Make any NPC in any game talk back with Fish Audio: skills, a self-hosted voice server and tools for AI coding agents, with a full GTA V example.

Make any NPC in any game talk back with Fish Audio: skills, a self-hosted voice server and tools for AI coding agents, with a full GTA V example. - christos-antono/NPCTalk

Summary

NPCTalk is a Python toolkit for adding voice-driven dialogue to game NPCs, shipped with a full GTA V example. It bundles Agent Skills, an npctalk CLI, and a self-hosted Fish Audio Drama 3 or S2 Pro voice server on Modal or fal, so an AI coding agent can wire speech-to-text, an LLM, voice cloning, and actions into a running game.

What it is useful for

Engineers can use NPCTalk to make single-player characters talk back when the player speaks, in a voice cloned from a recording they own. The skills include per-engine notes for GTA V, Unity, Unreal, Bethesda, Minecraft, and custom games, and the same building blocks are meant to work across titles. Adjacent needs are also covered: NPC actions and spoken orders via TypeSafe Jev, voice activity detection and push-to-talk input, persona authoring, and voice likeness measurement against held-out clips before claiming a setup is better. Skills live under skills/ and include make-npcs-talk, fish-voice-server, voice-references, npc-dialogue, npc-actions, voice-input, voice-likeness-eval, and npc-showcase-clips.

How engineers can use it

Install for an agent: Claude Code via /plugin marketplace add christos-antono/NPCTalk then /plugin install npctalk@npctalk; Codex with codex plugin marketplace add ... then codex plugin add npctalk@npctalk; Gemini CLI with gemini extensions install https://github.com/christos-antono/NPCTalk; VS Code/Copilot by enabling chat.plugins.enabled and "Install Plugin From Source"; Cursor by cloning (reads AGENTS.md and .cursor/mcp.json); OpenCode by cloning and running opencode. For skills only across any agent, npx skills add https://github.com/christos-antono/NPCTalk. The npctalk CLI is on PATH after a plugin or clone install, otherwise uv tool install git+https://github.com/christos-antono/NPCTalk.

Required tooling: Git, Python 3.10+, uv, ffmpeg. Required environment variables: NPCVOICE_TOKEN (Modal), FAL_KEY (fal), CEREBRAS_API_KEY (Cerebras), TYPESAFE_API_KEY (TypeSafe). The GTA V example additionally needs Windows, .NET SDK 8+, and the game.

The make-npcs-talk skill runs the loop the agent follows: recon the game, pick engines, stand up the voice server and test it with npctalk say, build reference voices, wire four hooks into the game (who's talked to, scene facts, voice output, actions), add voice input, add personas and actions, verify in-game with timing logs, measure voices, and write a field note. The README reports 1.2-2.2 second end-of-sentence-to-voice latency with push-to-talk and an additional 0.7 second wait for open mic.

Documented limitations: the project targets single-player games or self-run servers only; it never injects into online games with anti-cheat, and the GTA V docs warn against launching GTA Online with a script hook installed. Voice likeness numbers in the README are measured on one character (Lamar, ten test lines), and the README notes one early result did not survive a second run. For setup details beyond the CLI commands above, see examples/gta-v and docs/how-it-was-made.md.