DOCTOR AUDIO / TALK / AI IN GAME AUDIO
AI in Game Audio: Apocalypse or Liberation?
Where generative and assistive AI tools help a game audio pipeline, where they create rework, and how producers and audio leads can tell which is which.
Most discussions about AI in game audio run on two scripts: it is replacing composers, voice actors and sound designers, or it is a tool that solves everything. Both sides are confident. Both skip the question that matters in production: is the team still in control of the process?
This publication is adapted from a talk given at Nordic Game 2026. It documents what Flutu tested in real projects across voice over, music, sound design and the production layer around them, and sorts each use into one of two verdicts: liberation or apocalypse.
The reader is a producer, audio lead or technical sound designer deciding where these tools belong in a pipeline. The paper separates measured results from observed process changes, keeps limits next to each claim, and ends with a decision sequence another team can apply.
TL;DR
- Voice over is mostly liberation. Text to speech validates dialogue systems before recording, automated editing passes remove repetitive labor, and cloned voices make systemic layers viable. Lead performances stay with human direction.
- Music: assistive yes, directed composition no. Generative tools cannot be directed at the bar level an adaptive score needs. They work for diegetic, low-exposure music. Stem separation and DAW assistance speed up the musician without taking decisions away.
- Sound design: flat output, no system. Game SFX are built in layers the engine controls. Generative SFX returns a single flat file, so there is nothing to implement. A sourcing pipeline and a strong library gave better long-term results in our tests.
- The strongest case is the production layer. Scene breakdowns, naming conventions, technical documentation and first-pass tool prototypes carry almost no creative risk and remove real hours. This is where AI changed our workflow most.
AI in game audio: the wrong conversation
Most discussions about AI in game audio run on two scripts. One says AI is replacing composers, voice actors and sound designers. The other says AI is a tool that solves everything. Both sides are confident, and both are having the wrong conversation, because they treat AI as something a studio either embraces or resists.
In production the question is different. A tool enters a pipeline, and the useful question is whether the team is still in control of the process while using it.
This matters for producers and game directors because audio budgets are under pressure and the promise of reducing them with AI is attractive. Without a clear map of where these tools help and where they generate rework, the decision tends to be made on cost alone, and the cost of rework only shows up later in the schedule.
The gap between the public conversation and actual practice is also wide. A GameSoundCon survey found that around 15% of professional respondents were already exploring generative AI tools in game audio production. Our reading at Flutu is that real adoption is higher than this number, especially once you count technical workflows and production support rather than pure content generation. Many professionals avoid discussing how these tools are used internally because the topic is polarized, even when the tools are already part of the pipeline.
This whitepaper documents what we tested at Flutu, what worked, what did not, and the decision criteria we now use.
How we evaluated the tools
Before any AI tool enters the Flutu pipeline we run it in real production conditions and ask three questions: where it helps production, where it creates friction, and where it simply does not make sense. The same sequence was applied to every tool discussed in this paper.
Run the tool on a real task inside a live project, not on a demo prompt.
Voice over, music, sound design, or the technical and production layer.
Liberation: labor or risk removed, control kept. Apocalypse: control, identity or flexibility lost, savings turned into rework.
Verdicts are snapshots. Rerun the test when a tool changes materially.
One point runs through every section: audio leadership stays in the loop. The tools do not decide where quality comes from. A person responsible for quality does.
Voice over
Voice over has the strongest case for AI assistance today, for a structural reason. It has the highest asset count of any audio department. Multiply thousands of lines by several languages and the scale gets out of control quickly. Beyond performance itself there is a large amount of repetitive editing: trimming silence, cleaning breaths, basic editorial passes. This is necessary work that scales badly, and it is not the work sound designers want to spend their time on.
Illustrative diagram. The two highlighted stages are where AI enters a voice over pipeline in the cases below. A person approves every file before it ships.
Case 1: automated dialogue editing (Ubisoft La Forge, VoRACE)
Ubisoft La Forge built an internal tool called VoRACE to address exactly this. It automatically trims silence and breaths, applies a basic editorial pass, runs a text validation check, and only then sends the file to a sound designer for approval. What used to require listening to the same file many times becomes a single pass: approve or reject.
According to Ubisoft's GDC presentation, when tested on a project with around 80,000 dialogue lines the result was a 2x increase in delivery speed and a 10x reduction in manual workload. These are Ubisoft's measurements, not ours. The full breakdown is in their GDC talk: VoRACE: Tools for Automating Dialogue (GDC Vault).
Case 2: text to speech as a prototyping layer (Flutu, Mark of the Deep)
VO production spends a lot of time and money on casting and recording before anyone knows whether the dialogue system actually works. A common failure: an entire dialogue branch has to be rewritten after implementation, and by then every change costs a session.
On Mark of the Deep we populated the dialogue system with synthetic text-to-speech voices before recording a single actor. That was good enough to test pacing, implementation, dialogue logic and overall system behavior inside the game. The practical effect was that production decisions about the dialogue system were made before production money was spent on recording. This was an observed process change, not a measured cost saving.
Case 3: cloned voices for a systemic layer (Flutu, Final Exam)
On Final Exam we cloned our own team's voices to create the feeling of people trapped inside classrooms, whispering to the player. This is a secondary, systemic layer that would normally not survive a budget discussion. With voice cloning it became viable without a casting cost.
Verdict: mostly liberation, with clear boundaries
| Use case | Verdict | Why |
|---|---|---|
| Prototyping dialogue systems with TTS before recording | Liberation | Validates pacing and logic before casting costs |
| Automated editing passes (silence, breaths, text validation) | Liberation | Removes repetitive labor, keeps human approval |
| Systemic or background characters, crowd layers | Liberation | Low exposure, high volume |
| Secondary layers and production support | Liberation in many cases | Depends on exposure and direction needs |
| Lead performances and emotional storytelling | Apocalypse | Human direction and performance define the character |
Once you move into lead performances, emotional storytelling and character-defining moments, human direction and human performance still matter a lot. Trying to replace that with AI is where the apocalypse side appears.
Music
Music usually carries a significant portion of the audio budget, so the promise of reducing that cost with generative tools is attractive to producers.
Generative composition
We tested tools like Suno. Results can be surprising, but directing music with precision is still very difficult. A composer receives a brief like this:
"An epic battle track that loops between bars 9 and 17, harp intro, horns at the end, a timpani hit at bar 38."
That level of structural direction is normal for adaptive game music and it is where most generative systems fall apart. The output cannot be directed at the bar level, cannot be broken into the stems and transitions an adaptive score needs, and iteration becomes a prompting loop instead of a musical conversation.
Where generative music works today: diegetic and low exposure
A radio playing in the background. A distant speaker in a street scene. Low exposure, low stakes, low need for precise artistic control. For that production layer generative music is already useful and we use it.
Assistive tools inside the musician's workflow
The more interesting movement is AI used to assist musicians rather than replace them. Tools like Moises isolate stems, help study arrangements and speed up editing. DAWs such as Logic are integrating AI-assisted workflows directly into the creative process. These tools shorten mechanical steps while the musician keeps the decisions.
Verdict
| Use case | Verdict | Why |
|---|---|---|
| Directed composition, adaptive scores, emotional cues | Apocalypse | Cannot be directed with precision; creates rework and strips identity |
| Diegetic and low-exposure music | Liberation | Useful today, low artistic control needed |
| Stem separation, arrangement study, editing assistance | Liberation | Speeds up the musician without taking decisions away |
Forcing generative composition into directed music often creates more rework than savings. It is a shortcut that removes musical identity from the process. Used as assistance, the same technology does the opposite.
Sound design
Unlike film, game sound design is built in layers by nature. A gunshot is not one file. It is a transient, a tail, reverb reacting to the space, distant reflections. Vehicles are layered. Footsteps are layered. Everything reacts in real time to what the player does.
Those individual layers are what give the team control inside the engine. Without them there is no system, just a flat file with very little flexibility.
Illustrative diagram. The layer names are a simplified example of a gunshot, not a catalog.
The problem with generative SFX
This is the fundamental limitation of generative SFX right now. Most tools produce a single flat output, often at low quality, and they do not produce layers. Without layers there is no architecture to implement.
The best-known generative SFX platform today is probably ElevenLabs. For a studio or a solo developer without a strong sample library it is a legitimate starting point. It gives you something to work with.
At Flutu we have spent more than ten years building our own libraries: foley, creatures, vehicles, weapons, textures. Replacing that sourcing process with one that is not faster, does not give us layers and reduces implementation flexibility was not a good trade.
Verdict: where the apocalypse is most visible
| Use case | Verdict | Why |
|---|---|---|
| Fully generative SFX replacing professional sound design | Apocalypse | Flat output, no layers, heavy prompting loop, low impact on the final experience |
| Generative SFX as a starting point without a library | Acceptable | Gives material to work from; still needs layering and editing |
| Improving sourcing, custom recording, stronger libraries | Liberation (by comparison) | Better long-term results for a controllable, reactive system |
In real game production the challenge is not getting a sound. It is building a controllable, flexible and reactive system around that sound. In many cases a team will get better long-term results investing in its sourcing pipeline than brute-forcing generative SFX into production.
Beyond content: reducing operational friction
The most useful application of AI we found in game audio has little to do with generating sound or music. It is about reducing the mechanical work that surrounds creative decisions.
Well-documented productions scale more predictably. Better communication produces better iteration. And a lot of production time is lost to the mechanical work around decisions rather than the decisions themselves.
Documentation and breakdownsLLMs such as Claude and ChatGPT help us structure scene breakdowns, technical documentation and naming conventions. Before, a detailed sound design breakdown for a ten-minute scene could easily take an hour or more, mostly timestamping events and writing notes. Now the designer watches the scene, speaks observations out loud, and a structured production table is generated almost immediately. The designer still watches, still decides, still reviews the table. The hour of typing is what disappears. This is an observed change in our workflow, not a measured productivity figure.
Prototyping tools without waiting for engineeringAI-assisted scripting, sometimes called vibe coding: using AI tools to prototype scripts, editor tools and plugins without being an experienced programmer. Before, when an audio designer had a tool idea the process depended entirely on engineering availability. Now the audio team arrives much further along. We prototype first-pass editor tools, debug systems and validate ideas internally before bringing them to the engineering team. The output still needs cleanup, validation and technical direction.
AI is not replacing technical expertise. What it lowers is the friction between "we have an idea" and "we have something functional to show." This may be one of the strongest cases for AI in game audio today. Not replacing creativity, but helping creative and technical teams move faster without collapsing under production overhead.
What the evidence establishes
The claims in this paper rest on four kinds of evidence. They are categories, not a sequence. Each claim in the table below is tagged with one of them, and its limit sits in the same row.
- Measured result
- A number with context and a source. In this paper only the VoRACE figures qualify, and they are Ubisoft's measurement, not ours.
- Observed process change
- Something that happened in a Flutu production: a step removed, a decision made earlier, an idea that shipped. No time or cost measurement attached.
- Design intention
- How we chose to structure the evaluation and the decision sequence. A method, not a result.
- Pending validation
- Figures we quote from memory or estimate internally, such as hours per breakdown. Flagged, not claimed.
| Claim | Supporting evidence | Category | What it establishes | Limit |
|---|---|---|---|---|
| Automated dialogue editing cuts manual VO workload at scale | Ubisoft La Forge, VoRACE, GDC presentation, project of about 80,000 lines, reported 2x delivery speed and 10x less manual workload | Measured result (Ubisoft) | A measured result in a large AAA pipeline | Ubisoft's own measurement; requires engineering investment in an internal tool |
| TTS prototyping lets dialogue systems be validated before recording | Flutu, Mark of the Deep | Observed process change | Design and implementation decisions made before casting cost | No measured cost saving; depends on the dialogue system being testable in-engine |
| Voice cloning makes low-exposure systemic voice layers viable | Flutu, Final Exam, cloned team voices for a whispering NPC layer | Observed process change | An idea that would be cut on budget shipped | Consent and rights for cloned voices must be handled; not suitable for lead performances |
| Generative composition cannot be directed at the structural level adaptive scores need | Flutu internal tests with Suno | Observed process change | Limitation in bar-level direction, stems and transitions | Tools evolve; retest periodically |
| Generative music works for diegetic, low-exposure layers | Flutu production use | Observed process change | Acceptable quality for radios and distant speakers | Only where artistic control needs are low |
| Generative SFX does not produce the layers a game audio system needs | Flutu tests with ElevenLabs against a ten-year in-house library | Observed process change | Flat output, not faster than library sourcing | Different for teams without a library |
| LLM-assisted breakdowns remove the typing work of scene documentation | Flutu workflow change on scene breakdowns | Observed process change; hour figure pending validation | Reduction of a step that previously took about an hour per ten-minute scene | Qualitative; not measured across projects |
| AI-assisted scripting lets audio teams prototype tools before engineering | Flutu editor tool and debug prototypes | Observed process change | Ideas reach engineering as working prototypes | Output needs cleanup and technical direction |
Trade-offs and limits
Dependency on tool maturity
Every verdict in this paper is a snapshot. Generative tools change quickly and some limits described here will move. The evaluation method stays; the verdicts need periodic retesting.
Rework as the hidden cost
In the apocalypse cases (directed music, fully generative SFX, lead VO) the problem is rarely that the tool fails outright. It produces something, and the something costs more to fix than to make properly. Budgets that only count generation cost miss this.
Rights and consent
Voice cloning, even of the team's own voices, needs clear consent and usage terms. Generative music and SFX carry licensing questions that differ by platform and should be checked before shipping.
Loss of layers and architecture
Flat generated output removes the engine-side control that makes game audio a system. This is a structural limitation, not a quality one, and a better model does not fix it unless it outputs layers.
Where it does not serve
Teams treating audio as an isolated deliverable with no direction, no documentation and no ownership will not be helped by these tools. The tools accelerate whatever process already exists.
Status of what is described
VoRACE is an internal Ubisoft tool, not publicly available. The Flutu cases are production uses, not products. The documentation and prototyping workflows are practices, not packaged tools.
A wider note on the transition
AI is accelerating processes, lowering technical barriers and giving small teams access to workflows that would have required whole departments a few years ago. It is also creating instability across the market.
The distribution of that instability is uneven. Most AI infrastructure and computing power is concentrated in a small number of countries and companies, while large parts of the world still have limited internet access and digital infrastructure. The ITU estimates that roughly 2.6 billion people still do not have internet access. Teams in poorer countries may carry a disproportionate share of the transition's costs while having less access to its tools.
The tools themselves are becoming accessible to almost everyone with a connection. The difference is who is holding them and how those people think while using them. That points to investing less energy competing with automation and more in the skills that remain human: critical thinking, communication and leadership, complex problem solving, creative decision making.
Applying this to your project
For a producer or audio lead deciding where AI fits in a pipeline, this is the sequence we use. Each step narrows the next.
01 · Map the pipeline by exposure
Which assets does the player actually notice?
For each audio area, separate assets by how much the player notices them and how much direction they need. Lead VO and the main score are high exposure. Crowd whispers and a background radio are low exposure.
02 · Start with the low-exposure, high-volume work
Where is the best ratio of savings to risk?
That is where generative tools have the best ratio of savings to risk: TTS prototyping, diegetic music, systemic voice layers.
03 · Keep a human approval gate
Who says yes or no?
VoRACE works because a sound designer still approves every file. Any automated pass should end in a yes or no by a person responsible for quality.
04 · Check the output for layers before adopting a tool
Will the asset ever need engine-side control?
If a generative SFX or music tool returns a single flat file, decide whether the asset will ever need engine-side control. If yes, the tool is a starting point at most.
05 · Move AI to the production layer first
Where is the fastest return?
Scene breakdowns, naming conventions, technical documentation and tool prototypes carry almost no creative risk and remove real hours. This is usually the fastest return.
06 · Protect the high-exposure work
What stays with human direction?
Directed composition, lead performances and character-defining moments stay with human direction. The budget saved elsewhere can go here.
07 · Retest on a schedule
When do the verdicts expire?
Keep the verdicts as a living document and rerun the tests when a tool changes materially.
Apocalypse or liberation? For studios treating audio as an isolated deliverable with no direction, documentation or ownership, AI accelerates the path toward mediocrity. For studios with strong creative direction, documented processes and people responsible for quality from day one, AI becomes a powerful production layer.
The tools do not decide which one you get. The team does.
Questions We Can Answer
Why does Flutu publish technical whitepapers?
Around 99% of our projects are covered by strict non-disclosure agreements. That limits how much we can share through named case studies, project footage or internal production material.
Whitepapers give us a way to document how we have helped AA and AAA teams develop working methods, implementation standards and validation workflows, while respecting those confidentiality commitments. We focus on the methods, technical reasoning and evidence we can share without exposing protected project details.
This gives prospective partners a clearer basis for technical evaluation and due diligence: how we investigate systems, structure production work and check results. Each paper distinguishes what was implemented, what was observed and what remains unverified.
Does this mean Flutu uses AI to replace composers, actors or sound designers?
No. The cases in this paper use AI for prototyping, editing passes, systemic low-exposure layers, documentation and tool prototypes. Lead performances, directed composition and the core sound design system stay with human direction. Every automated step ends in a person approving the result.
Which tools are named here, and does that mean Flutu endorses them?
The paper names Suno, Moises, Logic, ElevenLabs, Claude and ChatGPT because they are the tools we actually tested or use, and the Ubisoft VoRACE tool because it is publicly documented. Naming a tool describes a test, not an endorsement. Each verdict is tied to the production context in which it was observed.
How often should these verdicts be revisited?
Whenever a tool changes materially, and at least once per project cycle. The verdicts are a snapshot of tests run before and during the Nordic Game 2026 talk. The evaluation sequence in the method section is what carries over; the individual results should be rerun.