Case study

Interactive audio & real-time systems

AMS

I co-designed AMS as an adaptive music system for Unreal Engine, developing its musical architecture, compositions and interaction rules while my collaborator built the Blueprint implementation that responded to gameplay states.

Interactive audio design, composition & product R&D

Years of involvement

Back to Unreal profile
Development of AMS, the Adaptive Music System, in Unreal Engine

AMS, or Adaptive Music System, began in 2019 when Unreal Engine developer Jorge from Coqui Games approached me about collaborating on an adaptive-music concept.

Jorge had the beginnings of the technical idea: an Unreal Engine system that could respond to gameplay through Blueprint logic. He knew I composed and produced music and asked whether I would be interested in exploring the concept with him.

The detailed system emerged through those early discussions.

We began exploring how adaptive music could work musically as well as technically: vertical remixing through compatible layers, different gameplay states such as exploration, tension and combat, transitions between tempos and sections, and how the system might avoid the obvious musical interruptions created by simply switching between complete tracks.

We also looked at existing approaches, including Elias and adaptive-music systems already available through the Unreal Marketplace, to understand where a simpler in-engine alternative might fit.

The collaboration quickly settled into a natural division of responsibilities. Jorge handled the core Unreal Engine and Blueprint implementation, while I focused on composition, production, interactive-audio design and the musical architecture required to make the system work convincingly.

Rather than treating music as a collection of finished tracks, I began composing it as interconnected systems of compatible musical material. Individual stems could be introduced, removed or given greater prominence as gameplay intensity changed, allowing the same underlying composition to move between states such as exploration, tension and combat.

We also experimented with transitions between musical sections and tempos, diagnostic audio for testing the system, procedural techniques and different ways of making sections interchangeable without creating obvious musical breaks. In some experiments I deliberately constrained material to common musical properties, such as the same key and 4/4 time signature, so sections could be exchanged while preserving harmonic and rhythmic continuity.

That required thinking differently about composition. I was no longer simply asking whether a piece of music sounded good from beginning to end. I had to consider what would happen if the system entered halfway through it, removed several layers, replaced one section with another or changed intensity because of an event I could not predict when writing the music.

We built test material and increasingly substantial demonstrations around those ideas, including exploration, tension and combat states and a fantasy-themed example designed to demonstrate how the system could reshape a soundtrack in real time.

The product concept developed alongside the technical and musical work. We explored packaging the project as a reusable Unreal Engine asset rather than something tied to a single game. The core Blueprint framework could be sold with high-quality demonstration music, with additional compatible soundtracks released separately as expansion packs.

The proposition was effectively to give Unreal Engine developers some of the adaptive capabilities associated with specialist systems such as Elias, but through a workflow operating directly inside the engine.

That combination was particularly interesting to me because the software and the content could be designed together. Instead of creating middleware and expecting composers to adapt to it afterwards, we could develop the musical rules and technical behaviour in parallel, learning from each side as the prototype evolved.

AMS reached a substantial prototype and content-development stage, but it was never commercially released. I attempted to restart development in 2020, but Jorge subsequently joined Epic Games and we agreed that we needed to respect the potential employment and conflict-of-interest implications around continuing or releasing collaborative Unreal Engine work. The project was therefore placed on indefinite hold.

Although AMS never became the commercial product we originally envisaged, the R&D became valuable in its own right. It gave me practical experience designing music as an interactive system rather than a fixed asset, working with Blueprint-driven gameplay logic, translating creative requirements into technical behaviours and thinking about how specialist technology could be packaged into a reusable product for other developers.

It also established an approach I have continued to use in later game-development work: creative and technical systems work best when they are designed together rather than one being added after the other.

Open to opportunities

Let’s talk about your next challenge

Discuss an opportunity