The machine plays too

When people hear that there are agents and an “AI sound engineer” in the live set, they tend to ask one of two questions. Either “so who's actually playing?” or, more politely, “and what do you do?” Both are fair. I'll try to answer them without the word “revolutionary”, which I've banned from this site.

A hundred years of handing things over

I spent a long time writing a book about how machines learned to make music, and the short version of it fits in a paragraph. Over roughly a century, people handed their work to machines one layer at a time. First the source of the sound itself, when a tone could come straight out of electricity. Then the memory of sound, on tape. Then performance, when the sequencer took over the obligation to play every note, and Giorgio Moroder's click in “I Feel Love” ran steadier than any drummer. Then the studio, which ended up inside a single computer. Then the rules of composition, then hearing, and most recently the composition itself.

Each of these steps was greeted by some people as a liberation and by others as the death of music. Neither side was quite right. After each step, music didn't die and wasn't liberated. It sounded different, and the people making it moved one layer up.

That last part is the one I care about. Every time a machine took over a job, the musician didn't disappear. He moved to the next thing the machine couldn't do yet, which was usually closer to deciding than to doing.

Where the line is in a live set

So here is how the work is split when Nobody Quits plays live, as it stands now. The system is still being built, so treat this as a description of intent more than a spec.

I keep the direction. That means where the set is going, how long the tension should be held before something gives, when a section has said enough, and which of several plausible next moves is the right one for this room tonight. I can say it in words (“darker, and don't release for a while”), with a controller, or by playing something. The system hears all three as the same kind of thing: an intention.

The machine takes the decisions that follow from that intention and that happen faster or more often than I can make them well. Agents translate “darker, hold it” into concrete moves: which filter drifts where over the next few minutes, how dense the percussion gets, which harmonic options stay on the table and which are closed. Generative parts produce the variations inside those limits, so a phrase that comes back is never quite the same phrase. And the AI sound engineer does what a good engineer at a desk does all night: listens, compares what it hears with how it should sound, and moves the faders, so that a build gets more intense without just getting louder.

It helps me to put it in the same terms as the book. The machine has the sound source, the memory, the note-by-note performance, the mixing desk and a good part of the ears. I have the intent and the right to say no.

Why the intent can't be handed over, yet

One of the less comfortable ideas I ran into while writing is that every generative system has a kind of gravity. Left alone, it drifts to the middle, towards whatever is most probable. A mastering assistant pulls a track towards the average curve of its genre. A prompt made of adjectives pulls a song towards the average of those adjectives. An agent that is asked to “build tension” with no other context will build the most typical tension it knows, which is exactly the kind I stopped wanting in 2013.

The only thing that pushes against that gravity is knowing in advance where you want to end up and recognising it when it shows up. There's nothing mystical about it. It's a direction held over time, plus the ability to say “no, not that one” quickly enough to matter. At the moment that's my part, and I don't think it's a small one. In a set that works as one continuous space, the direction is most of what's there.

I say “at the moment” deliberately. The book doesn't end with a guarantee that the last layer stays with people, and I'm not going to promise one here either.

What it's like on stage

In practice this feels less like operating a machine and more like playing with a band that's very fast, never tired, and has no opinions of its own about where the night should go. I say where. They find a way. Sometimes the way is better than what I had in mind, and then I follow it, which is also a decision. Sometimes it's the most probable thing in the world, and I veto it and ask again.

That's why the phrase on this site is “the machine plays too” and not “the machine plays”. The word “too” is doing the work. It means there's more than one player, that they have different jobs, and that one of them answers for the result.

A note on labels

AI is a tool inside this system, in the same sense that a sequencer or a compressor is. I don't put it on the poster, for the same reason nobody put “now with voltage control” on a techno flyer. If the set works, you'll hear a set. If you hear the technology first, something has gone wrong, and it's probably my fault rather than the machine's.