Tod Machover’s new symphony plays with AI while reflecting on democracy
MIT News ·
The MIT composer’s inventive “already and not yet,” premiering at the Boston Symphony Orchestra, uses new AI tools to create interplay with musicians while thematically exploring the fortunes of democracy.
Innovation, reflection, and celebration will blend at the Boston Symphony Orchestra this week, where MIT composer Tod Machover’s inventive new work “already and not yet” will see its world premiere on Thursday.
Machover, one of the world’s most prominent composers, has incorporated multiple ongoing interests into the new symphony, including AI tools interacting with the musicians, and a piece that is structured as a meditation on the crisis and promise of democracy.
“An orchestra is an interesting ensemble to use for thinking about how different groups of people can relate to each other,” says Machover, who is the Muriel R. Cooper Professor of Music and Media and faculty director at the MIT Media Lab, where he also serves as director of the Opera of the Future group.
He adds: “John Cage used to say one way to think of a piece of music is that it should be like the society you’d like to live in. I was always inspired by this idea, and thought of the orchestra — and constructed this new piece — in this way.”
If the innovation in “already and not yet” comes from its use of artificial intelligence, and the reflection pertains to democracy, the celebration involved in the event partly relates to MIT: The performances are coordinated with the MIT Media Lab’s 40th anniversary celebrations occurring this week. Machover has been a member of the Media Lab since it opened.
“In many ways, the piece is also an homage to the lab and to its spirit of invention and reinvention,” Machover says.
The acclaimed conductor Kent Nagano — a longtime collaborator and advocate for Machover’s music — will lead the Boston Symphony Orchestra’s performances, which are taking place Oct. 22-24 at Boston’s Symphony Hall.
Inspired by the possibilities
Machover’s extensive career has often featured the invention of new technologies within his compositions. Among many other technologically related projects, he has led the development of hyperinstruments, which can be responsive to the playing techniques of musicians and expand their range of expression; renowned cellist Yo-Yo Ma is among those who have performed Machover’s work using them.
Working across diverse musical forms, Machover’s 2010 opera “Death and the Powers” was a Pulitzer Prize finalist. He has also used locally sourced and community-created sounds as additional aural layers in his “City Symphonies,” which have been created around the world. The next project in the series, underway in Seoul, South Korea, is scheduled for premiere in September 2027.
“I’ve always been inspired by the possibilities of technology and the expressive potential it opens up,” Machover says. “It is kind of the core language of our time and something we have to grapple with so that it evolves in the most positive human-enhancing ways. I feel so fortunate that I have been able to pursue this vision at MIT, and particularly at the Media Lab, for all these years.”
Machover’s “already and not yet” reflects on democracy in multiple ways. One is through the form of the composition, which has three movements relating to different stages of democracy. In the first movement, “Smash Up,” strings and winds eventually get crowded out by what Machover calls the “oppressive, aggressive” sounds of brass, percussion, and electronics. The second movement, “Wake Up,” presents a moment to reflect and regroup, but little by little weaves together a thriving melodic and textural presence. The third movement, “Stand Up,” involves more vigorous interplay between all the musicians, inviting the players to participate in establishing a new kind of balance between individuality and community.
In this way, “already and not yet” conceptualizes the orchestra as a way of representing the stress and revival of democracy, and uses a “Live AI” system created at the Media Lab — along with layers of electronics crafted in Machover’s studio — to emphasize this struggle and progression. In the second movement of “already and not yet,” the AI tools aim to underscore the distinctiveness of each instrumentalist, establishing an ensemble of soloists; in the third movement, more broadly representing the regrowth of a polity, the AI system aims to find common ground, building new sonic bridges across the orchestra and extending it out into the hall using extra, specially installed loudspeakers.
“I’ve used the way the orchestra plays, how much is scripted and how much freedom they have, and how they negotiate their relationship with the conductor, as part of the story,” Machover says. “It’s not said explicitly, but I think the players and the audience will feel it. It’s a whole society up there on stage, and the liveliness and ever-changing nature of the AI systems for this piece both dramatize and encourage a new way of playing and listening.”
Machover credits two PhD candidates in his group at the Media Lab for their extensive work on the project: Manaswi Mishra, who designed and built the AI systems for the performances, and Ana Schon, who will be doing the complex sound mixing in Symphony Hall during each performance.
Reflections of an AI veteran
For Machover, the application of machine learning and AI is hardly a newfound idea; he has been exploring the concept for decades, giving him a distinctive perspective on this topic du jour.
“The current generation of AI, especially generative AI, has capabilities that are different than AI from 20 or 30 years ago,” Machover says. “It’s good at certain things that were hard to do 20 years ago, but less good at some things that were easier to do back then.”
When he designed a hypercello for Yo-Yo Ma, for instance, those systems would analyze Ma’s sound, playing techniques, and expressive intent in order to shape the extensions of the instrument. Today’s AI tools “are less good at analysis, but very good at detecting significant features and patterns in the audio itself, information that can be leveraged to powerful musical effect,” Machover says.
“Current systems aren’t so good at what we did [for Yo-Yo Ma], like trying to understand the performance or the music itself,” Machover says. “But evolving AI models do an incredible job — especially in Manaswi’s magical systems — at establishing an immediate, creative dialogue with musicians, producing sophisticated and surprising augmentations that in turn inspire performers or composers to discover truly new ideas.”
Ultimately, Machover says, “I think this piece — through its technology, its ‘story,’ and its sounds — is meant to take people on a journey of recognition about where we are at this moment in time and where we might be going. It’s meant to be an optimistic and inspiring piece.” He adds, “The idea, expressed in the work’s title, is that we’re so lucky to be at this place in this moment, even with everything that’s wrong and in critical need of repair. We really are. At the same time, it’s so urgent to realize that the vision of a just and flourishing society, for which the foundation exist, isn’t finished at all. Not only that, it’s in crisis. So this is the last time to be complacent, or discouraged.”
For all of that, Machover also expresses hope that the audience at Symphony Hall can simply engross themselves in the music itself, and find it compelling on its own terms.
“My biggest wish is that people listen to this piece as music — forget about the AI or the story and just get swept along,” Machover says. “There’s one melody in this piece that’s trying to express itself in the first movement, and appears in fragmented form in the second movement. In the third movement, it’s the center of everything, a kind of vision of resolve, hope, and purpose. If people come out of the performance and don’t have some part of that melody in their heads, I’d be surprised. I hope that audiences listen for that, try to hold on to it, and see where it might lead.”
The piece was commissioned by the Boston Symphony Orchestra and supported by the New Works Fund of the Massachusetts Cultural Council.