“We present a method to automatically generate music from literature.”
Let’s give that a second to sink in. Ready? OK.
Two technology-rooted artists (or, if you prefer, art-focused technologists) have devised a method for analyzing the emotional content and general profile of literature from “Alice in Wonderland” to “Clockwork Orange,” using that information to generate music designed to mirror the piece’s “general vibe.” You may disagree, but we think the result of a computer reading “Alice” brings up sepia-toned memories of shrinking potions and magical, mercurial cats:
Hannah Davis, a freelance artist and programmer, and Saif Mohammad, a research officer for the National Research Council of Canada, collaborated on TransProse, their unclassifiable technology — or new art form, depending on your comfort level with computers’ aesthetic capabilities — that David presented in beta form as her master’s thesis for New York University’s Interactive Telecommunications Program.
The project began with a different skeleton by Davis, who created a program that would create compositions from the grammar and structure of literary works. “You could hear the difference between Hemingway and David Foster Wallace,” she told us. But she soon moved on to the much more difficult task of trying to process the emotional content of a piece of literature, which presented immediate problems.
“I was trying a couple different things around sentiment analysis,” says Davis, who was “Really obvious — the list for ‘joy’ would include ‘joy’ and ‘smile.’ It was terrible.”
Davis had hit a wall, working with a set of “emotion words,” largely created by programmers, to scan texts for their emotional arcs. Eventually, her friend turned her towards the work of Mohammad, who created a lexicon of emotion words through crowdsourcing strangers for their emotional opinions on certain words. “There was a lot of work on sentiment analysis [taking place at the NRC], but I thought to work on emotional analysis, beyond just positive-negative, into joy and sadness, fear, anxiety.” The end result was the NRC Emotion Lexicon. At the discovery of Mohammad’s work, TransProse began to take shape. “I emailed him right away,” says Davis. “It automatically worked.”
With a reliable database in place, Davis could branch out into the nitty-grit of scanning texts and incorporating standard musical theory into her mechanized transliterations.
The practically conscious computer program eventually dug its roots deep into the history of human expression; “Grimm’s Fairy Tales,” Brian Eno’s work in generative ambience, the foundations of comparative literature, jazz. And a lot of deep computer theory on the relations between human emotion and computer’s understanding of it, culled from essays with names like “Proceedings of the 3rd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis.”
Related
“The National Research Center said that it’s possibly a new field,” Davis told us. “I haven’t found anything [like it]. When I did my thesis, you’re supposed to do outside research… I ended up having to look into any [examples of] turning one art form into another art form.”
In the future the pair of enterprising polymaths hope to profile portions of works that aren’t directly emotional but motif-based: think the dry sands of Arrakis in Frank Herbert’s “Dune.” “Here we’ve just explored emotions,” Mohammad said, “but it might be interesting to capture three ‘key things’ and have melodies for those three dimensions — whatever is most relevant for that novel. In ‘A Clockwork Orange’ there is a sense of deep helplessness, the passivity of not being able to do things. If we can capture that, it would be really interesting.”
The implications are daunting, the applications wide-ranging. “The more people see this, the more creatively they can use it,” Mohammad says. “We’ve thought about music for movie scripts or screenplays — we’ve had people request that. Another is audio-visual apps for ebooks — music playing into the text that is being read. Or going to an online bookstore… imagine you could press a button that plays you the emotional tone of the book.”
Many may feel uncomfortable with a computer expressing the emotions of our greatest works to us in our most sacred, ephemeral art form. It’s an understandable reaction. But, as the last twenty years have shown us, the future — of art, of technology — won’t belong to the close-minded. It never has.
You can find out more about TransProse at the project’s website. Learn more about Hannah Davis’ work here, and Saif Mohammad’s right here.