Coronavirus

Japan's 'Scary Beauty' Android Opera is a Metaphor for Our Relationship with Tech: Interview

Keiichiro
Masanori Naruse

Takehito Masui, Keiichiro Shibuya and Takashi Ikegami.

In 2012, Keiichiro Shibuya composed and performed the world's first vocaloid opera, titled The End, featuring 3D animated pop star Hatsune Miku. The cutting-edge Japanese composer's next project is an opera conducted and sung by an android equipped with artificial intelligence.

Called Scary Beauty, the project features the "lifelike" humanoid robot "Alter 2" created by roboticist Hiroshi Ishiguro of Osaka University embedded with an artificial neural network developed by artificial life researcher Takashi Ikegami of the University of Tokyo.

Billboard Japan spoke with Shibuya, Ikegami, and Takehito Masui of Warner Music Japan about the innovative endeavor:

Tell us how the Scary Beauty project came about.

Keiichiro Shibuya: I performed The End at the Ch√Ętelet theater in Paris in 2013, and on the final day, Jean-Luc Choplin, who was the director at the time, called me aside and asked, "What do you want to do next?" What immediately came to mind was my reply: "an opera with androids."

Then, in 2015, we put on a performance at the Palais de Tokyo in Paris using Prof. Ishiguro's androids. But the androids' movements were monotonous, and people in the audience left in succession. Also, Prof. Ishiguro and I had been releasing many highly abstract works, but from around 2010, I began sensing that abstract works were losing their appeal to audiences.

Why is that?

Shibuya: I think it's because of the internet and social network. People became used to these two very concrete stimuli and had no more room to interpret abstract things. And "reality worship" took the world by storm and accelerated. I think those are the reasons why.

That's how you came up with the idea of combining androids, music and artificial life?

Ikegami: Because androids are very concrete existences, the concept of artificial life becomes much clearer. The android in this project moves while singing, and the movement isn't programmed in advance. When audio input signals enter the "nerve cells" of the android embedded with an artificial neural network, the network organizes itself to avoid that stimuli. There's also an unstable rhythm generator inside, so they connect overall to produce stable dynamics on top of unstable dynamics. 

Scary Beauty was first unveiled in Adelaide, Australia, in September 2017, ahead of its Japan premiere.

Masui: Yes, and when we saw the video of the performance, the android looked like it was programmed. So we spoke with the CEO of Crypton Future Media, the company that developed Hatsune Miku, to incorporate the technology of moving the virtual idol in real time with an orchestra into our project.

What are you planning for the Japan premiere?

Shibuya: In the Australia production, I conducted while playing the piano, and the android sang my composition along with a 10-piece chamber orchestra. In the Japan premiere, the android will conduct and sing while the orchestra and I play the music under its baton. The lyrics, melody and number of bars will be input into the android beforehand, but it has no concept of BPM, so the conducting motion will be generated in real time.

Ikegami: The basic premise of artificial life is that it can generate its own motion patterns. So this android will rapidly re-generate its motion patterns based on the external information coming from the environment, such as the sound of the orchestra or its own voice, and based on its internal states such as the thousands of artificial neural network. But if "conducting for an orchestra" requires the expression of an conductor's mental complexity and a long term experience, then androids are incapable of doing it. Our major task is how to construct an actual mind in the artificial brain and body.

If the android changes the conduction pattern each time, wouldn't the actual performance be completely different from rehearsals?

Shibuya: I think the performance will change greatly each time, and it's possible that the quality of the performance could be lost. But that's what we wanted to do. Music today only offers beauty and good feelings and is like a service business in that respect. But this is creative work. I think there's more meaning, especially today, in offering a piece that takes on something absolutely new, that contains an element of uncertainty.

We didn't want to create a human conductor in the first place. We're aiming to show that when people forcibly comply with an android that conducts according to its own logic and system, a world that couldn't be seen with a human conductor and performers emerges. Nobody has ever done this before, so it's a discovery for music as well.

According to a certain statistic, the average length of time a Slack user is on the platform is 10 hours a day. Our show could be seen as a metaphor for the transitional relationship between technology and humans in that people are becoming more and more enslaved by technology.


THE BILLBOARD BIZ
SUBSCRIBER EXPERIENCE

The Biz premium subscriber content has moved to Billboard.com/business.


To simplify subscriber access, we have temporarily disabled the password requirement.