OpenAI presents Jukebox, a new AI model that generates genre-specific music with lyrics

The OpenAI artificial intelligence research lab today unveiled a new generative model that can make music called Jukebox. It’s technologically impressive, even if the results sound like soft versions of songs that may be familiar to you. According to OpenAI blog post, researchers decided to work on music why It’s hard. And even if they are not exactly what I would call music, the results the researchers obtained were impressive; There are recognizable chords and melodies and words (sometimes).

The way OpenAI did it was also fascinating. They used raw audio to train the model, who spits out raw audio in return, instead of using “symbolic music” like player pianos, because symbolic music doesn’t include voices. To get their results, the researchers first used convolutional neural networks to encode and compress raw audio, and then used what they call a transformer to generate new compressed audio that was then sampled to convert it back to raw audio. Have a board!

The approach is similar to how OpenAI developed a previous music-making AI called MuseNet, but Jukebox goes a step further by generating its own lyrics in collaboration (the company used the word “co-written”) with OpenAI researchers. Unlike MuseNet, which used MIDI data, these models were trained on a raw data set of 1.2 million songs (600,000 in English) and used metadata and lyrics extracted from LyricWiki. (Data on artists and genres were included to improve the production of the model). Still, the researchers write, there are limitations.

“While Jukebox represents a step forward in musical quality, coherence, length of audio sample, and the ability to condition artist, genre, and lyrics, there is a significant gap between these generations and human-made music “they write. “For example, while the songs generated show local musical coherence, follow traditional chord patterns, and may even feature impressive solos, we don’t hear larger familiar musical structures like repeating choruses.”

There are also other problems with the experiment. As writer and podcaster Cherie Hu pointed out on Twitter, Jukebox is potentially a copyright mess. (It’s worth noting that just this week, Jay-Z attempted to use copyright attacks to remove synthesized audio of himself from YouTube.)

All that said, Jukebox is a pretty fascinating achievement that pushes the limits of what’s possible. Even if the OpenAI musicians showed Jukebox that they thought it needed some work. Go listen for yourself!