If you’ve taken even a brief glimpse at our news coverage over the past few months, you’re likely to have spotted an emerging trend: fans using AI to transform their own vocals, or those of an existing artist, into vocals that sound like famous names such as Drake, The Weeknd, and Oasis’ Liam Gallagher.
It’s a fascinating development, and one that has seismic implications for the music industry. Though only a few notable examples have caught the public’s attention thus far, thousands of fans have already used AI to reimagine popular songs with new voices.
Ever wanted to hear AI Paul McCartney sing Billy Joel’s Piano Man? How about AI Kanye West and AI Rihanna taking on Eminem and Dido’s Stan? Fancy hearing AI Michael Jackson covering Rick Astley, or AI Kurt Cobain doing Blur? Now you can.
So how does it work? The majority of people creating these covers are using open-source software called SoftVC VITS Singing Voice Conversion, or So-VITS-SVC, to process vocals. This is an AI-powered deep learning model that can be trained by using audio files of any vocal timbre to convert vocal recordings into the singing voice it’s been trained on.
It isn’t the only model available that can do this, but it’s currently the most popular. By ripping the vocal stems from an artist’s recordings and using these to train So-VITS-SVC, users can teach the software how to process any vocal recording to sound like the artist in question.
There are a number of ways to access So-VITS-SVC. The software can be downloaded and run locally, but that process is a little convoluted (some coding is required) and it requires a powerful GPU. If you’d like to use So-VITS-SVC on your machine, YouTuber p3tro has made a great tutorial for that.
There are easier alternatives, though. Currently, the quickest and easiest way to process a vocal is to pay a visit to AI World, a popular Discord server for those experimenting with AI vocals, and use their automated bot. AI World’s bot makes use of the same software, but the process is vastly simplified.
Before you process a vocal, you’ll need to make sure that you have a clean a capella vocal recording. This can be achieved through the use of many paid-for tools that lift vocals from full mixes (RipX is excellent) but there are free options available, too, like VocalRemover. Once you have your a capella file ready (in WAV, MP3, MOV or MP4 format) head over to the AI World server, open up the channel ‘ai-bot-1’ and drop your audio file into the chatbox.
Accompanying the file in your message should be the text ‘-model @JuiceAI1’, with the word ‘model’ replaced by the name of the voice you’d like to emulate. Currently, 18 voices are available, and the list includes Kanye West, Drake, Ariana Grande, Taylor Swift, Justin Bieber, Kendrick Lamar, Travis Scott among others. Select the channel ‘available-voices’ in the Discord sidebar to view the exact model names.
Once you’ve specified the model in your message and attached the file, hit enter and the bot will return your processed audio in the chat. After downloading your processed audio, all that’s left to do is lay this file over the top of the instrumental version of your chosen track in a DAW such as Logic Pro or Ableton Live. After adjusting the relative levels of each track to achieve a balanced mix, all that’s left to do is bounce the final product.
If you’d like to experiment with voices beyond the selection available in the AI World Discord, you have several options. You could download and install So-VITS-SVC, as previously mentioned, and make use of the growing number of voice models being created and shared by the online community. This list includes a variety of names from Chris Cornell to Chief Keef and is updated regularly.
If you’re running So-VITS-SVC locally, you can train it yourself and create new models based on the voice of any artist you like: here’s another tutorial that runs you through that process.
One way to access AI voice conversion without installing So-VITS-SVC on your own computer is by making use of several community-created Google Colab notebooks that will run the required software for you on Google’s cloud servers. These are a little unpredictable, but relatively easy to follow: follow the instructions and hit the grey ‘play’ buttons on the left-hand side in order to advance through the steps in the process.
Finally, there are several paid-for services that offer a more streamlined way to accomplish the same results. Uberduck can imitate a decent range of voices, including older artists like David Bowie, Brian Wilson, Marvin Gaye and Freddie Mercury, and contemporary vocalists such as Lady Gaga, Kim Petras and Grimes. The service also allows you to create your own models with custom datasets, meaning you’ll be able to replicate any voice you choose. You can test Uberduck out for free, but to clone a voice, you’re required to subscribe.
Voicify is another alternative that can create AI covers using models trained on Drake, Juice WRLD, Pop Smoke, Michael Jackson, Eminem and more. Voicify’s range of available voices is smaller, but their tiered subscription plan is cheaper than Uberduck’s, and ranges from $8.99 to $89.99 per month. Voicify is also capable of producing new models using any voice you like, providing you have the data available to train it.
Before we conclude, it’s our duty to remind you that the legal status of AI covers is murky at best. Ghostwriter977’s AI-generated Drake and The Weeknd song, Heart on My Sleeve, hit the headlines and began racking up millions of views and streams almost immediately after being uploaded. Just a day or so later, though, it disappeared from streaming services, before being taken down from YouTube and replaced with a notice that referenced a copyright claim from Universal.
Though the legal boundaries of this kind of AI-powered vocal imitation haven’t yet been ironed out, we recommend erring on the side of caution here. While we maintain that it’s emphatically not a good idea to attempt to monetize any AI-generated music that imitate the voice of an existing artist – by uploading them to Spotify or YouTube, for example – we see no harm in playing around with this technology at home for your own amusement.
If you want to find out more about AI vocal processing, the Discord servers AI World and AI Hub are the best place to start.