Holly Herndon Releases AI Deepfake Tool That Lets Others Make Music With Her Voice
Holly Herndon has released a new artificial intelligence tool — which the composer is also referring to as her “digital twin” — called “Holly+” that allows users to upload any polyphonic audio and receive a new version of that music sung in Herndon’s own voice.
Herndon has been working extensively at the intersection of music and AI technology over the past several years (she previously earned a Ph.D. at Stanford’s Center for Computer Research in Music and Acoustics). In 2019, she released an album, Proto, which she made with the help of a neural network she developed named Spawn. While Herndon “grew” Spawn by exposing it to her own voice, the neural network ultimately created its own sounds instead of mimicking Herndon’s voice like Holly+.
Herndon released and developed Holly+ in collaboration with Never Before Heard Sounds. It’s as much a technological and artistic experiment, as it is a response to, and embrace of, the rise of deepfake technology. In a blog post, Herndon noted the spread of celebrity vocal deep fakes on YouTube, and predicted that “generating convincing spoken and sung voices will soon become standard practice for artists and other creatives.”
“Vocal deepfakes are here to stay,” Herndon added in a statement. “A balance needs to be found between protecting artists and encouraging people to experiment with a new and exciting technology. That is why we are running this experiment in communal voice ownership. The voice is inherently communal, learned through mimesis and language, and interpreted through individuals. In stepping in front of a complicated issue, we think we have found a way to allow people to perform through my voice, reduce confusion by establishing official approval, and invite everyone to benefit from the proceeds generated from its use.”
To tackle the complicated of issue of rights ownership, Herndon has established a Decentralized Autonomous Organization for Holly+. The Holly+ DAO will comprise a select group who received unique tokens from Herndon, with tokens essentially representing voting shares (Herndon plans to give the tokens to “collectors of my art, friends and family of the project, and other artists”). The members of the Holly+ DAO will vote on things like approving official usage of Holly+ and new tool creation; profits made will be split among DAO members and also go toward funding future projects and other expenses.
The vocal deepfake tool is the first of several projects Herndon and Never Before Heard Sounds have in development. “Further tools are in development to allow for spoken and sung phrases, and image generation, to be released later this year,” Herndon said.
