The Markup, now a part of CalMatters, uses investigative reporting, data analysis, and software engineering to challenge technology to serve the public good. Sign up for Klaxon, a newsletter that delivers our stories and tools directly to your inbox.

Bird singing and dancing, as practiced by the Cahuilla Band of Indians, tell a story about the creation of the world, and how the Cahuilla migrated to their current home in Southern California. Moving the same way your ancestors did, perhaps on the exact same land, makes you feel part of the past, present, and future all at once, said tribal member Emily Clarke. She’s done bird dance with her loved ones since she was 7 years old—an act, she said, not only of spirituality but also of perseverance, since bird dance is among the acts of Native American culture nearly eradicated by colonization and U.S. government policy.

So when Clarke heard that some generative artificial intelligence models, like Google’s Veo 3 and OpenAI’s Sora 2, can mimic the dance, her first thought was that it was wrong, distasteful and disrespectful. Then she wondered briefly if automated forms of bird dance could help preserve her culture—before deciding they can’t, since they will never replicate the conversations and community bonds that have helped give Cahuilla bird dance its distinct style and impassioned practitioners.

“It would miss the cultural and social importance, and without that, it’s not bird dancing,” she said.

Developers of AI systems are working continuously to do a better and better job of replicating complex human movement, including dance. Doing so has become a sort of Holy Grail in the field of generative AI due to the many technical challenges involved, but it remains an open question among dancers like Clarke of how much the technology will disrupt the world of dance as it progresses.

Clarke’s conflicting and uncertain thoughts about AI mirror those of other dancers across California interviewed by The Markup, who were, depending on the specific question or moment in time, optimistic, skeptical and concerned about AI’s incursion into their art form. Most settled into the view that AI is incapable of capturing the uniquely human aspects of dancing, including the cultures surrounding it, the pride and passion of dancers, the energy imparted by audiences, or the form’s essential element of improvisation.

Caption: From left, sisters Lily and Emily Clarke during a bird dance event at the Idyllwild Arts Academy in Idyllwild. The Clarke sisters belong to the Mountain Cahuilla tribe. Credit:Cierra Breeze Photography

The Markup and CalMatters tested four commercially-available AI video-generation models — OpenAI’s Sora, Google’s Veo, MiniMax’s Hailou, and Kuaishou’s Kling — and so far, dancers don’t have much to worry about. Late last year, we asked the four models to depict humans performing nine different dances, from the Apple dance popularized on TikTok to bird dance, folklorico and the Mashed Potato. All but one of our 36 tests returned a video of a human dancing, but all the videos failed to produce a video of a person performing the specific dance requested. Furthermore, a little under a third of the videos included many of the common issues seen in generative video, like inconsistencies in a subject’s appearance from frame to frame, abnormalities in movement, or too many limbs.

Video generation models still make obvious mistakes like liquefying arms and legs or sudden changes in clothing, but they are clearly improving. When we ran similar tests in late 2024, videos had more impossible limb movements and visual inconsistencies, failings that appear less frequently and are more difficult to spot today.

Clarke said of the AI-generated videos depicting Cahuilla Band of Indians bird dance: “None of these depictions are anywhere close to bird dancing, in my opinion. The regalia is only similar in that there are skirts with ribbons, but the songs and dance movements are completely off.”

Bird dance videos generated using, clockwise from top-left, OpenAI’s Sora 2, Google’s Veo 3.1, Kuaishou’s Kling 2.5 and MiniMax’s Hailou 2.3.

The Markup

Generative AI, technology that makes it possible to quickly generate audio, text, imagery or video with a simple text prompt, raises issues beyond the dance community. It’s often made with data scraped from the internet, meaning that creatives who share their work online risk having it used to train AI models.

Understanding how AI might affect the world of dance has also taken on new urgency as AI begins to influence the art and careers of creatives in adjacent fields and slop begins to crowd out images of real people on social media.

A nearly five-month strike in 2023 by Hollywood writers was prompted in part by concerns around the use of artificial intelligence in entertainment. An actors strike also addressed the effects of AI on performers’ work. Lawmakers are trying to sort out the policy implications; the California Legislature last year passed laws sponsored by the Screen Actors Guild to protect actors from getting turned into digital replicas without their consent. Next year, lawmakers will consider passage of a bill supported by Hollywood creatives which places digital fingerprints on copyright material so individuals can ask tech companies if their work was used to train their generative AI models and give them grounds to demand compensation or credit.

Most dancers are not unionized, although some are part of SAG-AFTRA, along with others who earn a living by moving their bodies, like stunt performers and martial artists. Within this group, video game motion actors are at the vanguard; they went on strike last year alongside voice actors as they sought protections from AI.

↩︎ link

Hip hop, folklorico, and the limits of tech

Edye Kelly, 23, first started dancing in a Bay Area hip-hop studio in 2016. She moved down to the Los Angeles area a couple of years ago to pursue a career in dance. She has been in a Daddy Yankee music video, and has performed at the Grammys as one of the dancers for R&B singer-songwriter Sza. She completed her first tour as a backup dancer with multi-platinum pop entertainer Usher.

Kelly heard creatives talk about how AI could affect their work during the SAG-AFTRA actors’ strike, but she doesn’t think many dancers know about the possibility that the technology could have a big impact on their livelihoods.

She said full-body motion technology is already beginning to encroach on human dancers’ territory: “It kind of already has happened, when you think of video games and how they use dancers for ‘Just Dance’ or ‘Michael Jackson: The Experience.’ ” Those video games use a combination of real dancers’ movements and motion technology, according to dancers who have talked online about their work for the games. Ubisoft, the maker of both games, on its website touts its use of AI “across the board” to “create believable worlds” and has posted to YouTube a video showing its capture process in action. A spokesperson for the company declined to comment on its use of software to simulate or augment dance.

Kelly, who earns money by dancing and teaching dance classes, is concerned that AI could eventually threaten dancers’ jobs and said she hopes dancers are fairly compensated if their likenesses or moves are used to train AI models.

But she struck an optimistic note, too, saying, “I don’t think AI could ever get to a point of perfectly mimicking humans … We are constantly evolving. They would always be behind in some kind of way.”

Nadia Arambula is an introvert, but that all melts away when she’s dancing. Arambula, 46, started folklorico dancing as a toddler in Guadalajara. Folklorico dance styles and clothing vary by state or region in Mexico, and as a child Arambula embraced the fashions of the Jalisco region by playing dress up in her mom’s colorful skirt and heels. When she was 5, her mom signed her up for dance classes at her daycare, and she’s been at it ever since. Today Arambula dances at events in San Diego at places such as Padres baseball games and community gatherings with her students at Ballet Folklórico El Tapatío.

Jalisco folklorico videos generated using, clockwise from top-left, OpenAI’s Sora 2, Google’s Veo 3.1, Kuaishou’s Kling 2.5 and MiniMax’s Hailou 2.3.

The Markup

Arambuila said she has danced nearly her entire life because she enjoys how the elegant, flowing drawing movements with skirts projects pride in Mexican culture and manifests joy and a sense of freedom. She also loves teaching insecure young people to get up on a stage and live out loud—to feel the sort of joy she feels when she dances.

Arambula believes it’s impossible for AI to mimic the emotion of a dancer, especially when it comes to interacting with the audience. The tech, she said, will never be able to transmit the grace, joy and emotions she feels when she dances.

“When I perform, I can feel the audience,” she said. “I give life to the people, and at the same time receive life from the people. It’s beautiful. There are no words that describe the feeling that I have when I perform and seeing the audience wants to be part of this moment.”

Shown some of Jalisco folklorico AI dance videos generated by The Markup and CalMatters, Arambula said they don’t come close to representing the Jalisco dance style she grew up with and teaches today. There’s no proper footwork, posture or skirt movement — all essential elements of the style. In some instances, AI makes a head turn without shoulders following or make part of a dress disappear for a moment, but she said she has to stop to catch the imperfections. She may not notice them if she scrolled past these videos in her social media feed without an indication they were made by AI.

Computer scientists have for years made it a grand challenge that their AI models could generate convincing dance videos on demand from textual descriptions — not unlike similar dreams of developing AI that can drive a car next to humans on city streets or one that can detect emergencies before any humans dial 9-1-1. Videos from YouTube and TikTok provided them with ample training data, helping AI systems generate movement without needing sensor-laden humans to literally dance for them, a process known as “motion capture” that is currently required for most video games and augmented reality worlds. (YouTube now prohibits the use of videos it hosts for training models unless users opt in. TikTok did not respond to repeated requests for comment.)

Researchers at UC Berkeley made important strides in 2018, when they used YouTube videos to train AI to do acrobatics like backflips, martial arts, and, perhaps most importantly, the Gangnam Style dance. Subsequent work by Berkeley researchers in 2019 used video of a ballerina on YouTube as training data. In 2022, a UCLA researcher, along with their collaborators, introduced Bailando, a generative dance AI model. The same year, Stanford researchers introduced EDGE, which can generate lengthy state-of-the-art breakdancing video based on only five seconds of music and video. In 2024, researchers from the University of Southern California and TikTok parent company Bytedance introduced AI models that imitate facial expressions and dance moves.

Making videos with AI has become even easier with the advent of generative AI models from companies like OpenAI, Midjourney and Runway.

Despite this progress, AI models still struggle to create natural movement. For example, it’s easy to make a video of a fireplace ablaze or a Hollywood explosion, said UC Berkeley computer science professor James O’Brien, but it’s hard to make a video of a wildfire impacted by strong winds. This limitation means AI models may do a poor job at imitating some dance moves today, but O’Brien said it’s important to appreciate that when AI developers fix their sights on a challenge, they can make vast improvement within a matter of months. Two years ago, for example, text-to-video models were incapable of producing videos of Will Smith eating spaghetti (which, like dance, is another defacto benchmark for AI models). Today such videos are not perfect, but they’re much more lifelike than they used to be.

Computer scientists are interested in being able to generate realistic videos of dance because the capability is a gateway to creating a range of photorealistic human movement, said Yuanhao Zhai, a recent Ph.D. graduate in computer vision from State University of New York at Buffalo. If you can generate video of a gymnast doing a backflip, for example, it will be easier to produce footage of a person walking down the block.

↩︎ link

Feeling dance moves under your skin

A ballerina figurine atop a music box is what inspired Emma Andre, 26, to dance. She was 3 years old and mesmerized by the miniature as it spun around a tiny piano, bent in an arabesque. She told her parents she wanted to go to dance class. A creative movement class led to lessons in ballet and contemporary dance, attendance at the Boston Conservatory, and training with dance companies in New York City. Today Andre choreographs dance performances and teaches dance to kids in Berkeley in the same kind of classes where she got her start.

Lately, she’s enjoyed movement in dance shows likeMy Obsession with Hamletmachine and theYerba Buena Garden ChoreoFest in San Francisco. She also enjoys working with her partner and fellow dancer Henry Winslow on a duet that she calls their dance theater idea farm. Last year they performed a three-part duet in front of an 11-person audience in their one-room apartment in Oakland, a feat that required some modifications to ensure Winslow didn’t stick a leg out a window or upset downstairs neighbors.

Henry Winslow, left, and Emma Andre rehearse a dance routine in their home in Oakland on Oct. 1, 2024. The pair, who are both life partners and dance partners, do not currently incorporate artificial intelligence into their practice but are open to the changes it could bring. Photos by Florence Middleton for The Markup

Credit:

Andre said she’s open to exploring the use of AI in dance, but believes that it is a uniquely human form of movement, incorporating elements that cannot be imitated by AI. Even if AI got every step right, it cannot capture the lessons and experiences that sometimes determine the next step for human dancers. A pirouette, a spin on one leg, is different every single time, she said.

“I do a lot of improvisation for my choreography, so I don’t even know what I’m gonna make. And so it’s really crazy to me that an AI could predict that?” she asked. “We talk a lot about fascia, which is the layer of connective tissue under your skin that goes throughout your entire body. And I feel like a lot of my movement stems from a fascia level. You can’t map that externally.”

What drives her to continue to dance is the joy of movement and connection, of meeting people and building relationships through dance. Ephemeral, beautiful, intimate moments of movement make dance what it is, she said. She’s also motivated to pass on that passion to young people — seeing them discover and feel creative and confident the same way she learned when she was a child fuels her interest in exploration.

“I have a choice, but I will always choose to dance,” she said.

Andre said she found a depiction of the Horton modern dance style generated using the Veo 3 model “staggeringly lifelike,” but she caught moments in a majority of the generated videos where something is“not quite right,” like when a dancer’s head is momentarily on backwards in one video, or when a ballet dancer bounces on to their tippy toes, but they’re wearing soft shoes, and you would only do that in hard toe ballet slippers.

Horton-style dance videos generated using, clockwise from top-left, OpenAI’s Sora 2, Google’s Veo 3.1, Kuaishou’s Kling 2.5 and MiniMax’s Hailou 2.3.

The Markup

“It’s really confusing to me why anyone would prefer this instead of hiring one of the thousands of really capable, really talented dancers in most major U.S. cities,” she said. “The whole point of dance is connecting with the human form.”

Winslow enjoys how dance connects him with Andre and other people in order to create something meaningful together, and delights in feeling the history of movement styles developed over generations that traveled across continents.

Sometimes a dance step is new and sometimes it’s something in the zeitgeist or you didn’t know you picked up from theater camp as a teenager. The majority of the time it isn’t new but something you learned over time, something programmed into a dancer’s nervous system over the span of days or decades, or mirror neurons that fire in the brains of experienced dancers when they watch other dancers perform. He questions how AI can imitate those human processes.

“Because we’ve been doing this for a while, there’s just a lot under the surface,” he said.

Andre said she stopped sharing her dance on Instagram in order to take a mental health break from the unhealthy dopamine dependency such platforms can enable. Posting dance videos on Instagram has helped Andre find jobs, but she said that if generative AI started imitating her favorite dance styles she would be less likely to post videos online in the future because she doesn’t like the idea that her body and movements can be copied and used by an AI model.

And she’s worried about constricting her posting that way because she believes dance is meant to be shared. If fewer dancers share their videos it can reduce learning and access. She has a contemporary dance degree from Boston Conservatory but learned how to do fouetté turns from a YouTube video and has drawn inspiration for how to choreograph dance performances from other videos posted on social media.

“My body and my dancing is mine, and the idea that that can just be siphoned through this process and then become part of AI without my consent is something that I don’t love the idea of,” she said.

Conversely, Arambula said she’s fine with the makers of AI scraping video of her performances because she’s confident that a machine cannot imitate the way she connects with an audience.

Emily Clarke said that if she found out her videos were part of data used to train video-generating AI, she too would stop publicly posting videos on social media, but she’s not worried about that negatively impacting community connection since it’s more common to learn bird dancing from an elder or another tribe member than it is to learn on social media.

Zion Harris is 25 years old and has been dancing since he was 5. He was 12 when he made it onto the Golden State Warriors’ Junior Jam dance team as a part-time employee of the NBA franchise, which is when he said he seriously began to think of making a living as a dancer.

Caption: Zion Harris, who recently danced on Bad Bunny’s Most Wanted Tour, at Heart WeHo in West Hollywood in Los Angels on Thursday, Sept. 19, 2024. Credit:Alisha Jucevic for The Markup

Now he’s doing it. Harris, who grew up in the Bay Area, has been on tour as a backup dancer with Puerto Rican rappers Bad Bunny and Daddy Yankee, getting to explore North and South America in the process. He has performed at the Grammys with country/hip-hop crossover star Lil Nas X. He’s been in a Coach commercial with Houston rapper Megan Thee Stallion.

Dance has helped him get through his “hardest times,” Harris said. “It blows my mind every day that I can do this.” His main sources of income are tours, music videos and stage performances. When there are human dancers at a concert, “it definitely helps with the crowd… and brings the artist’s vision alive,” he said. “Having a hologram, or digital images on a screen, would be boring.”

There are times when things get slow for Harris. But when he’s not on a job, he is training — taking dance classes or holding practice sessions with his fellow dancers.

Harris is signed with a casting agency that helps him book jobs, but social media plays a huge role in getting him noticed and sometimes helps him find opportunities directly. So the possibility of his moves and likeness being copied off social media platforms by AI is “definitely a big worry,” he said.

“Everything is becoming easy to replicate or use without consent,” Harris said. “But if we don’t post [on social media], our opportunities could stop. You really just have to take the gamble.”

↩︎ link

Dancers and educators experimenting with AI

Though some dancers are clearly concerned about having their work exploited to build artificial intelligence models, others are experimenting with incorporating AI into their craft. Performers associated with the dance tech nonprofit Kinetech have used AI in a number of ways since the organization formed in 2013, including in dance performances with robotic arms, or AI systems that use motion capture to mimic the steps of a live performer.

Regular contributors to Kinetech shows run the gamut from a computer scientist so stiff it looks like they can’t move their shoulders to dancers with little to no experience in technology, said Daiane Lopes de Silva, a dancer who cofounded Kinetech alongside technologist and director Weidong Yang.

Caption: Olesya Elfimova performs during DanceHack 2024 at Mills College at Northeastern University in Oakland on Sept. 29, 2024. Elfimova, along with other dancers and technologists, used technology such as AI to create performance pieces during a weekend-long workshop. Credit:Juliana Yamada for The Markup

Members of the group come together in order to use technology as a tool to highlight humanity, Wang said.

That play and experimentation takes place at a weekly “open lab” gathering in San Francisco’s Mission District. Amid exercises at an open lab last year ahead of the group’s annual DanceHack show, Lopes de Silva led warmups for about 30 dancers and technologists with snaps, claps, shoulder rolls, and an order to “make magic with your hands like an alchemist.”

Lopes da Silva thinks motion capture technology can help dancers, capturing their unique style of movement and serving as a tool for self reflection. She recently used motion capture to discover that she dances more enthusiastically on her right side than her left.

The best part about humans making art with machines, Yang told dancers and technologists at an open lab ahead of DanceHack, is when things go wrong.

“Artists like glitches,” he said. “That can lead to improvisation, unexpected events that cannot be reproduced.”

Ari Kalinowski runs a lab at Gray Area, a nonprofit arts organization known for putting human dancers on the same stage as robots. He supports lawsuits against companies like Stability AI and Midjourney, which artists accuse of copyright infringement. Neither company responded to requests to comment. At the same time, he also has been working with Kinetech on AI that can mimic and interact with dancers and wants dancers to explore dancing with AI agents.

Caption: Technologist and Delta_Ark Creative Director Ari Kalinowski at DanceHack 2024 at Mills College at Northeastern University in Oakland on Sept. 29, 2024. Credit:Juliana Yamada for The Markup

The art form can withstand integration with AI, he believes.

“People still play chess after Deep Blue beat Gary Kasparov,” he said. “We now have in addition to chess, augmented chess, and as an artist I’m not interested in replacing artists. I’m interested in expanding creativity to include human-agent interaction in different ways.”

Kate Ladenheim is an assistant professor of choreography at UCLA and self-described creative technologist. Ladenheim was once asked to choreograph a ballet about Anonymous, the hacker group, which “pointed me toward digital life as a really strong influence for my work,” Ladenheim said.

Ladenheim uses motion capture extensively and has explored the intersection of dance and technology in various other ways, including by teaching a class about modern dance as it relates to tech development, and working with a member of the Martha Graham Dance Company on a chatbot that responds in the voice of Graham, the famous dancer and choreographer.

Ladenheim said the impact of technology on dance is nothing new, and that fears about technology’s effect on labor have been around since the Industrial Revolution. Generative systems that are producing dance animations aren’t very good yet, in Ladenheim’s opinion. Still, they acknowledged that AI has the potential to get so adept that it brings up an “essential question for the field of choreography: What is the implication if we can (subtract the human) body from it?”

About half an hour away at the University of Southern California’s Glorya Kaufman School of Dance, associate professor d. Sabela grimes is facing AI head-on instead of fearing it. Many dancers have been early tech adopters, he said, embracing sampler and drum machines, and the internet as a way to share music files.

The professor encourages his dance students to use ChatGPT to help formulate questions. He also believes generative AI can help students in their choreography classes and to help teachers engage students. AI models are “going to be integrated in so many systems, so why not get ahead of it,” he said. Yet he added that there’s an important conversation to be had around “the ethics of plagiarizing people’s styles and using it for datasets.”

Should dancers be worried about AI replacing them? It depends on what audiences and artists want, grimes said: “If people value a virtual experience vs. a human experience, there will be a danger.”

Kat Lin is a technologist who has danced since she was 3 years old and taken part in Kinetech performances since 2018. She’s never seen Kinetech do shows that didn’t involve AI in some way. Though AI or metaphors about AI are part of many Kinetech dance performances, Lin doubts that AI can easily replicate their shows because they’re often improvised and technological glitches are treated like opportunities to explore the unexpected.

First: Olesya Elfimova, right, performs with Haley Rowland, left, during DanceHack 2024 at Mills College at Northeastern University in Oakland on Sept. 29, 2024. Last: Dancers perform “You Are Beautiful” based on the interactive AI system “Move Me” during DanceHack 2024 at Mills College at Northeastern University in Oakland on Sept. 29, 2024. Dancers and technologists gave the AI system choreography, which translated into spoken prompts during the performance. Photos by Juliana Yamada for The Markup

Credit:

There’s a lot of fearmongering today about AI taking jobs from creatives, Lin said, but she doesn’t view the relationship between AI and art as antagonistic. Humans might take inspiration from dancing with AI or learn things, like that their dance steps land heavier on their right side than their left. But Lin doesn’t believe AI is a threat to dancers’ jobs. “There’s always going to be something different about having a human stand in front of you and moving,” she said. “Nothing will ever replace that kind of energy, probably because humans are very social, and there’s just something magical to that.”

Lin said AI can’t imitate the community she enjoys at Kinetech open labs, which she has participated in since moving to San Francisco seven years ago. And it can’t mimic the connection she felt with Weidong Yang, who she met while doing a handstand. He joined her there, upside down with his legs against the wall, and she now considers him a great friend.

“I often say that getting to do dance pieces or be in rehearsal is just an excuse to hang out with friends,” she said. “Community is everything.”