“It was the first [Election Day simulation] that included artificial intelligence as a focus point,” Fontes, a Democrat, said in an interview. And unless you knew him or his staff well enough to ferret out the half-truths told in the clips, he said, the AI-powered fakes “wouldn’t necessarily be non-believable.” The fake of Fontes, for example, offered false but trivial facts about himself, like that he had a son who plays ice hockey.

The exercise in Arizona underscores how concerned election experts and public officials across the country have become about the rapid rise of generative artificial intelligence, which criminals, conspiracy theorists and U.S. adversaries are already using to perpetrate a range of scams. Fears are especially high in battleground states like Arizona, which Joe Biden won by just 10,000 votes four years ago.

Cait Conley, a senior adviser to the director of the Cybersecurity and Infrastructure Security Agency and one of the federal government’s top election officials, praised Arizona’s exercise as one of the “best ways” to ensure states are prepared for Election Day.

But she also acknowledged how much harder the fast-evolving tech could make things. “Heading into 2024, generative AI is likely to exacerbate many of the risks election officials already face,” she said.

What makes AI so troubling, Fontes said, is how it allows bad actors to use the very context clues people rely on to verify information from trusted sources — like the sound of a recognizable official’s voice — to fool them.

“It’s just kind of a faster, broader and deeper dissemination” of disinformation, he argued.

In March, the FTC warned that fraudsters were using AI voice clones to swindle Americans. A fake image of the Pentagon aflame temporarily tanked the stock market in May. And in September, Chinese actors used AI-generated images to spread the lie that American intelligence agencies were behind the Maui wildfires.

The mounting concerns around AI and disinformation have led to a new push on Capitol Hill to set safeguards for the technology before 2024.

At a Senate hearing in September, Sen. Amy Klobuchar (D-Minn.) urged lawmakers to approve bipartisan legislation she spearheaded to protect political candidates against targeted deep fakes — deceptive, AI-generated videos — arguing that the technology is already powerful enough to deceive voters.

“Given the stakes for our democracy, we cannot afford to wait,” she said at the hearing.

For now, though, state and local officials in battlegrounds like Arizona are largely trying to tackle the problem themselves.

The exercise in Arizona included AI-generated video of Fontes and another official, as well as audio of security staff in his office.

Fontes said he didn’t recall which software the state used to produce the voice fakes. But he said the video of him was trained on publicly available information, while that of his security expert was produced using special recording equipment. Fontes declined to go into further specifics on the fraudulent audio and video the state used due to concerns that bad actors could emulate them.

He said one of the main takeaways of his state’s two-day exercise is that AI doesn’t pose an entirely new threat to the vote, though he believes it could exacerbate old ones.

AI is “a new version of the same old threats: misinformation, disinformation, misleading information,” he said. But now, all of that is “much easier to generate and get out there.”

Arizona will organize two more simulations before the 2024 elections, Fontes said. And at least one other swing state, Michigan, plans to hold a similar election exercise using AI this spring, Secretary of State Jocelyn Benson told POLITICO previously.

CISA’s Conley also said the agency will help interested states hold no-cost exercises in the coming months and would welcome the opportunity to “bring similar opportunities” to states beyond Arizona and Michigan.

Steps like that could be key to staying ahead of AI. But Fontes stressed that it’s also on humans not to spin the inevitable Election Day hiccups out of proportion.

“We’ve had enough of the conspiracy theories. We’ve had enough of the mythologies. We’ve had enough of the, you know, storytelling nonsense from election deniers,” he said.

Read More