We already know not to trust everything we see online, but it’s about to become even harder to believe your own eyes.
OpenAI, the company behind ChatGPT, has just unveiled its new text-to-video tool powered by artificial intelligence. It is outrageously good – but that might not necessarily be a good thing.
Named Sora, the program is not yet publicly available, but example videos shared by the company and its CEO Sam Altman are simply mind blowing.
Woolly mammoths parade towards the ‘camera’ leaving clouds of powdered snow in their wake.
Snowflakes drift from the sky as a couple walks hand-in-hand through Tokyo.
A ‘drone’ shot pans across the familiar white and blue buildings of Santorini at dusk.
At first glance, nothing is amiss and this could easily be really footage – extinct animals excluded. All have been generated from simple prompts, and the firm says they were not edited in any way.
On closer inspection however, most videos have a few telltale giveaways.
Look closely at a stunningly complex Wild West scene and you’ll spot a horse with only two front legs and no behind. Close by, a dark, monstrous creature emerges from the river. Overall, the effect is very Star Wars cantina.
A trio of wolf pups playing on a gravel track multiplies as they tumble around, new pups spontaneously popping up and then merging back into their siblings.
And just as generative AI has struggled with digits and limbs in the past, the same is true – possibly more so – in video format. A clapping hand grows and loses fingers. A woman lying in bed becomes at one with her duvet, which appears to morph into her arm.
Most people seem to walk as though they’ve suffered a minor soiling incident.
OpenAI itself knows it is not yet perfect.
‘The current model has weaknesses,’ it says on its website. ‘It may struggle with accurately simulating the physics of a complex scene, and may not understand specific instances of cause and effect.
‘For example, a person might take a bite out of a cookie, but afterward, the cookie may not have a bite mark.’
But this is first generation, and it is frighteningly good.
In a year with around 60 elections across the globe and concern growing daily around the dangers of deep fakes, the idea that creating such realistic videos will soon be in the hands of anyone with the internet will exacerbate fears of voter manipulation.
OpenAI has pledged it is taking safety seriously, aiming to ensure AI-generated content is flagged, and to stop it being used for malicious purposes.
But given other similar software has already been used to create explicit video and to spoof the US president, stopping people using AI to upset and disrupt is easier said than done.
‘We’ll be taking several important safety steps ahead of making Sora available in OpenAI’s products,’ the company said. ‘We are working with red teamers – domain experts in areas like misinformation, hateful content, and bias – who will be adversarially testing the model.
‘We’re also building tools to help detect misleading content such as a detection classifier that can tell when a video was generated by Sora.’
However, despite their assurances, many are concerned by this gigantic leap forward in ability.
Rachel Tobac, a technical advisor to the US government, said her biggest concern was ‘how this content could be used to trick, manipulate, phish, and confuse the general public’.
Writing on X, she said: ‘AI text-to-video is here and we need to discuss the risks.
‘They mention in this thread that they’re considering the ways adversaries would leverage this content to harm thru [sic] red teaming but I’m still concerned.
‘This tool is going to be massively challenging to test and control under many, let alone most, adversarial conditions.’
Sharing examples of how it could be used to manipulate viewers, she suggested an adversary could use the tool to show a vaccine side-effect that doesn’t exist, or long lines in bad weather to convince people not to bother voting.
The new tool comes just days after AI expert Dr Roman Yampolskiy warned that there was ‘no evidence’ that AI can be controlled – and so should not be developed.
MORE : Artificial intelligence was taught to go rogue for a test. It couldn’t be stopped
MORE : AI Biden tells people not to vote on election day