OpenAI Sora Hands-On: Disappointing and Impressive in Equal Measure
After months of anticipation, OpenAI finally launched Sora, its AI video generator tool. We have been waiting to access Sora and generate AI videos to see if it lives up to the hype. We have now tested Sora on a variety of prompts and used its image-to-video feature as well. Now, go through our hands-on experience of Sora to learn about its capabilities and limitations.
Generate a Video of a Person Blowing Out Candles
To test Sora, I started with a simple prompt to see how it handles air movement and general physics. I ran the below prompt on Sora, but it refused to generate a video. Turns out, due to the “blow” word in the prompt, Sora refused to generate the video. So I changed it to “extinguishes” and it readily produced the video. It shows that Sora has been heavily restricted to avoid harmful video generation.
Never mind, coming to the generated video, it failed to follow my instructions. The person in the video didn’t blow out the candles. After multiple tries and having burned over 100 credits, Sora couldn’t generate the video I asked for.
A person blows out all the candles on a birthday cake
Generate a Video of a Cricket Match
Next, I asked Sora to generate a video of a cricket match between India and Australia. This was a good test to check Sora’s understanding of physics. Well, it generated a plausible-looking video, but the movement was wonky. As you can see in the video below, the bat magically disappears while the player runs.
It demonstrates that while Sora understands spatial coherence well — creating consistent individual frames — it lacks temporal coherence, which is the ability to generate sequences that follow logical, time-based rules. For example, a ball might fall at inconsistent speeds because the model doesn’t understand gravity or more broadly, the laws of physics.
I noticed that when humans are involved, Sora struggles to maintain temporal coherence. It reminds me of the early image generation models that often failed at rendering human features such as fingers, hands, eyes, etc.
India playing cricket against Australia
Image to Video Generation
Since Sora allows users to upload photos (not of humans, as of now), I took this opportunity to test how it transforms static images into videos. I uploaded an image of a mountain pathway and asked Sora to generate a video based on the image. I added a prompt instructing Sora to pan around and show the surroundings.
Again, Sora failed to follow my instructions. It created a sort of live video from the image and didn’t show the surroundings at all.
So I tried a different method and used Sora’s Remix feature to create something new based on the generated video. I asked Sora to add a river around the mountain and changed the Remix strength to Mild. While Sora added a flowing river, it changed the overall scene and the output looked like a watercolor painting.
Create an Abstract Video
On Sora’s Featured page, you will find many visually stunning videos, mostly abstract in nature. So I tried to produce an abstract video of a playful fox in a forest. And Sora did a wonderful job. It created a magical scene, showcasing great detail in cinematic light. For such videos, Sora is a great tool. You can go wild with your imagination and create surreal videos. Make sure to add detailed prompts for better results.
Create an Animated Video
In the final test, Sora created a beautiful animated scene of a deer, running in a mystical forest. I loved the slow movement and this time, it was perfectly rendered without any distortion. Again, if you wish to generate animated scenes, Sora is a worthy AI video generator.
Since this is the first model released by OpenAI for video generation, some imperfections are to be expected. We know that it’s running the Sora Turbo model which is faster, but doesn’t offer the full capabilities of the larger model. Nevertheless, it’s a promising development in the field of Generative AI. From here onwards, we can expect that it’s only going to get better.
Source link
/cdn.vox-cdn.com/uploads/chorus_asset/file/25789051/killswitch_2_dbrand.jpg)
