OpenAI has introduced Sora - a new video generation model capable of creating realistic video clips from simple text descriptions up to 1 minute long. This technology opens up unique possibilities for various industries, including entertainment, education, and marketing.

AI has been able to create videos before, but the generated results were often unsatisfactory, falling short of expectations in terms of both quality and realism. However, recent advancements in this field have been simply astounding, showcasing tremendous progress. Some videos are practically indistinguishable from real footage.

How does Sora work?

Sora utilizes advanced transformer architecture, similar to what is used in popular language models like GPT. Sora is specifically trained on a vast dataset of videos and textual prompts. This enables the model to understand the semantic connections between text and visual elements.

When Sora receives a text description, it breaks it down into a sequence of tokens. The model then uses its knowledge to create a sequence of images that correspond to the description. Additionally, Sora OpenAI has the ability to supplement existing videos with new frames. This means that videos can be expanded with additional scenes, objects or creatures can be inserted into the clips, and backgrounds can be changed.

Drawbacks

This model is not yet perfect. It may struggle with accurately modeling physics in complex scenes, such as fluid dynamics, collision animations, fabric simulation, hair, fire, etc. Additionally, Sora does not always understand causal relationships. For example, it may generate a person biting an apple but fail to show the bite mark on the apple itself. The model can also sometimes get confused with spatial details, such as left-right orientation or camera movement along a specific trajectory.

The Future of Sora

Currently, Sora is in the refinement stage. It is being tested by specialists from various industries for evaluation and feedback, but soon it will be available to ordinary users. As the model continues to learn and improve, we can expect it to generate even more realistic and captivating video clips.

Here you can watch videos created using Sora OpenAI.