On September 1, Fei-Fei Li's World Labs released Atlas.
There is a set of images on the release page that indeed look good. Several phones are mounted on tripods and fixtures, shooting the same scene from three to five angles, and then time stands still, the camera moves, circling around the characters, going to positions where real cameras have never been.
It's not unusual to see this technique, known as "bullet time," these days. What's different is that Atlas has also released three other things at the same time.
The first is depth, which involves assigning a distance value to each pixel in an image to indicate how far it is from the camera, effectively turning an ordinary photo into a terrain map colored by distance.
The second is the point cloud, where millions of points are scattered in space, each point remembering its own three-dimensional coordinates, and when combined, they form the shape of this room.
The third is 3D Gaussian Splat, which does not use triangular faces to build models, but instead uses a large number of small light spots with color, transparency, orientation, and thickness to render the entire scene, and can be re-rendered from any angle.
What these three things have in common is that they're measurable. The diameter of a cup, the distance from a table to a wall—all can be read directly as numerical values. The generated output is no longer just an image meant for human eyes; it can be fed into game engines, design software, or the control code for a robotic arm.
World Labs simply calls Atlas the World Simulator.
They don't use the term simulator lightly. Three months ago, they wrote an article themselves that specifically defined what kind of system deserves to be called by that name.
Visible and Tangible
The article, published in June this year, discusses what constitutes a world model, and concludes by categorizing existing systems into three levels: renderers, simulators, and planners.
The origin of this classification method dates back to 1943, when British psychologist Kenneth Craik proposed that humans can anticipate how an event will unfold because the brain has a miniature version of the real world. This idea was later adopted by control theory, neural networks, and reinforcement learning, and eventually evolved into three key terms: state, observation, and action.
Take a kitchen, for example.
The position of the cup on the table, the distance from the edge of the table in centimeters, the distance from the table to the wall, and whether the door is open or closed, all combined, constitute the current state of this kitchen.
A person standing at the door can only see part of the scene. The floor blocked by the refrigerator is out of his sight, and he has no idea how much water is left in the cup. The information he takes in through his eyes is called an observation.
He walked over and pushed the cup away, which was an action.
State, observation, and action - the number of these aspects that a system can manage determines its level.
The renderer draws the screen. When a person takes a step forward, it calculates what should be seen at that step, renders it realistically and seamlessly connects it to the previous frame, and the task is done. It doesn't need to know how tall the cup is or how far the table is from the wall, because all it outputs is an image.
It maintains the state of the simulator, storing numbers such as the position of a cup, the size of a table, and how far a door can be opened before hitting a wall, allowing these values to be retrieved at any time. Because it stores these numbers, other programs can be integrated, and a mechanical arm can be controlled to move around in this virtual kitchen.
The planner takes action. It not only knows that the cup is on the table and what will happen if it is knocked over, but also needs to determine whether to reach for it now, which one to pick up first, and from what angle to grasp it most stably.
Google DeepMind's Genie 3 is classified as the lowest level, merely a renderer, along with World Labs' own RTFM, which was released in October 2025.
Prior to this, the term "world model" was a catch-all phrase. Input a sentence, generate a walkable scene, and it's a world model. Change the official website slogan to use AI to simulate the world, and it's still a world model. This article draws a line for the first time, separating images from the world, and the virtual from reality.
In the past year, most demonstrations have stopped at the first level. Compared to purely text-generated videos, they have made significant progress, and the scene changes when you press the direction key.
However, what the renderer outputs is sufficient for human viewing but not usable by machines.
Atlas needs to be on the same level as a simulator. By their own definition, a simulator must maintain the entire state of the world, including physics.
Current tests focus on camera-condition generation and 3D reconstruction, and complex materials, fluids, and fabrics with multiple physical interactions have not been resolved, an issue that the entire industry has yet to address.
The technical name of Atlas is multimodal autoregressive diffusion transformer, with each of the four words being individually unremarkable.
Transformers and autoregressive models are used in large language models, which generate content one after another based on what has already been generated. Diffusion models are used for image and video generation, creating images from scratch. Multimodal refers to models that receive multiple types of input simultaneously.
The new locale in multimodal specifically includes four categories: text, images, camera poses, and depth maps.
The camera pose refers to the location and direction of the camera in 3D space when the photo was taken, which can be clearly described with six numbers. When combined with a depth map, each photo is no longer just a standalone image, but rather an image with its shooting location clearly marked.
All photos are tagged with the same set of coordinates, which is Atlas's shared spatial context.
For instance, standing at the entrance of the living room to take a photo of the sofa, and then walking to the window to take another photo. The two photos show the sofa in different sizes and angles, with half of it blocked by a coffee table. Ordinary models see two different images. Humans see the same sofa because they know they took a few steps and turned a certain degree.
What Atlas obtained was the data on "how many steps were taken and how many degrees were turned".
This data has another use that World Labs' previous product, RTFM, has already attempted. Its goal was to generate a world in real-time on a single H100 that could be explored continuously. As you move forward, it generates more, and the perspective changes as you turn. The problem lies in memory.
If someone were to wander through this virtual house for half an hour, and every frame generated during that time were fed into the model's context window, the model would indeed forget nothing. But as the context grows longer and longer, computational costs would give out first. If only the most recent few dozen frames were kept, computing power would be saved—yet by the time the model loops from the bedroom back to the kitchen, it would no longer remember where the table was placed twenty minutes earlier, and the same kitchen would be generated as two different versions.
World Labs calls its approach to RTFM "posed frames as spatial memory." Each frame stores not just the image itself, but also the camera's position and orientation in space when it was captured. When the model moves to a given location, the system doesn't need to sift through its entire memory—it simply retrieves a few nearby frames based on position and assembles the context needed for that particular generation.
Location serves as the index here. To see what the kitchen looks like, it's enough to pull up the frames captured nearby—when they were taken doesn't matter.
Human memory works in a similar way. If you try to recall what you did last Wednesday afternoon, you may not be able to remember, but if you walk into the meeting room, you can quickly recall who was sitting across from you and what was written on the whiteboard. Retrieving memories by time requires going through them one by one, while retrieving memories by location can be done in one step.
Reconstruction and Generation
There is also an experiment on Atlas's release page. Initially, only a photo of a garden is provided, and the model generates an image from a bird's-eye view. The photo only shows the garden, without capturing the surroundings, so the garden part is restored as is, and the adjacent buildings are supplemented by the model's prior knowledge from its training data.
The second photo shows a small hut next to it, which does not need to be filled in. The third photo reveals the house, and there is one less place that needs to be filled in.
According to World Labs, the more it sees, the less it imagines. Based on the data they provided, two or three photos are usually enough to restore a scene to a usable state, with a maximum of over 100 photos allowed at a time.
This experiment has integrated two things that were originally very separate into the same model.
To obtain a 3D scene on a computer, there were two approaches in the past. One was called reconstruction, which involved using photos taken on site to restore a real existing place to its original state, leaving blank areas that were not captured, even if it meant leaving a hole. The other was called generation, where a model would create a non-existent place based on the houses and lawns it had seen during training.
Atlas has it both ways, filling in the gaps with common sense where photos are lacking, and then replacing the fabricated parts with real ones as new photos become available, fabricating more when there are fewer photos and less when there are more.
I've always been fond of a project called Funes World, which travels the world taking photos of buildings with cameras, including famous ones like Pompeii, Troy, Cologne Cathedral, and the Temple of Heaven, as well as lesser-known houses that are renovated one day and then forgotten in their original form. They take photos from all angles, leaving enough overlap between them, and then use photogrammetry, NeRF, and Gaussian Splatting to reconstruct 3D models. The public archive now has over 2,100 models, covering more than 19 countries.
They claim to be making a backup of the real world, creating a physical world's GitHub.
Funes is taking a purely reconstructive approach to that road, snapping photos ahead of the bulldozers to ensure that the house is indeed shaped that way, and even if a piece is missing, they won't let a model fill it in.
Atlas faces a different kind of user. A wall in a kitchen, as long as the location and size are suitable, the result of a robotic arm bumping into it is the same, and the robotic arm doesn't care whether the pattern on the wall is photographed or coded.
So the question falls on another aspect, whether a partially fabricated world can be used to train a machine that will work in reality. As for whether the fabricated houses are realistic or not, that can be set aside for now.
In July, World Labs acquired robotics company SceniX, and the first batch of results released after the acquisition is in response to this move.
Several groups of robots were completely trained in simulators without using any real-world data and then directly installed on real robotic arms to perform tasks such as boxing, wire winding, and tube handling. Among them, several machines ran continuously and autonomously for an hour without human intervention.
What's more worth looking at is how they determine whether this simulator is qualified or not.
There's a standard that sounds reasonable: if the success rate is 80% in a simulator, it should also be 80% on a real device. However, this standard wasn't applied. The friction between the wall in the simulator and a real wall is different, and the weight distribution of the parts doesn't match, so the success rate is bound to be lower.
They changed their approach to this: first, they ran several scenarios 2,000 times each in a simulator, ranking their performance, then they moved these scenarios to a real device and ran them 100 times each, focusing on two things: whether the simulator's conclusion that scenario A outperformed scenario B held true on the real device, and whether the step where the simulator predicted a certain action would be most prone to errors was indeed the step where errors occurred on the real device.
In the disclosed tasks, these two matched up. As for the difference in success rates between the two, it doesn't matter, according to World Labs' own statement, a useful simulator doesn't need to have the same success rate as reality, it just needs to support decision-making in the same way as reality.
This method was used by people over a hundred years ago.
In the fall of 1901, the Wright brothers built a glider based on Lilienthal's published wing data, but it only achieved one-third of the predicted lift. The data from their predecessors was flawed, and re-measuring it would require sending people up into the air repeatedly. Therefore, they built a 6-foot-long wind tunnel out of wood, which allowed them to test several sets of wings in a single afternoon, and they tested over 100 different types that winter.
The wind in a wind tunnel and the sea breeze in North Carolina are two different things, but which pair of wings is better can be compared.
A world model does not need to replicate the entire universe, it only needs to retain the structure relevant to decision-making.
Failed experiences
At this point, the shape of World Labs' business comes into focus. Its product Marble, launched in 2025, can already generate explorable 3D worlds from text, images, and video. With the World API open, developers can call it directly. Atlas is the underlying model layer beneath these products.
In the $1 billion financing round in February this year, the investors came from a diverse range of companies. NVIDIA is interested in Physical AI, Autodesk in architecture, industrial design, and digital twins, while others such as AMD, Fidelity, and Sea also participated. In the same 3D world, directors focus on camera angles and lighting, architects on structure and scale, and robots require an environment for experimentation.
A world that is sufficiently universal can span several software industries that are separate from each other today.
It no longer makes sense to put companies that claim to be working on world models into the same benchmark table.
Genie 3 generates images, while Meta's V-JEPA 2 skips image generation and directly learns patterns from videos. NVIDIA's Cosmos integrates visual reasoning, world generation, and action prediction into the same model. Runway has shifted from video generation to simulation. They use the same terminology but are solving different problems.
There is only one divide. One camp believes that if the world is predicted well enough, 3D structures will emerge from the data on their own, and videos originally contain these rules. The other camp directly embeds camera positions, geometry, and physical constraints into the model, allowing the world to have structure from day one. Atlas does this by making camera pose and depth native inputs.
There's one thing that can't be avoided on this path: where the training data comes from.
Large language models have received a gift. Over the course of several decades, humans have written web pages, coded, and argued on forums, inadvertently accumulating a massive corpus of text.
There's no equivalent in the three-dimensional world. Spatial relationships are rarely systematically recorded, and the friction coefficient of a floor won't automatically be written into a webpage. Even more challenging is interaction - if a robot inserts a wire at an angle, the wire in reality is indeed at an angle, and someone has to go and straighten it before the next attempt. Text can be copied infinitely, but real-world experiences have to occur anew each time.
So what robots have lacked most, in fact, is failure. Making mistakes in the real world is too costly, but once simulators work well, robots can run thousands or tens of thousands of iterations inside them, rapidly accumulating experience.
Li Fei-Fei has been dealing with this issue for nearly twenty years.
In 2009, she and her team worked on ImageNet, reorganizing the scattered images on the internet according to WordNet's conceptual system. The complete index contained 14.19 million images and 21,841 synsets, and the ILSVRC, which later sparked the deep learning explosion, used only 1,000 of these classes. The topics of that generation of computer vision were very simple: given a machine a photo, let it say what was in it, what class it belonged to, and where it was located in the picture.
Seventeen years have passed, and this is no longer a problem. The cups in the kitchen in front, today's models can not only recognize them, but also generate a hundred cups that have never existed before out of thin air.
The difficulty lies in determining the position of the cup in the room, whether going around the table to observe it from behind would still be the same cup, and after someone pushes it away, where it should be and what state it should be in the next time you look. Recognizing an object and knowing how it exists in the world are two different things.
From ImageNet to Atlas, there is an unexpected continuous thread: how much information about the world needs to be put into machines for them to develop their own world.
In 1940, Argentine writer Adolfo Bioy Casares wrote a novel, The Invention of Morel.
The protagonist escaped to a deserted island, where he encountered a group of strange people. They went for walks, chatted, and danced every day, and everything seemed normal, but no one could see him. Even stranger, the same conversations would recur after a few days, with the same pauses in the same places.
It was only later that he discovered the island's scientist, Morel, had built a machine that had recorded a group of people on the island for an entire week. The sounds were recorded, the images were recorded, and Morel believed that even the sense of touch, smell, and temperature had been preserved. The machine, powered by the tides, replayed that week on the island in a loop.
Morel believed that when all senses are synchronized, the soul appears.
More than eighty years ago, this machine was written about, which now seems like an exaggeratedly absurd multimodal model, preserving all sensory evidence of reality and producing a replica that any observer cannot distinguish from the truth.
The protagonist fell in love with the woman Faustine in the projection. He stood in front of her every day, talking to her, hoping she would notice him. She never responded, because her part had been recorded weeks ago. The machine could replay that week 10,000 times, but it couldn't produce anything that hadn't happened at the time.
According to World Labs' own table, Morel built a perfect renderer.
Today's models can already restore a scene to be flawless. The difficult part is simulating something that has never been photographed before, such as someone walking in, reaching out, and knocking over a cup on the table.
Morel's island has everything, except for this one thing.