On September 1, Fei-Fei Li's World Labs released Atlas.
There is a set of images on the release page that look really good, with several phones mounted on tripods and fixtures, shooting the same scene from three to five angles, then time stands still, the camera moves, bypassing the characters, and goes to positions where real cameras have never been.
This tactic is not uncommon in the past two years. The difference lies in the three other things Atlas has simultaneously released.
The first is depth. By labeling how far each pixel in a frame is from the camera, an ordinary photo becomes a terrain map colored by distance.
The second is the point cloud, where millions of points are scattered in space, each point remembering its own three-dimensional coordinates, and when combined, they form the shape of this room.
The third is 3D Gaussian Splat, which does not use triangular faces to build models, but instead uses a large number of small light spots with color, transparency, orientation, and varying sizes to render the entire scene, and can re-render from any angle.
What unites these three things is that they can be measured. The diameter of a cup, the distance from a table to a wall—all can be read directly as numerical values. The generated output is no longer just an image for human eyes; it can be fed into a game engine, design software, or the control code for a robotic arm.
World Labs simply calls Atlas the World Simulator.
They don't use the term simulator lightly. Three months ago, they wrote an article themselves that specifically defined what kind of system deserves to be called by that name.
Tangible and Intangible
The article, published in June this year, discusses what constitutes a world model, and concludes by categorizing existing systems into three levels: renderers, simulators, and planners.
The origin of this classification method dates back to 1943, when British psychologist Kenneth Craik proposed that humans can anticipate the development of an event because the brain has a miniature version of the real world. This idea was later adopted by control theory, neural networks, and reinforcement learning, and evolved into three key terms: state, observation, and action.
Let's take a kitchen as an example.
The position of the cup on the table, its distance from the table's edge in centimeters, how far the table sits from the wall, and whether the door is open or closed—together, these define the kitchen's state at this moment.
A person stands at the door and can only see part of the scene. The floor blocked by the refrigerator is invisible to him, and he has no idea how much water is left in the cup. The information he takes in through his eyes is called an observation.
He walked over and pushed the cup away, which was an action.
The number of aspects a system can manage, including state, observation, and action, determines its level.
The renderer draws the screen. When a person takes a step forward, it calculates what should be seen at that step, renders it realistically and seamlessly connects it to the previous frame, and the task is done. It doesn't need to know how tall the cup is or how far the table is from the wall, because all it hands over is an image.
It maintains the state of the simulator, storing numbers such as the position of a cup, the size of a table, and how far a door can be opened before hitting a wall, allowing these values to be retrieved at any time. Because it stores these numbers, other programs can be integrated, and a mechanical arm can extend into this kitchen.
The planner takes action. It not only knows that the cup is on the table and what will happen if it is knocked over, but also needs to determine whether to reach for it now, which one to pick up first, and from what angle to grasp it most stably.
Google DeepMind's Genie 3 is classified as the lowest level, merely a renderer, along with World Labs' own RTFM, which was released in October 2025.
Before this, the term "world model" was a catch-all. Type a sentence and generate an immersive scene you could walk into—that was a world model. Change your website's tagline to claim you simulate the world with AI—also a world model. This article is the first to draw a line: visuals are visuals, and the world is the world.
In the past year, most demos have stopped at the first level. Compared to pure text-to-video generation, they have made significant progress, and the scene changes when you press the direction key.
However, the output from the renderer is good enough for human viewing but not suitable for machine use.
Atlas needs to be on the same level as simulators. By their own definition, simulators must maintain the entire state of the world, including physics.
Current tests focus on camera-condition generation and 3D reconstruction, and complex materials, fluids, and fabrics with multiple physical interactions have not been resolved, an issue that the entire industry has yet to address.
The technical name of Atlas is multimodal autoregressive diffusion transformer, with each of the four words being individually unremarkable.
Transformers and autoregressive models are used in large language models, which generate content one after another based on what has already been generated. Diffusion models are used for image and video generation, creating images from scratch. Multimodal refers to models that receive multiple types of input simultaneously.
The new locale in multimodal specifically includes four categories: text, images, camera poses, and depth maps.
The camera pose refers to the location and direction of the camera in 3D space when the photo was taken, which can be clearly described with six numbers. When paired with a depth map, each photo is no longer just a standalone image, but rather an image with its shooting location clearly marked.
All photos are tagged with the same set of coordinates, which is Atlas's shared spatial context.
For example, standing at the entrance of the living room to take a picture of the sofa, and then walking to the window to take another picture. The two photos show the sofa in different sizes and angles, with half of it blocked by a coffee table. An ordinary model sees two different pictures. A person sees the same sofa because they know they took a few steps and turned a certain degree.
What Atlas obtained was the data on "how many steps were taken and how many degrees were turned".
This data has another use that World Labs' previous product, RTFM, has already attempted. Its goal was to generate a world in real-time on a single H100 that could be explored continuously. As you move forward, it generates more, and the perspective changes as you turn. The problem lies in memory.
Assuming someone browses this virtual house for half an hour, if all the images generated during this half hour are stuffed into the context, the model will indeed not forget anything, but the context will become increasingly lengthy, and the computing power will be overwhelmed first. If only the last few dozen frames are kept, the computing power is saved, but when walking from the bedroom and back to the kitchen, the model will have already forgotten where the table was 20 minutes ago, and the same kitchen will be generated in two different ways.
World Labs refers to RTFM's approach as "posed frames as spatial memory." Each frame not only captures the image but also stores the camera's position and orientation in space at the time it was taken. When the model reaches a certain location, the system doesn't need to search through all of its memories, but instead retrieves a few frames based on proximity to the location, and then stitches them together to generate the necessary context for that instance.
Location serves as the index here. To see what the kitchen looks like, it's enough to pull up the frames captured near it—when they were shot doesn't matter.
Human memory works in a similar way. If you try to recall what you did last Wednesday afternoon, you may not be able to remember, but once you step into that meeting room, you can quickly recall who sat across from you and what was written on the whiteboard. Retrieving memories by time requires going through them one by one, while retrieving by location can be more direct.
Reconstruction and Generation
There is another experiment on Atlas's release page. Initially, only a photo of a garden is provided, and the model generates an image from a bird's-eye view. The photo only shows the garden, without capturing the surroundings, so the garden part is restored as is, and the adjacent buildings are supplemented by the model's prior knowledge from its training data.
The second photo shows a small hut next to it, which does not need to be retouched. The third photo reveals the house, and there is one less area that needs to be retouched.
World Labs' conclusion: the more it sees, the less it imagines. Based on their data, two or three photos are typically enough to reconstruct a scene for practical use, with a maximum of over a hundred photos that can be fed in at once.
This experiment has integrated two things that were originally very separate into the same model.
To obtain a 3D scene on a computer, there were previously two approaches. One is called reconstruction, which involves using photos taken on site to restore a real existing location to its original state, leaving blank any area that was not captured, even if it means leaving a hole. The other is called generation, where a model creates a non-existent place based on the houses and lawns it has seen during training.
Atlas has it both ways, filling in the gaps with common sense where photos are lacking, and then replacing the fabricated parts with real ones as new photos become available, fabricating more when there are fewer photos and less when there are more.
I've always been fond of a project called Funes World, which travels the world taking photos of buildings with cameras, including famous ones like Pompeii, Troy, Cologne Cathedral, and the Hall of Prayer for Good Harvests, as well as lesser-known houses that are renovated one day and then forgotten in their original form. They take photos from all angles, leaving enough overlap between them, and then use photogrammetry, NeRF, and Gaussian Splatting to reconstruct 3D models. The public archive now has over 2,100 models, covering more than 19 countries.
They claim to be making a backup of the real world, creating a physical world's GitHub.
Funes took the path of pure reconstruction. They clicked the shutter before the bulldozers arrived, demanding that the house look exactly as it did. Even if a piece was missing, they wouldn't let a model fill it in for them.
Atlas faces a different kind of user. For a wall inside a kitchen, as long as the position and dimensions line up, the result is the same when the robotic arm reaches out and hits it—whether the pattern on the wall was captured by a camera or generated by code is of no concern to the arm.
So the question falls to another place, a partially fabricated world, and whether it can be used to train a machine that will work in reality. As for whether the fabricated houses are realistic or not, that can be set aside for now.
In July, World Labs acquired robotics company SceniX, and the first batch of results released after the acquisition is in response to this move.
Several groups of robots were completely trained in simulators without using any real-world data and then directly installed on real robotic arms to perform tasks such as boxing, wire winding, and tube handling. Among them, several machines ran autonomously for an hour without human intervention.
What's more worth looking at is how they determine whether this simulator is qualified or not.
There's a standard that sounds reasonable: if the success rate is 80% in a simulator, it should also be 80% on a real device. However, this standard wasn't applied. The friction between the wall in the simulator and a real wall is different, and the weight distribution of the parts doesn't match, so the success rate is bound to be lower.
They changed their approach to this: first, they ran several scenarios 2,000 times each in a simulator, ranking their performance, then they moved these scenarios to a real device and ran them 100 times each, focusing on two things: whether the simulator's conclusion that scenario A outperformed scenario B held true on the real device, and whether the step where the simulator indicated a certain action was most likely to fail was also where it failed on the real device.
In the disclosed tasks, these two matched up. As for the difference in success rates between the two, it doesn't matter, according to World Labs' own statement, a useful simulator doesn't need to have the same success rate as reality, it just needs to support decision-making in the same way as reality.
This approach was used by someone over a hundred years ago.
In the fall of 1901, the Wright brothers built a glider based on Lilienthal's published wing data, but it only achieved one-third of the predicted lift. The data from their predecessors was flawed, and re-measuring it would require sending people up into the air one by one. So, they built a 6-foot-long wind tunnel out of wood, which allowed them to test several wing designs in a single afternoon, and they tested over 100 designs that winter.
The wind in a wind tunnel and the sea breeze in North Carolina are two different things, but which pair of wings is better can be compared.
A world model does not have to replicate the entire universe, it only needs to retain the structure related to decision-making.
Lessons from failure
By this point, the outline of World Labs' business has taken shape. Their product, Marble, launched in 2025, can generate explorable 3D worlds from text, images, and videos. After the World API is opened up, developers can directly call on it, and Atlas is the underlying model that supports these products.
In the $1 billion financing round in February this year, the investors came from a diverse range of companies. NVIDIA is interested in Physical AI, Autodesk in architecture, industrial design, and digital twins, while others such as AMD, Fidelity, and Sea also participated. In the same 3D world, directors care about camera angles and lighting, architects about structure and scale, and robots need an environment to experiment in.
A world that is sufficiently universal can span several software industries that are currently separate today.
It no longer makes sense to put companies that claim to be working on world models into the same benchmark table.
Genie 3 generates images, while Meta's V-JEPA 2 skips image generation and directly learns patterns from videos. NVIDIA's Cosmos integrates visual reasoning, world generation, and action prediction into the same model. Runway, on the other hand, has shifted from video generation to simulation. They use the same terminology but are solving different problems.
There is only one difference. One school of thought believes that if the world is predicted well enough, 3D structures will emerge from the data on their own, and videos originally contain these rules. The other school directly incorporates camera positions, geometry, and physical constraints into the model, allowing the world to have structure from day one. Atlas does this by making camera pose and depth native inputs.
There's one thing that can't be avoided on this path: where the training data comes from.
Large language models have received a gift. Over the course of several decades, humans have written web pages, coded, and argued on forums, inadvertently accumulating a massive corpus of text.
There's no equivalent in the three-dimensional world. Spatial relationships are rarely systematically recorded, and the friction coefficient of a floor won't automatically be written into a webpage. Even more challenging is interaction - if a robot inserts a wire at an angle, the wire in reality is indeed at an angle, and someone has to go and straighten it before the next attempt. Text can be copied infinitely, but real-world experiences have to occur anew each time.
So what robots have lacked most in the past is, in fact, failure. Making mistakes in the real world is too costly, but once simulators become reliable, robots can run thousands or even tens of thousands of iterations inside them, rapidly accumulating experience.
Li Fei-Fei has been dealing with this issue for nearly twenty years.
In 2009, she and her team worked on ImageNet, reorganizing the scattered images on the internet according to WordNet's conceptual system. The complete index contained 14.19 million images and 21,841 synsets, and the ILSVRC, which later sparked the deep learning boom, used only 1,000 of these classes. The topics of that generation of computer vision were simple: given a machine a photo, it would say what was in it, what class it belonged to, and where it was located in the picture.
Seventeen years have passed, and this is no longer a problem. The cups in the kitchen in front can not only be recognized by today's models, but also generate a hundred cups that have never existed before out of thin air.
The difficulty lies in determining the cup's position in the room, whether circling around the table to observe it from behind would still be the same cup, and after someone pushes it away, where it should be and what state it should be in the next time it is seen. Recognizing an object and understanding how it exists in the world are two different things.
From ImageNet to Atlas, an unexpected thread of continuity runs through: how much information about the world must humans feed into a machine before it grows a world of its own?
In 1940, Argentine writer Adolfo Bioy Casares wrote a novel, The Invention of Morel.
The protagonist escaped to a deserted island, where he encountered a group of strange people. They went for walks, chatted, and danced every day, and everything seemed normal, but no one could see him. Even stranger, the same conversations would recur after a few days, with the same pauses in the same places.
Later, he learned that a scientist on the island, Morel, had built a machine that recorded an entire week of a group of people's time on the island. The sounds were captured, the images were captured, and Morel believed that even touch, smell, and temperature had been preserved. The machine was powered by the tides, and that week played on a loop across the island.
Morel believes that when all senses are synchronized, the soul appears.
More than eighty years ago, this machine was written about, which now seems like an exaggeratedly absurd multimodal model, preserving all sensory evidence of reality and producing a replica that any observer cannot distinguish from the truth.
The protagonist fell in love with the woman Faustine in the projection. He stood in front of her every day, talking to her, hoping she would notice him. She never responded, because her part had been recorded weeks ago. The machine could replay that week 10,000 times, but it couldn't produce anything that hadn't happened at the time.
According to the chart from World Labs, Morel built a perfect renderer.
Today's models can already restore a scene to be flawless. The difficult part is capturing something that has never been photographed before, such as someone walking in, reaching out, and knocking over a cup on the table.
Morel's island has everything, except for this one thing.