In May this year, Jim Fan, head of NVIDIA's robotics division, declared in a public speech that VLA and teleoperation are dead.
At the time, the first reaction from the vast majority of people was: the AI circle, which never lacks extreme views, has added another new extreme view.
But unexpectedly, several months later, those words that had sounded somewhat exaggerated at the time actually began to become a reality.
自变量 just released TwinDex, a new dexterous operating system. With zero real-machine teleoperation data in the post-training phase, the robot completed a series of fine manipulation tasks in chemistry lab scenarios, including twisting bottle caps and pushing syringe plungers.
In the past, to teach robots such fine operations, it was often necessary to use real-machine remote control data for "hand-in-hand" instruction.
But this time, TwinDex directly cut out this part of the data demand and replaced it with only a few hundred lines of non-ontological data, with almost no robot content.
How is this achieved?
As the saying goes, a good demo is worth savoring.
In the demo released by TwinDEX this time, the most complete segment is a one-take chemical experiment:
From opening bottles for sampling, pipetting and test tube operations, to glass rod drainage and shaking observation, the entire process is autonomously executed by the strategy.
The entire experiment consists of 24 sub-actions, spanning three types of tools, involving multiple instances of bimanual coordination and tool switching, with each step requiring millimeter-level positioning and stability control.
Breaking it down, the three key aspects of TwinDEX actually concentrate on showcasing several types of capabilities that were notoriously difficult to master in past agile operations.
First, it requires extremely high precision and delicate operations.
When opening the toolbox, the robot needs to align its left and right index fingers with two narrow latches, accurately insert them, and then pull downwards to release, before using its index fingers and thumbs to grip the handle and open the lid.
The difficulty lies in whether the robot's arm can accurately enter a space with very small tolerance to complete the operation.
Similarly, injector pushing is also similar.
The index and middle fingers fix the syringe body, the thumb is aligned with the plunger to exert forward force, ensuring that the position is not deviated while precisely controlling the direction and force of the injection.
As embodied intelligence gradually enters laboratories, factories, and homes, such small-clearance, high-precision, and strong-contact operations will also become increasingly close to the basic skills of robots.
Second is the three-finger collaboration.
For example, in tasks such as using a broom and dustpan, three fingers form an envelope around the handle, allowing different fingers to share the burden of support and control.
Compared to having only two fingers, the third finger provides an additional point of contact and an extra way to stabilize objects.
This additional point of support is particularly crucial in tasks such as tool use and grasping long objects.
The third category is closer to human-like flexible operation and palm-based adjustment.
When opening a bottle cap, TwinDEX uses the index finger and thumb to grip the cap, and then relies on the lateral movement of the fingers to complete the rotation, with minimal need for a large wrist movement.
When flipping through a book, the robot's thumb first separates the top page, then completes the pinching, two-hand transfer, placement, and page-turning motions.
After personally going through this round, one of my most intuitive feelings is:
In the past, when dealing with this type of action, people would often instinctively think of more complex five-finger dexterous hands and large amounts of real-machine remote operation data.
But the demo above illustrates one thing: many delicate operations that were previously thought to require five fingers can actually be accomplished with just three.
To some extent, these three fingers have become more agile than many five-fingered hands.
However, the aforementioned aspects are just one side of TwinDEX, which is at the forefront and serves as a robot executor.
But before the robot actually gets down to work, it has another identity: a wearable, bodyless data collection system designed to match the execution end.
Unlike many solutions that use two different sets of hardware for data collection and robotic execution, TwinDEX adopts a design where data acquisition and execution are isomorphic.
In simpler terms, these three fingers are not only responsible for doing the work, but also for collecting data using the same structure, which is known as "what is collected is what is obtained".
The actions taken by collectors at the collection end can be more directly mapped to the robot execution end, without having to go through a completely different set of hardware to perform complex data migration.
This is also why the delicate operations mentioned earlier, which require extremely high precision in terms of finger position and force, can be more stably performed by robots.
Meanwhile, compared to directly remotely operating a cumbersome robot, collectors can complete actions more naturally and quickly, with better data collection and lower costs.
The experimental results also verified these two points.
In terms of collection efficiency, the number of effective trajectories that TwinDEX can produce per unit time is approximately 5.3 times that of traditional real-machine remote control.
With the same collector and the same amount of time, TwinDEX is able to accumulate a batch of truly usable training data more quickly.
As for the efficiency of data utilization, the independent variable found:
As the amount of data increases, strategies trained on both simulated data and real-machine remote operation data will continue to improve at the same rate and eventually converge to similar performance.
In other words, in this set of experiments, when training the model, simulated data can almost completely replace real remote control data, with an accuracy of nearly 100%.
So, as a data collector, TwinDEX solves not just "how to collect data faster".
More importantly, it has minimized the loss of accuracy when these data are actually migrated to the robot while improving collection efficiency.
And these two matters ultimately converge on the same question:
How to make non-ontological data truly usable for robots.
TwinDEX: Aligning Embodied Data Early
Strictly speaking, TwinDEX is not just a three-finger dexterous hand.
It is more like an agile operating system consisting of data collection hardware, robotic end-effectors, data processing pipelines, and model training processes.
Furthermore, simply giving the data-collection and robot-execution ends the same form won't naturally enable data migration.
The key aspect of TwinDEX lies in its design, which starts from the goal of completing training that relies solely on non-physical data and removing the need for real-machine remote operation data, and then works backward to design the entire system.
From the configuration of the agile hand, the movement of the collection equipment, to visual observation, time synchronization, data processing, and finally to model training, everything is actually centered around the same issue:
How can collected data be as close as possible to what robots actually need from the very start?
Having understood this, let's take a look at the most intuitive hardware part of TwinDEX.
In terms of the functional division seen in actual demos, the thumb and index finger undertake a large amount of work related to grasping, pinching, twisting, and fine-tuned operations, while the third finger can further provide enveloping, support, and stability.
The TwinDEX design as a whole can be distilled into three key attributes: agility, consistency, and scalability.
The first is agility, which has been repeatedly mentioned earlier.
TwinDEX did not opt for the commonly seen two-finger pincer like UMI, nor did it stack all the way to five fingers like a human hand, instead striking a balance between dexterity, stability, and engineering complexity.
It is commonly known that the more fingers and the closer the structure is to a human hand, the higher the theoretical limit for dexterous operation.
But at the same time, there are also more joints, drivers, sensors, as well as the resulting calibration, control and maintenance costs.
Conversely, while two-finger grippers are simple enough, they struggle to cover fine operations such as twisting, in-hand adjustment, and multi-point contact.
After undergoing basic grasping, in-place twisting, tool use, and palm inner operations, as well as comparisons with multiple candidate configurations, the independent variable was ultimately found:
Three fingers with nine degrees of freedom is currently the sweet spot, as it can cover the dexterous capabilities truly needed for most current tasks with relatively controllable complexity.
The second, and also the most crucial design concept for TwinDEX, is consistency.
The consistency referred to here means that TwinDEX's data collection end and final execution end maintain consistency in terms of movement method, contact method, visual observation, collection precision, and time synchronization.
This is also the key to TwinDEX being able to reduce the demand for real-machine remote control data.
In the past, although a lot of non-embodied data was readily available, when it actually came to migrating it to robots, a problem often arose: the collection end and the execution end did not share the same ontology.
How a person's hand moves and how a collection device records it may not be exactly reproducible on an actual robot, as there are usually kinematic differences and precision losses in between.
So, while this type of data is easy to scale, it is difficult to directly use for training target robots. Often, it is still necessary to supplement with a batch of real-machine remote control data to complete entity alignment and post-training.
TwinDEX's consistent design is intended to address this very issue.
It doesn't wait until the data collection is complete before trying to make the data adapt to the robot, but rather, during the hardware design phase, it makes the collection end and execution end as isomorphic as possible.
In other words, rather than waiting until the data collection is complete and then figuring out how to make it compatible with robots, the goal is to align the collection equipment and agile hands as much as possible from the outset.
In other words, the "ontology alignment" that originally occurred during the data processing stage has been moved forward to before the data is even generated.
This way, the loss of data when migrating from the physical entity to the robot can be minimized, reducing the need for additional real-machine remote control data supplementation.
That is precisely why, across the tasks covered in this experiment, TwinDEX was able to complete post-training using only a few hundred pieces of embodiment-free data—without adding any new teleoperation demonstrations from the target robot—and still execute the fine manipulations described above.
The third characteristic is scalability.
TwinDEX's data acquisition component is a wearable three-finger exoskeleton.
Its biggest advantage is that data collection does not require occupying a physical robot or being confined to a fixed workstation.
Once the operator puts on the device, they can directly perform actions such as twisting, pressing, and using tools.
Compared to operating robots remotely through a controller, this method is also closer to human natural operation: the hand can directly contact objects, obtain real force feedback, and does not require additional adaptation to spatial mapping and communication delays.
More importantly, data collection is no longer tied to the number of robots. Multiple operators can collect data simultaneously in different locations and then unify the data in the same processing and training pipeline.
During the collection process, the system synchronously records multimodal information including vision, joint states, and wrist poses, then applies calibration, time synchronization, and standardization to consolidate data from different operators and locations into a unified training space.
However, as the scale expands, new problems also arise.
Different devices and operators during the collection process inevitably introduce vibrations, drift, and positioning errors.
Therefore, the independent variables have also been designed in the model architecture and training process to enable the model to tolerate a certain range of collection errors, and ultimately learn the data into strategies that robots can execute in a closed loop.
At this point, the three design logics of TwinDEX have become clear:
Dexterity determines what can be picked and done; consistency determines whether the collected non-embodied data can be smoothly transferred to robots; and scalability determines whether this collection approach can truly scale.
The Tip of the Data Pyramid is Being Flattened
So what truly sets TwinDEX apart isn't just faster data collection—it demonstrates something more fundamental: high-quality data that closely mirrors robotic movements doesn't necessarily have to originate from physical robots.
Over the past two years, remote real-machine data has been at the top of the embodied data pyramid.
It offers the highest quality, as vision, joint states, and end-effector trajectories are naturally aligned—but it also carries the highest cost—
Each piece of data acquired must genuinely occupy one robot.
Meanwhile, robots need to be deployed and maintained, and humans also need to learn remote operation. Once the number of robots becomes the upper limit of data production capacity, it becomes extremely difficult to truly scale.
This is also one of the contexts behind Jim Fan's claim that teleoperation is dead.
For this reason, the entire embodied intelligence community has spent the past two years exploring new data collection strategies, including UMI, gloves/exoskeletons, and Ego first-person perspectives.
Their core idea is also to use more easily extensible non-physical entity data to reduce dependence on real machine data.
But here has always been a dilemma.
The freer the collection method, the easier it is to scale data; yet the further the collection end is from real robots, the harder the subsequent mapping and migration become.
In particular, the differences between human hands and robots in terms of structure, degrees of freedom, and contact methods will continuously introduce action redirection and precision loss.
TwinDEX has found a new sweet spot between scalability and the richness of physical information in data retention.
It does not completely abandon the constraints brought by the robot itself, but instead "copies" the action structure that the robot truly needs in advance to the data collection end.
Thus, in the tasks already demonstrated, only a few hundred pieces of non-embodied data were needed to complete part of the alignment work that previously, during the post-training phase, typically relied on real-robot teleoperation data.
Of course, objectively speaking, given that embodied data recipes are still far from converging, and considering the demands for task coverage and data diversity, we cannot yet declare teleoperation dead.
But at least one thing has become increasingly clear:
The strong binding between high-quality data and real-machine remote control is loosening.
If body-less data can gradually approach the training value of real-machine data during the post-training phase, then the next frontier worth exploring is a Scaling Law specific to body-less data.
However, before scaling up the unlabeled data volume to truly validate this Scaling Law, the independent variable has already offered its answer at this particular point in time.
The era in which every expansion to a new task must start from real-machine remote control data is coming to an end.
Project homepage: https://x2robot.com/pages/twindex