Trust · how we test
Devdoot™ is built so a robot tells you the truth about what it did and what it can't do. Here is how we test that, and every result so far, misses included.
Four answers, no guessing
Every task ends one of four ways.
It did the job, and checked with its own sensors that it really did.
Something makes the task unsafe, so it refuses before it moves.
Its sensors can't tell it enough to act or to be sure, so it stops and says so.
It tried, checked itself, and is telling you it didn't work.
How we test
The simulator knows the truth.
- Randomize. Every mission runs 20 times per situation, with the robot, the room and the faults changed each time.
- Break things on purpose. Wheels slip, sensors drop out, cameras go off, objects are out of reach.
- Grade against the truth. Only the simulator knows what really happened, so only it can say whether "Done" was true and whether "I don't know" was justified.
- Compare. The same robot runs without the honesty layer, so you can see what it prevents.
- Publish x of N. Every number is a count, including the times our robot was too cautious.
Latest results · simulation
Scorecards.
| Robot and mission | Faults tested | Said Done but was wrong | Without Devdoot™ | Hit something | Without Devdoot™ | Too cautious |
|---|---|---|---|---|---|---|
| Rover Drive a set distance | Wheel slip, blocked path, distance sensor lost | 0 of 50 | 48 of 100 | 0 of 100 | 18 of 100 | 30 |
| Classroom rover Stop 15 cm before the wall | Angled wall, lost echo, noisy sensor | 0 of 60 | 59 of 120 | 0 of 120 | 40 of 120 | 0 |
| Classroom rover Follow the line to the finish | Tight curves, breaks in the tape, flickering sensor | 0 of 77 | 49 of 120 | 0 of 120 | 0 of 120 | 3 |
| SO-101 arm Put the cube in the bin | Out of reach, two identical cubes, slippery cube, camera off or blinded | 0 of 78 | 88 of 140 | 0 of 140 | 0 of 140 | 2 |
"Too cautious" counts the runs where our robot stopped or said "I don't know" although the task could have been done. We report it because a robot that refuses everything is useless too.
Safety the robot enforces itself
The app asks. The robot decides.
- Lose contact with the controller for half a second and the motors stop.
- Something closer than 10 cm ahead and it won't drive forward.
- Speed is capped by mode, and the fastest mode switches off when the distance sensor is blind.
- Driving by hand while a mission is running is refused.
Checked by an automated test: 70 of 70 runs passed in simulation. The same test will grade the real robot.
What we don't claim
Where this stands.
- These results are from simulation. Real-robot scorecards will be published here when we have them, from the same tests.
- DEV 0.0 is a concept design; embodied products are not yet available.
- Robots in the shop are sold and supported by our partners, who are the seller of record.
- Your details from our forms are used only to answer you. We don't sell them.