Self-driving vehicles currently rely on a mix of cameras, radar and LiDAR to ensure they have all the data they need to navigate safely. However, Tesla intends to ultimately rely solely on cameras, using a neural network to achieve vision-only autonomous driving.
Such a system is highly desirable for a number of reasons. Most obviously, the technology effort per vehicle is reduced, reducing both cost and weight. And as Tesla CEO Elon Musk pointed out on Twitter back in April, "Vision has a lot more precision, so better to double vision than sensor fusion."
However, using Vision-only requires a lot of training, and that's where Dojo comes in. As TechCrunch reports, Dojo is a neural network training computer that Tesla plans to use to process the "huge amounts of video data" needed to train such a self-search-driving system. The problem is that Dojo doesn't exist yet, but Tesla has just unveiled the supercomputer it intends to use as the Dojo prototype. According to Andrej Karpathy, Director of AI at Tesla, the supercomputer consists of 5,760 GPUs that offer a performance of 1.8 EFLOPS (exaFLOPS) and are supported by 10 petabytes of NVMe storage with a connection speed of 1.6 TBps.
During a workshop presentation on autonomous driving at CVPR 2021, Karpathy explained how the LiDAR approach relies on creating an HD map in advance and then locating on that map while driving. Tesla's approach does everything locally and relies on the video feed from eight vehicle-mounted cameras. Karpathy says this is the much more difficult approach, but also much more scalable than the LiDAR+ HD map alternative because you just can't update the map data fast enough.
Tesla's solution is already advanced enough that the cameras do most of the heavy lifting and Karpathy confirmed cars started shipping three weeks ago without radar. The workshop video (start watching at the eight hour mark) shows you footage of the eight cameras Tesla relies on for the autonomous system. Now all Tesla needs to do is record lots of driving videos, store petabytes of data, and train its system to be competent and safe enough for all vehicles.
