From mid- to late-2020, Elon Musk has been making brief comments about 4D data gathering as a giant, “quantum” leap towards full autonomous driving. But few have taken notice of these bold statements.

Main Point: We think it's necessary to draw attention to Elon Musk’s comments on 4D by compiling all his comments in one place. These are bold statements for investors and customers to reflect upon.
In future posts, we will explain 4D data gathering/4D machine learning and draw out the significance of 4D for Tesla’s progress towards full autonomous driving. 

During the 2020 Q2 Earnings Call on July 22, 2020, Musk explained the improvements in Full Self Driving (FSD):

       "Well, the actual major milestone that's happening right now is really a transition of the autonomy system or the cars, like AI, if you will, from thinking about things in -- like two-and-a-half feet. It's like think -- things like isolated pictures and doing image recognition on pictures that are harshly correlated in time but not very well and transitioning to kind of a 4D, where it's like -- which is video essentially. You're thinking about the world in three dimensions and the fourth dimension being time. So that architectural change, which has been under way for some time but has not really been rolled out to anyone in the production fleet, is what really matters for Full Self-Driving.

        So what we've been doing, thus far, has really just been like 2D -- mostly 2D, and like I said, well correlated in time. So just hard to convey just how much better a fully 4D system would work -- does work. It's capable of things that if you just look -- looking at things as individual pictures as opposed to video -- basically, like you could go from like individual pictures to surround video, so it's fundamental. So the car will seem to have just like a giant improvement.

        We'll probably roll it out later this year. We'll be able to do traffic lights, stop, turns, thrust, everything, pretty much. And then it will be a long march of nines, essentially. How many nines of reliability are OK."

---

Since then, Musk has been dropping more tiny comments about FSD with 4D data on Twitter:

Notice how Musk says this new phase of FSD is a “fundamental architectural rewrite, not an incremental tweak.”


Musk links to this article: Tesla Published Patent “Generating Ground Truth For Machine Learning From Time Series Elements”


Musk is referring to a patent that Tesla filed on February 1, 2019, and finally published on August 6, 2020, entitled, “Generating ground truth for machine learning from time series elements” (available here), which details the science behind this next phase of FSD. This patent is obviously extremely important and merits further discussion later.

‍

On September 22, 2020, during Tesla’s Battery Day presentation, Musk said:

        "So we are now labeling in 3D video, so this is hugely different from the previously where we were labeling essentially a bunch of single images from the eight cameras, and they would be labeled at different times by different people, and some of the labels, you literally can’t tell what it is you’re labeling. So it basically made it sort of in some cases impossible to label, and the labels had a lot of errors. Now with our new labeling tools, we label it in video, so we actually label entire video segments in the system, so you get basically a surround video thing to label with the surround video and with time. So it’s now taking all cameras simultaneously and looking at how the image has changed over time and labeling that, and then the sophistication of the neural nets in the car and the overall logic in the car has improved dramatically.

        I think we’ll hopefully release a private beta of autopilot, of the full self-driving version of autopilot in, I think, a month or so, and then people will really understand just the magnitude of the change. It’s profound. So, yeah. Anyway, so you’ll see it. It’s just like a hell of a step change, but because we had to rewrite everything, labeling software, just the entire code base, it took us quite a while. The sort of new … I call it like 4D in the sense that it’s three dimensions plus time. It’s just taken us a while to rewrite everything, and so you’ll see what it’s like. It’s amazing. Yeah. It’s just clearly going to work."

---

Notice how Musk once again says that they had to “rewrite everything, labeling software, just the entire code base.” This new 4D FSD is not a small step, but a complete tear down and full rebuild in light of improved 4D data gathering and processing capabilities. 


Musk defines 4D as “three dimensions plus time.” With 4D, notice the change from labeling individual images to labeling video segments, which allows for a much richer, human-like understanding of the world and the complex behavior of other cars, animals, cyclists, and pedestrians.

‍

Musk is responding positively to this article: “Why Tesla’s Full Self-Driving Is Such A Big Deal: 4D Data Continuity & Trajectory Projection” This is an important article that merits further discussion later on.

‍

‍
These bold comments have been made in public for all to see and evaluate. 

‍

However -- these comments have received little attention, or at least have not received the attention they deserve. For example, research analyst Rob Hansen of Vontobel Quality Growth writes in his January 20, 2021, opinion piece, “Why Tesla is not a safe stock for long-term investors”: 

       "Tesla is the lone creator of autonomous vehicle technology that eschews an important vision system called Lidar (Light Detection & Ranging). This system allows a vehicle’s computer to create a 3D representation of the environment around a vehicle and provide a higher level of safety. The vast majority of experts agree that neglecting this technology in favor of cheaper radar and camera-based options poses safety risks longer term."


Financial analysts are prone to overlook and/or misunderstand the engineering aspects of tech companies like Tesla. Hansen seems unaware of the 4D technology that Tesla is developing and he follows the crowd in believing that LiDAR’s 3D mapping capabilities will be the key to safety for autonomous cars.


While many are unaware of Tesla’s 4D developments, others ignore these 4D developments because they don’t understand what 4D is, don’t understand how much richer 4D data is than 2D/3D data, and don’t understand how processing 4D data contributes to vastly improved and safer autonomous driving, “especially in large intersections with dense vehicle & pedestrian traffic” (tweeted by Elon Musk). 


In other words, the Fall 2020 release of Tesla’s Full Self Driving (FSD) Beta with its 4D capabilities was not a small step forward, but a giant leap forward with regards to safety and driving in complex situations. In future articles, we will explain what 4D data is and why it is the key to safe and excellent autonomous driving in complex, crowded situations.

Takeaway: Tesla is pulling far ahead of its rivals with 4D data, but few are noticing.‍