Why FIR sensing technology is essential for achieving fully autonomous vehicles
FIR technology is necessary for the deployment and mass market adoption of autonomous vehicles because it is the only sensor capable of providing the complete and reliable coverage needed to make AVs safe. Major automotive OEM, BMW is using thermal-imaging cameras as part of their sensor suites for all self-driving prototypes.
While other optical sensors used on cars only capture images visible to the human eye, FIR cameras provide a more comprehensive layer of perception. By scanning the infrared spectrum just above visible light, FIR cameras detect objects that a camera, radar, or LiDAR may miss. Moreover, unlike radar and LiDAR sensors that must transmit and receive signals, a FIR camera only collects signals, making it a “passive” technology. With no moving parts, a FIR camera can provide complete coverage of an AV’s surroundings simply by sensing signals from objects radiating heat.
Currently, there are three leading FIR sensor companies: Autoliv, FLIR systems and AdaSky. AdaSky is an Israeli startup that recently developed Viper, a high-resolution thermal camera that passively collects FIR signals, converts it to a high-resolution VGA video, and applies deep-learning computer vision algorithms to sense and analyze its surroundings.
click for larger image
Fig 3 A side-by-side comparison of a state-of-the-art camera with low light sensitivity and Viper shows objects undetected by current sensing technology are visible with a FIR solution. (Source: AdaSky)
With this advanced technology, FIR cameras can overcome the obstacles presented by complicated weather and lighting conditions that confound other sensing technologies. Compared to FIR cameras, today’s sensors are limited and no longer seem capable of delivering full autonomy. Even working together, current sensors cannot provide total or accurate coverage of a vehicle’s surroundings in every situation. Only FIR sensors can generate the needed, deeper layer of information that originates from a different band of electromagnetic spectrum to significantly increase performance for classification, identification, and detection of objects and of vehicle surroundings, both at near and far range.
By creating a visual representation of the vehicle’s surroundings, FIR fills the gaps left by other sensors to produce total detection and coverage in any weather condition and in any environment, whether it be urban, rural, or highway driving—or a combination of all three. For example, on the highway, it’s crucial to have long-ranging sensing so that if an object is detected, there is ample time for the vehicle to make the decision to stop—even as it travels at high speeds. In urban areas, having a wider field-of-view is prioritized to be able to detect pedestrians and cyclists on the sidewalk and at crosswalks.
While FIR is needed for Level-3 autonomous solutions, it is an essential enabler to Level-4 and up. To achieve Level-5 autonomy and, ultimately, bring fully autonomous vehicles to the mass market, AV developers anticipate that each vehicle should be equipped with several FIR cameras to enable wide coverage and a comprehensive understanding of its surroundings. Automakers favor using multiple FIR sensors because it delivers the highest level of safety. In fact, the U.S. Department of Transportation’s Federal Automated Vehicles policy requires redundancy for certain critical AV systems, and most OEM and tier-ones are gearing up to use multiple sensors and other components as fail-safe measures.
Every sensor has a scenario where it encounters weakness. No sensor is 100 percent accurate 100 percent of the time including FIR. This is why it is so crucial to have redundancy through a sensor fusion solution that combines LiDAR, Radar and FIR technology. As you can see in the diagram below, when you layer all of these technologies into a sensor fusion solution, you will get coverage for all areas and every scenario. This is how FIR will enable fully autonomous driving.
click for larger image
Fig 4. Sensor modalities for autonomous vehicles – coverage from CMOS, Radar, LiDAR and FIR. (Source: AdaSky)
Automakers’ goal of deploying autonomous vehicles on public roads by the beginning of the next decade cannot be achieved with the limited capabilities of today’s sensing solutions. Their persistent perception problems mean that vehicles cannot operate safely and reliably without the monitored control of a human driver, making true Level-5 autonomy impossible. FIR cameras are the only technology that can deliver complete classification, identification, and detection of a vehicle’s surroundings in any environment or weather condition and are, thus, the only sensing technology that can make the mass market adoption of fully autonomous vehicles a reality.
Yakov Shaharabani, CEO and Board Member for AdaSky, is a strategic thinker with extensive experience as a leader dealing with complex environments and highly tense situations. Much of Yakov’s leadership and strategic experience was gained in 34 years of service in the Israeli Air Force, from a young pilot, through the positions of a squadron leader and a base commander, up to the very senior positions, as one of the very few Generals in the Israeli Air Force. Yakov is the founder of SNH Strategies LTD, a company focusing on strategic consulting and strategic leadership education. He earned his B.S with honors in economics and computer sciences, and M.A. in National Resource Strategy (Cum Laude), The National Defense University (NDU), Washington D.C.