AI-SEE

AI-SEE [Artificial Intelligence enhancing vehicle vision in low visibility conditions]

The automotive industry is facing one of the most demanding challenges in its history: how to make automated travel safe in all conditions. There have been great advances towards automation with new vehicles increasingly equipped with driver assistance systems (ADAS). The biggest barrier now remaining to full automation is safe driving under poor weather and low visibility. The AI-SEE project aims to build a novel, robust sensing system supported by Artificial Intelligence (AI) that will enable automated travel in varied traffic, lighting and weather conditions. It will extend the Operational Design Domain (ODD) of automated vehicles (i.e. the scope of what they can do), taking the technology from SAE level 3 conditional automation) to level 4 (high automation) where vehicles drive themselves with no human interaction in most circumstances.

With advanced and autonomous vehicles entering the market, solving problems linked to illumination and weather conditions such as rain, fog and snow is key to ensuring a safe environment for drivers, passengers and pedestrians. However, to move from level 3 to level 4 requires solutions to four key challenges: (i) mass-production of powerful computing platforms (ii) improved sensing capabilities and lower-cost sensors (iii) necessary technical standards and (iv) infrastructure. AI-SEE is focusing primarily on the second challenge by increasing the environmental and situational awareness of vehicles.

Humans ‘see’ by combining stored memories and sensory input to interpret events and anticipate upcoming scenarios. Today’s automated vehicles cannot yet provide this inferential thinking, nor communicate in real-time with the environment. For automated vehicles to drive without human intervention, the information content from current sensors needs to be enhanced significantly. But this will create an increasingly large amount of data transmitted at huge data rates which, along with all the additional sensors, will quickly exceed the limits of in-vehicle storage space, and vehicle computational and energy resources.

Together, the high number of sensors needed for 360 degree environment perception and situation awareness and the high cost of LiDAR (Light Detection & Ranging) used for measuring distances to objects, represent significant barriers to the wider roll out of automated driving.^[1]

Taking technologies to the next level

AI-SEE will address these challenges by combining complex hardware and software development, creating automotive perception systems that go beyond today’s state-of-the-art. Its goal is to introduce reliable, secure, trustable sensors and software by implementing self-diagnosis, adaptation and robustness.

The AI-SEE concept is built on four main blocks:

1. A 24/365 high resolution adaptive all-weather sensor suite
2. An AI platform for predictive detection of prevailing environmental conditions including signal enhancement and sensor adaptation
3. Smart sensor data fusion to create the 24/365 adaptive all-weather robust perception system
4. A demonstrator and system validation plan, with testing carried out in simulations and in real-world environments in northern Europe

The project will deliver the first high-resolution adaptive multi-sensor suite building on ainnovativenovel AI perception-processing scheme for low visibility conditions.

Specifically, AI-SEE will create novel sensor hardware comprising an active polarimetric imager and congruent LIDAR data; a short-wave infrared (SWIR) LIDAR with a novel SPAD receiver architecture; a high resolution 4D MIMO radar and a gated SWIR-camera. To support the novel sensing system and improve localization performance in poor weather, the project will also take high definition (HD) dynamic mapping to a new level. In addition, to handle the multisensory data fusion, an AI platform will be built to advance early signal enhancement for robust perception.

Importantly, the project will develop sensor-near simulation models for all active sensors for artificial generation of synthetic inclement weather datasets. This is expected to revolutionise simulation, with conversion of good weather neural network datasets into inclement weather datasets – thereby saving large amounts of money and time in testing and validating inclement weather sensor performance. Moreover, a large outdoor weather data bank for testing, modelling and validation will also be created. All of which will lead to a paradigm shift in signal enhancement techniques and a competitive advantage for the European automotive industry.

Partnership beyond traditional automotive networks

The AI-SEE fault tolerant environment perception system and its sub-system are highly complex. Bringing them to the market calls for partnership beyond the traditional network of automotive OEMs and Tier 1 suppliers. To tackle the challenges of new hardware, data collection, and AI-supported signal enhancement and simulations, the AI-SEE project includes OEMs; Tier 1, 2 and 3 suppliers; and smaller engineering companies as well as academic and research institutes. Together, these partners not only have the necessary expertise, but also unique testing and development capabilities not found in any one European country alone.

Positioned for rapidly evolving market opportunities

The emergence of AD is transforming the automotive industry, bringing in new players such as online service providers and IT and telecommunication suppliers. However, the market outlook remains unclear. Predictions range from sales of two million L3+ vehicles in 2030 to 63 million in the same time period[1] . Nonetheless, automotive sensor sales are expected to grow at an average rate of 8%, with 14% growth in sales value, up to 2022. Overall, the sensor market was worth USD 11 billion in 2016 and it is expected to reach USD 23 billion by 2022, mainly due to the boom in imaging, radar and LiDAR sensors, which will respectively be worth USD 7.7 billion, USD 6.2 billion and USD 1.4 billion by 2022.^[2]

Being fast and focused will be key for success in this rapidly evolving landscape. AI-SEE will contribute by giving Europe a vital first in cutting-edge technologies for environmental perception. It will also allow European companies to compete in the supply of sensors related to adverse weather conditions where US exports are limited by security (defence technology) concerns. Furthermore, it will help Europe maintain its strong position (close to 40%) in the fast-growing LIDAR market, expected to reach USD 2.6 billion by 2030,^[3] and to seize opportunities in areas such as automotive night vision systems and automotive radar. In addition, AI-SEE’s results will support standardisation, which is essential for deployment of automated driving systems on a wide scale.

Automotive is one of Europe’s key industries accounting for 6.8% of EU GDP and 13.3 million jobs. Through its outcomes, AI-SEE will help Europe to retain its world-leading strengths in this domain, thereby safeguarding high-value jobs, economic growth and indirectly the social well-being of EU citizens.

^[1] Prices for individual LiDAR-sensors can reach up to 10 000 €. Garmin, 2019. LIDAR-Lite v3HP a Low-Cost Solution to Autonomous Building-Interior Mapping.

^[2] Electronic Specifier (2020). Sensing changes in the automotive sensor market.

^[3]   KnowMade (2018).  LiDAR for Automotive Patent Landscape.