AGRICULTURE
Smart Eyes See the Road to Autonomy
Combining Visual Odometry and GNSS+INS for Continuous Localization and Safety
While GNSS Data provides absolute positioning in open-sky areas closed structures and urban environments present a challenge for “satellite navigation” alone.
Recently, Hexagon | NovAtel integrated its high-precision positioning engine and TerraStar X correction technology with automotive-grade multifrequency GNSS chipsets, the Teseo APP and Teseo V from STMicroelectronics. The collaboration demonstrated GNSS-based precise localization on real roads in preparation for deployment of driverless cars.
Now the two companies are joined by AImotive, a vision-first software and hardware IP provider for automated driving, to meet the challenges associated with driving in any environment.
Recent demonstrations of the resulting single localization precise positioning platform shows tremendous promise for the advancement of both Advanced Driver Assistance Systems(ADAS) functionalities and SAE L4autonomy, High Automation (Figure 1 - see the PDF for Figure 1).
Inside the Stack
The key components of the multi-technology localization platform includes the software positioning engine from Novatel with TerraStar X correction technology, an ST Teseo V/APP automotive grade receiver GNSS/IMU chipset, and AImotive’s aiDrive, a modular software powered by artificial intelligence (AI) that integrates data from multi-frequency GNSS, high-precision inertial measurement units and visual odometry.
To understand how these all work together requires a look inside the powerful aiDrive software stack.
The software is comprised of five engines (Figure 2 - see the PDF for Figure 2), each responsible for a set of tasks within self-driving from perception through localization to control, according to AImotive. Each engine is divided into modules responsible for selected processes. Certain modules rely on AI for improved perception and decision-making performance, while formal algorithms throughout the system ensure safety.
At the heart of aiDrive is visual odometry. Visual odometry is a technique that uses multiple cameras to recognize the environment many times per second, and then correlates that data precisely with vehicle movement. The software uses the concept of ego motion—the 3D motion of a camera within an environment—to estimate the car’s moving position relative to lines on the road, parked vehicles or street signs observed from the car itself.
A simultaneous set of single images taken from different angles on the vehicle identifies features that are unique and tracks them across subsequent images. By back-calculation, the AI software tracks vehicle movement. The algorithm focuses on fusing visual and IMU data and other vehicle inputs such as wheel-tick information. Typically, the cameras record at 20 frames per second(20Hz), while the software reads the IMU at a standard 125 Hz setting and the wheel ticks at 40 Hz. However, these settings are optional and can be changed according to the user’s needs.
The aiDrive software can accelerate automated driving goals, regardless of platform. It integrates mono, stereo and fisheye camera setups and data gathered from sensors. As well, each addition of a sensor layer makes the system more robust. For instance, an optional step is to localize the data from the sensors to correspond with high-definition maps. As it is hardware-, middleware- and OS-agnostic with a modular architecture, aiDrive can be deployed as the core of standalone autonomous driving solutions, as part of ADAS solutions, or integrated by module.
Here’s how aiDrive works with the software positioning engine from NovAtel and automotive grade receiver chipsets.
Localization Inside and Out
The multi-technology localization platform incorporates a number of solutions working together as shown in Figure 3.
The ST Teseo GNSS and inertial sensors collect raw code and carrier phase GNSS measurements as well as acceleration and angular rate measurements from the IMU. Of note, STMicroelectronics’ new family of GNSS automotive-grade chipsets are specifically designed to support very demanding automotive applications with high-accuracy positioning, and navigation in most environmental conditions. The Teseo V and Teseo APP embed the latest STMicroelectronics GNSS engine innovation, which handles simultaneously multiple-frequency signals, and multi-constellation GNSS.
That data is processed by Novatel’s software positioning engine with TerraStar X correction technology to calculate the position of the vehicle with lane-level accuracy. The utilization of TerraStar X technology facilitates convergence time of under one minute to reach lane-level accuracy. Integrated with aiDrive’s AI-powered visual odometry, that same positioning is possible inside, for instance, a parking garage. For the localization platform, the ST Teseo data and the corrected measurements from the NovAtel software positioning engine are shared with the aiDrive autonomous driving software stack.
Road Testing Builds Trust
Recently, the three partners—NovAtel, STMicroelectronics and AImotive—put a proof of concept localization platform to a driverless test in difficult environments to demonstrate reliability and integrity and to build robustness of the solutions individually and together.
The platform operated through several loops in an underground parking garage, where a fusion of the visual odometry and IMU data provided continuous precise localization in a satnav-deprived environment. See the video “Solving localization for automated driving” (https://www.youtube.com/watch?v=VIIsXrpUBm4&feature=youtu.be)
Figures 4 and 5 (see the full PDF for Figures 4 and 5) show the results in progress, with yellow as that same data integrated with the AImotive visual odometry. In gray—largely hidden by the yellow track—is the ground truth.
In closed structure scenarios, continuous precise localization is handled by the fusion of visual odometry and IMU data. This ensures that accurate positioning information is available even several stories underground or in closed parking structures. When using the joint solution in a two-story underground parking garage, the absolute drift remained under 1 meter (.79m) after 340 meters driven. Upon exiting the garage, the overall longitudinal drift of the GNSS+INS solution was 21 meters, while that of the visual odometry came to 0.8 meters.
Similar results to the underground parking structure (Figure 5)were achieved in stop-and-go traffic through a tunnel. In a tunnel scenario, the absolute drift remained under the1 meter threshold while the GNSS+INS solution rose toward a maximum of10.45 meters, until it reacquired a GNSS signal upon exiting the tunnel and gradually returned to a near-accurate solution.
These scenarios demonstrate that the NovAtel/ST/AImotive solution provides accurate positioning even in environments where satellite connections are disrupted.
The continuous localization platform developed by NovAtel, STMicroelectronics and AImotive is designed to be ISO 26262 compliant.
Through a third-party audit, NovAtel assures process compliance with key automotive specifications ISO/TS 16949 and ISO 26262Functional Safety Design Assurance. In 2020, NovAtel completed an internal upgrade of the ISO/TS16949:2009 process framework to IATF 16949:2016. The company will also complete the implementation of the ASPICE SW process quality frame-work by December 2020.
STMicroelectronics’ Teseo APP is the world’s first ADAS/Autonomous Driving GNSS sensor, answering all positioning, integrity, and ISO26262security requirements for applications such as lane-departure warning, adaptive-cruise control (ACC), and high-accuracy functional safety for autonomous driving.
Similarly, AImotive follows industry standards such as ISO 26262and SOTIF, and aiSim achieved ISO26262 tool certification.
The interconnected development of the NovAtel, STMicroelectronics and AImotive technologies forms a modular platform to facilitate collaboration in the self-driving industry and leads to the creation of the safest automated driving systems possible.
Read the full PDF here: