Research on automotive vision algorithms: focusing on urban scenarios, BEV evolves into three technology routes.
1. What is BEV?
BEV (Bird's Eye View), also known as God's Eye View, is an end-to-end technology where the neural network converts image information from image space into BEV space.
Compared with conventional image space perception, BEV perception can input data collected by multiple sensors into a unified space for processing, acting as an effective way to avoid error superposition, and also makes temporal fusion easier to form a 4D space.
BEV is not a new technology. In 2016, Baidu began to realize point cloud perception at the BEV; in 2021, Tesla’s introduction of BEV draw widespread attention in the industry. There are BEV perception algorithms corresponding to different sensor input layers, basic tasks, and scenarios. Examples include BEVFormer algorithm only based on vision, and BEVFusion algorithm based on multi-modal fusion strategy.
2. Three technology routes of BEV perception algorithm
In terms of implementation of BEV technology, the technology architecture of each player is roughly the same, but technical solutions they adopt are different. So far, there have been three major technology routes:
Vision-only BEV perception route in which the typical company is Tesla;
BEV fused perception route in which the typical company is Haomo.ai;
Vehicle-road integrated BEV perception route in which the typical company is Baidu.
Vision-only BEV perception technology route: Tesla is a representative company of this technology route. In 2021, it was the first one to use the pre-fusion BEV algorithm for directly transmitting the image perceived by cameras into the AI algorithm to generate a 3D space at a bird's-eye view, and output perception results in the space. This space incorporates dynamic information such as vehicles and pedestrians, and static information like lane lines, traffic signs, traffic lights and buildings, as well as the coordinate position, direction angle, distance, speed, and acceleration of each element.
Tesla uses the backbone network to extracts features of each camera. It adopts the Transformer technology to convert multi-camera data from image space into BEV space. Transformer, a deep learning model based on the Attention mechanism, can deal with massive data-level learning tasks and accurately perceive and predict the depth of objects.
BEV fused perception technology route: Haomo.ai is an autonomous driving company under Great Wall Motor. In 2022, it announced an urban NOH solution that underlines perception and neglects maps. The core technology comes from MANA (Snow Lake).
In the MANA perception architecture, Haomo.ai adopts BEV fused perception (visual Camera + LiDAR) technology. Using the self-developed Transformer algorithm, MANA not only completes the transformation of vision-only information into BEV, but also finishes the fusion of Camera and LiDAR feature data, that is, the fusion of cross-modal raw data.
Since its launch in late 2021, MANA has kept evolving. With Transformer-based perception algorithms, it has solved multiple road perception problems, such as lane line detection, obstacle detection, drivable area segmentation, traffic light detection & recognition, and traffic sign recognition.
In January 2023, MANA got further upgraded by introducing five major models to enable the transgenerational upgrade of the vehicle perception architecture and complete such tasks as common obstacle recognition, local road network and behavior prediction. The five models are: visual self-supervision model (automatic annotation of 4D Clip), 3D reconstruction model (low-cost solution to data distribution problems), multi-modal mutual supervision model (common obstacle recognition), dynamic environment model (using perception-focused technology for lower dependence on HD maps), and human-driving self-supervised cognition model (driving policy is more humane, safe and smooth).
Vehicle-road integrated BEV perception technology route: in January 2023, Baidu introduced UniBEV, a vehicle-road integrated solution which is the industry's first end-to-end vehicle-road integrated perception solution.
Features:
Fusion of all vehicle and roadside data, covering online mapping with multiple vehicle cameras and sensors, dynamic obstacle perception, and multi-intersection multi-sensor fusion from the roadside perspective;
Self-developed internal and external parameters decoupling algorithm, enabling UniBEV to project the sensors into a unified BEV space regardless of how they are positioned on the vehicle and at the roadside
In the unified BEV space, it is easier for UniBEV to realize multi-modal, multi-view, and multi-temporal fusion of spatial-temporal features;
The big data + big model + miniaturization technology closed-loop remains superior in dynamic and static perception tasks at the vehicle side and roadside.
Baidu’s UniBEV solution will be applied to ANP3.0, its advanced intelligent driving product planned to be mass-produced and delivered in 2023. Currently, Baidu has started ANP3.0 generalization tests in Beijing, Shanghai, Guangzhou and Shenzhen.
Baidu ANP3.0 adopts the "vision-only + LiDAR" dual redundancy solution. In the R&D and testing phase, with the "BEV Surround View 3D Perception" technology, ANP3.0 has become an intelligent driving solution that enables multiple urban scenarios solely relying on vision. In the mass production stage, ANP3.0 will introduce LiDAR to realize multi-sensor fused perception to deal with more complex urban scenarios.
3. BEV perception algorithm favors application of urban NOA.
As vision algorithms evolve, BEV perception algorithms become the core technology for OEMs and autonomous driving companies such as Tesla, Xpeng, Great Wall Motor, ARCFOX, QCraft and Pony.ai, to develop urban scenarios.
Xpeng Motors: the new-generation perception architecture XNet can fuse the data collected by cameras before multi-frame timing, and output 4D dynamic information (e.g., vehicle speed and motion prediction) and 3D static information (e.g., lane line position) at the BEV.
Pony.ai: In January 2023, it announced the intelligent driving solution - Pony Shitu. The self-developed BEV perception algorithm, the key feature of the solution, can recognize various types of obstacles, lane lines and passable areas, minimize computing power requirements, and enable highway and urban NOA only using navigation maps.
Global and China Automotive Operating System (OS) Industry Report, 2023-2024
Chinese operating systems start to work hard In 2023, Chinese providers such as Huawei, Banma Zhixing, Xiaomi, and NIO made efforts in operating system market, launched different versions with competi...
Automotive RISC-V Chip Industry Research Report, 2024
Automotive RISC-V Research: Customized chips may become the future trend, and RISC-V will challenge ARM
What is RISC-V?Reduced Instruction Set Computing - Five (RISC-V) is an open standard instructio...
Passenger Car CTP (Cell to Pack), CTC (Cell To Chassis) and CTB (Cell to Body) Integrated Battery Industry Report, 2024
Passenger Car CTP, CTC and CTB Integrated Battery Industry Report, 2024 released by ResearchInChina summarizes and studies the status quo of CTP (Cell to Pack), CTC (Cell To Chassis) and CTB (Cell to ...
Software-defined Vehicle Research Report, 2023-2024 - Industry Panorama and Strategy
1. How to build intelligent driving software-defined vehicle (SDV) architecture?
The autonomous driving intelligent platform can be roughly divided into four parts from the bottom up: hardware platf...
Automotive DMS/OMS (Driver/Occupant Monitoring System) Research Report, 2023-2024
In-cabin Monitoring study: installation rate increases by 81.3% in first ten months of 2023, what are the driving factors?
ResearchInChina released "Automotive DMS/OMS (Driver/Occupant Monitoring Sys...
Automotive Functional Safety and Safety Of The Intended Functionality (SOTIF) Research Report, 2024
As intelligent connected vehicles boom, the change in automotive EEA has been accelerated, and the risks caused by electronic and electrical failures have become ever higher. As a result, functional s...
Autonomous Driving Map Industry Report,2024
As the supervision of HD map qualifications tightens, issues such as map collection cost, update frequency, and coverage stand out. Amid the boom of urban NOA, the "lightweight map" intelligent drivin...
Automotive Vision Industry Research Report, 2023
From January to September 2023, 48.172 million cameras were installed in new cars in China, a like-on-like jump of 34.1%, including:
9.209 million front view cameras, up 33.0%; 3.875 million side vi...
Automotive Voice Industry Report, 2023-2024
The automotive voice interaction market is characterized by the following:
1. In OEM market, 46 brands install automotive voice as a standard configuration in 2023.
From 2019 to the first nine month...
Two-wheeler Intelligence and Industry Chain Research Report, 2023
In recent years, two-wheelers have headed in the direction of intelligent connection and intelligent driving, which has been accompanied by consumption upgrade, and mature applications of big data, ar...
Commercial Vehicle Telematics Industry Report, 2023-2024
The market tends to be more concentrated in leading companies in terms of hardware.
The commercial vehicle telematics industry chain covers several key links such as OEMs, operators, terminal device ...
Automotive Camera Tier2 Suppliers Research Report, 2023
1. Automotive lens companies: "camera module segment + emerging suppliers" facilitates the rise of Chinese products.
In 2023, automotive lens companies still maintain a three-echelon pattern. The fir...
China Passenger Car Navigate on Autopilot (NOA) Industry Report, 2023
Intelligent driving is evolving from L2 to L2+ and L2++, and Navigate on Autopilot (NOA) has become a layout focus in the industry. How is NOA advancing at present? What are hotspots in the market? Wh...
Automotive Telematics Service Providers (TSP) and Application Services Research Report, 2023-2024
From January to September 2023, the penetration of telematics in passenger cars in China hit 77.6%, up 12.8 percentage points from the prior-year period. The rising penetration of telematics provides ...
Passenger Car Intelligent Chassis and Chassis Domain Controller Research Report, 2023
Passenger Car Intelligent Chassis and Chassis Domain Controller Research Report, 2023, released by ResearchInChina combs through three integration trends of brake-by-wire, steer-by-wire, and active su...
Automotive Smart Cockpit Design Trend Report, 2023
As the most intuitive window to experience automotive intelligent technology, intelligent cockpit is steadily moving towards the deep end of “intelligence”, and automakers have worked to deploy intell...
China Automotive Multimodal Interaction Development Research Report, 2023
China Automotive Multimodal Interaction Development Research Report, 2023 released by ResearchInChina combs through the interaction modes of mainstream cockpits, the application of interaction modes i...
Automotive Smart Surface Research Report, 2023
Market status: vehicle models with smart surfaces boom in 2023
From 2018 to 2023, there were an increasing number of models equipped with smart surfaces, up to 52,000 units in 2022 and 256,000 units ...