Research on automotive vision algorithms: focusing on urban scenarios, BEV evolves into three technology routes.
1. What is BEV?
BEV (Bird's Eye View), also known as God's Eye View, is an end-to-end technology where the neural network converts image information from image space into BEV space.
Compared with conventional image space perception, BEV perception can input data collected by multiple sensors into a unified space for processing, acting as an effective way to avoid error superposition, and also makes temporal fusion easier to form a 4D space.
![视觉算法 1_副本.png](/UpLoads/Article/2023/视觉算法%201_副本.png)
BEV is not a new technology. In 2016, Baidu began to realize point cloud perception at the BEV; in 2021, Tesla’s introduction of BEV draw widespread attention in the industry. There are BEV perception algorithms corresponding to different sensor input layers, basic tasks, and scenarios. Examples include BEVFormer algorithm only based on vision, and BEVFusion algorithm based on multi-modal fusion strategy.
![视觉算法 2_副本.png](/UpLoads/Article/2023/视觉算法%202_副本.png)
2. Three technology routes of BEV perception algorithm
In terms of implementation of BEV technology, the technology architecture of each player is roughly the same, but technical solutions they adopt are different. So far, there have been three major technology routes:
Vision-only BEV perception route in which the typical company is Tesla;
BEV fused perception route in which the typical company is Haomo.ai;
Vehicle-road integrated BEV perception route in which the typical company is Baidu.
Vision-only BEV perception technology route: Tesla is a representative company of this technology route. In 2021, it was the first one to use the pre-fusion BEV algorithm for directly transmitting the image perceived by cameras into the AI algorithm to generate a 3D space at a bird's-eye view, and output perception results in the space. This space incorporates dynamic information such as vehicles and pedestrians, and static information like lane lines, traffic signs, traffic lights and buildings, as well as the coordinate position, direction angle, distance, speed, and acceleration of each element.
![视觉算法 3_副本.png](/UpLoads/Article/2023/视觉算法%203_副本.png)
Tesla uses the backbone network to extracts features of each camera. It adopts the Transformer technology to convert multi-camera data from image space into BEV space. Transformer, a deep learning model based on the Attention mechanism, can deal with massive data-level learning tasks and accurately perceive and predict the depth of objects.
![视觉算法 4_副本.png](/UpLoads/Article/2023/视觉算法%204_副本.png)
BEV fused perception technology route: Haomo.ai is an autonomous driving company under Great Wall Motor. In 2022, it announced an urban NOH solution that underlines perception and neglects maps. The core technology comes from MANA (Snow Lake).
In the MANA perception architecture, Haomo.ai adopts BEV fused perception (visual Camera + LiDAR) technology. Using the self-developed Transformer algorithm, MANA not only completes the transformation of vision-only information into BEV, but also finishes the fusion of Camera and LiDAR feature data, that is, the fusion of cross-modal raw data.
![视觉算法 5_副本.png](/UpLoads/Article/2023/视觉算法%205_副本.png)
Since its launch in late 2021, MANA has kept evolving. With Transformer-based perception algorithms, it has solved multiple road perception problems, such as lane line detection, obstacle detection, drivable area segmentation, traffic light detection & recognition, and traffic sign recognition.
In January 2023, MANA got further upgraded by introducing five major models to enable the transgenerational upgrade of the vehicle perception architecture and complete such tasks as common obstacle recognition, local road network and behavior prediction. The five models are: visual self-supervision model (automatic annotation of 4D Clip), 3D reconstruction model (low-cost solution to data distribution problems), multi-modal mutual supervision model (common obstacle recognition), dynamic environment model (using perception-focused technology for lower dependence on HD maps), and human-driving self-supervised cognition model (driving policy is more humane, safe and smooth).
![视觉算法 6_副本.png](/UpLoads/Article/2023/视觉算法%206_副本.png)
Vehicle-road integrated BEV perception technology route: in January 2023, Baidu introduced UniBEV, a vehicle-road integrated solution which is the industry's first end-to-end vehicle-road integrated perception solution.
Features:
Fusion of all vehicle and roadside data, covering online mapping with multiple vehicle cameras and sensors, dynamic obstacle perception, and multi-intersection multi-sensor fusion from the roadside perspective;
Self-developed internal and external parameters decoupling algorithm, enabling UniBEV to project the sensors into a unified BEV space regardless of how they are positioned on the vehicle and at the roadside
In the unified BEV space, it is easier for UniBEV to realize multi-modal, multi-view, and multi-temporal fusion of spatial-temporal features;
The big data + big model + miniaturization technology closed-loop remains superior in dynamic and static perception tasks at the vehicle side and roadside.
![视觉算法 7_副本.png](/UpLoads/Article/2023/视觉算法%207_副本.png)
Baidu’s UniBEV solution will be applied to ANP3.0, its advanced intelligent driving product planned to be mass-produced and delivered in 2023. Currently, Baidu has started ANP3.0 generalization tests in Beijing, Shanghai, Guangzhou and Shenzhen.
Baidu ANP3.0 adopts the "vision-only + LiDAR" dual redundancy solution. In the R&D and testing phase, with the "BEV Surround View 3D Perception" technology, ANP3.0 has become an intelligent driving solution that enables multiple urban scenarios solely relying on vision. In the mass production stage, ANP3.0 will introduce LiDAR to realize multi-sensor fused perception to deal with more complex urban scenarios.
3. BEV perception algorithm favors application of urban NOA.
As vision algorithms evolve, BEV perception algorithms become the core technology for OEMs and autonomous driving companies such as Tesla, Xpeng, Great Wall Motor, ARCFOX, QCraft and Pony.ai, to develop urban scenarios.
Xpeng Motors: the new-generation perception architecture XNet can fuse the data collected by cameras before multi-frame timing, and output 4D dynamic information (e.g., vehicle speed and motion prediction) and 3D static information (e.g., lane line position) at the BEV.
Pony.ai: In January 2023, it announced the intelligent driving solution - Pony Shitu. The self-developed BEV perception algorithm, the key feature of the solution, can recognize various types of obstacles, lane lines and passable areas, minimize computing power requirements, and enable highway and urban NOA only using navigation maps.
![视觉算法 8_副本.png](/UpLoads/Article/2023/视觉算法%208_副本.png)
Chinese OEMs (Passenger Car) Going Overseas Report, 2024--Germany
Keywords of Chinese OEMs going to Germany: electric vehicles, cost performance, intelligence, ecological construction, localization
The European Union's temporary tariffs on electric vehicles in Chi...
Analysis on DJI Automotive’s Autonomous Driving Business, 2024
Research on DJI Automotive: lead the NOA market by virtue of unique technology route.
In 2016, DJI Automotive’s internal technicians installed a set of stereo sensors + vision fusion positioning syst...
BYD’s Layout in Electrification, Connectivity, Intelligence and Sharing and Strategy Analysis Report, 2023-2024
Insight: BYD deploys vehicle-mounted drones, and the autonomous driving charging robot market is expected to boom.
BYD and Dongfeng M-Hero make cross-border layout of drones.
In recent years,...
Great Wall Motor’s Layout in Electrification, Connectivity, Intelligence and Sharing and Strategy Analysis Report, 2023-2024
Great Wall Motor (GWM) benchmarks IT giants and accelerates “Process and Digital Transformation”.
In 2022, Great Wall Motor (GWM) hoped to use Haval H6's huge user base to achieve new energy transfo...
Cockpit AI Agent Research Report, 2024
Cockpit AI Agent: Autonomous scenario creation becomes the first step to personalize cockpits
In AI Foundation Models’ Impacts on Vehicle Intelligent Design and Development Research Report, 2024, Res...
Leading Chinese Intelligent Cockpit Tier 1 Supplier Research Report, 2024
Cockpit Tier1 Research: Comprehensively build a cockpit product matrix centered on users' hearing, speaking, seeing, writing and feeling.
ResearchInChina released Leading Chinese Intelligent Cockpit ...
Global and China Automotive Wireless Communication Module Market Report, 2024
Communication module and 5G research: 5G module installation rate reaches new high, 5G-A promotes vehicle application acceleration
5G automotive communication market has exploded, and 5G FWA is evolv...
ADAS and Autonomous Driving Tier 1 Suppliers Research Report, 2024 – Chinese Companies
ADAS Tier1s Research: Suppliers enter intense competition while exploring new businesses such as robotics
In China's intelligent driving market, L2 era is dominated by foreign suppliers. Entering era...
Automotive Gateway Industry Report, 2024
Automotive gateway research: 10BASE-T1S and CAN-XL will bring more flexible gateway deployment solutions
ResearchInChina released "Automotive Gateway Industry Report, 2024", analyzing and researching...
Global and China Electronic Rearview Mirror Industry Report, 2024
Research on electronic rearview mirrors: electronic internal rearview mirrors are growing rapidly, and electronic external rearview mirrors are facing growing pains
ResearchInChina released "Global a...
Next-generation Zonal Communication Network Topology and Chip Industry Research Report, 2024
The in-vehicle communication architecture plays a connecting role in automotive E/E architecture. With the evolution of automotive E/E architecture, in-vehicle communication technology is also develop...
Autonomous Delivery Industry Research Report, 2024
Autonomous Delivery Research: Foundation Models Promote the Normal Application of Autonomous Delivery in Multiple Scenarios
Autonomous Delivery Industry Research Report, 2024 released by ResearchInCh...
Global Autonomous Driving Policies & Regulations and Automotive Market Access Research Report, 2024
Intelligent driving regulations and vehicles going overseas: research on regional markets around the world and access strategies. "Going out”: discussion about regional markets aroun...
China Passenger Car HUD Industry Report, 2024
HUD research: AR-HUD accounted for 21.1%; LBS and optical waveguide solutions are about to be mass-produced. The automotive head-up display system (HUD) uses the principle of optics to display s...
Ecological Domain and Automotive Hardware Expansion Research Report, 2024
Automotive Ecological Domain Research: How Will OEM Ecology and Peripheral Hardware Develop? Ecological Domain and Automotive Hardware Expansion Research Report, 2024 released by ResearchInChina ...
C-V2X and CVIS Industry Research Report, 2024
C-V2X and CVIS Research: In 2023, the OEM scale will exceed 270,000 units, and large-scale verification will start.The pilot application of "vehicle-road-cloud integration” commenced, and C-V2X entere...
Automotive Intelligent Cockpit Platform Configuration Strategy and Industry Research Report, 2024
According to the evolution trends and functions, the cockpit platform has gradually evolved into technical paths such as cockpit-only, cockpit integrated with other domains, cockpit-parking integratio...
Analysis on Huawei's Electrification, Connectivity, Intelligence and Sharing,2023-2024
Analysis on Huawei's Electrification, Connectivity, Intelligence and Sharing: Comprehensive layout in eight major fields and upgrade of Huawei Smart Selection
The “Huawei Intelligent Driving Business...