Socionext and Tohoku University Significantly Accelerate Deep Learning-based SLAM Processing

Yokohama, Japan, October 12, 2021 --- Socionext Inc. has collaborated with a research group from Tohoku University, led by Professor Takayuki Okatani, and developed a new method that can reduce the processing time required for SLAM (Simultaneous Localization and Mapping), which is essential for devices that perform autonomous control, down to 1/60th of the time needed by conventional technologies. The new method enables advanced SLAM processing even on SoCs for edge devices with limited CPU performance and power consumption, and is expected to be applied to self-driving cars, AGVs (Automatic Guided Vehicle), robots, drones, and other devices that perform autonomous control, as well as devices such as AR (augmented reality) glasses.

The research work has been accepted as an oral presentation paper for ICCV, one of the most prestigious conferences in the field of computer vision. This new method is presented at the conference, held online from October 11 through 17. https://iccv2021.thecvf.com/home

SLAM can be classified into two main types according to the sensing method used to acquire 3D information around an object. One is LiDAR (Light Detection and Ranging), which measures distance using a laser. The other is Visual SLAM, which uses camera images. Visual SLAM is evolving significantly because the cameras used are relatively inexpensive and many applications can be expected by combining the method with various control processes using image recognition in addition to simultaneous localization.

Image recognition technology has made remarkable progress in recent years with the introduction of deep learning, and the use of deep learning has become an important factor in the evolution of Visual SLAM. However, the huge amount of computation required for bundle adjustment (BA), which optimizes the unknown 3D information of landmarks based on the feature points and camera orientation, is the bottleneck of conventionally proposed methods. This can make practical processing difficult for edge-type SoC devices with limited CPU processing power. (Figure 1)

As a solution to this challenge, the research team proposed a method of approximate computation by inference using "Graph Network (GN)"^[1], a kind of graph neural network. The new method consists of the inference of updated information from keyframe and landmark information input by GN blocks (Figure 2), and convergence to the final value by a multi-stack GN structure (Figure 3). These have enabled inference processing that requires less computation compared to the standard bundle adjustment using the conventional Levenberg-Marquardt method.

 

The research team implemented Visual SLAM bundle adjustment using this new inference method and compared it with "g2o"^[2], a method widely used today. PC simulation results confirmed that the new method can reduce the processing time to 1/60 compared to g2o (Fig. 4).

Socionext will accumulate experiences from this research work and establish Visual SLAM technology as one of the offerings for the company's custom SoC solutions. The company will propose innovative performance improvements for customer systems in fields that require image recognition, such as industrial equipment and mobility. In addition, the company will continue research and development on processing efficiency improvement by the new inference method, and expand its usage to new customer applications outside of image recognition.

ICCV2021 https://iccv2021.thecvf.com/home
Held online from October 11 to 17, 2021

Authors:
Tetsuya Tanaka, Yukihiro Sasagawa from Socionext, and Takayuki Okatani from Tohoku University

Subject:
Learning to Bundle-adjust: A Graph Network Approach to Faster Optimization of Bundle Adjustment for Vehicular SLAM

Notes:
[1] “Relational inductive biases, deep learning, and graph networks”
Source: https://github.com/deepmind/graph_nets, Battaglia et al.
[2] “G2o: A general framework for graph optimization”
Source: 2011 IEEE International Conference on Robotics and Automation, Kmmerle et al.