OPTICAL FLOW-BASED LIDAR ODOMETRY

Abstract

In this study, data obtained from a 360-degree (or 2D) Light Detection and Ranging sensor, which is commonly used for Simultaneous Localization and Mapping, were converted into an image/frame matrix to apply optical flow methods. Key features on the resulting image matrix were identified as corners using the Shi-Tomasi method. The positional changes of these key features during each complete measurement cycle of the LiDAR sensor were calculated using the Lucas-Kanade method, an optical flow technique. These positional changes were analyzed to estimate the vehicle’s displacement within the environment. The developed method was tested in a simulation environment with different object densities, moving linearly at different speeds, both constant and variable.

Keywords

• Lidar odometry
• Navigation
• Optical-Flow
• Simultaneous Localization and Mapping (SLAM)

References

1. C. Cadena et al., “Past, Present, and Future of Simultaneous Localization and Mapping: Toward the Robust-Perception Age,” IEEE Transactions on Robotics, vol. 32, no. 6, pp. 1309–1332, Dec. 2016, doi: 10.1109/TRO.2016.2624754.
2. H. Durrant-Whyte and T. Bailey, “Simultaneous localization and mapping: part I,” IEEE Robot Autom Mag, vol. 13, no. 2, pp. 99–110, Jun. 2006, doi: 10.1109/MRA.2006.1638022.
3. A. Eldemiry, Y. Zou, Y. Li, C.-Y. Wen, and W. Chen, “Autonomous Exploration of Unknown Indoor Environments for High-Quality Mapping Using Feature-Based RGB-D SLAM,” Sensors, vol. 22, no. 14, p. 5117, Jul. 2022, doi: 10.3390/s22145117.
4. M. Bryson and S. Sukkarieh, “Observability analysis and active control for airborne SLAM,” IEEE Trans Aerosp Electron Syst, vol. 44, no. 1, pp. 261–280, Jan. 2008, doi: 10.1109/TAES.2008.4517003.
5. M. U. Diginsa, N. Shafie, N. Yusuf, and S. U. Suleiman, “Review: Issues and Challenges of Simultaneous Localization and Mapping (SLAM) Technology in Autonomous Robot,” International Journal of Innovative Computing, vol. 13, no. 2, pp. 59–63, Nov. 2023, doi: 10.11113/ijic.v13n2.408.
6. S. Zheng, J. Wang, C. Rizos, W. Ding, and A. El-Mowafy, “Simultaneous Localization and Mapping (SLAM) for Autonomous Driving: Concept and Analysis,” Remote Sens (Basel), vol. 15, no. 4, p. 1156, Feb. 2023, doi: 10.3390/rs15041156.
7. M. E. Tokgöz, A. Yusefi, İ. Toy, and A. Durdu, “Enhanced Obstacle Detection in Autonomous Vehicles using 3D LiDAR Mapping Techniques,” in 2024 23rd International Symposium INFOTEH-JAHORINA (INFOTEH), IEEE, Mar. 2024, pp. 1–4. doi: 10.1109/INFOTEH60418.2024.10495979.
8. Jonghyuk Kim and S. Sukkarieh, “Autonomous airborne navigation in unknown terrain environments,” IEEE Trans Aerosp Electron Syst, vol. 40, no. 3, pp. 1031–1045, Jul. 2004, doi: 10.1109/TAES.2004.1337472.

9. I. Cvišić, J. Ćesić, I. Marković, and I. Petrović, “SOFT‐SLAM: Computationally efficient stereo visual simultaneous localization and mapping for autonomous unmanned aerial vehicles,” J Field Robot, vol. 35, no. 4, pp. 578–595, Jun. 2018, doi: 10.1002/rob.21762.
10. Y. Alborzi, B. S. Jalal, and E. Najafi, “ROS-based SLAM and Navigation for a Gazebo-Simulated Autonomous Quadrotor,” in 2020 21st International Conference on Research and Education in Mechatronics (REM), IEEE, Dec. 2020, pp. 1–5. doi: 10.1109/REM49740.2020.9313875.
11. L. von Stumberg, V. Usenko, J. Engel, J. Stuckler, and D. Cremers, “From monocular SLAM to autonomous drone exploration,” in 2017 European Conference on Mobile Robots (ECMR), IEEE, Sep. 2017, pp. 1–8. doi: 10.1109/ECMR.2017.8098709.
12. Y. Tian, H. Yue, B. Yang, and J. Ren, “Unmanned Aerial Vehicle Visual Simultaneous Localization and Mapping: A Survey,” J Phys Conf Ser, vol. 2278, no. 1, p. 012006, May 2022, doi: 10.1088/1742-6596/2278/1/012006.
13. H. Qin et al., “Autonomous Exploration and Mapping System Using Heterogeneous UAVs and UGVs in GPS-Denied Environments,” IEEE Trans Veh Technol, vol. 68, no. 2, pp. 1339–1350, Feb. 2019, doi: 10.1109/TVT.2018.2890416.
14. L. Mochurad, Y. Hladun, and R. Tkachenko, “An Obstacle-Finding Approach for Autonomous Mobile Robots Using 2D LiDAR Data,” Big Data and Cognitive Computing, vol. 7, no. 1, p. 43, Mar. 2023, doi: 10.3390/bdcc7010043.
15. Q. Zeng, X. Tao, H. Yu, X. Ji, T. Chang, and Y. Hu, “An Indoor 2-D LiDAR SLAM and Localization Method Based on Artificial Landmark Assistance,” IEEE Sens J, vol. 24, no. 3, pp. 3681–3692, Feb. 2024, doi: 10.1109/JSEN.2023.3341832.
16. J.-S. Kim, G. Lee, and Y. S. Kim, “2D-LiDAR-Sensor-Based Retaining Wall Displacement Measurement System,” Applied Sciences, vol. 12, no. 22, p. 11335, Nov. 2022, doi: 10.3390/app122211335.
17. Z. YILMAZ and L. BAYINDIR, “Lidar-based Robot Detection and Positioning using Machine Learning Methods,” Balkan Journal of Electrical and Computer Engineering, vol. 10, no. 2, pp. 214–223, Apr. 2022, doi: 10.17694/bajece.984744.
18. Z. Yılmaz and L. Bayındır, “Simulation of Lidar-Based Robot Detection Task using ROS and Gazebo,” European Journal of Science and Technology, pp. 513–529, Oct. 2019, doi: 10.31590/ejosat.642840.
19. H. Wang, C. Wang, and L. Xie, “Intensity-SLAM: Intensity Assisted Localization and Mapping for Large Scale Environment,” IEEE Robot Autom Lett, vol. 6, no. 2, pp. 1715–1721, Apr. 2021, doi: 10.1109/LRA.2021.3059567.
20. Q. Zou, Q. Sun, L. Chen, B. Nie, and Q. Li, “A Comparative Analysis of LiDAR SLAM-Based Indoor Navigation for Autonomous Vehicles,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 6907–6921, Jul. 2022, doi: 10.1109/TITS.2021.3063477.
21. K. Yan and B. Ma, “Mapless Navigation Based on 2D LIDAR in Complex Unknown Environments,” Sensors, vol. 20, no. 20, p. 5802, Oct. 2020, doi: 10.3390/s20205802.
22. L. Mu, P. Yao, Y. Zheng, K. Chen, F. Wang, and N. Qi, “Research on SLAM Algorithm of Mobile Robot Based on the Fusion of 2D LiDAR and Depth Camera,” IEEE Access, vol. 8, pp. 157628–157642, 2020, doi: 10.1109/ACCESS.2020.3019659.
23. W. Wei, B. Shirinzadeh, R. Nowell, M. Ghafarian, M. M. A. Ammar, and T. Shen, “Enhancing Solid State LiDAR Mapping with a 2D Spinning LiDAR in Urban Scenario SLAM on Ground Vehicles,” Sensors, vol. 21, no. 5, p. 1773, Mar. 2021, doi: 10.3390/s21051773.
24. R. Ren, H. Fu, and M. Wu, “Large-Scale Outdoor SLAM Based on 2D Lidar,” Electronics (Basel), vol. 8, no. 6, p. 613, May 2019, doi: 10.3390/electronics8060613.
25. Z. Wang, Y. Chen, Y. Mei, K. Yang, and B. Cai, “IMU-Assisted 2D SLAM Method for Low-Texture and Dynamic Environments,” Applied Sciences, vol. 8, no. 12, p. 2534, Dec. 2018, doi: 10.3390/app8122534.
26. L.-H. Chen and C.-C. Peng, “A Robust 2D-SLAM Technology With Environmental Variation Adaptability,” IEEE Sens J, vol. 19, no. 23, pp. 11475–11491, Dec. 2019, doi: 10.1109/JSEN.2019.2931368.
27. A. Aboutaleb, A. S. El-Wakeel, H. Elghamrawy, and A. Noureldin, “LiDAR/RISS/GNSS Dynamic Integration for Land Vehicle Robust Positioning in Challenging GNSS Environments,” Remote Sens (Basel), vol. 12, no. 14, p. 2323, Jul. 2020, doi: 10.3390/rs12142323.
28. D. Kong, X. Li, Y. Hu, Q. Xu, A. Wang, and W. Hu, “Learning a Novel LiDAR Submap-Based Observation Model for Global Positioning in Long-Term Changing Environments,” IEEE Transactions on Industrial Electronics, vol. 70, no. 3, pp. 3147–3157, Mar. 2023, doi: 10.1109/TIE.2022.3169849.
29. Z. Chen et al., “Improved-UWB/LiDAR-SLAM Tightly Coupled Positioning System with NLOS Identification Using a LiDAR Point Cloud in GNSS-Denied Environments,” Remote Sens (Basel), vol. 14, no. 6, p. 1380, Mar. 2022, doi: 10.3390/rs14061380.
30. A. Ammar, H. Ben Fredj, and C. Souani, “Accurate Realtime Motion Estimation Using Optical Flow on an Embedded System,” Electronics (Basel), vol. 10, no. 17, p. 2164, Sep. 2021, doi: 10.3390/electronics10172164.
31. K. Blachut and T. Kryjak, “Real-Time Efficient FPGA Implementation of the Multi-Scale Lucas-Kanade and Horn-Schunck Optical Flow Algorithms for a 4K Video Stream,” Sensors, vol. 22, no. 13, p. 5017, Jul. 2022, doi: 10.3390/s22135017.
32. S. Shi, D. Zhang, C. Zhang, Z. Chen, C. Feng, and B. Fan, “Large Displacement Optical Flow Estimation Based on Robust Interpolation of Sparse Correspondences,” IEEE Access, vol. 8, pp. 227360–227372, 2020, doi: 10.1109/ACCESS.2020.3045764.
33. A. Dashkevich and O. Vodka, “Computational Tool for Analysis of Strains Based on Optical Flow Approach,” in 2020 IEEE KhPI Week on Advanced Technology (KhPIWeek), IEEE, Oct. 2020, pp. 382–386. doi: 10.1109/KhPIWeek51551.2020.9250075.
34. K. Souhila, O. Djekoune, D. Djebrouni, and D. Meriche, “On the Use of Optical Flow in Robot Navigation,” in 2007 IEEE International Conference on Signal Processing and Communications, IEEE, 2007, pp. 1287–1290. doi: 10.1109/ICSPC.2007.4728562.
35. K. Dawson, M. Tistarelli, and D. Vernon, “On the usefulness of optical flow for robotic part manipulation,” in Proceedings IROS ’91:IEEE/RSJ International Workshop on Intelligent Robots and Systems ’91, IEEE, pp. 117–122. doi: 10.1109/IROS.1991.174436.
36. Chi-Cheng Cheng, Kun-Hsin Ho, Hui-Ting Li, and Gwo-Long Lin, “Image following using the feature-based optical flow approach,” in Proceedings of the IEEE Internatinal Symposium on Intelligent Control, IEEE, pp. 350–355. doi: 10.1109/ISIC.2002.1157788.
37. F. F. Khalil and P. Payeur, “Optical flow techniques in biomimetic UAV vision,” in International Workshop on Robotic Sensors: Robotic and Sensor Environments, 2005., IEEE, 2005, pp. 14–19. doi: 10.1109/ROSE.2005.1588329. K. Kontis and H. Zare‐Behtash, “Optical Flow Visualization,” in Encyclopedia of Aerospace Engineering, Wiley, 2010. doi: 10.1002/9780470686652.eae069.
38. Z. Tu et al., “A survey of variational and CNN-based optical flow techniques,” Signal Process Image Commun, vol. 72, pp. 9–24, Mar. 2019, doi: 10.1016/j.image.2018.12.002.
39. A. Bruhn, J. Weickert, and C. Schnörr, “Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods,” Int J Comput Vis, vol. 61, no. 3, pp. 211–231, Feb. 2005, doi: 10.1023/B:VISI.0000045324.43199.43. S. Oron, A. Bar-Hille, and S. Avidan, “Extended Lucas-Kanade Tracking,” 2014, pp. 142–156. doi: 10.1007/978-3-319-10602-1_10.
40. K. Blachut and T. Kryjak, “Real-Time Efficient FPGA Implementation of the Multi-Scale Lucas-Kanade and Horn-Schunck Optical Flow Algorithms for a 4K Video Stream,” Sensors, vol. 22, no. 13, p. 5017, Jul. 2022, doi: 10.3390/s22135017.
41. A. Mitiche and A.-R. Mansouri, “On Convergence of the Horn and Schunck Optical-Flow Estimation Method,” IEEE Transactions on Image Processing, vol. 13, no. 6, pp. 848–852, Jun. 2004, doi: 10.1109/TIP.2004.827235. G. Farnebäck, “Two-Frame Motion Estimation Based on Polynomial Expansion,” 2003, pp. 363–370. doi: 10.1007/3-540-45103-X_50.
42. J. Cobos, L. Pacheco, X. Cufi, and D. Caballero, “Integrating visual odometry and dead-reckoning for robot localization and obstacle detection,” in 2010 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR), IEEE, May 2010, pp. 1–6. doi: 10.1109/AQTR.2010.5520874.
43. C. Zheng et al., “FAST-LIVO2: Fast, Direct LiDAR–Inertial–Visual Odometry,” IEEE Transactions on Robotics, vol. 41, pp. 326–346, 2025, doi: 10.1109/TRO.2024.3502198.
44. J. Zhang et al., “LOFF: LiDAR and Optical Flow Fusion Odometry,” Drones, vol. 8, no. 8, p. 411, Aug. 2024, doi: 10.3390/drones8080411.
45. M. E. Tjahjadi and F. D. Agustina, “Single image orientation of UAV’s imagery using orthogonal projection model,” in 2017 International Symposium on Geoinformatics (ISyG), IEEE, Nov. 2017, pp. 18–23. doi: 10.1109/ISYG.2017.8280668.
46. W. Ding, S. Hou, H. Gao, G. Wan, and S. Song, “LiDAR Inertial Odometry Aided Robust LiDAR Localization System in Changing City Scenes,” in 2020 IEEE International Conference on Robotics and Automation (ICRA), IEEE, May 2020, pp. 4322–4328. doi: 10.1109/ICRA40945.2020.9196698.
47. B. Zhou, Z. Tang, K. Qian, F. Fang, and X. Ma, “A LiDAR Odometry for Outdoor Mobile Robots Using NDT Based Scan Matching in GPS-denied environments,” in 2017 IEEE 7th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER), IEEE, Jul. 2017, pp. 1230–1235. doi: 10.1109/CYBER.2017.8446588.
48. X. Guo, X. Lin, L. Zhao, Z. Zhu, and J. Chen, “An Unsupervised Optical Flow Estimation for Lidar Image Sequences,” in 2021 IEEE International Conference on Image Processing (ICIP), IEEE, Sep. 2021, pp. 2613–2617. doi: 10.1109/ICIP42928.2021.9506376.
49. V. Vaquero, A. Sanfeliu, and F. Moreno-Noguer, “Hallucinating Dense Optical Flow from Sparse Lidar for Autonomous Vehicles,” in 2018 24th International Conference on Pattern Recognition (ICPR), IEEE, Aug. 2018, pp. 1959–1964. doi: 10.1109/ICPR.2018.8545803.
50. E. Ilg, N. Mayer, T. Saikia, M. Keuper, A. Dosovitskiy, and T. Brox, “FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Jul. 2017, pp. 1647–1655. doi: 10.1109/CVPR.2017.179.
51. M. Jaimez, J. G. Monroy, and J. Gonzalez-Jimenez, “Planar odometry from a radial laser scanner. A range flow-based approach,” in 2016 IEEE International Conference on Robotics and Automation (ICRA), IEEE, May 2016, pp. 4479–4485. doi: 10.1109/ICRA.2016.7487647.
52. A. Censi, “An ICP variant using a point-to-line metric,” in 2008 IEEE International Conference on Robotics and Automation, IEEE, May 2008, pp. 19–25. doi: 10.1109/ROBOT.2008.4543181.
53. Z. He, Y. Wang, and H. Yu, “Feature-to-Feature based Laser Scan Matching in Polar Coordinates with Application to Pallet Recognition,” Procedia Eng, vol. 15, pp. 4800–4804, 2011, doi: 10.1016/j.proeng.2011.08.897.
54. M. Jaimez, J. Monroy, M. Lopez-Antequera, and J. Gonzalez-Jimenez, “Robust Planar Odometry Based on Symmetric Range Flow and Multiscan Alignment,” IEEE Transactions on Robotics, vol. 34, no. 6, pp. 1623–1635, Dec. 2018, doi: 10.1109/TRO.2018.2861911.
55. Jianbo Shi and Tomasi, “Good features to track,” in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition CVPR-94, IEEE Comput. Soc. Press, 1994, pp. 593–600. doi: 10.1109/CVPR.1994.323794.
56. R. J. Mstafa, Y. M. Younis, H. I. Hussein, and M. Atto, “A New Video Steganography Scheme Based on Shi-Tomasi Corner Detector,” IEEE Access, vol. 8, pp. 161825–161837, 2020, doi: 10.1109/ACCESS.2020.3021356.
57. C. Harris and M. Stephens, “A Combined Corner and Edge Detector,” in Procedings of the Alvey Vision Conference 1988, Alvey Vision Club, 1988, pp. 23.1-23.6. doi: 10.5244/C.2.23.
58. N. Kanopoulos, N. Vasanthavada, and R. L. Baker, “Design of an image edge detection filter using the Sobel operator,” IEEE J Solid-State Circuits, vol. 23, no. 2, pp. 358–367, Apr. 1988, doi: 10.1109/4.996.
59. K. Mikolajczyk and C. Schmid, “Indexing based on scale invariant interest points,” in Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, IEEE Comput. Soc, pp. 525–531. doi: 10.1109/ICCV.2001.937561.
60. J. R. Winkler, “Error Analysis and Condition Estimation of the Pyramidal Form of the Lucas-Kanade Method in Optical Flow,” Electronics (Basel), vol. 13, no. 5, p. 812, Feb. 2024, doi: 10.3390/electronics13050812.
61. N. Koenig and A. Howard, “Design and use paradigms for gazebo, an open-source multi-robot simulator,” in 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566), IEEE, pp. 2149–2154. doi: 10.1109/IROS.2004.1389727.
62. Y. Maruyama, S. Kato, and T. Azumi, “Exploring the performance of ROS2,” in Proceedings of the 13th International Conference on Embedded Software, New York, NY, USA: ACM, Oct. 2016, pp. 1–10. doi: 10.1145/2968478.2968502.
63. M. Siwek, J. Panasiuk, L. Baranowski, W. Kaczmarek, P. Prusaczyk, and S. Borys, “Identification of Differential Drive Robot Dynamic Model Parameters,” Materials, vol. 16, no. 2, p. 683, Jan. 2023, doi: 10.3390/ma16020683.
64. F. Nascimento Martins and A. Santos Brandão, “Motion Control and Velocity-Based Dynamic Compensation for Mobile Robots,” in Applications of Mobile Robots, IntechOpen, 2019. doi: 10.5772/intechopen.79397.