Collaborative Research: SHF: Small: Enabling Efficient 3D Perception: An Architecture-Algorithm Co-Design Approach

This research project designs software-hardware collaborative mechanisms to boost the execution efficiency of 3D perception (point cloud) algorithms by an order of magnitude. Achieving this goal requires designing fundamentally new algorithmic primitives, offline and run-time systems, and hardware architectures that tame the irregular computation and memory patterns in 3D perception algorithms. This research unlocks next-generation software innovation in emerging domains driven by 3D perception, such as autonomous driving, Augmented/Virtual Reality, and precision agriculture. The research agenda is complemented by an educational/outreach agenda. The PIs are (1) offering summer “introduction to computing” courses to high school students from the Rochester Central School District, (2) engaging students in RIT’s National Technical Institute for the Deaf program through experiencing 3D sensing and research activities, (3) introducing new courses/modules in both UR and RIT on 3D perception, both on algorithms and hardware systems, and (4) offering undergraduate students opportunities for hands-on experience in emerging application domains and hardware acceleration. This research project addresses the fundamental mismatch between the irregularities in point-cloud algorithms and today’s hardware architectures, which are primarily optimized for 2D image- and video-processing algorithms that are regular stencil pipelines operating on structured data. The key intellectual merit is the pursuit of new algorithms and system architectures that reduce/eliminate irregular computation and memory accesses in 3D perception. The technical contribution is three-fold: 1) efficient, yet generally applicable, hardware building blocks required to accelerate point cloud algorithms, 2) run-time systems that dynamically adapt to operating constraints (e.g., hardware resources, performance, energy) in an application-aware and data-aware manner, and 3) a new class of efficient-by-construction 3D perception algorithms that leverage the small data volume in single-beam point clouds. The algorithm-hardware co-designed system not only accelerates current 3D perception algorithms, but also provides a computing substrate so that 3D perception can be pervasively used as a building block in future application domains. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.