Published: Sep 30, 2023 by

Yi Du

Robotic Perception and Mapping: Frontier Vision & Learning Techniques

This workshop aims to present the latest advancements and frontier techniques in computer vision and machine learning that are expected to have a significant impact on robotic perception and mapping and set the direction of research in the next 5-10 years. Through a series of invited and contributed talks by renowned academic leaders and researchers, the event will discuss frontier technologies for robotic perception and mapping with particular focus on addressing existing computer vision challenges such as dealing with dynamic environments and non-rigid objects, trade-offs between scalability (capturing large environments over long periods of operation without running out of memory) versus expressivity (capturing precise details about the environment characteristics, including geometry, semantics, dynamics, topology), as well as addressing machine learning challenges such as reducing training and inference time of machine learning models, fitting large models on small robotic platforms, trade-offs between pre-training and fine-tuning environment models, and ensuring generalization and robustness. To encourage interaction among participants, the workshop will feature panel discussions, posters, and spotlight talks. The event will adopt a hybrid format with both in-person and remote participants. All talks and accepted contributions will be published on the workshop’s webpage to expand its reach and impact.

This workshop is a follow-up to the well-received ICRA 2022 workshop on “Robotic Perception and Mapping: Emerging Techniques,” which had the largest attendance across all workshops with over 1,000 participants. This follow-up workshop will provide a new perspective by inviting a new set of speakers and discussing frontier research on computer vision and machine learning (instead of mapping, which was the focus of the previous workshop). To differentiate the workshop from traditional computer vision and machine learning conferences, the talks are particularly focused on techniques with direct application in robotic perception and autonomy.

Expected Outcome

By facilitating discussion among invited speakers, participants, and authors of contributed papers, the workshop aims to study and answer the following fundamental questions:

What are the latest results and frontier techniques in computer vision and machine learning that are expected to have a high impact on robotic perception and mapping in the next 5-10 years?
What are the latest trends in learning-based perception technologies, such as novel scene representations, neural implicit scene encoding, radiance fields, and dynamic scene reconstruction?
How will the latest computer vision and machine learning technologies (e.g., large language models) be utilized in robotics, what are the barriers to transferring these technologies to real robots, and how to overcome them?
What are open problems, gaps, and failure cases in robotic perception and mapping, and how should the community plan the research roadmap to address these challenges?

Invited Speaker

Konstantinos Alexis

Norwegian University of Science and Technology

Joydeep Biswas

University of Texas, Austin

Wolfram Burgard

Technical University of Nuremberg

Katerina Fragkiadaki

Carnegie Mellon University

Golnaz Habibi

University of Oklahoma

Christoffer Heckman

University of Colorado Boulder

Chad Jenkins

University of Michigan

Michael Kaess

Carnegie Mellon University

Lingjie Liu

University of Pennsylvania

Katie Skinner

University of Michigan

Sai Vemprala

Scaled Foundations

Ge Yang

Massachusetts Institute of Technology

Panelists:

Wolfram Burgard

Technical University of Nuremberg

Ayoung Kim

Seoul National University

Davide Scaramuzza

University of Zurich

Sebastian Scherer

Carnegie Mellon University

Teresa Vidal-Calleja

University of Technology Sydney

Event Schedule - October 5, 2023

Date: October 5, 2023
Time: 8:30 AM - 3:50 PM (US Eastern Time)
Location: Room 310A/B, Huntington Place, Detroit, Michigan, USA

Session 1 (Chaired by Chen Wang)

8:30-8:40: Welcome message by organizers
8:40-9:00: Konstantinos Alexis
9:00-9:20: Katerina Fragkiadaki
9:20-9:40: Ge Yang
9:40-9:50: Spotlight talk 1: Jiaxu Xing

10:20-10:50: Coffee break & posters

Session 2 (Chaired by Golnaz Habibi)

10:20-10:40: Wolfram Burgard
10:40-11:00: Lingjie Liu
11:00-11:20: Christoffer Heckman
11:20-12:00: Panel discussion (Chaired by Nikolay Atanasov)
- Panelists: Wolfram Burgard, Ayoung Kim, Davide Scaramuzza, Sebastian Scherer, Teresa Vidal-Calleja

12:00-1:00: Lunch break & posters

Session 3 (Chaired by Kevin Doherty)

13:00-13:20: Michael Kaess
13:20-13:40: Joydeep Biswas
13:40-14:00: Golnaz Habibi
14:00-14:10: Spotlight talk 2: Giovanni Cioffi

14:10-14:40: Coffee break & posters

Session 4 (Chaired by Kevin Doherty)

14:40-15:00: Chad Jenkins
15:00-15:20: Katie Skinner
15:20-15:40: Sai Vemprala
15:40-15:50: Closing remarks

Call For Papers/Posters/Videos

We cordially invite researchers to submit short papers, extended abstracts, posters, and/or videos. We accept original papers, as well as in-review or recently accepted manuscripts. Submitted contributions can describe work in progress, preliminary results, novel concepts, or industrial applications.

All manuscripts are limited to 4+n pages (i.e., additional pages over 4 are only allowed for references), should use the IEEE standard two-column conference format (see IROS 2023 website), and must be in the PDF format with a size less than 20 MB. We encourage authors to submit a video for their manuscript as supplementary material. All video submissions must be in mp4 format with a size of less than 100 MB.

All submissions will be peer-reviewed. Authors who submit a paper are expected to provide (up to) 3 single-blind reviews for the papers submitted to this workshop. Submissions will be selected by workshop organizers based on the reviews, their originality, relevance to the workshop topics, contributions, technical clarity, and presentation. All accepted manuscripts will be presented as posters during the workshop, which will be displayed throughout the day. Two top contributions will be selected for 10-minute oral presentations at spotlight sessions. Accepted posters and videos will be posted on the workshop website. You can contact the corresponding organizer with any questions: Dr. Kaveh Fathian, fathian@ariarobotics.com.

TOPICS OF INTEREST:

Novel 3D representations including implicit and explicit neural representations, compressed point clouds, meshes, signed distance functions, occupancy, and semantic maps
Dynamic and non-rigid 3D reconstruction
Language models for perception and spatial reasoning
Test-time incremental neural network training for perception and mapping
Self-supervised feature and environment model training
Semantic scene understanding, detection, and segmentation
Multi-modal sensing for multi-modal scene reconstruction
Vision-based navigation and estimation
Robust localization in uncertain and dynamic environments
Uncertainty estimation and introspective failure detection in machine learning and perception
Few-shot generalization and robustness to distribution shift in mapping and SLAM domains
Certifiable and interpretable learning techniques for perception
Optimization and graphical models for perception
Synergetic learning and model-based techniques for perception

Submission Platform:

To submit your contributions, please follow these steps: Submission Platform

You will need to create a new account if you have not used Microsoft CMT before.

Important Dates:

Submission Deadline: Sunday, Aug 20, 2023, by 11:59 PM EDT
Reviews due: Tuesday, Sep 5, 2023, by 11:59 PM EDT
Acceptance Notification: Tuesday, Sep 12, 2023
Camera-ready submission: Monday, Sep 18, 2023, by 11:59 PM EDT
Workshop: Thursday, October 5, 2023
Location: Room 310A/B, Huntington Place, Detroit, Michigan, USA

Accepted Paper List

ID	Paper Title	Author Names
4	Markerless Camera-to-Robot Pose Estimation via Self-supervised Sim-to-Real Transfer	- Jingpei Lu (University of California, San Diego)* - Florian Richter (University of California San Diego) - Michael Yip (UC San Diego)
5	Importance Sampling: Degradation-Aware Alternative to Voxelization in Robot Pose Estimation	- Pavel Petracek (Czech Technical University)* - Nikhil Khedekar (Norwegian University of Science and Technology) - Morten Nissov (NTNU) - Kostas Alexis (Norwegian University of Science and Technology) - Martin Saska (Czech Technical University in Prague)
6	HDVIO: Improving Localization and Disturbance Estimation with Hybrid Dynamics VIO	- Giovanni Cioffi (University of Zurich)* - Leonard Bauersfeld (University of Zurich) - Davide Scaramuzza (University of Zurich, Switzerland)
7	Attending Multiple Visual Tasks for Own Failure Detection	- Jiaxu Xing (University of Zurich)* - Boyang Sun (ETH Zurich) - Hermann Blum (ETH Zurich) - Roland Siegwart (ETH Zurich) - Cesar Cadena (ETH Zurich)
8	Faster Optimization in S-Graphs Exploiting Hierarchy	- Hriday Bavle (University of Luxembourg)* - Jose Luis Sanchez-Lopez (University of Luxembourg) - Javier Civera (Universidad de Zaragoza) - Holger Voos (SnT - Interdisciplinary Centre for Security, Reliability and Trust)
10	Motion Segmentation from a Moving Monocular Camera	- Yuxiang Huang (University of Waterloo)* - John Zelek (University of Waterloo)
11	Adaptive Graduated Non-Convexity for Pose Graph Optimization	- Seungwon Choi (Seoul National University) - Wonseok Kang (Seoul National University) - Jiseong Chung (Seoul National University) - Jaehyun Kim (Seoul National University) - Tae-wan Kim (Seoul National University)*
12	Cluster-based Dynamic Object Filtering via Egocentric Motion Detection for Building Static 3D Point Cloud Maps	- Pengcheng Cao (University of California, San Diego)* - Thomas Bewley (University of California, San Diego) - Falko Kuester (UC San Diego)
13	Human-Inspired Topological Representations for Visual Object Recognition in Unseen Environments	- Ekta U Samani (University of Washington) - Ashis G. Banerjee (University of Washington)*
15	Real-Time and Recurrent Latent Map Propagation for Dynamic Outdoor Environments	- Joseph G Wilson (University of Michigan)* - Minghan Zhu (University of Michigan, Ann Arbor) - Kira Barton (University of Michigan) - Maani Ghaffari Jadidi (Univ. of Michigan)
16	HARDNAV - Simulator for Benchmarking Robust Navigation and Place Recognition in Large, Confusing and Highly Dynamic Environments	- Tomáš Musil (Czech Technical University in Prague)* - Matěj Petrlík (Czech Technical University in Prague) - Martin Saska (Czech Technical University in Prague)
17	The Harmonic Exponential Filter for Recursive Nonparametric Estimation on Motion Groups	- Steven A Parkison (Mila)* - Miguel Saavedra-Ruiz (Université de Montréal ) - Ria Arora (Universite de Montreal) - James Forbes (McGill University) - Liam Paull (Mila)
18	Pre-Trained Object-Centric Representations for Robotics Composed From “What” and “Where” Foundation Models	- Junyao Shi (University of Pennsylvania)* - Jianing Qian (University of Pennsylvania) - Yecheng Ma (University of Pennsylvania) - Dinesh Jayaraman (University of Pennsylvania)
19	Long-range UAV Thermal Geo-localization with Satellite Imagery	- Jiuhong Xiao (New York University)* - Daniel Toertei (Technology Innovation Institute) - Eloy Roura (Technology Innovation Institute) - Giuseppe Loianno (New York University)

Organizers & Committee:

Senior Organizers:

Academic leaders and government researchers with extensive experience in organizing workshops at internationally recognized conferences that establish the primary workshop objectives and program.

Nikolay Atanasov

University of California San Diego

Luca Carlone

Massachusetts Institute of Technology

Jonathan How

Massachusetts Institute of Technology

John Leonard

Massachusetts Institute of Technology

Carlos Nieto

US Army Research Laboratory

Sebastian Scherer

Carnegie Mellon University

Workshop Committee:

Junior faculty and researchers that implement the workshop vision, oversee submitted contributions (papers/posters/videos), disseminate the contributions, advertise the workshop, and chair sessions during the event.