Click here to join the COMPARE Slack!

Charter for Strategic Development and Ecosystem Functions

To support the development, benchmarking, and deployment of robot systems, an open-source ecosystem (OSE) is under development through the National Science Foundation (NSF) Pathways to Enable Open-Source Ecosystems (POSE) program.

The OSE will facilitate the development and dissemination of the open-source assets for robotic manipulation, i.e., robot hardware, software, benchmarking practices. It will create a community-driven platform that researchers and developers can share and learn about these open-source resources, find tools to easily utilize them, collaborate on developing systematic robot experimentation methodologies, and disseminate their findings effectively. As such the OSE will address the issues facing the advancement of robot manipulation due to the lack of systematic development and benchmarking methodologies, as well as the unique challenges related to physical assets (both equipment and benchmarking tools), preventing the robotic manipulation community from effectively utilizing such resources compared to other research domains like computer vision.

The OSE will support a wide variety of open-source assets (OSAs) available or to be developed in the future for robot manipulation, including physical object sets and hardware designs, digital datasets and simulations, instructional task protocols and metrics, and functional algorithms and robotic behaviors. 

Participants in the OSE will include all stakeholders of these OSAs, including developers (those who develop and maintain the OSA), contributors (those who iterate on an existing OSA), and users (those who implement OSAs in their research and development practices). 

OSE structure, governance, functions, and activities will be facilitated by a distributed set of OSE participants in various roles (e.g., leadership and participatory positions), diverse in terms of terms of backgrounds, research domains, and developmental maturity, spanning industry, academia, and government entities. 

Development of the OSE will be conducted transparently, with continuous input solicited from the community and outputs shared from the initial core development team, through a series of workshops, meetings, surveys, and asynchronous communication via online platforms.

It is intended that the Open Source Security Foundation (OpenSSF) best practices for their Free/Libre and Open Source Software (FLOSS) projects be followed for all OSAs utilized within the OSE, although additional consideration will be needed for non-code-based OSAs such as those that are physical and instructional.

During Phase I of the project (2022 - 2023), the scope of the OSE shall be investigated and discovered, towards a proposal for Phase II (2024 - 2026). During Phase II, the OSE will be formally developed and established through the execution of multiple activities and functions, including, but not limited to, the following:

• Establish COMPARE clearinghouse, a comprehensive and organized repository of OSAs with mechanisms to support contribution of new OSAs by developers, iterations of OSAs by contributors, and implementation of OSAs for experimentation usage. 

• Establish advisory committees and working groups for reviewing OSA development, implementation, and benchmarking with OSE participants, with groups established for particular applications (e.g., bimanual manipulation, human-robot handovers, etc.).

• Establish collaborative test facilities for conducting interlaboratory experiments and round robin testing of OSAs and robotic solutions that utilize OSAs to evaluate their robustness and demonstrate replicable performance.

• Propose new research conference tracks and journals specific to the OSE topics of interest, including open-source benchmarking and dataset generation.

• Provide roadmaps and frameworks for OSA development maturity with supporting templates and processes for effectively reaching each stage.

• Provide webinars, tutorials, and instructions for OSE participation as a developer, contributor, and user, for usage of the previously described assets and activities.

• Define OSE-specific standards for OSA structure, deployment, and interoperability, for potential support by standards development organizations (SDOs) like IEEE or ASTM.

• Provision OSE participants with a common set of facilities, including robotic equipment, tools, hardware OSAs, software OSAs, and access to remotely accessible supporting infrastructure, as needed for participating organizations, with eligibility of such provisions to be based on a set of established criteria.

• Coordinate large-scale events across the OSE such as distributed hackathons, grand challenges, and competitions to ignite targeted, active development of OSAs, robotic capabilities, and/or benchmarking of robot systems. 

• Identify strategies for a sustainable OSE, i.e. establishing monetary (company or government partnerships, membership mechanisms) and human resources (governance structure) for long-term operation.

Principal Investigators

Contact

If you are interested in contributing to the development of the COMPARE open-source ecosystem or have any questions or comments, please e-mail Adam Norton at adam_norton@uml.edu