(1) Hardware requirements for dexterous manipulation
A dexterous manipulation system relies on multiple subsystems with components working together, each with varying characteristics and physical configurations to produce a dexterous capability, including:Â
Subsystems (hands, wrists, fingers, thumbs, vision, arms),Â
Components (actuators, tendons, tactile sensors, cameras, finger tips, palms), andÂ
Characteristics (compliance, backdrivable, compute, form factor, degrees of freedom, resolution).Â
Benchmarking: What existing benchmarking efforts can be leveraged? Where are there gaps? How capable are simulations compared to physical tests?
Reproducibility: What challenges exist to reproduce hardware for dexterity? Is it a materials issue, control/behavior reproducibility, degradation over time?
Standards: What existing standards can support dexterous hardware? What should standards specify? Fabrication processes, performance thresholds, etc.?
Other: What hardware could be common to ease development and/or adoption of dexterity? Data collection hardware, end-effectors, sensors?
(2) Tactile data representation to drive dexterous manipulation
With the prevalence of tactile data used in dexterous manipulation, a common representation of tactile sense data beyond the typical P(x,y,t) approach may greatly benefit the community
Decouple robot perception and control from the underlying sensor hardware, enabling tactile data from diverse sensing technologies to be encoded into a shared latent space
Unlock sensor interchangeability to allow robots to operate across different tactile platforms without requiring redesign or retraining of control policiesÂ
Benchmarking:Â What new metrics, benchmarks, and test methods would a common data representation enable? Would it invalidate any existing benchmarking?
Reproducibility: How useful are current tactile datasets? Do they allow for reproducible results, benchmarks, etc.? Would a common format improve this?
Standards: Can such a data representation be supported by a standard? If so, what parameters should it include? Format, resolution, fidelity, data collection method?
Other: Would you modify your current data format to match a new, comm one? Why or why not? How should it be formatted to be most beneficial for your work?
(3) Dexterity for deformable object manipulation
Manipulating deformable objects like textiles require advanced robot perception, planning, and control to enable the dexterity needed to perform tasks like grasping, transferring, folding, piling, fitting, etc.
Simulating deformables and contact (rigid on flexible, flexible on flexible) is a key bottleneck for development and benchmarking
Tasks like belt tensioning or sheet fitting require not only dexterity, but also for significant strength, both of which may not be present in current robot manipulation systems
Benchmarking:Â What types of objects, tasks, and metrics should be used in benchmarking dexterous manipulation of deformables? Materials, applications?
Reproducibility: How reproducible are results of deformable object manipulation experiments? Are they trustworthy? What conditions are most difficult to reproduce?
Standards: What standards exist to support deformable object specification (e.g., garment/textile industry)? Data formats of simulations? Task specification methods?
Other: What are the known unknowns and unknown unknowns for representing deformables? What are current datasets lacking to support dexterity?Â
(4) Defining dexterity: capabilities, tasks, and characteristicsÂ
There is no unified definition or framework for dexterity, and it can be defined from multiple perspectives:Â
Robot capabilities (e.g., motion primitives between robot and object),Â
Task actions (e.g., in-hand object translation/rotation, wrist range of motion), and
Characteristics (e.g., degrees of freedom of arm/hand/finger, obstructed sensor views during performance, proprioception/vision/tactile).
Existing performance-based tests defined for human dexterity (e.g., Purdue pegboard test) and robot benchmarking tools (e.g., NIST-ATB, WBCD)
Benchmarking: Are there dexterity-specific metrics to aid in defining levels of dexterity? Current benchmarks only use success/failure, lacking detail
Reproducibility: What additional considerations are there for reproducing a dexterous capability? Harsher or relaxed performance thresholds? Force? Accuracy?
Standards: What terminology should be defined and standardized (other than “dexterity”)? What stakeholders are needed to shape these standards?
Other: Are there taxonomies of dexterity (or related concepts) that should be leveraged? What requirements are there to consider a robot “dexterous”?