Sequentially grasping multiple objects with multi-fingered hands is common in daily life, where humans can fully leverage the dexterity of their hands to enclose multiple objects. However, the diversity of object geometries and the complex contact interactions required for high-DOF hands to grasp one object while enclosing another make sequential multi-object grasping challenging for robots. In this paper, we propose SeqMultiGrasp, a system for sequentially grasping objects with a four-fingered Allegro Hand. We focus on sequentially grasping two objects, ensuring that the hand fully encloses one object before lifting it and then grasps the second object without dropping the first. Our system first synthesizes single-object grasp candidates, where each grasp is constrained to use only a subset of the hand's links. These grasps are then validated in a physics simulator to ensure stability and feasibility. Next, we merge the validated single-object grasp poses to construct multi-object grasp configurations. For real-world deployment, we train a diffusion model conditioned on point clouds to propose grasp poses, followed by a heuristic-based execution strategy. We test our system using 8 × 8 object combinations in simulation and 6 × 3 object combinations in real. Our diffusion-based grasp model obtains an average success rate of 65.8% over 1600 simulation trials and 56.7% over 90 real-world trials, suggesting that it is a promising approach for sequential multi-object grasping with multi-fingered hands.
Overview of SeqMultiGrasp. (a) Contact candidates for different grasp types, where red and blue dots are used for pinch-like and side grasps, respectively. (b) Multi-object grasp configuration generation, where validated single-object grasps are merged into feasible multi-object grasps. (c) Real-world grasp proposal process, where a diffusion model conditioned on point clouds generates grasp poses. (d) Illustration of heuristic-based sequential grasping execution.
We thank Haozhe Lou, Zhiyuan Gao, Yiyang Ling, Chaoyi Pan, and Haoran Geng for their helpful technical advice and discussions. We are especially grateful to Yuyang Li for his invaluable help and support.
@article{he2025sequential,
title={Sequential Multi-Object Grasping with One Dexterous Hand},
author={Sicheng He and Zeyu Shangguan and Kuanning Wang and Yongchong Gu and Yuqian Fu and Yanwei Fu and Daniel Seita},
journal={arXiv preprint arXiv:2503.09078},
year={2025}
}