PP-Tac: Paper Picking Using
Tactile Feedback in Dexterous Robotic Hands
Pei Lin*, Yuzhe Huang*, Wanlin Li*, Jianpeng Ma, Chenxi Xiao†, Ziyuan Jiao†
* Denotes equal contribution; † Co-corresponding authors
Picking various paper-like objects from various terrains.
Robots are increasingly envisioned as human companions, assisting with everyday tasks that often involve manipulating deformable objects. Recent advancements in robotic hardware and embodied AI algorithms have expanded the range of tasks robots can perform. However, current systems still struggle with handling thin, flat objects like paper and fabric due to limitations in motion planning and perception. This paper introduces PP-Tac, a robotic system designed specifically for handling paper-like objects. We developed a multi-fingered robotic hand equipped with high-resolution tactile sensors that provide omnidirectional feedback, enabling slippage detection and precise friction control with the material. Additionally, we created a grasp trajectory synthesis pipeline to generate a dataset of flat-object grasping motions and trained a diffusion policy for real-time control. This policy was then transferred to a real-world hand-arm platform for extensive evaluation. Our experiments, involving both everyday objects (e.g., plastic bags, paper, cloth) and more challenging materials (e.g., kraft paper handbags), achieved a success rate of 87.5%. By leveraging tactile feedback, our system also adapts to varying surfaces beneath the objects. These results demonstrate the robustness of our approach. We believe PP-Tac has significant potential for applications in household and industrial settings, such as organizing documents, packaging, and cleaning, where precise handling of flat objects is essential.
Paper
Latest version: arXiv or here.
Code and Tutorial
Tactile Sensor
At first, let me show you some results of our new designed visual-based tactile sensor!In-the-wild Generalization Experiments
(1) Randomly placed book📕
(2) Randomly placed keyboard⌨️ and book📕
(3) Randomly placed plate🍽️
Human Interference
During plastic bag grasping trials, we systematically introduced controlled perturbations to evaluate the robustness of the grasping process.Other Ability
We posit that the proposed framework exhibits strong extensibility and can be readily adapted to various manipulation tasks beyond the current scope.(1) In-hand object reorientation
By synthesising different fingertip trajectories of manipulation, our model can achieve in-hand object reorientation with only tactile and proprioceptive feedback.
Rotate an orange in hand
Pinch and roll a light bulb
(2) Continuous global exploration of object geometry
The fundamental challenge lies in maintaining continuous finger-object contact through tactile feedback, which inherently enables global exploration and manipulation capabilities.
Contact
If you have any questions, please feel free to contact Pei Lin.