Manipulation and Grasping

Picking up a cup seems trivial. You do it dozens of times a day without conscious thought. But consider what is actually happening: your visual system estimates the cup's pose and geometry; your brain plans a grasp that accounts for the cup's weight, fragility, and likely contents; your hand moves to a pre-grasp position; your fingers close with just enough force to secure the cup without crushing it; and throughout the lift, tactile sensors in your fingertips monitor slip and adjust grip force continuously. Reproducing this capability in a robot is one of the hardest open problems in robotics. Manipulation — physically interacting with and reshaping the arrangement of objects in the world — is where robotics meets its deepest challenges.

Contact Mechanics and Grasp Quality

A grasp is a set of contacts between robot fingers and an object that constrains the object's motion. Contact mechanics describes how forces are transmitted through each contact point. The key concept is friction: a contact point can exert a force along the contact normal (pushing) and, up to a limit, along the contact tangent (friction). This friction limit is described by the friction cone: the set of all force directions that can be sustained at a contact without slipping. If the applied force direction lies inside the friction cone, the contact holds. Outside the cone, the finger slips. A grasp is said to achieve force closure if the set of contacts can apply an arbitrary wrench (force + torque) to the object. A force-closed grasp can resist disturbances from any direction — the object cannot be pulled, pushed, or twisted out of the hand. Force closure is the gold standard for reliable manipulation. Grasp quality metrics quantify how well a grasp can resist disturbances. The epsilon quality metric measures the largest wrench ball that fits inside the set of achievable wrenches — intuitively, how much force in the worst direction the grasp can resist before slipping. High-quality grasps have contacts distributed around the object with large friction cones in critical directions. Grasp synthesis algorithms take object geometry as input and output a set of contact points (or finger configurations) that achieve a high-quality grasp. Analytic approaches compute force closure from contact geometry exactly. Learning-based approaches train on large datasets of grasp attempts to predict grasp success probability directly from sensor data — typically point clouds from depth cameras.

End effectors — the robot's 'hands' — vary enormously in design, and the right choice depends heavily on the task. Parallel-jaw grippers have two flat jaws that close symmetrically. They are simple, reliable, and force-controlled. They work well on objects with parallel flat surfaces (boxes, cylinders) but struggle with irregular shapes. They are the dominant gripper in industrial pick-and-place. Three-finger grippers (like the Robotiq 2F-140 or the Barrett Hand) add a third degree of flexibility, enabling power grasps (wrapping around objects) and precision pinch grasps (fingertip contact for delicate objects). They cover a wider range of object geometries. Dexterous hands, like the Shadow Dexterous Hand with 24 degrees of freedom, replicate human hand anatomy. They can manipulate objects within the hand — rolling a pen, turning a key — without setting them down. But they are mechanically complex, expensive, fragile, and extremely difficult to control. In-hand manipulation remains a frontier research problem. Soft grippers, made from compliant silicone or pneumatic chambers, use their physical compliance to conform passively to irregular object shapes without complex grasp planning. They are robust to uncertainty in object pose and shape. Their weakness is that they provide less precise force control and lower grasping strength than rigid grippers.

Sensing and Perception for Manipulation

Grasping requires perceiving the object: its position, orientation, geometry, and surface properties. Modern manipulation pipelines typically combine multiple sensing modalities. Depth cameras (structured-light or time-of-flight) produce point clouds — 3D maps of visible surfaces. Algorithms like PointNet or GraspNet process point clouds directly to predict grasp poses or contact points. Pose estimation algorithms localize known objects (estimating their 6-DOF pose from the point cloud) to enable more precise grasp planning. Tactile sensing is the sense robots most conspicuously lack relative to humans. Human fingertips have about 2,500 tactile receptors per square centimeter, providing rich information about contact force, texture, slip, and object hardness. Most commercial grippers have no tactile sensing at all. Research tactile sensors use arrays of pressure-sensitive elements (capacitive or piezoelectric), embedded cameras filming deforming gel pads (GelSight), or barometric chambers. With tactile feedback, robots can detect incipient slip (a slight reduction in friction that precedes a drop) and increase grip force just enough to prevent it — a reflex humans perform unconsciously. The closed-loop grasp control pipeline is: (1) visual estimation of object pose, (2) grasp synthesis to compute finger placement, (3) arm motion to pre-grasp configuration, (4) finger closing with force control, (5) tactile monitoring during the lift, (6) grip adjustment in response to detected slip or load changes.