Generalizable Volumetric DYnamics MOdeling for Robot Hair Manipulation
Anonymous Authors
Overview
TL;DR
A study of model-based approaches for robot hair manipulation.
DYMO-Hair, a unified, model-based robot system for visual goal-conditioned hair styling, evaluated across diverse hairstyles in simulation and real-world settings.
A 3D volumetric dynamics model for hair combing, generalizable to diverse unseen hairstyles, enabled by a compact 3D latent space of diverse hairstyles and an action-conditioned latent state editing mechanism.
A hair simulator with a novel Position-Based Dynamics (PBD) method for strand-level, contact-rich hair-combing simulation.
Hair State Representation for Dynamics Modeling
Static State
Strands
Point Cloud
Voxel Grid [Proposed]
Grid Corresponding Strand Segments [Proposed]
Dynamics Process
Strands
Point Cloud
Voxel Grid [Proposed]
Grid Corresponding Strand Segments [Proposed]
Design Highlight
Representing hair state as a high-resolution volumetric occupancy grid with a 3D orientation field, enabling:
Capturing both the spatial distribution and the flow direction of hair, achieving high fidelity.
Efficient and robust state estimation from multi-view RGB-D data in the real world.
More structured processing with highly optimized neural operators and dense voxel-level supervision for learning.
Maintaining acceptable accuracy when resolution is sufficiently high.
Hair Combing Dynamics Learning
Design Highlight
A novel two-phase neural dynamics learning paradigm, designed for generalizable volumetric hair dynamics modeling.
Phase 1: Pre-training a compact 3D latent space for diverse hair states (various hairstyles and deformations).
Phase 2: Adapting the pre-trained model with a ControlNet-style framework for dynamics learning, formulating the dynamics as an action-conditioned latent state editing process.
With all three constraints, the hair maintains a realistic shape.
The twist constraint, in particular, preserves the strand's curvature and prevents gravity-induced oversmoothing.
Dynamics Learning Results on Unseen Hairstyles
Key Takeaway
Our model outperforms all baselines in capturing local deformation for unseen hairstyles, highlighting the effectiveness and generalizability of our model on hair-combing dynamics modeling.
Hair Styling Simulation Results on Unseen Hairstyles
All Method Comparisons on Hard Cases
Goal
DYMO-Hair (Ours)
Rule-based System
PC-GNN System
V-UNet System
V-FiLM System
Goal
DYMO-Hair (Ours)
Rule-based System
PC-GNN System
V-UNet System
V-FiLM System
Goal
DYMO-Hair (Ours)
Rule-based System
PC-GNN System
V-UNet System
V-FiLM System
DYMO-Hair (Ours) on More Various Messy Cases
Goal
Case A
Case B
Goal
Case A
Case B
Goal
Case A
Case B
Goal
Case A
Case B
Goal
Case A
Case B
Goal
Case A
Case B
Key Takeaway
DYMO-Hair successfully styles a wide range of unseen hairstyles with varying difficulty, surpassing both the state-of-the-art rule-based system and all model-based baselines.
Compared to the state-of-the-art rule-based system, DYMO-Hair reduces the final geometric error by 22% and achieves a 42% higher absolute success rate on average.
This highlights the benefit of incorporating a dynamics model and demonstrates the superior performance of our advanced model in boosting closed-loop manipulation at the system level.
Hair Styling Real-world Tests on Wigs
Rule-based System
DYMO-Hair (Ours)
Rule-based System
DYMO-Hair (Ours)
Key Takeaway
Real-world tests exhibit zero-shot transferability of DYMO-Hair to wigs, achieving consistent success on challenging unseen hairstyles where the state-of-the-art rule-based system fails.
Compared to the state-of-the-art rule-based system, DYMO-Hair is more robust to environmental variations, generalizes across hair lengths, and can effectively capture spatially long-horizon goals.
