3D HAND SHAPE AND MOTION RECONSTRUCTION FROM 2D VIDEOS

Abstract

Hand tracking and 3D reconstruction have emerged as crucial technologies in fields like computer vision, augmented reality, and human-computer interaction. This project aims to create a robust pipeline that transforms 2D hand video sequences into accurate 3D models of hands and their motions. The system leverages the Attention Collaboration-based Regressor (ACR) for two-hand reconstruction and the MANO model for hand shape and pose estimation. Through methods like 2D hand detection, keypoint estimation, and temporal optimization, the system enables smooth and realistic hand movement reconstruction. This technology has practical applications in areas such as sign language interpretation, virtual reality, and robotic control, overcoming challenges like hand occlusion, fast movements, and complex interactions. The results demonstrate significant accuracy and temporal consistency, making the system well-suited for real-time applications.