Reconstruction of a 3d Mesh of a Person

Simple SummaryContent extracted from patent full text and abstract with AI.

This invention presents a method and device for reconstructing an accurate 3D mesh of a person from a single 2D image using a combination of neural networks, advanced mathematical modelling, and camera geometry. By predicting both 2D and 3D vertex data from the image and combining them via articulated mesh models (like SMPL) and camera information, the system can estimate not only the person's shape but also their precise pose, all from just one photo. The approach utilizes a new solver (PLIKS) that makes the reconstruction process both analytically robust and suitable for end-to-end machine learning.

Use CasesContent extracted from patent full text and abstract with AI.

• Medical imaging and patient modeling for surgery planning or radiological procedures.
• Virtual try-on systems for fashion and e-commerce, where users can see clothing on their own body shapes from a single photo.
• Animation, gaming, and film production for generating 3D characters from photographs.
• Augmented reality (AR) and virtual reality (VR) applications requiring real-time, accurate 3D human representations.
• Surveillance and security systems to recognize and track human posture and identity with limited camera data.
• Ergonomic assessment or sports performance analysis using quick 3D body reconstruction.

BenefitsContent extracted from patent full text and abstract with AI.

• Enables accurate 3D human body reconstruction from just one image, making it highly accessible and practical.
• Reduces the need for expensive or multiple camera setups, making 3D data capture easier and lower cost.
• Allows incorporation of real camera parameters for more precise depth and shape estimation compared to weak-perspective methods.
• Adapts easily to different body models, including neutral, gendered, or special postures (e.g., lying down), with minimal retraining.
• Fully differentiable and end-to-end trainable, increasing accuracy over time with machine learning.
• Can add environment or geometric constraints without retraining, providing flexibility for specialized needs (e.g., medical, motion capture).
• Improves performance over state-of-the-art solutions across public 3D pose and shape estimation benchmarks.

Technical Classifications (CPCs)

Inventors & Applicants

Patent Abstract

A 3D mesh of a person shall be reconstructed based on one single 2D image. For this purpose, 2D vertex projections and 3D vertex projections (6) of the person are predicted based on the single 2D image (1). An approximated pose (13) is estimated from the 3D vertex projections (6). The shape and/or pose, i.e. the 3D mesh (14) of the person, is computed from the predicted 2D vertex projections and from the approximated pose (13) by using a pregiven camera model and an articulated 3D mesh model of a human body.