Real-time Expression Transfer for Facial Reenactment

Simple SummaryContent extracted from patent full text and abstract with AI.

This invention provides a method for tracking and manipulating a human face in a video. It separates facial geometry from expressions, allowing expressions to be changed in real time. The system then generates new video where the facial expression is modified based on user input or other data.

Use CasesContent extracted from patent full text and abstract with AI.

• Live facial expression editing for video calls or streaming
• Creating realistic digital avatars for virtual meetings or gaming
• Film or entertainment industry for facial reenactment and dubbing
• Social media apps offering facial animation or filters
• Accessibility tools for assisting communication through facial expression manipulation

BenefitsContent extracted from patent full text and abstract with AI.

• Allows real-time facial expression changes in videos
• Enables more natural interaction in virtual environments or digital avatars
• Improves the realism of facial reenactment for film, media, or communication
• Separates facial structure from expression, allowing targeted editing without altering identity
• Supports a variety of applications from entertainment to accessibility

Technical Classifications (CPCs)

Inventors & Applicants

Patent Abstract

A computer-implemented method for tracking a human face in a target video includes obtaining target video data of a human face; and estimating parameters of a target human face model, based on the target video data. A first subset of the parameters represents a geometric shape and a second subset of the parameters represents an expression of the human face. At least one of the estimated parameters is modified in order to obtain new parameters of the target human face model, and output video data are generated based on the new parameters of the target human face model and the target video data.