A selfie-based 3D avatar is an inferred reconstruction, not a perfect extraction of the person photographed.
Research has established that detailed facial surfaces can be reconstructed from individual photographs, although pose, illumination, occlusion and missing views remain difficult. A foundational CVPR paper demonstrated an end-to-end neural network for detailed 3D face reconstruction from a single image, while later work such as AvatarMe pursued render-ready facial reconstruction from uncontrolled photographs. More recent research, including FaceLift at ICCV 2025, continues pushing single-image reconstruction towards higher-quality, full-head results.
The technology has improved enormously. The underlying uncertainty has not disappeared.
How AI Finds Depth Where the Photograph Contains None
Depth begins with relationships rather than isolated pixels. One pixel cannot explain a face, but thousands arranged around an eye, jaw or nose contain patterns from which shape can be inferred.
Occlusion provides one of the strongest clues. When the nose hides part of a cheek, or hair covers the forehead, the machine learns that the concealed structure probably continues behind the obstruction. Humans perform this completion unconsciously; a machine must learn it through data and optimisation.
Lighting provides another clue, but it can also deceive. A highlight may indicate a raised surface, while a shadow may reveal curvature. Yet dramatic lighting, makeup, compression artifacts and phone-camera processing can cause the algorithm to confuse illumination with geometry.
Facial landmarks then give the model a more stable map. Google’s current MediaPipe Face Landmarker identifies facial landmarks and expressions in images or video and explicitly lists virtual-avatar creation among its possible applications. A reconstruction system can use comparable landmark information to estimate pose, expression and facial proportions before generating a mesh.
| Visual clue | Likely inference | Common source of error |
|---|---|---|
| One eye appears smaller | That side of the face may be farther from the camera | Lens distortion or facial asymmetry |
| One ear is partly hidden | The head is probably rotated | Hair, clothing or cropping |
| Shadow beside the nose | Nose direction and projection | Uneven or coloured lighting |
| Jaw disappears behind a hand | The jaw continues behind the obstruction | Incorrect completion of hidden geometry |
| Forehead highlight | Possible surface curvature | Oily skin, flash or image enhancement |
| Moving expressions in video | Separation of identity from expression | Motion blur and inconsistent tracking |
A short selfie video can offer more evidence than a single photograph because it exposes the face from several moments or angles. Even then, the software must reconcile movement, expression and lighting into one coherent identity.
Shape, Appearance, Identity and Animation Are Different Problems
A visually impressive avatar must solve several connected problems that are too often collapsed into the vague label of “AI.”
Geometry determines the structure of the head. Texture provides its visible surface. Identity preservation decides whether the result remains recognisably you rather than becoming a generic attractive face. Rigging makes the model movable. Animation drives expressions, speech or body motion.
A system can perform well in one layer and fail badly in another. It may generate smooth geometry while erasing a distinctive nose. It may preserve facial resemblance from the front but become implausible from the side. It may look realistic as a still image but deform unnaturally when the mouth opens.
This is why the original makeAvatar.ai proposition was technically ambitious. It did not merely promise a decorative portrait. It proposed rigged and animated output that could be used elsewhere—a difference comparable to receiving a photograph of a car versus receiving a controllable 3D car model that can be placed inside a game.
Modern face-landmark models, neural rendering, generative reconstruction and richer 3D datasets have made the pipeline more capable. The FaceScape research dataset, for example, reported 16,940 textured 3D faces captured from 847 subjects across 20 expressions, illustrating the scale and variation required to teach systems how identity and expression interact.
