IC-1468DECAF's optimization-based fitting degrades under significant self-occlusion where the hand covers more than half the face

Qingxuan Wu, Zhiyang Dou, Sirui Xu, Soshi Shimada, Chen Wang, Zhengming Yu, Yuan Liu, Cheng Lin, Zeyu Cao, Taku Komura, Vladislav Golyanik, Christian Theobalt, Wenping Wang, Lingjie Liu

SourceDICE: End-to-end Deformation Capture of Hand-Face Interactions from a Single Image

The paper demonstrates that DECAF, which depends on accurate initial keypoint estimates to drive its test-time optimization, produces poor reconstructions when significant occlusion is present. In cases where the hand covers more than half the face, the 2D keypoint estimators (MediaPipe, 3DDFA) fail to produce accurate initial estimates, causing DECAF's iterative fitting to converge to incorrect hand-face relationships. This is shown qualitatively in Figure 8 with several example images.

Evidence

observational

Caveat

Shown qualitatively with a small number of examples in Figure 8; no systematic evaluation over a defined set of occluded cases is reported

Model

DECAF

Concepts

Failure mode

Datasets

DECAF [eval]

Methods

DECAF [primary]

Related findings

IC-1467

Extraction

automatic-extraction