KBody: Towards general, robust, and aligned monocular whole-body estimation

Nikolaos Zioulis1, James F. O'Brien1, 2,
1Klothed Technologes Inc., 2UC Berkeley
Paper (Long) Supplementary Paper (Short) arXiv Video

KBody offers robust monocular body fitting across a wide variety of images.

Abstract

KBody is a method for fitting a low-dimensional body model to an image. It follows a predict-and-optimize approach, relying on data-driven model estimates for the constraints that will be used to solve for the body’s parameters.

Acknowledging the importance of high quality correspondences, it leverages “virtual joints” to improve fitting performance, disentangles the optimization between the pose and shape parameters, and integrates asymmetric distance fields to strike a balance in terms of pose and shape capturing capacity, as well as pixel alignment.

We also show that generative model inversion offers a strong appearance prior that can be used to complete partial human images and used as a building block for generalized and robust monocular body fitting.

Overview of our KBody framework.

Partial Images

KBody produces higher quality fits when receiving partial images as input compared to other approaches.

Shape Variety

KBody can handle a variety of shapes without sacrificing pose capturing performance.

KBody SHAPY
KBody PyMAF-X
KBody SMPL-X

References

PyMAF-X: Towards Well-aligned Full-body Model Regression from Monocular Images.

SHAPY: Accurate 3D Body Shape Regression using Metric and Semantic Attributes.

SMPL-X: Expressive Body Capture: 3D Hands, Face, and Body from a Single Image.

BibTeX

@inproceedings{zioulis2023kbody,
  author    = {Zioulis, Nikolaos and O'Brien, James F.},
  title     = {KBody: Towards general, robust, and aligned monocular whole-body estimation},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
  month     = {June},
  year      = {2023},
}