Lateral: 3D Measure Me

Our task was to create 3D models of people from a single 2D image typically taken from a smart phone. The models would then be used in down stream phone-based applications for clothing based use cases.

Given we only had a total of one month to work on this project our scope was limited to reviewing and running open-source papers and code solutions to this problem.

We had to work with several different programming languages and deep learning frameworks as well as specific library versions of CUDNN. We leveraged docker containers for each environment.

Overall BodyNet (Approach 4) proved to be the best solution for reconstructing 3D volumetric models of people in real world coordinates. Later we fitted gender specific SMPL models to the bodynet output. We found that span error in measurements ranged from 2 to 9% in subjects (our staff) photographed.

Approach 1: HMR, End to end recovery of Human Shape & Pose
Original Paper: https://akanazawa.github.io/hmr/
Open Source Tensor Flow & Pytorch implementations

Approach 2: Unite The People
Original Paper: http://up.is.tuebingen.mpg.de
Open Source Caffe2 implementation

Approach 3: Dense Pose
Original Paper: https://arxiv.org/abs/1802.00434
Open Source Caffe2 Implementations

Mapping between pixel coordinates, and patch coordinates (I,U,V)

plotting patch 1 U, V coordinates from normal & scaled images

Approach 4: Bodynet
Original Paper: https://arxiv.org/abs/1804.04875
Open Source Torch & Matlab