Pose_to_SMPL

Fitting SMPL Parameters by 3D-pose Key-points

The repository provides a tool to fit SMPL parameters from 3D-pose datasets that contain key-points of human body.

The SMPL human body layer for Pytorch is from the smplpytorch repository.

Setup

1. The smplpytorch package

• Run without installing: You will need to install the dependencies listed in environment.yml:

  • conda env update -f environment.yml in an existing environment, or
  • conda env create -f environment.yml, for a new smplpytorch environment

• Install: To import SMPL_Layer in another project with from smplpytorch.pytorch.smpl_layer import SMPL_Layer do one of the following.

  • Option 1: This should automatically install the dependencies.

    git clone https://github.com/gulvarol/smplpytorch.git
    cd smplpytorch
    pip install .

  • Option 2: You can install smplpytorch from PyPI. Additionally, you might need to install chumpy.

    pip install smplpytorch

2. Download SMPL pickle files

• Download the models from the SMPL website by choosing "SMPL for Python users". Note that you need to comply with the SMPL model license.
• Extract and copy the models folder into the smplpytorch/native/ folder (or set the model_root parameter accordingly).

3. Download Dataset

• Download the datasets you want to fit

  currently support:

  • HumanAct12
  • CMU Mocap
  • UTD-MHAD
  • Human3.6M
  • NTU
  • HAA4D

• Set the DATASET.PATH in the corresponding configuration file to the location of dataset.

Fitting

1. Executing Code

You can start the fitting procedure by the following code and the configuration file in fit/configs corresponding to the dataset_name will be loaded (the dataset_path can also be set in the configuration file):

python fit/tools/main.py --dataset_name [DATASET NAME] --dataset_path [DATASET PATH]

2. Output

• Direction: The output SMPL parameters will be stored in fit/output

• Format: The output are .pkl files, and the data format is:

  {
  	"label": [The label of action],
  	"pose_params": pose parameters of SMPL (shape = [frame_num, 72]),
  	"shape_params": pose parameters of SMPL (shape = [frame_num, 10]),
  	"Jtr": key-point coordinates of SMPL model (shape = [frame_num, 24, 3])
  }
  

Citation

This repo is part of the Pangea project, if you find it useful for your research, please consider citing:

@article{li2023isolated,
  title={From isolated islands to pangea: Unifying semantic space for human action understanding},
  author={Li, Yong-Lu and Wu, Xiaoqian and Liu, Xinpeng and Dou, Yiming and Ji, Yikun and Zhang, Junyi and Li, Yixing and Tan, Jingru and Lu, Xudong and Lu, Cewu},
  journal={arXiv preprint arXiv:2304.00553},
  year={2023}
}