WormPose pipeline

There are several commands to run one at a time, please run all the commands from the same folder.

1 - Assemble a dataset

image0

WormPose analyzes single worm videos where the simple non-coiled centerlines have already been calculated with another worm tracker, for example the Tierpsy tracker. The tierpsy format is supported out of the box for a single worm per video, check the Use with Tierpsy tracker guide to setup a Tierpsy dataset.

There is also a simple dataset used in the tutorials, which is a folder of images associated with a HDF5 file for the features. You can also assemble your own dataset composed of worm videos associated with their features (instructions).

2 - Generate training data

image1

The datagen command will create binary files *.tfrecord containing labeled images for training and evaluation, as well as a configuration file. This step can take some time, a system with several CPU cores is recommended.

wormpose datagen DATASET_LOADER_NAME DATASET_PATH

where DATASET_LOADER_NAME is “tierpsy” or “sample_data” or your custom loader name.

Warning

This command generates a file “config.json” storing the information about the data: image size, dataset loader, resize factor etc. This file needs to exist for the following steps to work. If you did not generate the training dataset and just want to predict new data for example, you need to obtain the configuration file (or regenerate a new one with the same parameters) and use the option –config to pass the path of the configuration file to the next scripts.

Troubleshooting

For the error dataset_loader “tierpsy” (or other) not found in the package entry points, please reinstall WormPose. If using Google Colab, please restart the runtime. If running WormPose source code directly, please run pip install -e . at the root level.

3 - Train the neural network

The neural network will be trained on the data generated from the previous command. This step can take some time, a GPU is recommended.

wormpose train DATASET_PATH

4 - Predict

image2

The network predicts the videos of the dataset, then there is a processing step to discard wrong results and obtain a final time series with a consistent head-tail orientation.

wormpose predict DATASET_PATH

Extra optional commands

  • Postprocess the results with interpolation and smoothing

wormpose postprocess DATASET_PATH

  • Export the results in another format (only WCON format supported for Tierpsy data)

wormpose export DATASET_PATH

  • Visualize the predictions as images

wormpose viz DATASET_PATH

  • Calibrate to fine tune the image score threshold used during the prediction step

wormpose calibrate DATASET_LOADER_NAME DATASET_PATH

  • Evaluate the trained model on new synthetic images

wormpose evaluate DATASET_PATH

Outputs

All output files will be stored in a “experiments” folder in the directory where the scripts are ran, or use the --work_dir option to specify a custom directory.

+-- experiments (or the value of work_dir)
|   +-- dataset_name
|   |   +-- config.json
|   |   +-- training_data  // contains the generated files used for training the network
|   |   |   +-- *.tfrecord
|   |   +-- models  // contains the trained models
|   |   +-- results  // contains the video analysis results
|   |   |   +-- video_name0
|   |   |   |   +-- results.h5  // original and new angles and skeletons, image scores
|   |   |   |   +-- images_results.zip  // Optional, when the script visualize_results is run. Visualize predicted skeletons overlaid on top of the original frame.
|   |   |   +-- video_name1
|   |   |   |    ...