Release dev2

.gitignore

@@ -1,7 +1,159 @@
 build
 dist
 dannce.egg-info
-__pycache__
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+# PyInstaller
+#  Usually these files are written by a python script from a template
+#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+#   For a library or package, you might want to ignore these files since the code is
+#   intended to run in multiple environments; otherwise, check them in:
+# .python-version
+
+# pipenv
+#   According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
+#   However, in case of collaboration, if having platform-specific dependencies or dependencies
+#   having no cross-platform support, pipenv may install dependencies that don't work, or not
+#   install all needed dependencies.
+#Pipfile.lock
+
+# poetry
+#   Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
+#   This is especially recommended for binary packages to ensure reproducibility, and is more
+#   commonly ignored for libraries.
+#   https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
+#poetry.lock
+
+# PEP 582; used by e.g. github.com/David-OConnor/pyflow
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# PyCharm
+#  JetBrains specific template is maintainted in a separate JetBrains.gitignore that can
+#  be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
+#  and can be added to the global gitignore or merged into this file.  For a more nuclear
+#  option (not recommended) you can uncomment the following to ignore the entire idea folder.
+#.idea/
+
 dannce/engine/__pycache__/
 dannce/labeling/__pycache__/
 old_repo/
@@ -31,6 +183,11 @@ demo/markerless_mouse_1/DANNCE/predict_results/
 demo/markerless_mouse_1/DANNCE/train_results/AVG/weights*
 demo/markerless_mouse_1/DANNCE/predict_test/
 
+demo/markerless_mouse_1/DANNCE/train_test_ln/
+demo/markerless_mouse_1/DANNCE/train_test_in/
+demo/markerless_mouse_1/DANNCE/train_test_in_dgp/
+demo/markerless_mouse_1/DANNCE/train_test_ln_dgp/
+
 demo/markerless_mouse_1/COM/train_test/
 demo/markerless_mouse_1/COM/train_test/logs/*
 demo/markerless_mouse_1/COM/train_test/*.pickle
@@ -54,3 +211,8 @@ demo/markerless_mouse_2/DANNCE/predict_results/copy*
 
 demo/markerless_mouse_1/videos/
 demo/markerless_mouse_2/videos/
+
+tests/configs/alabel3d_temp_dannce.mat
+tests/configs/log_file.txt
+
+.vscode/

README.md

@@ -21,8 +21,8 @@ DANNCE (3-Dimensional Aligned Neural Network for Computational Ethology) is a co
 
 |      OS               | Python | TensorFlow | CUDA | cuDNN | PyTorch |
 |:---------------------:|:------:|:----------:|:----:|:-----:|:-------:|
-| Ubuntu 16.04 or 18.04 |  3.7.x |   2.2.0 - 2.3.0  | 10.1 |  7.6  |  1.5.0 - 1.7.0  |
-| Windows 10            |  3.7.x |   2.2.0 - 2.3.0  | 10.1 |  7.6  |  1.5.0 - 1.7.0  |
+| Ubuntu 16.04 or 18.04 |  3.7.x |   2.6.0  | 11.1 |  8.1  |  1.9.1  |
+| Windows 10 **_Not yet retested_**           |  3.7.x |   2.6.0  | 11.1 |  8.1  |  1.9.1  |
 
 We recommend installing DANNCE using the following steps:
 
@@ -35,13 +35,13 @@ cd dannce
 2. If you do not already have it, install [Anaconda](https://www.anaconda.com/products/individual).
 
 3. Set up a new Anaconda environment with the following configuration: \
-`conda create -n dannce python=3.7 cudatoolkit=10.1 cudnn ffmpeg`
+`conda create -n dannce python=3.7 cudatoolkit=11.1 cudnn=8.1 ffmpeg -c nvidia -c conda-forge`
 
 4. Activate the new Anaconda environment: \
 `conda activate dannce`
 
 5. Install PyTorch: \
-`conda install pytorch=1.7 -c pytorch`
+`conda install pytorch=1.9.1 -c pytorch`
 
 6. Update setuptools: \
 `pip install -U setuptools`
@@ -98,9 +98,9 @@ The demo should take less than 2 minutes to run on an NVIDIA Titan X, Titan V, T
 
 Please see the *Wiki* for more details on running DANNCE and customizing configuration files. 
 
-## Using DANNCE on your data
+# Using DANNCE on your data
 
-### Camera Calibration
+## 1. Camera Calibration
 To use DANNCE, acquisition cameras must calibrated. Ideally, the acquired data will also be compressed. Synchronization is best done with a frametime trigger and a supplementary readout of frame times. Calibration is the process of determining the distortion introduced into an image from the camera lens (camera intrinsics) and the position and orientation of cameras relative to one another in space (camera extrinsics). When acquiring our data, we typically calibrated cameras in a two-step process. We first used a checkerboard to find the camera intrinsics. We then used an 'L-frame' to determine the camera extrinsics. The L-frame is a calibrated grid of four or more points that are labeled in each camera. A checkerboard can also be used for both procedures. We have included two examples of calibration using MATLAB (in `Calibration/`).
 
 Some tips:
@@ -120,12 +120,12 @@ Cameras tested:
 3. Basler ace aca1920-155uc
 
 
-## Formatting The Data
+## 2. Formatting The Data
 DANNCE requires a set of videos across multiple views and a `*dannce.mat` file that contains camera calibration parameters, a structure that synchronizes frames across views, and in the case of training, the 3D labels. 
 
 We recommend setting up individual project folders for each video recording session, as in the dannce demos (`./demo`), although the dannce configuration files are flexible enough to support more custom file and directory organizations. The demo project folders also contain examples of all of the following formatting information.
 
-**video directories**.
+**2.1-video directories**.
 
 DANNCE requires a parent video directory with *n* sub-directories, one for each of *n* cameras. Within each subdirectory, videos must be named according the frame index of the first frame in the file. For example, for a three-camera system, the video directory must look like:
 
@@ -144,44 +144,77 @@ DANNCE requires a parent video directory with *n* sub-directories, one for each
 |\_\_+--0.mp4
 
 
-**configuration files**.
+**2.2-configuration files**.
 
 `DANNCE` uses two configuration files and one data file. 
 
 - *main config*, e.g. `configs/*.yaml`. This file defines data and model hyperparameters. It can be reused across experiments.  
 - *io config*, e.g. `demo/markerless_mouse_1/io.yaml`. This file defines input data and ouput directories. It is used for a single experiment. 
 - *dannce.mat*, e.g. `demo/markerless_mouse_1/label3d_dannce.mat`. This file contains three cell arrays of matlab structures. `params` stores the camera parameters for each camera. `sync` stores a vector that synchronizes all cameras. `labelData` stores the frame identities and 3d labels for hand-labeled frames. This file can be produced automatically with `Label3D.exportDannce()`.
 
-**camera calibration parameters**.
+**2.3-camera calibration parameters**.
 Dannce requires structs for each camera containing the camera's rotation matrix, translation vector, intrinsic matrix, radial distortion, and tangential distortion. If you use our included calibration scripts, you can convert the output to the required format with `utils/convert_calibration.m`. 
 
 A properly formatted calibration struct has the following fields, `['R','t','K','RDistort','TDistort']`. 
 
-**synchronization files**.
+**2.4-synchronization files**.
 DANNCE requires a set of sync structs, one for each camera, which define frame synchrony across the different cameras over time. If you know your cameras are reliably synchronized at all times (e.g. via hardware triggering), these files can be generated with the aid of `dannce/utils/makeSyncFiles.py`. Once your video directories are set up correctly, sync files can get generated by running `python dannce/utils/makeSyncFiles.py {path_to_videos} {acquisition_frame_rate} {number_tracked_landmarks}`, where {.} denotes variables you must replace with relevant values. See the `makeSyncFiles.py` docstring for more information.
 
 If your cameras are not natively synchronized, but you can collect timestaps for each frame, sync files should be generated by `dannce/utils/preprocess_data.m`, which will generate sync files from a properly formatted `.mat` file listing the frameID for each camera at each timepoint. See `/dannce/utils/example_matchedframs.mat` file for how these timestamp data should be formatted before running `preprocess_data.m`.
 
-## Hand-Labeling
+## 3. Hand-Labeling
 For fine-tuning DANNCE to work with your animal and system, we developed a labeling GUI, which can be found in a separate repo: https://github.com/diegoaldarondo/Label3D. The `Label3D` repository should be cloned with DANNCE automatically as a submodule when using `git clone --recursive https://github.com/spoonsso/dannce` When labeling is completed, the labels can be used to train DANNCE and the COMfinder network (see below) after converting the Label3D files to DANNCE format using `Label3D.exportDannce()`.
 
-### Training and Predicting with the COMfinder U-Net
+## 4. Training and Predicting with the COMfinder U-Net
 DANNCE requires a reasonable estimate of the 3D position of the animal in each frame. We obtain this by triangulating the 2D center of mass (COM) of the animal in each frame. You can use your favorite method to find an estimate of the animal COM in each frame, but we trained a 2D U-Net to do it. Our U-Net typically requires some additional training data to get it working on new views, new environments, and new species. If working with hand-labeled data, your same data structures can be used to train both the COMfinder network and the DANNCE network.
 
-Given formatted data, a properly organized directory structure, and a config file (see config and demo folder, and wiki), navigate to your project folder and run 
-`com-train /path/to/main_com_config.yaml`
+### 4.1 Training COMfinder U-Net
+1. Get the data formatted and directory structure organized as described in section 2.1
+2. Get the main config and other config files set as described in 2.2. Note, different _io.yaml_ and _dannce.mat_ config files need to be present in each project directory.(see config and demo folder, and wiki)
+3. Navigate to your project folder and run `com-train /path/to/main_com_config.yaml` The trained COM network weights will go into `com_train_dir` defined in the io.yaml. 
 
+### 4.2 Generating COM predictions
 After training, run 
 `com-predict /path/to/main_com_config.yaml`
-to generate center of mass predictions.
+to generate center of mass predictions. 
 
-### Training and Predicting with DANNCE
+The COM predictions may be stored in a separate file or appended to the `label3d_dannce.mat` file.
 
-Once the COM is found, the main DANNCE network can be trained by running: 
+## 5. Training and Predicting with DANNCE
+
+### 5.1 Training DANNCE
+1. Get the COM predictions for your experiments (Section 4.1 and 4.2)
+2. If you want to run Dannce from your experiments directory, you can simply train dannce by running
+`dannce-train /path/to/main_config.yaml`
+3. The training step for DANNCE can be run from a directory other than the project directory. To do this one needs to define the `label3d_dannce.mat` file and corresponding video directories using the `exp` option. Check the expanded io.yaml
+4. Once the io.yaml is set appropriately, dannce can be trined by running 
 `dannce-train /path/to/main_config.yaml`
 
-After training, run
+### 5.2 Generating DANNCE Predictions
+1. Navigate to the project folder
+2. Make sure that the label3d_dannce.mat and io config files are present and properly configured. The io file should point to the weights file which needs to be loaded.
+3. Also verify the directory structure for videos. Note, path to videos can be modified in the io to point to exact video file location, and will default to `./videos`.
+4. Run `dannce-predict /path/to/main_config.yaml` from within an experiment/project folder to make 3D predictions using the trained model.
 
-`dannce-predict /path/to/main_config.yaml` from within an experiment folder to make 3D predictions using the trained model.
 
 Consult the demo folder for directory and config file formatting examples
+
+__*Note*__: All the dannce scripts should be ideally run from the project folders. This is because, dannce scripts look for all config files (except main_config), videos and `dannce.mat` file inside the directory from which they are called. `dannce-train` could be run from a separate folder, but the directory paths need to be adjusted accordingly in both io config and main config. 
+
+__*Note*__: If you are running dannce scripts on a slurm cluster, there are additional config files that can be set according to your node configurations. Please refer to Wiki section Dannce on Slurm HPC for more details
+
+## Contributing
+
+Before you push into the repository, make sure to run the standard tests on the code. The `tests` directory contains all the standard tests to be run. You will need to have the weights and videos in order to run the tests.
+
+After your changes are done, the tests can be run by using the following simple steps:
+ - Go to the dannce install directory
+ - Do a pip install
+ - run `tests/tests.sh`
+```
+    cd dannce
+    pip install -e .
+    sh tests/tests.sh
+``` 
+
+If you think the functionality you have included needs to be tested separately, you can include a test for the same under `tests/cli_test.py`