A toolkit for the dynamic study of elastic kinematics

Burchardt ([email protected]), van der Sande, Kehy, Gamba, Ravignani, Pouw ([email protected])

Biological structures are defined by elements like bones and cartilage, and elastic elements like muscles and membranes. Computer vision advances have enabled automatic tracking of animal skeletal poses. However, the elastic and soft-tissues of organisms, like the nose of Elephant seals, or the buccal sac of frogs, have been poorly studied as no computer vision methods are optimized for tracking such elastic kinematics. This leaves major gaps in different areas in biology. In the area of primatology, most critically, the function of air sacs is widely debated and many questions exist about their role in communication and human language evolution. Moving towards the dynamic study of soft-tissue elastic structures, we present a toolkit for the automated tracking of semi-circular elastic structures in biological video data. The toolkit contains unsupervised computer vision tools (using Hough transform) and supervised deep learning (by adapting Deeplabcut) methodology to track inflation of laryngeal air sacs or other biological spherical objects (e.g., gular cavities).

Toolkit

	Component
✅	Hough transform to detect semi-circles (unsupervised method): https://wimpouw.github.io/AirSacTracker/AirSacTracking_with_Hough.html
✅	Deeplabcut + Landau circle estimation (supervised method): https://wimpouw.github.io/AirSacTracker/DLC+.html
✅	Open dataset on Air Sac- and Articulatory Kinematics in Siamang: https://doi.org/10.34973/6apg-q804

Manuscript pipeline

(pipeline image)

Download trained DLC resnet 101 model (> 100mb)

500K iterations
280K iterations

Installation / requirements

See requirements.txt for each module. You can install the requirements by entering in your terminal 'pip -r requirements.txt' (after navigating to the folder where the requirements.txt is located)

File structure

• Project -> contains all the code and materials for the manuscript
• Docs -> contains all the github pages
• Toolkit -> contains the hough and DLC+ code

Code contributors

Lara Burchardt, Yana van der Sande, Wim Pouw

Reference

Burchardt, L., van de Sande, Kehy, M., Ravignani, A., Pouw, W. (2024). A toolkit for the dynamic study of air sacs in siamang and other elastic circular structures. Plos Computational Biology. DOI: https://doi.org/10.1371/journal.pcbi.1012222