A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives

Extracting behavioral measurements non-invasively from video is stymied by the fact that it is a hard computational problem. Recent advances in deep learning have tremendously advanced predicting posture from videos directly, which quickly impacted neuroscience and biology more broadly. In this primer we review the budding field of motion capture with deep learning. In particular, we will discuss the principles of those novel algorithms, highlight their potential as well as pitfalls for experimentalists, and provide a glimpse into the future.Comment: Review, 21 pages, 8 figures and 5 boxe

AmadeusGPT: a natural language interface for interactive animal behavioral analysis

The process of quantifying and analyzing animal behavior involves translating the naturally occurring descriptive language of their actions into machine-readable code. Yet, codifying behavior analysis is often challenging without deep understanding of animal behavior and technical machine learning knowledge. To limit this gap, we introduce AmadeusGPT: a natural language interface that turns natural language descriptions of behaviors into machine-executable code. Large-language models (LLMs) such as GPT3.5 and GPT4 allow for interactive language-based queries that are potentially well suited for making interactive behavior analysis. However, the comprehension capability of these LLMs is limited by the context window size, which prevents it from remembering distant conversations. To overcome the context window limitation, we implement a novel dual-memory mechanism to allow communication between short-term and long-term memory using symbols as context pointers for retrieval and saving. Concretely, users directly use language-based definitions of behavior and our augmented GPT develops code based on the core AmadeusGPT API, which contains machine learning, computer vision, spatio-temporal reasoning, and visualization modules. Users then can interactively refine results, and seamlessly add new behavioral modules as needed. We benchmark AmadeusGPT and show we can produce state-of-the-art performance on the MABE 2022 behavior challenge tasks. Note, an end-user would not need to write any code to achieve this. Thus, collectively AmadeusGPT presents a novel way to merge deep biological knowledge, large-language models, and core computer vision modules into a more naturally intelligent system. Code and demos can be found at: https://github.com/AdaptiveMotorControlLab/AmadeusGPT.Comment: demo available https://github.com/AdaptiveMotorControlLab/AmadeusGP

KNEE AND ANKLE MUSCLES COACTIVATIONS IN BREASTSTROKE SWIMMING KICK AND RECOVERY: EXPLORATORY APPROACH

The specificities of body position in breaststroke induce important lower limbs solicitations for the swimmers to propel themselves efficiently. Coactivations around the knee and ankle might appear during the powerful leg extension (i.e. push) and for leg replacement (i.e. recovery). The purpose of this exploratory study is to determine muscle activations and coactivations during these two phases at three different velocities. The EMG of four muscles was recorded (BF, RF, GAS and TA). The results showed important activations of the four muscles in the push, contrary to the recovery. However, no significant differences were found for the coactivations in the two phases and for the three velocities. These findings denoted the important resistances occasioned by aquatic environment, both in push and recovery phases, necessitating muscle coactivations to stabilise joints

NEUROMUSCULAR ACTIVATION DURING ROTATION AND PUSH-OFF PHASES OF BACKSTROKE TO BREASTSTROKE TURNING TECHNIQUES IN AGE-GROUP SWIMMERS

The aim of this study was to assess and compare, through electromyography, the neuromuscular activation during the rotation and push-off phases of four backstroke to breaststroke swimming turns. Eight male swimmers volunteered in this study, comparing the open turn, the back flip turn and the crossover turn. The crossover turn was the one that most activated the studied muscle. Erector spinae (ES) and rectus abdominis (RA), as well as latissimus dorsi (LD) were the main activated muscles during rotation phase. Gastrocnemius medialis (GM) and Tibialis anterior (TA) were mainly activated muscles during the explosive action of the push-off phase. These results provided better understanding about neuromuscular contributions during rotation and push-off of turning performance

Breaststroke swimmers moderate internal work increases toward the highest stroke frequencies

A model to predict the mechanical internal work of breaststroke swimming was designed. It allowed us to explore the frequency–internal work relationship in aquatic locomotion. Its accuracy was checked against internal work values calculated from kinematic sequences of eight participants swimming at three different self-chosen paces. Model predictions closely matched experimental data (0.58±0.07 vs 0.59±0.05 J kg−1 m−1; t(23)=−0.30, P=0.77), which was reflected in a slope of the major axis regression between measured and predicted total internal work whose 95% confidence intervals included the value of 1 (β=0.84, [0.61, 1.07], N=24). The model shed light on swimmers ability to moderate the increase in internal work at high stroke frequencies. This strategy of energy minimization has never been observed before in humans, but is present in quadrupedal and octopedal animal locomotion. This was achieved through a reduced angular excursion of the heaviest segments (7.2±2.9° and 3.6±1.5° for the thighs and trunk, respectively, P<0.05) in favor of the lightest ones (8.8±2.3° and 7.4±1.0° for the shanks and forearms, respectively, P<0.05). A deeper understanding of the energy flow between the body segments and the environment is required to ascertain the possible dependency between internal and external work. This will prove essential to better understand swimming mechanical cost determinants and power generation in aquatic movements

Explosive lower limb extension mechanics: An on-land vs. in-water exploratory comparison

International audienc

Different Muscle-Recruitment Strategies among Elite Breaststrokers

International audienc

Ecological kinematics and electromyography approach of the lower limb in breaststroke

International audienc

Knee and ankle muscle coactivations in breaststroke swimming kick and recovery: Exploratory approach

International audienc