Skip to main content

Main menu

  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Blog
    • Collections
    • Podcast
  • TOPICS
    • Cognition and Behavior
    • Development
    • Disorders of the Nervous System
    • History, Teaching and Public Awareness
    • Integrative Systems
    • Neuronal Excitability
    • Novel Tools and Methods
    • Sensory and Motor Systems
  • ALERTS
  • FOR AUTHORS
  • ABOUT
    • Overview
    • Editorial Board
    • For the Media
    • Privacy Policy
    • Contact Us
    • Feedback
  • SUBMIT

User menu

Search

  • Advanced search
eNeuro
eNeuro

Advanced Search

 

  • HOME
  • CONTENT
    • Early Release
    • Featured
    • Current Issue
    • Issue Archive
    • Blog
    • Collections
    • Podcast
  • TOPICS
    • Cognition and Behavior
    • Development
    • Disorders of the Nervous System
    • History, Teaching and Public Awareness
    • Integrative Systems
    • Neuronal Excitability
    • Novel Tools and Methods
    • Sensory and Motor Systems
  • ALERTS
  • FOR AUTHORS
  • ABOUT
    • Overview
    • Editorial Board
    • For the Media
    • Privacy Policy
    • Contact Us
    • Feedback
  • SUBMIT
PreviousNext
Research ArticleResearch Article: Methods/New Tools, Novel Tools and Methods

A Layered, Hybrid Machine Learning Analytic Workflow for Mouse Risk Assessment Behavior

Jinxin Wang, Paniz Karbasi, Liqiang Wang and Julian P. Meeks
eNeuro 23 December 2022, 10 (1) ENEURO.0335-22.2022; https://doi.org/10.1523/ENEURO.0335-22.2022
Jinxin Wang
1Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, NY 14642
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Paniz Karbasi
2Lyda Hill Department of Bioinformatics and BioHPC, University of Texas Southwestern Medical Center, Dallas, TX 75390
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Liqiang Wang
2Lyda Hill Department of Bioinformatics and BioHPC, University of Texas Southwestern Medical Center, Dallas, TX 75390
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Julian P. Meeks
1Department of Neuroscience, University of Rochester School of Medicine and Dentistry, Rochester, NY 14642
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • ORCID record for Julian P. Meeks
  • Article
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF
Loading

Article Figures & Data

Figures

  • Extended Data
  • Figure 1.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 1.

    A, Architecture of analytic workflow and behavioral experiment overview. Mouse movement and body parts are tracked by using DeepLabCut software. Behavioral features (e.g., distance, angle_1, angle_2, and velocities of snout and body center) are calculated using DeepLabCut outputs and are used to train a random forest (RF) and a hidden Markov model (HMM) with equal numbers of states, separately. The per-frame predictions from RF and HMM are passed to a second HMM layer (reHMM). The predictions from RF and HMM plus predominate positional features are used to train a third HMM (“reHMM+”). B, Diagram of the behavioral test arena and DeepLabCut labeling. Mice are tracked by an overhead camera during video recording. For DeepLabCut labeling, four mouse body parts (snout, left ear, right ear, and the base of tail) are labeled. Eight labels equally spaced on a circle are used to label the Petri dish. C, Graphical representation of derivative features. D, Ethogram including six behavioral states for mouse risk assessment behavior, including approaching (APP), exploration (EXP), investigating-odor (IVO), hiding (HID), heading-out (HDO), and leaving (LEA).

  • Figure 2.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 2.

    Schematic diagrams of experimental design of risk assessment behavior test. A, Experimental design of Study 1. B, Experimental design of Study 2. C, Experimental design of Study 3.

  • Figure 3.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 3.

    Plots of ground truth (GT) and performance of random forest (RF). A, Graphical representations of the GT behavioral states. Each dot denotes the mouse position (midpoint between the ears) during manually annotated frames, with color indicating the normalized instantaneous velocity of the animal center. At right are polar plots indicating the direction of the mouse head relative to the horizontal axis. B, Feature importance for the RF. C, Confusion matrix for the RF versus the GT. D, Feature importance of the optimized RF, which included rolling median filters of each derivative feature with temporal sliding windows (5, 10, 15, 30, and 60 frames) E, Confusion matrix for the optimized RF versus GT. F, Confusion matrix for the optimized RF versus GT for frames not included in the training set (n = 2218).

  • Figure 4.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 4.

    Performance of the hidden Markov model (HMM). A, Graphical representations of HMM behavioral states for a 6-state classification [HMM(6)]. Each dot denotes the mouse position (midpoint between the ears) during manually annotated frames, with color indicating the normalized instantaneous velocity of the animal center. At right are polar plots indicating the direction of the mouse head relative to the horizontal axis. B, Confusion matrix for the HMM(6) versus the GT. C, State transition matrix of hidden Markov model for 6-state classification. D, Confusion matrix for the HMM for 11-state classification [HMM(11)] versus the GT. E, State transition matrix of hidden Markov model for 11-state classification. Additional data can be found in Extended Data Figures 4-1, 4-2, and 4-3.

  • Figure 5.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 5.

    Performance of reHMM and reHMM+. A, Graphical representations of hidden behavioral states, as predicted by the reHMM. Dots represent the midpoint of the two ears and the color represents the velocity of the body center. Polar plots represent the angle between the head direction vector and the horizontal x-axis. B, Confusion matrix for reHMM versus the GT. C, Graphical representations of hidden behavioral states, as predicted by the reHMM+. D, Confusion matrix for reHMM+ versus the GT.

  • Figure 6.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 6.

    TMT induced a heightened level of fear-like behavior in mice. Analysis was conducted for a 2-min window starting with the first IVO event. A, Confusion matrix for the merged reHMM+ states versus the RF states. For simplicity, the three reHMM+ states matching the RF states approaching (APP), heading-out (HDO), and leaving (LEA) were merged into the back-and-forth (B-A-F) state. Right, Graphical representation of the interpretation of merged reHMM+ states. Red indicates state 0 [hiding(HID)]. Yellow indicates state 1 [exploration(EXP)]. Orange indicates state 2 [back-and-forth(B-A-F)]. Green indicates state 3 [investigating-odor (IVO)]. B, Occupancy analysis for the reHMM+ state IVO. C–E, Transition probabilities between listed reHMM+ states. F, Heatmap illustrating behavioral sequence aligned to the first IVO event. Each row indicates one mouse. Each column indicates the time (0.25 s). The color code is identical to the description in panel A. G, Behavioral sequence similarity, as evaluated by Euclidean distance. H, Plot of linear discriminant analysis; *p < 0.05, one-way ANOVA followed by multiple comparisons tests. For this experiment, 24 mice (n = 8 for each treatment group) were available for analysis. Additional data can be found in Extended Data Figure 6-1.

  • Figure 7.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 7.

    Snake feces stimulated risk assessment behaviors in mice. A–C, Occupancy analysis for listed reHMM+ states. D–F, Transition probabilities between listed reHMM+ states. G, Heatmap illustrating behavioral sequences aligned to the first IVO event. Each row indicates one mouse. Each column indicates the time (0.25 s). Red indicates state 0 [hiding(HID)]. Yellow indicates state 1 [exploration(EXP)]. Orange indicates state 2 [back-and-forth(B-A-F)]. Green indicates state 3 [investigating-odor (IVO)]. H, Behavioral sequence similarity, as evaluated by Euclidean distance. I, Plot of linear discriminant analysis; *p < 0.05, one-way ANOVA followed by multiple comparisons tests; ns: not significant. For this experiment, 16 mice were available for analysis. Additional data can be found in Extended Data Figures 7-1 and 7-2.

  • Figure 8.
    • Download figure
    • Open in new tab
    • Download powerpoint
    Figure 8.

    Snake feces extract promoted risk assessment behaviors in mice. A, B, Occupancy analysis for listed reHMM+ states. C–F, Transition probabilities between listed reHMM+ states. G, Heatmap illustrating behavioral sequence. Each row indicates one mouse. Each column indicates the time (0.25 s). Red indicates state 0 [hiding(HID)]. Yellow indicates state 1 [exploration(EXP)]. Orange indicates state 2 [back-and-forth(B-A-F)]. Green indicates state 3 [investigating-odor (IVO)]. H, Behavioral sequence similarity, as evaluated by Euclidean distance. I, Plot of linear discriminant analysis; Letter codes (e.g., “a”, “b”, “ab”, etc.) identify statistically distinct groups as assessed by one-way ANOVA followed by multiple comparisions tests. For this experiment, six mice were available for analysis. Additional data can be found in Extended Data Figures 8-1 and 8-2.

Extended Data

  • Figures
  • Extended Data Figure 4-1

    Overview of hidden Markov model for 11-state classification. A, Expectation-Maximization (EM) algorithm for hidden Markov model for 11-state classification [HMM(11)] relative to the number of training iterations. B, Bayesian Information Criterion (BIC) scores of HMM(11) relative to the number of hidden states. C, Graphical representations of hidden behavioral states, as predicted by HMM(11). For each behavioral state, dots represent the midpoint of two ears, and the color represents the velocity of the body center. The right polar plot represents the angle between the head direction vector and the horizontal x-axis. Download Figure 4-1, TIF file.

  • Extended Data Figure 4-2

    Performance of the hidden Markov model for 23-state classification. A, Confusion matrix for the HMM for 23-state classification [HMM(23)] versus the GT. B, State transition matrix of hidden Markov model for 23-state classification. Download Figure 4-2, TIF file.

  • Extended Data Figure 4-3

    Overview of hidden Markov model for 23-state classification. A, Expectation-Maximization (EM) algorithm for hidden Markov model for 23-state classification [HMM(23)] relative to the number of training iterations. B, Bayesian Information Criterion (BIC) scores of HMM(23) relative to the number of hidden states. C, Graphical representations of hidden behavioral states, as predicted by HMM(23). The dots represent the midpoint of two ears, and the color represents the velocity of the body center. The right polar plot represents the angle between the head direction vector and the horizontal x-axis. Download Figure 4-3, TIF file.

  • Extended Data Figure 6-1

    Additional analysis of male mouse behavioral responses to TMT and female mouse urine. A, Interval between two consecutive IVOs. B, Duration of each IVO. C, Total number of IVO. D, Latency of the first IVO. E, Total movement distance; *p < 0.05, one-way ANOVA followed by multiple comparison’s tests. A total of 24 mice (n = 8 for each treatment group) were available for analysis. Additional data can be found in Extended Data Figure 6-1 and Extended Data 6-2. Download Figure 6-1, TIF file.

  • Extended Data Figure 7-1

    Additional analysis of male mice behavioral responses to female mouse urine, snake feces, and TMT. A, Interval between two consecutive IVOs. B, Duration of each IVO. C, Total number of IVO. D, Latency of the first IVO. E, Total movement distance; *p < 0.05, one-way ANOVA followed by multiple comparison’s tests. For this experiment, 16 mice were available for analysis. Download Figure 7-1, TIF file.

  • Extended Data Figure 8-1

    Additional analysis of male mouse behavioral responses to snake feces and its extracts. A, Interval between two consecutive IVOs. B, Duration of each IVO. C, Total number of IVO. D, Latency of the first IVO. E, Total movement distance; *p < 0.05, one-way ANOVA followed by multiple comparison’s tests. For this experiment, six mice were available for analysis. Download Figure 8-1, TIF file.

  • Extended Data 6-2

    This is a .csv file containing the ReHMM+ transition matrix presented in Figure 6. Download Extended Data 6, CSV file.

  • Extended Data 7-2

    This is a .csv file containing the ReHMM+ transition matrix presented in Figure 7. Download Extended Data 7, CSV file.

  • Extended Data 8-2

    This is a .csv file containing the ReHMM+ transition matrix presented in Figure 8. Download Extended Data 8, ZIP file.

  • Extended Data 1

    This .zip directory contains six files with example data and code used in this manuscript, along with selected outputs from the reHMM+ workflow. The file named DeepLabCut_raw_data.csv contains a spreadsheet with consolidated raw tracking data outputs from DeepLabCut, which were the inputs for the results presented in Figures 3 and 4 and Extended Data Figures 4-1, 4-2, and 4-3. The files Analytic workflow_1_define behavioral feature.ipynb, Analytic workflow_2_RF and HMM classification.ipynb, and Analytic workflow_3_reHMM and reHMM+ classification.ipynb contain Python notebooks implementing the workflows presented in Figures 3–8 and associated Extended Data Figures Line 43 in Analytic workflow_1_define behavioral feature.ipynb is a generic call to load data, which can be used to load DeepLabCut_raw_data.csv. Executing this code will generate a series of .csv file outputs, which are used later. Line 41 in Analytic workflow_2_RF and HMM classification.ipynb is a generic call to load the .csv file outputs from Analytic workflow_1_define behavioral feature.ipynb. Executing this code will generate a series of .csv file outputs, which are the outputs of the initial RF and HMM models. This also generates .sav files, which contain the model specifications themselves for future use if needed. Line 76 of Analytic workflow_3_reHMM and reHMM+ classification.ipynb is a generic call to load the .csv file outputs from Analytic workflow_2_RF and HMM classification.ipynb. Executing this code will generate .csv files, which contain the reHMM and reHMM+ outputs, as well as .sav files which contain the model specifications themselves for future use if needed. Plotting and analysis of results is user specific, and can be done as needed using general plotting and statistical analysis functions available in Python (e.g., numpy, matplotlib, etc.). Download Extended Data 1, CSV file.

Back to top

In this issue

eneuro: 10 (1)
eNeuro
Vol. 10, Issue 1
January 2023
  • Table of Contents
  • Index by author
  • Masthead (PDF)
Email

Thank you for sharing this eNeuro article.

NOTE: We request your email address only to inform the recipient that it was you who recommended this article, and that it is not junk mail. We do not retain these email addresses.

Enter multiple addresses on separate lines or separate them with commas.
A Layered, Hybrid Machine Learning Analytic Workflow for Mouse Risk Assessment Behavior
(Your Name) has forwarded a page to you from eNeuro
(Your Name) thought you would be interested in this article in eNeuro.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Print
View Full Page PDF
Citation Tools
A Layered, Hybrid Machine Learning Analytic Workflow for Mouse Risk Assessment Behavior
Jinxin Wang, Paniz Karbasi, Liqiang Wang, Julian P. Meeks
eNeuro 23 December 2022, 10 (1) ENEURO.0335-22.2022; DOI: 10.1523/ENEURO.0335-22.2022

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Respond to this article
Share
A Layered, Hybrid Machine Learning Analytic Workflow for Mouse Risk Assessment Behavior
Jinxin Wang, Paniz Karbasi, Liqiang Wang, Julian P. Meeks
eNeuro 23 December 2022, 10 (1) ENEURO.0335-22.2022; DOI: 10.1523/ENEURO.0335-22.2022
Twitter logo Facebook logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • Significance Statement
    • Introduction
    • Materials and Methods
    • Results
    • Discussion
    • Acknowledgments
    • Footnotes
    • References
    • Synthesis
    • Author Response
  • Figures & Data
  • Info & Metrics
  • eLetters
  • PDF

Keywords

  • hidden Markov model
  • machine learning
  • quantification of behavior
  • random forest
  • risk assessment behavior

Responses to this article

Respond to this article

Jump to comment:

No eLetters have been published for this article.

Related Articles

Cited By...

More in this TOC Section

Research Article: Methods/New Tools

  • Cell Density Impacts Population Activity in Human iPSC-Derived Neural Networks
  • Adapting a Two-Photon Scanning Microscope for Simultaneous Single-Photon Imaging of an Infrared Dopamine Sensor
  • Age-Related Decline in Myelin Markers and Oligodendrocyte Density in Rhesus Macaque Prefrontal Cortex
Show more Research Article: Methods/New Tools

Novel Tools and Methods

  • Cell Density Impacts Population Activity in Human iPSC-Derived Neural Networks
  • Adapting a Two-Photon Scanning Microscope for Simultaneous Single-Photon Imaging of an Infrared Dopamine Sensor
  • Age-Related Decline in Myelin Markers and Oligodendrocyte Density in Rhesus Macaque Prefrontal Cortex
Show more Novel Tools and Methods

Subjects

  • Novel Tools and Methods
  • Home
  • Alerts
  • Follow SFN on BlueSky
  • Visit Society for Neuroscience on Facebook
  • Follow Society for Neuroscience on Twitter
  • Follow Society for Neuroscience on LinkedIn
  • Visit Society for Neuroscience on Youtube
  • Follow our RSS feeds

Content

  • Early Release
  • Current Issue
  • Latest Articles
  • Issue Archive
  • Blog
  • Browse by Topic

Information

  • For Authors
  • For the Media

About

  • About the Journal
  • Editorial Board
  • Privacy Notice
  • Contact
  • Feedback
(eNeuro logo)
(SfN logo)

Copyright © 2026 by the Society for Neuroscience.
eNeuro eISSN: 2373-2822

The ideas and opinions expressed in eNeuro do not necessarily reflect those of SfN or the eNeuro Editorial Board. Publication of an advertisement or other product mention in eNeuro should not be construed as an endorsement of the manufacturer’s claims. SfN does not assume any responsibility for any injury and/or damage to persons or property arising from or related to any use of any material contained in eNeuro.