Article Figures & Data
Figures
- Figure 1.
The EthoProfiler 10-cage data acquisition system. A, Frontal view of the setup, showing video cameras at the front, two cages per shelf level, and a backlighting panel at the rear, all mounted on a MetroMax shelf. B, Close-up of one shelf, with two adjacent Tecniplast 1145T cages housing mice of different strains (C57BL/6 on the left, SWISS on the right). Each cage contains the following: (a) a wire lid with ad libitum access to food, (b) a water bottle, and (c) paper bedding. Each camera (d) faces one cage, while the backlighting panel (e) provides illumination. Infrared background lighting in the processed videos was more uniform than in the daylight images shown here (Fig. 2). C–E, 3D schematics of one shelf: (C) overview, (D) front view (similar to panel B), and (E) side view.
- Figure 2.
The eight basic behavior categories as defined in Jhuang et al. (2010). Representative frames from the annotated database are shown. Monochromatic images were captured using an infrared (IR) background light and a camera filter blocking wavelengths below 850 nm.
- Figure 3.
The DeepEthoProfile CNN model architecture. Convolutional layers are shown in red, normalization layers in green, pooling layers in blue, and fully connected layers in yellow. The gray boxes represent the 11 input frames.
- Figure 4.
Distribution of annotated frames for each behavior category. The dataset comprises 3,240,000 frames, equivalent to 36 h of video material.
- Figure 5.
Confusion matrices for animal behavior detection rates using data acquired with EthoProfiler. Rows represent the manually annotated behaviors. A, The sum of confusion matrices from 10 leave-one-out tests. In each test, the model was trained on data from nine mice and tested on the remaining mouse. B, Results of training on 90% of the data and testing on the remaining 10%, with all reviewed annotations included in the training set. C, Results from the partial review of the initial human annotation data, where columns represent the reviewed behavior annotations.
- Figure 6.
Mean behavior durations monitored over 48 h. A, Data from SWISS mice (n = 5). B, Data from C57BL/6 mice (n = 5). The data were acquired using the EthoProfiler setup and processed with DeepEthoProfiler. The shaded bar at the top of each figure indicates the dark phase. Data from the first hour after introducing the mice to their new cages were excluded.
- Figure 7.
Confusion matrices for animal behavior detection rates using the dataset from Jhuang et al. (2010). Rows represent the manually annotated behaviors. A, The sum of confusion matrices from 12 leave-one-out tests on the “full database.” In each test, the model was trained on data from 11 videos and tested on the remaining video. B, The sum of confusion matrices from five tests on the “clipped database.” In each test, the model was trained on 50% of the videos (randomly selected) and tested on the remaining 50%. C, Results from the partial review of the initial human annotation, as originally published in Jhuang et al. (2010).
Tables
- Table 1.
Comparison of the accuracy of the model across mouse strains with leave-one-out test results
Mouse Strain DeepEthoProfile Review agreement Frame accuracy Macroaccuracy Frame accuracy Macroaccuracy C57BL/6 80% 81% 84.4% 87.0% SWISS 85.5% 84.5% 87.8% 88.6% - Table 2.
Representation of previously published classification models tested on the data from Jhuang et al. (2010) alongside results obtained with DeepEthoProfile
Method Frame Acc (%) Macro Acc (%) Accuracy for each behavior (%) Drink Eat Groom Hang Mm Rear Rest Walk Full database DeepEthoProfile 74.2 78.7 91 90 70 97 66 80 91 77 Jhuang et al. (2010) 78.3 76.4 72 75 70 92 83 70 94 55 Human (3) 71.6 75.8 78 87 57 91 64 66 95 68 Jiang et al. (2017) 72.9 Jiang et al. (2019) 81.5 79.2 60.3 81.1 82.2 95.8 74.6 74.9 97.1 67.7 CleverSys (3) 60.9 64.0 63 73 30 82 64 35 96 69 Le and Murari (2019) 73.5 76.5 Clipped database DeepEthoProfile 97.6 96.8 98 99 96 100 90 95 100 97 Jhuang et al. (2010) 93 Jiang et al. (2017) 95.9 90.7 72.4 95.7 97.4 97.6 69.8 94.9 99.5 98.3 Dollar (2) 82.2 70.5 41 69 88.4 80.8 32.2 57.9 98.8 96.1 Wang et al. (2015) (1) 96.1 88.5 45.9 92 98.3 98.8 80.6 95.2 99 97.9 Wang et al. (2017) (1) 94.5 87.6 50.8 90.1 97 96 77.8 93.2 97.1 99.1 Jiang et al. (2019) 97.9 94.3 77 97.3 99.1 99.2 86.1 96.7 99.1 99.6 I3D (4) 96.9 R(2 + 1)D (4) 96.3 Nwokedi et al. (2023) (5) 82 Results marked with (1) were published in Jiang et al. (2019), those with (2) were published in Jiang et al. (2017), and those with (3) appeared in Jhuang et al. (2010). The methods marked with (4) were tested in Nguyen et al. (2019). The results from (5) were trained on the “full database” and tested on the “clipped database”.
Extended Data
Download Extended Data, ZIP file.
In this issue
