Article Figures & Data
- Figure 1.
A, Setup design. Head-fixed mice are acclimatized to whisker painting and trained to use their whiskers to contact a piezo-film touch sensor. A Pixy camera was used to track whiskers in real time (left), a high-speed color camera was used simultaneously to acquire data. B, Paradigm for whisker task. A sound-cue initiates the trial. The animal whisks one of the two painted whiskers into contact with a piezo-film sensor, and if contact reaches threshold, the animal obtains a liquid reward. There is a minimum inter-trial interval of 10 s. C, Capturing whisker motion in real time. The movement and location of the D1 (green, S = 1, signature 1) and D2 (red, S = 2, signature 2) whiskers shown at two time points, frame 1 and frame 2 (below). The wave form of whisker data reflects the spatial location and the dimensions of the tracked box around the whisker. The waveforms in the middle show the movement of the two whiskers, toward and away from each other.
- Figure 2.
A real-time automated multiple whiskers tracking. A, Pixy data from D1 and D2 whiskers (left, raw and smoothed) or β and γ whiskers (right, smoothed), as a mouse performs five auditory go-cue-triggered trials. A mouse moves a whisker into contact with a piezo-film sensor (bottom). Contact with the sensor triggers a reward. The cue onset and the reward trigger times are marked below the whiskers movement traces. Note that the spatial location of the D1 and D2 whiskers is distinct; the position of the two whiskers rarely overlap. In these trials, the distance between the two whiskers ranged from approximately 2 to 10 mm (distances converted into arbitrary units that denote spatial location). B, Average position during task performance. The D1 and D2 whiskers move differently (left): the average position of the two whiskers at rest is different (before zero), and the average position of the two whiskers at contact is different (at zero). The D2 whisker, which contacts the piezo-film sensor and is rostral to the D1 whisker, moves more than the D1 whisker. In contrast, the two arc whiskers’ position overlaps at rest and at contact, but even here, the average motion of the whisker used to make contact with the sensor is different from the motion of the adjacent whisker. C, Tracking performance by tracking whisker movement over days. The performance of an animal trained in the go cue task was monitored by monitoring the motion of the B2 whisker over days of training. The go-cue-triggered motion of the B2 whisker is task related by d 9 of training (compared with the imperceptible motion of the same whisker after the cue on d 1). D, The contact-triggered motion is also faster and larger by d 9, compared with its motion on d 1 (on the left).
- Figure 3.
Post-hoc automated tracking. A, Diagram of a Pixy camera capturing whiskers motion previously recorded with a high-speed video camera and played back in slow motion on a monitor. B, Comparison of the high-fidelity signature of the D1 and D2 whiskers (top and bottom), recaptured automatically by the Pixy camera in slow motion (orange) with the data acquired in real time (black). C, Motion of the two points on the whisker pad and one whisker are tracked in real time and post hoc in slow motion. The motion of the D1 whisker and the pad-point under the D1 whisker, and the second pad point under the A2 whisker could be tracked easily in real time and the same trials could be examined post hoc with analysis of the slow-motion playback of high-speed video data. The motion of the whisker pad appears to be a filtered version of the whisker motion. The motion of the D1 whisker in both real time (left) and post hoc (right) reveals differences in the set-point of protraction on each of the five trials, but real-time pixy data captures the entire envelope of both the whisker and the pad motion (bottom, expanded record of trial above on right).
- Figure 4.
Single whisker tracking in infrared light. Top, Pixy image of whisker painted with yellow UV light sensitive paint, illuminated with infrared light only and automatically tracked in real time. Bottom, Output from Pixy camera showing periods with infrared (IR ON) and without infrared (IR OFF) illumination.
- Figure 5.
Pixy tracking of two limbs. A, schematic of a mouse head-fixed on a treadmill and the two forepaws, one painted green, the other painted red are tracked with a Pixy camera. B, The positon and velocity of the treadmill and the alternating up and down motion of the limbs are tracked in real time.
- Figure 6.
Tracking head rotation and location of freely moving animals. A, The head rotation (top), x (middle), and y (bottom) coordinates of animal position were simultaneously tracked. Four time points corresponding to the four frames (right) are shown, where the animals head direction, and position in the box change from moment to moment. B, The animal’s position over 3 min was tracked and a heat map of the preferred location was created; red, more time and blue, less time. C, The location of two animals in the same enclosure can be distinctly tracked, including each animals head rotation, and position. Pixy tracking is shown by the boxes around the animal’s head.
Bermejo et al. (1996) Knutsen et al. (2005) Voigts et al. (2015) Ritt et al. (2008) O'Connor et al. (2010) Gyory et al. (2010) Perkon et al. (2011) EMG Pixy
(our method)1 Tracking principle Opto Video Video Video Video Video Video Muscle Opto/video 2 Spatial element Single point Multiple points Multiple points Multiple points Multiple points Multiple points Multiple points NA Multiple points 3 Real time Yes No No No No No No Yes Yes 4 Individual whisker Yes Yes Yes Yes Yes No No No Yes 5 Number of single identifiablewhiskers One whisker on each side One whisker on each side One whisker on each side Three whiskers Five whiskers N/A N/A NA Two whiskers (up to seven in principle) 6 Whisker removal Yes Yes Yes Yes Yes N/A N/A No No 7 Limitation Whisker thickness Contrast andresolution Contrast and resolution Contrast and resolution Contrast and resolution Contrast and resolution Contrast and resolution NA Illumination and color 8 Method shows Single whisker Single row C1-4 whiskers Single whiskers Multiple whiskers Two rows Full whisker pad Whisker pad Two whiskers/one whisker and two pad/two paws 9 Head tracking No Yes Yes Yes No Yes Yes Yes Yes 10 Head tracking requirement N/A Additional light source for the eye Tip of nose Contour edge/whisker base N/A No requirement No requirement Wire in muscle Marker glued to head 11 Compatible in IR Yes Yes Yes Yes Yes Yes Yes Yes Yes (single whisker) 12 Algorithm flexibility Yes (not automatic) No No No No No No Yes (with wires) Yes 13 Unrestrained animal No Yes Yes Yes No Yes Yes Yes Yes (whole animal) Here we compare 13 different features of 7 earlier tracking methods, including optoelectronic (Opto), electromyography (EMG), and other whisker tracking algorithms combined with high-speed videography to our Pixy based method. The elements that we compared here: 1) Tracking principle: Tracking algorithms based on videography, optoelectronic methods like beam breaking, EMG or color. 2) Spatial coordinate system: Beam breaking has a distinct (single or multiple) spatial coordinate, while videography can track over multiple spatial locations. 3) Real-time at any frequency. 4, 5, 6) Number of objects tracked: A single whisker, or multiple individual whiskers, with or without plucking or removing whiskers. 7) Limiting element of each method: Lighting, contrast, resolution and length of whiskers for videography, or color and painting for Pixy, 8) Output: Single whisker, multiple whisker or whisker and whisker pad. 9, 10) Head direction tracking method: possibility, and whether the eye need or tip of the nose or a color needs to be tracked. 11) Ability to tack in infrared red light: All the high speed cameras can work with infrared light, as can EMG and optoelectronic methods. The pixy camera is limited in this context because it can only be used to track a single spatially distinct point with a pixy camera. 12) The flexibility in tracking multiple body parts: Cameras can be used for tracking any object, but optoelectronic methods, and EMGs, and even automated tracking video systems have to be optimized or positioned for tracking the object of interest. 13) The ability to use the system in unrestrained animals.
- Video 1.
Real-time tracking of D1 and D2 whiskers. Left panel shows the real-time data transmitted from Pixy to data files. The top right panel shows the simultaneously acquired high-speed video of the two whiskers, and the bottom right shows Pixy view. The D2 whisker is painted red and shows up as the red waveform on the top left, the D1 whisker is painted green and is the green waveform on the left. The yellow/black boxes are the text mark indicators, showing that Pixy is transmitting data in real time via the USB interface. The positions of the two whiskers do not overlap. They are not at the same point in space at the same time, in the videos or in the waveforms. The set point of both whiskers changes from moment to moment (time 5 s in the video, to 8 s in the video). The actual distance moved in millimeters can be seen in both the high-speed and the Pixy video.
- Video 2.
Pixy analysis of slow-motion video data. The color high-speed video can be played back in slow motion (left panel). Pixy camera and Pixymon (middle panel) can be used to track the position of the two whiskers and the data can be extracted into a data file (right panel).
- Video 3.
Pixy in infrared illumination. A single painted whisker shown in the video on the right is tracked in real time (left panel) with infrared illumination. At 3 s into the video, the infrared light is turned off, whisker color signature disappear as well. When the light is turned on again, the whisker can be tracked again.
- Video 4.
Tracking forepaws movements. The painted limbs can be tracked for alternating up and down movement in real-time. The red trace is for the UP/Down movement of the right limb. The green traces is for the left limb. The treadmill position and velocity are also shown in the traces below.
- Video 5.
Tracking a single animal head rotation/direction and position in real time. Pixy camera tracks a multicolored piece of Styrofoam fixed on animal head-plate in regular light condition. The red traces on the top left shows the angle of head direction, while the blue traces in the middle left and green trace in bottom left shows the horizontal and vertical movement, respectively.
In this issue
