Introduction

Rett syndrome is a neurodevelopmental disorder usually caused by a mutation on the X-linked methyl-CpG-binding protein 2 (MECP2) gene.[1] The condition affects females approximately 1 per 9,000 live female births,[2] and is characterized by a loss of functional hand use and language skills in early childhood with the development of hand stereotypies and impaired mobility.[3] These developmental issues are complicated by frequent occurrence of comorbid conditions. Recent large cross-sectional studies have found marked variability in phenotype in part explained by the type of genetic mutation.[4, 5] With regard to motor abilities, girls and women with mutation p.R270X or p.R168X present with a more severe phenotype, whereas those with p.R133C, p.R294X, and C-terminal deletions are more likely to walk.[4, 5]

Early clinical descriptions suggested that neurological impairments such as hypotonia and weakness in the early years and dystonia and bradykinesia in the later years[6] impacted motor function. It was also reported that the gait may be rigid with a lack of co-ordinated movements of the upper extremities and an unsteady wide base.[6] We previously found that most girls with Rett syndrome learn to sit and approximately half learn to walk during their early development.[7] Some individuals are able to maintain the ability to walk through adulthood,[8] but others develop bradykinesia and increased muscle tone, and with increasing difficulty maintaining upright postures, the ability to walk is lost.[6]

We previously adapted the Gross Motor Function Measure[9, 10] and included several additional items to form a smaller scale suitable for administration to those with Rett syndrome.[11] Families participating in the Australian Rett Syndrome Database (ARSD) were asked to video their daughter performing a set of functional activities and also complete a parallel parent-report checklist. Observing the videos, we classified each motor skill according to the amount of assistance required and demonstrated substantial to excellent inter-rater reliability for coding each of the items.[11] Using principal components analysis, 15 items were reduced to two subscales, one describing general motor skills and the other more complex gross motor skills.[12] This initial validation was promising: the factors were conceptually consistent with different aspects of motor function and we found a general decline in motor skill capacity with increasing age.[12] However, the ARSD is now custodian to a substantially larger dataset and additional examination of the measurement properties the Rett Syndrome Gross Motor Scale (RSGMS) is justified. Clear understanding of the measurement properties of scales appropriate to Rett syndrome is critical in this era of clinical trials for neurodevelopmental disorders.

Therefore, the objectives of the current study were to replicate a principal components analysis of the RSGMS using our current larger dataset and to assess relationships of motor scores with genotype and age. We also sought to investigate the consistency of observed and parent-reported ratings and to describe the test-retest and within-subject reliability of the RSGMS using clinical data collected at the Danish Rett Syndrome Center.

Materials and Methods

Results

Validation

Video data were available for 255 individuals including 170 (66.7%) collected in 2012, 65 (25.5%) in 2007 and 20 (7.8%) in 2004. Table 1 shows the distribution of the sample by mutation type and age group.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Frequency distribution of mutation type and age group categories for individuals included in the principal components and regression analyses.

https://doi.org/10.1371/journal.pone.0147555.t001

Principal components analysis of the 15 items resulted in the extraction of three factors that accounted for 82% of total data variance (Table 2). Nine items describing skills of sit to stand, standing, walking, side stepping, turning, walking on a slope and stepping over an obstacle loaded strongly on to factor one which was named ‘Standing and Walking’ because the items related to weight bearing activities (Eigenvalue 9.8 and accounting for 43.4% of the variance). Three items describing skills of moving from the floor to standing, picking up an object from the floor from standing and running loaded onto factor two and this was named ‘Challenge’ because of the complexity of the skills (Eigenvalue 1.5 and accounting for 20.9% of the variance). Three items describing sitting on the floor, on a chair and on a stool loaded onto factor three which was named ‘Sitting’ (Eigenvalue 0.92 and accounting for 17.5% of the variance). The Cronbach’s alpha coefficient for the total scale was 0.96, for ‘Standing and Walking’ 0.97, for ‘Challenge’ 0.85 and for ‘Sitting’ 0.83.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Factor loadings for individual scale items onto each of the three factors.

https://doi.org/10.1371/journal.pone.0147555.t002

The median (interquartile range [IQR]) was 10/45 (IQR 1, 28) for the total score and 3/27 (IQR 0, 20) for the Standing and Walking subscale. Not surprisingly, the median (IQR) score for the Sitting subscale of 6/9 (IQR 0, 20) was higher than that for the Challenge subscale of 0/9 (IQR 0, 1). As the majority of individuals were unable to attain better than minimal level of skills included in the Challenge subscale, the score was dichotomised into a binary outcome (0, ≥1). Multivariate relationships between motor scores, age group and mutation are shown in Table 3. In general, scores decreased with increasing age although not significantly for the total and Standing and Walking scores. Compared to children younger than eight years, teenagers received three points less for the Sitting subscale (95% confidence interval [CI] -6, 0; p = 0.038) as did adults (95%CI -6, 0; p = 0.021). Also compared to children younger than eight years, adults were 75% more likely to be unable to score on any of the Challenge subscale items (odds ratio [OR] 0.25; 95%CI 0.10, 0.65; p = 0.004) (Table 3).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Relationships between total and subscale scores, age group and genotype.

https://doi.org/10.1371/journal.pone.0147555.t003

Those with mutations such as a large deletion, p.Arg255* or p.Arg270* generally had lower scores than those with mutations such as p.Arg133Cys, p.Arg294* or p.Arg306Cys. Compared to those with the large deletion, those with the p.Arg133Cys (median 21; 95%CI 3, 39; p = 0.025), p.Arg294* (median 24; 95%CI 5, 43; p = 0.015) or p.Arg306Cys (median 23; 95%CI 3, 43; p = 0.022) mutation achieved higher total scores. Similarly and compared to those with the large deletion, those with the p.Arg133Cys (median 19; 95%CI 6, 32; p = 0.003), p.Arg294* (median 15; 95%CI 2, 28; p = 0.027) or p.Arg306Cys (median 21; 95%CI 7, 35; p = 0.003) mutation achieved higher Standing and Walking scores. Sitting scores were relatively more consistent across the mutation groups but compared to the large deletion, those with the p.Arg306Cys achieved five points more (95%CI 0, 10; p = 0.051). Challenge scores were low across the mutation groups but those with the p.Arg294* had more than four times the odds of achieving a score greater than 0 than those with a large deletion (95%CI 0.82, 23.37; p = 0.084). Those with the p.Arg294* consistently achieved high motor scores across the different scores (Table 3).

Comparing observed and parent-reported levels of assistance for each item, Kappa values indicated moderate to good agreement with values ranging from 0.47 for sit to stand to 0.75 for running (Table 4). Substitution of one observed score with a parent-reported score had minimal effects on the total and each of the subscale scores. The differences in scores were less than half of one point for each scale, ranging from -0.16 to 0.41 for the total score, -0.15 to 0.26 for the Standing and Walking subscale, -0.19 to 0.01 for the Challenge subscale and 0.17 to 0.42 for the Sitting subscale (Table 4).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 4. Kappa values for observed and parent-reported item scores and differences between observed scores and the modified observed scores when one item replaced with the parent reported score.

https://doi.org/10.1371/journal.pone.0147555.t004

Reliability

Thirty-eight girls and women participated in the test-retest analyses at a median (IQR) age of 16.9 (6.8, 34.7) years (S1 Table). All mutation categories were represented (C-terminal deletion [n = 4], early truncating [n = 3], large deletion [n = 4], p.Arg106Trp [n = 1], p.Arg133Cys [n = 1], p.Arg168* [n = 1], p.Arg255* [n = 1], p.Arg270* [n = 1], p.Arg294* [n = 4], p.Arg306Cys [n = 2], p.Thr158Met [n = 10] and other [6]). Assessments were conducted a median (IQR) of seven (7, 7) days apart. Reliability of the two tests for each of the total and subscale scores was strong and ICC (95%CI) values are shown in Table 5. The standard error of measurement was 1.5 for the total score and the minimal detectable difference was 4 points on the 45-point scale, indicating that an observed difference on the same individual of at least this magnitude would be necessary to be 95% confident that the difference was greater than measurement error. The standard error of measurement and minimal detectable difference values for the total and subscale scores are shown in Table 5.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 5. Test retest consistency and within subject measurement error for the total and subscale scores (N = 38).

https://doi.org/10.1371/journal.pone.0147555.t005

Discussion

We have extended our previous assessment of a 15-item gross motor assessment scale for Rett syndrome, and analysed its measurement properties in a larger sample of girls and women. Principal components analysis indicated a three factor structure comprising subscales in relation to sitting, standing, walking and more challenging motor skills, and the total score and each subscale demonstrated strong internal consistency. Mutations associated with a milder phenotype achieved better gross motor scores and overall, there was decline in total and subscale scores with increasing age. There was moderate to good agreement between observed and parent reported skills and use of parent reported data for a single item had minimal effect on scores. Repeatability of the measure in a clinical setting was excellent.

We previously used principal components analysis in an earlier study using data from the ARSD (n = 99) and derived two factors, one to calculate a general motor skills score and the other for more complex motor skills.[12] We have now replicated this analysis using our current larger dataset and found a three factor solution. The Eigenvalues for the factors Standing and Walking, and Challenge were greater than 1.0 and the Eigenvalue for Sitting was just under 1.0, but this latter value was clearly higher than the remaining factors and with strong factor loading, we interpreted the data to indicate the presence of three distinct motor profiles. Standing and Walking represented skills in relation to transitioning from sitting to standing, standing and walking, Challenge represented more difficult transition skills and running, and Sitting represented different methods of sitting. Conceptually, the Challenge and Sitting factors would be appropriate to address variation in those who are more mildly or severely affected and this supports the validity of a three factor solution.

Mutation type accounts for substantial variation in general severity in Rett syndrome.[4, 5, 17] Compared to those with a large deletion and taking into account the effect of age, those with the p.Arg133Cys, p.Arg294* or p.Arg306Cys had better gross motor skills as indicated by higher total scores and the Sitting and Standing and Walking subscale scores. These findings are consistent with the literature. For example, mutations such as the p.Arg133Cys, p.Arg294* or p.Arg306Cys mutations are generally associated with better functional abilities and milder clinical severity, whereas mutations such as the p.Arg270* and the large deletion are generally associated with poorer functional abilities and a more severe clinical severity.[4, 5, 17, 18] Additionally, we found that those with the p.Arg294* mutation had higher scores also for the Challenge subscale. These were the most difficult skills to achieve and for this group, performance in day to day settings would likely enable a much richer capacity to move independently and negotiate complexities within the environment. Predicted relationships between our new factor structure and mutation support the construct validity of the scale.

Age was also associated with gross motor function. Compared with early childhood and taking into account the effects of mutation type, motor scores were substantially reduced in those older than 19 years. The literature indicates that some girls with Rett syndrome will maintain the ability to walk when adults[8, 19] and this was observed in our dataset for the Standing and Walking subscale and total scores on account of the protective effect of a milder mutation. However, others had poorer skills when older,[6, 12] possibly related to the effects of neurological impairments such as bradykinesia[6] and progressive scoliosis. Unexpectedly, sitting scores declined significantly from the teenage years. Decline in sitting at an earlier age could relate also to the impairments of dystonia, bradykinesia and progressive scoliosis but conceivably, could relate to longer time spent sitting, often in a wheelchair with less opportunity for practice of gross motor activities. The alignment of our observations in relation to effects of age in the literature also supports the construct validity of the scale.

We previously assessed chance-corrected agreement for each of the gross motor items when video was assessed by two trained observers and we demonstrated substantial to excellent inter-rater reliability.[12] Comparing observed versus parent reported scores, chance-corrected agreement was poorer. For some items, the Kappa statistic was in lower half of the range for “moderate” agreement and therefore the lower bound of the (95%CI) borders were near the cutpoint between the values classified as “moderate” and “poor” agreement. This could be because parents have different interpretations of what each level of assistance comprised or there could be variability across a day or between days. We requested that video data were collected when the girls or women were well in their everyday settings and so we expect that the observed functional abilities were similar to what would be performed on a regular daily basis. These data highlight the need for clear and precise discussion with parents if estimating skill levels that have not been observed. There was little effect on summed scores when one item of parent report data was used, likely because some parents over- and others under-report their daughter’s skill levels. Therefore, regression can be used as a method of filling in missing values when assessing group data or parent reported data can be used in the calculation of scores if not every item can be observed during a clinical assessment. For total and subscale scores, the high Cronbach’s alpha values indicated strong internal consistency and the high ICC values indicated excellent repeatability. Our findings therefore provide multiple insights into the reliability of the RSGMS with favourable evidence for the internal consistency of the scale and stability of measurement. Finally, for any individual, we are confident that a change in the total score of 4 points would be greater than measurement error. During an assessment, this increase would identify individual improvement and could also be a feasible target when aiming to improve gross motor skills in Rett syndrome.

Rett syndrome is a rare disorder[2] but database infrastructure can be effective in the recruitment and analysis of large sample sizes[13]. The current study has accumulated gross motor data on 293 individuals by combining the resources of the ARSD with those of the multidisciplinary Danish Rett Syndrome Center. The development of well-validated measures is critical in our current era of clinical trials for neurodevelopmental disorders[20] and our data suggest that the RSGMS has potential to be useful in this regard. We acknowledge the limitation that not all skills were observed on every videoed assessment. However, three quarters had complete data and 10 or more of the 15 skills could be scored on more than 95% of videos. Principal components analysis using pairwise rather than listwise deletion allowed us to maximise the use of our data.[14] Additional analyses are necessary to continue building the case for the validation of the RSGMS. For example, future analyses could include relationships between scale scores and general clinical severity, the magnitude of scoliosis, longitudinal trajectories, and importantly assessment of responsiveness to change.

The RSGMS is a measure of everyday performance of gross motor skills for use in Rett syndrome. The measure is feasible to administer and allows assessment of important skills of everyday living. Using a large dataset, our study has allowed comprehensive evaluation of the validity and reliability of the RSGMS and we suggest it has a role to play in clinical monitoring and as an outcome measure in clinical trials.

Supporting Information

Acknowledgments

The authors would like to thank those families who participate in the Australian Rett Syndrome Study and gave their time to participate in this study and to Ms Orla McIlroy for assistance with video coding. We extend our gratitude to the Australian Paediatric Surveillance Unit (APSU) and the Rett Syndrome Association of Australia for their ongoing support in case ascertainment. At the Danish Center for Rett Syndrome, we would like to thank all participating girls and women as well as their families and caregivers. We are also very thankful to all the therapists for conducting assessments. Additionally, we greatly appreciate the assistance from staff in the pre-school/school and day centre settings for their help in facilitating the assessments.