The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery

https://doi.org/10.1016/j.schres.2003.09.011Get rights and content

Abstract

Studies of neurocognitive function in patients with schizophrenia use widely variable assessment techniques. Clinical trials assessing the cognitive enhancing effect of new medications have used neurocognitive assessment batteries that differed in content, length and administration procedures. The Brief Assessment of Cognition in Schizophrenia (BACS) is a newly developed instrument that assesses the aspects of cognition found to be most impaired and most strongly correlated with outcome in patients with schizophrenia. The BACS requires less than 35 min to complete in patients with schizophrenia, yields a high completion rate in these patients, and has high reliability. The BACS was found to be as sensitive to cognitive impairment in patients with schizophrenia as a standard battery of tests that required over 2 h to administer. Compared to healthy controls matched for age and parental education, patients with schizophrenia performed 1.49 standard deviations lower on a composite score calculated from the BACS and 1.61 standard deviations lower on a composite score calculated from the standard battery. The BACS composite scores were highly correlated with the standard battery composite scores in patients (r=0.76) and healthy controls (r=0.90). These psychometric properties make the BACS a promising tool for assessing cognition repeatedly in patients with schizophrenia, especially in clinical trials of cognitive enhancement.

Introduction

The cognitive deficits of schizophrenia are profound and devastating. Patients with chronic schizophrenia demonstrate impairments that range between one and a half to two standard deviations below healthy controls on several key dimensions of cognition, especially verbal memory, working memory, motor speed, attention, executive functions and verbal fluency Saykin et al., 1991, Keefe, 1995, Harvey and Keefe, 1997, Heinrichs and Zakzanis, 1998. These components of cognitive dysfunction in schizophrenia also have an impact on functional outcome Green, 1996, Green et al., 2000 as they are all correlated with poor functional abilities.

Since the second generation of antipsychotic medications became widely available, dozens of studies of the neurocognitive impact of these medications have been completed. While most of these studies have concluded that second generation antipsychotics improve neurocognitive function, the interpretation of these results has been challenged, in part, by the variable test batteries used in each study (Harvey and Keefe, 2001). There is no standard, easily administered test battery that specifically and efficiently assesses the important cognitive deficits in patients with schizophrenia. Instead, current test batteries differ widely in content, duration, procedures, and implementation. The limited generalizability of neurocognitive findings across studies has reduced the ability of clinicians and researchers to make clear conclusions about the relative impact of the various antipsychotic medications on cognition in schizophrenia. The absence of a standard measure has also challenged studies of adjunctive treatments for cognition in patients with schizophrenia Friedman et al., 2001, Friedman et al., 2002, Evins et al., 2000, Heresco-Levy et al., 1996.

Another drawback of current assessments of cognition in treatment studies of patients with schizophrenia is that many of the neurocognitive assessment batteries used are long and complex. Most neuropsychological assessment batteries used in schizophrenia studies have been adapted from clinical neuropsychology, which assesses the entire profile of neuropsychological strengths and weaknesses in individuals. These batteries of tests may require several hours to administer. Their adaptation for schizophrenia research has kept some of the length that was originally necessary for individual assessment, but may no longer be necessary for research such as clinical trials that compares groups of patients. In fact, the length of some assessment batteries may be a limiting factor in assessing patients repeatedly throughout a clinical trial (Harvey and Keefe, 2001).

In contrast to these standard assessment techniques in schizophrenia research and clinical practice, cognitive function in patients with dementia is usually assessed with a widely used cognitive assessment tool, such as the Mini Mental Status Examination (Folstein et al., 1975), Dementia Rating Scale (Gardner et al., 1981), or Alzheimer's Disease Assessment Scale (Rosen et al., 1984). These instruments can be easily administered in typical treatment settings such as a physician's office, and are routinely employed as indicators of treatment change in clinical trials of anti-dementia drugs. The administration of These tests shed light on the global severity of patients' cognitive deficits, track progression, and measure cognitive changes.

The availability of a quick and efficient tool for measuring cognition in patients with schizophrenia could be an extremely useful guide for clinicians making decisions about potential rehabilitation and for researchers implementing clinical trials to assess cognitive improvement. There are several batteries of tests that are available or in development for the purposes of brief cognitive assessment.

Several computerized batteries, such as the Cambridge Neuropsychological Test Automated Battery (CANTAB) (Robbins et al., 1996), the CDR Cognitive Assessment System (Hunter et al., 1997), and the CogTest Battery (Cogtest, 2002) have been applied to schizophrenia samples, and have the option of reduced length. However, portability, regular software and hardware version changes, and increased patient and tester burden present implementation challenges. Other batteries, such as the Cognitive Screening Instrument for Schizophrenia (CSIS), a very short (10 min) set of abbreviated tests (Scot Purdon, personal communication), are currently in development.

The work completed with the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Randolph, 1998, Gold et al., 1999, Hobart et al., 1999 has clearly demonstrated the utility of brief assessment approaches. The RBANS is capable of providing reliable and valid assessments of patients with schizophrenia in various cognitive domains Gold et al., 1999, Hobart et al., 1999, Wilk et al., 2002. As noted in the RBANS manual, however, the test was originally developed as a screening measure primarily for elderly subjects, and the test is heavily weighted by memory, language, and visual–perceptual subtests. In addition, the item difficulties are most relevant to the types of impairment likely to be observed in patients with dementing illnesses such as Alzheimer's Disease, with ceiling effects in some domains. Although the RBANS is clearly sensitive to some of the impairments observed in schizophrenia, and it has utility as a screening tool for patients with schizophrenia, it lacks measures of motor, executive, and working memory performance that may be particularly important targets for cognitive enhancement in schizophrenia. These limitations suggest the need for a measure specifically designed for use in schizophrenia clinical trials that preserves the desirable features of the RBANS: brief administration and scoring time, portablity, repeatability, and availability of alternate forms.

The Brief Assessment of Cognition in Schizophrenia (BACS) has been developed for clinical trials with these key features. In addition, the domains of cognitive function that are assessed by the BACS are those found to be consistently impaired, and consistently related to outcome, in schizophrenia: verbal memory, working memory, motor speed, attention, executive functions and verbal fluency. The BACS is fully portable, and is designed to be easily administered by a variety of testers, including nurse clinicians, psychiatrists, neurologists, social workers, and other mental health workers. It is designed to require about 30 min of testing time with minimal extra time for scoring and minimal training demands.

The development of a psychological assessment instrument should determine the instrument's test–retest reliability, sensitivity, criterion-referenced validity (i.e. comparison to a standard measure), and comparability of alternative forms. This study aims to determine the quality of the BACS in these domains of reliability and validity.

Section snippets

Subjects

Healthy controls were recruited from a variety of sources, including bulletin board postings, newspaper advertisements, and word of mouth. They were interviewed with the selected sections of the Structured Clinical Interview for DSM-IV Axis I Disorders, Clinical Version, for healthy people and were required not to have an Axis I disorder according to DSM-IV criteria, nor any relevant neurologic illness such as a history of brain trauma.

Patients were recruited from the inpatient and outpatient

Test–retest reliability, sensitivity and practice effects of tests without alternate forms

Table 2 lists the means and standard deviations for all of the measures from the BACS that do not have alternate forms. These measures were repeated with the same form regardless of which version of the BACS was administered. All tests demonstrated significant differences (P<0.001) between controls and patients. The intra-class correlations (ICC) between performance from test session 1 and test session 2 for each measure are also included. Each test had one measure that produced an ICC of 0.79

Discussion

The Brief Assessment of Cognition in Schizophrenia, or BACS, an easily administered pen- and -paper battery of neurocognitive tests, demonstrated high reliability and concurrent validity with a standard battery of tests in schizophrenia patients and healthy controls with similar ages, racial backgrounds and parental education. The BACS was as sensitive to the cognitive deficits of schizophrenia as a standard battery of neuropsychological tests that took nearly four times as long to administer.

Acknowledgements

This work was supported by a grant from Eli Lilly. Mark Appelbaum provided statistical consultation. Neurocognitive data were collected, in part, by Adam Vaughn, Susan Shortel, Matthew Dukes, Trina Walker and Joseph Kang. Healthy controls at Mount Sinai were assessed by Adam Brickman and Julie Kim.

References (33)

  • J.M Gold et al.

    Repeatable battery for the assessment of neuropsychological status as a screening test in schizophrenia: I. Sensitivity, reliability, and validity

    Am. J. Psychiatry

    (1999)
  • M.F Green

    What are the functional consequences of neurocognitive deficits in schizophrenia?

    Am. J. Psychiatry

    (1996)
  • M.F Green

    Mediators between neurocognitive deficits and community outcome in schizophrenia

    J. Adv. Schizophr. Brain Res.

    (2002)
  • M.F Green et al.

    Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the “right stuff”

    Schizophr. Bull.

    (2000)
  • P.D Harvey et al.

    Cognitive impairment in schizophrenia and implications atypical neuroleptic treatment

    CNS Spectr.

    (1997)
  • P.D Harvey et al.

    Studies of cognitive change with treatment in schizophrenia

    Am. J. Psychiatry

    (2001)
  • Cited by (1142)

    View all citing articles on Scopus
    View full text