The Brief Assessment of Cognition in Schizophrenia: reliability, sensitivity, and comparison with a standard neurocognitive battery
Introduction
The cognitive deficits of schizophrenia are profound and devastating. Patients with chronic schizophrenia demonstrate impairments that range between one and a half to two standard deviations below healthy controls on several key dimensions of cognition, especially verbal memory, working memory, motor speed, attention, executive functions and verbal fluency Saykin et al., 1991, Keefe, 1995, Harvey and Keefe, 1997, Heinrichs and Zakzanis, 1998. These components of cognitive dysfunction in schizophrenia also have an impact on functional outcome Green, 1996, Green et al., 2000 as they are all correlated with poor functional abilities.
Since the second generation of antipsychotic medications became widely available, dozens of studies of the neurocognitive impact of these medications have been completed. While most of these studies have concluded that second generation antipsychotics improve neurocognitive function, the interpretation of these results has been challenged, in part, by the variable test batteries used in each study (Harvey and Keefe, 2001). There is no standard, easily administered test battery that specifically and efficiently assesses the important cognitive deficits in patients with schizophrenia. Instead, current test batteries differ widely in content, duration, procedures, and implementation. The limited generalizability of neurocognitive findings across studies has reduced the ability of clinicians and researchers to make clear conclusions about the relative impact of the various antipsychotic medications on cognition in schizophrenia. The absence of a standard measure has also challenged studies of adjunctive treatments for cognition in patients with schizophrenia Friedman et al., 2001, Friedman et al., 2002, Evins et al., 2000, Heresco-Levy et al., 1996.
Another drawback of current assessments of cognition in treatment studies of patients with schizophrenia is that many of the neurocognitive assessment batteries used are long and complex. Most neuropsychological assessment batteries used in schizophrenia studies have been adapted from clinical neuropsychology, which assesses the entire profile of neuropsychological strengths and weaknesses in individuals. These batteries of tests may require several hours to administer. Their adaptation for schizophrenia research has kept some of the length that was originally necessary for individual assessment, but may no longer be necessary for research such as clinical trials that compares groups of patients. In fact, the length of some assessment batteries may be a limiting factor in assessing patients repeatedly throughout a clinical trial (Harvey and Keefe, 2001).
In contrast to these standard assessment techniques in schizophrenia research and clinical practice, cognitive function in patients with dementia is usually assessed with a widely used cognitive assessment tool, such as the Mini Mental Status Examination (Folstein et al., 1975), Dementia Rating Scale (Gardner et al., 1981), or Alzheimer's Disease Assessment Scale (Rosen et al., 1984). These instruments can be easily administered in typical treatment settings such as a physician's office, and are routinely employed as indicators of treatment change in clinical trials of anti-dementia drugs. The administration of These tests shed light on the global severity of patients' cognitive deficits, track progression, and measure cognitive changes.
The availability of a quick and efficient tool for measuring cognition in patients with schizophrenia could be an extremely useful guide for clinicians making decisions about potential rehabilitation and for researchers implementing clinical trials to assess cognitive improvement. There are several batteries of tests that are available or in development for the purposes of brief cognitive assessment.
Several computerized batteries, such as the Cambridge Neuropsychological Test Automated Battery (CANTAB) (Robbins et al., 1996), the CDR Cognitive Assessment System (Hunter et al., 1997), and the CogTest Battery (Cogtest, 2002) have been applied to schizophrenia samples, and have the option of reduced length. However, portability, regular software and hardware version changes, and increased patient and tester burden present implementation challenges. Other batteries, such as the Cognitive Screening Instrument for Schizophrenia (CSIS), a very short (10 min) set of abbreviated tests (Scot Purdon, personal communication), are currently in development.
The work completed with the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS) Randolph, 1998, Gold et al., 1999, Hobart et al., 1999 has clearly demonstrated the utility of brief assessment approaches. The RBANS is capable of providing reliable and valid assessments of patients with schizophrenia in various cognitive domains Gold et al., 1999, Hobart et al., 1999, Wilk et al., 2002. As noted in the RBANS manual, however, the test was originally developed as a screening measure primarily for elderly subjects, and the test is heavily weighted by memory, language, and visual–perceptual subtests. In addition, the item difficulties are most relevant to the types of impairment likely to be observed in patients with dementing illnesses such as Alzheimer's Disease, with ceiling effects in some domains. Although the RBANS is clearly sensitive to some of the impairments observed in schizophrenia, and it has utility as a screening tool for patients with schizophrenia, it lacks measures of motor, executive, and working memory performance that may be particularly important targets for cognitive enhancement in schizophrenia. These limitations suggest the need for a measure specifically designed for use in schizophrenia clinical trials that preserves the desirable features of the RBANS: brief administration and scoring time, portablity, repeatability, and availability of alternate forms.
The Brief Assessment of Cognition in Schizophrenia (BACS) has been developed for clinical trials with these key features. In addition, the domains of cognitive function that are assessed by the BACS are those found to be consistently impaired, and consistently related to outcome, in schizophrenia: verbal memory, working memory, motor speed, attention, executive functions and verbal fluency. The BACS is fully portable, and is designed to be easily administered by a variety of testers, including nurse clinicians, psychiatrists, neurologists, social workers, and other mental health workers. It is designed to require about 30 min of testing time with minimal extra time for scoring and minimal training demands.
The development of a psychological assessment instrument should determine the instrument's test–retest reliability, sensitivity, criterion-referenced validity (i.e. comparison to a standard measure), and comparability of alternative forms. This study aims to determine the quality of the BACS in these domains of reliability and validity.
Section snippets
Subjects
Healthy controls were recruited from a variety of sources, including bulletin board postings, newspaper advertisements, and word of mouth. They were interviewed with the selected sections of the Structured Clinical Interview for DSM-IV Axis I Disorders, Clinical Version, for healthy people and were required not to have an Axis I disorder according to DSM-IV criteria, nor any relevant neurologic illness such as a history of brain trauma.
Patients were recruited from the inpatient and outpatient
Test–retest reliability, sensitivity and practice effects of tests without alternate forms
Table 2 lists the means and standard deviations for all of the measures from the BACS that do not have alternate forms. These measures were repeated with the same form regardless of which version of the BACS was administered. All tests demonstrated significant differences (P<0.001) between controls and patients. The intra-class correlations (ICC) between performance from test session 1 and test session 2 for each measure are also included. Each test had one measure that produced an ICC of 0.79
Discussion
The Brief Assessment of Cognition in Schizophrenia, or BACS, an easily administered pen- and -paper battery of neurocognitive tests, demonstrated high reliability and concurrent validity with a standard battery of tests in schizophrenia patients and healthy controls with similar ages, racial backgrounds and parental education. The BACS was as sensitive to the cognitive deficits of schizophrenia as a standard battery of neuropsychological tests that took nearly four times as long to administer.
Acknowledgements
This work was supported by a grant from Eli Lilly. Mark Appelbaum provided statistical consultation. Neurocognitive data were collected, in part, by Adam Vaughn, Susan Shortel, Matthew Dukes, Trina Walker and Joseph Kang. Healthy controls at Mount Sinai were assessed by Adam Brickman and Julie Kim.
References (33)
- et al.
“Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician
J. Psychiatr. Res.
(1975) - et al.
Guanfacine treatment of cognitive impairment in schizophrenia
Neuropsychopharmacology
(2001) - et al.
A double-blind placebo controlled trial of donepezil adjunctive treatment to risperidone for the cognitive impairment of schizophrenia
Biol. Psychiatry
(2002) - et al.
Test–retest reliability of a new form of the Auditory Verbal Learning Test (AVLT)
Arch. Clin. Neuropsychol.
(1994) - et al.
Performance of patients with schizophrenia on a pen and paper visuospatial working memory task with short delay
Schizophr. Res.
(1997) - Cogtest plc. Cogtest(tm): Computerised Cognitive Battery for Clinical Trials. (2002). Retrieved November 8, 2002, from...
- et al.
Impaired attention, genetics, and the pathophysiology of schizophrenia
Schizophr. Bull.
(1994) - et al.
Placebo-controlled trial of glycine added to clozapine in schizophrenia
Am. J. Psychiatry
(2000) - et al.
Mattis Dementia Rating Scale: internal reliability study using a diffusely impaired population
J. Clin. Neuropsychol.
(1981) - et al.
Auditory working memory and Wisconsin Card Sorting Test performance in schizophrenia
Arch. Gen. Psychiatry
(1997)
Repeatable battery for the assessment of neuropsychological status as a screening test in schizophrenia: I. Sensitivity, reliability, and validity
Am. J. Psychiatry
What are the functional consequences of neurocognitive deficits in schizophrenia?
Am. J. Psychiatry
Mediators between neurocognitive deficits and community outcome in schizophrenia
J. Adv. Schizophr. Brain Res.
Neurocognitive deficits and functional outcome in schizophrenia: are we measuring the “right stuff”
Schizophr. Bull.
Cognitive impairment in schizophrenia and implications atypical neuroleptic treatment
CNS Spectr.
Studies of cognitive change with treatment in schizophrenia
Am. J. Psychiatry
Cited by (1142)
Montreal Cognitive Assessment (MoCA) as a screening tool for cognitive impairment in early stages of psychosis
2024, Schizophrenia Research: CognitionPersonal recovery in first-episode psychosis: Beyond clinical and functional recovery
2024, Schizophrenia ResearchRevisiting delusion subtypes in schizophrenia based on their underlying structures
2024, Journal of Psychiatric Research