Objective: Meta-analysis is a way of categorizing and quantitatively summarizing multiple research studies of a common phenomenon. By statistically combining the findings from many studies, the magnitude of an intervention's effect on an outcome across studies can be estimated. However, there are several factors which affect the validity of meta-analytic findings. One factor, the quality of the studies included in the meta-analytic review, is the focus of this presentation. Quality is defined as an evaluation of the design of a study, assessing the internal and external validity. No research design is free of weaknesses, although some researchers are more effective in controlling for threats to internal and external validity than others. For over 20 years, many studies have been conducted to evaluate the effectiveness of interventions for caregivers of persons with dementia. The findings from these studies have demonstrated no, low, moderate, and high effectiveness of the interventions on study outcomes. It has been proposed that the results may vary due to methodological limitations. The objective of this study is to evaluate the quality of studies testing interventions to lessen the negative impact or enhance the positive aspects of caring for a family member with dementia.
Design: The design of this study was meta-analytic, however the aspect of the study reported here is quality analysis.
Population: The population for the study was all research, published and unpublished, testing interventions for persons caring for a family member with dementia. The sample consisted of 138 studies conducted between 1980 - 2001.
Concept: The concept under study was quality, a rating of the internal and external validity of studies included in a meta-analytic review.
Methods: A quality rating scale was developed for the study from three published essays on quality indicators (design aspects, sampling methods, intervention specificity, and measurement congruence). The quality instrument was evaluated for validity by two meta-analytic experts. Each quality indicator was operationally defined and researchers were given extensive training on quality ratings and on detecting quality indicators. Each study was rated for quality on study design, sample selection methods, eligibility criteria, intervention description, outcome description, outcome measures, and outcome measure frequency. Each study was coded for quality independently by two researchers to establish interrater reliability which varied between .92 and .98. Intrarater reliability was also assessed and varied between .96 and 1.00.
Findings: The range of possible quality scores was 0 - 21. Scores for the studies ranged from 6 - 21, with a mean score of 14.5.
Conclusions: Quality ratings for the family caregiver intervention studies were moderate. Problematic areas included weak designs, lack of randomization, poor intervention specificity, missing point estimates (means) and variability (standard deviations).
Implications: Synthesis summaries can illuminate research directions. Researchers conducting intervention studies with family caregivers should address the quality aspects of research to increase the validity of and confidence in the study outcomes. Such attention could elevate the state of the science and thus, clarify the effectiveness of said interventions. Yet, some scholars argue that methods to increase quality such as experimental design (randomization) and intervention specificity are not applicable to this population because of the problems such as denying or delaying an intervention for a needy caregiver (randomization into the control group). Additionally, some researchers state that interventions cannot be strictly specified due to individual needs of family caregivers. These dilemmas are not easily solved. This meta-synthesis clarifies the quality of caregiver intervention studies and suggests directions for future research.
Back to Posters
Back to The Advancing Nursing Practice Excellence: State of the Science