Evaluation of the Validity and Reliability of the Revised Universal Design in Healthcare Education Instrument

Saturday, 21 July 2018: 8:50 AM

Janet A. Levey, PhD, RN-BC, CNE
Nursing, Concordia University Wisconsin, Mequon, WI, USA
Esteban Montenegro-Montenegro, MEd
Institute of Measurement, Methodology, Analysis and Policy (IMMAP), Texas Tech University, Lubbock, TX, USA

Purpose: The specific aim of this presentation is to discuss the refinement of the Universal Design in Healthcare Education (UDinHE) instrument, face and content validity, and stable reliability indexes.

Healthcare educators teaching in nursing, medicine, dentistry, physical/occupational therapy, physician assistant, and other allied professional fields are preparing a diverse workforce in response to the growing healthcare needs of a heterogeneous community (American Hospital Association, 2015, 2016). Diversity in postsecondary colleges and universities extends beyond ethnicity and includes students with disabilities. Healthcare workers in concordance with patient populations are critical to improved healthcare access and patient outcomes, enhanced communication and understanding of the patient’s context, and increased adherence to treatment plans (American Hospital Association, 2015, 2016; La Veist & Pierre, 2014). Evidence is revealing underrepresented minority healthcare workers practice in underserved communities reducing gaps in access to care (Wayne, Kalishman, Jerabek, Timm, & Cosgrove, 2010). Although healthcare workers are becoming more diverse, a concerted effort must be put forth by postsecondary healthcare educators to address strategies for reducing barriers for underrepresented groups, like students with disabilities, and increase their graduation rate from professional healthcare programs.

To achieve the goal of reducing barriers for students in healthcare academia, universal design (UD) is an innovative teaching pedagogy which focuses on inclusive teaching approaches in the development of curricula and instructional delivery (Burgstahler, 2015; Levey, in-press, 2016, 2017). The UD framework proactively reduces barriers during creation and delivery of course content, materials, assessments, and learning environments for learners, including those with disabilities (Burgstahler, 2015; McGuire, 2011). The UD framework is applicable to all learning environments; including the classroom, online, hybrid, clinical, and simulation and skill labs, for little or no cost to the postsecondary institution.

The practice of UD is embedded in marketing, communication, business, engineering, and elementary through secondary education degree programs, but not in professional healthcare programs (e.g., nursing, medical, physical/occupational therapy, physician assistant). Reasons UD has not been adopted by postsecondary healthcare educators are perceptions of UD and what constitutes inclusive teaching practices (Dallas, Upton, & Sprong, 2014; Levey, in-press, 2014, 2016, 2017). To date, no valid instrument exists to measure healthcare educators' perceptions towards inclusive teaching methods based in UD. Assessing healthcare educators’ perceptions towards UD is the first step in designing and measuring the effectiveness of an educational intervention.

Methods: The initial pilot and study of the Universal Design in Healthcare Education (UDinHE) formerly known as the Inclusive Teaching Strategies in Nursing Education instrument showed promising results, as the exogenous model fit the sample data and most scales had acceptable reliability indexes. However, the endogenous model fell slightly below established parameters (Levey, 2017); an expected result the first time an instrument is tested (Brown, 2015). The revised instrument was broadened to include educators teaching in professional healthcare sciences.The UDinHE instrument is a three-structured model of Knowledge of UD, System Support for UD, and Perceptions of UD based on Rodgers’ Diffusion of Innovation Theory (2003) and demographic questionnaire. The UDinHE contains 33-items using a 5-pt Likert scale ranging from Strongly Agree to Strongly Disagree to assess healthcare educators’ perceptions towards UD. The preliminary evaluation of the instrument consisted of assessed face and content validity of the revised instrument. A convenience sample at the primary investigator’s university assessed the internal consistency and stable reliability over a 2-week period. Data collection occurred during the 2017-2018 academic year. Deans and Program Directors of professional healthcare at the researcher’s university forwarded the survey to faculty currently teaching in their programs (e.g., nursing, physical/occupational therapy, physician assistants, and pharmacy). A sample size of 34 paired surveys were needed to calculate internal consistency indexes (effect size of 0.5, alpha, 0.05, and Power of 0.80). Consent was recorded when participants linked to the survey and results reported as an aggregate.

Results: A panel of nine experts in UD working in postsecondary education reestablished the face and content validity of the UDinHE. The overall scale content validity index (S-CVI) was .98. An S-CVI index of .80 or greater is acceptable (Lynn, 1986; Waltz, Strickland, Lenz, 2010). Internal consistency analysis revealed all subscales had a reliability coefficient (Cronbach’s alpha) greater than 0.70, except for the social system at 0.62 for the 22 paired surveys. The low alpha on the social system subscale may be related to the timing of the study during a time of the school year when faculty are navigating a new school year and systems. As hypothesized, there was no significant differences between Time 1 (M = 2.56, SD = 0.51) and Time 2 (M = 2.57, SD = 0.52) (t = -0.26, p = 0.79, d= -0.06) or changes in mean scores indicating stability of the instrument over time. All correlations between subscales were statistically significant at p <.001 (two-tailed) and ranged from 0.53 to 0.83 meaning there is a strong relationship between the constructs.

Conclusion: The UDinHE instrument obtained face and content validity and demonstrated stable reliability. A confirmatory factor analysis is the next step in the project. The instrument provides one way to measure healthcare educators’ perceptions towards UD and practices reflective of inclusive teaching methods. The UDinHE has the potential of assessing a program’s accessibility for students in relationship to disability law requirements. Graduating a diverse student body that mirrors society is imperative in meeting the healthcare needs of vulnerable populations (American Hospital Association, 2017). For this reason, the effectiveness of an educational intervention focusing on the adoption of UD by healthcare educators warrants future study.