Design: Test development, psychometric estimates of reliability and validity.

Population, Sample, Setting, and Years: One hundred fifty-seven UAPs from 2002 to 2003.

Concept or Variables Studied Together or Intervention and Outcome Variable(s): Knowledge competency in identified areas with the potential to affect patient safety (math, patient data collection, medical terminology, and reporting abnormal data).

Methods: Test construction, test- retest reliability, inter-rater reliability, content validity, construct validity (predictive models of skill competence for knowledge), and item analysis.

Findings: Content Validity Index was 1.0 after five rounds with content experts. Inter-rater agreement was 72%, yet the level of agreement was significantly related to test scores (r= .458. p= .008). There was no significant difference in test one and test two scores. Test scores were significantly different in those that did and did not demonstrate vital sign competency (t=-2.411, df=30, p=. 022). Using a cut score of 17 or greater resulted in identification of 87% masters and 100% non-masters with a Phi Coefficient of .412 (p=. 020). These correct classifications were maintained in test-retest results.

Conclusions: Acceptable estimates of reliability and validity were achieved. A cut score of 17 will result in acceptable classification of masters and non-masters. Implications: Methods to test UAP essential knowledge and skills are fundamental to support practicing nurses and promote patient safety. This research has resulted in a criterion-referenced test that can be used to screen UAPs prior to a clinical interview.

Back to Posters I
Back to 15th International Nursing Research Congress
Sigma Theta Tau International
July 22-24, 2004