The Institute of Medicine, 2010 report recommends that nurses continue their education with ongoing competency assessment and validation in order to ensure safe quality care in a rapidly changing and diverse healthcare environment. Historically nursing competency has been measured by annual skills fair focused on evaluating clinical skills, not abilities or understanding of the science behind the skills. Competency is defined as not only capabilities, but also the achievement of desired outcomes, with measurements reflecting nursing abilities beyond technical skills. Simulation is embraced as a component of continuing education and an effective means of systematically validating competencies in a controlled environment.
State of the Science:
Human Patient Simulation provides the opportunity for participants to react to high risk situations without any risk to patient safety. The use of a validated rubric with established inter-rater reliability and simulation is increasing being used by healthcare organizations for validating nurses competency in the collection and interpretation of data, clinical judgement, clinical reasoning, and communication which are essential to patient safety.
Statement of Purpose:
Conduct competency assessment within a high fidelity human simulated environment for the nurses at a 25 bed critical access medical center, using a validated and reliable measurement tool. Informatics Solution: Human Patient Simulation is widely used in nursing academia, and provides an effective means of systematically validating competencies in a controlled environment.
Methods and Procedure:
Step 1: A needs assessment tool will be used to identify competency needs.
Step 2: Development of simulation scenario(s) by educators with expertize in simulation scenario development and adult medical surgical nursing. The scenarios will include identified skills while allowing for assessment of patient, interpreting of patient data and managing the patient. The scenario will include multiple mini-scenarios which will allow for the staff to progress in meeting the complexity of the competencies being measured. The simulation scenario content will be assessed for clarity and validity by a panel of experts who are knowledgeable in adult medical surgical nursing for more that two years.
Step 3:pilot testing of scenario will be performed with a medical surgical nurses who are not participating in the study, in order to ascertain construct validity. Efficiency measure will be established by determining the amount of time required to complete simulation scenario; which will be used to determine the time that will be required by each participant to complete the simulated activity.
Step 4: The Creighton Competency Evaluation Instrument (C-CEI®) will be used to assess participants’ response. C-CEI is a validated quantitative evaluation tool used to evaluate participants' performance in a clinical simulated environment; and focuses on 23 general nursing behaviors divided into four categories: assessment, communication, clinical judgement, and patient safety. A panel of nurse educators will use the C-CEI to determine what is expected as performance criteria for each of the 23 general behaviors based on the simulation scenario, using the C-CEI discussion worksheet provided by the creators of the tool. A consensus will be reached on the required passing score used to determine competency.
Step 5: The clinical educators who will serve as validators will receive online training for using the C-CEI tool from the Creighton College of Nursing website on how to use the tool. Inter-reliability will be achieved by having the validators watch two simulation exercise videos: one with no error in performance and the other with performance errors. Inter-rater reliability will be assessed using the Kappa statistic and percentage agreement among raters with an acceptable level of agreement set at 80%.
Step 6:Approximately one month prior to the competency assessment scheduled events, participants will be notified of the schedule (Jones, Carson, and Mancini, 2002). On the day of the event, participants will be given an orientation to the simulation room, the equipment, and the high fidelity patient simulator. Trained instructors will conduct the simulation by following strict guidelines for sequence of events and responses. Each participant will be given the same time frame to accomplish as many steps as possible. The participants will be informed that they will have the defined time and instructed to complete as much of the care of the patient as possible. The validator will independently score each participant. Debriefing will be conducted as an opportunity for participants to reflect on actions and take corrective on any key element missed during the simulation exercise.
Step 7: Statistical analysis will be conducted on the data to evaluate the effectiveness of using simulation to measure the nurses level of competency. The participants’ feedback of their satisfaction and perception of the method will also be evaluated using a survey questionnaire. This survey questionnaire is administered annually to assess staff’s satisfaction with the competency assessment process and methods. The survey questionnaire has no established validity and reliability scores, thus results from the survey questionnaire will not form a part of the publication, but as a pilot testing for the tool to establish content and validity.
SWOT Analysis of Use of Simulation Technology for Competency Assessment.
As simulation becomes a viable and desirable option in the evaluation of dimensions of competencies such as critical thinking, interpersonal skills and technical skills, the purpose of this paper is to critically analyze the strengths, weaknesses, opportunities and treats to using simulation technology for evaluation of clinical competency.
Strengths
- The literature is robust in reporting positive faculty and student perspectives related to implementation of simulation and its impact on outcomes in nursing education.
- There are published simulation scenario and evaluation tools developed and designed to require the nurse to demonstrate critical and reflective skills.
- Simulated experience for continued competency assessment which resembles an actual event that requires the practitioner to make critical decisions while demonstrating discipline specific competencies without jeopardizing patient safety.
- Another benefit of using simulation is that the exercise can be videotaped, allowing multiple evaluators the opportunity to analyze the practitioner’s proficiency
- Debriefing as part of simulation exercises may offer the opportunity for valuable reflective learning and clarification of content and concepts provided during the simulation scenario.
Weaknesses
- Lack of research to support strong simulation evaluation rubric measurements to ensure reliability and validity of simulation evaluation tools. Simulation evaluation tools have been studied in a controlled simulation environment, however no published study of a larger random control study to test for validity and inter-rater reliability.
- Simulation is also not standard across nursing programs and tools developed cannot capture completely the effectiveness of simulation when it is measuring differing variations of implementation of simulation.
- The cost and time commitments in the development of scoring methods, the selection and design of simulation experiences.
- Simulation does not completely capture reality; thus the need to validate if proficiencies demonstrated in the simulated environment are, in fact, present in the patient care setting.
Opportunity
- If simulation scenarios were standardized in nursing programs, it would be a platform for more standardized studies and measurement tools for evaluation
- The use of standardize simulation scenarios for evaluation research exploits the strength of having a consistent subject for observers to evaluate, which will provide control over source of variability between what is evaluated by each observer
- 3. Research is needed to identify characteristics of health professionals who are at higher risk for failing to demonstrate ongoing clinical competency
Threats to Implementation
- 1. Cost and time commitment of the endeavor of the development and testing of simulation scenarios; training and establishing inter-rater reliability of evaluation tool
- 2. Identifying and recruiting educators with experience in developing simulation scenarios as the design of the scenario will require participant to demonstrate critical and reflective thinking through evaluation, communication, interdisciplinary collaboration, and skill performance.
Ethical Consideration:
Participants will be informed of purpose of project.
- Fidelity maintained by using validators not assigned to facility.
- Remediation will be provided, and outcome not tied to work performance
Conclusion
Effective competency validation method requires a dynamic process dependent of the skills or behavior to be assessed, or the practice setting, and the expertise of the staff member. High fidelity human simulators and assessment rubrics have demonstrated efficacy in assessing nurses’ ability to conduct patient assessment, clinical judgment, clinical reasoning, communication and patient safety.