Psychometric Testing of a New Instrument to Measure Writing Self-Efficacy: A Proposed Mixed-Methods Delphi Study

Sunday, 22 July 2018

Kim M. Mitchell, MN1
Diana E. McMillan, PhD, RN2
Michelle M. Lobchuk, PhD, RN2
Nathan C. Nickel, PhD3
(1)Department of Nursing, Red River College, Winnipeg, MB, Canada
(2)College of Nursing, University of Manitoba, Rady Faculty of Health Sciences, Winnipeg, MB, Canada
(3)Community Health Sciences, University of Manitoba, Winnipeg, MB, Canada

Purpose: The purpose of this proposed study will be to develop and test an instrument to measure writing self-efficacy based on a model of constructivist writing. It asks the question: What measurement items will form the most reliable and valid measure of writing self-efficacy?

Background

Writing is a pedagogical strategy that facilitates critical thinking, identity development, and ways of knowing. The emotional and metacognitive demands of writing often impact personal agency, creating barriers or facilitators to student writing development.1 Qualitative research2 has established that writing assignments are anxiety provoking, fraught with frustrations, and complicated by contextual challenges such as inconsistencies of instructor expectations and grading. Some of the emotional upheaval associated with writing stems from the reality that writing is more than a means to an end or a grade, it is also a process of meaning making and self-reflection.1 Only one strategy will effectively improve student writing: more writing. Writing self-efficacy, defined as “everything inherent in a writers’ belief about their ability to write within a specific context,”3 is required to motivate students to seek help with their writing when needed. Developing a sense of writing self-efficacy in students is what keeps students motivated to improve their writing craft.4 The ways of knowing writing assignments provide, translate into increased practitioner depth of thinking – a necessary requirement in a health care environment driven by evidence-informed practice.5

Writing Self-Efficacy and Nursing

A small body of research is emerging in the literature exploring writing self-efficacy and its impact on nursing education.4,6,7 These studies have demonstrated that using writing pedagogies, which target building self-efficacy primarily through scaffolded support, can improve student sense of self-efficacy from pre-to-post writing course. An across the curriculum exploration identified that writing self-efficacy fluctuates based on context where low self-efficacy was found to be an identifying factor in students who were at risk for program delays or attrition.8

Writing Epistemology and the Measurement of Self-Efficacy

Writing research has progressed through three phases of epistemological thought: 1) product views, which explore surface elements of writing; 2) cognitive process views, which focus on the recursive elements of planning, composing, and revising; and 3) social constructionist views which explore writing as a way of knowing and meaning.9 A synthesis of published writing self-efficacy instruments3 has identified that most writing self-efficacy instruments explore the concept using surface markers visible in a product (grammar, sentence structure, organization), or by process tasks of writing which need to be mastered (outlining, researching, avoiding distractions), to the neglect of the creative, emotional, relational, contextual, and identity forming constructivist elements of writing which can drive a writer’s passion, or paralyze them from progress.

Theoretical Framework

The questionnaire will be developed considering the constructivist elements of writing proposed by Mitchell.9 This model proposes that five dimensions exist in constructivist perspectives of writing:

Identity: writing voice, nursing identity, past experiences with writing, and reflexivity;

Creativity: including idea development, synthesis, risk taking, and originality;

Emotions: can drive or inhibit writing and are present during planning, writing, and feedback stages;

Relational Aspects: including relationships built with instructors and interpretive relationships formed with authors while reading sources in preparation for writing;

Context: perceived difficulty of evaluators, the stakes for the writing task, disciplinary conventions and values, and familiarity with the content knowledge and type of document assigned.

Method

The method proposed for this instrument development study is a modified Delphi method.10 Modified Delphi studies incorporate both qualitative and quantitative approaches in developing, evaluating, and testing the items on a new instrument in order to assess its content and construct validity.11 Four phases will be implemented in order to achieve this goal.

Phase I – Item development. Items will be created based on an extensive review of the literature and personal experience with writing and writing instruction. Between 25 and 50 items will be developed corresponding to the five domains of the constructivist model of writing. All items will be positively worded to avoid cognitive interruption in the participant, improve internal consistency, and avoid factor structures which cluster all the negative items together.12 In Phases II and III, the scale will be presented, in the form of an electronic sliding scale, to an expert panel and to undergraduate students, anchored with disagree to agree, with possible scores for each item ranging from 0-100 to increase response variability.13

Phase II – Delphi panel. The developed items will be presented via an online survey to a purposive panel of approximately 10 international nurse educators and writing experts along with a description of the theoretical and practical intent of the instrument. Experts will be approached through personal connection or social media. The experts will be asked to rate each item’s applicability to the proposed scale on a 0-100 scale. Respondents will also be asked to comment qualitatively on each item in terms of edits to wording, the item’s interpretability, and to provide rationale if they believe an item should be removed. Opportunity will be given to the experts to suggest additional items. Following expert feedback, the scale items will be revised. Items will be removed if the average applicability score does not fall in the agreement range or has a standard deviation range which falls below neutral.14 The revised scale will be re-presented to the panel and this process will be repeated until consensus is obtained. A minimum of two Delphi rounds are expected.

Phase III – Think-aloud interviews. The revised questionnaire will be presented electronically, while face-to-face with the researcher, to 8-12 undergraduate nursing students, with varying writing backgrounds, educational experiences, and cultural backgrounds, from a local university. When research participants engage with the items on questionnaires, they often recall memories or visualize past experiences which contribute to their decision-making while scoring an item.15 Think-aloud interviews will be used and will be tape recorded and transcribed. Enrolled participants will be asked to read each item, score themselves and then explain verbally why they decided upon the score they gave. Participants will also be encouraged to relate examples and past experiences with writing and/or being graded which might have contributed to the score they may have chosen. These interviews will be used to establish that the items are being interpreted in the intended manner. The scale will be revised according to findings. Items which trend toward causing a confused or indifferent response will be revised or removed from the scale. A secondary analysis of these interviews will also be conducted using content analysis to identify participants’ common themes of writing self-efficacy experiences.

Phase IV – Psychometric testing. A sample of 10-20 nursing students for every finalized item on the instrument will be recruited from three local nursing programs. The developed scale will be presented concurrently with an established writing self-efficacy instrument, the Post-Secondary Writerly Self-Efficacy Scale.16 Demographic information will also be collected from students such as institution, year in program, age, and gender. Cronbach’s alpha will be calculated for reliability. Factor analysis will be the primary method of construct validity. Exploratory factor analysis will be used to identify the scale’s latent variables.17 Confirmatory factor analysis will explore the fit of the instrument to the theoretical model.18

Significance

  • A writing self-efficacy instrument based on constructivist principles is more in line with nursing’s values of holism and relationality.
  • A new writing self-efficacy instrument will be beneficial for researchers to contribute to the growing need for writing research in the nursing discipline.
  • A multidimensional scale will assist educators in identifying areas for targeted instructional interventions across nursing curriculums.
  • Writing pedagogies need to reflect a broader understanding that writing is more than an graded outcome. It is a challenging and emotional endeavour that preys on a writer’s self-efficacy.
  • Understanding writing self-efficacy will assist with understanding factors that contribute to student attrition and delayed graduation from nursing programs.