The Educational Impact of the Evaluation Tool for Child Life Interns: A Mixed Methods Study

Abstract

Background: The Association for Child Life Professionals (ACLP) requires those seeking to become a Certified Child Life Specialist (CCLS) complete a minimum of 480 hours of supervised clinical experience prior to the certification exam. Trainees’ skills are evaluated using the Evaluation Tool for Child Life Interns. This tool consists of 54 skill-based items scored with a 7-point Likert-based rating scale (with a Not Applicable (NA) option) and space for open-ended narrative feedback. Skill based items are within three separate domains: Assessment, Intervention and Professional Responsibility. There are no published research studies available to date on the use or effectiveness of the tool. Purpose: This study seeks to explore the educational impact of the tool using two sources of data: frequency of NA ratings across the three domains and narrative feedback from preceptors in comment sections of the tool. Methods: Using an explanatory sequential mixed methods design, quantitative data (proportion of NA ratings) and qualitative data (preceptors’ comments) from evaluation tools obtained from 45 learners spanning the academic years 2011 – 2016 at McMaster University’s Post Graduate Diploma Program in Child Life. Mean proportion of NAs was evaluated using a 2 (time: internship 1 and internship 2) x 3 (domain: assessment, intervention and professional responsibility) two-way repeated measures analysis of variance ANOVA. Qualitative data was explored using thematic analysis of preceptors’ narrative comments for skill-based items where at least 1 NA rating was selected. Results: Analyses of quantitative data revealed NA ratings to be substantially greater in the professional responsibility domain (22%) relative to both assessment (<1%) and intervention (3%) domains (p-value < 0.001). There was no main effect for time. Qualitative analyses showed that the use of NA was primarily due to the intern not having the opportunity to experience or demonstrate the skill being rated. Four themes emerged from the thematic analysis of narrative feedback (constructive feedback, action plans, clinical examples with patients and families, little to no opportunity for skill development). Conclusion: The Evaluation Tool for Child Life Interns offers quantitative and qualitative feedback to child life learners with benefits and challenges to its use. The subtasks in the assessment and intervention domains represent child life practice across various placement sites. Within the professional responsibility domain, skills being assessed require further review through stakeholder input to ensure they reflect current practice. The narrative feedback provided by preceptors is detailed and constructive. The tool is discouraged for use as a single summative assessment to make decisions of competency and is recommended for formative and summative use as one method of feedback within a program of assessment.