Measures to Monitor Program Fidelity
PLL’s treatment fidelity protocol is based upon triangulating multiple data sources. The use of triangulation provides investigators with assurance that clinicians are using the treatment model as written. According to Moncher and Prinz (1991), triangulation is rarely used but absolutely necessary if treatment fidelity is to be addressed.
To address the issue of treatment fidelity and triangulation, the following measures are used for both PLL Group Therapy and PLL Family Therapy components:
Video Supervision Measure (VSM) – (Therapist Model Adherence Measure)
The Video Supervision Measure (VSM) is a checklist that assesses both content and process of the delivery of the PLL model. The PLL supervisor reviews video recordings of both group and coaching sessions. These recordings are rated using the VSM and the resulting scores are used to track each clinician’s fidelity level.
|Video Supervision Measure (VSM) – (Therapist Model Adherence Measure)|
Varies depending on Group or Family Therapy (Coaching) session
Videos are submitted for both Groups and Family Therapy (Coaching) sessions twice per month until Advanced Model Adherence is achieved.
|Process for submitting videos for VSM analysis:||
For Beginner, Intermediate, and Advanced model adherence levels, the clinician records a Motivational Interview, all six PLL Groups, and all four core coaching phases for review.
|Procedure for Scoring:||
Each key theoretical construct is rated in areas of content and/or process. Content domains are rated with either a “Yes” or “No”. Process domains are rated on a Likert scale from 1 (Extremely Poor) to 5 (Extremely Well). Model protocol adherence is assessed based on overall scoring of the VSM.
The PLL Dashboard is a proprietary software application that calculates "real time" outcomes. The dashboard has the benefit of illustrating how each individual PLL therapist is doing in adhering to the model’s protocol for engaging the family and moving them through both group and family therapy.
Group and Family Therapy (Coaching) Protocol Checklists - (Therapist Model Adherence Measure)
Group Protocol Checklist: Each of the six groups has a checklist of key concepts specific to that group that were either covered or not covered by the therapist. Therapists use both a Yes/No scale (Content items) and a Likert scale (Process items) to measure how closely they followed the Leader's Guide.
Family Therapy (Coaching) Protocol Checklist: Two measures are used in this area. First, the Family Therapy Coaching Family Plan (PowerPoint) is a snapshot of the therapist’s adherence to the PLL Family Therapy Manual. Second, the Family Therapy Phase II Pre-Session Preparation Worksheet is used to fill out specific areas that assist in model adherence and in learning the key undercurrent construct of the PLL Model.