Feasibility randomised controlled trial comparing TRAK-ACL digital rehabilitation intervention plus treatment as usual versus treatment as usual for patients following anterior cruciate ligament reconstruction

Design—a randomised controlled feasibility trial

This study was a parallel arm, individually randomised, feasibility RCT comparing postoperative ACL rehabilitation TAU with TAU plus TRAK-ACL. It is reported in line with the Consolidated Standards of Reporting Trials (CONSORT) reporting standards extension for pilot and feasibility trials.17 The trial was supported by the PRIMENT clinical trials unit, a registered UK Clinical Trials Collaboration and a trial steering committee was established to oversee the trial processes.

Patient and public involvement (PPI)

This study won an award for patient involvement. TRAK-ACL was developed with a PPI group in a London NHS hospital.13 18 The group participated in choosing and developing content for the website, ensuring the website met an acceptable standard of diversity and inclusion and the design of the feasibility study. One member of the PPI group sat on the trial steering committee. The opinions of NHS Physiotherapists contributed to the development of the TRAK-ACL website and design of the feasibility study which was informed by the findings of two previous usability and acceptability studies carried out by this team of researchers.13 14

Recruitment

The study took place at three NHS sites; a large University Hospital Foundation Trust, in an ethnically and socio-economically diverse English metropolitan area (site 1), a large South of England Trust covering a mixed urban and rural population (site 2) and large University Health Board in Wales, also mixed urban and rural population (site 3). Recruitment opened in July 2018 at sites 1 and 2. Site 2 was unable to recruit patients and was closed. Site 3 was added in November 2018 and recruitment closed in March 2019. Sites 1 and 3 are reported hereafter.

Sample size

Feasibility study sample sizes are usually 50–70 participants.19–21 However, we looked to the ACL rehabilitation clinical trial literature for estimates of retention and compliance in similar studies.22 The literature suggested that we could expect between 20% and 25% loss to follow-up.23 24 We therefore estimated 75% retention. We estimated that 25 in each arm would give a 95% CI of 0.60 to 0.85, suggesting that we could be 95% certain that at least 60% of the target population would remain in the trial for at least 6 months.

Randomisation

The trial statistician used computer-generated random number sequences to draw up a spread sheet where trial participant numbers could be added sequentially. Participants were allocated to the intervention or control arm by a member of the research team. Randomisation was performed after informed consent and baseline data were collected by the blinded assessor.

Intervention

Participants randomised to the intervention arm received treatment as usual (TAU) plus TRAK-ACL website. The intervention was delivered by qualified chartered physiotherapists working in the musculoskeletal out-patient setting.

TRAK-ACL website

TRAK-ACL is a DHI whose content was drawn from the literature on ACL rehabilitation. It was designed to support patients after ACL reconstruction by reinforcing teaching and exercise prescriptions given by a physiotherapist in face-to-face care. It includes an extensive exercise library and information from the ACL literature and physiotherapists and orthopaedic surgeons experienced in managing patients with ACL reconstruction. The exercises and information were provided phase by phase (early, middle, advanced and return to sport) as videos, animations, infographs and in written text format. Interactive features such as personal goal setting, progress logs and dashboards of progress were informed by the Behaviour Change Wheel framework for intervention design and are known to promote engagement with rehabilitation behaviours.12 The TRAK website can be accessed at: https://spas.cs.cf.ac.uk/trakacl/.

A training package was provided for physiotherapists using TRAK-ACL, including how to induct patients to TRAK-ACL. The TRAK-ACL intervention and the training programme are described in online supplemental file 1 and reported according to the Template for Intervention Description and Replication (TIDieR) guidelines.25 Each patient was given a TRAK-ACL alphanumeric login and they were invited to seek extra support with using TRAK-ACL if needed. Tablet computers were provided to participating sites and Wi-Fi availability was ensured before the study began.

The treating physiotherapist inducted the patient on how to use the TRAK-ACL website and give the participant a login during their first face-to-face consultation. The patient was shown a playlist of exercises which were individualised to their needs by the physiotherapist. Goals were set in discussion with the patient. Both goals and exercise playlists were progressed according to the patient reaching the rehabilitation milestones. All the rehabilitation exercises were on the TRAK-ACL website.

Control group

Physiotherapy TAU varied across the included sites and is described in detail in online supplemental file 1. Common features included face-to-face time with physiotherapists, monitoring of the recovery from surgery and achievement of early rehabilitation goals. The duration of care was not restricted at either site and was expected to last for 6–12 months according to patient needs. A progression through phases of care was evident. The main difference between sites was the mode of delivery: at sites 1 and 2 care was delivered in an ACL group and at site 3 care was delivered though individual one-to-one appointments, with the option of attending a generalised lower limb class. The timing of appointments was different at each site. At sites 1 and 2 appointments were on a weekly basis dropping to fortnightly. At site 3 the scheduling was based on patient need and service capacity.

Outcome measures

The primary outcomes were feasibility outcomes (recruitment and retention) plus measures to inform the parameters of a future trial.26 Study outcomes were taken at baseline, 3 and 6 months.

Data collection

Outcomes were collected at physiotherapy appointments at baseline, 3 and 6 months, except quadriceps strength, which was collected at site 1 at 3 and 6 months only. Outcome assessors and the lead researcher were blinded to treatment allocation but neither patients nor treating physiotherapists could be blinded as they needed to use the DHI.

Data analysis

All primary outcome measures were summarised separately by study arm. Differences in secondary outcomes between arms were estimated using linear and logistic regression. Quantitative data were analysed as follows: binary and other categorical measures such as recruitment, retention and adverse events were summarised using frequencies and percentages. Continuous measures were summarised using means and SDs (or medians and IQRs for skewed distributions). The precision of estimates was assessed using 95% CIs. Power analyses were conducted to calculate the sample size necessary to detect an effect of the intervention in a future RCT with the KOOS as the primary outcome. Intention to treat principles were applied to data for all recruited patients.31 Progression of a full-scale trial would be considered if all of the feasibility criteria based on the primary outcomes were met.

Recruitment

Flow of participants through the trial is illustrated using a CONSORT flow diagram shown in figure 1. Fifty-nine people were assessed for eligibility across two sites, of whom eight people declined to participate and 51 were randomised. Of these, 26 were allocated to TRAK-ACL and 25 to TAU.

Characteristics at baseline

There were 51 study participants overall. The TAU arm and the TRAK-ACL arm were well-matched for baseline characteristics, presented in table 1.

Retention: numbers analysed

Retention was measured by the proportion of participants providing outcome data at 3 and 6 months. In total, 44 patients were retained to the study at 3 months and 40 at 6 months. Over the course of the study, three people were lost to follow-up in the TRAK-ACL arm and eight were lost to follow-up in TAU (table 3).

Completeness of outcome data

The completeness of outcome data suggested that the number, complexity and time taken to complete items were all acceptable to participants, indicating that data collection for a fully powered RCT will be feasible. All but one outcome measure had 100% item completion for each outcome (table 4). The CSRI was the exception in that it was only completed by 54% of participants at baseline, and then 69% and 89% at 3 and 6 months.

TRAK-ACL usage data

Usage of the TRAK-ACL intervention was measured by the number of logins to the website, videos watched and behaviour change functions used, for example, exercise log, goal setting or weekly progress. These findings are displayed in full detail in online supplemental information 2—TRAK-ACL usage data. In summary, the median number of logins per patient participant was 5 with IQR of 3–13. The median (IQR) patient logins per week was 4 (2–7). The time between patients’ first and last login was a mean of 18 weeks (12.2SD), which suggests that users continue to engage over time and across phases of care. The median (range) physiotherapist logins per week was 2 (0–5). The median (range) logins per physiotherapist over the duration of the trial was 11.5 (6–18.5). The logins of physiotherapists and patients over 60 weeks, given in figure 2, showed consistency of patient usage as physiotherapist usage dwindles and disappears over the last 20 weeks.

There were no reported adverse events for participants enrolled on this feasibility trial.

Analysis of secondary outcomes

The analysis of secondary outcomes is discussed in online supplemental information 3. The results show that the primary outcome of a future substantive RCT would be the KOOS at 6 months follow-up. A power calculation assuming a SD of 17.2671 on the KOOS estimates that a sample size of 172 participants (86 per study arm) will be required to detect a nine point difference on the KOOS between intervention and control groups with 90% power and 5% alpha. After inflation for 20% loss to follow-up, this figure increases to 108 participants per arm, 216 in total.

Principal findings

The aim of the study was to determine the feasibility of an RCT comparing TRAK-ACL plus TAU to TAU in the management of postoperative patients with ACL. The findings suggest a future RCT to determine effectiveness, as measured by the KOOS, would be feasible, as patients were successfully recruited and retained and provided adequate outcome data, with the exception of the CSRI. Further work may be needed to improve data collection for an assessment of cost-effectiveness. The findings on usage suggest that patients and physiotherapists engaged with the TRAK-ACL intervention, in terms of number of log-ins, uptake of material and duration of engagement.

Recruitment and retention

Eighty-six per cent of those approached about the study were recruited. The recruitment target of four patients per month was exceeded at site 1 where there was a dedicated researcher to facilitate recruitment. Site 2 recruitment was affected by changes to orthopaedic team leading to a significant drop in patients with ACL reconstruction. This could have been anticipated by better engagement with orthopaedics through the planning stage. It is known that recruitment challenges can be a key reason why randomised trials fail32 so a robust recruitment outcome was important.33 Recruitment method was highlighted as a key barrier to participation in previous DHI studies where in one example less than half of the physiotherapists allocated to delivering DHI actually recruited patients34 and in another only 30% of eligible patients were recruited.35 Strategies such as personalising the intervention (personal exercise plans) and dedicated staff support were implemented in the current study which may have contributed to recruitment success.36

At 6-month follow-up 78% of participants were retained, which exceed the criteria set for this study of less than 30% drop-outs. Eysenbach et al described two types of attrition; non-usage attrition (non-use of the DHI) and drop out attrition (lost to follow-up in the trial).37 Some patient loss can be explained by typical drop out from physiotherapy over the duration of ACL rehabilitation,38 and dropout rates can exceed 30% in trials for DHI and exercise.14 39 This study suggests that patients with ACL reconstruction will engage with the DHI and trial process over time and that the role of the research assistant seems key to facilitating outcome collection.

Usage of TRAK-ACL

Usage data from TRAK-ACL show that there was consistent engagement by some users over time, indicating improved access to care. For the current study, a progression criterion of three logins per week per patient was set based on current evidence.4 Use of the behaviour change mechanisms such as logs, goal setting and educational and motivational content was accessed by up to 50% of patient TRAK-ACL users which may have improved these patients adherence to the target behaviour.

The usage data shown in this trial are similar to reported use data for health websites and apps used alongside treatment as usual.35 40–42 Additionally, levels of usage found in this study are similar to those found for other physiotherapy digital intervention feasibility studies14 43 and may indicate a growing acceptability of DHIs. A future trial would benefit from addressing physiotherapist engagement which may be affected by factors such as time and willingness to adopt new technology, understanding of new technology, the extent to which new technology meets user needs and the workplace support to maximise the potential of new technology.34 44

Outcomes

The gathering of clinical outcomes such as KOOS, the self-efficacy score, EQ-5D-5L and work productivity and impairment was also shown to be feasible in this trial. There was 100% completion by all patients who were retained to the trial at 6 months in four out of five outcomes, which exceeded the feasibility progression criterion of 80% completion set for this study. The exception to good outcome collection was the CSRI. It is considered a valuable tool of health economic analysis and should be included where possible.45 46 In a future RCT, a digital rather than paper version could facilitate cleaner more usable data. Likewise, strength testing would be a key outcome of a future trial.3 47 The method of collecting strength testing across sites was not standardised and in a future trial isokinetic muscle strength testing of quadriceps and hamstrings at 180°/s should be considered. Feasibility of strength testing was successful however, clinical trial standard strength testing requires standardised conditions and calibration of machines.48

Strengths and limitations

The strength of this study was the success of recruitment pathways, outcome measures and intervention toward determining feasibility. Progression to a full RCT would be recommended as the feasibility progression criteria for recruitment, retention, completeness of outcome measures at 6 months and adverse outcomes have all been met. The total sample size calculated for a full RCT was 216 participants (allowing for drop-outs). This would be achievable based on the recruitment to this study but would require 12 months’ recruitment over a minimum of five sites.

Not all patient participants engaged with all aspects of the behaviour change mechanisms built into TRAK-ACL and physiotherapists did not appear to sustain engagement with TRAK-ACL through to the return to sport phase of rehabilitation. Exploring this in a qualitative study would have strengthened this study and design of a future RCT. Measuring the number of face-to-face physiotherapist contacts would help evaluate if TRAK-ACL resulted in reduced physiotherapist contacts. The study was limited by the specificity of usage measurement that TRAK-ACL was capable of when considered against the AMUsED framework, a standard for reporting usage in digital trials.49 In a future trial, TRAK-ACL would need more specific usage measurement capabilities.

New findings

Patient engagement with educational resources and exercises on the DHI occurs across multiple phases of the rehabilitation.

Digital health tools such as the TRAK-ACL website may provide an opportunity to teach and reinforce the key lessons and exercises at each stage of rehabilitation, as well as engage patients through behaviour change functions.

DHI may be crucial for patients who do not have access to physiotherapy throughout rehabilitation.