early 2011 – evaluating the effectiveness of a study in which patients participated remotely, without ever having to visit a study site. How would the results of the REMOTE study compare with those of a similar, completed, conventional study? Would this innovative research approach be validated?There were plenty of good reasons for Pfizer to try to answer these questions. “Virtual” trials would make research participation possible for patients who live far from investigative sites, or for patients who have trouble moving about. Virtual trials may reduce patient withdrawal rates, as remote access would likely be more convenient and require less of a time commitment from patients. They also have the potential to save research dollars.
Results of the REMOTE trial
Despite its potential, within a year, the REMOTE trial was in jeopardy. In its trial announcement, Pfizer said it had planned to enroll 600 patients, but in March 2012 reported having trouble meeting that target. In May 2012, Pfizer terminated further enrollment, able to randomize only 18 patients into the study.
Disappointing? Yes. End of the story? Not at all.
Last week, Pfizer formally published the results of the REMOTE pilot in Contemporary Clinical Trials. They concluded “the efficacy observed was consistent with results from conventional trials.” (Detailed results from the ClinicalTrials.gov website can be found here.) Though enrollment was quite low, that REMOTE results were consistent with those of the conventional study is a positive outcome, crucial for the viability of virtual trials.
Pfizer further concluded that “with simplification of multi-step screening and testing, web-based trials or their component parts should provide a participant-friendly approach to many clinical trials.” The first part of the statement, the piece about “simplification,” comes from analyzing what went wrong with enrollment. The second part of the statement comes from observing what components worked well.
Characteristics of the REMOTE trial
* Phase IV safety and efficacy study of Detrol LA for treatment of overactive bladder
* Recruitment, ID verification, screening questionnaires, and informed consent all conducted online
* 24/7 on-call support
* Instructional video available, developed by technology partner Mytrus
* Lab supplies and blinded study medication (or placebo)
delivered to patient homes
* Blood and urine tests conducted during a home visit or at
clinic local to patient
* Bladder diary maintained by patient using small mobile
device supplied by Pfizer
* Periodic self-assessments completed by patient via
online portal
* Individual clinical trial results shared with patient after study completion
What Went Wrong?
Through observation and patient feedback, Pfizer uncovered some of the operational factors that significantly contributed to the dismal enrollment. Identity verification had been conducted by asking a potential subject some facts from his/her personal history, such as the hospital in which the patient was born, the model of car the patient owned in 1997, the street on which the patient lived in 1985. Pfizer anticipated it would be able to validate the patient responses using publicly available sources. It turned out the necessary data was often publicly unavailable, leading to automatic exclusion. Also, the data entry software was unforgiving, and correcting mistakes was difficult. Lastly, a patient who successfully navigated the ID verification process would sometimes fail to proceed to the next step simply because confirmation email from Pfizer was blocked as SPAM. At least two of those issues can be easily fixed. (1) Data entry/interface software can be improved; ask any human factors expert. (2) Patients can be advised to add the appropriate domain name to their Safe Senders lists and email can be sent that is devoid of the characteristics SPAM filters love to hate.
What Went Well?
Answer #1: online video-based informed consent. Pfizer’s technique received positive feedback from both IRBs and federal regulators. The online format allowed consent to be standardized across sites, and subject understanding was verified using a quiz. Answer #2: EDC tools (ePRO, online portal, lab reporting, direct IVRS interface). They allowed for real-time edit checking, study and subject compliance monitoring, and early access to safety signals. Pfizer observed a significant drop in queries per subject than is typical. Both online video informed consent and the EDC tools with which Pfizer gained experience during the REMOTE pilot are the types of components, to which Pfizer referred in its final report, that could be used to enhance any study.
What’s Next?
Pfizer plans to conduct another virtual trial, REMOTE 2.0, in Europe. They have an opportunity to fix the issues they encountered during the pilot as well as introduce some other patient-centric strategies. Pfizer plans to offer additional assistance to patients during the screening process and to engage patients’ primary care physicians who can be a familiar presence in what can otherwise seem a foreign process. They also plan to issue iPad-like communication devices to facilitate communication with patients who spend little time online.
Surely, all eyes will be on the progress of enrollment and on-going process of patient engagement. What else will REMOTE 2.0 teach us?
by Laurie Meehan
[Some of the preceding information comes from a June 2013 talk at the ACRON Symposium, given by Miguel Orris, then Senior Director of Clinical Sciences at Pfizer.]
I propose to make "connected" studies in Hungary or Denmark. Those countries are in the lead in many aspects of Telemedicine...Thank you for your initiated Writing in the Clinical Trial Issue.
ReplyDeleteIf you hear of any Telemedicinal (I wonder if that's a word) being run, let us know! Thanks.
DeleteGreat commentary, thanks. I would be interested in their cost per trial participant and the funds spent on advertising and enrollment, it was very well publicized and seems odd they ended up with so few participants. Do they know what the logjam is here?
ReplyDeleteLooks like the logjam was their screening interface and ID validation. I'm curious to know how many potential patients they lost; in other words, what would recruitment have looked like if not for faulty procedures.
DeleteI got 0 participants in my study which I never conducted but I cured cancer! WTF! Are you serious comparing a study that enrolled less than 3% of its target enrollment to Ph III data? I'm sure there are lots of things that could be, can be, will be learned about virtual patients, but NOT from this study! I bet it also came in under budget which should be a factor in future trials!!!
ReplyDeleteSometimes experiments work and sometimes they don’t, but wouldn’t you agree that either way you learn things? Obviously, REMOTE ’s enrollment procedures were terribly flawed. Other elements of the study worked better. Pfizer thinks it learned enough to make changes and try again. I think that's a move forward; clearly you disagree.
DeleteI agree with Rob, we need transparency in methods and resource costs to learn anything. We plan to be transparent about success and failure so others can replicate and strengthen healthcare research rather than all of us struggling with the same reinvented distorted wheel. Who cares what Pfizer learned if it is inaccessible it is invisible speculation and a waste of space without information we can use. I can clearly see some issues from a Psych, neuroscience and logistical point of view by watching the video. Did they ever do a systematic review before they started on behavioral change/platforms/etc if they did we don't see it so can't see what they attempted to address or what they missed.
ReplyDeleteThanks, Laurie, for your post. I appreciate the conversation thread and feedback.
ReplyDeleteThe REMOTE study was an initiative in which Pfizer invested to enhance the way in which we develop our medicines. We are already using the output from the modules within REMOTE that were successful, incorporating tools such as remote monitoring, eConsent, and other components in our studies wherever they can add value by improving patient engagement, participation, and trial efficiency.
At the same time we have worked to be extremely transparent about the REMOTE project. We have shared through countless public forum -- from podcasts and tweetchats to industry conferences and media responses. That transparency has helped spark other companies to conduct their own experiments and investments improving study conduct and patient participation. The REMOTE study only carried the ball so far down the field, and we hope others can pick up and continue to move us all farther downfield toward our desired goal.
As a first-time pilot, many systems needed to be developed to enable the project. Inferences can not be made from the one-time investments for REMOTE to future trials. Our teams have done our own modeling for the expected return based on our study costs, and I would expect others interested in using such tools would do their own modeling in the course of their diligence to evaluate a potential ROI.
I believe we have shared significantly more details about this project than most other preceding internal industry efforts to impact this space. Doing so certainly can open up to critics, and I welcome constructive feedback. While I understand there may always be a hunger for details, I take that as a signal that we are going the right direction.
Disclosure: I work at Pfizer and am an author on the referenced REMOTE publication, but the opinions expressed are my own and not necessarily those of my employer or co-authors.
Thank you Craig for taking the initiative and for building the FDA approved fully online trial. This is huge for all of us. Our team has watched the space from the beginning and cheered you on. It is unlikely that any innovation of this magnitude would score 100% on all levels on its first run out of the gate. A small no frills initiative would not have yielded the needed information to improve the process.
ReplyDeleteI think many are not critical at all, they are curious and hungry for information to see the true potential of the internet as a place of research and learning. Added to this you are quite right as I know other industry efforts that were kept fully under wraps, with the trials not registered and not reported. Thanks for sharing what you have done. Is there a list of links somewhere that you would recommend as reflecting your position and purpose?
Nice comment, Amy. Thanks!
Delete