A Modern Risk-based Monitor

Investigator fawn on her.  Coordinators call on her.
She is the very Model of a Modern Risk-based Monitor.



With a little help from Gilbert & Sullivan, Polaris describes the quintessential Risk-based Monitor.
(Sung to the tune of "I Am the Very Model of a Modern Major General.")

Operetta notwithstanding, we've been closely watching the evolution of RBM in the industry and interacting with many of the significant players from sites, sponsors, CROs, and IT solution vendors.
If you Subscribe to our Newsletter this month, we'll send you a free compilation of our articles on RBM.

No singing.

What Does Risk-based Monitoring Mean for QA Auditing?

By now, we all know that risk-based monitoring isn’t just about changing the role of the CRA; it’s transforming the way clinical studies are managed.  So what does that mean for QA teams who audit these new processes?  Polaris president Celine Clive led a roundtable discussion about RBM and its implications for auditing at November’s North Carolina Regulatory Affairs Forum (NCRAF) meeting.

It’s a slippery subject.  Traditional monitoring relied on the gold standard of on-site visits every 4-8 weeks and 100% SDV.  RBM is not replacing this standard with another.  RBM is a framework for customizing a monitoring approach for each study, and guess what -- “custom” is a lot trickier to audit than “standard.”

Two New RBM Papers from TransCelerate

From One, Come Two
If you follow the TransCelerate RBM initiative closely, since its July update you’ve been eagerly anticipating the availability of the paper entitled “TransCelerate BioPharma: Central Monitoring Model and Technology Specification Proposals to Enable Risk-Based Monitoring Adoption.”  Now that the paper is publically available, your patience has been rewarded twice.  TransCelerate chose to divide the single document into two, one focusing on central monitoring, the other focusing on RBM-enabling technologies.  Publishing separate documents was a good decision.  As you’ll see, each topic is certainly rich enough to warrant individual treatment.

Is the SDV in Risk-based Monitoring Misunderstood?

What’s in a name?  Not much according to Shakespeare.  But the man who penned “A rose by any other name would smell as sweet” was a 16th century poet and playwright, not a 21st century clinical research professional.  For us, names matter.  Despite recent efforts to standardize the definition of Source Data Verification (SDV), the term still means different things to different people, and that needs to be navigated very carefully. 

A Curated Collection of Tweets from ACRP 2014

Some Polaris staff members attended ACRP last month in San Antonio. It was a great conference. See what we and other attendees had to say about a few key sessions here.

(This Twitter compilation was extremely easy to curate and annotate using a free service called Storify. You don’t even need to have a Twitter account to do it. In fact, you don’t even need a Twitter account to follow a conference hashtag. Feel free to contact us via our website if you’d like to learn how; we'd be glad to share!)

Summary of TransCelerate’s Update to its Risk-based Monitoring Position Paper

The 7-page document provides an update on TransCelerate’s Risk-based Monitoring (RBM) initiative.  It includes a list of lessons learned, shares FDA feedback on RBM pilot studies, and provides links to RBM training developed by TransCelerate (TCB) member companies.  It also outlines 2014 publishing plans.

Risk-based Monitoring vs Traditional Monitoring: the ADAMON Project

Many clinical research professionals are RBM believers, convinced that a risk-based approach to monitoring represents an effective and more efficient alternative to the tradition of frequent on-site visits and 100% SDV.  Many others are cautiously open to the idea, and are waiting for news of successful RBM studies to start filling the industry journals and online forums.  From those same journals and forums, we know that RBM studies have been, and are currently being, conducted.  In collaboration with CluePoints, GSK Vaccines is piloting a central statistical monitoring approach.  Sanofi and Lilly are currently piloting the TransCelerate methodology published in May.  Bayer and other sponsors are using an analysis of EDC data and metadata to steer their on-site monitoring activities.  Many CROs are offering their sponsor clients alternative monitoring solutions; Quintiles has conducted upwards of 60 studies using a number of different RBM implementations, Health Decisions has had years of experience conducting trials using its adaptive monitoring approach, and ICON has just announced its ICONIK Monitoring service after several years of piloting it alongside the traditional approach.  Finally, almost every RBM webinar (RBM-inar?) I’ve attended has mentioned that government and academic clinical trials commonly employ a RBM strategy.

But it’s been quiet.  The big headlines necessary to assuage lingering doubts have not been forthcoming.  Even as we begin to hear about RBM successes, how will we know if RBM is as good as traditional techniques? 

Assessment of FDA’s Final Guidance on Risk-based Monitoring

Yesterday, FDA finalized its 2011 draft guidance on risk-based monitoring (RBM). Without preamble, here’s what has changed.

Compared with the draft version of the guidance, the final version:

• Reinforces, even strengthens, its support of RBM

• Suggests “more intensive monitoring” does not necessarily indicate on-site monitoring

• Provides specific examples of critical/non-critical data and tasks that could be accomplished remotely

• Emphasizes the importance of protocol and CRF design

• Notes that, under RBM, delegating monitoring obligations to a CRO requires more clarity, communication, and evaluation than traditional monitoring

Unraveling Risk-based Monitoring Terminology

“Judging by the sheer number of conference presentations, forum discussions, and webinars, not to mention all the new vendor offerings built to facilitate the process, risk-based monitoring might be one of the most frequently discussed topics in clinical research today.”

That sentence opened our December blog post on risk-based monitoring, and it’s as true today as it was then. (In fact, I’ll say it’s even truer today, if only to spark debate about truth being an absolute.)

The more a topic is discussed, the more words people use to discuss it.  Of course, a few subject-specific, well-defined terms can facilitate conversation, but over time, meanings blur.  Industry-established definitions give way to corporate-centric, implementation-specific usage.  We’ve all seen this recipe for miscommunication before.  The German and the Frenchman know that they speak different languages and can take measures to assure accurate translation.  Two people who speak the same language and use the same words may not even realize they might not be saying the same thing.

This blog post looks at the terminology used in what are arguably the three most important and oft-quoted papers on Risk-based Monitoring in the industry:

·        FDA draft guidance Oversight of Clinical Investigations – A Risk-based Approach

·        EMA Reflection Paper on Risk-based Quality Management in Clinical Trials

·        TransCelerate BioPharma Position Paper on Risk-based Monitoring Methodology

I carefully reviewed these three documents looking for consistencies on which we can rely, and inconsistencies of which we need to be cautious.  For each set of terms below, I first present the conclusions I’ve drawn about usage consensus, or lack thereof, and then suggest some tips to avoid confusion in discussions you might have with colleagues and clients.  The sane among you will probably be interested in only my conclusions and suggestions; you linguists, skeptics, and masochists may be interested in the supporting details I added to the end of the post.

Sorting out terminology and definitions is a tricky business.   If you have different interpretations than I present here, or can add to this analysis, please post a comment and we can start a discussion.

Remote versus Off-Site Monitoring

Conclusion:

Sources agree that these terms are synonymous and used to describe traditional on-site monitoring activities that, thanks to technology, are now able to be conducted elsewhere.

Tip:

You can use these terms interchangeably, but continue reading to avoid confusion with the term “central”.

Remote/Off-Site versus Central Monitoring

Now that we’ve established that remote monitoring and off-site monitoring can be used interchangeably, let’s compare the pair to central monitoring.

Conclusion:

Sources agree that central monitoring includes remote/off-site monitoring activities, but includes other functions, as well, predominately statistical analysis. 

Tip:

Sometimes the literature uses “remote/off-site” and “central” interchangeably, but that’s risky.  Due to its statistical component, “central monitoring” has the widest range of interpretations of all the terminology in use, and some of the interpretations are very different from the comparatively straight-forward “remote/off-site monitoring” definition.  Read on.

(BTW, FDA uses “centralized”; TransCelerate uses “central”.  Tomato, tomahto.)

Central Monitoring and Statistical Analysis

Conclusion:

“Central monitoring” implies that data across study sites are collected centrally, and available for statistical analysis.  The type and extent of  statistical analysis that is performed to adapt a monitoring plan can vary significantly.   Sometimes analysis is limited to Key Risk Indicators or critical data points, and sometimes more rigorous statistical testing is performed on every data item, as any outlying element can be indicative of data quality.

Tip:

When using the term “central monitoring” outside a narrow culture, make sure to clarify whether you mean a slight superset of remote/off-site monitoring, statistical analysis of Key Risk Indicators, or something more.  Different computing platforms offer different features; if it’s appropriate, mentioning the IT platform that is being used could clarify the type of central monitoring being discussed.

Adapted versus Adaptive Monitoring

If you’ve stuck with this analysis thus far, you deserve some lighter commentary…

Conclusion:

Usage is too loose to be of much danger.

Tip:

Use interchangeably with giddy abandon.

Centralized versus Centralised Monitoring

Conclusion:

It depends on which side of the pond you are.

Tip:

Stick to verbal discourse and there is no difference.  (Tomato, tomayto.)

Closing Thought (before I lose you all)

My college French professor was the first to point out to me that irregular verbs, the verbs that refuse to follow regular conjugation patterns, are the most common verbs in every language – the verbs to be, to go, to do, to have, to see, etc. They’re used so often that they bend and blur and evolve on their own. I think the jumble of definitions that arises with an oft-discussed topic like risk-based monitoring is somehow analogous to the unstructured forms of irregular verbs. The more they get thrown around, the more they go their own way.

 

By Laurie Meehan

We at Polaris hope you find this information helpful. You can visit us at our website, www.polarisconsultants.com or you can contact us at info@polarisconsultants.com.
 

--------------------------------------------------------------------------------------------

Supporting Detail (as promised/threatened)

Remote versus Off-site Monitoring:

The FDA guidance does not formally define either of these two terms, but says, “source data verification and other activities traditionally performed by on-site monitoring can now often be accomplished remotely, as both trial data and source data typically become part of the central submission.”  The TransCelerate paper equates the two terms, and formally defines both as “monitoring activities as defined either within process documents or in the MP that occur away from the study site location (e.g. at a Monitor’s home or in a sponsor representative’s office.)”  The EMA paper uses neither term.

Remote/Off-site versus Central Monitoring:

The definition of central monitoring in the FDA guidance consists of 8 bullet items, only some of which are the technology-enabled remote/off-site version of traditional on-site monitoring.  The TransCelerate paper says, “Off-site Monitoring Activities are performed by Monitors and can be distinguished from Central Monitoring which could also be performed by Monitors or other roles within clinical operations or by other functions (e.g. Statistics, Data Management, Safety).”

Central Monitoring and Statistical Analysis:

The TransCelerate paper focuses on analyzing Key Risk Indicators to determine and adapt a monitoring strategy.  The appendices are ripe with examples that show how thresholds can be used to dictate actions and trigger on-site visits.  The paper doesn’t preclude analyzing all study data (both critical and non-critical) for outliers, but doesn’t discuss much about the role that this type of statistical analysis might play or provide details on its use.  The definition of “central monitoring” in the FDA guidance is more general.  “Conduct aggregate statistical analyses of study data to identify sites that are outliers relative to others” might refer to Key Risk Indicators.  It might also refer to the “analysis-of-every-data-element” approach that some platform vendors have implemented to detect outlying, hence higher risk, sites.  The EMA paper speaks only of the potential to develop central monitoring systems that use statistical methodology, and leaves it to future papers to delve into more detail.

Adapted versus Adaptive Monitoring
Of the three sources, the EMA reflection paper uses the term “adapted” the most, but never formally defines it.  “Adapted,” as it relates to monitoring, can refer to the practice of tailoring monitoring procedures for each protocol.  (“Safety monitoring procedure adapted to each trial”)  It can also refer to mid-study changes to the monitoring procedures that were set out at the beginning of the study.  (“[Priorities] should be carefully set out so that risk analysis is carried out and control measures are designed in a way that is continuously adapted to them.”)  The TransCelerate paper uses the term “adaptive,” almost as generically, when it defines risk-based monitoring to be an adaptive approach.

Centralized versus Centralised Monitoring:

No factual foundation whatsoever; I went out on a limb.

Examining the Slow Transition to Risk-based Monitoring

In August 2011, the FDA published its draft guidance to industry, Oversight of Clinical Investigations – A Risk-based Approach.  The EMA, MHRA, and European Commission have also published papers in support of risk-based or adaptive monitoring.  Judging by the sheer number of conference presentations, forum discussions, and webinars, not to mention all the new vendor offerings built to facilitate the process, risk-based monitoring might be one of the most frequently discussed topics in clinical research today.

The FDA guidance encourages organizations to forgo the former one-size-fits-all, gold standard of monitoring:  100% source data verification (SDV) and on-site visits every 4 to 6 weeks.  There are two distinct themes at play here.  First, information technology reduces, if not obviates, the need to review study data on-site.  That site visits might be less frequent as a result cannot be a wholly unexpected consequence.  A move towards remote data monitoring, and any associated decrease in on-site visits, does not necessarily represent a philosophical shift for FDA, so much as an acknowledgement of the inevitability of advancing technology.  Had the new guidance been titled Oversight of Clinical Investigations – A Remote Monitoring Approach, the industry might not have raised a collective eyebrow.

A far more philosophical departure is FDA’s position that traditional absolutes, such as100% SDV, are often overkill -- they don’t necessarily produce more reliable results or offer greater subject protection than less intensive approaches would.  Instead, FDA says that decisions about frequency, intensity, and type of monitoring should be made according to the risk that each study, and even each site, presents.  Variables to consider in these decisions are therapeutic area, protocol complexity, investigational product safety, population vulnerability, investigator experience, EDC capabilities, and study phase.

Verifying anything less than 100% of source data is easier, cheaper, and faster than verifying all of it.  Applying more resources to critical areas and fewer resources to less important, less risky areas sounds obvious.  So why haven’t sponsors and CROs scrambled to implement these process changes as fast as possible? 

Assessing risk is hard.  You can download all the risk assessment tools you want, but it doesn’t change the fact that striking a smart balance between risk and benefit is the result of careful analysis, long experience, and good judgment.  You’ll probably never know if you overestimate the risk you face.  You’ll just end up performing unnecessary monitoring procedures that offer little benefit, but only after you’ve spent a lot of time and money on a useless risk assessment effort.  If you underestimate the risk you face and fail to sufficiently monitor the study, you defeat the entire essential purpose of study oversight, jeopardizing the safety of your patients and the reliability of your results.

Another reason sponsors and CROs have moved slowly towards an adaptive monitoring approach rests with the study sites themselves.  Without 100% SDV, sites need to integrate quality management into their processes, rather than rely on monitors to identify questionable data.  Risk-based monitoring requires more vigilance from site staff than is necessary with the traditional monitoring approach, and may require additional site training. 

Despite these difficulties, some sources report that companies are nevertheless beginning to implement risk-based approaches to clinical oversight.  In January, the Applied Clinical Trials website published a trend analysis developed by Medidata Insights.  Among their client companies, Medidata reports that median SDV percentage rates have been dropping steadily since 2007. 

Still, the perception is that transition to risk-based monitoring is slow.  An Outsourcing-Pharma article recently quoted the CEO of Annex Clinical who says firms are still checking every piece of data associated with a clinical trial, including data not central to data integrity or subject protection.  That same publication summarized the opinions of many attendees and speakers at this year’s Partnerships for Clinical Trials conference.  They believe that the guidances on risk-based monitoring lack clarity, and are urging regulatory authorities to make more solid rules.  Even Big Pharma concedes it has to understand risk-based monitoring better.  In September, Abbott, AstraZeneca, Boehringer Ingelheim, Bristol Myers Squibb, Eli Lilly, GlaxoSmithKline, Johnson and Johnson, Pfizer, Roche’s Genentech division, and Sanofi formed the non-profit TransCelerate BioPharma, with the goal of improving the quality of clinical studies and accelerating the development of new medicines.  Among their stated objectives is the development of risk-based site monitoring approaches and standards.

If Big Pharma is still struggling to understand comprehensive risk-based monitoring, smaller biotech companies are even further from implementation.  Smaller companies are less likely to have the resources and experience necessary to conduct thorough risk assessments and then translate them into adapted monitoring plans.  Also, small biotechs are more likely to conduct significantly smaller studies, which may translate into smaller risk tolerance.

And then there’s this:  Big Pharma is notoriously risk averse when it comes to deciding what biotech firms to purchase or what new products to license.  A due diligence team needs to present an airtight case to its management, who, in turn, wants to make sure the company’s investment will be sound.  Smaller firms understand this.  They know that they may only have “one shot on goal” to either partner or sell, and are concerned that their Big Pharma suitors will be dissatisfied with anything less than 100% SDV and frequent on-site visits. 

So what happens from here?  Transition to a risk-based monitoring approach may continue to inch along at its current pace.  Organizations like TransCelerate may develop standards which set the stage for large-scale adoption.  Maybe a final version of the FDA guidance will offer clearer parameters for implementation.   Whatever happens in the industry at large, it’s likely that the need to reduce clinical development costs will motivate some companies to aggressively move toward risk-based approaches, with or without accepted industry standards or further regulatory guidance.  They’ll be the ones to watch.

By Laurie Meehan