Here's to Your eTMF Health

Twenty years ago, when I was employed at a small pharmaceutical company, we’d work every weekend for six months to get our New Drug Application assembled, QA’d, and packaged up to meet the FDA filing date. When the van carrying the completed application finally drove away to deliver the files to Washington, DC, we’d raise a glass of champagne and say “cheers” to a job well done.

Fast forward to today—the process for filing NDAs may be a little different, but the objectives are basically the same. We still assess, track, and report on biomarkers and other indicators of health status in patients during clinical trials to identify outcomes. Years ago, “metrics” was the accepted buzzword—now the common term is “Key Performance Indicators” or KPIs. Regardless of what you call it, the goal is to have an objective assessment of your success across key measures. In clinical trials, KPIs can include data collection, monitoring visits, number of subjects enrolled, number of SAEs, etc., but the same idea can be applied to all areas of operation, including the technology and service vendors employed by sponsors and CROs.

Companies have not historically tracked metrics for Trial Master Files (TMFs), since for so long they were all on paper. But as the industry has gradually moved towards eTMF systems, the ability to track more robust KPIs has exploded. Many companies new to eTMF struggle with what KPIs to collect. When I spent time designing paper CRFs many years ago (which are thankfully close to extinct), many companies collected data fields “just in case,” and much of the data collected was never used or analyzed, resulting in a significant waste of personnel and financial resources. Not tracking any KPIs is bad, but tracking KPIs excessively should also be avoided. Study teams should be able to leverage KPI data to meaningfully manage the health of the eTMF for a particular study, a program, or even across a compound.

At TransPerfect, we consult with many clients during implementation of our eTMF and study start-up solutions on best practices for measuring KPIs. Here are some examples of metrics and documentation that we feel are important in order to effectively measure performance:

• Number and status of site activation packages
• Number of active investigative sites
• Number of activated countries
• Number of amendments and sites, with all amendment approvals and documents needed on file (protocol signature pages, IRB/EC approval, IRB/EC approved ICF, etc.)
• Expiring and expired documents (such as medical licenses and other certifications that have an expiration date)
• Sites that will soon “expire,” meaning they are due for IRB/EC re-review based on the IRB/EC requirements for their institution
• Monitoring visit documentation per site (reports, follow up letters, etc.)
• Rejected TMF documents (due to poor quality, missing signatures, incorrect study, or a myriad of other reasons)
• Cycle time on site approval – Regulatory package distribution to activated site date
• Cycle time of TMF documents submitted to date published
• Number of times sites were contacted during the study start-up phase to get site activation packages completed
• “Uptime” of servers to ensure constant availability of the system
• Number of helpdesk tickets submitted as well as completion statuses on both the study and user level

The KPIs above are just a sampling of the metrics that are vital to managing the quality of the eTMF and study start-up process on an ongoing basis. Unfortunately most companies—if they measure eTMF KPIs at all—scramble to collect these KPIs at the end of the study prior to study archival, or worse yet, once they are informed that the sponsor (if they are a CRO), FDA, or other regulatory authority will be coming in for a inspection. This happens more than you would expect, as many companies do not see the TMF as being on the critical path—until they have a poor regulatory inspection or an internal audit of the TMF.

We encourage our clients to identify and agree on KPIs early on in the eTMF setup so that they can consistently be measured within and across studies, which helps companies stay in compliance with regulatory requirements, as well as giving study stakeholders and their management teams access to key information that is crucial to the business. Push notifications from eTMFs are a necessity, as they provide near-real-time access to mission-critical information enabling key decisions to be made rapidly and decisively.

Monitoring the health of your eTMF is a lot like monitoring your personal health. Staying in front of it can prevent disease or provide the opportunity for early intervention should an issue come up. No matter how you give a toast in your culture—whether it’s cheers, kanpai, salud, cin cin, or something else—we at TransPerfect want you to be armed with the best KPIs so you can toast to the success of your clinical trials and the ultimate approval of your product. Cheers to your eTMF health!