The Challenge
A global pharmaceutical company was running 15 concurrent Phase II/III oncology trials across multiple therapeutic areas. Each trial had its own RBQM approach:
- • Inconsistent KRIs: Each study team defined their own KRIs, making cross-trial comparison impossible
- • Fragmented dashboards: Some trials used Medidata Detect, others used custom R Shiny apps, others used Excel
- • Slow risk detection: Risk signals took 4+ weeks to surface due to manual reporting cycles
- • No knowledge sharing: Lessons learned in one trial weren't applied to others
- • Audit inconsistency: Regulatory inspectors found different RBQM maturity levels across trials
The VP of Clinical Operations needed a standardized RBQM framework that could scale across the entire oncology portfolio while maintaining flexibility for protocol-specific risks.
The Solution
I was brought in as Senior Clinical Data Science Lead to design and implement a standardized RBQM program. The engagement lasted 18 months and included:
Phase 1: Framework Design (Months 1-3)
- Core KRI Library: Designed 25 standardized KRIs applicable across all oncology trials (data quality, enrollment, safety, efficacy)
- Protocol-Specific Add-Ons: Created methodology for study teams to add custom KRIs for unique protocol risks
- Quality Tolerance Limits (QTLs): Validated thresholds using historical data from 8 completed oncology trials
- Closed-Loop Workflows: Documented escalation paths and mitigation procedures for each KRI
Phase 2: Technical Implementation (Months 4-9)
- Power BI Dashboard: Built automated dashboards pulling data from Medidata Rave, Oracle CTMS, and Safety Database
- ETL Pipeline: Designed Python-based ETL process to refresh dashboards daily (previously monthly)
- Automated Alerts: Configured email alerts when KRIs breach thresholds, with CTMS task creation
- Role-Based Access: Created dashboards for CRAs, Study Managers, and Executive Leadership with appropriate data views
Phase 3: Rollout & Training (Months 10-18)
- Pilot Trials: Rolled out to 3 trials first, refined based on feedback
- Global Training: Trained 120+ CRAs, Study Managers, and Data Managers across US, EU, and APAC
- Documentation: Created SOPs, user guides, and training videos
- Ongoing Support: Provided 6 months of post-rollout support and KRI tuning
The Results
Quantitative Outcomes
Qualitative Outcomes
- Cross-Trial Benchmarking: Study Managers could now compare performance across trials, identifying best practices
- Audit Readiness: FDA pre-approval inspection found "robust and well-documented RBQM program"
- Executive Visibility: C-suite gained real-time portfolio view, enabling data-driven resource allocation
- Knowledge Retention: Internal team now maintains and extends the system independently
Real-World Example: Early Risk Detection
In Month 14, the dashboard detected an unusual pattern at Site 042: tumor assessment data was being entered on time, but 40% of assessments showed "stable disease" (SD) compared to 15% portfolio average.
The automated alert triggered a root cause analysis. The CRA discovered the site was using an outdated RECIST 1.0 criteria instead of RECIST 1.1, systematically misclassifying partial responses as stable disease.
Impact: Site was retrained within 48 hours. Historical data was corrected before database lock. Without the KRI, this would have been discovered during statistical analysis—too late to fix.
Lessons Learned
- 1.Standardization ≠ Rigidity: The core KRI library provided consistency, but allowing protocol-specific add-ons ensured buy-in from study teams
- 2.Automation is Key: Daily dashboard refreshes (vs. monthly) made the difference between proactive and reactive risk management
- 3.Training Must Be Role-Specific: CRAs needed different training than Study Managers. One-size-fits-all training failed in pilot trials
- 4.Executive Sponsorship Matters: VP-level support ensured adoption. Without it, RBQM would have remained optional