Every year, pharma companies spend tens of millions of dollars on real-world evidence programmes that fail to deliver what was promised. Not because the data didn’t exist. Not because the analysts weren’t capable. But because the strategic assumptions behind the programme were wrong from the start — shaped by myths that have become so embedded in the industry that almost nobody questions them anymore.
Here’s what I’ve noticed working across Continental European markets: the real-world evidence space is saturated with advocacy. Every data vendor, every CRO, every conference panel will tell you that RWE is transformative. What they won’t tell you is where it breaks down. And the silence around those limitations is creating some of the most expensive strategic mistakes I’ve seen pharma teams make.
This post isn’t here to sell real-world evidence. It’s here to help you use it correctly — which means being honest about both its power and its boundaries. The five myths below are the ones I’ve seen cost organisations the most. Some of them will be familiar. A few might surprise you.
If you’ve ever watched an RWE submission get pushed back by a payer or questioned by a regulatory reviewer, there’s a good chance at least one of these myths was somewhere in the chain of decisions that led there.
Before diving in, if you’re already thinking about what analysis-ready real-world evidence looks like in practice, our post on analysis-ready real-world evidence and healthcare data for pharma is worth reading alongside this one.
Setting the Stage: What RWE Actually Is — and Isn’t
Before dismantling the myths, let’s establish a precise definitional foundation — because most of the misconceptions below stem from conflating two distinct concepts.
Real-world data (RWD) is the raw input. Health insurance claims records, electronic health records, disease registry entries, patient-generated data — these are real-world data sources. They capture what happened in routine clinical practice: what was prescribed, what was billed, what diagnostic codes were recorded.
Real-world evidence (RWE) is the analytical output — the conclusions drawn from that data through appropriate study design, rigorous methodology, and careful interpretation. Data doesn’t become evidence just by existing. It becomes evidence through the analytical process applied to it.
This distinction matters enormously. A claims dataset is not evidence of anything. It’s raw material. The evidence is what emerges when you apply appropriate epidemiological methods to answer a specific, pre-defined question. Sherman et al. established this taxonomy clearly back in 2016, and the FDA’s Framework for Real-World Evidence formalised it for regulatory purposes. Most of the myths below collapse this distinction in ways that lead to significant strategic errors.
Myth 1: “RWE Can Replace Randomised Controlled Trials”
This is the most dangerous myth in the space. And it’s not a fringe view — I’ve heard it from CMOs, from boards, and from senior commercial leaders who’ve been pitched it by vendors with a financial interest in them believing it.
Let me be direct: RWE and RCTs answer fundamentally different questions. A randomised controlled trial establishes efficacy — does this drug work under tightly controlled conditions, in a selected population, against a defined comparator? Real-world evidence establishes effectiveness — does this drug work in routine clinical practice, in the full heterogeneity of real patient populations, in actual treatment sequences? These are not the same question. Neither replaces the other. They’re complementary — and treating them as substitutes is where strategies fall apart.
Where does the regulatory landscape actually stand in 2025–2026? The FDA has been expanding its acceptance of RWE through its Advancing Real-World Evidence Program, and recent analyses of FDA drug approvals have shown growing use of RWE to support supplemental indications and post-approval label expansions. EMA published its reflection paper on non-interventional studies using real-world data in 2025, reinforcing the role of RWE within a defined methodological framework.
But read that context carefully. The regulatory momentum is concentrated in specific contexts: supplemental indications, external control arms in rare disease programmes where randomisation is genuinely infeasible, post-approval safety and effectiveness commitments. Primary efficacy evidence for novel drug approvals still requires randomised trial data. Full stop. FDA guidance is explicit on this, and EMA’s position is consistent.
The most useful conceptual tool here is Hernán and Robins’ target trial emulation framework — the idea that any observational analysis should explicitly emulate the randomised trial it’s attempting to approximate, defining the same eligibility criteria, treatment strategies, follow-up period, and outcome definitions. Their work doesn’t argue that observational studies replace trials — it argues that the rigour of trial design thinking should discipline every observational analysis. That’s a very different claim.
If your evidence strategy rests on the assumption that a real-world data analysis will substitute for prospective trial data in a primary efficacy submission, the consequences of discovering that assumption is wrong can run to years of delay and hundreds of millions in remediation costs.
Myth 2: “Claims Data Is Accurate Enough for Any Research Question”
This one tends to live in commercial analytics teams and data science functions. The dataset is large. The coverage seems broad. The coding looks consistent. So the assumption is: we can answer whatever question we throw at it.
Here’s what most teams miss: claims data was built for billing, not research. It captures what was billed — not what was clinically intended, not what the patient actually experienced, not what the outcome was. That structural reality has specific, consequential implications for which research questions it can credibly answer.
ICD-10 diagnostic coding in administrative data is driven partly by reimbursement logic, not purely by clinical reality. Conditions may be undercoded because they weren’t relevant to the billing claim. Comorbidities may be overcoded because they increase reimbursement. Coding practices vary by institution, by specialty, and by region — creating systematic biases that don’t look like noise in aggregate statistics but are absolutely real in the underlying data. Iezzoni’s foundational work on administrative data validation documented this dynamic decades ago. It hasn’t gone away. The downstream consequence of this is cohort misclassification — which is why phenotype validation in claims data is the methodological step that separates studies that hold up to scrutiny from those that don’t.
Prescription fill data — which gets treated as a proxy for adherence constantly — measures whether a prescription was dispensed. It does not measure whether the patient took the medication. In therapeutic areas where adherence is a major driver of outcome variation, this gap is not a rounding error. It’s a structural limitation that invalidates adherence-dependent conclusions.
The FDA addressed this directly in its 2024 guidance on assessing EHR and medical claims data, establishing specific criteria for evaluating fitness for purpose in regulatory contexts. The RECORD statement from Benchimol et al. provides the reporting standards that allow reviewers to assess whether these limitations have been appropriately acknowledged and addressed.
The correct framework isn’t “is this data accurate?” It’s “is this data fit for this specific research question?” Those are very different questions. Claims data is excellent for treatment pattern analysis, healthcare utilisation, and broad epidemiological characterisation. It has genuine limitations for outcomes research requiring clinical endpoints, for safety surveillance requiring granular adverse event capture, and for comparative effectiveness analyses requiring detailed confounder adjustment. Knowing which category your question sits in is not optional — it’s the foundational step in study design.
Myth 3: “More Data Means Better Evidence”
This is the myth that data engineering teams and technical leadership are most prone to. And it’s intuitively appealing — if a study with fifty thousand patients is good, surely a study with ten million patients is better? More data means more statistical power, less sampling error, more representative findings.
Except that’s not how this works.
A poorly designed observational study with ten million patients produces worse evidence than a well-designed study with fifty thousand. Worse — not just comparably bad. Because scale amplifies bias. When a study has systematic confounding baked into its design, larger sample sizes don’t dilute that confounding — they make spurious associations appear statistically significant with increasing confidence. You end up more certain about a wrong answer.
The most consequential real-world data quality issues in pharma research are design problems, not data volume problems. Confounding by indication — where the reason a treatment was selected is correlated with the outcome — is the endemic challenge in observational pharmacoepidemiology. Immortal time bias — where the period between cohort entry and treatment initiation is incorrectly attributed to the treated group — has invalidated entire classes of observational findings. Selection bias in how cohorts are defined, informative censoring in how follow-up is handled, missing data patterns that are never random — none of these are solved by adding more patients.
Schneeweiss et al.’s work on longitudinal study design visualisation and Hernán’s arguments on causal inference in observational data both make the same essential point: the design question should come before the data question. Always. What causal claim are you trying to make? What confounders need to be addressed? What analytical approach — propensity score matching, inverse probability weighting, difference-in-differences — is appropriate for this specific inferential task? ISPOR-ISPE good practice guidance formalises this sequence.
The GRACE principles for recognising high-quality comparative effectiveness studies assess study quality on design rigour, not dataset size. HTA bodies applying these frameworks to RWE submissions are evaluating methodology first. A massive dataset with methodological weaknesses will get harder questions from reviewers, not easier ones — because reviewers understand that scale amplifies bias.
Myth 4: “RWE Is Only Useful After Approval”
This is the myth that clinical development teams and R&D leadership hold most often. Real-world data is for post-approval pharmacovigilance. For HEOR submissions after launch. For label expansion programmes. It’s not something the development team needs to think about until the drug is on the market.
That view has become increasingly costly to maintain.
Real-world data adds measurable value throughout the development lifecycle — and the strategic windows it enables are only available before approval. By the time you’re in post-approval mode, those windows have closed.
Disease natural history analysis from real-world data informs trial design. Understanding how patients actually progress through treatment sequences in clinical practice — what gets prescribed first, what gets added, when patients switch, what the real-world rate of discontinuation looks like — tells you things about trial design and endpoint selection that no preclinical model can approximate. Dagenais et al. documented the expanding use of RWD throughout the development lifecycle in their 2022 review.
External control arms using historical real-world data are gaining genuine regulatory traction in rare disease programmes where randomisation is genuinely infeasible. The FDA’s 2023 guidance on externally controlled trials provides the framework — and Thorlund et al.’s primer on synthetic and external controls outlines the methodological requirements in practical terms.
Trial site selection, inclusion and exclusion criteria optimisation, patient population feasibility assessment — all of these benefit from real-world data analysis before a single patient is enrolled. Madabushi et al.’s work on model-informed drug development makes the case for integrating real-world evidence into the lifecycle from early development rather than treating it as a post-market afterthought.
The opportunity cost of the post-approval-only mindset is substantial. Teams that build RWE capabilities early can use them to make better development decisions, design better trials, and arrive at approval with a richer evidence base. Teams that wait until post-approval are playing catch-up.
Myth 5: “If the Data Exists, We Can Answer the Question”
This is the myth most commonly held by procurement teams, senior leaders commissioning studies, and business development functions evaluating pipeline assets. The logic sounds reasonable: you need to answer a question about treatment patterns in a given therapeutic area, a provider tells you they have data on that therapeutic area, therefore you can answer your question.
Data availability does not equal analytical fitness. These are different things, and confusing them creates the most common and most expensive RWE project failures I’ve seen.
Geographic coverage gaps mean a dataset that appears comprehensive may have thin or unrepresentative coverage for the specific markets you need. Therapeutic area depth varies enormously — a dataset excellent for oncology may have insufficient granularity for rare metabolic conditions. Data recency issues mean that a dataset licensed in early Q2 may actually reflect clinical practice from nine or twelve months earlier, which matters enormously for competitive intelligence applications. Linkage quality between data modules — claims to pharmacy, inpatient to outpatient — affects whether longitudinal analyses are even possible.
The fitness-for-purpose principle — matching data source selection to the specific evidence question — is the correct framework, and both Makady et al. and Garrison et al. have articulated it in the context of coverage and payment decisions. The ISPOR Real-World Evidence Transparency Initiative sets guiding principles specifically to address the mismatch between data availability and analytical credibility.
The correct sequence is: define the analytical question first, specify the data requirements it implies, then evaluate which available sources meet those requirements. Most teams do this in reverse — they find out what data a vendor has, then figure out what questions they can ask of it. That reversal is where fitness-for-purpose failures begin.
Understanding which data sources match specific use cases requires deep market knowledge that most internal teams don’t have. The Pan-European data landscape across Continental European markets is genuinely complex — providers whose datasets are appropriate for commercial analytics may be entirely inappropriate for HEOR submissions or regulatory evidence, and the distinctions are not always visible from a vendor pitch deck.
When RWE Is Genuinely Not the Right Answer
Most RWE content carefully avoids this section. I’m not going to do that — because intellectual honesty about when real-world evidence falls short is precisely what distinguishes rigorous advisory from vendor advocacy.
There are contexts where RWE cannot substitute for trial data, and no amount of analytical sophistication changes that.
Primary efficacy evidence for novel drug approvals. Regulators still require randomised trial data for the primary efficacy claim. This is not a limitation of current RWE methodology that will be resolved by better data or more sophisticated causal inference. It reflects a fundamental epistemic difference between efficacy under controlled conditions and effectiveness in routine practice — a difference that regulators are right to take seriously. Kesselheim and Doherty assessed these challenges directly.
Situations where confounding by indication cannot be adequately controlled observationally. When treatment selection is so strongly determined by disease severity, patient frailty, or prescriber preferences that no available covariate adjustment can break the association, observational evidence simply cannot support causal comparative effectiveness claims. This isn’t always obvious at the study design stage — which is precisely why it requires epidemiological expertise to identify before resources are committed.
Analytical questions requiring clinical variables that administrative data structurally cannot capture. Performance status, symptom severity, patient-reported outcomes, detailed pathological staging — if your research question depends on these and your dataset doesn’t have them, no analytical technique compensates for their absence. A recent CIOMS Working Group report from Hennessy et al. articulates this boundary in the context of regulatory decision-making.
Recognising these limitations is not a sign that real-world evidence has failed. It’s a sign of methodological maturity. The teams that understand where RWE stops being appropriate are the same teams whose RWE submissions hold up to scrutiny — because they’ve only used it where it’s defensible.
What Good RWE Strategy Actually Looks Like
Having been honest about where real-world evidence goes wrong, let me close with what going right actually looks like.
Good RWE strategy starts with the evidence question, not the dataset. Before any conversation about data sources, there should be a precise, pre-specified statement of what you’re trying to establish, for what regulatory or payer audience, in what market context, using what comparator and outcome definition. Blonde et al.’s work on interpreting real-world clinical data and FDA’s experience with its Advancing RWE Program both reinforce this principle: question first, data second.
Good RWE strategy matches study design to the regulatory or payer context before selecting a data source. What will a national HTA body accept as evidence for comparative effectiveness? What methodological standards does EMA’s guidance require? What does the ISPOR-ISPE framework recommend for the analytical design you’re considering? These are not afterthoughts — they define the design requirements before you know which data source can meet them.
Good RWE strategy acknowledges limitations transparently rather than papering over them. Reviewers at HTA bodies and regulatory agencies have seen every attempt to minimise a data limitation in a dossier. The studies that survive scrutiny are the ones where limitations are explicitly addressed — with pre-specified sensitivity analyses, with methodological robustness checks, with honest discussion of what the findings can and cannot support.
And good RWE strategy combines real-world and trial evidence where each adds distinct and complementary value. Neither is inherently superior. The intelligent question is always: what does each source add that the other cannot provide?
The challenge many pharmaceutical companies face isn’t finding data — it’s determining which combination of analytical approaches and data sources aligns with their specific evidence requirements, across the relevant markets, for the right regulatory and payer audiences. Navigating that complexity requires expertise in both pharmaceutical evidence strategy and the evolving Pan-European data landscape. Expert advisors who understand both your business objectives and the methodological requirements can accelerate your path to credible evidence while avoiding the costly mismatches this post has described.
If you’re designing a real-world evidence programme — or questioning whether an existing one is on the right track — contact us to discuss your specific evidence requirements. A strategic consultation is the right first step before committing to a study design or data source.
Sources & References
- Sherman, R. E., et al. (2016). Real-world evidence — what is it and what can it tell us? New England Journal of Medicine, 375(23), 2293–2297. https://www.nejm.org/doi/full/10.1056/NEJMsb1609216
- U.S. Food and Drug Administration (2018). Framework for FDA’s Real-World Evidence Program. https://www.fda.gov/media/120060/download
- Hernan, M. A., & Robins, J. M. (2016). Using big data to emulate a target trial when a randomized trial is not available. American Journal of Epidemiology, 183(8), 758–764. https://academic.oup.com/aje/article/183/8/758/2739044
- Innes, et al. (2025). Real-world evidence in new drug and biologics license application approvals during fiscal years 2020–2022. Clinical Pharmacology & Therapeutics. https://ascpt.onlinelibrary.wiley.com/doi/10.1002/cpt.3688
- European Medicines Agency (2025). Reflection paper on the use of real-world data in non-interventional studies to generate real-world evidence. https://www.ema.europa.eu
- U.S. Food and Drug Administration (2021). Considerations for the use of real-world data and real-world evidence to support regulatory decision-making for drug and biological products. https://www.fda.gov
- Benchimol, E. I., et al. (2015). The RECORD statement. PLOS Medicine, 12(10). https://journals.plos.org/plosmedicine
- U.S. Food and Drug Administration (2024). Real-world data: assessing electronic health records and medical claims data to support regulatory decision-making. https://www.fda.gov
- Iezzoni, L. I. (2005). Validating administrative data in health services research. Medical Care, 43(10). https://journals.lww.com
- Schneeweiss, S., et al. (2019). Graphical depiction of longitudinal study designs in health care databases. Annals of Internal Medicine, 170(6), 398–406. https://www.acpjournals.org
- Hernán, M. A. (2018). The C-word: scientific euphemisms do not improve causal inference from observational data. American Journal of Public Health, 108(5), 616–619. https://ajph.aphapublications.org
- Dreyer, N. A., et al. (2018). GRACE principles. American Journal of Managed Care, 16(6), 467–471. https://www.ajmc.com
- ISPOR-ISPE Special Task Force. Good practices for real-world data studies of treatment and/or comparative effectiveness. https://www.ispor.org
- Dagenais, S., et al. (2022). Use of real-world evidence to drive drug development strategy. Clinical Pharmacology & Therapeutics, 111(1), 77–89. https://pubmed.ncbi.nlm.nih.gov/34839524/
- U.S. Food and Drug Administration (2023). Considerations for the design and conduct of externally controlled trials. https://www.fda.gov/media/164960/download
- Thorlund, K., et al. (2020). Synthetic and external controls in clinical trials. Clinical Epidemiology. https://pubmed.ncbi.nlm.nih.gov/32440224/
- Madabushi, R., et al. (2022). Role of model-informed drug development approaches. Clinical Pharmacology & Therapeutics. https://pmc.ncbi.nlm.nih.gov/articles/PMC9097888/
- Makady, A., et al. (2017). Policies for use of real-world data in health technology assessment. Value in Health, 20(3), 520–532. https://www.valueinhealthjournal.com
- Garrison, L. P., et al. (2007). Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. Value in Health, 10(5), 326–335. https://www.valueinhealthjournal.com
- ISPOR (2022). Real-world evidence transparency initiative: guiding principles. https://www.ispor.org
- Kesselheim, A. S., & Doherty, M. (2021). Real-world evidence in drug regulation. Health Affairs, 40(2). https://www.healthaffairs.org
- Hennessy, et al. (2025). Real-world data and real-world evidence in regulatory decision making: report summary from the CIOMS Working Group XIII. Pharmacoepidemiology and Drug Safety. https://onlinelibrary.wiley.com
- Smith, et al. (2025). US Food and Drug Administration’s Advancing Real-World Evidence Program: initial experience. Clinical Pharmacology & Therapeutics. https://ascpt.onlinelibrary.wiley.com
- Blonde, L., et al. (2018). Interpretation and impact of real-world clinical data for the practicing clinician. Advances in Therapy, 35(11), 1763–1774. https://link.springer.com
Continue reading: Analysis-Ready Real-World Evidence and Healthcare Data for Pharma | How Data Science Transforms Pharma: RWD, Claims Analytics & AI Modelling
Category: Real World Evidence Strategy & Analytics | Pharma Analysis & Research
