Medical evidence is evaluated by asking whether the research was designed and conducted in a way that produces trustworthy, relevant and sufficiently precise information for the question being considered. Evaluation includes examining the study design, population, intervention or exposure, comparator, outcomes, follow-up, statistical analysis, missing data, sources of bias and applicability. A result cannot be judged solely by whether it appears in a scientific journal or whether its statistical test reaches a conventional significance threshold. Researchers and clinicians need to determine how likely the findings are to reflect the underlying truth and how closely that truth applies to the patient or population of interest.
The first step is identifying whether the study design fits the question. Randomised controlled trials are generally well suited to estimating causal effects of interventions because random allocation can balance confounding factors. Cohort studies are valuable for prognosis, incidence and associations between exposures and outcomes. Case-control studies can be efficient for rare outcomes. Diagnostic accuracy studies require appropriate reference standards and patient populations. Systematic reviews are used to synthesise multiple studies. The methodological strength of a study is therefore context-dependent; a design that is excellent for one question may be inappropriate for another.
Risk of bias is a central consideration. Bias is systematic error that can distort an estimated effect. Selection bias can occur when the participants included differ systematically from the target population or when allocation is flawed. Performance bias can arise when knowledge of treatment affects care or behaviour. Detection bias can occur when outcome assessment differs between groups. Attrition bias can arise from differential loss to follow-up, while selective reporting can occur when investigators emphasise favourable outcomes and omit or minimise unfavourable ones. Modern appraisal tools assess these and other domains explicitly.
Confounding is another major concern, particularly in observational research. A confounder is a factor associated with both the exposure and outcome that can create or distort an apparent association. For example, an observed association between a treatment and outcome could partly reflect differences in disease severity, age or other characteristics between treated and untreated groups. Researchers can attempt to control confounding through randomisation, restriction, matching, stratification, regression modelling, propensity methods and other approaches, but statistical adjustment cannot necessarily eliminate all residual confounding.
Precision is assessed using statistical uncertainty, commonly through confidence intervals. A narrow confidence interval generally indicates a more precise estimate, while a wide interval indicates greater uncertainty. Sample size, outcome frequency and variability influence precision. Statistical significance and precision are not the same thing. A very large study can detect a tiny difference with high statistical confidence, while a clinically important effect in a small study may have a wide confidence interval. Evidence evaluation therefore considers the size and uncertainty of the effect rather than relying solely on a p-value.
External validity or applicability concerns whether findings can reasonably be transferred to the patients or circumstances under consideration. A trial may have excellent internal validity but exclude older adults, people with multiple diseases, pregnant people or those taking common medicines. Its results may therefore be less certain for those groups. Researchers consider differences in baseline risk, disease severity, treatment availability, adherence, healthcare settings and outcome definitions when judging applicability. Evidence can be methodologically strong yet still have limited relevance to a particular patient if the populations or circumstances differ substantially.
When multiple studies address the same question, evidence can be synthesised through systematic review and, where appropriate, meta-analysis. Reviewers assess the methodological limitations of included studies, consistency of results, directness to the question, precision and potential publication bias. The GRADE framework, widely used for assessing certainty of bodies of evidence, considers risk of bias, inconsistency, indirectness, imprecision and publication bias, with additional considerations possible for non-randomised evidence.
Good evidence evaluation also considers harms, absolute effects, competing outcomes and whether the result is clinically meaningful. Researchers distinguish relative risk from absolute risk because a large relative reduction can correspond to a small absolute benefit when baseline risk is low. They consider adverse effects and treatment burden alongside benefits rather than evaluating efficacy in isolation. Publication status, funding and conflicts of interest are also relevant to interpretation, although funding source alone does not prove that findings are biased. Overall, medical evidence is strongest when appropriate study designs converge on consistent findings, methods minimise bias, estimates are sufficiently precise, outcomes are clinically meaningful and the evidence is directly applicable to the decision at hand.
