The Member State Coordination Group on Health Technology Assessment (HTACG) guidance on the validity of clinical studies

The European Health Technology Assessment (EU HTA) guidance on Multiplicity of Hypothesis Testing (MHT) [1] resembles an algebra no-solution equation. The hypothetico-deductive method used in inferential statistics, which this guidance references, relies on conclusive reasoning. However, the guidance fails to provide a clear resolution, leaving the matter to individual Member States (MS) despite its critical impact on statistical precision, a key pillar of validity in clinical trials for comparative effectiveness assessments in Joint Clinical Assessments (JCAs).

This review of the scope of the European Health Technology Assessment (EU HTA)’s guidance on clinical trial validity in its randomized controlled trials (RCTs) highlights several key issues that undermine its practical application and effectiveness, including misconceptions, errors, and inconsistencies

The European Parliament, in its resolution of March 2nd, 2017, on European Union (EU) options for improving access to medicines, called on the Commission to propose legislation on a European system for health technology assessment (HTA) to harmonize transparent HTA criteria to assess the added therapeutic value and relative effectiveness of health technologies compared with the best available alternative (1). In December 2021, the EU-HTA R, formally known as ‘Regulation (EU) 2021/2282 of the European Parliament and of the Council of December 15th, 2021’ (2), on HTA and amending Directive 2011/24/EU (3) was adopted. It entered into force with implementation starting in January 2022 for oncology products and advanced therapeutic medicinal products (ATMP) in January 2028 for orphan designated products and in January 2030 for all products.

The practice of Health Technology Assessment (HTA) inherently involves moral, epistemological, and ontological commitments, which shape how assessments are conducted, interpreted, and used for decision-making

Health Technology Assessment (HTA) is shaped by moral (what makes a health technology desirable or acceptable, e.g., beneficence, justice), epistemological (how reliable knowledge is obtained and validated, e.g., evidence standards, uncertainty handling), and ontological (what effects and outcomes are considered real and relevant for assessment) commitments. The EU Joint Clinical Assessment (JCA), intended to harmonize relative effectiveness evaluation, embeds these commitments implicitly rather than explicitly, creating transparency and consistency issues. Critical concerns include: rigid reliance on the PICO framework privileging RCTs and quantifiable outcomes, proliferation of PICOs leading to infeasible evidence requirements, inadequate handling of multiplicity and post-hoc analyses, conflation of certainty and uncertainty, and exclusion of qualitative evidence and stakeholder input. Guidance documents, criticized for methodological weaknesses (e.g., outdated tools, poor external validity assessment, nominal p-values acceptance), further undermine epistemic robustness. These structural flaws risk producing assessments that are non-inclusive, reductionist, and epistemologically inconsistent. Future EU HTA frameworks should explicitly align on foundational commitments and integrate stakeholder perspectives to ensure transparency, scientific credibility, and ethical legitimacy.