FormalPara Key Points for Decision Makers

This study gives insight into the current available information given directly in HTA dossiers that widely differs across countries and highlights the consideration of other factors beyond clinical factors.

Stakeholder persuasion and unmet needs were the relative common factors for decision making across all HTA organizations.

1 Introduction

The overall expenditure for healthcare is increasing more than the total gross domestic product (GDP) of most countries globally. A report on public health spending by the World Health Organization (WHO) between 2000 and 2016 revealed an average annual rise of 6% and 4% in the total healthcare expenditure in low/middle and high-income countries, respectively [1]. The increasing economic burden of the healthcare sector can be mostly attributed to hospital care costs, physicians and clinical services and prescription drug costs [2, 3]. Cancer and hepatitis C are two therapeutic areas associated with the highest economic burden, as reported by the WHO [4, 5].

Health Technology Assessment (HTA) organizations can make decisions on financial implications of any intervention to the healthcare system of a country based on different economic analyses like budget impact analysis and cost-effectiveness analysis [6].

The decisions made by HTA organizations have a great impact on the therapeutic market. Based on the HTA recommendations, the patient market access to any therapeutic may increase or decrease. Pharmaceutical companies also observe the trends of HTA recommendations made in the past when designing and conducting clinical trials for drugs. This helps them in generating necessary evidence required for applying for reimbursement of their products [7].

Various studies have been conducted to explore the patterns of decision making by HTA organizations. Studies have highlighted disparities in HTA recommendations related to HTA processes, contextual aspects [8], methodological differences [9], levels of scientific evidence and patient evidence, levels of uncertainties [10] and willingness-to-pay thresholds [11]. Two studies, Dakin et al. [12] and Nicod [13], compared the characteristic elements and factors considered by HTA organizations for decision making. The assessment approach of the HTA organization of the United Kingdom (UK)—the National Institute for Health and Care Excellence (NICE)—was studied by Rawlins et al. [14]. They reported that the decisions of NICE are influenced by two types of judgements—scientific and social values. The former considers the clinical and economic evidence while the latter considers the severity of the underlying illness, end-of-life treatments, stakeholder persuasion, significant innovation, disadvantaged populations and children [14]. In recent years, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Special Task Force has suggested the inclusion of 12 potential elements of value in healthcare for broadening the evaluation items of HTA organizations into cost-effectiveness evaluation of new interventions [15].

There are only a few studies comparing the overall process of decision making in HTA organizations in developed countries, such as Japan, which introduced a pilot HTA programme in Japan in April 2016. The objective of the present study was to understand the differences in the process and factors involved in decision making for HTA organizations across eight countries—France, Germany, Italy, Spain, UK, Australia, Canada and Japan. Inclusion of the 12 potential elements of value (suggested by ISPOR) across the HTA organizations of these eight countries was also explored.

2 Methods

2.1 Selection of Drugs and Health Technology Assessment (HTA) Organizations

The targeted literature review (TLR) was focused on four target drugs for cancer (Opdivo® [nivolumab], Keytruda® [pembrolizumab], Kadcyla® [ado-trastuzumab emtansine], Ibrance® [palbociclib]) and five target drugs for hepatitis C (Daklinza® [daclatasvir], Sunvepra® [asunaprevir], Viekirax® [ombitasvir], Harvoni® [ledipasvir/sofosbuvir], Sovaldi® [sofosbuvir]) that were evaluated up until August 2019. The dossiers of these nine target drugs were compared across the following HTA organizations—Agencia de Evaluación de Tecnologías Sanitarias de Andalucía (AETSA; Spain), Agenzia Italiana del Farmaco (AIFA; Italy), Center for Outcomes Research and Economic Evaluation for Health/Central Social Insurance Medical Council (C2H/Chuikyo; Japan), Canadian Agency for Drugs and Technologies in Health/pan-Canadian Oncology Drug Review (CADTH/pCODR; Canada), Haute Autorité de Santé (HAS; France), Institute for Quality and Efficiency in Healthcare (IQWiG; Germany), NICE (UK) and Pharmaceutical Benefits Advisory Committee (PBAC; Australia). In Spain, there are seven autonomous regions with HTA agencies (Catalonia, Andalusia, the Basque Country, Valencia, Galicia, Navarre and the Canary Islands) that are responsible for their own healthcare systems [16]. AETSA was chosen for this review as Andalucia is the largest region in Spain with 8,405,294 inhabitants (July 2018) [17]. Descriptive information is provided in Supplemental Table S1 (see electronic supplementary material [ESM]).

Different types of assessments were included: original submission, extension of indication, resubmission and unavailable status. However, the publication status was labelled as completed for each. Cancer and hepatitis C drugs were selected due to their high economic burden [3, 4] and the aforementioned eight HTA organizations were selected due to similarities in their countries’ health expenditures and GDP per capita [18, 19].

2.2 Data Collection

The TLR was conducted from two sources—HTA accelerator (HTAA; IQVIA’s HTA dossier literature platform) and the evidence or dossiers available on the homepages of HTA organizations from the respective countries [20, 21]. The articles were selected by one researcher and cross-checked by another researcher to ensure article selection met the selection criteria.

These dossiers were evaluated for factors mentioned in Sect. 2.3. Factors found in the HTAA database were verified by a local person in charge of each country and verified through the homepage and dossiers of each target HTA organization. Additional factors were also collected from HTA dossiers by C.F., R.D.M., H.M. and R.M. (see Acknowledgements section). English versions of the dossier were used, if available, and otherwise Google translate was used to translate French, German, Italian and Spanish dossiers to English, as was done by Verghese et al. [22]. These additional factors were extracted by C.F. and H.M. and cross-checked by R.D.M and R.M. to ensure that the collected information adhered to the definitions provided in Sect. 2.3.

Each original, resubmission and extension of indication dossiers were evaluated individually. If multiple subgroups, such as cohorts or genotypes, were evaluated in one single dossier, each factor found was only counted once, meaning a factor is not necessarily reported for all listed subgroups. Furthermore, each factor was evaluated as being reported regardless of whether that factor was directly considered or involved in the decision-making process.

The details of the population/intervention/comparison/outcome/study design (PICOS) criteria and the inclusion/exclusion criteria for each dossier in the TLR are provided in Supplemental Table S2 and Supplemental Table S3, respectively (see ESM).

2.3 Variables

An initial list of 34 evaluation factors was created from studies conducted by Nicod [13], Rawlins et al. [14] and Lakdawalla et al. [15]. This list was then refined by removing duplicates and redefining some factors to arrive at our own list comprising 28 factors, which were grouped into the following four categories for data analysis: type of economic evaluation, clinical uncertainties and issues, consideration factors (consisting of disease considerations, population considerations and treatment considerations, which were identified separately) and additional elements of value (see Supplemental Table S2in the ESM for more detail). The economic evaluations were observed based on the economic analysis provided in the dossiers, such as incremental cost-effectiveness ratio (ICER) with quality-adjusted life-years (QALY), cost only or no analysis at all. The clinical factors were observed from two viewpoints—uncertainties and issues. Uncertainties are defined as unclear or insufficient clinical evidences that hindered the ability to obtain a solid judgement of the assessed technology reported by the HTA organizations. On the other hand, issues are defined as clinical factors that were expressed as being incorrect or problematic by the HTA organizations. They are identified by the exact and/or similar expressions in the dossier.

A detailed list of the 28 factors along with their definitions is provided in Supplemental Table S4 (see ESM).

2.4 Data Analysis

The data analysis was performed using R version 3.6.0 and the graphs were produced using R package ‘ggplot2’. The data collected from the HTAA database and HTA dossiers was transformed into categorical, nominal or dichotomous variables. Descriptive statistics were used to measure the frequencies of each element to compare across the eight HTA organizations. Bar charts were used in order to compare the number of cases per factor across the organizations.

For quantitatively understanding the association between the factors considered across various HTA organizations, corresponding analysis was used. In this analysis, categorical variables were explored using visual technique-based R packages, namely ‘FactoMineR’ and ‘factoextra’ [23, 24]. Pearson’s chi-square test was performed to determine the presence of any significant relationship between two categorical variables. In this test, if the null hypothesis was rejected, the categorical variables were assumed to be dependent while if it was not rejected, the categorical variables were assumed to be independent [25]. Eigenvalues (values corresponding to the amount of information retained by each dimension) were examined to determine the number of dimensions to finally plot the correspondence analysis, where each axis represented a dimension. The cumulative variance was obtained by adding all the eigenvalues.

For additional elements of value, HTA organization and the number/frequency of the elements of value for cancer and hepatitis C evaluations stratified by HTA organization in dossiers were compared using Fisher’s exact test with Holm’s correction, p < 0.05.

The data analyses were performed according to the four categories mentioned in Sect. 2.3.

3 Results

3.1 Dossiers Used in This Study

A total of 189 dossiers (original, resubmission, extension of indication and unknown) were included in the study (Table 1). Over a span of 7 years, 147 cancer-related and 42 hepatitis C-related dossiers were found in the target HTA organizations up to August 2019.

Table 1 Number of dossiers used in the study stratified by Health Technology Assessment organization and disease
Full size table

PBAC had the highest number of dossiers (n = 51), while C2H had the lowest (n = 7). This is due to the number of resubmissions and extensions of indication that vary by country. Resubmissions and extensions of indication account for 45% (n = 23) and 44% (n = 22), respectively, of total PBAC dossiers. More than 50% of dossiers by AIFA, CADTH/pCODR, HAS, IQWiG and NICE were extensions of indication for cancer drugs. Also, no dossiers for hepatitis C drugs were presented by AIFA.

Being a relatively new HTA organization (since 2018), the C2H evaluates multiple diseases in one assessment for a drug, unlike the other HTA organizations (e.g. a single assessment was done for the use of Opdivo in melanoma, lung cancer and renal cancer). Also, two drugs—Ibrance and Keytruda—were not included by the C2H, as the HTA results and details of these drugs were not publicly available.

Moreover, the gap between the dates of marketing authorizations approved by the European Medicines Agency (EMA) and decision dates by HTA organizations in European Union countries is shown in Supplemental Table S5 (see ESM).

3.2 Economic Evaluation Types

The economic evaluations considered by HTA organizations, stratified by disease, are shown in Fig. 1. Irrespective of disease, all dossiers by NICE, CADTH/pCODR and C2H used strictly ICER with QALY.

Fig. 1
figure 1

Percentages of economic evaluation types used by selected organizations and diseases. Economic evaluation types consisted of cost only, incremental cost-effectiveness ratio with quality-adjusted life-year and not available. The percentages were reported by selected organizations and diseases (cancer and hepatitis C). AETSA Agencia de Evaluación de Tecnologías Sanitarias de Andalucía, AIFA Agenzia Italiana del Farmaco, C2H Center for Outcomes Research and Economic Evaluation for Health, CADTH Canadian Agency for Drugs and Technologies in Health, HAS Haute Autorité de Santé, IQWiG Institute for Quality and Efficiency in Healthcare, NICE National Institute for Health and Care Excellence, PBAC Pharmaceutical Benefits Advisory Committee, pCODR pan-Canadian Oncology Drug Review

Full size image

For IQWIG, all cancer-related dossiers were evaluated using cost-only analysis. On the other hand, only half of the hepatitis C-related dossiers were evaluated using cost-only analysis and the other half had no mention of any economic evaluations. For HAS, about 55% of dossiers showed cost-effectiveness analysis using ICER with QALY while the remaining dossiers did not present any economic analysis.

For PBAC, about 85% of cancer-related dossiers presented cost-effectiveness analysis using ICER with QALY while the remaining were either cost-only analysis or no economic analysis at all. For about 75% of the hepatitis C-related dossiers, cost effectiveness was evaluated using ICER with QALY, while for the remaining, no economic analysis was presented.

AETSA and AIFA did not mention any economic evaluations in their dossiers.

3.3 Clinical Uncertainties and Issues

The overview of uncertainties and issues reported in cancer dossiers across HTA organizations is presented in Fig. 2. For cancer, the highest number of uncertainties observed in all HTA organizations were related to clinical benefit of the drug under consideration, with the percentage ranging from 27.3% (n = 6) for HAS to 73.9% (n = 17) for NICE. The second most common uncertainty reported in cancer dossiers pertained to comparators (percentages reported for uncertainties ranged from 13.0% [n = 3] for NICE to 26.1% [n = 6] for IQWiG). PBAC was the only HTA organization in which cancer dossiers reported uncertainties pertaining to the safety of drugs under consideration (n = 8; 18.6% of the submitted dossiers). Study design was reported as an uncertainty in three (13.6%) cancer dossiers by HAS and one (12.5%) cancer dossier by AIFA. Relatively few uncertainties were seen across all organizations for the remainder of the factors including population and sample. No data was found for C2H. Regarding the issues reported, a lot of variations were observed across all HTA organizations; for example, AETSA and C2H reported no study-design–related issues while AIFA reported issues in the study design for 50.0% (n = 4) of their submissions.

Fig. 2
figure 2

Percentages of cancer clinical uncertainties and issues mentioned by Health Technology Assessment (HTA) organization. Uncertainties are defined as unclear or insufficient clinical evidences that hindered the ability to obtain a solid judgement of the assessed technology reported by the HTA organizations. Issues are defined as clinical factors that were expressed as being incorrect or problematic by the HTA organizations. AETSA Agencia de Evaluación de Tecnologías Sanitarias de Andalucía, AIFA Agenzia Italiana del Farmaco, C2H Center for Outcomes Research and Economic Evaluation for Health, CADTH Canadian Agency for Drugs and Technologies in Health, HAS Haute Autorité de Santé, IQWiG Institute for Quality and Efficiency in Healthcare, NICE National Institute for Health and Care Excellence, PBAC Pharmaceutical Benefits Advisory Committee, pCODR pan-Canadian Oncology Drug Review

Full size image

The overview of uncertainties and issues reported in hepatitis C dossiers across HTA organizations is presented in Fig. 3. For hepatitis C, similar trends in uncertainties and issues were observed compared to cancer dossiers. The most common uncertainty reported was clinical benefit and a lot of variations were observed in the reported issues. However, due to the small sample size of hepatitis C dossiers included in this study, any inference from the present results must be made with caution.

Fig. 3
figure 3

Percentages of hepatitis C clinical uncertainties and issues mentioned by Health Technology Assessment (HTA) organization. Uncertainties are defined as unclear or insufficient clinical evidences that hindered the ability to obtain a solid judgement of the assessed technology reported by the HTA organizations. Issues are defined as clinical factors that were expressed as being incorrect or problematic by the HTA organizations. AETSA Agencia de Evaluación de Tecnologías Sanitarias de Andalucía, AIFA Agenzia Italiana del Farmaco, C2H Center for Outcomes Research and Economic Evaluation for Health, CADTH Canadian Agency for Drugs and Technologies in Health, HAS Haute Autorité de Santé, IQWiG Institute for Quality and Efficiency in Healthcare, NICE National Institute for Health and Care Excellence, PBAC Pharmaceutical Benefits Advisory Committee, pCODR pan-Canadian Oncology Drug Review

Full size image

3.4 Consideration Factors Reported by Each HTA Organization

The consideration factors analysed across the target HTA organizations are presented in Fig. 4. Overall, the non-availability of hepatitis C dossiers by AIFA; very limited information publicly released by AETSA, AIFA and C2H; and the low number of hepatitis C dossiers by NICE made it more difficult to observe trends for consideration factors across HTA organizations.

Fig. 4
figure 4

Consideration factors across Health Technology Assessment (HTA) organizations stratified by diseases (cancer and hepatitis C). Evaluated consideration factors were categorized as follows: disease considerations (Disease nature affecting the patient/Severity of disease, Rarity, orphan status, National priority and End-of-life), population considerations (Stakeholder persuasion, Disadvantaged population and Children) and treatment considerations (Unmet needs, Issues around current treatment alternative, Complex pathway—no best practice or advance, Innovation, Indirect benefit from the treatment and Adverse events manageable/non-significant). AEs man/non-sig. Manageable/nonsignificant adverse events, AETSA Agencia de Evaluación de Tecnologías Sanitarias de Andalucía, AIFA Agenzia Italiana del Farmaco, C2H Center for Outcomes Research and Economic Evaluation for Health, CADTH Canadian Agency for Drugs and Technologies in Health, Dis. Nature/Sev. Disease nature/Severity, Disadv. Pop. Disadvantaged population, HAS Haute Autorité de Santé, IQWiG Institute for Quality and Efficiency in Healthcare, Issues alt. treat. Issues around alternative treatment, NICE National Institute for Health and Care Excellence, PBAC Pharmaceutical Benefits Advisory Committee, pCODR pan-Canadian Oncology Drug Review, STKH persuasion Stakeholder persuasion

Full size image

3.4.1 Disease Considerations

Disease severity was reported by four organizations—HAS, NICE, PBAC, CADTH/pCODR. CADTH did not mention disease factors for hepatitis C while AETSA, AIFA and IQWiG did not mention disease factors at all. National priority was observed for hepatitis C at the highest rate (n = 5; 62.5%) by PBAC. NICE reported end-of-life for cancer at the highest rate (n = 20; 87.0%) of all organizations. The rare/orphan status disease factor in cancer was only presented by NICE, but at a low rate (n = 1; 4.3%).

3.4.2 Population Considerations

Amongst population factors, stakeholder persuasion was the most common factor reported across HTA organizations with a proportion of 75% or over. IQWiG and CADTH/pCODR mentioned stakeholder persuasion for 100% of the dossiers for both cancer and hepatitis C. However, HAS, AETSA, AIFA either included it at a low rate or did not mentioned it at all. NICE noted disadvantaged population factor for both hepatitis C drugs evaluated.

3.4.3 Treatment Considerations

NICE was the only organization that reported nearly all factors related to treatment for both cancer and hepatitis C evaluations. NICE was also the only HTA organization to mention the complex pathways of treatment in cancer, but at a low rate (n = 2; 8.7%). The treatment factor observed in the highest proportion for both diseases across multiple HTA organizations was unmet needs. Six out of eight and five out of eight organizations mentioned unmet needs for cancer and hepatitis C, respectively, at a high rate ranging from 54.5% (HAS, n = 12) to 100% (AIFA, n = 8). Another factor, manageable or non-significant adverse events (AEs), in cancer was also noted by six out of eight organizations, at rates ranging from 4.5% (HAS, n = 1) to 78.3% (IQWiG, n = 18). AIFA reported innovation and unmet needs for all cancer dossiers evaluated.

The correspondence analysis illustrating the relative association between HTA organizations and the factors considered for evaluating cancer dossiers is presented in Fig. 5. Test of independence showed a statistically significant association between HTA organizations and consideration factors (χ2=234.62, p < 0.0001). There was a 66.6% variance was accounted by the two-dimensional representations of the first two eigenvalues.

Fig. 5
figure 5

Correspondence analysis for cancer drug evaluation consideration factors. Test of independence showed statistically significant association between Health Technology Assessment organizations and consideration factors (χ2 = 234.62, p-value <0.0001). A variance of 66.6% (horizontal axis 45.0% and vertical axis 21.6%) was accounted for by the two-dimensional representations of the first two eigenvalues. As an example, National Institute for Health and Care Excellence (NICE) was in favour of additional factors such as alternative treatment and end-of-life and innovation, which in their plots are close together. AETSA Agencia de Evaluación de Tecnologías Sanitarias de Andalucía, AIFA Agenzia Italiana del Farmaco, C2H Center for Outcomes Research and Economic Evaluation for Health, HAS Haute Autorité de Santé, IQWiG Institute for Quality and Efficiency in Healthcare, NICE National Institute for Health and Care Excellence, PBAC Pharmaceutical Benefits Advisory Committee, pCODR pan-Canadian Oncology Drug Review

Full size image

HAS, PBAC, pCODR and NICE mentioned the maximum number of factors for their drug evaluations. The most common attributes observed across these organizations were unmet needs, stakeholder persuasion, disease nature affecting the patient and indirect benefit. NICE was distinguished from other organizations for also noting additional factors such as issues around alternative treatment and end-of-life. IQWiG, on the other hand, was uniquely associated with manageable or non-significant AEs. AETSA, AIFA and C2H reported fewer factors while evaluating drugs and in addition to their small sample size of dossiers, no confident remarks could be made.

3.5 Additional Elements of Value Suggested by ISPOR Special Task Force

The additional elements of value for healthcare suggested by the ISPOR Special Task Force are presented in Fig. 6. PBAC was found to be the only HTA organization currently mentioning the use of most of the additional elements suggested by ISPOR in its dossiers. The most common additional elements mentioned by PBAC were adherence-improving factors, fear of contagion, equity and scientific spillover. HAS, NICE, CADTH/pCODR and C2H were found to be using one or two additional value elements while AETSA, AIFA and IQWiG were using none. The Fisher’s test result can be found in Supplemental Table S6 (see ESM). The table shows the significant difference between IQWiG versus C2H and versus PBAC (p = 0.0016 and p = 0.0087, respectively, p < 0.05) and HAS versus C2H (p = 0.0272, p < 0.05).

Fig. 6
figure 6

Number/frequency of considered additional elements of value for cancer and hepatitis C evaluations stratified by Health Technology Assessment (HTA) organization. The number in the parentheses under each organization’s name describes the number of dossiers. AETSA Agencia de Evaluación de Tecnologías Sanitarias de Andalucía, AIFA Agenzia Italiana del Farmaco, C2H Center for Outcomes Research and Economic Evaluation for Health, CADTH Canadian Agency for Drugs and Technologies in Health, HAS Haute Autorité de Santé, IQWiG Institute for Quality and Efficiency in Healthcare, NICE National Institute for Health and Care Excellence, PBAC Pharmaceutical Benefits Advisory Committee, pCODR pan-Canadian Oncology Drug Review, Reduction of uncert. diag value of reduction of uncertainty due to a new diagnostic

Full size image

4 Discussion

With increasing economic burden in the healthcare sector [1], the role of HTA organizations becomes highly important for providing reimbursement relief [26]. Furthermore, the increasing contribution of HTA organizations in deciding the cost effectiveness of new interventions makes them even more significant [6]. There has been a high variability observed in the recommendation trends of HTA organizations across countries for the same intervention [10, 11, 27]. Hence, it becomes important for stakeholders, such as healthcare professionals, patients and pharmaceutical companies, to understand the methodologies and factors considered by HTA organizations for decision making.

The present study reviewed four types of factors for the decision making across eight HTA organizations—economic evaluation types, clinical uncertainties/issues, consideration factors and additional value elements.

It is important to note that the economic evaluations across HTA organizations are utilized for different stages of the HTA process. Cost-effectiveness analysis using ICER with QALY was evaluated by CADTH/pCODR, NICE and PBAC for the purpose of decision making around reimbursements. IQWiG also presented economic evaluations for the decision making of reimbursements using budget impact analysis for all cancer and 50% of the hepatitis C dossiers. On the other hand, HAS and C2H utilized cost effectiveness using ICER with QALY for price adjustments rather than for reimbursement decision making [28, 29]. Note that Japan only formally introduced HTA and, more specifically, cost-effectiveness analysis in April 2019, after initiating a pilot programme that started in April 2016 [30].

The clinical factors were evaluated in two areas—uncertainties and issues. We observed more uncertainties (approximately 80%) than issues (approximately 20%) for clinical benefits across all HTA organizations. The uncertainties and issues reported across organizations were variable and were broadly related to comparators, population, safety and sample size. Since these factors also indirectly influence the clinical benefit of an intervention, any uncertainties and issues pertaining to them would have automatically led to the higher rate of uncertainties and issues for clinical benefit. These findings were in line with the results of a study conducted by Nicod that observed high rates of uncertainties for clinical benefit and study design for NICE [13]. It is expected that clinical uncertainties and issues have an impact on the HTA decisions regarding reimbursements. In fact, in Japan, cost-effectiveness analysis of any intervention is only conducted if it has met all the criteria of clinical evidence. This might also be the reason for the absence of any clinical uncertainties by C2H since their main objective is to assess cost-effectiveness analyses [28].

Amongst consideration factors, stakeholder persuasion and unmet needs were the most common factors for decision making across all HTA organizations. Stakeholder persuasion is usually voiced by patients and/or healthcare professionals and HTA bodies can decide whether to accept it or not. Unmet needs are related to clinical efficiency, safety and patient demand. Given that the interventions included in this study were not children-specific or for a rare disease, it was expected that children and rare/orphan status factors would not be regarded in most HTA dossiers. For hepatitis C drug assessments, NICE discussed certain minority ethnic groups as disadvantaged populations. In addition, end-of-life was also mentioned by NICE given that they do give special consideration to this factor, specifically for cancer dossiers [31]. However, it is difficult to confidently conclude that disadvantaged population and end-of-life factors are unique to NICE, recognizing that other HTA organizations might equally be considering these factors, but they might simply not be stated in the HTA dossiers. As seen in Nicod [13], unmet needs is a highly observed factor. However, there are some differences in results as this study evaluates cancer and hepatitis C drugs rather than orphan drugs. For example, the rates for rare/orphan status and national priority would be more significant for orphan drugs.

For potential elements of value in healthcare, the ISPOR Special Task Force has suggested certain factors to broaden the considerations for decision making. Most of the suggested elements, such as fear of contagion (5 of 12), equity (5 of 6) and scientific spillovers (4 of 4) were being reported by PBAC only. PBAC mentions fear of contagion for hepatitis C treatments that have the potential to reduce disease transmission. It also considers equity for the wellbeing of the general public as the awareness of options for hepatitis C treatment increases. Scientific spillover was mentioned pertaining to the probable creation of new similar treatments for hepatitis C in the future.

In this study, the correspondence analysis was performed to display the relationship between HTA organization and consideration factor as a useful tool to clarify the relationship between categorical variables. However, correspondence analysis is not suitable for a hypothesis test. In the future, statistical analysis to reveal the relationship based on a hypothesis test is needed for better decision making.

There are other limitations of this research. This research could not determine included or absent factors because of missing information about the existence of the factors in HTA dossiers. Therefore, the factors were raised up as potential factors to be included in each HTA dossier. There is a reliance on Google translation of non-English dossiers, reflecting possible language misinterpretation. However, Ethan et al. (2012) showed that Google translation provides a certain degree of accuracy, with some caution required [32]. Due to the low sample size of hepatitis C dossiers, caution is recommended when interpreting the results. Also, although HTA organizations in countries that have similar health expenditure were selected to remove the heterogeneities of different approvals and reimbursements for the drugs investigated, and to correspond the findings from the documents in selected HTA organizations, a lot of uncertainties remained in our results. These uncertainties are associated with the public accessibility or transparency of the HTA process and result. Akehurst et al. [9] reported that there was a lack of transparency in the long duration process for some products. Epstein and Espín [33] compared the process and results of European HTA agencies and reported most organizations such as HAS, NICE and IQWIG are far from perfect in terms of transparency [9, 33].

This study employed a new approach, which compared the HTA process for drugs of interest across developed countries including Japan. This is because a pilot HTA programme was introduced in Japan in April 2016.

5 Conclusion

This study has provided a view into the additional factors that could be leveraged in broadening the knowledge of reimbursement processes. Although clinical factors play a predominant role in the decision to reimburse medicines, NICE and PBAC were found to be the HTA organizations with the most comprehensive list of additional criteria. Furthermore, CADTH/pCODR, HAS and IQWiG also showed a direction for decision making that extends beyond clinical evidence. If the decision-making process of HTA were clearly outlined with more public accessibility or transparency into the considered factors, there would be more transparency in HTA systems, leading to better understanding amongst stakeholders about decision making.