Research and analysis

Executive summary: Methodological options for estimating the causal impacts of the UK’s trade remedies on trade and economic performance

Published 2 August 2024

Executive Summary

Purpose of this report

1.1 This report aims to facilitate the Trade Remedies Authority’s (“TRA”) evaluation of its existing and future trade measures. It does this by investigating the strengths and weaknesses of the application of methodological tools to estimate the causal impacts of the UK’s trade remedy measures. These tools are investigated using case studies on trade patterns and the performance of the UK industries.

1.2 The report is not intended to provide commentary or evaluate existing trade remedy measures, but instead provide a toolkit for future policymakers to draw upon when undertaking their own evaluations. The case studies contained in the report are wholly illustrative – and in some cases based on synthetic data to help illustrate the analysis – meaning that they should not be read as providing an assessment of the impacts of the remedies described.

Summary of tools used in trade evaluation

1.3 The main focus of this report is investigating evaluation models which have been based on econometric methods, where differences pre-and post- a trade remedy are observed, whilst also taking into account what might have happened in the absence of the trade remedy (for example using some sort of ‘control’ group). This begins with a literature review of evaluation approaches undertaken within the academic literature and by policymakers. The key econometric methods considered in this report are:

• Gravity models: Gravity models form the seminal trade literature and the foundation to capture the conceptual determinants of bilateral trade. At their heart, they explain bilateral trade flows as a function of size of economies and the distance between economies, sector specific factors (e.g. technological innovation within industry) and trade remedy. When observations before and after a trade remedy are available, gravity models can be used to detect changes in trade flows as a result of trade remedies.

• Difference-in-differences (diff-in-diff): Diff-in-diff seeks to identify the impact of trade remedies by comparing actual outcomes (import volumes, firms profits and so forth) to those in a counterfactual scenario i.e. the scenario that would have occurred absent the trade remedy. In essence, the approach looks at the change in outcome (e.g. import volumes) before and after a trade remedy for a ‘treated’ group (i.e. a group of companies/sectors subject to a trade remedy). It then ‘nets off’ (subtracts) the change the same outcome (e.g. import volumes) for a ‘control’ group (i.e. a group not affected by the trade remedy; e.g. a group in an adjacent sector or the same sector in a different country). By seeking to take account of what would have happened absent the intervention (by looking at a ‘control’ group), the technique is intended to provide confidence that the identified impact is truly ‘causal’.

• Synthetic control (SCM): Akin to diff-in-diff, SCM seek to isolate causal impacts by focussing on a counterfactual scenario (i.e. absent the trade remedy). Unlike diff-in-diff, and as the name suggests, the SCM involves constructing an artificial counterfactual. In particular, SCM creates a counterfactual by weighting a number of unimpacted “units” that are combined to form a counterfactual. The units used often come from a wider pool of potential units, denoted the “donor pool”. In the context of trade remedies, this means predicting an outcome (e.g. import volumes) as if the trade remedy had not been imposed.

• Event studies: Event study methodologies can be used to estimate the impact of a specific change. They work similarly to SCM in that the counterfactual model is created to estimate the evolution of the variables of interest across time. Once this is complete, the event study method directly computes an “event day” impact, that is the difference between the estimated counterfactual and the observed variable on an event day. This methodology works well for single events or multiple discrete events. However, this methodology is not best placed to estimate contemporaneous factors, such as trade remedies.

• Bayesian methods (BCI): Bayesian econometrics is based on Bayes theorem, and posits an alternative interpretation method for results of empirical estimations. The methodology involves specifying a likelihood function which incorporates the probability of observing your data given the model specification and specifying so-called “priors”. These priors reflect beliefs of parameters before observing the data. This can be based on prior knowledge or one can allow for the data to dominate.

1.4 There are occasions when econometric methods are not suitable, or it is not possible to implement such approaches. For example, in instances where data is not available or time/resources are limited. In these circumstances, it may be appropriate to use simple empirical approaches such before-during-after analysis (where variables of interest are compared before and after the imposition of a trade remedy) or trend analysis (which investigates changes in variables over time and examines changes in trajectories and fluctuations).

1.5 In some circumstances it may not be possible to observe the response of variables of interest to the imposition of a trade remedy at all. For example, data may be completely absent, or the remedy may simply be to ‘do nothing’ (e.g. to retain an existing trade remedy). In this case, data-driven approaches may not be suitable (or possible) for evaluating trade remedies. In these circumstances, techniques based on economic theory (sometimes called simulation methods) may help shed light the impacts of trade remedies. Some of the some of the most widely deployed approaches are as follows include Computable General Equilibrium (CGE) models and microeconomic models (e.g. ‘new trade’ models essentially which essentially model how firms compete in the marketplace).

Evaluation framework

1.6 With the different evaluation tools and techniques in place, the question becomes: under what circumstances should the different approach be deployed? To allow for this question to be answered, a guiding framework for methodology selection has been developed: a key output of this research. The framework is not meant to be a prescriptive “must follow” approach but gives an initial indication of methodologies that one may wish to consider. The framework is given in two forms: a tabular format and a decision-tree format.

Tabular format

1.7 The tabular version of the framework proceeds through two steps: one that helps identify which methods are feasible; and a second that helps identify the appropriateness of different feasible methodologies (depending on data and other consideration factors).

1.8 Table 1 below identifies the main considerations for feasibility across the methodologies. The table identifies whether a methodology is feasible if the answer is “Yes” to the question (with red meaning “not viable”, amber meaning “potentially viable” and green meaning “viable”). For example, if there are no valid comparators, then diff-in-diff, synthetic control, Bayesian methods and event studies are all unlikely to be feasible. Likewise, if only pre-initiation data is available (and no post- trade remedy initiation data is available) then economic theory (simulation) methods may be the only option available.

Table 1: Consideration of factors that make econometric methodologies feasible

Notes: Dark grey indicates “Not Viable” if answer to question is “Yes”; light grey indicates “Potentially Viable” if answer to question is “Yes”; and white indicates “Viable” if answer to question is “Yes”.

Source: GT Analysis.

1.9 Once Table 1 has been used to identify feasible methodologies, further data specific factors should be considered when making a choice between methodologies. These considerations are indicated in Table 2 below. The colours illustrate the circumstances in which the methods may be most appropriately deployed. For example, Bayesian methods maybe the most appropriate approach if data is very noisy and contains structural breaks, interpreting the results of the analysis as genuinely causal is an important factor to consider and ample time is available. Conversely, if time is of the essence and there is less of a desire to test whether a trade remedy has truly caused an impact, then a before-during-after comparison may suffice.

Table 2: Consideration of factors that make a particular method appropriate, conditional on the method being feasible

Notes: White means that the methodology copes well, light grey means that the methodology can be adapted to accommodate the factor, and dark grey means that the methodology cannot be adapted with the relevant factor.

Source: GT Analysis

Decision tree format

1.10 The principles expressed in the above tables can be presented in the form of a decision tree: the following diagram how the factors in Table 1 and Table 2 should be navigated when the TRA is deciding on a suitable model. It expresses method selection as flow of thought; the method of choice is a terminal node that follows naturally from conditions that have been satisfied in previous nodes. It begins by considering fundamental features of the data and the prevailing context, and thereafter leads the user through more nuanced details such that the remaining set of viable models is more tailored to the situation at hand. The conclusions of the decision tree and those in Table 1 and Table 2 regarding model suitability in a given context are consistent as they are governed by the same principles.

1.11 Table 1, Table 2, and Figure 1 are not intended to be read prescriptively. It is not being proposed that a particular methodology be used with certainty and exclusivity, but rather, it is conditional on the necessary circumstances warranting it. It is to be read as general guidance for the contexts and constraints that make some methodologies more suitable for a given problem, and which methodologies may struggle with the data or context.

Figure 1: Decision tree for methodology selection

Illustrative case study applications

1.12 With the above framework for methodology selection in place, four case studies were undertaken intended to:

• illustrate how the above framework can be applied in practice to make choices between methods;
• show why the selection of the appropriate method is so important (and that selecting an inappropriate method can result in misleading results);
• illustrate some of the practical challenges that may be encountered in pursuing these approaches and how they can be overcome; and
• uncover important practical lessons-learned for evaluating the impact of trade remedies.

1.13 The case studies do not evaluate the true impact of the trade remedies in question. In fact, case study 3 and 4 even use synthetic data. Therefore, the focus should not be on the results of the models but rather on the process of selecting the appropriate evaluation method based on contextual factors and issues that may arise during impact estimation.

Illustrative Case Study 1 – AD0012 Aluminium Extrusions from the People’s Republic of China (PRC)

Context

1.14 This case study covers the application of ex-post counterfactual analysis to assess the impact of imposition of a new anti-dumping measure in the UK. The trade remedy is an anti-dumping measure applied to Aluminium Extrusions imported from the PRC. The TRA initiated an investigation into the matter on 21 June 2021 and later concluded in its provisional and final determinations, dated 17 August 2022 and 16 December 2022 respectively, that these goods were being dumped into the UK and this was causing injury to UK industry. The impact of the trade remedy on the import volumes of nine target commodities was assessed.

Analytical approach

1.15 The data used to assess the impact of the trade remedy was trade data at the 8-digit commodity-level obtained from UK Trade Info. Although the trade remedy was applied to commodities at the 10-digit level, the data that was available was at the 8-digit level (i.e. several 10-digit commodity codes fall into a single 8-digit commodity code). Therefore, the affected commodities were matched to their nearest 8-digit level commodity code and analysis was undertaken at this level. Following the methodology selection framework above – and noting the abundance of data and potential to construct a useful control group – the methodologies used to estimate the causal impact of the trade remedy were the Synthetic Control Method (i.e. SCM) and Bayesian methods (i.e. BCI).

Results

1.16 The results from the analysis are shown in the tables below.

Table 3: Estimated effect for each commodity using synthetic control method (SCM)

Notes: This table presents the results from using the SCM for each commodity. The first column shows the aggregate impact of the trade remedy on the volume of imports in kilotons (i.e. the sum of the impact of the trade remedy in each quarter). The second column shows the average impact as a proportion of the predicted imports for every quarter in the post-initiation period. The final column shows p-value associated with the average impact of the trade remedy. Values below 0.05 indicate that the average impact is statistically significant at the 5% significance level.

Source: GT analysis.

Table 4: Estimated effect for each commodity using Bayesian causal impact (BCI)

Notes: This table presents the results from using the BCI for each commodity. The first column shows the aggregate impact of the remedy on the volume of imports in kilotons. The second column shows the average impact as a proportion of the predicted imports for every quarter in the post-initiation period. The final column shows p-value associated with the average impact of the remedy. Values below 0.05 indicate that the average impact is statistically significant at the 5% significance level.

Source: GT analysis.

1.17 These models led to similar findings with respect to certain commodities: the trade remedy led to a notable and statistically significant reduction in the volume of imports. However, the findings of models diverged with respect to other commodities. In particular, BCI was able to estimate causal impacts more precisely and thus detected statistically significant trade remedy impacts for a greater number of commodities. This may be due to the ability of the BCI to generate more precise and less biased estimates of causal impacts when the underlying data is noisy.

Key lessons

1.18 Additional themes that were captured within this case study include:

• Counterintuitive trade remedy impacts: Evidence was found of the trade remedy having the potentially counterintuitive impact of increasing imports for some commodities that should have been made relatively more expensive as a result of the anti-dumping measure. This is likely due to imperfect coverage between the unit of analysis (i.e. the 8-digit level commodity codes) and the unit targeted by the measure (i.e. the 10-digit commodities). Notably, some the 10-digit commodities within the 8-digit classifications investigated were within the scope of the trade remedy while others were out of scope. Therefore, some of the estimates reported at the 8-digit level were a combination of impacts due to the trade remedy and import movements for commodities not targeted by the trade remedy. Import movements for non-targeted commodities may be due to (i) substitution between the targeted commodity and the non-targeted commodities; (ii) attempts by importers to circumvent the trade remedy or (iii) exogenous factors unrelated to the trade remedy. Further analysis of import behaviour at the 10-digit level is required to both empirically separate the true impact of the trade remedy from external factors, and to understand the mechanisms driving the imports of non-targeted commodities within the same 8-digit level commodity code as targeted commodities.

• Anticipation effects: Evidence was found of importers anticipating the trade remedy and adjusting their importing behaviour in response to the trade remedy prior to the initiation date. The initiation of a similar EU trade remedy investigation prior to the initiation date of the corresponding trade remedy investigation of the UK may have led importers to pre-empt the trade remedy taking effect in the UK. It is suggested that the initiation date is back dated within the analysis in order to capture these anticipation effects within the causal impact estimates. The approach taken by the evaluator to uncover causal impacts would be different if the pre-initiation effects were found to be due to exogenous factors unrelated to the trade remedy.

• Comparator commodities poorly predicting observable imports: Results showed that the BCI was unable to accurately predict observed imports during the pre-initiation period due to poor explanatory power of the comparator commodities. Including variables that may explain some of the variation of in import volumes (e.g. exchange rates or more qualitatively similar commodities) could reduce the noise in the model and generate more precise estimates of the trade remedy impact.

• Substitution and complementary effects: Comparator commodities that may have been affected by the trade remedy because they are substitutes of or complements for the target commodity should be removed from the sample. Efforts were made to minimise the number of commodities that were affected by spillovers, but there is evidence to suggest that not all such commodities were. The conceptual and quantitative measures to minimise the number of comparator commodities that may be impacted by spillovers and the recommended process is detailed in Appendix 1: Case Study 1: AD0012 Aluminium Extrusions from the PRC.

Illustrative Case Study 2 – TD0014 Heavy Plate from the PRC

Context

1.19 This case study illustrates how models from economic theory can be used to explore the impact of trade remedies when there is no observable counterfactual from real-world data (e.g. because there has been no recent change in remedy or data is not available).

1.20 To illustrate the application of these models, the case study considers duties imposed by the UK on Heavy Plate from the PRC. An import tariff of around 70% has been imposed on these imports since 2017. Importantly, TRA’s conclusion through its transition review was not to change the scope/form/level of the measure, meaning that it is not possible to observe changes in demand, market share and so forth. This means that econometric approaches to evaluation are not suitable. More information about the case is available in the TRA’s recommendation to the Secretary of State (TRA, 2023). The case study focuses on the impact on market share but other variables such as profits, consumer welfare and so forth can also be explored through these models.

Analytical approach

1.21 The case study uses a variant of the models developed by authors including Brander, Spencer and Krugman in the early 1980s (part of the ‘New Trade Theory’ literature). Essentially, a Cournot oligopoly model (adapted to capture international trade and tariffs) is constructed to simulate how firms compete in the domestic market. This model is then ‘calibrated’ with real-world information (for example about existing prices, firms’ costs and market shares) to explore the impact of trade remedies (in particular, the effect of retaining an import tariff).

Results

1.22 The relationship between the import tariff and the market share of the domestic firm can be plotted, by calibrating the model with real world data. As the import tariff rises it increases the foreign firm’s effective costs in the domestic market, giving a competitive advantage to the domestic firm. The domestic firm’s market share rises and the foreign firm’s market share falls (Figure 2).

Figure 2: Impact of an import tariff

Source: GT Analysis.

1.23 This information can be used to derive an illustration of the impact of retaining the import tariff in this setting. The import tariff being applied is circa 70%, resulting in a domestic firm market share of around 67% (as per the TRA findings). The model can then be used to construct a counterfactual – and estimate the market share – for other tariff levels to help evaluate the impact of the import tariff. For example:

• in a counterfactual where a 50% tariff was applied (for example if 50% was the most-favoured nation (MFN) tariff), the model would imply a domestic market share of 40%. This would imply that, compared to that counterfactual, the impact of retaining the import tariff is to uphold the market share of the domestic firm by around 27 percentage points.

• in a counterfactual where the import tariff is removed altogether, the implied domestic market share would fall (a difference of almost 50 percentage points). In other words, according to the model, retaining the import tariff results in around half of the market purchasing from the domestic producer that would otherwise procure from the foreign supplier.

Key lessons

1.24 Models from economic theory can help give useful indications of the impacts of trade remedies, and illustrate the economic mechanisms through which trade remedy affects variables of interest. They may be especially useful where data is not available to construct a counterfactual scenario. However, the models are inevitably oversimplifications of real-world dynamics. In addition, different theoretical models serve different purposes and calibrating them is fraught with uncertainty. As a result, they may be best thought of tools to help illustrate impacts and give an indication of possible size of effects in the context of significant uncertainty (rather than techniques capable of providing precise estimates of trade remedy impacts).

Illustrative Case Study 3 – TD0004 and TS0005 Biodiesel from the United States and Canada

Context

1.25 This case study pertained to the UK changing the scope of two trade remedies imposed by the European Commission: an anti-dumping measure (i.e. TD0004) and a countervailing measure (i.e. TS0005). The initial measures began in July 2009 and imposed anti-dumping and countervailing duties on imports of biodiesel originating in the US and consigned from Canada. On 11 August 2020, the TRA initiated a transition review of the original EU measure to assess whether the measure should be varied or revoked in the UK. The recommendation revoked the measure in relation to HVO biodiesel, but it maintained the measure with relation to FAME biodiesel, and this change in scope was instated from 30 January 2021.

Analytical approach

1.26 This case study uses synthetic firm-level data as real-world firm-level data was not available due to confidentiality issues. However, in collaboration with the TRA, synthetic firm-level data was create, resembling the data that the TRA would have access to when it undertakes its investigations. The synthetic data attempted to mirror this real-world data by containing similar variables and a realistic level of granularity and frequency. The synthetic data was a quarterly sample spanning from 2010 to 2023, contained firm-level variables such as costs, sales, revenues and profit, and consisted of one firm that was exposed to the trade remedy and, depending on the specification, between 10-30 comparator firms that were not exposed to the trade remedy.

1.27 The impacts of the trade remedy were estimated using the SCM and BCI (following the methodology selection framework above). However, these impacts were known beforehand as they were constructed as part of the data. The performance of the methodology was assessed by measuring how close the estimates were to the true impact of the trade remedy.

Results

1.28 The baseline case investigates the impact of the trade remedy with 10 firms, assuming the effect of the trade remedy was constant over time and the underlying volatility in the data was relatively low. The results from estimating the SCM and BCI on this data are summarised in the tables below. The reported results are the true and estimated aggregate impact of the trade remedy in the entire post-initiation period and the average proportional impact of the trade remedy.

Table 5: SCM results with constant trade remedy impact and low volatility

Notes: This table shows the impact of the trade remedy on the synthetic firm assuming that the impact of the trade remedy reduces cost by 10% and is constant over time. The columns indicating the aggregate effect show the true and estimated aggregate impacts of the trade remedy (i.e. the sum of the impact of the trade remedy in each quarter). The columns indicating the proportionate impact of the trade remedy show the impact of the trade remedy as a percentage across each quarter in the post-initiation period. The p-value represents the statistical precision of the estimate produced by the model. Values below 0.05 indicate that the estimate of the average impact of the trade remedy is statistically significant, whereas numbers above 0.05 indicate that the estimate of the average impact is not statistically different from 0.

Source: GT Analysis.

Table 6: BCI results with constant trade remedy impact and low volatility

Notes: This table shows the impact of the trade remedy on the synthetic firm assuming that the impact of the trade remedy reduces cost by 10% and is constant over time. The columns indicating the aggregate effect show the true and estimated aggregate impacts of the trade remedy (i.e. the sum of the impact of the trade remedy in each quarter). The columns indicating the proportionate impact of the trade remedy show the impact of the trade remedy as a percentage across each quarter in the post-initiation period. The p-value represents the statistical precision of the estimate produced by the model. Values below 0.05 indicate that the estimate of the average impact of the trade remedy is statistically significant, whereas numbers above 0.05 indicate that the estimate of the average impact is not statistically different from 0.

Source: GT Analysis.

Key lessons

1.29 The SCM and BCI to were implemented on synthetic firm-level under varying scenarios. The scenarios investigated included: (i) low versus high data volatility; (ii) a constant trade remedy impact versus a time-varying trade remedy impact; (iii) 10 comparator firms vs 30 comparator firms; and (iv) a sample consisting of some unsuitable comparators versus a sample consisting of no unsuitable comparator. The results showed that the best performing model in terms of both accuracy and precision across the board was the BCI, and that a sample with no unsuitable comparators yielded the best results. This finding was robust to all the modifications made to the underlying data. This result is likely due to the ability of the BCI to generate more precise estimates of causal impacts despite the underlying data being noisy or the true impact of the trade remedy being small and time-varying.

1.30 This case study also underscored the importance of removing unsuitable comparators from the sample before undertaking causal analysis. Keeping unsuitable comparators in the sample generates bias that may even cause models that are highly robust to noise (i.e. the BCI) to perform poorly, especially as the data becomes noisier, the number of comparators changes and as the impact of the trade remedy becomes more complex.

Illustrative Case Study 4 – TS0023 Stainless Steel Bars and Rods from India

Context

1.31 This case study is based on a trade remedy initially imposed by the EU on Stainless Steel Bars and Rods originating in India, which was revoked after a TRA transition review. The case study illustrates an approach to evaluating trade remedies when data is limited and explores some of their key challenges.

Analytical approach

1.32 Since the case study data is limited to just a small number of periods, the methododology selection framework above reveals that some of the more advanced techniques (e.g. SCM and BCI) are unlikely to be suitable. Nevertheless, potential data on comparators does exist, so a difference-in-differences (“diff-in-diff”) estimation is employed. As with Case Study 3, a synthetic data set was constructed for sales (since it was not possible for TRA to share data owing to confidentiality considerations).

1.33 Diff-in-diff estimators rely on the parallel trends assumption. This assumption states that the pre-trade remedy trends of the comparator and treated firm are parallel. This assumption is necessary to interpret causal impacts from a diff-in-diff estimation.

Results

1.34 The results for this case study are estimated in three scenarios:

• Parallel trend;
• Parallel trend with noise; and
• Non-parallel trend.

1.35 Data was constructed to create an actual drop in sales of £10,500.

1.36 The estimation results for the parallel trend estimation are presented below.

Table 7: Estimation results for the parallel trends analysis

Notes: Significance codes: p<0.001; p<0.01; p<0.05

Standard errors are reported in parentheses.

Source: GT Analysis.

1.37 When the pre-trade remedy trends are parallel, estimation yields a close to accurate estimation of the fall in sales. When noise is added into the estimation, the results of the estimation are further from the actual drop. The results from the noisier estimation are presented below.

Table 8: Estimation results for diff-in-diff with a parallel trend and noise

Notes: Significance codes: p<0.001; p<0.01; p<0.05

Standard errors are reported in parentheses.

Source: GT Analysis.

1.38 The results identify a fall in sales, but of £10,630 rather than £10,500. This occurs as the level of noise is comparable to the underlying trend, and with only three observations, it is more difficult to identify the underlying trend. Significant noise will interfere with identifying an underlying trend, unless the frequency or quantity of observations is large enough to ascertain a trend despite the noise.

1.39 The analysis is also conducted for a comparator firm that does not have a pre- trade remedy parallel trend. The results of this estimation are presented below.

Table 9: Diff-in-diff estimation with non-parallel trends

Notes: Significance codes: p<0.001; p<0.01; p<0.05

Standard errors are reported in parentheses.

Source: GT Analysis.

1.40 This estimation results in an estimated fall in sales of £15,590. This is significantly larger than the actual drop of £10,500. This highlights the importance of the parallel trends assumption holding.

Key lessons

• Whilst diff-in-diff estimations require minimal data, there are still strict criteria that must be met for the interpretation of diff-in-diff to be causal.
• If there is significant noise, the trend may be more difficult to identify. This can be solved by obtaining more data, or more frequent data.
• If there is no appropriate comparator or the parallel trend assumption does not hold, then alternative methods should be undertaken, or further data should be collected.

Key lessons and recommendations

1.41 The key aim of this report was to develop a framework for evaluation methodology selection that the TRA can use to select suitable approaches for the causal analysis of trade remedies in a way that is tailored to the various contexts that the TRA are likely to encounter. The methodology selection framework presented above (and detailed in Section 6) was developed with this aim in mind. This framework delivers the TRA with more than just a toolkit for causal analysis; it provides the TRA with a guide for selecting the right tools for the right task.

1.42 The case studies illustrate how this framework can be applied to four scenarios to which the TRA applies trade remedies. Each case study was unique in terms of the context of the trade remedy that was applied, the nature of the counterfactual problem implied by the trade remedy and the data that was available for analysis. In each case, the framework was used to guide the selection of the methodology. When comparison of multiple methodologies was undertaken, the methodology that was deemed most suitable by the framework delivered the best results.

1.43 The high-level take-away from these results is that the framework can be relied upon to choose causal approaches, from a suite of feasible methodologies, that are tailored to the nuances of the context. However, the framework should not be read prescriptively; the framework and its proposed methodologies should not be followed blindly and judgement is necessary. Further, the framework should not be interpreted as proposing a particular method to be used with certainty and exclusivity. Specifically:

• Methodologies within the framework have assumptions that must be met for causal impacts of trade remedies to be identified; and
• If these assumptions are not met, other methodologies should be considered.

1.44 The framework is intended to serve as general guidance for the contexts and constraints that make some methodologies more suitable and others less so.

1.45 In practice, conditional on a set of methods being feasible, it would be ideal to apply all of them and use the findings of each method to triangulate the most credible answer. This can be done by implementing multiple methodologies, where feasible, and then:

• Assessing the degree of similarity in their findings;
• Identifying potential factors that drive differences in results, should divergences arise;
• Evaluating the importance of each finding based on the merits and limitations of each approach in light of the context;
• Being transparent with any assumptions that have been made due to limited evidence;
• Being transparent with all shortcomings in the methodologies considered and caveat results accordingly; and
• Coming to a reasonable and well-balanced conclusion.

1.46 The fourth point is particularly important when the TRA is led by the framework to use approaches that are less capable of producing causal estimates but are the most suitable due to time and complexity constraints. The framework has been endowed with the flexibility of proposing approaches that the TRA can use to deliver expedient results when there is a shortage of time, data or expertise. However, these approaches tend to be less able to produce causal findings when compared to more robust yet resource intensive alternatives. The TRA may wish to ensure that all results are caveated appropriately when endeavouring to use the framework to deliver results at pace. Furthermore, in cases where the TRA relies of purely theoretical approaches, they could view the findings of these approaches as illustrative rather than precise and causal.

Data collection considerations

1.47 Situations may arise when methodologies that would be most capable of delivering causal estimates will be infeasible due to data constraints, and the TRA may be limited to second-best alternatives that produce less reliable and robust results. The TRA faces many data constraints that can be inflexible in the short-term, particularly with respect to firm-level data. The TRA’s database of firm-level information has historically depended on the voluntary data contributions of domestic importers, domestic like-good producers, and firms in the wider supply chain of domestic producers and importers. The nature of the data that TRA acquires for investigations is sparse, and usually consists of a small number of firms with limited time periods. This heavily limits the methodologies that the TRA can feasibly apply when investigating trade remedies; data-hungry approaches such as the SCM and BCI would likely unworkable.

1.48 The following recommendations could potentially help the TRA enhance their data capabilities:

• Collect more granular data or for longer time periods: increasing the number of time periods that can be assessed will create opportunities for the TRA to implement a greater number of robust approaches for causal inference. This can be done either by collecting data at a higher frequency (e.g. at the monthly or quarterly level) or by incorporating a greater number of pre-trade remedy years into the sample. It may be possible for the TRA to encourage participation of firms to cooperate with greater data demands by stressing the importance of data in coming to sound determinations;
• Encouraging firms to participate: Whilst compelling firms to participate may not be possible, strong encouragement of participation could lead to better insights from the data. However, there may be an argument for this in scenarios where the trade remedy in under review is likely to have widespread and significant effects on UK industry and the wider economy. In such cases, making correct deductions about the impact of a trade remedy can have large economic ramifications on the UK and it would allow the TRA to give make the best approaches feasible by having ample data; and
• Make the most of what is available: if the TRA were able to harmonise all the data that has been collected from previous investigations into a single database, it may be able to leverage this information in future investigations. This may allow the TRA to increase the number of time periods and comparators contained in future data samples by appending relevant information from prior investigations. In theory, this could expand the data available to the TRA in the long-term without jeopardizing the participation of its questionnaires’ respondents in the short-term. For this, the TRA would need to ensure that it adheres to any data storage and utilisation terms within the data provision agreements it made with respondents from past investigations.

Practical considerations for causal analysis

1.49 The framework provides the TRA with guidance for how to select feasible and suitable methodological approaches for the causal assessment of trade remedies in subject to various contexts and constraints. However, there are many considerations that the TRA must bear in mind when practically undertaking causal analysis with these approaches in order to meaningfully interpret their findings. These considerations include many of the key lessons gleaned from the case studies within this report but also go beyond them. They include:

• The unit of treatment and unit of analysis: in general, there can be a divergence between how the trade remedy is implemented practice and how this is captured in the data. For instance, the trade remedies are imposed at the 10-digit commodity code level, whilst analysis was undertaken at the 8-digit level. This resulted in counterintuitive findings for commodities that had imperfect overlap between these two levels of classification with respect to trade remedy exposure. Such divergences must be identified and addressed, or the results must be caveated accordingly;
• Pre-initiation effects: effects of a trade remedy that materialise prior to the initiation of the trade remedy investigation may be indicative of factors that must be accounted for in causal analysis;
• Spillover effects: these may arise when commodities that are out of scope for a trade remedy are impacted due to substitution effects, complementary effects or other effects connected to the trade remedy. Failing to account for spillover effects may result in the estimated impact of the trade remedy being heavily biased;
• Volatility matters: the underlying volatility of the data has implications for the performance of the many of the methodologies considered within this report. Numerous ways to mitigate the amount of noise in the data were proposed and executed in this report; and
• Sample selection: the TRA gathers its firm-level data on a volunteer basis. This creates sample selection issues of varying degrees. Sample selection occurs when the sample is not representative of the general market or population of interest. In its most extreme form, sample selection can render ex-post counterfactual analysis infeasible because there are no comparators for the firms that are affected by the trade remedy. The only recourse in such a situation would be to rely on empirical or theoretical methods that do not rely on a counterfactual. Less extreme forms of sample selection can be overcome by implementing approaches that are robust to sample selection (these are beyond the scope of this report) or caveating the findings and resulting conclusions appropriately.