With the major role that obtaining evidence now plays in research, we should be able to generate theories that correspond well with the real world. Progress has been made in this direction in recent decades, but there is still some way to go. This enterprise has two aspects: the negative one of identifying which parts of traditional economics should be jettisoned, and the positive one of building theory from observations, generalisations, and explanations of observed phenomena.

In this article, I examine a variant of the negative aspect, in which evidence has been used, but the conclusions reached have been dominated by a default way of thinking derived from old-fashioned schematic notions about the way the world works. By exploring the case of international free trade agreements, I argue that a complicated multi-causal reality has been portrayed in over-simple terms, and this has resulted in wildly inaccurate predictions. I focus not on the more abstract modelling approaches, but rather on a methodology that incorporates evidence into the analysis. This is to illustrate the point that the mere fact that one uses evidence is not sufficient to generate an account that corresponds well with the real world, as long as gaps in understanding are filled by naïve extrapolations.

International free trade agreements are ubiquitous. Canada and the EU have recently negotiated the Comprehensive Economic and Trade Agreement (CETA). Twelve countries (not including China) are involved in negotiating a Trans-Pacific Partnership (TPP). Discussions are underway for a potential Transatlantic Trade and Investment Partnership (TTIP). Numerous smaller ones are in the works.

How much do we know about the consequences of these agreements? Even if they increase trade, what then follows in terms of employment, wage levels, etc? I will argue here that our knowledge is seriously deficient. The case presented is not about the merits or demerits of such treaties, but rather whether or not we have sufficient evidence to make an informed judgement about those merits. This is not the same as the more general issue of openness to international trade; a national policy of outward orientation or export-based growth may have different consequences from a free trade agreement between countries.

The central problem is the existence of other major economic forces. Elegant models can be constructed, with a ceteris paribus assumption. But their predictions need to be evaluated following implementation, and this can only be done successfully if it is possible to evaluate the attributable consequences, teased out from all the economic events that followed implementation.

NAFTA and its consequences
There is a large literature on this topic. I focus here on the North Atlantic Free Trade Association (NAFTA), which came into force on 1 January 1994. The advantages are that it was of a size that would be expected to have large effects; that there has been sufficient time to evaluate these; and that NAFTA has been thoroughly studied, in terms both of predictions beforehand and of subsequent events. In particular I concentrate on Mexico, because Canada had an earlier treaty with the US that would have had larger consequences, and the effects on America were relatively small because of the size of its economy relative to the other participants.

Predictions based on a variety of modelling approaches and assumptions were published in the early 1990s. Probably the most cited of these is the thorough 1992 study by Hufbauer and Schott, which used a ‘historical’ approach.³ This report is closely argued, based on detailed analysis of a wide range of evidence, including a seven-volume World Bank report. The authors state that their perspective is ‘responsibly optimistic’, so they do not claim to be entirely non-partisan. Finally, it has the advantage that the same authors revisited the topic in 2005.⁴ Accordingly, I will compare their 1992 and 2005 reports in order to ascertain how closely their predictions were realised in reality. Whilst I do not claim that such a comparison is representative of the state of the art of this type of prediction — no doubt it is possible to find examples of accurate prediction in this very large literature — it does illustrate the main points.

In 1992, Hufbauer and Schott reviewed seven studies, including their own. All predicted a rise both in Mexico-to-US exports and in US-to-Mexico exports, with a wide range of values. Three predicted an increase in Mexican exports to the rest of the world, and three predicted a decrease. Predictions of the Mexican trade balance varied from -12 to +4 billion USD. Three studies predicted that Mexican employment would rise, two predicted no change, and one predicted a fall. Of the five studies that had predictions for the Mexican wage rate, four predicted a rise and one predicted no change.⁵ Thus, taking these reports as a group, there was considerable uncertainty over the likely impacts of a future NAFTA, although admittedly some of this could be attributed to uncertainty over the precise content that NAFTA would have at the time they were writing.

For brevity, I will focus just on the study by Hufbauer and Schott themselves. Their main predictions for Mexico were:

• the current account deficit would rise, allowing much-needed imports to flow in;

• exports would rise, therefore so would tradable sector employment (estimated as 609,000 jobs);

• it would be the ‘beginning of the end for the maquiladora program … the raison d’être for most maquiladoras will soon fade away’ (p. 334);⁶

• increased competition would stimulate higher productivity and international competitiveness — ‘the most important objective of the NAFTA’ (p. 4);

• the wage level would remain unchanged;

• the peso would appreciate by 29 per cent due to increased capital inflows, reducing the cost of living.

As it turned out, the economic trajectory of Mexico immediately after NAFTA implementation was dominated by the peso crisis of December 1994, in which the peso depreciated from 3.4 to 7.2 per US dollar, rather than appreciating as predicted, partially recovering to 5.8 four months later. The crisis was triggered by political events, but had economic roots — a current account deficit, not considered in the analysis by Hufbauer and Schott, that could no longer be funded.⁷

Following the peso crisis,⁸ real wages fell 22 per cent by 1997, largely recovering by 2003. In contrast, productivity grew (except during 1996-2001), continuing a pre-NAFTA trend — although it was judged disappointing, except in some specific industries such as autos. Employment in maquiladoras doubled to over a million. The overall rise in employment was over 8 million, mainly in services. The current account deficit fell in 1995, at the expense of a 6.9 per cent fall in GDP, and remained small by pre-NAFTA standards thereafter.

Clearly these changes are not necessarily attributable to NAFTA. That is the point: other economic forces are likely to be more powerful than free trade agreements. The most important of these were:

• movements in the exchange rate, especially the peso crisis of 1994-95;
• other liberalization initiatives, e.g. the earlier switch from ISI to an export-oriented model;
• rural/agricultural policies, largely aimed at shrinking unproductive small-scale agriculture;
• the Progresa programme (see below);
• the rise of China, making it more difficult to compete internationally;
• oil prices;
• the US trade cycle and growth, which strongly affect the Mexican economy.

But neither are the actually-occurring changes compatible with Hufbauer and Schott’s predictions of NAFTA’s effects. For the exchange rate, current account and maquiladora employment, the actual trend was opposite to that predicted. Wages dropped sharply rather than staying constant. Although competitiveness — the main justification for NAFTA — increased in a few sectors, it was by far less than predicted. Employment increased by a great deal more, but in services and maquiladoras, not in non-maquiladora export industries. One could do better tossing a coin.

A central problem in the evaluation of free trade agreements is that ideology obscures reality. The 1995-96 fall in wages was blamed on NAFTA by its opponents. The post-crisis growth in exports and thus in employment was attributed to NAFTA by its enthusiasts — but occurred even more strongly in the maquiladora sector (unaffected by NAFTA) than elsewhere. The first is explicable by the December 1994 crisis when the peso crashed, which was not closely linked with NAFTA, and the second by the knock-on effects of this devaluation — which explains why the maquiladora sector expanded rather than withering away. Growth in services may have been aided by the lower wages. Yet the major player here, the exchange rate, did not feature in the detailed analyses of the Mexican economy that were carried out in preparation for NAFTA, including the one by Hufbauer and Schott. Similar remarks apply to GDP growth, which averaged 2.9 per cent in 1994-2004 — far short of converging towards the US.

Even the most proximal of effects of the agreement, an increase in trade flows, was less affected by NAFTA than the raw figures suggest, because the trade flows were largely accounted for by the rise in maquiladoras.⁹ Mexico-to-US exports rose from 40.4 billion US dollars in 1993 to 139.0 in 2003, when maquiladora output was 77.5; US-to-Mexico exports rose from 41.5 to 97.2 when maquiladora input was 59.1. Because maquiladoras have no linkages elsewhere in the economy, they cannot be expected to have the normal positive externalities — but their inputs and outputs are still counted as imports and exports in the statistics.

Investment largely shifted from portfolio investment to FDI, as predicted. However, as Hufbauer and Schott say, ‘Contrary to the expectation that foreign investment would be concentrated in the lowest-skilled activities, the principal impact of FDI in manufacturing was to raise the demand for semi-skilled workers and the wage premium paid to them’. In fact, FDI mostly flowed into existing assets, not into increasing production.¹⁰

Other issues feature in the 2005 report that were not prominent in the earlier one. There was a sharp contraction in the number of small and medium sized enterprises, consistent with increased exposure to competitive pressure. There were falls in poverty and inequality – likely to have resulted mainly from the increase in employment following the peso devaluation, and from the Progresa programme of conditional cash payments.¹¹ Another unexpected development was related to dispute settlement: ‘In practice, however, the rules … have fostered litigation by business firms against a broader range of government activity than originally envisaged’. Thus, concentration in enterprise ownership, changes in poverty/inequality, and the extent of dispute settlement, as well as the devaluation, were not even included in the list of predicted effects. And as we have seen, that list of predictions did not turn out to correspond with what actually happened post-NAFTA.

Conclusions from this cautionary tale
What can we learn from this? We know from the rise of East Asia that trade openness can lead to increased competition, and thence to productivity growth. But it does not necessarily have this effect, and introducing a free trade agreement does not turn a relatively unproductive manufacturing sector into a world-beating one. Indeed, exposure to intensified international competition can be detrimental. More broadly, even if trade expansion does occur as a result of such an agreement, its consequences for employment, wage levels, etc cannot be deduced in a simple way.

These conclusions may seem obvious. In any case, the original Hufbauer and Schott report was over twenty years ago — surely things are better now? Unfortunately not; the same approach is still being taken in the current attempts to predict the consequences of TTIP. A good example is a study that combines a sophisticated general equilibrium model with econometric estimates based on the ideal datasets, in order to predict future trade flows.¹² The link from there to real GDP is then taken as automatic, reminiscent of the Hufbauer and Schott attempt to predict employment changes by doing a simple calculation based on changes in imports and exports. And the predictions ignore wider forces that will likely prove more important than TTIP itself, just as Hufbauer and Schott did with NAFTA. The world is just not that straightforward; the consequences of trade growth depend crucially on the context.

This before-after comparison shows the limits of our current understanding of the economy. It also uncovers a tendency in the work of some economists to gloss over that fact, using unjustified notions derived from old-fashioned schematic thinking that is not supported by the evidence.

In addition, we can learn a great deal by making comparisons of this kind — both from what was correctly predicted and from what was not. Such a comparison can highlight where our knowledge is relatively secure, and where it is empirically inadequate. It can also make a contribution to the positive aspect of evidence-based economics, by identifying our current strengths and weaknesses and by suggesting better hypotheses — but this topic is beyond the scope of the present article.

Notes:

1. Joffe M. Can economics be evidence-based? http://www.res.org.uk/view/art4aApr14Features.html
2. Imperial College, London
3. Hufbauer GC and Schott JJ. North American free trade: issues and recommendations. Institute for International Economics, Washington DC, 1992.
4. Hufbauer GC and Schott JJ. NAFTA revisited — achievements and challenges. Institute for International Economics, Washington DC, 2005. A comparison of these two reports has previously been carried out by Grumiller. This is parallel to the present article, but differs in that it focuses particularly on computable general equilibrium (CGE) models. He reaches similar conclusions. See Grumiller J-A. Ex-ante versus ex-post assessments of the economic benefits of Free Trade Agreements: lessons from the North American Free Trade Agreement (NAFTA). OFSE, Vienna, 2014. Available at http://www.oefse.at/en/publications/briefing-papers/detail-briefing-paper/publication/show/Publication/Ex-ante-versus-ex-post-assessments-of-the-economic-benefits-of-Free-Trade-Agreements/ [accessed 6 March 2015]
5. These are summarised in table 3.4 of Hufbauer and Schott, 1992.
6. Maquiladoras are firms with special legal status. They typically import components for assembly (mainly by ex-rural unskilled female workers), and export the finished product. Before NAFTA they were granted tax advantages, that were then eroded by NAFTA. This sector is highly sensitive to the peso/dollar exchange rate and to the health of the US economy.
7. It was however predicted by others, e.g. Stanley Fischer in 1988 and Nora Lustig in 1992: Fischer, S. ‘Real balances, the exchange rate, and indexation: real variables in disinflation’, Quarterly Journal of Economics 1988; 103(1): 27-49. Lustig N. 1992. Mexico: the remaking of an economy. Washington, DC: Brookings Institution Press.
8. This account is based on Hufbauer & Schott 2005, except where indicated, adhering to the before/after comparison.
9. See Blecker RA. ‘The North American economies after NAFTA’. International Journal of Political Economy 2003; 33(3): 5-27.
10. Salas C. ‘Between unemployment and insecurity in Mexico: NAFTA enters its second decade’, in Scott R, Salas C, Campbell B. Revisiting NAFTA: still not working for North America’s workers. Economic Policy Institute, Washington DC. Briefing Paper 173, 2006, 33-52. http://www.epi.org/publication/bp173/ [accessed 6 March 2015]
11. Progresa was created in 1997, rebranded as Oportunidades in 2002, and subsequently as Prospera. Randomised controlled trials have demonstrated its effectiveness in reducing poverty and improving education, nutrition and health. Similar achievements have been documented elsewhere in the world, e.g. Brazil.
12. Felbermayr G, et al. Macroeconomic potentials of transatlantic free trade: a high resolution perspective for Europe and the world. CESifo, Munich. Working Paper 5019, 2014. See pp 11-12 for the translation of trade flows into GDP in the model.

From Issue no. 169, April 2015, pp.20-22