Two years ago, Paul Romer warned the incoming President of the World Bank that the Bank should outsource its research function because “diplomacy and science cannot both thrive under the same roof.”

The Bank depends on good relations with member countries. Romer had been World Bank Chief Economist from 2015 through his early 2018 departure, winning the Nobel Prize in Economics later in 2018.

And Romer warned that the Bank’s diplomatic function was too easily protected at the expense of its research.

On Friday, Romer’s warning was amply vindicated with the release of a scathing independent review of the Bank’s Ease of Doing Business Index.

For the past twenty years, the Bank’s Ease of Doing Business Index has been a touchstone for empirical economists. While country rankings can grab headlines, and New Zealand often bagged first place, the Index has been far more important in research.

How do differences in cultural and political institutions affect the ease of doing business in different places? Do differences in colonial background still matter? To what extent do reforms undertaken in one country influence its neighbours?

Important questions require reliable indicators on how business works in different places. And the World Bank’s Ease of Doing Business Index has provided indicators aplenty. Their headline rankings are based on series of sub-indices tallying how long it takes to start a business, how hard it is to deal with getting construction permits (New Zealand ranks seventh), the ease of getting electricity and registering property – and far more.

While some empirical work would trace out what gives rise to differences in the ease of doing business, other work would look at the consequences of those differences. How harmful is it for economic growth if it is hard to enforce contracts or resolve insolvencies? How important is it that minority investors be protected? Do improvements in the ease of doing business help to attract investment and improve quality of life?

Friday’s report says that the last two iterations of the World Bank Ease of Doing Business report cannot be relied upon. The Bank has paused the next report and is undertaking review and audit.

What happened?

Let’s go back to Paul Romer.

Romer left the Bank after pointing out potential errors in Chile’s Index ranking. Changes in the Index’s construction may have been motivated by a desire to push Chile down the rankings, because of its change in government. Two weeks after having apologised to Chile and having announced that the Bank would review prior iterations of the Index, Romer left.

A year later, Romer provided his warning to the incoming President. While he did not warn specifically about pressure over Chinese statistics, he noted that multilateral institutions need embrace ambiguity over, for example, allowing “Chinese Taipei” to compete in the Olympics – before warning that maintaining diplomatic conformity “is not compatible with scientific research.”

In January of this year, the International Bank for Reconstruction and Development commissioned a review of the 2018 and 2020 Indices. The Review noted that the Bank, going into the 2018 Index, was particularly sensitive to concerns from China because each country’s share of the Bank’s funding was under review and China’s contribution was to increase.

The review found high-ranking Chinese officials “repeatedly expressed their concerns” about China’s ranking. The team responsible for the Index considered methodological changes “that might boost China’s ranking”. Bank Chief Executive Krisalina Georgieva was reportedly directly involved.

Simeon Djankov, the economist who had helped found the Index, and who was advisor to Bank Chief Executive Georgieva during this period, was reportedly involved in helping find the changes necessary to boost China’s score and ranking.

For the 2020 Index, Djankov reportedly instructed the team to alter scores for Jordan and Saudi Arabia. The Report appears to catch Djankov in presenting falsehoods as part of the investigation, and reports staff feared retaliation from him for challenging decisions.

The Bank also provides advice to countries on improving policy. The Report warns that the Bank’s advisory services to China and Saudi Arabia led to lobbying for changes improving those countries’ rankings.

For empirical economists, the problem is grave. How much can we trust results from the Ease of Doing Business Index, and how far back does the problem go? Hundreds of published papers and working papers rely on this work.

For the Bank, the problem Romer pointed out remains: can the Bank be a reliable producer of research and statistics while massaging diplomatic concerns among its funder-members?

And, for the rest of us, how much can we trust multilateral institutions when individual countries can bring inordinate pressure to bear on the work they produce, and face no sanction for having done so? The President of the World Bank at the time, Krisalina Georgieva, who disputes the investigation’s findings, now heads the International Monetary Fund.