Why Big Data Won’t Save Central Planners from the Knowledge Problem

From FEE.org (Apr. 11):

Among statisticians, economists, and business executives, “Big Data” is all the rage. Large and detailed data sets that, until recently, couldn’t even be stored on a computer are now managed and analyzed using innovative statistical techniques. Hopes are high that these advances will improve scientists’ ability to predict human behavior. Some enthusiasts even speculate that Big Data will render markets obsolete, enabling central planning of the economy. Big Data is more than a buzzword, but its potential is often wildly overstated.

How Are We Using Big Data?

To make sense of enormous databases, statisticians have developed innovative analytical tools, including machine learning, A/B testing, and natural-language processing. Storage and computation speeds have also improved in recent years.

……………….

Of course, Big Data raises important ethical concerns. Data are often drawn from oblivious populations. Although Internet users typically sign a Terms of Service contract granting social media companies permission to record behavior, the legalistic jargon in these agreements usually leave users unaware of the scope and scale of surveillance. The Cambridge Analytica debacle clearly demonstrates these problems (though it does not imply government would be a better steward of information).

Governments, on the other hand, make no such pretensions about obtaining permission. Officials in the Xinjiang region of China use Big Data analytics to track the movements of the Uighur ethnic minority. The Associated Press reported thousands of Uighurs have been sent to political indoctrination camps and prevented from communicating with relatives abroad. The risk of a data-fueled dystopia is not trivial.

Big Data Does Not Equal Big Knowledge

Some Big Data enthusiasts are somewhat reminiscent of 20th century “market socialists.” For example, a 2017 article by economists Binbin Wang and Xiaoyan Li of Sichuan University for the World Review of Political Economy contends that Big Data solves many of the problems faced by 20th-century socialism. Tech journalists and even entrepreneurs like Alibaba's Jack Ma have expressed similar sentiments. Since today’s governments have far finer-grained knowledge of citizens, the argument goes, central planning could be more viable.

Big Data represents an important scientific advance, but it is fundamentally inadequate to achieve these more ambitious goals. The problem is not merely a time-lag of data collection or the inability to predict future innovations and sudden changes in preferences (though these limitations are also important).

The reason that Big Data can’t enable central planning is that any data on economic activity is inextricably predicated on the existence of markets.The algorithms which private firms use to better predict demand and supply rely on an incoming flow of market data. Without a market, that data ceases to exist.

It is only from market transactions that prices emerge. Consumer behavior is only conceivable when consumers have the freedom to choose between products. Profits and losses reflect the performance of actual firms and market participants engaged in rivalrous competition. [read more]