Date: 17-12-2016
Source: The Economist
China invents the digital totalitarian state
The worrying implications of its social-credit project
GARY SHTEYNGART’S novel of 2010, “Super Sad True Love Story”, is set in a near future when the Chinese yuan is a global currency and people all wear an “apparat” around their neck with RateMe Plus technology. Personal details are displayed in public on ubiquitous Credit Poles, posts on street corners with “little LED counters at eye level that registered your Credit ranking as you walked by.” The protagonist’s are summed up thus:
LENNY ABRAMOV. Income averaged over five-year span, $289,420 yuan-pegged…Current blood pressure: 120 over 70. O-type blood…Thirty-nine years of age, lifespan estimated at eighty-three…Ailments: high cholesterol, depression…Consumer profile: heterosexual, nonathletic, non-automotive, nonreligious…Sexual preferences: low-functioning Asian/Korean…Child abuse indicator: on…Last purchases: bound, printed, nonstreaming Media artifact” [ie, book].
The novel is a fictional dystopia about the destruction of privacy. China’s Communist Party may be on its way to inventing the real thing. It is planning what it calls a “social-credit system”. This aims to score not only the financial creditworthiness of citizens, as happens everywhere, but also their social and possibly political behaviour. It is not yet clear how extensive the system will be, nor whether it will work, nor how far it will withstand the criticism ranged against it in the state-controlled media. But an outline is complete and some of the building blocks are in place. The early signs are that China is starting on the most ambitious experiment in digital social control in the world.
A pilot scheme in Suining county, in Jiangsu province north of Shanghai, gives clues about what such a system might mean in practice. Starting in 2010, the local government awarded people points for good behaviour (such as winning a national honour of some kind) and deducted points for everything from minor traffic offences to “illegally petitioning higher authorities for help”. Those who scored highest were eligible for rewards such as fast-track promotion at work or jumping the queue for public housing.
The project was a failure. The data on which it was based were patchy. Amid a public backlash, a report in China Youth Daily, a state-owned newspaper, criticised the system. It said “political” data (such as petitions) should not have been included, declaring that “people should have rated government employees and instead the government has [rated] the people.” Another state-run newspaper, Beijing Times, even compared the scheme with the “good citizen” certificates issued by Japan during its wartime occupation of China.
But the party and government seem undaunted, issuing outline plans for the social-credit system in 2014 and more detailed guidelines this year. About 30 local governments are collecting data that would support it. The plan appears hugely ambitious, aiming explicitly to influence the behaviour of a whole society. By 2020, Chinese officials say, it will “allow the trustworthy to roam everywhere under heaven while making it hard for the discredited to take a single step.”
The project is a response to the party’s biggest problems: the collapse of confidence in public institutions, and the need to keep track of the changing views and interests of China’s population (without letting them vote). It seeks to collect information on the honesty of ordinary citizens, public officials and companies alike.
A question of trust
Despite years of economic growth, popular discontent at widespread corruption has grown stronger. A series of scandals about everything from shoddy housing to out-of-date vaccines has led to public cynicism about companies and the government’s ability to enforce rules. Social-credit scoring aims to change that by cracking down on the corrupt officials and companies that plague Chinese life. And it aims to keep a closer track on public opinion. In a society with few outlets for free expression, big data might paradoxically help make institutions more accountable.
But it could also vastly increase snooping and social control. In other countries there have been many scare stories about Big Data leading to Big Brother. Most have proven false. But China is different. It is a one-party state, with few checks on its power, a tradition of social control and, in President Xi Jinping, a leader even more prone to authoritarianism than his immediate predecessors. The extent of social-credit scoring will depend on what the government intends, whether the technology works and how the party responds to public concerns.
Start with intent. The “planning outline” published in 2014 said the government “pays high regard to the construction of a social-credit system”—suggesting the project has the imprimatur of Mr Xi and Li Keqiang, the prime minister. Social credit, it declared, “is an important component …of the social-governance system”: in other words, it is part of governing the country.
The paper did not set out how the system would work but was clear about its aims. They are to strengthen confidence in the government by improving its efficiency through big data; to crack down on companies that cheat and sell unsafe goods; and to “encourage keeping trust and punish breaking trust…throughout the entire society”. Social credit, it concluded, would be “an important basis for…building a harmonious socialist society”.
Getting to know you
Such thinking is in keeping with the party’s long record of using bureaucratic tools to restrict freedom and invade privacy in the name of public order. Almost everyone has a hukou (household registration) document that determines where citizens can get public services. Most people once had a dang’an (personal file) containing school and work reports, and salary details. Both controls have been relaxed, notably the dang’an. But both still exist.
Increasing numbers of people in government, state-owned firms and universities are required to hand over their passports “for safe keeping”. Holders of passports in some parts of the restless regions of Xinjiang and Tibet have also been told to hand them over to the police.
Punishments and rewards for behaviour are woven into the government’s activities. The one- (now two-) child policy remains the extreme example of a supposed greater good trampling over private interests. But it is not the only one. The Elder-care law of 2013 requires all adult children, on penalty of fines or jail, to visit parents over 60 “often” (the courts define what counts as often). A few people have been fined under the law and one official said their offences might be entered onto their dang’an, though there is no sign that this has been done.
China has “an administrative rewards system” in which hundreds of thousands of people a year receive honours and titles, such as “outstanding cadre”, “spiritually advanced individual” and “civilised village”. Winners get money, a higher pension, better health insurance and the right to jump the queue for public housing. The honours system is valued by the leadership. Last year, all seven members of the country’s highest decision-making body, the Standing Committee of the Politburo, attended the awards ceremony of the National Model Worker programme.
Wholesale surveillance, increasingly of the digital sort, is a central pillar of Chinese communist rule. A system of block-by-block surveillance called “grid management” is being set up in several parts of the country: police and volunteers keep tabs on groups of a few hundred people, supposedly to ensure the rubbish is collected and disputes resolved. It is part of a tradition of self-policing that stretches back to the Song dynasty in the 11th century.
Newer forms of monitoring involve the ubiquitous use of closed-circuit television cameras. In 2009 China had 2.7m of them; now it may have overtaken America as the country with the largest number of CCTV devices. According to Jack Ma, head of Alibaba, China’s largest internet firm, the company’s home town of Hangzhou has more surveillance cameras than New York, a somewhat larger city.
As internet use has grown (see chart), so have China’s comprehensive controls in cyberspace—from the Great Firewall, the system that blocks access to tens of thousands of websites (Economist.com among them); to the Golden Shield, an extensive online surveillance system; and the Great Cannon, a tool to attack hostile websites. China’s cyber-censors can suspend internet or social-media accounts if their users send messages containing sensitive terms such as “Tibetan independence” or “Tiananmen Square incident”.
The scale of the data-collection effort suggests that the long-term aim is to keep track of the transactions made, websites visited and messages sent by all of China’s 700m internet users. That would be enormously ambitious but probably not impossible. According to leaked documents, America’s National Security Agency can collect 42bn internet records a month and 5bn mobile-phone location records a day.
To make such surveillance work, the government has to match the owners of devices with the digital footprints they leave. So laws passed in 2012 and 2016 require internet firms to keep their customers’ real names and other personal information. But there are lots of fake registrations. And it is unclear how censors plan to tackle virtual private networks, which mask a user’s IP address.
Who’s naughty and nice
The emerging social-credit system builds on this history of monitoring and control of people’s private lives. Lists are central to the project: you need lists of identities to order the data you gather. And lists are a Chinese speciality. China’s tourist authority keeps a no-fly list for ill-mannered travellers, who can be banned from going abroad for up to ten years. The Cyberspace administration keeps a “white list” of favoured media firms that may sell their articles to other outlets. And so on.
The list at the heart of the social-credit system is called the “judgment defaulter’s list”, composed of those who have defied a court order. If two people or companies have a contract dispute, or if couples are fighting over a divorce or child support, the parties can go to a civil court for judgment. If the losing party then defaults on payment, he, she or it is put on the list. Names of offenders are displayed on an electronic crawl outside court houses. According to the supreme court, there were 3.1m defaulters on the list at the end of 2015.
All countries have problems enforcing civil judgments in financial cases, so the list may not look unusual. But it is. It is exceptionally long, and made available to dozens of government departments and party organisations, all of which can apply their own sanctions to defaulters. People on the list can be prevented from buying aeroplane, bullet-train or first- or business-class rail tickets; selling, buying or building a house; or enrolling their children in expensive fee-paying schools. There are restrictions on offenders joining or being promoted in the party and army, and on receiving honours and titles. If the defaulter is a company, it may not issue shares or bonds, accept foreign investment or work on government projects. By August 2016 defaulters had been stopped from buying airline tickets about 5m times. This goes far beyond normal legal enforcements.
Sins with Chinese characteristics
From blacklisting debt-defaulters the system could be expanded a bit, say, to keep track of companies that sell poisoned milk or build shoddy houses. Yet guidelines issued in May and September suggest it could go much further. They call the defaulters list “an important component of social-credit information”, implying that it is part of a larger system, and that financial offences are only one category of wrongdoing. Other sorts of “untrustworthy behaviour” meriting attention include: “conduct that seriously undermines…the normal social order…seriously undermines the order of cyberspace transmissions”, as well as “assembling to disrupt social order [and] endangering national defence interests”. Such broad categories imply the system could be used to rate and punish dissent, expressions of opinion and perceived threats to security.
Although not spelled out clearly, the guidelines could, on the face of it, allow the state to integrate its many databases: everyone’s hukou and dang’an, information from electronic surveillance, the tourist blacklist, the national model-worker programme and more. Even regulations on video games published in December say that firms and gamers that violate the rules could be blacklisted and inscribed in the social-credit database. At worst, the social-credit project could become a 360-degree digital-surveillance panopticon.
That may sound like scaremongering. After all, Google, Facebook, data-brokers and marketing companies in Western countries—even American presidential-election campaigns—all hold vast quantities of personal information without causing serious harm to civil liberties, at least not so far.
But China treats personal information differently from the West. In democracies, laws limit what companies may do with it and the extent to which governments can get their hands on it. Such protections are imperfect everywhere. But in China they do not exist. The national-security law and the new cyber-security law give the government unrestricted access to almost all personal data. Civil-liberty advocates who might protest are increasingly in jail. And, according to America’s Congressional Research Service, companies that hold data, such as Alibaba, Baidu (China’s largest search engine) and Tencent (which runs a popular social-messaging app) routinely obey government demands for data.
Big-data systems in democracies are not designed for social control. China’s explicitly would be. And because its leaders consider the interest of the party and society to be the same, instruments of social control can be used for political purposes. Earlier this year, for instance, the party asked China Electronics Technology Group, one of the country’s largest defence contractors, to develop software to predict terrorist risks on the basis of people’s job records, financial background, consumption habits, hobbies and data from surveillance cameras. Sifting data to seek terrorists can easily morph into looking for dissidents. It is telling that Western intelligence agencies have tried to use data-mining schemes to identify individual terrorists, but failed because of an excess of “false positives”.
So can a vast social-credit system work? The Chinese face two big technical hurdles: the quality of the data and the sensitivity of the instruments to analyse it. Big-data projects everywhere—such as the attempt by Britain’s National Health Service to create a nationwide medical database—have stumbled over the problem of how to prevent incorrect information from fouling the system (this undermined the Suining experiment, too). Problems of bad data would be even more onerous in a country of 1.3bn people. Vast treasuries of data would also give big incentives for cyber-criminals to steal or change information.
How to analyse the data would be equally problematic. The feature of the social-credit system that has attracted the most attention and alarm is the notion of ascribing “credit scores” (points) to social and political activity. Here, the model seems to be America’s marketing industry. Companies work out credit scores that predict people’s patterns of consumption based on things such as job security, health risks and youth delinquency. But errors abound. The World Privacy Forum, a non-profit organisation, says credit scores are based on hundreds of data points with no standards of accuracy, transparency or completeness. As the report concluded, “error rates and false readings become a big issue.” Garbage in, garbage out.
What could go wrong?
The government is well aware of these difficulties. It has allowed an unusual amount of discussion on them in state-run media, suggesting it may be testing the waters before deciding how far to plunge in. A recent high-level “social-credit summit” in Shanghai, for example, talked about how scores can be checked, and mistakes rectified; many argued that legal protections needed to be improved. Zhang Zheng, director of the China Credit Research Centre at Peking University, said multiple problems remain unsolved, and that the administration needed to be reined in.
A commentary in Beijing Times complained about plans to punish people who do not pay their electricity bills by limiting foreign travel and bank borrowing. “I have never opposed the establishment and improvement of a credit-information system,” wrote the author, Yang Gengshen. “I am only against using credit to expand the power of the strong and further compress the space for civil rights.”
Much about the social-credit system remains unclear. The government has not yet determined whether it wants the system mainly for cracking down on crooks or to go the full Big Brother. It is uncertain about how much of the information it holds should be incorporated into the system. The surveillance technology is largely untested at the vast scale of China. And the fragmentation of China’s intelligence agencies would have to be overcome.
But the government is creating the capacity for a long-tentacled regime of social control. Many of the elements are ready: the databases; the digital surveillance; the system of reward and punishment; and the we-know-best paternalism. What remains is to join the pieces together. If and when that is done, China would have the world’s first digital totalitarian state. As another character in “Super Sad True Love Story” writes to a friend: “This is what happens when there’s only one party and we live in a police state.”
