Is China’s corruption-busting AI system ‘Zero Trust’ being turned off for being too efficient?

Despite being restrict­ed to just 30 coun­ties and cities, arti­fi­cial intel­li­gence sys­tem has already helped snare 8,721 offi­cials System cross-ref­er­ences big data to eval­u­ate work and per­son­al lives of mil­lions of gov­ern­ment workers

What would you do if you had a machine to catch a thief? If you were a cor­rupt Chinese bureau­crat, you would want to ditch it, of course.

Resistance by gov­ern­ment offi­cials to a ground­break­ing big data exper­i­ment is only one of many chal­lenges as the Chinese gov­ern­ment starts using new tech­nol­o­gy to nav­i­gate its giant bureaucracy.

According to state media, there were more than 50 mil­lion peo­ple on China’s gov­ern­ment pay­roll in 2016, though ana­lysts have put the fig­ure at more than 64 mil­lion – slight­ly less than the pop­u­la­tion of Britain.

Xi Jinping tells judiciary and law enforcement agencies to ‘scrape away the poison’ [1]

To turn this behe­moth into a seam­less oper­a­tion befit­ting the infor­ma­tion age, China has start­ed adapt­ing var­i­ous types of sophis­ti­cat­ed tech­nol­o­gy. The for­eign min­istry, for instance, is using machine learn­ing to aid in risk assess­ment and deci­sion mak­ing for China’s major invest­ment projects overseas.

Beijing has been devel­op­ing a nation­wide facial recog­ni­tion sys­tem using sur­veil­lance cam­eras capa­ble of iden­ti­fy­ing any per­son, any­where, around the clock with­in sec­onds. In Guizhou, a cloud sys­tem tracks the move­ments of every police­man with a live sta­tus report.

Major Chinese telecom­mu­ni­ca­tion com­pa­nies such as ZTE [2] have won gov­ern­ment con­tracts to devel­op blockchain tech­nol­o­gy to pre­vent the mod­i­fi­ca­tion of gov­ern­ment data by unau­tho­rised peo­ple or organisations.

President Xi Jinping has repeat­ed­ly stressed the neces­si­ty of pro­mot­ing sci­en­tif­ic and tech­no­log­i­cal inno­va­tions such as big data and arti­fi­cial intel­li­gence (AI) in gov­ern­ment reform.

Anti-corruption teams installed at China’s state banks, insurance companies [3]

The chal­lenge is imple­ment­ing that vision on the ground. Look no fur­ther than an anti-cor­rup­tion AI sys­tem dubbed by the researchers work­ing it as “Zero Trust”.

Jointly devel­oped and deployed by the Chinese Academy of Sciences and the Chinese Communist Party’s inter­nal con­trol insti­tu­tions to mon­i­tor, eval­u­ate or inter­vene in the work and per­son­al life of pub­lic ser­vants, the sys­tem can access more than 150 pro­tect­ed data­bas­es in cen­tral and local gov­ern­ments for cross-reference.

According to peo­ple involved in the pro­gramme, this allows it to draw sophis­ti­cat­ed, mul­ti­ple lay­ers of social rela­tion­ship maps to derive behav­iour analy­ses of gov­ern­ment employees.

This was “par­tic­u­lar­ly use­ful” in detect­ing sus­pi­cious prop­er­ty trans­fers, infra­struc­ture con­struc­tion, land acqui­si­tions and house demo­li­tions, a researcher said.

The sys­tem is not with­out its weak­ness­es, however.

AI may quick­ly point out a cor­rupt offi­cial, but it is not very good at explain­ing the process it has gone through to reach such a con­clu­sion,” the researcher said. “Although it gets it right in most cas­es, you need a human to work close­ly with it.”

Police chief kills himself after two colleagues detained in corruption probes [4]

The sys­tem can imme­di­ate­ly detect unusu­al increas­es in bank sav­ings, for instance, or if there has been a new car pur­chase or bid­ding for a gov­ern­ment con­tract under the name of an offi­cial or one of his fam­i­ly or friends.

Once its sus­pi­cions have been raised it will cal­cu­late the chances of the action being cor­rupt. If the result exceeds a set mark­er, the author­i­ties are alerted.

A com­put­er sci­en­tist involved in the pro­gramme who asked not to be named said that at that stage a supe­ri­or could then con­tact the per­son under scruti­ny and per­haps help him avoid “going down the road of no return with fur­ther, big­ger mistakes”.

The Zero Trust exper­i­ment has been lim­it­ed to 30 coun­ties and cities, just 1 per cent of the country’s total admin­is­tra­tive area. The local gov­ern­ments involved, includ­ing the Mayang Miao autonomous coun­ty in Hunan province, are locat­ed in rel­a­tive­ly poor and iso­lat­ed regions far away from China’s polit­i­cal pow­er centres.

Another researcher involved in the pro­gramme said the idea was to “avoid trig­ger­ing large-scale resis­tance among bureau­crats”, espe­cial­ly the most pow­er­ful ones, to the use of bots in governance.

Since 2012, Zero Trust has caught 8,721 gov­ern­ment employ­ees engag­ing in mis­con­duct such as embez­zle­ment, abuse of pow­er, mis­use of gov­ern­ment funds and nepotism.

Crushing victory’: what’s next for Xi Jinping’s war on corruption? [5]

While some were sen­tenced to prison terms, most were allowed to keep their jobs after being giv­en a warn­ing or minor punishment.

Still, some gov­ern­ments – includ­ing Mayang coun­ty, Huaihua city and Li coun­ty in Hunan – have decom­mis­sioned the machine, accord­ing to the researchers, one of whom said they “may not feel quite com­fort­able with the new technology”.

None of the local author­i­ties respond­ed to requests for comment.

Zhang Yi, an offi­cial at the Commission for Discipline Inspection of the Chinese Communist Party in Ningxiang, Hunan province, said his agency was one of the few still using the system.

It is not easy … we are under enor­mous pres­sure,” he said, insist­ing that the main pur­pose of the pro­gramme was not to pun­ish offi­cials but to “save them” at an “ear­ly stage of corruption”.

We just use the machine’s result as ref­er­ence,” Zhang said. “We need to check and ver­i­fy its valid­i­ty. The machine can­not pick up the phone and call the per­son with a prob­lem. The final deci­sion is always made by humans.”

Zhou Yongkang, Bo Xilai among elite prisoners in China’s ‘tigers’ cage’ [6]

Since Xi rose to pow­er in 2012, more than 1.4 mil­lion par­ty mem­bers and gov­ern­ment employ­ees are esti­mat­ed to have been dis­ci­plined, includ­ing lead­ers like for­mer secu­ri­ty tsar Zhou Yongkang and for­mer Chongqing strong­man Bo Xilai.

A par­ty dis­ci­pli­nary offi­cial in Xiushui coun­ty, Jiangxi, who took part in the Zero Trust project said no gov­ern­ment offi­cials were will­ing to pro­vide the nec­es­sary data.

But they usu­al­ly com­ply with a bit of pres­sure,” said the offi­cial, who asked not to be named because of the sen­si­tiv­i­ty of the issue.

Xi Jinping has repeat­ed­ly stressed the need to pro­mote tech­no­log­i­cal inno­va­tion in gov­ern­ment reform. Photo: Xinhua

Disciplinary offi­cials need to help sci­en­tists train the machine with their expe­ri­ence and knowl­edge accu­mu­lat­ed from pre­vi­ous cas­es. For instance, dis­ci­pli­nary offi­cials spent many hours man­u­al­ly tag­ging unusu­al phe­nom­e­na in var­i­ous types of data sets to teach the machine what to look for.

Some offi­cials might fab­ri­cate data, but the machine can com­pare infor­ma­tion from dif­fer­ent sources and flag dis­crep­an­cies. It can even call up satel­lite images, for instance, to inves­ti­gate whether the gov­ern­ment fund­ing to build a road in a vil­lage end­ed up in the pock­et of an offi­cial, the researchers said.

Xi Jinping takes aim at more top generals as anticorruption drive rolls on [7]

The sys­tem is still run­ning in Xiushui, but its fate is uncer­tain. Some offi­cials have ques­tioned the machine’s right of access to sen­si­tive data­bas­es because there is nei­ther a law nor reg­u­la­tion autho­ris­ing a com­put­er or robot to do so.

No won­der the sys­tem is being decom­mis­sioned by coun­ties and cities that had signed up, and those still using it are fac­ing enor­mous pres­sure, with the researchers see­ing lit­tle or no hope of rolling it out nationwide.

The Zero Trust hump notwith­stand­ing, arti­fi­cial intelligence’s for­ay into oth­er gov­ern­ment sec­tors con­tin­ues as the gov­ern­ment is deter­mined to apply cut­ting-edge tech­nol­o­gy to its advan­tage. AI clerks, for exam­ple, have been recruit­ed in some courts to read case files and help judges process law­suits with high­er speed and accuracy.

Last month, a court in Shanghai became the first ever in China to use an AI assis­tant at a pub­lic hear­ing, Xinhua reported.

The machine, code-named “206”, has the abil­i­ty to record con­ver­sa­tions, show evi­dence such as sur­veil­lance cam­era footage when men­tioned by lawyers, and com­pare tes­ti­monies to help judges spot dis­crep­an­cies, the report said.

One judge was quot­ed as say­ing it would reduce the like­li­hood of a wrong verdict.

Chen Binglin

Is China’s cor­rup­tion-bust­ing AI sys­tem ‘Zero Trust’ being turned off for being too efficient?