Big data and artificial intelligence have brought us into an age of the algorithm economy. Deep-learning methods now shape how information spreads online, how the digital economy develops and how information services are delivered, and the economic returns spill well beyond the firms that own them.
But these algorithms are data-driven: their behaviour is not directly controlled and their decision logic is hard to explain, which raises questions of fairness and privacy. Worse, the black-box quality makes them hard to govern, increases the chance that capital captures them, and casts a shadow over the sector. The negative effects are already visible. Platforms whose algorithmic decisions harm users are a recurring story, conventional supervision does not reach them, and the result is a trust problem for regulators.
The asymmetry
That trust problem has a root: firms hold an asymmetric advantage in data and algorithms. Ownership and control of the underlying data sit mainly with platform companies, which design, run, test and analyse the algorithms end to end and therefore hold the stronger position. Regulators, in an algorithmic society, find themselves pushed to the margin and risk losing oversight of the data and algorithms that matter. A black box means private interests can capture the public interest, and capital can sidestep the constraint of public authority. It also conceals the algorithm’s own defects, which can trigger safety problems, while making it hard for a regulator to examine what the algorithm actually does. Both the substance and the instruments of regulation are limited, and correction comes late.
Differential pricing is the standard example: the same product or service priced lower for a new user than for a loyal one, a quiet form of market discrimination that damages fairness, consumers and public confidence. Dynamic pricing algorithms carry a further risk of collusion — several of them running at once can raise prices and concentrate a market without anyone deciding to. Beyond commerce, the black box leaves many algorithmic processes opaque and leaves citizens’ right to know largely unenforced.
China’s 14th Five-Year Plan for national informatisation calls for an orderly governance system for digital development. The regulations on managing algorithmic recommendations in internet information services, issued in January 2022, explicitly rule out algorithmic discrimination, differential pricing and the engineering of compulsive engagement. What is missing is the rest of the apparatus: the laws, the supervisory standards, the monitoring technology and the platforms.
What regulation can reach
Meeting that gap means building a legal framework on three levels — legal liability, administrative liability and ethical responsibility — and building trust between the parties. On legal liability, the state has to set and refine rules that give multiple actors a shared standard for inspecting the black box. On administrative liability, the focus is review, supervision and accountability: technical standards and codes of practice that require a platform’s algorithms to comply with the law and serve the public interest, weakening or removing the harm the black box causes. On ethical responsibility, the focus is embedding algorithmic ethics, so that platforms are required to design to the values and norms society generally holds.
Governing design, use and supervision end to end needs a complete standards system — baseline standards, risk assessment, security grading and supervisory standards — applied by tier and by category, with technical indicators that are comprehensive, clear and workable, and that keep the system observable and controllable in real time. Firms should cooperate with the process, publish the information the standards require and raise transparency. Civil society has a role too: third-party scrutiny is what balances the asymmetry.
Black-box risk is hard to define, hard to supervise, hard to prevent, hard to evidence and hard to attribute. Addressing that needs joint work between research institutes, firms, universities and government on the underlying theory and core technology — measurement and assessment, behaviour monitoring, risk protection, game evolution, mutation tracing — and a richer set of supervisory instruments, so that risk stays controllable at every stage. Automated, standardised governance also needs intelligent evaluation and monitoring across the algorithm’s whole lifecycle, and a supervisory platform that balances explainability, privacy, fairness and robustness.
Liu Xianglong is a professor and doctoral supervisor at Beihang University’s School of Computer Science, and deputy director of its State Key Laboratory of Software Development Environment.
Republished for information. The original text has not been independently verified and does not necessarily reflect our views.
Want to talk through what you just read?
We work end to end, from product design to launch. Tell us about your scenario and we can start there.
Shenzhen XZS Technology Co., Ltd. · Tel 0755-82494862 · 13760424861