How secure is the Great Firewall of China?
Why China should be nervous about its tech monoculture—and so should the West
This post is about…
…why I think the Great Firewall of China (GFW) is systemically insecure against the types of attacks its adversaries might want to lever against it
…how some of the forces that make the GFW insecure—monoculture and centralization—have also come to plague the Western internet
…why we need basic research to dig ourselves out of this hole
Warning: wonky.
For hundreds of years, China faced persistent attacks from Eurasian nomads. Starting in earnest with the first Qin emperor in the 2nd century BC and continuing until the 17th century AD, generations of Chinese emperors spent tremendous government money to build an enormous wall: the Great Wall of China.
China's Great Firewall (GFW) really does live in the legacy of this project. Given an asymmetric threat (equestrian archers → randos posting online), the Chinese state spent massive capital erecting a barrier (wall → firewall) that both keeps the threat out and keeps the subjects in.
For the uninitiated: China's GFW censors all content on the Chinese internet. It blocks content coming into China from abroad and content posted domestically within China's internet, including fully encrypted content.
Put aside for a moment any moral or political feelings you might have about censorship, China, or the GFW. I'm here to ask a different question. How secure is this thing?
The Great Firewall has to do things that are hard to do securely
Ponder for a moment what the GFW needs to do. It needs to inspect every packet1 that travels over the Chinese internet. Now, there are obviously optimizations available. But, no matter how you slice it, the GFW is doing lots of parsing. Think just about its Winnie-the-Pooh problem or just its Tiananmen Square problem. It needs to identify images and videos that depict either of these verboten things. That means opening all sorts of formats and cobbling together codecs, including extremely obscure ones. (You can’t let subversive memes through just because they’re, say, .pcx files. There's only one parser available, and it hasn't been updated in ten years? Too bad! Ship it!). As any security person knows, parsing random stuff from the internet is extremely dangerous. Inputs can be anything. And when attackers have access to the parsers and codecs you're using (which they almost always do), they can find a bug in one of those codecs, and from that bug, potentially break out into the rest of your systems. That means that any bug in any parser could be exploited at any of the system’s edges—that is, anywhere the GFW is deployed, which, in China, is everywhere.
Now, it’s worth clarifying that the GFW is not some monolithic codebase run exclusively by the Chinese state. Plenty of in-the-loop-but-functionally-private companies, like Baidu, Weibo, or Douyin (TikTok's Chinese equivalent) are likely deputized to suppress political sensitive content on their own platforms. They likely do so with internally built tools, and if those tools don't work well enough, someone at those companies likely gets a very stern phone call from a government agency. This ‘deputizing’ of domestic tech companies takes some load off of the state's GFW infrastructure and also introduces some heterogeneity into the overall state censorship ecosystem. But the GFW still has lots of work to do. With content that happens outside of major platforms, and particularly with traffic that comes from abroad, the GFW cannot rely on its domestic ‘private’ sector to block everything. It must use its infrastructure, which must do lots and lots of parsing. And every time it does, an attack opportunity arises.
So, what are the odds that CCP's state GFW infrastructure has zero vulnerabilities in any parser or, indeed, in any part of its codebase that ingests unsanitized bits from the public web? Well, consider: even Apple, something of a security 'high-water mark' as far as technology goes, had a pretty serious bug in one of its image parsers, one that easily could have become the 'bad' sort of Apple 0-day, the kind that could feature in an exploit chain that ends with a remote attacker all up in your iPhone, iCloud, etc. And image parsing is only one of the things the GFW needs to do! It needs also to monitor natural language texts, probably with a large language model2 which has God-knows-what manner of breakout, model/data extraction vulnerability, susceptibility to adversarial examples/dataset poisoning, etc.
No third party has a good incentive to look for bugs in the GFW
Apple also has an advantage GFW lacks: third parties have an incentive to disclose the vulnerabilities they find. Some minimal background: In practice, the people who build stuff do an imperfect job of finding the bugs in what they build. There are a variety of reasons for this perennial bug-blindness: limited time/money/manager interest, a psychological orientation toward making the thing work vs. breaking it, lack of clear internal checks/incentives for getting security wrong/right, etc. So, in practice, companies (and governments!) rely on third parties—users, outside security firms, and often, straight-up randos—to identify security vulnerabilities in their products. Now, Apple has a tremendous number of users, many of whom care very deeply about their Apple (especially iPhone) device being reasonably secure under a variety of threat models—many humdrum, some quite exotic. These users are very happy to pay Apple a premium to ensure those devices remain reasonably secure under the threat models they care about. Paying that premium may look like paying more for each iPhone compared to each Android,3 but it could also look like paying good hackers to do adversarial stuff to iPhones and, when they find something, telling Apple to “please fix this right away, thank you.” In other words, to do free security work for Apple.
The GFW has many users, but it only has one paying customer: the Chinese state. Everyone else in the world has either (1) no intrinsic motivation to find vulnerabilities or (2) an incentive to look for vulnerabilities and never disclose them. Now, China experts will here object: China has robust safe-harbor laws. Disclosing bugs to the government is well-protected, and maybe even well-incentivized, for Chinese citizens. Granted. The question is: who's going to go looking? All the Chinese companies ‘deputized’ by the Chinese state to keep the internet ‘clean’ almost certainly do so within their private networks. Naughty Douyin (Chinese TikTok) posts are censored before they hit the state-run GFW infrastructure. What incentive does Douyin have to go looking for bugs outside their network boundary, in what (from their perspective) amounts to end-user content delivery infrastructure? The only bugs in the GFW they care about are the ones that prevent them from pushing bits to users—not the ones that get you root on CCP computers. Patriotic netizens could go looking, but from available research, the hacking competitions seem to be oriented more toward hacking Western software than red-teaming politically sensitive domestic infrastructure.
Bureaucracy may work against the GFW’s security interests
Combine this with what we know about how the GFW is built and the picture looks even worse. Our best information comes primarily from the work of Xiao Qiang, who uses pretty much every OSINT source available to figure out how the GFW is made. The result is a lot of very "government" looking group photos of agencies with innocuous-sounding names. But from these photos, we learn a picture. The first thing we learn: the GFW is slow to upgrade, partly because it's critical infrastructure (push a bad commit and PRC's whole consumer internet goes down), and partly because, well, this is government: things need a billion stamps of approval to go live.
One colorful footnote on CCP bureaucracy: The GFW actually gets harder to circumvent around big CCP events (like party congresses. I talked to the authors about this result, and the best explanation any of us could come up with is that, around these events, party leaders become more sensitive about their careers and, therefore, more tolerant of false positives (innocuous things getting blocked by accident). Imagine getting a phonecall from an important honcho in the Party.
“Hey! I'm seeing [thing that bodes poorly for my political career]. Why?”
“Uh, well, we can definitely look into—”
“Block it!”
“Sure, we can do that, but you should know: we'll need to collect a good amount of data, then we'll need to A/B test our filters to make sure they don't block benign content, then we'll need to get approval from—”
“Block it! By Wednesday!”
“Got it.”
Click.
The second thing we learned from Qiang’s work: a lot of GFW code is built by academics.4 While this may bode well for the GFW's ability to tackle hard problems and get proof-of-concept solutions (may bode well—I'm an academic; I hesitate to flatter our ranks), it bodes poorly for the code quality of the most “research-y” parts of the GFW’s infrastructure. As an academic whose research code hydrates production systems: Academics write insecure code. This is so well-understood within the security community as to be a trope.
China’s adversaries have great incentives to attack the GFW
Let’s assume a bug exists. What can you do with a bad bug? Well, if you find a foothold (say, reading memory on a GFW machine), you could create an exploit chain that ends with you popping shells inside of Chinese telecoms, Chinese government agencies, etc. In other words, insider access to GFW infrastructure. You could tamper with censorship rules or just block everything: a massive denial-of-service. In fact, the GFW has such an enormous attack surface, and incentives are so unfavorable to the CCP that I would wager that the Western intelligence community already has some level of GFW access.
So what have we learned? Well, the sociotechnical assemblage that is the GFW has...
...a large attack surface area (because it has to read everything that might enter a user's proverbial eyeballs)
...a bad chance of learning about bugs from others (because of a lack of incentive internally and incentive misalignment externally)
...issues patching quickly, and maybe even a tendency to generate insecure (i.e., research-y) code in sensitive places.
The result is a GFW that systematically favors adversaries—not just for circumvention but also for attackers. Those attackers could…
…censor or un-censor content (globally or for specific users!)
…turn off internet access in arbitrary parts of the country
…spy on Chinese citizens
…move onward to other Chinese government/telecom/private-sector systems
So what? Either China's adversaries know about this or I'm wrong. Why post about it here, where it might—a small chance, but might—help the CCP harden the GFW (or help the West take down Chinese systems, depending on which horse your money is on)?5
Because the whole episode shows that centralized infrastructure is—and this is a technical term—whack, and the U.S.'s private sector internet is succumbing to many of the same problems.
The U.S. internet has become a monoculture, too
I have written at great length about content distribution networks (CDNs). In short, CDNs deliver most of the content on the public internet, at least in the West. Some of them are owned and vertically integrated with the giants you know and ‘love:’ Google, Meta, etc. But every website that's not owned by one of those big names basically uses one of three companies for CDN service: Akamai, Cloudflare, or Fastly. That's a triopoly, not unlike the cell service triopoly of AT&T, Verizon, and T-Mobile. Not exactly a Brandesian, small-business paradise, but not necessarily a national security threat either. Here's the problem. The web is highly interdependent. Even big firms rely on software developed by small firms and even individual hackers/open-source contributors. From the perspective of innovation, that's probably a good thing, as Berk & Saxenian (2024) persuasively argue. But, since the small guys all rely on one of three CDNs, the whole internet relies on... any one of those CDNs. When Fastly goes down, Amazon—a company famous for owning servers—goes down with it.
Would an adversary of the West actually attack a big, commercial CDN like Cloudflare? Well, Russia & China have both invested heavily in ‘internet sovereignty’.6 If those countries rely minimally on Western CDNs and know that they do, attacking CDNs presents minimal risks to their domestic internet infrastructure. And, if they want to do damage (or just earn some temporary control over, e.g., what Americans see on the news), their incentives to attack are there, too.
Could an adversary get into a CDN like Cloudflare? One almost did! (And Cloudflare was less than forthcoming about it when it happened).
We have our own homogeneity problems in the West. This CDN business is just a part of the puzzle. In cloud computing, Amazon, Google, and Microsoft share their own triopoly. In AI, OpenAI+MS, Google, and Meta do. Vertical integration within these companies, across product lines, creates their own types of centralization—like in DNS. Like in AI.
CDNs have values—and we want them to
Before getting into the problem, let’s describe what’s not the problem. The problem is not service providers having politics.
Zooming out: The triopoly of CDNs in the U.S. is an inter-network (i.e., between ISPs) trust layer. It uses data to broker untrusted TCP/UDP connections. So is the Great Firewall. What it means to broker TCP/UDP connections—specifically, what it means to do so “correctly”—really is a question of values—that is, of politics. Cloudflare has terminated service for political reasons, as well it should have. We want service providers to have values, which will, at times, require them to acknowledge their politics. Doing so is worth the economic risk. We cannot have an ungoverned internet.
The difference between our triopoly (Cloudflare, Akamai, and Fastly) and the GFW is the political economy in which each operates. U.S.-domiciled companies are, at least in theory, accountable to democratically elected lawmakers and susceptible to users voting with their feet (their customers could move from Cloudflare to Fastly if they wanted to enact a boycott). From where I stand, that situation is not half bad.
In many ways, our triopoly of big, commercial CDNs—Cloudflare, Akamai, and Fastly—is something of an (if you’ll allow me) “Great Freedom Wall:” it blocks stuff in a liberal (as in -ism) way. And, as far as an inter-AS “trust layer” goes, our Western Great Freedom Wall is better from a security perspective than a national GFW made up of one. It has more heterogeneity, and the incentives to report bugs are higher when you and your customers have the same concerns (fast delivery plus attack mitigation) than when one party cares a lot more about success than the other (compared to the Communist Party of China, Douyin cares relatively less about Winnie the Pooh memes—at least intrinsically). In other words, three is better than one, and customer relationships are (marginally) better at aligning service providers’ incentives toward security.
…but we need resilience, too
Still, Alannis Morasette: isn't it ironic? The U.S. Military apparatus designed the internet to be so radically without a single center of failure that it, like the freeway system, would be difficult to disrupt during war. Throughout its commercial rise, the internet’s boosters (from earnest-but-wrong elected officials to super-cringe-in-retrospect California ideology types) cited this feature as…
…secure (as in resilient)
…freedom-enhancing
…inherently compatible with that Western, free-market capitalism most closely associated with the United States
How is it that this very system produced a centralized network—one that is, in my view at least, not tremendously resilient in war, at least not any longer—so not resilient that it has actually become something of a national security liability for the U.S.? And—worse—how has it become a network that shares those features with a communist country, where a single, state-run infrastructure interdicts literally all consumer internet access?
Some basic research questions (and a note on CrowdStrike…)
In the midst of writing this piece, a bad CrowdStrike upgrade caused the largest IT meltdown in history, grounding thousands of flights (including one I was booked on). Crowdstrike is not internet infrastructure per se—but, using the internet, it achieves economies of scale. From those economies of scale, it can attract more customers—a virtuous feedback loop.
The downside? This dynamic breeds monocultures.
“…whether we want to govern platforms through competition, or want to accept that they're inherently monopolistic and regulate them instead?” (Khan, 2017)
When everyone uses the same thing, bad things happen when that thing goes down. Intuitive enough. The policy question is: if things get better when they’re bigger (because they can take advantage of economies of scale), how do we navigate the tradeoffs between the benefits of resilience and the benefits of scale?
Over the last three or four years, I've become convinced that the resolution to this situation has to do with competition policy—that the way we think about what constitutes the internet's core needs a refresh, as my collaborator Tejas Narechania and I have written, but also, more recently, that we're a lot less sure than we ought to be where economies of scale really come from in big technology sectors. We’re not as sure as we should be where vertical integration improves service, and where vertical integration is just a way of extracting rent.
To resolve these problems, we need to ask new questions. For example: Is there an essential tradeoff between service providers' governability and their scale? (Would more, smaller-scale CDNs mean each CDN is less governable, potentially giving rise to more 8chans?) If so, how do we balance governability and resilience?
A (non-exhaustive) set of questions we’d need to answer in order to manage tech monoculture effectively
Where do economies of scale really come from in internet-enable technologies? Which economies of scale benefit customers—and in what ways?
How do we navigate the tradeoffs between the benefits of resilience and the benefits of scale?
What do we do about things we can’t make smaller without also making them worse?
Is there an essential tradeoff between the governability of service providers and their scale?
All of these questions originate from a single question, one best asked by Lina Khan in her landmark Amazon's Antitrust Paradox (2017): “…whether we want to govern platforms through competition, or want to accept that they're inherently monopolistic and regulate them instead?”
More later.
Thanks to Xiao Qiang.
A packet is the smallest atomic chunk of data in the internet stack.
With Chat XiPT, perhaps? Maybe, maybe not. On the one hand, it's probably not practical to have the ‘sanitized’ (politically correct from CCP's perspective) model you make available to the public and also the model you use to identify all the politically sensitive stuff it reads on the public web. On the other hand, the datasets, fine-tuned models, and prompts that help you build the former almost certainly help you build the latter.
In the case of government contracts, paying a premium for iPhones because they're secure can kickstart a virtuous cycle of internal investment in security, which has arguably been the case for Apple. On one hand, the standoff between Apple and the FBI over the San Bernadino shooter's iPhone was the US Federal Government against one of its more important contractors. On the other hand, it was proof in the pudding that Apple was 'delivering the goods' on security and incented the USG to figure out how much it costs to defeat their own security.
In one case, the Russian security firm Kaspersky accused the USG of exploiting a bug in iPhones to spy on Russians (and then had the gall to demand the bug bounty). If their allegations are true, then—even when the Western intelligence community does hoard Apple 0-day—security firms in Western adversaries have an incentive to disclose the bug! That's how good Apple is at this whole security thing. As with all innovation, it’s about the ecosystem!
Whether Apple remains the vendor of choice for Russia or whether they prefer to create an on-shore or Chinese competitor remains to be seen. Of course, they'll face a strucutral problem with on-shoring that infrastructure: that infrastructure will have the same incentive alignment problems with security that the GFW has!
These academics often publish work that appears to be about some general computer science topic but, in fact, has some dual use for the GFW. That means that, through these academics’ collaborations, the CCP may be getting non-Chinese academics to contribute to GFW infrastructure unwittingly. A fascinating topic in its own right.
A brief note re: "doesn't this help the CCP?” Securing software—especially big software—isn't just about code. It's about institutions. I think the very idea of a censorship apparatus tuned to the tastes of a small group will face structural disadvantages against attacks. That suspicion is as much what motivates this post as it is what motivates my concern about the concentration of infrastructure among private companies in the West!
Russia has claimed at various times to have disconnected itself from the global internet. But it's good to take these claims with a big grain of salt. Ideally, we’d want BGP route announcements or some other independently verifiable signal to confirm these reports.
Excellent article Dr. Merrill!!