The Department of Homeland Security has sided with Apple, Amazon and Supermicro, which vehemently denied claims contained in a Thursday Bloomberg report alleging that China installed miniature "spy chips" the size of a grain of rice on motherboards built by Supermicro and used by around 30 companies. Bloomberg, citing 17 unidentified intelligence and company sources, claims that the chip can compromise data on the servers, giving China a backdoor into some of the world's largest companies and organizations.
The three tech giants have all published statements on their websites refuting the report, while Bloomberg is sticking by their story.
The DHS statement reads in part:
The Department of Homeland Security is aware of the media reports of a technology supply chain compromise. Like our partners in the UK, the National Cyber Security Centre, at this time we have no reason to doubt the statements from the companies named in the story. Information and communications technology supply chain security is core to DHS's cybersecurity mission and we are committed to the security and integrity of the technology on which Americans and others around the world increasingly rely.
On Thursday, Apple contested the Bloomberg report, saying that their own internal investigation had found no evidence of the spy chips, and that neither the company nor their law enforcement contacts were aware of any FBI investigation into the matter.
Apple's recently retired general counsel, Bruce Sewell, told Reuters he called the FBI's then-general counsel, James Baker, last year after being told by Bloomberg of an open investigation of Super Micro Computer, a hardware maker whose products Bloomberg said were implanted with malicious Chinese chips.
"I got on the phone with him personally and said, 'Do you know anything about this?," Sewell said of his conversation with Baker. "He said, 'I've never heard of this, but give me 24 hours to make sure.' He called me back 24 hours later and said 'Nobody here knows what this story is about.'" -CNBC
Multiple Apple sources, "three of them very senior executives," told BuzzFeed News: "We tried to figure out if there was anything, anything, that transpired that’s even remotely close to this ... We found nothing."
A senior security engineer directly involved in Apple’s internal investigation described it as “endoscopic,” noting they had never seen a chip like the one described in the story, let alone found one. “I don’t know if something like this even exists,” this person said, noting that Apple was not provided with a malicious chip or motherboard to examine. “We were given nothing. No hardware. No chips. No emails.”
Equally puzzling to Apple execs is the assertion that it was party to an FBI investigation — Bloomberg wrote that Apple “reported the incident to the FBI.” A senior Apple legal official told BuzzFeed News the company had not contacted the FBI, nor had it been contacted by the FBI, the CIA, the NSA, or any government agency in regards to the incidents described in the Bloomberg report. This person’s purview and responsibilities are of such a high level that it’s unlikely they would not have been aware of government outreach. -BuzzFeed
The Bloomberg report claims that there is an ongoing government investigation into China's use of a "tiny microchip" that found its way into servers that were widely used throughout the US military and intelligence infrastructure, from Navy warships to DoD server farms. The probe began three years ago after the US intelligence agencies were tipped off by Amazon. And three years later, it remains ongoing.
Nested on the servers’ motherboards, the testers found a tiny microchip, not much bigger than a grain of rice, that wasn’t part of the boards’ original design. Amazon reported the discovery to U.S. authorities, sending a shudder through the intelligence community. Elemental’s servers could be found in Department of Defense data centers, the CIA’s drone operations, and the onboard networks of Navy warships. And Elemental was just one of hundreds of Supermicro customers.
During the ensuing top-secret probe, which remains open more than three years later, investigators determined that the chips allowed the attackers to create a stealth doorway into any network that included the altered machines. Multiple people familiar with the matter say investigators found that the chips had been inserted at factories run by manufacturing subcontractors in China.
As we previously reported:
The story begins with a Silicon Valley startup called Elemental. Founded in 2006 by three engineers who brilliantly anticipated that broadcasters would soon be searching for a way to adapt their programming for streaming over the Internet, and on mobile devices like smartphones, Elemental went about building a "dream team" of coders who designed software to adapt the super-fast graphics chips being designed for video gaming to stream video instead. The company then loaded this software on to special, custom-built servers emblazoned with its logo. These servers then sold for as much as $100,000 a pop - a markup of roughly 70%. In 2009, the company received its first contract with US defense and intelligence contractors, and even received an investment from a CIA-backed venture fund.
- Elemental also started working with American spy agencies. In 2009 the company announced a development partnership with In-Q-Tel Inc., the CIA’s investment arm, a deal that paved the way for Elemental servers to be used in national security missions across the U.S. government. Public documents, including the company’s own promotional materials, show that the servers have been used inside Department of Defense data centers to process drone and surveillance-camera footage, on Navy warships to transmit feeds of airborne missions, and inside government buildings to enable secure videoconferencing. NASA, both houses of Congress, and the Department of Homeland Security have also been customers. This portfolio made Elemental a target for foreign adversaries.
Like many other companies, Elementals' servers utilized motherboards built by Supermicro, which dominates the market for motherboards used in special-purpose computers. It was here, at Supermicro, where the government believes - according to Bloomberg's sources - that the infiltration began. Before it came to dominate the global market for computer motherboards, Supermicro had humble beginnings. A Taiwanese engineer and his wife founded the company in 1993, at a time when Silicon Valley was embracing outsourcing. It attracted clients early on with the promise of infinite customization, employing a massive team of engineers to make sure it could accommodate its clients' every need. Customers also appreciated that, while Supermicro's motherboards were assembled in China or Taiwan, its engineers were based in Silicon Valley. But the company's workforce featured one characteristic that made it uniquely attractive to China: A sizable portion of its engineers were native Mandarin speakers. One of Bloomberg's sources said the government is still investigating whether spies were embedded within Supermicro or other US companies).
But however it was done, these tiny microchips somehow found their way into Supermicro's products. Bloomberg provided a step-by-step guide detailing how it believes that happened.
- A Chinese military unit designed and manufactured microchips as small as a sharpened pencil tip. Some of the chips were built to look like signal conditioning couplers, and they incorporated memory, networking capability, and sufficient processing power for an attack.
- The microchips were inserted at Chinese factories that supplied Supermicro, one of the world’s biggest sellers of server motherboards.
- The compromised motherboards were built into servers assembled by Supermicro.
- The sabotaged servers made their way inside data centers operated by dozens of companies.
- When a server was installed and switched on, the microchip altered the operating system’s core so it could accept modifications. The chip could also contact computers controlled by the attackers in search of further instructions and code.
In espionage circles, infiltrating computer hardware - especially to the degree that the Chinese did - is extremely difficult to pull off. And doing it at the nation-state level would be akin to "a unicorn jumping over a rainbow," as one of BBG's anonymous sources put it. But China's dominance of the market for PCs and mobile phones allows it a massive advantage.
One country in particular has an advantage executing this kind of attack: China, which by some estimates makes 75 percent of the world’s mobile phones and 90 percent of its PCs. Still, to actually accomplish a seeding attack would mean developing a deep understanding of a product’s design, manipulating components at the factory, and ensuring that the doctored devices made it through the global logistics chain to the desired location - a feat akin to throwing a stick in the Yangtze River upstream from Shanghai and ensuring that it washes ashore in Seattle. "Having a well-done, nation-state-level hardware implant surface would be like witnessing a unicorn jumping over a rainbow," says Joe Grand, a hardware hacker and the founder of Grand Idea Studio Inc. "Hardware is just so far off the radar, it’s almost treated like black magic."
But that’s just what U.S. investigators found: The chips had been inserted during the manufacturing process, two officials say, by operatives from a unit of the People’s Liberation Army. In Supermicro, China’s spies appear to have found a perfect conduit for what U.S. officials now describe as the most significant supply chain attack known to have been carried out against American companies.
Some more details from the report are summarized below:
The government found that the infiltration extended to nearly 30 companies, including Amazon and Apple.
- One official says investigators found that it eventually affected almost 30 companies, including a major bank, government contractors, and the world’s most valuable company, Apple Inc. Apple was an important Supermicro customer and had planned to order more than 30,000 of its servers in two years for a new global network of data centers. Three senior insiders at Apple say that in the summer of 2015, it, too, found malicious chips on Supermicro motherboards. Apple severed ties with Supermicro the following year, for what it described as unrelated reasons.
Both Amazon and Apple denied having knowledge of the infiltration (Amazon eventually acquired Elemental and integrated it into its Amazon Prime Video service). Meanwhile, the Chinese government issued a conspicuous non-denial denial.
- In emailed statements, Amazon (which announced its acquisition of Elemental in September 2015), Apple, and Supermicro disputed summaries of Bloomberg Businessweek’s reporting. "It’s untrue that AWS knew about a supply chain compromise, an issue with malicious chips, or hardware modifications when acquiring Elemental," Amazon wrote. "On this we can be very clear: Apple has never found malicious chips, 'hardware manipulations’ or vulnerabilities purposely planted in any server," Apple wrote. "We remain unaware of any such investigation," wrote a spokesman for Supermicro, Perry Hayes. The Chinese government didn’t directly address questions about manipulation of Supermicro servers, issuing a statement that read, in part, "Supply chain safety in cyberspace is an issue of common concern, and China is also a victim." The FBI and the Office of the Director of National Intelligence, representing the CIA and NSA, declined to comment.
Bloomberg based its story on interviews with 17 anonymous sources, including 6 former government intelligence officials. One official told BBG that China's long-term goal was "long-term access" to sensitive government secrets.
- In all, 17 people confirmed the manipulation of Supermicro’s hardware and other elements of the attacks. The sources were granted anonymity because of the sensitive, and in some cases classified, nature of the information.
- The companies’ denials are countered by six current and former senior national security officials, who - in conversations that began during the Obama administration and continued under the Trump administration - detailed the discovery of the chips and the government’s investigation. One of those officials and two people inside AWS provided extensive information on how the attack played out at Elemental and Amazon; the official and one of the insiders also described Amazon’s cooperation with the government investigation. In addition to the three Apple insiders, four of the six U.S. officials confirmed that Apple was a victim. In all, 17 people confirmed the manipulation of Supermicro’s hardware and other elements of the attacks. The sources were granted anonymity because of the sensitive, and in some cases classified, nature of the information.
One government official says China’s goal was long-term access to high-value corporate secrets and sensitive government networks. No consumer data is known to have been stolen.
Notably, this revelation provides even more support to the Trump administration's insistence that the trade war with China was based on national security concerns. The hope is that more US companies will shift production of sensitive components back to the US.
- The ramifications of the attack continue to play out. The Trump administration has made computer and networking hardware, including motherboards, a focus of its latest round of trade sanctions against China, and White House officials have made it clear they think companies will begin shifting their supply chains to other countries as a result. Such a shift might assuage officials who have been warning for years about the security of the supply chain—even though they’ve never disclosed a major reason for their concerns.
As one government official reminds us, the extent of this attack cannot be understated.
- With more than 900 customers in 100 countries by 2015, Supermicro offered inroads to a bountiful collection of sensitive targets. "Think of Supermicro as the Microsoft of the hardware world," says a former U.S. intelligence official who’s studied Supermicro and its business model. "Attacking Supermicro motherboards is like attacking Windows. It’s like attacking the whole world."
But perhaps the most galling aspect of this whole scandal is that the Obama Administration should have seen it coming.
- Well before evidence of the attack surfaced inside the networks of U.S. companies, American intelligence sources were reporting that China’s spies had plans to introduce malicious microchips into the supply chain. The sources weren’t specific, according to a person familiar with the information they provided, and millions of motherboards are shipped into the U.S. annually. But in the first half of 2014, a different person briefed on high-level discussions says, intelligence officials went to the White House with something more concrete: China’s military was preparing to insert the chips into Supermicro motherboards bound for U.S. companies.
And thanks to Obama having dropped the ball, China managed to pull off the most expansive infiltration of the global supply chain ever discovered by US intelligence.
- But that’s just what U.S. investigators found: The chips had been inserted during the manufacturing process, two officials say, by operatives from a unit of the People’s Liberation Army. In Supermicro, China’s spies appear to have found a perfect conduit for what U.S. officials now describe as the most significant supply chain attack known to have been carried out against American companies.
The inconspicuous-looking chips were disguised to look like regular components but they helped China open doors that "other hackers could go through" meaning China could potentially manipulate the systems being infiltrated (as a reminder, these chips were found in servers used in the US drone program).
- The chips on Elemental servers were designed to be as inconspicuous as possible, according to one person who saw a detailed report prepared for Amazon by its third-party security contractor, as well as a second person who saw digital photos and X-ray images of the chips incorporated into a later report prepared by Amazon’s security team. Gray or off-white in color, they looked more like signal conditioning couplers, another common motherboard component, than microchips, and so they were unlikely to be detectable without specialized equipment. Depending on the board model, the chips varied slightly in size, suggesting that the attackers had supplied different factories with different batches.
- Officials familiar with the investigation say the primary role of implants such as these is to open doors that other attackers can go through. “Hardware attacks are about access,” as one former senior official puts it. In simplified terms, the implants on Supermicro hardware manipulated the core operating instructions that tell the server what to do as data move across a motherboard, two people familiar with the chips’ operation say. This happened at a crucial moment, as small bits of the operating system were being stored in the board’s temporary memory en route to the server’s central processor, the CPU. The implant was placed on the board in a way that allowed it to effectively edit this information queue, injecting its own code or altering the order of the instructions the CPU was meant to follow. Deviously small changes could create disastrous effects.
- Since the implants were small, the amount of code they contained was small as well. But they were capable of doing two very important things: telling the device to communicate with one of several anonymous computers elsewhere on the internet that were loaded with more complex code; and preparing the device’s operating system to accept this new code. <strong>The illicit chips could do all this because they were connected to the baseboard management controller, a kind of superchip that administrators use to remotely log in to problematic servers, giving them access to the most sensitive code even on machines that have crashed or are turned off.
- This system could let the attackers alter how the device functioned, line by line, however they wanted, leaving no one the wiser. To understand the power that would give them, take this hypothetical example: Somewhere in the Linux operating system, which runs in many servers, is code that authorizes a user by verifying a typed password against a stored encrypted one. An implanted chip can alter part of that code so the server won’t check for a password—and presto! A secure machine is open to any and all users.