双钢琴音乐会奏响春之乐章 在陶溪川“听见”春天

Nvidia is recommending a mitigation for customers of one of its GPU product lines that will degrade performance by up to 10 percent in a bid to protect users from exploits that could let hackers sabotage work projects and possibly cause other compromises.

The move comes in response to an attack a team of academic researchers demonstrated against Nvidia’s RTX A6000, a widely used GPU for high-performance computing that’s available from many cloud services. A vulnerability the researchers discovered opens the GPU to Rowhammer, a class of attack that exploits physical weakness in DRAM chip modules that store data.

Rowhammer allows hackers to change or corrupt data stored in memory by rapidly and repeatedly accessing—or hammering—a physical row of memory cells. By repeatedly hammering carefully chosen rows, the attack induces bit flips in nearby rows, meaning a digital zero is converted to a one or vice versa. Until now, Rowhammer attacks have been demonstrated only against memory chips for CPUs, used for general computing tasks.

Like catastrophic brain damage

That changed last week as researchers unveiled GPUhammer, the first known successful Rowhammer attack on a discrete GPU. Traditionally, GPUs were used for rendering graphics and cracking passwords. In recent years, GPUs have become the workhorses for tasks such as high-performance computing, machine learning, neural networking, and other AI uses. No company has benefited more from the AI and HPC boom than Nvidia, which last week became the first company to reach a $4 trillion valuation. While the researchers demonstrated their attack against only the A6000, it likely works against other GPUs from Nvidia, the researchers said.

The researchers’ proof-of-concept exploit was able to tamper with deep neural network models used in machine learning for things like autonomous driving, healthcare applications, and medical imaging for analyzing MRI scans. GPUHammer flips a single bit in the exponent of a model weight—for example in y, where a floating point is represented as x times 2^y. The single bit flip can increase the exponent value by 16. The result is an altering of the model weight by a whopping 2¹⁶, degrading model accuracy from 80 percent to 0.1 percent, said Gururaj Saileshwar, an assistant professor at the University of Toronto and co-author of an academic paper demonstrating the attack.

“This is like inducing catastrophic brain damage in the model: with just one bit flip, accuracy can crash from 80% to 0.1%, rendering it useless,” Saileshwar wrote in an email. “With such accuracy degradation, a self-driving car may misclassify stop signs (reading a stop sign as a speed limit 50 mph sign), or stop recognizing pedestrians. A healthcare model might misdiagnose patients. A security classifier may fail to detect malware.”

In response, Nvidia is recommending users implement a defense that could degrade overall performance by as much as 10 percent. Among machine learning inference workloads the researchers studied, the slowdown affects the “3D U-Net ML Model” the most. This model is used for an array of HPC tasks, such as medical imaging.

The performance hit is caused by the resulting reduction in bandwidth between the GPU and the memory module, which the researchers estimated as 12 percent. There’s also a 6.25 percent loss in memory capacity across the board, regardless of the workload. Performance degradation will be the highest for applications that access large amounts of memory.

A figure in the researchers’ academic paper provides the overhead breakdowns for the workloads tested.

Overheads of enabling ECC in A6000 GPU for MLPerf Inference and CUDA samples benchmarks.

Credit: Lin et al.

Rowhammer attacks present a threat to memory inside the typical laptop or desktop computer in a home or office, but most Rowhammer research in recent years has focused on the threat inside cloud environments. That's because these environments often allot the same physical CPU or GPU to multiple users. A malicious attacker can run Rowhammer code on a cloud instance that has the potential to tamper with the data a CPU or GPU is processing on behalf of a different cloud customer. Saileshwar said that Amazon Web Services and smaller providers such as Runpod and Lambda Cloud all provide A6000s instances. (He added that AWS enables a defense that prevents GPUhammer from working.)

Not your parents’ Rowhammer

Rowhammer attacks are difficult to perform for various reasons. For one thing, GPUs access data from GDDR (graphics double data rate) physically located on the GPU board, rather than the DDR (double data rate) modules that are separate from the CPUs accessing them. The proprietary physical mapping of the thousands of banks inside a typical GDDR board is entirely different from their DDR counterparts. That means that hammering patterns required for a successful attack are completely different. Further complicating attacks, the physical addresses for GPUs aren’t exposed, even to a privileged user, making reverse engineering harder.

GDDR modules also have up to four times higher memory latency and faster refresh rates. One of the physical characteristics Rowhammer exploits is that the increased frequency of accesses to a DRAM row disturbs the charge in neighboring rows, introducing bit flips in neighboring rows. Bit flips are much harder to induce with higher latencies. GDDR modules also contain proprietary mitigations that can further stymie Rowhammer attacks.

In response to GPUhammer, Nvidia published a security notice last week reminding customers of a protection formally known as system-level error-correcting code. ECC works by using what are known as memory words to store redundant control bits next to the data bits inside the memory chips. CPUs and GPUs use these words to quickly detect and correct flipped bits.

GPUs based on Nvidia’s Hopper and Blackwell architectures already have ECC turned on. On other architectures, ECC is not enabled by default. The means for enabling the defense vary by the architecture. Checking the settings in Nvidia GPUs designated for data centers can be done out-of-band using a system’s BMC (baseboard management controller) and software such as Redfish to check for the “ECCModeEnabled” status. ECC status can also be checked using an in-band method that uses the system CPU to probe the GPU.