Most engineers deploying confidential AI VMs seem to almost have no idea what's actually happening under the hood. Here's what AMD SEV does, and what it doesn't.
AMD SEV (Secure Encrypted Virtualization) is a hardware-based security technology integrated into AMD EPYC processors. Its primary purpose is to protect data in use by encrypting the memory of virtual machines (VMs).In traditional virtualization, the hypervisor (the software managing the VMs) has full visibility into the memory of every guest VM. This creates a security risk: if the hypervisor is compromised (or if the cloud provider is untrusted), the data inside your VM can be read or tampered with. AMD SEV solves this by cryptographically isolating the VM from the hypervisor. That sounds great right? I mean a marvel of computing security, fixing a major issue in computing.
You might be wondering what is the difference between AMD SEV and Intel SGX . Here is the schtick about AMD SEV imagine building a fortress around your entire house. Everything inside the house (the Operating System, all applications, and data) is safe from the outside world (the Cloud Provider/Hypervisor), then by some chance if a thief manages to get inside the house (e.g., a malware infection in your Guest OS), they can steal everything. Intel SGX on the the other hand is like placing a steel vault inside a room in your house. Even if the house has no walls and thieves (compromised OS or Hypervisor) are roaming freely, they cannot get into the vault. Only the specific items you put in the vault are safe. I bet you can already tell which one is favored by the cloud computing giants (of course it is AMD SEV) though Intel SGX can be prove to be quite difficult to use as you have build your entire application around it. Is it worth the trade of?
The answer to that is for you to decide, the big guys (Azure, Google Cloud, Nvidia) are pushing for it massively and not without reasons. Model Weights are too expensive to risk and Training Data is too regulated to expose. AMD SEV (paired with NVIDIA GPUs) provides the hardware-level guarantee that not even the computer owner can watch what the computer is thinking. You have to admit that is pretty good take it for example you and I own banks we want to train a model for fraud detection (I used banks so you understand that this is purely hypothetical) and we need a clean room where we can not even peek to see what is going on inside then sure AMD SEV solves that like a champ. And as stated through out this entire blog we all Know it is not all rainbows and sunshine.
I spent a week staring at AMD SEV's security model trying to find the weak point. It took longer than I expected the encryption layer is genuinely solid. But then I looked past the data and started watching the behavior. How long each operation takes. Which memory addresses get touched. The shape of the computation itself. SEV doesn't hide that. And for AI inference where the shape of the computation can leak information about the model or the input that's a problem that nobody in the confidential computing space is really addressing yet. Imagine we launch the fraud detection model, and someone does some findings goes to the room and notices how the room itself was designed and decorated they could get a pretty good idea of what was going on inside, take it like this if I balloons in a room I don't think it is going to be a funeral.
Top comments (0)