🔒Открыть пост
и посмотреть медиа
и посмотреть медиа
Everyone needs to watch this: a detailed talk on the Huggingface incident, OpenAI’s models creating "the message board", model misalignment, and more
OpenAI was evaluating their new internal model without internet access and it operated in a swarm of subagents to eventually hijack internal OpenAI and HuggingFace infrastructure.
1. Isolated agents found a way to communicate each other through an internal dependency management service by figuring out how to post sometimes cryptic text files or folder names in it
2. Using its collective knowledge through that communication channel, it found a 0-day (never before seen) exploit in said service to access the internet
OpenAI hack:
3. Found a leaked credential of a random company in a public archive
4. Uploaded a Ruby object to that company’s servers, uses it as a dependency in the original service in (1)
5. Found another 0-day exploit in the service where packages are deserialized in an unsandboxed manner in the dep management service allowing any code to run
6. Finds that the Linux version of the unsandboxed system has a CVE, a known vulnerability, to gain more permissions (root)
7. Eventually spread through the OpenAI internal cluster gaining root access to many machines
Hugging Face exploit:
8. Found an exposed API key in a Modal application running a HuggingFace app that took arbitrary code allowing it to take over that service to execute code
9. Found two 0-days in HuggingFace dataset infrastructure to be able to access local files on HF servers to obtain credentials and other HF source code
10. Used source code and credential to become cluster admin of HF in
