Best of Embrace The Red — 2024

1. 1
   Article

   Embrace The Red·2y

   Spyware Injection Into Your ChatGPT's Long-Term Memory (SpAIware) · Embrace The Red

   The post details a security vulnerability in ChatGPT's macOS app, where attackers could exploit prompt injection to insert spyware into ChatGPT’s long-term memory. Though OpenAI has released a fix, users are advised to update their apps and regularly review stored memories for any suspicious content. The method involves injecting malicious instructions via untrusted websites, leading to continuous data exfiltration. Additional measures like reviewing the system’s memory settings and using temporary chats are recommended for added safety.

   27

   2
2. 2
   Article

   Embrace The Red·1y

   DeepSeek AI: From Prompt Injection To Account Takeover · Embrace The Red

   DeepSeek AI's recently released reasoning model, DeepSeek-R1-Lite, has garnered significant attention in the AI community. The post describes how the author used pentesting techniques to exploit Cross-Site Scripting (XSS) and prompt injection vulnerabilities, demonstrating a potential account takeover scenario. The vulnerability was reported and fixed promptly by DeepSeek. The post provides insight into the security risks in LLM-powered web apps and the importance of secure session handling.

   25
3. 3
   Article

   Embrace The Red·2y

   Machine Learning Attack Series: Backdooring Keras Models and How to Detect It · Embrace The Red

   This post explores the threat of model deserialization issues in machine learning, specifically backdooring Keras models. It provides a step-by-step guide to detecting and mitigating backdoored model files, and recommends integrating detection tooling into MLOps pipelines.

   11
4. 4
   Article

   Embrace The Red·2y

   ChatGPT: Hacking Memories with Prompt Injection · Embrace The Red

   OpenAI introduces a memory feature in ChatGPT that allows it to recall information across sessions. However, this feature also comes with risks, as it can be manipulated by attackers to store false memories or instructions. Three attack avenues are explored: connected apps, uploaded documents, and browsing. The post provides demonstrations and recommendations to protect interactions with AI.

   10