5 min readJust now

–

Author: Berend Watchus. Independent non profit AI & Cyber Sec Researcher. [Publication for: OSINT Team, online magazine]

Press enter or click to view image in full size

Press enter or click to view image in full size

[… unless you add a cryptographic layer etc]

The Inevitability of Semantic Leakage: Why “Safe” Image Embeddings Were Never Really Safe

A new research paper exposes a fundamental vulnerability in how AI systems handle images. Companies have long claimed that storing compressed “embeddings” instead of actual images protects user privacy. This research systematically demonstrates why that claim was always misleading — and why it had to be.

Read the full paper: Semantic Leakage from Image Embeddings by Chen, Xu, Eliott, Li, and Bjerva (2026).

What Are Embeddings? A Simple Explanation

Meet Harry, Bob, and Mary — three people using AI services:

Harry uploads photos to an AI service. The service doesn’t store his actual photos. Instead, it converts images into “embeddings” — lists of numbers like: [0.23, -0.81, 0.45, 0.12, ...] with hundreds or thousands of values.

The company tells Harry: “We only store these numbers, not your photos. Your privacy is protected!”

Bob shares medical imaging data with a platform. They convert his X-rays into embeddings and promise: “These are just mathematical representations, not your actual images.”

Mary uses a learning app that creates embeddings of her study patterns and shares them with researchers: “These are compressed, anonymized representations. Nobody can identify you.”

The Problem

These number lists aren’t random. They’re designed to preserve meaning. Similar photos get similar numbers. Beach photos cluster together. Hospital photos cluster together. Tuesday evening study sessions cluster together.

And if the numbers preserve meaning, then someone can extract that meaning back out — without reconstructing the original image.

What The Researchers Found

The researchers developed SLImE (Semantic Leakage from Image Embeddings) and tested it on GEMINI (Google), COHERE, CLIP (OpenAI), and NOMIC — major embedding systems that power countless AI services.

They proved that from embeddings alone, they could extract:

• Text descriptions matching original captions with 50%+ accuracy
• Objects, relationships, and scene context
• Semantic information that persists even after multiple transformations

Most alarmingly: this works with minimal information — even a single data point enables substantial extraction.

Read the technical details in the full paper

Real-World Impact: The Services You Actually Use

GEMINI embeddings (Google): Harry builds a photo app using Google’s GEMINI embedding API. Google claims: “We only process embeddings, not store actual photos.”

But the research proved that from GEMINI embeddings alone, they could extract descriptions like “beach, family, children, outdoor setting.” If users post similar content publicly, someone could match those posts to “anonymous” embeddings and learn about all their other photos.

CLIP embeddings (OpenAI): Bob’s medical imaging startup uses CLIP. They share embeddings with researchers: “Just mathematical representations, not actual X-rays.”

The researchers extracted semantic information like “chest cavity imaging, medical scan, anatomical patterns” — recovering meaningful medical information without reconstructing pixels.

COHERE embeddings: Mary’s learning platform uses COHERE’s API and shares embeddings with researchers: “Fully anonymized learning patterns.”

The research showed they could recover “mathematics content, repeated review patterns, late-night sessions” — exactly what trains student-risk algorithms.

NOMIC embeddings: A startup uses NOMIC’s open-source model and shares “anonymized” embeddings with partners.

The researchers proved NOMIC leaks semantic information just like the proprietary models.

The verdict: Across ALL four systems, semantic leakage is consistent and fundamental.

My Take: This Was Always Obvious

Here’s what frustrates me: of course embeddings leak semantic information. How could they not?

This isn’t a bug — it’s the entire point.

Words have meanings. You cannot create a representation that enables semantic search while stripping out all meaning. That’s a logical contradiction.

Maps show routes. A navigation system must preserve spatial relationships to be useful. You cannot know “these locations are near a hospital” while knowing nothing about locations or hospitals.

Embeddings encode semantics. They’re explicitly optimized for semantic retrieval — trained to keep similar content close together. That’s their job specification.

The researchers formalize this beautifully: embeddings preserve “local semantic neighborhoods.” Similar images cluster together numerically. This clustering IS the utility. It’s also the vulnerability.

The Core Contradiction

You cannot optimize for semantic preservation and semantic privacy simultaneously. These are contradictory objectives.

• Better semantic preservation = better AI performance = worse privacy
• Better privacy = worse semantic preservation = worse AI performance

There is no technical magic that escapes this trade-off. You cannot preserve “useful semantic relationships” while eliminating “privacy-relevant semantic relationships” — because they’re the same relationships.

The researchers show this empirically: even after multiple lossy transformations (alignment, retrieval, discretization, caption generation), semantic information persists. Why? Because semantic structure is the load-bearing element of the system.

If you compress a map so thoroughly you can’t determine which roads lead where, you no longer have a useful map. If you compress an embedding so thoroughly it no longer preserves semantics, you no longer have a useful embedding. The utility IS the vulnerability.

What This Means Practically

Companies claiming “we only store embeddings, so your privacy is protected” are either:

1. Fundamentally misunderstanding their own technology
2. Deliberately misleading users
3. Both

Embeddings are not anonymization. The paper proves this across GEMINI, COHERE, CLIP, and NOMIC. The vulnerability isn’t specific to one company — it’s fundamental to how ALL embedding systems work.

The technology cannot be “fixed.” This isn’t a bug that can be patched. Differential privacy and cryptographic techniques exist, but they fundamentally degrade performance — because they must degrade semantic preservation to improve privacy.

The Way Forward

I appreciate that the researchers are exposing this systematically, even if it confirms what should have been theoretically obvious. We need empirical validation because companies won’t change their marketing claims based on theoretical arguments alone.

The solution: Honest communication about trade-offs. Stop claiming embeddings provide strong privacy protection. They don’t. They can’t. The mathematics doesn’t allow it. It’s inherent to the very concept of a descriptive code — if the code describes something accurately enough to be useful, it reveals information about what it describes. You cannot have a description that both captures meaning and hides meaning simultaneously.

If you’re using embedding-based systems for sensitive data:

• Understand you’re trading privacy for functionality
• Implement additional protections (differential privacy, secure enclaves, access controls)
• Accept that these protections will reduce performance
• Be transparent with users about these trade-offs

The semantic leakage isn’t a security failure — it’s a feature working exactly as designed. The failure is in pretending otherwise.