How Deepfake Detection Works: Three Layers Explained

No single method can reliably tell you whether an image, video, or audio clip is real. The generators improve every few months, and any one defense eventually develops blind spots. The solution is not a better single test. It is multiple tests, layered together, each catching what the others miss.

Key Takeaways

• Deepfake detection works best when three independent layers verify the same content: provenance (cryptographic proof of origin), AI classification (statistical analysis of pixels and frequencies), and biological signal analysis (temporal and physiological patterns in faces and voices)
• C2PA provenance is near-certain when present, but most content online does not carry it yet
• AI classifiers catch content from any source, including tools with no provenance. They are the broadest net but are locked in an arms race with generators.
• Biological analysis catches face swaps and reenactment deepfakes that pixel-level classifiers miss, by checking whether a person in a video moves, blinks, and speaks like a real human
• When all three layers agree, confidence is high. When they disagree, that disagreement itself is informative.

Layer 1: Provenance — Where Did This Come From?

Provenance is the chain of custody for a piece of media. Think of it like a receipt stapled to a photograph that records which tool created it, when, and every edit made afterward.

C2PA Content Credentials

The leading standard is C2PA (Coalition for Content Provenance and Authenticity), backed by Adobe, Microsoft, Google, OpenAI, and major camera manufacturers ^[1]. A C2PA-signed image carries cryptographically sealed metadata. If the image was generated by DALL-E, Firefly, or Gemini, the manifest says so. If someone edits the image afterward, that edit is recorded.

When C2PA metadata is present and valid, it provides a near-certain answer. No statistical guessing required.

Watermarks (SynthID and others)

A related approach is watermarking, which embeds an imperceptible signal directly in the pixel values rather than in metadata. Google's SynthID embeds statistical patterns into images and audio from Google's AI models ^[2]. Meta has deployed similar watermarking in their AI content.

The key difference: C2PA metadata can be stripped by re-saving or screenshotting. Watermarks survive these transformations because they live in the pixels themselves. SynthID detection is now available through Google's detector portal (with a waitlist), through the Gemini app, and through "About this image" in Chrome and Search. OpenAI has also extended SynthID to their generated content and opened API access for verification. Over 100 billion items have been watermarked as of mid-2026 across Google, OpenAI, ElevenLabs, and Kakao ^[2].

Where provenance falls short

According to the Content Authenticity Initiative, fewer than 5% of images circulating online carry any form of provenance metadata ^[3]. That means this layer alone leaves you blind to most of the media you encounter.

Provenance is powerful when present. But most content has none. That is where the other layers come in.

Layer 2: AI Classification — Does This Look Machine-Made?

AI classifiers examine the content itself and predict whether it was generated or manipulated by AI. This layer does not depend on the creator doing anything right. It works on any media, from any source, with or without metadata.

Modern classifiers operate at multiple levels:

• Pixel-level analysis detects artifacts in texture, lighting, and geometry that generators produce
• Frequency-domain analysis identifies statistical fingerprints left by specific generator architectures (GANs leave different traces than diffusion models)
• Compression-aware detection accounts for the degradation that happens when images move through social media and messaging apps

What classifiers catch

• AI-generated content from any source, including open-source tools with no provenance
• Content that has been stripped of all metadata
• Images from generators the model has been trained on (Midjourney, DALL-E, Stable Diffusion, Flux, and others)

Where classifiers fall short

Classifiers are the broadest net, but they have a specific blind spot: content where the pixels are partially real. Face swaps take a real video, replace only the face, and blend the result. The surrounding pixels are genuine. This is where Layer 3 becomes critical.

Layer 3: Biological Signal Analysis — Does This Person Behave Like a Real Human?

This is the layer that catches what pixel analysis misses. Instead of asking "do these pixels look machine-made," it asks "does this person move, blink, and speak like a real human being?"

Face swap and reenactment deepfakes are the fastest-growing category of harmful synthetic media. They are used in video call fraud (the Hong Kong $25M case), interview impersonation, non-consensual intimate imagery, and political disinformation. They are also the hardest to catch with pixel analysis alone, because much of the content is real.

Biological signal analysis examines:

• Temporal coherence across video frames. Real faces have consistent geometry as they move. Face swaps often produce subtle inconsistencies at the boundary between the swapped face and the original head, especially during rotation ^[5].
• Blinking patterns. Early deepfakes famously failed to blink realistically. Modern generators have improved, but the statistical distribution of blink timing and duration still differs between real and synthetic faces.
• Lip-audio synchronization. In reenactment deepfakes, the lip movements are driven by a different audio source. The alignment between phonemes (speech sounds) and mouth shapes can drift in ways that are imperceptible to humans but measurable by temporal models.
• Facial muscle dynamics. Real human expressions involve coordinated movement across dozens of facial muscle groups. Deepfakes often produce expressions that are individually plausible but temporally simplified, activating fewer muscle groups simultaneously than a real face would.

What biological analysis catches

• Face swap deepfakes where the surrounding pixels are genuine
• Reenactment videos where someone else's expressions drive the target face
• Lip-sync manipulations where audio has been replaced or altered
• Video call deepfakes where an impersonator drives a synthetic face in real time

Where biological analysis falls short

Biological analysis is narrow but deep. It only applies to content with faces, but on that content, it catches manipulations that pixel classifiers miss entirely.

Why All Three Layers Together

Each layer has gaps. But their gaps do not overlap.

Scenario A: AI-generated image from DALL-E

Two layers confirm. High confidence.

Scenario B: Deepfake from open-source Stable Diffusion, no metadata

Provenance cannot help, but the classifier catches it.

Scenario C: Face swap deepfake in a video call recording

This is the critical case. The classifier struggles because most pixels are real. Biological analysis catches the manipulation through temporal patterns that only appear across multiple frames.

Scenario D: Real photograph, heavily compressed via WhatsApp

All available layers agree the content is authentic, even after compression.

How Witness Uses All Three Layers

When you submit media to Witness, it runs all applicable checks:

1. Provenance check. Reads C2PA manifests and content credentials. If provenance confirms AI generation (for example, "created by Adobe Firefly" or "generated by DALL-E"), that is reported as a high-confidence signal before any statistical analysis runs.

1. AI classification. Detection models analyze the content for statistical patterns consistent with AI generation. This runs on every image and video, regardless of whether provenance data exists.

1. Biological signal analysis. On video containing faces, dedicated models analyze temporal patterns across frames to detect face swaps and reenactment. This layer activates automatically when faces are detected in the video.

The results are combined into a confidence assessment that accounts for agreement and disagreement across all active layers. A face swap detection from Layer 3 combined with inconclusive results from Layer 2 is reported differently than a case where all layers agree.

What This Means for You

You do not need to understand temporal coherence or C2PA manifests. The layers run automatically. What matters is this: the media you are worried about gets checked from multiple angles, each designed to catch what the others might miss.

If you have a photo that seems off, a video call recording that felt wrong, or an image you need to verify, you can check it at witness.vision/scan. Free, no account required.