Poisoning the Well

Introduction

This is basically a writeup on a structural vulnerability in large-scale behavioral data collection - the kind that doesn't require breaking any systems, bypassing any auth, or doing anything that looks like an attack. The vuln is simpler than that: the signal can be poisoned, and once it is, the whole system degrades in ways that are hard to reverse.

Large-scale data collection - the kind Meta runs - works because it assumes the behavioral data it's ingesting reflects human preferences. That assumption is load-bearing. Pull it out and the targeting precision collapses, the model weights get contaminated, and you end up with a surveillance system that's increasingly blind to what actual people actually want (This is the goal more or less with this idea).

The interesting part is that this isn't theoretical. It's already partially happening at baseline - before any coordinated effort. The math below is just tracing how far along the curve we already are, and how little it takes to push it further.

Meta's (or any collection company in this particular case) data collection methods rests on a single assumption: that its behavioral data reflects actual human preferences. The entire value of behavioral surveillance rests on the precision of that signal.

This post argues that precision is mathematically fragile, already partially collapsed at baseline, and sits within striking distance of total failure.

1 The Baseline

Meta's own SEC filings disclosed that false accounts represented approximately 4-5% of worldwide MAU as of Q4 2022, with duplicate accounts adding another 11%. [1]

Meta stopped disclosing these figures in SEC filings after FY2023. [2]

Despite removing 3.5 billion fake accounts in 2025 alone and 38 billion over eight years - the false account percentage remained stable. [3]

The creation rate matches the removal rate. The system is at steady state saturation.

2 The Signal to Noise Model

Define these variables:

• N = total accounts
• S = synthetic accounts
• p = S/N (synthetic ratio)

Signal-to-noise ratio:

SNR = (1 - p) / p

SNR alone understates the damage. Targeting accuracy does not degrade linearly.

3 The Targeting Accuracy Degradation Model

Advertiser confidence collapses nonlinearly, so it behaves more or less like a phase transition. Modeled as an inverted logistic function:

s(p) = 1 / (1 + e^(20 x (p - 0.15)))

Where:

• p_c = 0.15 is the critical threshold (inflection point of collapse)
• k = 20 (note that this is a modeling choice) captures steepness of the transition, consistent with ML model robustness literature on label noise sensitivity [5]

4 Detection Will Not Save It

Meta claims 99.7% proactive detection of fake accounts. This figure applies to low-quality bots with high behavioral distinguishability (a).

For coordinated high quality synthetic accounts - residential IP, device fingerprint, human-plausible behavioral patterns etc, a is low.

P_d(a, p) = 0.997 x a x (1 - p / p_max)

Where p_max = 0.30 is the saturation point where detection breaks down from resource exhaustion.

At a = 0.10 (note that this is high quality accounts):

Effective synthetic ratio after detection:

p_eff = p x (1 - P_d)

At p_injected = 0.15:

p_eff = 0.15 x (1 - 0.050) = 0.1425 s(0.1425) = 0.537

Detection saves Meta in particular approximately 3.7 percentage points of accuracy at the critical threshold. It does not prevent collapse.

5 The Irreversibility Problem

The above models current targeting accuracy, the particular problem is that poisoned training data does not leave when accounts are removed.

Model contamination follows a sort of a leaky integrator - the same mathematics as viral spread in epidemiology:

dM/dt = y x p x (1 - M) - L x M

At steady state:

M*(p) = R_c x p / (1 + R_c x p)

Where R_c = y/L is the contamination reproduction ratio.

Estimated from Meta's continuous retraining architecture: [4]

• y = 0.80 (contamination absorption rate, high due to retraining)
• L = 0.05 (natural decontamination rate, low - cause of old weights)
• R_c = 16    M*(p) = 16p / (1 + 16p)

At Meta's current baseline of p = 0.05, 44% of model weights are already contaminated by synthetic behavioral patterns - and this is before any coordinated injection effort. [4]

6 Combined Model

s_total(p) = s(p_eff) x (1 - M*(p))

The combined degradation effect on targeting precision:

7 Error Margin

All parameters carry a ±11% uncertainty range. The table below shows s_total at baseline (p = 0.05) and critical threshold (p = 0.15) across best and worst case scenarios for Meta.

Under no reasonable parameter combination does Meta's targeting exceed 55% of theoretical maximum at current synthetic account levels.

The error bars here do not save Meta from this.

8 Finding

At Meta's current disclosed baseline of p = 0.05, s_total = 0.497. [1]

Meta's behavioral targeting is already operating at approximately 50% of theoretical maximum accuracy, which is funny. A surveillance system that is already half blind at baseline.

Distance from current state to critical threshold:

delta_p = 0.15 - 0.05 = 0.10

Accounts needed to cross it:

S = 0.10 x 3.58B / 0.90 = 398 million accounts

Once the threshold is crossed, recovery requires approximately 12 months of clean data minimum - M decays as e^(-L x t) with L = 0.05 per training cycle and targeting precision does not recover on the same timescale as the mathematics.

9 This is happening at scale already

Meta's fake account removal rate already implies the injection threshold is being approached organically - just without the explicit goal of data quality degradation. [3]

The idea here is reframing the question: what happens when coordinated synthetic injection targets advertising data quality specifically, as a privacy protection mechanism, rather than for influence operations or spam?

The mathematics here suggests that behavioral surveillance stops being technically viable as a privacy invasion mechanism.

Summary

Basically the idea that I am trying to convey is that, when bot/fake accounts increases, it poisons the datasets companies use to sell data to advertisers, hence, increasing user privacy by noise - it is very hard to handle datasets that are filled with "false" data and sell them, this creates a dilemma, deleting all the data collected after a specific date/instance, actual user data gets deleted as well, creating a deadlock for these companies.

The author is not responsible for exploitation or damages caused by any use of the idea in any way.