Toxic
On-device moderation that flags hateful, abusive and threatening text in 23 European languages, before the message is sent.
Catch hate speech before it posts.
Toxic scores three things separately, so a product can treat them differently. Hate speech means public incitement to violence or hatred against a protected group. Abuse means insults, ranked by how severe they are. A threat means violence against a person or group.
On Multilingual HateCheck, a public suite of hand-written examples, Toxic gets about 85 out of 100 across the seven EU languages. Mistral's Shieldstral gets 79 and wants a 16GB GPU; the guard models sized for a phone get 65.
0.847 on real hate speech, from 80MB.
0.847 macro F1 on real Multilingual HateCheck across the seven EU languages it covers, from an 80MB model that runs on device. The closest guard model through the same harness is Mistral's Shieldstral-1.0-3B at 0.786, and it wants a 16GB GPU; the ones sized for a phone score about 0.65.
Measured on the weights that ship, not extrapolated from a training checkpoint; three runs of the same recipe average 0.834. The threshold is 0.40 for the hate score and 0.50 for the other two. Out of domain, on corpora with no shared lineage: 0.724 on textdetox across 5 languages and 0.658 on offenseval2020 across 3. The seven EU languages are scored on real Multilingual HateCheck; the other 15 are scored on synthetic sets, because HateCheck has no real set for them, and those figures are development estimates rather than comparable scores. Latency has not been measured yet.
Use cases
Catch hate speech before it posts.
Warn before the message is sent
Score outgoing messages on the device before they are sent, so hateful, abusive or threatening content can trigger an inline warning or a review path without ever reaching your servers.
Comment triage without infrastructure
Run every incoming comment through Toxic locally in the client and send only the likely hate speech to a moderator queue. The rest never needs a human, and no comment costs a cloud call.
Moderate in 23 European languages
One model file covers 23 languages, from English, Spanish, and Polish through Bulgarian, Estonian, and Irish. No per-language model, and no separate language-detection step.
What the model does
- Three scores per text: hateful, abusive, threat.
- Ten target signals under the hate score: race, color, religion, descent, national or ethnic origin, sexual orientation, gender, disability, age, other.
- 23 languages from one model file, with a recommended setting per language in the model metadata.
- Runs fully on device. No text leaves the device to be scored.
Specs
- Languages
- 23, from English, German, and French through Greek, Lithuanian, and Irish
- On-device size
- 80MB Core ML (4-bit); 91MB LiteRT (int4); 100MB ONNX (browser)
- Model
- XLM-R-base, 12 layers, 159.6M params
- Accuracy
- 0.847 macro F1 on the 7 EU languages in Multilingual HateCheck, shipped weights; 23 languages supported
- Platforms
- iOS and macOS (Core ML), Android (LiteRT), browser (ONNX); weights on Hugging Face, not in the SDK yet
Toxic is a triage aid, not a verdict. Send flags to a human reviewer or a heavier local tier instead of auto-removing.
Toxic reacts to hateful words even when someone is quoting or condemning them, so counter-speech and news reporting can trip the flag. At the default setting it flags 26 to 30% of the non-hateful text in the seven EU languages it was measured on, and 12% in English.
Toxic separates hate speech from ordinary text well, and hate speech from plain rudeness much less well, so an angry customer can read as hateful.
FAQ
What is Toxic?
On-device moderation that flags hateful, abusive and threatening text in 23 European languages, before the message is sent.
Does Toxic run on device?
Yes. Toxic runs on the device, with no server call, so the data stays with the user.
Is Toxic available yet?
Toxic is in closed beta. Request early access on this page.
How much does Toxic cost?
Every model is free up to 100k monthly active devices per SDK. Unlimited inference per user. Contact us for custom licenses.
How accurate or fast is Toxic?
0.847 macro F1 on real Multilingual HateCheck across the seven EU languages it covers, from an 80MB model that runs on device. The closest guard model through the same harness is Mistral's Shieldstral-1.0-3B at 0.786, and it wants a 16GB GPU; the ones sized for a phone score about 0.65.
Early access
Tell us what you are building and we will get you set up.