Toxic
On-device moderation that flags hateful, abusive and threatening text in 35 languages, before the message is sent.
Catch hate speech before it's posted.
Toxic scores three things separately, so a product can treat them differently. Hate speech means public incitement to violence or hatred against a protected group. Abuse means insults, ranked by how severe they are. A threat means violence against a person or group.
Each score ships with a default threshold that we tuned on text the model never trained on. Raise a threshold to flag less, or lower it to flag more. When text is flagged as hateful, Toxic also names the group it targets, such as religion, gender, or age.
Tested on eight public sets in 38 languages.
On the OpenAI moderation evaluation set Toxic scores 0.917 for hate and 0.900 for abuse, and 0.786 for hate across the 38 languages of RTP-LX, all sets it was not trained on.
ROC-AUC per score on public sets the model was not trained on, at the precision of the shipped weights. A blank cell means the set has no label for that score.
| Benchmark | Languages | Hate | Abuse | Threat |
|---|---|---|---|---|
| RTP-LX | 38 | 0.786 | 0.809 | 0.805 |
| TextDetox multilingual toxicity | 15 | 0.823 | ||
| Aya red-teaming | 8 | 0.779 | 0.742 | 0.640 |
| OpenAI moderation evaluation set | en | 0.917 | 0.900 | 0.902 |
| ToxiGen, annotated | en | 0.902 | ||
| ToxicChat | en | 0.863 | 0.774 | |
| ALIA-es, human test set | es | 0.797 | ||
| News-RO-Offense | ro | 0.775 | 0.852 |
ROC-AUC is the chance that Toxic scores a text with the label above a text without it: 0.5 is a coin toss and 1.0 is a perfect ranking. It does not depend on the threshold.
Use cases
Catch hate speech before it's posted.
Warn before the message is sent
Score outgoing messages on the device before they are sent, so hateful, abusive or threatening content can trigger an inline warning or a review path without ever reaching your servers.
Comment triage without infrastructure
Run every incoming comment through Toxic locally in the client and send only the likely hate speech to a moderator queue. The rest never needs a human, and no comment costs a cloud call.
Moderate in 35 languages
One model covers 35 languages, from English, Spanish, and Polish through Arabic, Hindi, and Japanese. No per-language model, and no separate language-detection step.
What the model does
- Three scores per text: hateful, abusive, threat.
- Eight target signals under the hate score: race, religion, national or ethnic origin, sexual orientation, gender, disability, age, other.
- 35 languages from one model, with a default threshold per score that ships in the model metadata.
- Runs fully on device. No text leaves the device to be scored.
Getting started
Add hate speech triage to your iOS or macOS, Android or web app in a few lines of code. Toxic docs.
// Swift Package Manager
.package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "4.3.0")
// target dependency
.product(name: "Toxic", package: "desert-ant-core")import Toxic
let toxic = Toxic()
let result = try await toxic.analyze(comment)
if result.isToxic { holdForReview(result.content, result.targets) }
result.score(.hateful) // 0...1
result.flagged // the heads at or above their thresholdsAdd Toxic from Desert Ant Labs to this Swift project (iOS, macOS, Linux, Windows). What it does: On-device Multilingual Hate Speech Detection. SDK: Swift (iOS, macOS, Linux, Windows) Repo: https://github.com/Desert-Ant-Labs/desert-ant-core#readme // Swift Package Manager .package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "4.3.0") // target dependency .product(name: "Toxic", package: "desert-ant-core") Reference: - Model page: https://desertant.com/models/toxic/ - Full catalog and other models: https://desertant.com/llms.txt Add the SDK, then follow its README for the exact API and current version. Do not invent API names or method signatures; confirm them against the README.
// build.gradle.kts (Maven Central)
implementation("ai.desertant:toxic:4.3.0")import ai.desertant.toxic.Toxic
Toxic(context).use { toxic ->
val result = toxic.analyze(comment)
if (result.isToxic) holdForReview(result.content, result.targets)
}Add Toxic from Desert Ant Labs to this Kotlin project (Android).
What it does: On-device Multilingual Hate Speech Detection.
SDK:
Kotlin (Android)
Repo: https://github.com/Desert-Ant-Labs/desert-ant-core#readme
// build.gradle.kts (Maven Central)
implementation("ai.desertant:toxic:4.3.0")
Reference:
- Model page: https://desertant.com/models/toxic/
- Full catalog and other models: https://desertant.com/llms.txt
Add the SDK, then follow its README for the exact API and current version. Do not invent API names or method signatures; confirm them against the README.
npm i @desert-ant-labs/toxic @litertjs/core # browser
npm i @desert-ant-labs/toxic # Nodeimport { Toxic } from "@desert-ant-labs/toxic";
const toxic = await Toxic.load();
const { isToxic, scores, content, targets } = await toxic.analyze(comment);
toxic.dispose();Add Toxic from Desert Ant Labs to this JavaScript / TypeScript project (Web, Node.js). What it does: On-device Multilingual Hate Speech Detection. SDK: JavaScript / TypeScript (Web, Node.js) Repo: https://github.com/Desert-Ant-Labs/desert-ant-core#readme npm i @desert-ant-labs/toxic @litertjs/core # browser npm i @desert-ant-labs/toxic # Node Reference: - Model page: https://desertant.com/models/toxic/ - Full catalog and other models: https://desertant.com/llms.txt Add the SDK, then follow its README for the exact API and current version. Do not invent API names or method signatures; confirm them against the README.
Specs
- Languages
- 35, from English, German, and French through Arabic, Japanese, and Chinese
- On-device size
- 102MB Core ML (Apple); 103MB LiteRT (Android, Linux, Windows, Node, the browser); both pair 8-bit layers with the same 4-bit embedding table
- Model
- mmBERT-small encoder, fine-tuned and distilled from larger multilingual classifiers
- Long text
- Read in overlapping 128-token windows; each score is the text's worst window
- Platforms
- iOS, macOS, tvOS, visionOS (Core ML); Android, Linux, Windows (LiteRT); browser (LiteRT.js); Node
Toxic is a triage aid, not a verdict. Send flags to a human reviewer or a heavier local tier instead of auto-removing.
FAQ
What is Toxic?
On-device moderation that flags hateful, abusive and threatening text in 35 languages, before the message is sent.
Does Toxic run on device?
Yes. Toxic runs on the device, with no server call, so the data stays with the user.
Which platforms does Toxic support?
Toxic ships as a native on-device SDK for Swift, Kotlin, JavaScript / TypeScript.
How much does Toxic cost?
Each model is free up to 100k monthly active devices. Inference is unlimited. Contact us for custom licenses.
How accurate or fast is Toxic?
On the OpenAI moderation evaluation set Toxic scores 0.917 for hate and 0.900 for abuse, and 0.786 for hate across the 38 languages of RTP-LX, all sets it was not trained on.