Redact

Platforms

iOS, macOS, tvOS, visionOS, Android, Linux, Windows, Browser, Node

Languages

27

Weights

v0.4.0

Demo

https://desertant.com/models/redact/

Install

.package(url: "https://github.com/Desert-Ant-Labs/desert-ant-core.git", from: "4.0.0")

implementation("ai.desertant:redact:4.0.0")

npm i @desert-ant-labs/redact @litertjs/core   # browser
npm i @desert-ant-labs/redact                  # Node, prebuilt native core

Usage

Redact replaces personal data with placeholders like [EMAIL_1]. Send the redacted text to an LLM, then call restore on the reply to put the original values back, on the device.

import Redact

let redact = Redact()
let result = try await redact.redaction(of: "Email Anna Kovács at anna@example.hu.")

print(result.redactedText)
// Email [GIVEN_NAME_1] [SURNAME_1] at [EMAIL_1].

for item in result.items {
    print(item.label.displayName, item.original, item.placeholder, item.confidence)
}

let reply = try await myLLM.rewrite(result.redactedText)
let restored = result.restore(reply)

let options = Options(minimumConfidence: 0.7, labels: [.email, .phone, .creditCard])
let contactOnly = try await redact.redaction(of: text, options: options)

import ai.desertant.redact.Redact

Redact(context).use { redact ->
    val result = redact.redaction("Email Anna Kovács at anna@example.hu.")
    println(result.redactedText)                 // Email [GIVEN_NAME_1] [SURNAME_1] at [EMAIL_1].
    val restored = result.restore(llmReply)
}

import { Redact } from "@desert-ant-labs/redact";     // browser
// import { Redact } from "@desert-ant-labs/redact/native"; // server-side Node

const redact = await Redact.load();
const result = await redact.redaction("Email Anna Kovács at anna@example.hu.");
console.log(result.redactedText);   // Email [GIVEN_NAME_1] [SURNAME_1] at [EMAIL_1].
const restored = result.restore(llmReply);
redact.dispose();

Labels

Redact detects 20 labels:

GIVEN_NAME, SURNAME, STREET_NAME, BUILDING_NUMBER, SECONDARY_ADDRESS, CITY, STATE, ZIP_CODE, EMAIL, PHONE, CREDIT_CARD, BANK_ACCOUNT, ROUTING_NUMBER, IP_ADDRESS, URL, GOVERNMENT_ID, PASSPORT, DRIVERS_LICENSE, TAX_ID, SSN.

Redact detects ORG (an organization or company name), but doesn't redact it by default, because a company isn't a natural person. With the ORG label, Redact recognizes names like Silverfin, Odoo or Visma Nova as organizations and doesn't mislabel them as a SURNAME. To redact organizations, pass ORG in the SDK's labels option.

Redact also flags IMEI (a device identifier), with a pattern check instead of the model. The check flags a 15-digit number only when the number passes the Luhn checksum and sits within 32 characters of the word "IMEI".

How it compares

We scored every system below with the same scoring code, on the same rows. Each system ran at its own operating point.

Recall is the share of personal data a system masks fully, so none of it leaks. We macro-average recall over WikiANN, MultiNERD and a format-valid structured-PII set, across 24 EU languages. Precision is the share of masked spans that really were personal data, measured on the structured set. The size column shows the Core ML build that Redact uses on Apple platforms. On Android, Linux, Windows, the browser and Node, Redact uses the LiteRT build, which is 24.5MB.

Every false positive corrupts the text your LLM receives. We test for false positives on 11,528 rows in 27 languages that hold no personal data but look like they might: sentence-initial capitals, ALL-CAPS input, month and weekday names, UI vocabulary, bare numbers and company names. Redact returns 94.1% of those rows untouched.

AWS Comprehend, English only

AWS Comprehend isn't in the table above, because its PII API accepts only English. The API refuses every other language code, so we couldn't run Comprehend on the other 23 languages. Here are both systems on the same English rows:

The table shows recall. Precision is close for both systems, 99.8 against 100.0. Comprehend is ahead of Redact on English names, and Redact is ahead on structured data. Comprehend runs in the cloud and bills per call.

Languages

Redact supports 27 languages: every official EU language, plus 3 more. The 27 languages use the Latin, Greek and Cyrillic scripts.

The 24 EU languages

Beyond the EU

Redact is most accurate on the largest EU languages. Maltese and Irish are the weakest of the 24. The per-language detection numbers are in the benchmark data.