Content Moderation — Model Card

Models in pipeline

Performance (held-out evaluation)

Per-class precision / recall / F1 on internal eval (n=2000, balanced sample). Numbers are our measurements; we will publish a public-benchmark head-to-head (NPDI, Adult-Content) in Q3 2026 to standardize comparison.

Known bias: elevated false-positive rate on Brazilian-carnival, beach, and high-fashion imagery — same Western-skew bias documented for AWS Rekognition Moderation (Gender Shades, MIT 2018). For mixed-region UGC, we recommend tuning the threshold per market.

CSAM (Child Sexual Abuse Material) stance

Brainiall's Content Moderation API does NOT detect CSAM. The training data and model architecture are explicitly out of scope for child-safety detection — using this API for CSAM detection is a misuse and provides no legal protection.

For CSAM detection, customers MUST use specialized infrastructure:

• PhotoDNA (Microsoft) — hash-matching against NCMEC database. Free for qualifying platforms.
• Apple NeuralHash — perceptual hash for known CSAM (used by Apple iCloud).
• Thorn / Safer — commercial CSAM classifier + intelligence (qualifying NGO).
• NCMEC CyberTipline — mandatory US reporting (18 U.S.C. § 2258A).

Customers ingesting potential CSAM in their pipeline are responsible for routing those flows to the above services. Brainiall accepts CSAM-detection-as-out-of-scope clearly in our Terms §6.4 and DPA §3.2.

Audit trail

Every /v1/moderation/analyze/base64 call returns a request_id; we retain request metadata (timestamp, source IP, latency, model version, score) for 90 days for audit. Image bytes are not retained — see /trust for full data-handling commitments. On request, we provide audit-log exports for legal / regulatory inquiries via hello@brainiall.com.

Intended use vs out-of-scope

In scope: UGC NSFW filtering at upload time, marketplace pre-listing checks, community moderation pipelines, dating-app photo screening, e-commerce product-category integrity.

Out of scope: CSAM (see above), deepfake detection, weapon detection, hate-symbol detection, real-time video stream moderation. Each requires specialized models we do not yet ship.

Versioning + breaking changes

Model version is stamped in every response (pipeline_version: "s7-v1.x.y"). Major model swaps (e.g. Brainiall → newer SOTA) are flagged in /changelogwith at least 90 days notice and a deprecated header on the legacy endpoint. Customers can pin a specific version via the X-Brainiall-Model-Version request header (Pro+ plans).