A plain-language glossary of content moderation terms

Short answer. Content moderation is the work of deciding what gets published in a shared space and what does not. Its vocabulary comes partly from the trust-and-safety teams of large platforms and partly from machine learning. The terms below are the ones a volunteer admin actually meets, each with a note on why it matters in practice.

Moderation has acquired a vocabulary of its own, partly from the trust-and-safety teams of large platforms and partly from machine learning. Much of it is useful once translated. This glossary explains the terms a volunteer admin is likely to meet, in the order of the alphabet, with a note on why each matters in practice.

A to C

Appeal. A request to have a moderation decision looked at again. Even a small page benefits from having a way to ask, if only an email address, because some decisions will be wrong.

Astroturfing. Coordinated activity designed to look like spontaneous public feeling. On a confession page it takes the form of many submissions pushing the same line, written to appear independent. See brigading.

Automated moderation. The use of software to screen content with no human involved, or before a human sees it. It is useful for sorting and for catching the obvious, and unreliable as a final judge. Explained in How AI content moderation works.

Blocklist. A list of words or phrases that cause a post or comment to be hidden or held. Quick to set up and blunt in effect: it catches innocent uses and misses creative spellings.

Brigading. A group organising in one place in order to descend on another — submitting, commenting or reporting en masse. The volume is meant to look like consensus.

CAPTCHA. A test intended to tell people from programs. Effective against bots; adds friction and can raise accessibility and privacy concerns.

Classifier. A model trained to give a score for how strongly content matches a category such as "harassment" or "sexual". It produces a number, not a decision.

Community guidelines. The published rules of a space. The term is used both for a platform's rules, which you must follow, and for your own page's rules, which you write.

Content warning. A short notice ahead of material some readers may find distressing, so that they can choose whether to read on. Helpful when used sparingly; ignored when placed on everything.

Context collapse. What happens when words written for one audience are read by another. A joke between friends, published to five thousand strangers, is no longer the same statement.

CSAM. Child sexual abuse material. Illegal to possess or distribute. It must be reported and never forwarded, even for the purpose of reporting it.

D to H

Decision log. A record of non-obvious moderation decisions and the reasons for them. The cheapest tool a team has for staying consistent.

Defamation. A false statement, communicated to others, that damages a person's reputation. In writing or other permanent form it is libel. See Defamation and confession pages.

Doxxing. Publishing someone's identifying or private information — address, workplace, phone number, identity documents — in order to expose them to harassment. A criminal offence in Malaysia since 2025 when done with that intent or effect.

Escalation. Passing a decision to someone with more authority or experience, or requiring a second person to agree. A team needs a clear list of what triggers it.

Evasion. Deliberate attempts to get round a filter: swapping letters for numbers, inserting spaces, using lookalike characters, or placing text inside an image.

False negative. Harmful content that a check failed to flag. The mistake you learn about from a complaint.

False positive. Harmless content that a check flagged wrongly. The mistake you learn about from an annoyed submitter.

Flag. A marker that content needs attention. It may be raised by software or by a user's report. A flag is a question, not an answer.

Hash matching. Comparing a file's digital fingerprint with a database of known harmful files. Large platforms use it to detect known abuse imagery without anyone having to view it.

Hate speech. Attacks on people on the basis of characteristics such as race, religion, nationality, gender, sexuality or disability. Definitions differ between platforms and between countries.

Human in the loop. A system design in which software makes recommendations and a person makes the final decision. The sensible default for anything with consequences.

I to N

Identifiability. Whether a person could be recognised from a piece of content by people who know them. It does not require a name.

Impersonation. Pretending to be a specific person or organisation. On a confession page this includes submissions written as though by someone else in order to embarrass them.

Jigsaw identification. Identifying a person by putting together several details, each harmless on its own. Course, year and society role, taken together, may point to exactly one person.

Large language model (LLM). The technology behind chat assistants. It can be given instructions in ordinary language and asked to judge content against them. Flexible, and capable of being confidently wrong.

Metadata. Information about a file as distinct from its content: when a photo was taken, on what device, and sometimes where. It can identify a person even when the image itself does not.

Moderation queue. The list of items waiting for a decision. The age of the oldest item in it is the best single measure of a team's health.

Named-person detection. Automated flagging of submissions that appear to contain a personal name, so that a human can consider whether someone is being identified. It detects that a name is present. It cannot judge whether publishing it is harmful.

Non-consensual intimate imagery. Sexual images shared without the consent of the person shown, including images that have been fabricated. Sometimes called "revenge porn", a term many people avoid because it implies that the victim did something to provoke it.

NSFW. "Not safe for work" — shorthand for sexually explicit or graphic content, and the usual label for classifiers that detect it.

P to S

Pile-on. A large number of people directing criticism or abuse at one person in a short time. Each participant sends a single message; the target receives hundreds.

Pre-moderation. Reviewing content before it is published. This is how confession pages work, and it is why the page's admins are so clearly responsible for what appears.

Post-moderation. Publishing first and reviewing afterwards, usually when something is reported. This is how comment sections normally work.

Proof of work. A small computational task that a browser must complete before a form will accept a submission. It is imperceptible to one person and costly to someone sending thousands. It limits abuse without identifying anybody.

Rate limiting. Restricting how often one source can perform an action within a period of time. The main defence against flooding.

Recall. In machine learning, the proportion of all harmful items that a check manages to catch. High recall means that little gets through; it usually comes with more false positives. (Separately, some submission tools use "recall" to mean a submitter withdrawing their own post.)

Precision. The proportion of flagged items that really were harmful. High precision means that flags can be trusted; it usually comes with more misses.

Republication. Repeating a statement that someone else originally made. In defamation law, the person who repeats a statement is responsible for it as though they had written it.

Safe messaging. Established guidance on discussing suicide and self-harm in ways that do not increase the risk to vulnerable readers: no methods, no sensational language, always with sources of help. See When a submission mentions self-harm.

Shadow banning. Limiting the visibility of a user's content without telling them. Controversial because it is, by design, not transparent.

Sockpuppet. A second account or identity used by one person to appear to be several. On an anonymous form, every flooder is effectively operating sockpuppets.

T to Z

Takedown. The removal of published content, usually in response to a request or complaint. See How to handle a takedown request.

Threshold. The score above which a classifier's output is treated as a flag. Lower it and you catch more and wrongly flag more; raise it and the reverse happens. There is no correct value, only a trade-off suited to your purpose.

Triage. Sorting incoming items by urgency and difficulty before dealing with them, so that attention goes where it is needed.

Trust and safety. The name large platforms give to the teams and policies that deal with abuse, harmful content and user protection. A volunteer admin team is doing the same job at a smaller scale.

Two-person rule. A requirement that two people agree before a sensitive action is taken. On a small team it is the main safeguard against a single person's bad day.

User-generated content (UGC). Material created by the users of a service and not by its operators. A confession page consists entirely of UGC that its admins have chosen to publish — and that choice is why they answer for it.

Vicarious trauma. The cumulative effect on moderators of repeated exposure to distressing material. It is real, it affects volunteers as well as professionals, and it is a reason to rotate duties and keep sessions short.

Using the vocabulary

Knowing the terms matters less than asking the questions behind them. When a tool claims to be "99% accurate", ask whether that figure refers to precision or recall, and on which languages it was measured. When someone says a post "wasn't naming anyone", ask whether the person was identifiable. And when a flood of submissions all say the same thing, ask whether you are looking at a community's opinion or at a brigade.

Common questions

What is a false positive in content moderation?

Harmless content that a check flagged wrongly — the mistake you learn about from an annoyed submitter. Its opposite, a false negative, is harmful content that a check failed to flag, which you learn about from a complaint.

What is brigading?

A group organising in one place in order to descend on another — submitting, commenting or reporting en masse. The volume is meant to look like consensus.

What is doxxing?

Publishing someone's identifying or private information — address, workplace, phone number, identity documents — in order to expose them to harassment. It has been a criminal offence in Malaysia since 2025 when done with that intent or effect.

What is the difference between pre-moderation and post-moderation?

Pre-moderation reviews content before it is published, which is how confession pages work and why their admins are so clearly responsible for what appears. Post-moderation publishes first and reviews afterwards, usually when something is reported, which is how comment sections normally work.

What is jigsaw identification?

Identifying a person by putting together several details, each harmless on its own. A course, a year and a society role, taken together, may point to exactly one person.

What do precision and recall mean for a moderation tool?

Recall is the proportion of all harmful items that a check manages to catch; precision is the proportion of flagged items that really were harmful. High recall usually comes with more false positives, and high precision with more misses, so a claim of "99% accurate" means little until you know which one it refers to.