One moderation stack, four engines — and a report for every frame
We moved every moderation model into a single service, and opened up what each engine scored on each image and each video frame.
Until this week VibeFlow ran its moderation models in two places. The submission app screened text; a separate detector service screened images and video. Both grew their own copy of the same taxonomy, and the two drifted — a rule added on one side never reached the other.
Everything now runs in one service. A submission is screened by four independent engines, all of them on every post:
- A named-person pre-check that runs locally at no API cost — honorifics (Dr, Prof, Encik, Ustaz, Cikgu), bin/binti lineage markers, Malaysian Chinese and Indian surnames, and Chinese character name sequences.
- OpenAI omni-moderation-latest, for category-level harm scores.
- Meta Llama 4 Scout, which reasons about whether a name is being used to defame someone and whether a claim is unverifiable — the two judgements that make up most of what actually gets reported on a confession page.
- An on-premise NSFW classifier running on our own hardware, over every image and every sampled video frame.
None of these is a standby for another. They run in parallel, and any one of them flagging holds the post for review.
The bigger change is what you can now see. The detector was always scoring every frame of a video — but the admin panel only showed a single verdict, so a clean pass looked identical to moderation never having run. Your moderation panel now shows the evidence: per image, and per video frame, what the NSFW classifier scored, what Scout vision said, and what omni-moderation returned. A video clip expands into a frame-by-frame table.
That matters most on the borderline cases. When a post sits at 38% NSFW on one frame and 2% on the rest, you can see exactly which frame to look at instead of guessing.
What happens after the engines have run
A flag does not delete anything. A flagged submission is set aside for the page's admins, who are notified, and it stays there with its evidence until one of them decides. A submission that nothing flagged waits in the ordinary queue for an admin to approve it.
There is one setting that changes this. A page can turn on Auto Publish, and then a submission that passes every engine is published without an admin reading it first. It is off by default. The engines are good at what is explicit and poor at what is merely unkind, so a clean pass means that nothing obvious was found — not that the post is fit to publish. Our guide on how AI moderation works, and where it fails, goes through the failure cases one by one.