Scapu hosts thousands of pieces of user content daily. Moderation is not a feature. It is the foundation of trust.
Traditional moderation techniques are too slow. Server-side processing adds roundtrip latency. We needed a faster, cheaper, and private solution.
The Fail-Fast Pipeline
The system follows a strict hierarchical order. We ask one question first. Can we prove this content is unsafe quickly?
The pipeline has four steps.
1. Local cache check. This takes less than 1 millisecond. If we have seen the content before, we know the verdict instantly.
2. Text scanner. This runs in 10 to 50 milliseconds. It catches over 90 percent of violations.
3. Visual scanner. This runs in 100 to 300 milliseconds. We only call this when the text passes inspection.
4. Manual moderation fallback. Humans step in when the model is uncertain.
This structure saves processing power. Since most of our content are text pubs, and most text pubs are safe, we avoid expensive image classification for the majority of use cases by scanning text first.
Layer 1: SmartTextModerator_v2
The text engine runs entirely on a browser-like environment. It normalizes user input and converts leetspeak to standard forms. For example, it maps @ to a, $ to s, and 3 to e. It also removes extra spaces and repeated characters.
The engine detects compound threats. Two safe words can become dangerous together. "Harm" and "you" are safe alone. "Harm you" is a threat. Our proximity detection finds these patterns when the words appear close to each other.
The engine also measures intensity. All-caps text increases the confidence score by 15 percent. Multiple exclamation marks add up to 20 percent. Repeated characters add 5 percent. This helps us catch borderline violations that might otherwise slip through.
Layer 2: Visual Intelligence with NSFWJS
For images and videos, we use TensorFlow.js. The model runs locally. No data leaves the browser. This protects user privacy and avoids compliance issues.
Images are stored on a custom Database. Browsers block cross-origin requests to the database. We built a proxy endpoint. The admin requests images through /api/proxy-image. The proxy fetches the image, adds CORS headers, and returns it to the client. This adds about 50 milliseconds of latency.
We decode images off the main thread. We use a Web Worker for decoding. This reduces UI jank by 73 percent. Users can scroll while images process in the background.
The Scoring Formula
The model returns five scores: P*rn, H*ntai, S*xy, Neutral, and Drawing. We turn these into a flag decision with a simple formula.
Threshold = 1 - sensitivity.
Flag if (P*rn + H*ntai) > threshold or S*xy > (threshold + 0.1).
We add a buffer for Sexy content. False positives on Sexy are more common and more frustrating. The buffer reduces those errors.
Administrators adjust sensitivity in real time. Strict mode lowers the threshold. Relaxed mode raises it. Communities choose their own tolerance.
The Cache Layer
We previously scaned every image every time, which was wasteful. Now we cache results in IndexedDB.
When content enters the viewport, we check the cache first. If we find a result, we update the UI instantly. No scan required. If not, we run the full pipeline and store the result.
Results are permanent unless the admin clears them. The cache makes repeated moderation tasks instantaneous.
We also preload the cache. When the dashboard loads, we fetch the first 100 results from the database. We warm up the cache before the user starts scrolling.
UI and UX Strategies
Our page size is 500 items. Scanning all 500 at once would freeze the browser. We use an Intersection Observer. We only scan content that is visible or within 200 pixels below the viewport. This reduces CPU usage by 85 percent.
The moderation console sits in the bottom-right corner. It shows live logs. Each item appears with its scores and verdict. Moderators see the AIs reasoning in real time.
Bulk operations are simple. The system indexes all flagged items. One click purges every flagged item in the current view. The operation is transactional and near-instantaneous.
Security Decisions
Content never leaves the admin's browser. The model runs locally. This prevents data leaks and also eliminates per-scan costs.
If the model fails to load, the system falls back to manual moderation and does not assume content is safe. When in doubt, we require human review.
User-generated content can include malicious regex patterns which can crash the browser. We escape every dynamic pattern using a strict function. This prevents ReDoS attacks.
What We Learned
We initially optimized for the happy path with the assumption that most content was safe. That worked until a wave of spam hit, which triggered a sampling mechanism. We detect changes in the distribution of safe versus unsafe content. If the sample shows more violations, we increase sensitivity automatically.
Caching creates a false sense of completeness as moderators assume cached means verified. To mitigate this, we currently differentiate three states in the UI. Verified by our custom model. When content is cached and needs re-scan, it is manually reviewed.
Since no moderation system is perfect, our system catches about 95 percent of violations; the remaining 5 percent fall into a grey area. We do not block them automatically and flag them for human review. This avoids false positives while maintaining a high safety bar.
The Road Ahead
We are currently working on version 3.0, which combines text and visual analysis. For example, an image of a person with text that says "call this number" might be safe alone. Together, they may constitute a scam. A Multi-modal analysis will catch these.
We are also building adaptive sensitivity, which will learn from moderator overrides. If moderators consistently unflag a certain type of content, the model will adapt.
Our team is optimizing for mobile and our early tests show mobile moderators are 30 percent faster than desktop. We will lean into this.
Final Note
Building this system was challenging and rewarding. It combines performance optimization, user experience, machine learning, and security. It makes Scapu safer for everyone.
Continuous improvement is an ever-evolving process, and we look to keep improving our system. Every day presents a new challenge and we have the right team to rise to the occasion.
Matthew Akinloye is a Founding Software Engineer at Scapu. He builds infrastructure that keeps millions of users safe and passionate about moderation systems.
Download Scapu today to join the movement.