The Moderation Service API provides endpoints for managing and enforcing content moderation rules across various types of messages. This API enables you to create, read, update, and delete (CRUD) rules and keyword lists, ensuring your platform maintains a safe and compliant environment for users.
Below is an overview of the key functionalities provided by the Moderation Service API:
- The Rules Management endpoints allow you to define and manage moderation rules to handle inappropriate content based on various conditions
- The Keywords Management endpoints allow you to create and manage lists of keywords or regex patterns used for content moderation.
- The list blocked messages endpoint allows you to fetch all the violated messages.
- Fetches the historical revisions of a rule, detailing all updates made over time.
The default rules include text profanity, AI-based moderation for images and videos, custom profanity (custom messages), contact details, and email content, ensuring comprehensive content moderation.
The text profanity and custom profanity filters scan for words, including those matching regex patterns, and drop the message if a violation is detected.
The AI-based image and video moderation system scans for violations in images and videos and discards the message if a violation is detected. It allows the attachment of the following labels to a rule, enabling moderation for specific categories of images and videos.
Following are the labels that are available for Image/Video moderation:
| Labels | Definitions |
|---|---|
| Violence | Instruments or devices used to cause harm or damage to living beings, structures, or systems. This includes firearms (e.g., guns, rifles, machine guns, etc.), sharp weapons (e.g., swords, knives, etc.), explosives and ammunition (e.g., missiles, bombs, bullets, etc.). |
| Alcohol | The act of drinking alcoholic beverages from bottles or glasses of alcohol or liquor. |
| Gambling | The act of participating in games of chance for a chance to win a prize in casinos, e.g., playing cards, blackjacks, roulette, slot machines at casinos, etc. |
| Drugs & Tobacco | Small, solid, often round or oval-shaped tables or capsules. The act of inhaling, exhaling, and lighting up burning substances including cigarettes, cigars, e-cigarettes, hookah, or joint. |
| Rude gestures | Visual depiction of a hand gesture with middle finger is extended upward while the other fingers are folded down. |
| Explicit Nudity | Human male/female genitalia, including the penis (whether erect or flaccid), the scrotum, and any discernible pubic hair. |
| Non-Explicit Nudity | Human posterior part where the majority of the skin is visible from the neck to the end of the spine. This term does not apply when the individual's back is partially or fully occluded. |
| Swimwear or underwear | Human clothing for female/male swimwear (e.g., one-piece swimsuits, bikinis, tankinis, swim trunks, boardshorts, swim briefs, etc) |
| Visually Disturbing | Human bodies that are extremely thin and undernourished with severe physical wasting and depletion of muscle and fat tissue. |
| Hate symbols | Visual depiction of symbols, flags, or gestures associated with Nazi Party. Images containing extremist and terrorist group flags. |
When a rule is violated, the message will be blocked, and there is an option to either kick or ban the user in groups, or block the user in one-on-one interactions.