Types of content moderation APIs
We will quickly go through the key features of the moderation APIs in this section.
Note
A reference to the documentation for all APIs can be found at https://docs.microsoft.com/nb-no/azure/cognitive-services/content-moderator/api-reference.
Image moderation
The image moderation API allows you to moderate images for adult and inappropriate content. It can also extract textual content and detect faces in images.
When using the API to evaluate inappropriate content, the API will take an image as input. Based on the image, it will return a Boolean value, indicating whether the image is appropriate or not. It will also contain a corresponding confidence score between 0 and 1. The Boolean value is set based on a set of default thresholds.
If the image contains any text, the API will use OCR to extract the text. It will then look for the same adult or racy content as text moderation, which we will get to shortly.
Some content-based applications may not want to display any personally identifiable information, in which case it can be wise to detect faces in images. Based on the information retrieved in the face-detection evaluation, you can ensure that no user content contains images of people.
Text moderation
Using the text moderation API, you can screen text against custom and shared lists of text. It is able to detect personally identifiable information and profanity in text. In this case, personally identifiable information is the presence of information such as email addresses, phone numbers, and mailing addresses.
When you submit a text to be moderated, the API can detect the language used, if it is not stated. Screening text will automatically correct any misspelled words (to catch deliberately misspelled words). The results will contain the location of profanities and personal identifiable information in the text, as well as the original text, autocorrected text, and the language. Using these results, you can moderate content appropriately.