Interactive Audio Moderation

The Jocloud interactive audio moderation helps to moderate audio streams in real time particularly for Jocloud interactive audio solutions. Specifically, it can detect improper information in audio streams and can provide management and control suggestions.
Once a room (roomId) moderation task starts, the system pulls audio data from Jocloud transport system and performs segmented moderation periodically.

1. Introduction

The Jocloud interactive audio moderation service is used to detect illegal information of real-time audio streams, and an asynchronous callback returns an illegal result.
The followingmoderation category action and corresponding labelare provided:

Moderation TypeDescriptionActionLabel
Sexual panting recognitionDetect voiceprint features in audio files, and recognize illegal features such as sexual pantingpornnormal: normal
moan: Sexual panting
Sensitive word recognitionTranslate text information of audio files, and recognize illegal sensitive contentsantispamnormal: normal
terrorism: terrorism
porn: porn
illegal: illegal
politics: politically sensitive contents
abuse: abuse
ad: advertisement cheating
feudalism: feudalism
religion: religiously sensitive contents
affairs: affairs
contraband: contrabands
minors: minors
banned-website: banned websites

2. Restrictions

Restriction CategoryDescription
Saving durationThe system will save suspected illegal audio clips, and return URL of files and a detection result back to the user.
The saving duration is 3 hours, and the files may become invalid beyond that duration. Please export them in time.
Duration limitAn individual live streaming detection task can last for 24 hours at maximum.
Concurrency restrictionEach user’s real-time stream detection tasks run simultaneously cannot exceed 200 by default. Please contact customer supports for more concurrency supports.
Stream pulling timeoutThe task will stop when the live streaming is stopped or the stream pulling cannot be implemented for over 10 minutes due to other causes; and this event will be returned to the service side for determining whether to initiate the task request again.

3. APIs

3.1 Start a Task

Initiate a Request

ItemDescription
Request MethodPOST
Request Protocol HTTPS
Request Domain Nameai.jocloud.com
Request Pathapp/{appId}/v1/audio/stream/start?traceId=xxxxx
Request ParametertraceId is a uuid string for fault locating during troubleshooting. You should use different values for requests.
Request HeaderContent-Type: application/json;charset=UTF-8
token: Authentication token, generated on the basis of[identity authentication](/cn/product_category/content_audit_service/integration_and_start/integration_and_start.md#3-Identity authentication)
Request Bodyjson string, defined as follows

Request Content

NameData TypeRequiredDescription
roomIdStringYesRoom ID, unique identifier for a moderation task
uidsString arrayNoUser ID list of moderated users. If the list is empty, all users in the roon are being moderated.
sceneIdIntegerNoScenario ID customized by the developer and used for subdividing service scenes
contextJSONNoContext of the service side, which is passed through during callback
actionsString arrayYesModeration category, including the following values:
- porn: audio feature moderation (sexual panting recognition)
- antispam: audio moderation (sensitive word recognition)
statusCallbackStringNoHTTP callback address for moderation status (HTTP and HTTPs protocols are supported)
resultCallbackStringNoTTP callback address for moderation result (HTTP and HTTPs protocols are supported)
sequenceStringNoThis value is used for a signature in a callback notification request. This field is required during the callback. See result callback description for detailed application method.
  1. Moderation tasks are identified by roomId uniquely. If the same roomId has started moderation tasks for multiple times, the system will return an error.
  2. In this case, suspend existing moderation tasks and then update the task moderation configuration.

Response Content

Data in a respond message body is JSON, and the specific fields are described as below:

NameTypeRequiredDescription
codeIntegerYesError code that is identical with HTTP status code and is also extended
- 2xx indicates success
- 4xx indicates request error
- 500 indicates server error
See related descriptions for specific values
messageStringYesError message description
traceIdStringYestraceId content in the pass-through parameter
roomIdStringYesRoom ID, unique identifier for a moderation task
sceneIdIntegerNoScenario ID customized by the developer and used for subdividing service scenes
contextJSONNoContext in the corresponding request
timestampIntegerYesCurrent unix timestamp in seconds

3.2 Result Callback

Callback Time

  1. When the real-time stream involves illegal information, call an address in resultCallback to notify the user side of the moderation result;
  2. When the task has abnormal situations, such as stream pulling failure or decoding failure, call an address in statusCallback to notify the user side of the task status.
  • When starting the task, resultCallback and statusCallback can be null. In this case, you can use the search API to get task status and result.
  • If you need to use the callback method to get result and task status, make sure the HTTP callback service is stable;
  • When the access to a callback address fails or the returned status code is not 200, Jocloud background prompts Try Again for no More Than 5 Times. .

Callback Method

Use ** HTTPS POST** method to access the URL provided by the user, and the status and result are filled in the HTTP request body in a JSON string format. See the next section for specific formats.
Add one checksum item in header of HTTP request to validate whether the contents are legal, preventing the contents from tampering.

A checksum string is generated based on the following steps:

Generate a checksum value based on sequence + body string data in task starting parameters by using SHA256 algorithm.

Callback of Recognition Result

NameData TypeRequiredDescription
roomIdStringYesRoom ID, unique identifier for a moderation task
sceneIdIntegerNoScenario ID customized by the developer and used for subdividing service scenes
contextJSONNoContext parameter during the pass-through of the task initiating request
timestampIntegerYesCurrent unix timestamp of server in seconds
uidStringYesUser ID
results[]JSON arrayNoReturn result data. Each element indicates a processing result of one action. For the specific structure, see Table result as below.

Table result

NameTypeRequiredDescription
codeIntegerYesError code. 200 indicates success, consistent with status code of HTTP
messageStringYesDescription information for errors
actionStringYesDetection type, corresponding to a detection type (actions) in the call request
labelStringYesDetection result label with a value related to action. For specific values, see moderation types above and corresponding label specification table.
rateFloating-point numberYesProbability of detection result label, with the value range: [0.00 - 1.00]. The greater the value, the higher the credibility.
suggestionStringYesOperation suggested to perform, with the value range:
- pass: normal, no need of operating;
- review: suspected, an uncertain detection result, and needing further manual moderation
- block: Illegal, suggested to punish illegal contents
urlStringNoAddress for saving illegal audio clips (Note: Only illegal audio clips will be saved for 3 hours. Otherwise, the URL may become invalid after more than 3 hours. Please export them in time.)
textStringNoText content of audio clips, provided only when the moderation category is "antispam".
extraData[]JSON arrayNoExtended information of the detection result, provided only when the moderation category is “antispam”. For details, see Table antispam-extraData.

Table antispam-extraData

NameTypeRequiredDescription
hintJson arrayNoHit keyword
labelStringNoType of hit keyword
rateFloating-point numberNoMeaningless, always "1.0"

Callback of Task Status

NameData TypeRequiredDescription
roomIdStringYesRoom ID, unique identifier for a moderation task
sceneIdIntegerNoScenario ID customized by the developer and used for subdividing service scenes
contextJSONNoContext field during the pass-through of the starting task
statusStringYesTask statuses (running - being running, stopped - stopped normally, error - error occurred)
errCodeIntegerYesTask status code
errMessageStringYesDescription information
timestampIntegerYesCurrent unix timestamp in seconds

3.3 Search Results

Request Method

ItemDescription
Request MethodPOST
Request Protocol HTTPS
Request Domain Nameai.jocloud.com
Request Pathapp/{appId}/v1/audio/stream/results?traceId=xxxxx&roomId=xxxxxx
Request ParametertraceId is a uuid string, and used for fault locating during troubleshooting. You should use different values for requests.
roomId is the room ID corresponding to the moderation task to be searched.
Request HeaderContent-Type: application/json;charset=UTF-8
token: Authentication token, generated on the basis of[identity authentication](/cn/product_category/content_audit_service/integration_and_start/integration_and_start.md#3-Identity authentication)

Returned Content

NameTypeRequiredDescription
codeIntegerYesError code that is identical with HTTP status code and is also extended
- 2xx indicates success
- 4xx indicates request error
- 500 indicates server error
See related descriptions for specific values.
messageStringYesError message description
traceIdStringYestraceId content in the pass-through parameter
roomIdStringYesRoom ID, unique identifier for a moderation task
sceneIdIntegerNoScenario ID customized by the developer and used for subdividing service scenes
contextStringNoUser context, context in the corresponding task starting parameter
statusStringYesTask statuses (running - being running, stopped - stopped normally, error - error occurred)
errCodeIntegerYesTask status code
errMessageStringYesDescription information
timestampIntegerYesCurrent unix timestamp in seconds
results[]results arrayNoSave multiple recent result data. When the call succeeds (code==200), each element in the array is a structure in Table 2 results.

Table. results

NameTypeRequiredDescription
uidStringYesUser ID
timestampIntegerYesTime of getting the recognition result (unix timestamp in seconds)
result[]result arrayNoSave multiple result data of action recognition. Each element in the array is a structure in Table. result.

3.4 Stop a Detection Task

Request Method

ItemDescription
Request MethodPOST
Request Protocol HTTPS
Request Domain Nameai.jocloud.com
Request Pathapp/{appId}/v1/audio/stream/stop?traceId=xxxxx&roomId=yyyy
Request ParametertraceId is a uuid string, and used for fault locating during troubleshooting. You should use different values for requests.
roomId is the room ID corresponding to the moderation task to be stopped.
Request HeaderContent-Type: application/json;charset=UTF-8
token: Authentication token, generated on the basis of[identity authentication](/product_category/content_audit_service/integration_and_start/integration_and_start.md#3-Identity authentication)

Response Content

Data in return body is JSON, and the specific field is described as below:

NameTypeRequiredDescription
codeIntegerYesError code that is identical with HTTP status code and is also extended
- 2xx indicates success
- 4xx indicates request error
- 500 indicates server error
See related descriptions for specific values.
messageStringYesError message description
traceIdStringYestraceId content in the pass-through parameter
roomIdStringYesRoom ID, unique identifier for a moderation task
sceneIdIntegerNoScenario ID customized by the developer and used for subdividing service scenes
contextJSONNoContext in the corresponding request
timestampIntegerYesCurrent unix timestamp of server in seconds

3.5 Error Codes

The returned error code of json object is described as follows.

Error CodeError Description
200Succeeded
4xxRequest parsing error
5xxInternal processing error
3012Task stopped*, cause: stream timeout (10 min)

4. Call Sample

The following demonstrates the call example of python code.

4.1 Request to Start a Task

# -*- coding: utf-8 -*-
#! python3.5

import requests
import uuid
import base64

host = "https://ai.jocloud.com"

appid = 1234567890  # Your service ID
restful_id = '**********'  # Your certificate ID
restful_secret = '**********'  # Your certificate key
traceid = str(uuid.uuid4())
dataid = str(uuid.uuid4())

# url
url = host + '/app/%s/v1/video/live/start?traceId=%s' % (appid, traceid)

# headers
headers = {
    "content-type": "application/json"
}

auth = base64.b64encode(("%s:%s" % (restful_id, restful_secret)).encode('utf-8'))
headers['token'] = 'Base ' + auth.decode()

# Live streaming URL to be recognized
streamurl = 'rtmp://mytest.com/live/123456789'

# Context required by the service side, which facilitates follow-up processing of the callback message, for example: 
context = {
    'myid': 123,
    'myname': 'test'
}

# Recognition result and status callback address. Trigger the callback of a notification of recognition result and status through http POST.
resultCb = 'http://192.168.1.100:12345/result'
statusCb = 'http://192.168.1.100:12345/status'

# content
values = {
    'actions': ['v-porn'],
    'url': streamurl,
    'streamId': "demo-test",
    'resultCb': resultCb,
    'statusCb': statusCb,
    'sequence': 'test',
    'context': context
}

# request
res = requests.post(url, json=values, headers=headers)
print ('code=%s, data=%s\n' % (res.status_code, res.text))

The returned call results are as below:

{
    "code": 200,
    "message": "OK",
    "traceId": "f35a8abb-db6c-48f2-af7e-f6c0921b7b0d",
    "taskId": "acee5522-b198-4325-9434-61b41eb2e08a",
    "timestamp": 1576567090,
    "streamId": "demo-test",
    "context": {
        "myid": 123,
        "myname": "test"
    }
}

4.2 Search Task Status and Result

# -*- coding: utf-8 -*-
#! python3.5

import requests
import uuid
import base64

host = "https://ai.jocloud.com"

appid = 1234567890  # Your service ID
restful_id = '**********'  #  Your certificate ID
restful_secret = '**********'  # Your certificate key
task_id = "acee5522-b198-4325-9434-61b41eb2e08a"  # Fill in the taskId content returned when the task starts.

traceid = str(uuid.uuid4())
dataid = str(uuid.uuid4())

# url
url = host + '/app/%s/v1/video/live/results?traceId=%s&taskId=%s' % (appid, traceid, task_id)

# headers
headers = {
    "content-type": "application/json"
}

auth = base64.b64encode(("%s:%s" % (restful_id, restful_secret)).encode('utf-8'))
headers['token'] = 'Base ' + auth.decode()

# request
res = requests.get(url, headers=headers)
print ('url=%s\ncode=%s\ndata=%s\n' % (url, res.status_code, res.text))

The returned call results as below:

{
    "code": 200,
    "message": "OK",
    "traceId": "fe0a8278-0ed6-4c74-80ab-fb1066605227",
    "taskId": "acee5522-b198-4325-9434-61b41eb2e08a",
    "timestamp": 1576567239,
    "streamId": "demo-test",
    "context": {
        "myid": 123,
        "myname": "test"
    },
    "results": [
        {
            "result": [
                {
                    "action": "v-porn",
                    "label": "normal",
                    "rate": 0.99952548742294312,
                    "suggestion": "pass",
                    "timestamp": 1576567232,
                    "url": "http://jocloud-ai-openapi.oss-cn-hangzhou.aliyuncs.com/live/10034/cccc2dcc-5472-4fbf-ae1b-5a906ea46c46.jpg"
                }
            ],
            "timestamp": 1576567233
        },
        {
            "result": [
                {
                    "action": "v-porn",
                    "label": "normal",
                    "rate": 0.99910759925842285,
                    "suggestion": "pass",
                    "timestamp": 1576567227,
                    "url": "http://jocloud-ai-openapi.oss-cn-hangzhou.aliyuncs.com/live/10034/f5540914-c5d4-41b6-8080-42a218f16a34.jpg"
                }
            ],
            "timestamp": 1576567228
        },
        {
            "result": [
                {
                    "action": "v-porn",
                    "label": "normal",
                    "rate": 0.9999052882194519,
                    "suggestion": "pass",
                    "timestamp": 1576567222,
                    "url": "http://jocloud-ai-openapi.oss-cn-hangzhou.aliyuncs.com/live/10034/50bf73a6-2555-48c2-8f33-13d2002f7e76.jpg"
                }
            ],
            "timestamp": 1576567223
        },
        {
            "result": [
                {
                    "action": "v-porn",
                    "label": "normal",
                    "rate": 0.99971604347229004,
                    "suggestion": "pass",
                    "timestamp": 1576567217,
                    "url": "http://jocloud-ai-openapi.oss-cn-hangzhou.aliyuncs.com/live/10034/4696a8a5-78f8-4c32-b696-aebbf79685bc.jpg"
                }
            ],
            "timestamp": 1576567218
        },
        {
            "result": [
                {
                    "action": "v-porn",
                    "label": "normal",
                    "rate": 0.99987661838531494,
                    "suggestion": "pass",
                    "timestamp": 1576567212,
                    "url": "http://jocloud-ai-openapi.oss-cn-hangzhou.aliyuncs.com/live/10034/6002efb8-e6d4-4527-afd8-1fd4a951d655.jpg"
                }
            ],
            "timestamp": 1576567213
        }
    ],
    "status": "running"
}

4.3 Stop the Task

# -*- coding: utf-8 -*-
#! python3.5

import requests
import uuid
import base64

host = "https://ai.jocloud.com"

appid = 1234567890  # Your service ID
restful_id = '**********'  # Your certificate ID
restful_secret = '**********'  # Your certificate key
traceid = str(uuid.uuid4())
dataid = str(uuid.uuid4())

# headers
headers = {
    "content-type": "application/json"
}

auth = base64.b64encode(("%s:%s" % (restful_id, restful_secret)).encode('utf-8'))
headers['token'] = 'Base ' + auth.decode()

traceid = str(uuid.uuid4())
task_id = 'acee5522-b198-4325-9434-61b41eb2e08a' # Fill in the taskId content returned when the task starts.

# url
url = host + '/app/%s/v1/video/live/stop?traceId=%s&taskId=%s' % (appid, traceid, task_id)

# request
res = requests.post(url, json={}, headers=headers)
print ('url=%s\ncode=%s\ndata=%s\n' % (url, res.status_code, res.text))

The returned call results are as below:

{
    "code": 200,
    "message": "OK",
    "traceId": "9eb28f1d-a307-4e93-bcf4-66e531aa12f1",
    "taskId": "acee5522-b198-4325-9434-61b41eb2e08a",
    "timestamp": 1576567306
}

5. Version Description

VersionDateDescription
v1.0.02020-05-09Initial version
<