Video Layout

Introduction

When multiple users publish live streaming in different rooms or in one room, the platform can also mix multiple audio/video streams from specified rooms into one path for pushing to CDN or to the cloud for recording.

When starting to mix streams through RESTful API, you can set the display position, video screen size, watermark, background and SEI of the user's video screen by selecting the layout templateid or custom options to achieve the best display effect.

Implementation

  1. Call startrecord to start record task and set mixconfig parameters.
  • Layout template need to set templateid and output .

  • Custom layout need to set intput and output .

    Notices: Watermark (time stamp, text or picture), background or SEI can be set when you fill the values of watermark, background, base64seidata in output .

  1. (Optional) In order to update layout, you can call updatemixrecord to update mixconfig parameters .
  2. Call stoprecord to stop record task and mix stream task.

Parameters

ParameterTypeLayout templateCustom layoutDescription
intputJSONArrayNoRequiredInput parameter. Do not fill it when using Layout template .
outputJSONRequiredRequiredOutput parameter
templateidintRequiredNoLayout template ID:
1: Horizontal Layout
2: Vertical Layout
3: Tiled Layout
4: Corner Layout
5: 5 Users Horizontal Layout
Only valid for stream mixing in one room

input

ParameterTypeVideo and audioVideoAudioDescription
uidstringCustomCustomCustomSource stream user ID
roomidstringCustomCustomCustomRoom ID of the source stream user
streamtypeint012Stream type:
0: video and audio
1: video
2: audio
zOrderintCustomCustomOptionalLayer (0 indicates the bottommost layer and 1 is the first layer from bottom to top, and so on, up to 16 layers)
layoutXintCustomCustomOptionalTop left x-coordinate in canvas ( coordinate system: abscissa axis extended rightward and ordinate axis extended downward )
Value range: [0, 1920]
layoutYintCustomCustomOptionalTop left y-coordinate in canvas
Value range: [0, 1920]
layoutHintCustomCustomOptionalHeight in canvas
Value range: (0, 1920]
layoutWintCustomCustomOptionalWidth in canvas
Value range: (0, 1920]
cropintCustomCustomOptionalZooming display options
0: centered display of input stream, with black sides filled
1: filling in input stream as much as possible, and centering and cropping
cropXintOptionalOptionalOptionalAn x coordinate of a cutout region in an original image if an input is a cutout image from the original image
Value range: [0, 1920]
cropYintOptionalOptionalOptionalAn y coordinate of a cutout region in an original image if an input is a cutout image from the original image
Value range: [0, 1920]
cropHintOptionalOptionalOptionalHeight of a cutout region if an input is a cutout image from the original image
Value range: (0, 1920]
cropWintOptionalOptionalOptionalWidth of a cutout region if an input is a cutout image from the original image
Value range: (0, 1920]
alphafloatOptionalOptionalOptionalTransparency in mixed flow, value range: [0, 1.0]. 0 means completely transparent, 1.0 means completely opaque.

output

ParameterTypeVideo and audioVideoAudioDescription
streamtypeint012Stream type:
0: video and audio
1: video
2: audio
videoheightintCustomCustomOptionalVideo height, value range: [16, 1920], integral multiple of 4.
(If not, round to the nearest multiple of 4)
videowidthintCustomCustomOptionalVideo width, value range: [16, 1920], integral multiple of 4.
(If not, round to the nearest multiple of 4)
videobitrateintCustomCustomOptionalVideo bit rate in kbps, value range: (0, 10000]
videofpsint15 recommended15 recommendedOptionalVideo frame rate, value range: (0, 60]. The default value is 15fps. If video frame rate is set higher than 60, the service will be adjusted to 60.
videogopint30 recommended30 recommendedOptionalVideo GOP, value range (0, 1000), unit: fps, default value: 30fps.
Adopting calculation method: videogop = videofps × Key frame interval (seconds). When it is not satisfied, it will round up and adjust the videogop.
videoencodeint100100OptionalVideo encoding
100: H264 (not support H265)
audiobitrateint128Optional128Audio bitrate (kbps), value: 128.
audiochannelint2Optional2Number of audio tracks.
2: Stereo
Value is 2.
audiosampleint48000Optional48000Audio sampling rate, value is 48000.
audioformatint2Optional2Audio code
2: AAC
Value is 2.
patchintOptionalOptionalOptionalVideo frame patching method:
0: not patching the frame
1: patching black screen
2: patching the last frame
3: patching the specified picture
patchUrlstringOptionalOptionalOptionalWhen video frame patching method is 3, specifying the picture address with the length not more than 256 bytes
watermarkJSONOptionalOptionalOptionalWatermark
backgroundJSONOptionalOptionalOptionalBackground image and background color
base64seidatastringOptionalOptionalOptionalCustomize the video frame SEI information. The input string is Base64 encoded, and the length of original string does not exceed 2048 bytes. Server encodes the original content into each video frame. This setting does not work for pure audio output.

templateid

Horizontal Layout

The first user appears in the large-window picture at the upper side, with other users in the small-window pictures at the lower side. The small-window pictures hover above the large-window picture. Two rows of small windows can be provided at most, each of which can contain 8 pictures. In total, 17 user pictures can be supported in the small windows.

  • For the large image, if the aspect ratio of the picture is inconsistent with the aspect ratio of the video stream, the output video will be scaled without cropping.
  • For small pictures, if the aspect ratio of the picture is inconsistent with the aspect ratio of the video stream, the output video will be cropped to fill the small picture.
  • If a user only publishes audio, it will not occupy the layout position.

The effect is as below:
img

Vertical Layout

The first user appears in the large-window picture at the left side, with other users in the small-window pictures at the right side. Two rows of small windows can be provided at most, each of which can contain 8 pictures. In total, 17 user pictures can be supported in the small windows.

  • For the large picture, if the aspect ratio of the picture is inconsistent with the aspect ratio of the video stream, the output video will be scaled without cropping.
  • For small pictures, if the aspect ratio of the picture is inconsistent with the aspect ratio of the video stream, the output video will be cropped to fill the small picture.
  • If a user only publishes audio, it will not occupy the layout position.

The effect is as below:
img

Tiled Layout

Automatically adjust the size of each picture based on the number of users. The picture sizes of all users are the same, and 16 pictures (4 x 4) can be supported in total.

  • If the aspect ratio of the picture is inconsistent with the aspect ratio of the video stream, the output video will be cropped to fill the picture.
  • If a user only publishes audio, it will not occupy the layout position.

The effect is as below:
img

Corner Layout

The first user occupies the whole large window, and one small window hovers at each corner. This layout supports four small windows at most, and includes five pictures in total. At present, only a fixed layout is supported. The canvas size is 1280*720, and the size of each small window is 300*300. The distance between the small window and the large window is 10 pixels.

  • For the large picture, if the aspect ratio of the picture is inconsistent with the aspect ratio of the video stream, the output video will be scaled without cropping.
  • For small pictures, if the aspect ratio of the picture is inconsistent with the aspect ratio of the video stream, the output video will be cropped to fill the small picture.
  • If a user only publishes audio, it will not occupy the layout position.

The effect is as below:
img

5 Users Horizontal Layout

The first user occupies the whole large window, other users arrange and display the small window screens below, supporting totally 4 small windows at most, and the small window screens are floating above the large window screen. At present, only a fixed layout is supported. The canvas size is 1280*720, and the size of each small window is 300*300. The distance between the small window and the large window is 16 pixels.

  • For the large picture, if the aspect ratio of the picture is inconsistent with the aspect ratio of the video stream, the output video will be scaled without cropping.
  • For small pictures, if the aspect ratio of the picture is inconsistent with the aspect ratio of the video stream, the output video will be cropped to fill the small picture.
  • If a user only publishes audio, it will not occupy the layout position.

The effect is as below:
img

watermark

NameTypeRequiredDescription
timestampJSONNoTimestamp watermark
textListJSON ArrayNoText watermark
imageListJSON ArrayNoImage watermark

timestamp

NameTypeRequiredSample valueDescription
xintYes0The horizontal axis coordinate when taking the upper left corner as the origin, with the value range: [0, 1920), and does not exceed the width of the output picture.
yintYes0The vertical axis coordinate when taking the upper left corner as the origin, with the value range: [0, 1920), and does not exceed the height of the output picture.
formatstringYes"%H:%M:%S"Posix time display format. The following are supported:
"%Y/%m/%d": displayed as 2020/01/01
"%Y-%m-%d": displayed as 2020-01-01
"%H:%M:%S": displayed as 13:00:00
"%Y/%m/%d %H:%M:%S": displayed as 2020/01/01 13:00:00
"%Y-%m-%d %H:%M:%S": displayed as 2020-01-01 13:00:00
fontstringNo"NotoSansCJK"Font format, only support value "NotoSansCJK".
sizeintYes28Font size, with the value range: (0, 100].
colorstringNo"FFFFFF"Font color, RGB color definition, with the value range: 000000-FFFFFF, letters are all uppercase or all lowercase.
Default value is "FFFFFF" which means white.
backgroundColorstringNo"000000"Background color, RGB color definition, with the value range: 000000-FFFFFF, letters are all uppercase or all lowercase.
Default value is "000000" which means black
alphafloatNo1.0Transparency, with the value range: [0.0, 1.0]. 0.0 means completely transparent, and 1.0 means completely opaque.

textList

NameTypeRequiredSample valueDescription
xintYes0The horizontal axis coordinate when taking the upper left corner as the origin, with the value range: [0, 1920), and does not exceed the width of the output picture.
yintYes0The vertical axis coordinate when taking the upper left corner as the origin, with the value range: [0, 1920), and does not exceed the height of the output picture.
contentstringYes Display content string.
fontstringNo"NotoSansCJK"Font format, only support value "NotoSansCJK".
sizeintYes28Font size, with the value range: (0, 100]
colorstringNo"FFFFFF"Font color, RGB color definition, with the value range: 000000-FFFFFF, letters are all uppercase or all lowercase.
Default value is "FFFFFF" which means white.
backgroundColorstringNo"000000"Background color, RGB color definition, with the value range: 000000-FFFFFF, letters are all uppercase or all lowercase.
Default value is "000000" which means black
alphafloatNo1.0Transparency, with the value range: [0.0, 1.0]. 0.0 means completely transparent, and 1.0 means completely opaque.

imageList

NameTypeRequiredSample valueDescription
xintYes0The horizontal axis coordinate when taking the upper left corner as the origin, with the value range: [0, 1920), and does not exceed the width of the output picture.
yintYes0The vertical axis coordinate when taking the upper left corner as the origin, with the value range: [0, 1920), and does not exceed the height of the output picture.
widthintYes100Image width, with the value range: (0, 1920), and does not exceed the width of the output picture.
heightintYes100Image height, with the value range: (0, 1920), and does not exceed the height of the output picture.
urlstringYes Image URL address, the length does not exceed 256 bytes
scaleintYes0Zoom/crop mode.
0: Zoom and crop in equal proportions.
1: Stretch to fill the entire canvas.
alphafloatNo1.0Transparency, with the value range: [0.0, 1.0]. 0.0 means completely transparent, and 1.0 means completely opaque.

background

NameTypeRequiredSample valueDescription
colorstringNo"000000"Background color, RGB color definition, with the value range: 000000-FFFFFF, all letters are uppercase or all lowercase.
Default value is "000000" which means black.
imageJSONNo/Background image, when both color and image are filled in, image value will cover color value.

image

NameTypeRequiredSample valueDescription
urlstringYes/Image URL address, the length does not exceed 256 bytes.
scaleintYes0Zoom/crop mode.
0: Zoom and crop in equal proportions.
1: Stretch to fill the entire canvas.

Notices

  • Layout template is not supported during cross-room stream mixing.
  • Please contact Customer Support for customizing other layout template.

Was this page helpful?

Helpful Not helpful
Submitted! Your feedback would help us improve the website.
Feedback
Top
<