HTML5 Video Editor Integration API
Deployment guide
The editor server instance could be installed locally using docker.
To get the docker-compose configuration files please contact info@solveigmm.com
Host requirements
RAM at least 2GB, 2 CPU ( AVX instructions set is required by mongodb)
HDD 20 GB of free space (Docker images size is 10 GB)
GPU to with minimum OpenGL 2.0 GLSL 1.10 support (optional, for chromakey feature)
Install docker compose
To install Docker Compose for your host please use https://www.docker.com/
Ubuntu 22.04:
sudo su
apt-get update
apt-get upgrade
apt-get install docker-compose
Windows:
Follow installation instruction from official package https://www.docker.com
The WSL 2 option should be set to provides better performance than the Hyper-V backend
MacOS:
Follow installation instruction from official package https://www.docker.com
For the MACOS host docker should be run with ordinary user (without su privileges)
Configuration notes
Replace MASTER_SERVER_IP variable in docker_build_solveig/master/.env by your server IP address on the network (localhost and 127.0.0.1 will not work)
For the server instance there should be ports exposed and accessible for MASTER_SERVER_IP. Please expose it before running the server:
9092 – kafka
27017- mongodb
80 – nginx front
3478(TCP and UDP) - coturn
The main docker configuration file:
docker_build_solveig/master/docker-compose.yml
To run the server:
docker-compose up -d --force-recreate
To stop the server:
docker-compose down
The Editor is available by URL ( Google Chrome browser is recommended):
http://localhost
The test media files are located at ./smm_cloud_editor_v2/shared/files
You can try to put your own there to see it in media tab or import media files via browser
Chromakey requirements
For "chromakey" feature the host instance should have GPU with minimum OpenGL 2.0 GLSL 1.10 support.
The gpu driver should be installed and configured properly.
Amazon instances with gpu: https://docs.aws.amazon.com/dlami/latest/devguide/gpu.html
G4ad instance GPU driver installation
To install driver at G4ad instance:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-amd-driver.html
Alternative way to install drivers:
sudo su
wget --referer http://support.amd.com http://repo.radeon.com/amdgpu-install/22.20.3/ubuntu/jammy/amdgpu-install_22.20.50203-1_all.deb
apt install ./amdgpu-install_22.20.50203-1_all.deb
apt update
amdgpu-install --usecase=graphics --no-32
wget https://repo.radeon.com/amdgpu-install/22.40.5/ubuntu/jammy/amdgpu-install_5.4.50405-1_all.deb
apt install ./amdgpu-install_5.4.50405-1_all.deb
apt-get update
amdgpu-install --usecase=graphics --no-32
External image library sources requirements
Pexels and Unsplash images source types requires developer’s account creation:
https://www.pexels.com/api/documentation/#authorization
https://unsplash.com/developers
The Api keys are configured at "docker_build_solveig/master/.env" :
#PEXELS_API_KEY=<your_api_key>
#UNSPLASH_API_KEY=<your_api_key>
#UNSPLASH_API_SECRETE=<your_api_key_secrete>
#UNSPLASH_APP_NAME=<your_unsplash_app_name>
Registration
To make custom integration with your serivce it is required to register a developer account to get an apiKey and configure integration options:
- Provide UI customization options. See Custom UI demo as an example
- companyName - Used at preview screen instead of default "Solveig Multimedia"
- appTitle - Used at page's header and "title" tag instead of default "Online Video Editor"
- appDescription - Used as html "meta:description" tag
- favIconUrl - ICO logo is used as site's icon
- svgLogoUrl - SVG logo image used at prview screen and header
- customCSSUrl - custom CSS url that allows to change html elements styles and layout
- customJSUrl - custom JS url that allows to run external scripts.Requires 'Access-Control-Allow-Origin' header at script host to prevent browser CORS block.
- callBackUrl to receive events that are performed by the user while using the editor. See CallBack events for details
- Provide custom API urls for media files. See Custom URLs specification and Generate media info for details
- fetchFileUrl to fetch media information
- mediaTabContentUrl to fetch media files to display at media tab,
- fetchThumbnailUrl to fetch media file’s thumbnail to display at timeline
- outputS3Bucket to upload output results to Amazon S3 Bucket. See S3 bucket configuration for details
Before open the editor's session for specific user use Create user session API to get the session’s id
To open an editor use the following url:
https://smarteditingonline.solveigmm.com/?session_id=<id>
Custom UI demo
The demo is implemented at url https://smarteditingonline.solveigmm.com/custom
To restore default UI use https://smarteditingonline.solveigmm.com/default
The demo is implemented using the following Developer profile:
{
"name":"Example Name",
"eMail":"example@solveigmm.com",
"companyName": "My Company Name",
"appTitle": "My Editor",
"appDescription":"My editor description",
"favIconUrl":"https://deveditor.solveigmm.com:9011/favicon.ico",
"svgLogoUrl":"https://deveditor.solveigmm.com:9011/logo.svg",
"customCSSUrl": "https://deveditor.solveigmm.com:9011/custom.css"
}
The custom.html example that opens user session with the associated developer's profile apiKey that was created using /api/dev_profiles:
<html>
<script>
async function open() {
const apiKey = 'f06fe180f54efa48e41ceff6849b032f'
const name = 'UserName'
const resp = await fetch('/api/users', {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json'
},
body:JSON.stringify({
apiKey,
name
})
})
const {id} = await resp.json()
location.replace(`/?session_id=${id}`)
}
open()
</script>
<body>
</body>
</html>
The custom.css example is used to change main panels background color, button colors and audio icons:
/* change button colors */
.MuiButton-root.btn--primary {
color: #fff;
background-color: #5cb85c;
}
.MuiButton-root.btn--primary:hover {
color: #fff;
background-color: #4cae4c;
}
/* change background panels color */
.header-solveig {
background: #6d6d6d;
}
.preview-player-toolbar {
background: #6d6d6d;
}
.timeline-toolbar{
background: #6d6d6d;
}
.timeline-footer {
background: #6d6d6d;
}
/* change audio icons */
:root {
--audio-asset-background:url(https://deveditor.solveigmm.com:9011/audio_background.png);
--audio-asset-icon:url(https://deveditor.solveigmm.com:9011/audio_icon.svg);
--voice-asset-background:url(https://deveditor.solveigmm.com:9011/audio_background.png);
--voice-asset-icon:url(https://deveditor.solveigmm.com:9011/voice_icon.svg);
}
Custom URLs specification
To provide custom media files within the user's session, the urls specified during developer registration should be implemented at developer server.
The urls should be configured to respond with the ‘Access-Control-Allow-Origin’ header with the value where the editor is running and 'Access-Control-Allow-Headers' 'content-type'
All request contain an Authorization header:
"Authorization": <auth_token>
auth_token
- the token that was assigned to user's session at apiAuthToken
fetchFileUrl
The fetchFileUrl is used to get media information.
Request:
GET
fetchFileUrl/<fileId>
Parameters:
fileId
- the media file's id
Response:
JSON encoded object dev_media_file_entry_object :
{
"id":<string>,
"previewUrl":<string_url>,
"name":<string>,
"filetype":<string>,
"mediaType":<string>,
"duration":<number>,
"size":<number>,
"width":<number>,
"height":<number>,
"thumbnail":<base64_encoded_png_image>,
"audioTracks":[<string>, <string>]
}
id
- defines file id
previewUrl
- url for playing media file
name
- defines file name of entry
filetype
- defines file's type, could be: "mp4"
, "mov"
, "mp3"
, "png"
,"m4a"
,"jpg"
,"mxf"
, "folder"
, "webm"
mediaType
- defines entry media type, could be: "video"
, "audio"
, "image"
duration
- is duration in 100ns units,
size
- file size in bytes,
width, height
- is video resolution in pixels of source media
thumbnail
- is png image converted to base64 encoded string.
audioTracks
- array of audio tracks names
mediaTabContentUrl
The mediaTabContentUrl is used to get media tab content.
Request:
GET
mediaTabContentUrl?start_number=<number>&count=<number>&filter=<string>&sort_field=<string>&sort_order=<string>&search=<string>
Parameters:
start_number
- the start number of first element in requested media list
count
- the max count of requested entries
filter
- defines filter by mediaType (optional, could be comma separated combination of multiple items)
sort_field
- defines sort field
name
- by name of the filecreateTime
- by creation time (default)size
- by sizeduration
- by duration
sort_order
- defines order in which entries should be sorted(optional)
asc
- ascendingdesc
- descending (default)
search
- defines search string that will be applied to entries(optional)
Response:
{
"total_count":<number>,
"entries":[
<dev_media_file_entry_object>,
<dev_media_file_entry_object>,
<dev_media_file_entry_object>,
<dev_media_file_entry_object>,
]
}
Description:
total_count
- defines total count of files
entries
- An array of dev_media_file_entry_object
fetchThumbnailUrl
The fetchThumbnailUrl is used to get a single thumbnail in order to display it at timeline.
Request:
GET
fetchThumbnailUrl?id=<fileId>&time=<number>
Parameters:
id
- the media file’s id
time
- the thumnbail’s time in 100ns units
Response:
JSON encoded object
{
"data":<base64_encoded_png_image>
}
Description:
data
- is png image converted to base64 encoded string.
Generate media info
The media file information required for customized urls implementation could be prepared in advance.
Generate media info API
Use existing html5editor backend server to generate media info.
All request should contain an Authorization header:
"Authorization": <valid_api_key>
valid_api_key
- the valid api developer's key
Create media info task
Request:
POST
/api/task/mediainfo
Request Body:
{
"originalUrl": <string>, (optional)
"awsBucket": { (optional)
"awsBucketName":<string>,
"awsAccessKey":<string>,
"awsSecreteKey":<string>
}
}
Description:
originalUrl
- file's url of the orignal media file
awsBucket
- AWS bucket credentials object. (Note: aws bucket processing works only with ENABLE_KEEP_MEDIAINFO_DATA=true)
Response:
{
"id": <taskId>
}
Get media info
Request:
GET
/api/task/mediainfo/<taksId>
Response:
Status 202 - The media file processing is in progress
Status 200 - The media file was processed
JSON mediainfo object:
{
"info":{
"name":<string>,
"filetype":<string>,
"mediaType":<string>,
"duration":<number>,
"size":<number>,
"width":<number>,
"height":<number>,
"thumbnail":<base64_encoded_png_image>,
"previewUrl":<string_url>,
"audioTracks":[<string>, <string>]
},
"thumbnails":[{
"time":<number>,
"data":<base64_encoded_png_image>
}]
}
Description:
info
- should be used to implement fetchFileUrl, and mediaTabContentUrl. The fields description at dev_media_file_entry_object.
thumbnails
- should be used to implement fetchThumbnailUrl
Generate media info locally
The local installed docker container(provided on demand) helps to generate media info locally
Input media files config
The editor requires media information on media files that could be prepared by locally installed editor's instance.
Prefetched media info sample
The media information will be stored at locally installed editor's database. The media files will be downloaded to fetch media info and removed after operation completed. The input media files could be processed by url or aws S3 bucket
Configure .env docker variables:
MASTER_SERVER_IP - the editor's ip address
ADMIN_SECRETE - secrete to manage developer profiles.
Register developer profile
Register developer profile via API /api/dev_profiles
Use ADMIN_SECRETE as Authorization header
Replace MASTER_SERVER_IP to actual value
{
"fetchFileUrl": "http://MASTER_SERVER_IP/api/mediafiles",
"mediaTabContentUrl": "http://MASTER_SERVER_IP/api/mediafiles",
"fetchThumbnailUrl":"http://MASTER_SERVER_IP/api/clip_thumbnail"
}
The result is DEV_API_KEY created.
Create media info task
Using DEV_API_KEY as Authorization header create media info task request /api/task/mediainfo and wait for import process completed.
Specify aws bucket credentials as follow:
awsBucket: {
"awsBucketName":<string>
"awsAccessKey":<string>,
"awsSecreteKey":<string>,
}
Or specify originalUrl to media file:
originalUrl: <string>
Register user's session
Register user's session with DEV_API_KEY via /api/users
{
"apiKey": DEV_API_KEY,
"apiAuthToken": DEV_API_KEY
}
The result is USER_SESSION_ID created.
Open user's session
Open the user's session via url
http://MASTER_SERVER_IP?session_id=USER_SESSION_ID
NodeJS sample:
The NodeJS sample that prefetch mediainfo by url and creates user's session:
https://solveigmm.com/files/prefetch_mediainfo_session.zip
Ingest media by url sample
In this examples the user's session will be created with the import process started. The media information and the file will be stored locally.
Create the user's session
Pass session Authorization token using websocket connection (see Authorization section)
Import media by url
Start import process using /api/import/media/url
Python sample:
The Python sample that opens user's session with import process:
https://solveigmm.com/files/open_html5editor.zip
Output S3 bucket configuration
To upload output results to Amazon S3 Bucket outputS3Bucket should be configured with the following fields:
{
"awsBucketName":<string>,
"awsEndPoint":<string>,
"awsAccessKey":<string>,
"awsSecreteKey":<string>,
}
The example permission policy configuration for Amazon is the following:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"s3:ListBucket"
],
"Resource": [
"arn:aws:s3:::<bucketName>"
]
},
{
"Effect": "Allow",
"Action": [
"s3:PutObject",
"s3:GetObject",
"s3:PutObjectAcl"
],
"Resource": [
"arn:aws:s3:::<bucketName>/*"
]
}
]
}
CallBack events
The editor sends POST requests with JSON object body to the callback url that was specified during developer registration.
Common event structure
{
"type": <string>,
"userData": <string>,
"apiAuthToken": <string>
"payload": <object>
}
type
- the event type
userData
- the data that was passed at user’s session registration
apiAuthToken
- the apiAuthToken that was passed at user’s session registration
payload
- contains event dependant fields
Check event at callbackUrl
Each event’s POST request contains the Authorization header that is set to the unique_user_session_id that was created by Registration api ("/api/users"). You can use it to restrict posting of unauthorised data to callback.
Project created
Called when the user created new project
type:
project_create
payload:
{
"projectId":<unique_project_id>,
"thumbnail":<base64_image_string>
}
Project updated
Called when the user make some changes to project
type:
project_update
payload:
{
"projectId":<unique_project_id>,
"thumbnail":<base64_image_string>
}
Render project started
Called when the user starts project rendering
type:
render_start
payload:
{
"projectId":<unique_project_id>,
}
Render project completed
Called when the project rendering completed
type:
render_complete
payload:
{
"projectId":<unique_project_id>,
"downloadUrl":<output_file_url>,
"error":<error_message>
}
HTML5 Video Editor - Backend Server API
Authorization
auth_token is used to identify user in API requests,
Backend uses WebSocket connection associated with auth_token in order to track if the user is online.
For the new connection auth_token is created as follow (JS):
const client = new WebSocket('ws://backendHost:backendPort/websocket')
client.onopen = () => {
const authToken = Date.now().toString()
client.send(JSON.stringify({
type: 'user_event',
payload: { authToken },
}))
}
client.onmessage = message => {
const dataFromServer = JSON.parse(message.data)
if (dataFromServer.type === 'ping') {
client.send(JSON.stringify({
type: 'pong',
}))
}
}
Create user
Request:
POST /api/users
Request Body:
{
"name": <string>,
"avatarUrl":<string>,
"data": <json_object>,
"apiKey":<string>,
"apiAuthToken": <string>
}
Description:
All properties are optional
name
- name of the user
avatarUrl
- url to user's profile avatar
data
- custom user’s data json_object
apiKey
- associates the user with the the developer's profile
apiAuthToken
- used as Authorization header with custom urls overriden by developer's profile (should be used with apiKey)
Response:
JSON encoded object:
{
"id": <string>
}
Description:
id
- users’s id
Update existing user
Request:
PATCH /api/users/<id>
Request Body:
userObject as for create user request
Response:
JSON encoded object:
{
"id": <string>
}
Description:
id
- users’s id
Get user data
Request:
GET /api/users/<id>
Parameters:
id
- user’s id
Response:
{
"name": <string>
"data": <string>
}
Create Developer's Profile
Managing developer's profile requires Authorization header to be set as specified in ADMIN_SECRETE config variable
Request:
POST /api/dev_profiles
Request Body:
{
"name":<string>,
"eMail":<string>,
"callbackUrl":<string>,
"fetchFileUrl":<string>,
"mediaTabContentUrl":<string>,
"fetchThumbnailUrl":<string>,
"outputS3Bucket": {
"awsBucketName":<string>,
"awsEndPoint":<string>,
"awsAccessKey":<string>,
"awsSecreteKey":<string>,
}
"companyName": <string>,
"appTitle": <string>,
"appDescription":<string>,
"favIconUrl":<string>,
"svgLogoUrl":<string>
"customCSSUrl": <string>,
"customJSUrl": <string>,
"imageLibSrcType": <string>,
"pexelsApiKey":<string>,
"unsplashApiKey": <string>,
"unsplashApiSecrete":<string>,
"unsplashAppName":<string>,
"pexelsApiKey":<string>,
}
Description:
Required fields:
name
- The name of the profile
eMail
- The profiles' e-mail
Optional fields:
callbackUrl
- The url to recieve callbacks from the editor
fetchFileUrl, mediaTabContentUrl, fetchThumbnailUrl
- Custom media files' urls
outputS3Bucket
- Bucket to upload output media files
companyName
- Displayed on empty preview window
appTitle
- Displayed at header and browser tab
appDescription
- Meta description tag of the web page
favIconUrl
- Url to the icon that used as browser tab icon
svgLogoUrl
- Url to the icon at header and empty preview window
customCSSUrl
- Url to plaintext css file with overriden styles
customJSUrl
- Url to plaintext js file with code
imageLibSrcType
- coma separated image sources (could be json, unsplash, pexels)
unsplashApiKey
, unsplashApiSecrete
, unsplashAppName
- credentials to use unsplash.com as source for images library
pexelsApiKey
- credentials to use pexels.com as source for images library
Response:
JSON encoded object:
{
"apiKey":<string>,
}
Update existing developer's profile
Request:
PATCH /api/dev_profiles/<api_key>
Request Body:
dev_profile_object as for create request
Response:
JSON encoded object:
{
"apiKey": <string>
}
List dev profiles
Request:
GET /api/dev_profiles?start_number=0&count=10
Parameters:
start_number
- profiles num to start with
count
- count per page
Response:
{
"totalCount":<number>,
"profiles":[<dev_profile_object>]
}
Description:
totalCount
- defines total count of profiles
profiles
- An array of dev_profile_object
Delete dev profile
Request:
DELETE /api/dev_profiles/<api_key>
Get media files list
Request:
GET /api/mediafiles?start_number=0&count=20&filter=video&sort_field=createTime&sort_order=asc&search=abc&folder=<id>&auth_token=<valid_token>
Parameters:
start_number
- the start number of first element in requested media list
count
- the max count of requested entries
filter
- defines applied filter to all files by types could be video, audio, image, folder, voiceover(optional, could be comma separated combination of multiple items)
sort_field
- defines sort field
name
- by name of the filecreateTime
- by creation time (default)size
- by sizeduration
- by duration
sort_order
- defines order in which entries should be sorted(optional)
asc
- ascendingdesc
- descending (default)
search
- defines search string that will be applied to entries(optional)
Response: Description: Used to get information on single media file Headers: Request: Parameters: Response: JSON encoded media_file_entry_object object: media_error_description_object: Used to get status on media_file_entry_object Headers: Request: Parameters: Response: JSON encoded object Used to delete media_file_entry_object Headers: Request: Parameters: Response: Creates new folder with specified name Request: Headers: Request Body: Description: Response: JSON encoded object Removes folder with all it’s content recursively with subfolders Request: Headers: Parameters: Response: Request: Headers: Parameters: <id>- requested folder id Request Body: Description: Response: JSON encoded object: Headers: Request: Parameters: Request Body: Description: NOTE: in case case of moving files to root folder /api/folders/root/entries should be used Fetches list of audio library files.
To initiate audio library files fetch process use /api/import/audiolib_json
sort_field Request: Parameters: Response: Response is the same as for api /api/mediafiles Used to get information on audio library file Request: Response: Response is the same as /api/mediafiles/ Fetches list of existing categories Request: Parameters: Response: JSON array describing media_file_entry_object's: <auidolib_category_object> is JSON object Fetches list of existing categories Request: Parameters: Response: JSON array describing media_file_entry_object's: <auidolib_mood_object> is JSON object The Image library api could use different sources: The "imageLibSources" payload variable of "user_info" websocket message contains the list of available sources for the current user session. Request: Parameters: Response: JSON encoded object Request: Parameters: Response: JSON encoded image_library_object: Request: Parameters: Response: JSON object: To get thumbnails for the clip on timeline from prepared database of thumbnails use if To get list of thumbnails for the media file with step and requested count Request: Parameters: Response: JSON encoded object Description: Request: Parameters: Response: JSON encoded object Description: Editing Task is passed via POST request the request body should contain json object with XTL format task Request: Headers: Request Body: Description: Example of xtl_task: Response: JSON encoded object Description: To get task’s status use GET request Request: Headers: Parameters: Response: JSON encoded object Description: engine_error_description_object: To stop task use DELETE request Request: Headers: Parameters: Response:
JSON encoded object Project passed via POST request the request body should contain json object with project’s data Request: Headers: Request Body: Description: Response: JSON encoded object: Description: Error Codes: Request: Headers: Parameters: Request Body: Description: Response:
JSON encoded object: Error Codes: To get project’s data use GET request Request: Headers: Parameters: Response: JSON encoded <project_object>: Description: Error Codes: Request: Headers: Parameters: Response: JSON array describing media_file_entry_object’s: Description: Error Codes: Request: Headers: Parameters: Response: JSON array describing media_file_entry_object’s: Description: Error Codes: Used to delete project Headers: Request: Parameters: Response: Error Codes: The action history is saved at backend to keep project’s changes between autosave intervals and ability to restore the changes if the user disconnected between autosave intervals.
The order of POST requests will define the order of the entries returned by GET
The total actions count for the project configured by ACTION_HISTORY_DEPTH (default value 1000) backend variable. Exceeding this count will remove earlier actions. The actions are linked to the project, removing the project removes all actions related to it. Action passed via POST request the request body should contain json object with action’s data. On new action all action with Request: Headers: Request Body: Description: Response: JSON encoded object: Description: Error Codes: In order to undo/redo action, the “canceled” field should be updated. Request: Headers: Parameters: Request Body: Description: Response: JSON encoded object: Error Codes: Request: Headers: Parameters: Response: JSON array describing media_file_entry_object’s: Description: The action object: The entries are in the order that they were POSTed. Error Codes: For editing with text overlays or image overlay it’s required to put image data of rendered text or image to use it in editing task, it will be automatically deleted after user disconnected. Request: Headers: Request Body: Description: Response:
JSON encoded object Create an upload session on response Headers: Request: Request Body: Description: Response: JSON encoded object Error Codes: Headers: Request: Request Body: Description: Response: Status 200 - the media file was uploaded
Status 206 - more data expected Use the Range header in the response to determine where to start the next chunk. The header contains information on which bytes the server has received. Do not assume that the server received all bytes sent in the previous request. For example, Status 5xx - upload request was interrupted Initiates process of importing media by url at backend side. Headers: Request: Request Body: Description: Response: JSON encoded object Error Codes: Initiates process of importing media by aws bucket credentials at backend side. Headers: Request: Request Body: Description: Response: JSON encoded object Error Codes: Initiates process of fetching audio library files using json list.
The json list’s absolute path is set by env variable AUDIOLIB_JSON_LIST (default uses /var/www/smm_cloud_editor_v2/audioLib.json)
The media absolute path is configured by AUDIOLIB_FILES_DIR env variable The example of audioLib.json: Initiate importing process. To get the progress and errors use GET Request: Response: JSON encoded object Error Codes: Request: Response: JSON encoded object Error Codes: The json list’s absolute path is set by env variable IMAGELIB_JSON_LIST (default uses /var/www/smm_cloud_editor_v2/imageLib.json) The media absolute path is configured by IMAGELIB_FILES_DIR env variable The example of imageLib.json: Fields Description: The import process and control it's status uses the same API as for audio library with the url /api/import/imagelib_json Creates import project session on response the id of the new project will be available Headers: Request: Request Body: Description: Response: JSON encoded object Error Codes: Headers: Request: Request Body: Description: For example, Response: Use the Range header in the response to determine where to start the next chunk. The header contains information on which bytes the server has received. Do not assume that the server received all bytes sent in the previous request. For example Range bytes=0-524287 - means that server has received the first 524,288 bytes ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range ) Headers: Request: Response: JSON encoded object Description: Headers: Request: Response: JSON encoded object Voiceover is implemented using kurento media server Creates webrtc connection and start sending voice data to server Headers: Request: Request Body: Description: Response: JSON encoded object Error Codes: Start record voiceover data to file Headers: Request: Request Body: Description: Response: JSON encoded object Error Codes: Headers: Request: Description: Response: JSON encoded object Error Codes: Get backend live statistics. Request: Response: JSON encoded object Description: Request: Response: JSON encoded object Description Version string contains 1 - major, 0 - minor, 20 - year, 06 - month, 18 - dayfolder
- defines the folder with auth_token
- user’s auth_token{
"total_count":4,
"entries":[
<media_file_entry_object>,
<media_file_entry_object>,
<media_file_entry_object>,
<media_file_entry_object>,
]
}
total_count
- defines total count of files
entries
- An array of media_file_entry_objects
Get information on media file
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationGET
/api/mediafiles/<id>
id
- media file id{
"id":<string>,
"name":<string>,
"type":<string>,
"filetype":<string>,
"mediaType":<string>,
"duration":<number>,
"size":<number>,
"width":<number>,
"height":<number>,
"thumbnail":<base64_encoded_png_image>,
"previewUrl":<string_url>,
"originalUrl"::<string_url>,
"audioTracks":[<string>,<string>],
"status":<string>,
"progress":<number>,
"error": <media_error_description_object>,
"taskId": <string>,
"categoryId":<string>,
"moodId":<string>
}
id
- defines file idname
- defines file name of entrytype
- defines entry type, could be "test"
, "uploaded"
, "url"
, "output"
test
- test file available to all usersuploaded
- files that was uploaded by userurl
- file is imported from urloutput
- files that was exported/renderedfiletype
- defines entry type, could be: "mp4"
, "mov"
, "mp3"
, "png"
,"m4a"
,"jpg"
,"mxf"
, "folder"
, "webm"
mediaType
- defines entry media type, could be: "video"
, "audio"
, "image"
duration
- is duration in 100ns units,size
- file size in bytes,width, height
- is video resolution in pixels of source mediathumbnail
- is png image converted to base64 encoded string.previewUrl
- url for playing media fileoriginalUrl
- url for original media file. In case of using proxy file for previewUrlaudioTracks
- array of audio tracks namesstatus
- defines media file status if some operation is in progress, could be
render
- status for project output file that is rendering nowupload
- status for media file that user is uploading nowimport
- status for media file that is importing from url nowconvert
- status for media file that is converting nowprepare_thumbnails
- status for media file that is involved in thumbnails building process.ready
- status indicates that on media file there will no further operations expectedprogress
- determine progress of the current stageerror
- determine error description JSON object <media_error_description_object>taskId
- determine id of render task used for output files (optional)categoryId
- category id of the media file, used with audio library files (optional)moodId
- mood id of the media file, used with audio library files (optional)
Media error description object
{
"type": <error_type>,
"message": <text>,
"title": <error_title>,
}
error_type
could be:
exceeded_file_size
- Exceeded single file size limitexceeded_per_user_size
- Exceeded per user size limitlow_disk_space
- low disk space on serverunsupported_media
- media file is not supportedinternal_server_error
- Internal server errornot_found
- media file not foundused_in_projects
- media file is used in projectstext
- contains error descriptionerror_title
- the error title
Get status on media file
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationGET
/api/mediafile/status?&id="1233f2"
id
- media file id{
"status":"upload",
"progress":90,
"error": <media_error_description_object>
}
Delete media file
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationDELETE
/api/mediafiles/<id>?force=1
id
- media file idforce
- ignore error{
"error": <[media_error_description_object](#media_error_description_object)>
}
Working with folders
Create folder
POST /api/folders
"Authorization": "valid_auth_token"
valid_auth_token
- was received on authorization{
"name": "folder_name"
"folder": "folder_id"
}
name
- the name of the folder,folder
- the folder’s id to create new folder in{
"id":"2sdv342s8834"
}
Delete folder
DELETE
/api/folders/<id>?force=1
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationid
- folder’s idforce
- ignore error{
"error": <media_error_description_object>
}
Update folder name
PATCH
/api/folders/<id>
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorization{
"name": <folder_new_name>,
}
name
- name of the folder. Optional{}
Move media file(s), folders to another folder
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationPUT
/api/folders/<id>/entries
<id>
- requested folder id{
"files": [],
"folders": [],
}
files
- an array of file ids to move to folder ( Optional)folders
- an array of folder ids to move to folder ( Optional)
Audio library
Get audio library files list
GET /api/audiolib_files?type=music&start_number=0&count=20&sort_field=createTime&sort_order=asc&search=abc&category_id=<id>&mood_id=<id>
start_number
- the start number of first element in requested media listcount
- the max count of requested entriestype
- selects the list of specific type could be:
music
- audio tracks with music composition (default)sound
- audio samples of specific sounds or noisescategory_id
- optional field to filter by specific categorymood_id
- optional field to filter by specific moodsort_field
- defines sort field
name
- by name of the file (default)createTime
- by creation timesort_order
- defines order in which entries should be sorted(optional)
asc
- ascending (default)desc
- descendingsearch
- defines search string that will be applied to entries(optional) Get information on audio library file
GET
[/api/audiolib_files/<id>](#Get_information_on_media_file "/api/audiolib_files/<id>")
Get categories list for audio library files
GET /api/audiolib_files/categories?type=music
type
- field to define type of the list could be:
music
- audio tracks with music composition (default)sound
- audio samples of specific sounds or noises{
"entries":[
<auidolib_category_object>,
<auidolib_category_object>,
<auidolib_category_object>,
<auidolib_category_object>,
],
}
{
"id": <unique_id>
"name": <category_name>
"entriesCount": 20
}
Get Moods list for audio library files
GET /api/audiolib_files/moods?type=music
type
- field to define type of the list could be:
music
- audio tracks with music composition (default)sound
- audio samples of specific sounds or noises{
"entries":[
<auidolib_mood_object>,
<auidolib_mood_object>,
<auidolib_mood_object>,
<auidolib_mood_object>,
],
}
{
"id": <unique_id>
"name": <mood_name>
"entriesCount": 20
}
Images library
Get images library files list
GET
/api/imagelib_files/<source>?page=<number>&count=<number>&search=<string>&sort_field=createTime&sort_order=asc&category_id=<string>
source
- defines the image library sourcepage
- the page numbercount
- the count of entries per pagecategory_id
- optional field to filter by specific categorysort_field
- defines sort field
name
- by name of the file (default)createTime
- by creation timesort_order
- defines order in which entries should be sorted(optional)
asc
- ascending(default)desc
- descendingsearch
- defines search string that will be applied to entries(optional){
entries: [<image_library_object>]
total_count:<number>
}
entries
- an array of image_library_objectstotal_count
- number of all entries matched to request Get information on image library file
GET
/api/imagelib_files/<source>/<id>
source
- defines the image library sourceid
- defines image file id{
"id":<string>,
"name":<string>,
"width":<number>,
"height":<number>,
"thumbnail":<string>,
"previewUrl":<string>,
"originalUrl":<string>,
"webpageUrl":<string>,
"userName":<string>,
"size":<number>,
"filetype":<string>,
}
name
- the title of the imagewidth
, height
- is image resolution in pixels of source mediathumbnail
- is png image converted to base64 encoded string or urlpreviewUrl
- url for previeworiginalUrl
- original url for mediawebpageUrl
- web page of the image resource on external resource(optional)userName
- the user’s name that image resource belongs to (optional)size
- the size of the original image in bytes (optional)filetype
- defines entry type (could be: "png", "jpg") Get categories list for image library files
GET /api/imagelib_files/<source>/categories?page=<number>&count=<number>search=<string>
source
- defines the image library sourcepage
- the page numbercount
- the count of entries per pagesearch
- defines search string that will be applied to entries(optional){
"entries":[<imagelib_category_object>],
"total_count": <number>
}
entries
- an array of image_library_category_objectstotal_count
- number of all entries matched to request{
"id": <sting>,
"name": <string>,
"entriesCount": <number>,
"description":<string>,
"parent": <string>,
}
id
- unique category idname
- the category nameentriesCount
- the entries count in this categorydescription
- the description for category(optional)parent
- the parent category id (optional)
Thumbnails for clip on timeline
cached=true
parameter in requestcached=false
thumbnails will be generated in real timeGET
/api/clip_thumbnails?id=0&start_time=10000000&step=5000000&count=20&cached=true
id
- the media file’s id ( see /api/mediafiles)start_time
- the time in 100ns units for the start timestep
- the time in 100ns units time stepcount
- count of requested thumbnailscached
- should get thumbnails from prepared database{
"status":"",
"thumbnails":[
{
time:0
data:"base64_encoded_png_image"
},
{
time:5000000
data:"base64_encoded_png_image"
}
]
}
status
- contains error message, empty string means operation success,thumbnails
- an array of thumbnails objects,time
- thumbnail time in 100ns units,data
- base64_encoded_png_image
Get single thumbnail
GET
/api/clip_thumbnail?id=0&time=10000000
id
- the media file’s id ( see /api/mediafiles)time
- thumnbail’s time in 100ns units{
"data":<base64_encoded_png_image>
}
data
- is png image converted to base64 encoded string.
Editing task
Create new task
POST
/api/task/editing
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorization{
"xtl_task": <valid_xtl_task_string>
}
auth_token
- User gets this token after authorization<timelines version="2">
<timeline>
<group >
<track video="1" audio="1" accuracy="frame">
<clip src="media_file_id" start="00:00:01:00" stop="00:00:06:00" timeFormat="time10ms" />
<clip src="media_file_id" start="00:00:07:00" stop="00:00:12:00" timeFormat="time10ms" />
</track>
</group>
</timeline>
</timelines>
{
"taskId":"2sdv342s8834"
"fileId": "output_23234234234.mp4"
"error": <error_description_message>
}
taskId
- an id of newly created taskfileId
- identifies output Media Fileerror
- contains error description (optional)
Editing task operation status
GET
/api/task/editing?id=<task_id>
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationid
- requested task’s id{
"id":<task_id>,
"progress":50,
"status":"running",
"error": <engine_error_description_object>,
"outputUrls":[
"http://35.234.23.3/output_files/43234/32343/name.mp4"
]
}
id
- an id of task
progress
- task progress in percent
status
- string description of current task status could be:
running
- task is runningcompleted
- task was successful completedaborted
- task was aborted by user or error messagedownloading
- downloading media files before trimming (in case of “url” file types)uploading
- uploading saved result (e.g. to amazon bucket)error
- task’s Engine error description object
outputUrls
- links to output media file(s)
Engine error description object
{
"type": <type>,
"message": <message>,
"code": <code>,
}
type
- could be:
engine_error
- Edit engine errordownload_error
- unable to download input mediaupload_error
- upload errormessage
- contains error description
code
- internal engine error code
Stop editing task
DELETE
/api/task/editing?id=<task_id>
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationid
- requested task’s id{
"status":"",
}
Projects
Create new project
POST
/api/projects
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorization{
"name": <project_name>,
"data": <project_data>,
"thumbnail": <base_64_string>,
"createTime": <unix_timestamp_ms>
}
name
- name of the project.data
- actual project’s data.thumbnail
- the project thumbnailcreateTime
- the project createTime (Optional). If not set the backend time will be used{
"id": "B7pAdG3HJUZ25uJkwuuVcQH6Hkndx6gARcV76EV4"
}
id
- project’s id400
- bad request404
- no user or no project with id found500
- internal server error
Update existing project
PATCH
/api/projects/<id>
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorization<id>
- requested project id{
"name": <project_name>,
"data": <project_data>,
"thumbnail": <base64_string>,
"modifTime":<unix_timestamp_ms>
}
name
- name of the project. Optionaldata
- actual project’s data. Optionalthumbnail
- thumbnail for the project. OptionalmodifTime
- project’s modification time. Optional. If not set the backend time will be used.{}
400
- bad request404
- no user or no project with id found500
- internal server error
Get existing project’s data
GET
/api/projects/<id>
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorization<id>
- requested project id{
"id":<project_id>,
"data":<project_data>,
"name":<project_name>,
"createTime":<unix_timestamp>,
"modifTime":<unix_timestamp>,
"thumbnail": <base64_string>
}
id
- an id of the projectdata
- actual project’s dataname
- name of the projectcreateTime
- project’s creation timemodifTime
- last project’s modification timethumbnail
- thumbnail for the project400
- bad request404
- no user or no project with id found500
- internal server error
Get projects list
GET /api/projects?start_number=<number>&count=<number>&data=<0|1>&search=<string>
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationstart_number
- the start number of first element in requested project listcount
- the max count of requested entriesdata
- fill <project_object>’s data field for non zero valuesearch
- defines search string that will be applied to entries(optional){
"total_count":<number>,
"entries":[
<project_object>,
<project_object>,
<project_object>,
<project_object>,
],
}
total_count
- defines total count of files
The project_object description could be found in /api/project
400
- bad request404
- no user or no project with id found500
- internal server error
Check media presence in projects
GET /api/projects/presence?file_id=<id>
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationfile_id
- defines file's id to search{
"presence":<boolean>,
}
total_count
- defines total count of files
The project_object description could be found in /api/project
400
- bad request404
- no user or no project with id found500
- internal server error
Delete project
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationDELETE
/api/projects/<id>
id
- project id{}
400
- bad request404
- no user or no project with id found500
- internal server error
Actions history
Create new action
canceled=”true”
field will be removed, before posting action that will remove canceled actions the project should be in sync.POST
/api/actions
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorization{
"id": <action_id>,
"project": <project_id>,
"name": <action_name>,
"data": <action_data>,
"createTime": <unix_time_ms>,
}
id
- action’s idprojectId
- project’s id to which action belongs toname
- the name of the actiondata
- the action datacreateTime
- the action’s createTime (Optional). If not set the backend time will be used{
"id": "1632798732"
}
id
- actions’id400
- bad request404
- no user or no project with id found500
- internal server error
Undo Redo action
PATCH
/api/actions/<id>
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorization<id>
- requested action id{
"modifTime": <unix_time_ms>
"canceled": true
}
canceled
- true means action was undoed false action is activemodifTime
- the time of modification. If not set backend time will be used{}
400
- bad request404
- no user or no project with id found500
- internal server error
Get actions list
GET /api/actions?project=<valid_project_id>
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorizationproject
- the project id that actions belongs to{
"entries":[
<action_object>,
],
}
{
"id": <action_id>,
"name": <action_name>,
"data": <action_data>,
"canceled": <boolean>,
"createTime": <unix_time_ms>
"modifTime": <unix_time_ms>
}
id
- action id.name
- name of the action.data
- actual action’s data.canceled
- "true"
- action was canceled (undo operation was done), "false"
- action is activecreateTime
- action’s creation time unix time in msmodifTime
- action’s modification time unix time in ms400
- bad request404
- no user or no project with id found500
- internal server error
Overlay image upload
POST
/api/overlay_img
"Authorization": <valid_auth_token>
valid_auth_token
- was received on authorization{
"data":"base64_encoded_png_image"
}
data
- base64 encoded_png_image{
"status":"",
"id":<overlay_img_id>,
}
status
- contains error message, empty string means operation success,id
- an id of overlay_img
Local media upload
upload_session_id
will be available. The upload session is valid within 24 hours. Upload media file parts with specifying upload_session_id
. The progress of current session could be gotten via upload_session_id
Create an upload session
"Authorization": <valid_auth_token>
POST
/api/upload/media
{
"size":12345
"name":"my media.mp4"
"folder":"folder_id"
}
valid_auth_token
- User gets this token after authorizationsize
- is number of bytes of file data, which will be transferred in subsequent requestsfolder
- destination folder’s id (optional){
"id":<upload_session_id>,
}
id
- an id of uploaded media400
- bad request404
- no user found500
- internal server error507
- Upload limit exceeded error514
- Unsupported media
Put media file parts to upload session
"Authorization": <valid_auth_token>
"Content-Length": <chunk_size>
"Content-Range": bytes <range_start>-<range_end>/<total_size>
"Content-Type": "application/octet-stream"
PUT
/api/upload/media?id=<upload_session_id>
<binary_data>
valid_auth_token
- User gets this token after authorizationchunk_size
- number of bytes in the current chunk<range_start>-<range_end>/<total_size>
- show which bytes in the file you upload.
For example, Content-Range: bytes 0-524287/2000000
shows that you upload the first 524,288 bytes (256 x 1024 x 2) in a 2,000,000 byte file ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Range )Range bytes=0-524287
- means that server has received the first 524,288 bytes ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range )
Import media by URL
Create an url import request
"Authorization": <valid_auth_token>
POST
/api/import/media/url
{
"url":"https://example.com/file.mp4"
"folder":"folder_id"
}
valid_auth_token
- User gets this token after authorizationfolder
- destination folder’s id (optional){
"id":<imported_media_id>,
}
id
- an id of imported media400
- bad request404
- no user found500
- internal server error
Import AWS bucket
Create an aws bucket import request
"Authorization": <valid_auth_token>
POST
/api/import/media/awsbucket
{
"awsAccessKey":<key>,
"awsSecreteKey":<secrete>,
"awsBucketName":<bucket_name>,
}
valid_auth_token
- User gets this token after authorizationawsAccessKey, awsSecreteKey, awsBucketName
- Credentials to access aws bucket.{
"id":<awsbucket_id>,
"error":<error description>,
}
id
- an id of imported aws_bucketerror
- description if present400
- bad request404
- no user found500
- internal server error
Import audio library files from json list
{
"entries": [
{
"name": "Bubble_G_Funk_Remastered.mp3",
"relativePath": "Bubble_G_Funk_Remastered.mp3",
"category": "Electro",
"type": "music"
},
{
"name": "cats.mp3",
"relativePath": "cats.mp3",
"category": "animals",
"type": "sound"
},
]
}
Create an audio library files import request
POST
/api/import/audiolib_json
{
}
500
- internal server error Get an audio library files import process status
GET
/api/import/audiolib_json
{
"processedCount": 1
"totalCount": 256
"error":<error description>,
}
error
- contains error description if presentprocessedCount
- contains count of files already handledtotal count
- contains total number of files500
- internal server error Import images library files from json list
{
"entries": [
{
"name": "Chair",
"relativePath": "architecture1.jpg",
"category": "Architecture"
},
{
"name": "Blue Ocean",
"url": "https://smarteditingonline.solveigmm.com/files/ocean.jpg",
"category": "Nature",
"subCategory":"Ocean",
"width":1920,
"height":1080,
"thumbnail":"https://smarteditingonline.solveigmm.com/files/ocean.jpg",
"size":241888,
"filetype":"jpg",
"userName":"SolveigMM",
"userWebPage":"https://solveigmm.com"
}
]
}
name
- The name of the image displayed in mediarelativePath
- The relative path to IMAGELIB_FILES_DIR of the image on local driveurl
- The url of the original imagecategory
- The root category name for the imagesubCategory
- The subcategory name for the imagewidth, height
- The original image dimensionsthumbnail
- The thumbnail url to be displayed in media (the default size required 174x113, other size will be scaled automatically)size
- The file size of the image displayed in popupfiletype
- The file type of the image displayed in popupuserName, userWebPage
- used to reference image author Import Project
Create an import session
"Authorization": <valid_auth_token>
POST
/api/upload/project
{
"size":12345
"name":"edlproject.zip"
"fps":29.97
"createFolder":true
}
valid_auth_token
- User gets this token after authorizationsize
- is number of bytes of file data, which will be transferred in subsequent requestsname
- the name of the uploaded project file (optional)fps
- the fps that will be used at the import stage e.g. for edl that is required(optional)сreateFolder
- import media files to the folder with the project name or to root media (optional){
"id":<project_id>,
}
id
- an id of the new project400
- bad request500
- internal server error Put project file parts to import session
"Authorization": <valid_auth_token>
"Content-Length": <chunk_size>
"Content-Range": bytes <range_start>-<range_end>/<total_size>
"Content-Type": "application/octet-stream"
PUT
/api/upload/project?id=<project_id>
<binary_data>
valid_auth_token
- User gets this token after authorizationchunk_size
- number of bytes in the current chunk<range_start>-<range_end>/<total_size>
- show which bytes in the file you upload.Content-Range: bytes 0-524287/2000000
shows that you upload the first 524,288 bytes (256 x 1024 x 2) in a 2,000,000 byte file (
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Range )Status 200
- the media file was uploadedStatus 206
- more data expectedStatus 5xx
- upload request was interrupted Get import project session progress and status
"Authorization": <valid_auth_token>
GET
/api/upload/project?id=<project_id>
{
"stage": <import_project_stage>,
"progress": 0
"error": "error description"
}
import_project_stage
- the status of the import process, could be:
"upload"
- the data is uploading now"unpack"
- the data is unpacking"import_media"
- import media files"ready"
- the import process completedprogress
- the progress value of the current stageerror
- error description if present Cancel import project session
"Authorization": <valid_auth_token>
DELETE
/api/upload/project?id=<project_id>
{}
Voiceover recording
Create voiceover recording session
"Authorization": <valid_auth_token>
POST
/api/voiceovers
{
"sdpOffer":<sdp_offer>
"iceCandidates":<array_of_local_ice_candidates>
}
offer
- SDP offer string created by browser webRtcPeer.generateOffericeCandidates
- local ice candidates array gathered by browser{
"id":<new_voiceover_id>
"sdpAnswer": <remote_sdp_answer>
"iceCandidates": <array_of_remote_ice_candidates }
}
500
- internal server error Start voiceover recording
"Authorization": <valid_auth_token>
PUT
/api/voiceovers/<id>
{
"name": <the file name for_recording>
}
id
- recording idname
- new recording file name (optional) , the backend will set name automatically if missing{
error: <error description>
}
error
- Error description if present500
- internal server error Stop and delete voiceover recording session
"Authorization": <valid_auth_token>
DELETE
/api/voiceovers/<id>?keepRecording=1
id
- recording idkeepRecording
- flag determines keeping media file recorded during the session{
}
500
- internal server error Statistics
GET
/api/stat
{
"activeUsersCount":1,
"activeBuildThumbsTasksCount":0,
"activeRenderTasksCount":1,
"activeImportFromURLTasksCount":0,
"activeImportFromLocalDriveCount":0
}
activeUsersCount
- Count of active useractiveBuildThumbsTasksCount
- Count of active build timeline thumbnails tasksactiveRenderTasksCount
- Count of active render tasksactiveImportFromURLTasksCount
- Count of files that are downloading from url nowactiveImportFromLocalDriveCount
- Count of files that are uploading from local drive now
Backend version
GET
/api/version
{
"version":"1.0.2006.18",
}