HTML5 Video Editor Integration Guide

Deployment

The editor server instance could be installed locally using docker.

To get the docker-compose configuration files please contact info@solveigmm.com

Host requirements

RAM at least 2GB, 2 CPU ( AVX instructions set is required by mongodb)

HDD 20 GB of free space (Docker images size is 10 GB)

GPU to with minimum OpenGL 2.0 GLSL 1.10 support (optional, for chromakey feature)

Install docker compose

To install Docker Compose for your host please use https://www.docker.com/

Minimal docker version required 20.10.21

Ubuntu 22.04:

    sudo su
    apt-get update
    apt-get upgrade
    apt-get install docker-compose

Windows:

Follow installation instruction from official package https://www.docker.com

The WSL 2 option should be set to provides better performance than the Hyper-V backend

MacOS:

Follow installation instruction from official package https://www.docker.com

For the MACOS host docker should be run with ordinary user (without su privileges)

Configuration notes

Replace MASTER_SERVER_IP variable in docker_build_solveig/.env by your server IP address on the network (localhost and 127.0.0.1 will not work)

For the server instance there should be ports exposed and accessible for MASTER_SERVER_IP. Please expose it before running the server:

9092 – kafka
27017- mongodb
80 – nginx front
3478(TCP and UDP) - coturn

The main docker configuration file:

docker_build_solveig/docker-compose.yml

To run the server:

docker-compose up -d --force-recreate

To stop the server:

docker-compose down

The Editor is available by URL ( Google Chrome browser is recommended):

http://localhost

The test media files are located at ./smm_cloud_editor_v2/shared/files

You can try to put your own there to see it in media tab or import media files via browser

Chromakey requirements

For "chromakey" feature the host instance should have GPU with minimum OpenGL 2.0 GLSL 1.10 support.

The gpu driver should be installed and configured properly.

Amazon instances with gpu: https://docs.aws.amazon.com/dlami/latest/devguide/gpu.html

G4ad instance GPU driver installation

To install driver at G4ad instance:

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/install-amd-driver.html

Alternative way to install drivers:


Works with the kernel version linux-image-5.19.0-1029-aws

sudo apt update
sudo apt remove linux-image-*

Click "No" in a dialog to confirm removal current kernel

sudo apt install linux-image-5.19.0-1029-aws linux-headers-5.19.0-1029-aws linux-modules-extra-5.19.0-1029-aws
sudo apt-mark hold 5.19.0-1029-aws
sudo apt autoremove


Change /etc/default/grub

GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 5.19.0-1029-aws"

sudo update-grub
sudo reboot

uname -r (should display 5.19.0-1029-aws)

sudo su
wget https://repo.radeon.com/amdgpu-install/22.40.5/ubuntu/jammy/amdgpu-install_5.4.50405-1_all.deb
apt install ./amdgpu-install_5.4.50405-1_all.deb
apt-get update
amdgpu-install --usecase=graphics --no-32
sudo reboot

External image library sources requirements

Pexels and Unsplash images source types requires developer's account creation:

https://www.pexels.com/api/documentation/#authorization

https://unsplash.com/developers

The Api keys are configured at "docker_build_solveig/.env" :

#PEXELS_API_KEY=<your_api_key>

#UNSPLASH_API_KEY=<your_api_key>

#UNSPLASH_API_SECRETE=<your_api_key_secrete>

#UNSPLASH_APP_NAME=<your_unsplash_app_name>

Customization

By default each user has options for customization since it has linked developer's profile (configured via ENABLE_DEV_PROFILE_FOR_USERS=TRUE environment variable).

You can adjust basic styles and text definitions via "Customize editor" UI.

With the developer's profile you can configure:

User interface styles and text definitions:
- companyName - Used at preview screen instead of default "Solveig Multimedia"
- appTitle - Used at page's header and "title" tag instead of default "Online Video Editor"
- svgLogo - Logo image used at preview screen and header
- customCSS - custom CSS that allows to change html elements styles
- customJS - custom JS that allows to run external scripts.
- customLanguage - custom JSON with text definitions.
See dev_profile_object for details.
callBackUrl to receive events that are performed by the user while using the editor. See CallBack events for details
Custom urls that will be used by the editor to fetch media assets information. See Override media assets information API and Generate media info for details
outputS3Bucket to upload output results to Amazon S3 Bucket. See S3 bucket configuration for details

If the server configured with disabled customization for regular users (ENABLE_DEV_PROFILE_FOR_USERS=FALSE) to configure UI it is required to create a developer's user profile to get an apiKey using /api/developer_profiles.

The customization UI will be available at https://smarteditingonline.solveigmm.com/?auth_token=<apiKey>

To create regular users sessions with customized interface use /api/users. The apiKey field should identify the developer's profile. The user's session will be available at https://smarteditingonline.solveigmm.com/?auth_token=<userId>

Override media assets information API

The Developer profile allows to override media assets information API by custom implementation at your server. The following variables available:

fetchFileUrl - to override fetching media information /api/mediafiles/<id>
mediaTabContentUrl to override fetch media files to display at media tab /api/mediafiles
fetchThumbnailUrl to override fetch media file's thumbnail to display at timeline/api/clip_thumbnails

The urls responses should include the headers to prevent CORS policy issues:

'Access-Control-Allow-Origin':'*'
'Access-Control-Allow-Headers':'content-type'

All request from the editor will contain an Authorization header with the token that was assigned to user's session at apiAuthToken

"Authorization": <auth_token>

Generate media info

The media file information required for Custom media assets could be prepared in advance.

Use existing html5editor backend server to generate media info.

All request should contain an Authorization header:

"Authorization": <valid_api_key>

valid_api_key - the valid api developer's key

Create media info task

Request:

POST
/api/task/mediainfo

Request Body:

{
  "originalUrl": <string>, (optional)
  "awsBucket": {           (optional)
    "awsBucketName":<string>,
    "awsAccessKey":<string>,
    "awsSecreteKey":<string>
  }
}

Description:

originalUrl - file's url of the orignal media file

awsBucket - AWS bucket credentials object. (Note: aws bucket processing works only with ENABLE_KEEP_MEDIAINFO_DATA=true)

Response:

{
  "id": <taskId>
}

Get media info

Request:

GET
/api/task/mediainfo/<taksId>

Response:

Status 202 - The media file processing is in progress

Status 200 - The media file was processed

JSON mediainfo object:

{
  "info":{
    "name":<string>,
    "filetype":<string>,
    "mediaType":<string>,
    "duration":<number>,
    "size":<number>,
    "width":<number>,
    "height":<number>,
    "thumbnail":<base64_encoded_png_image>,
    "previewUrl":<string_url>,
    "audioTracks":[<string>, <string>]
    },
  "thumbnails":[{
    "time":<number>,
    "data":<base64_encoded_png_image>
  }]
}

Description:

info - should be used as a response for custom api urls implementation ( fetchFileUrl, mediaTabContentUrl at devProfile ).

thumbnails - should be used to implement custom api url for timeline thumbnails (fetchThumbnailUrl at devProfile )

Custom media assets examples

Import media by url example

In this example the user's session will be created with the import process started.

Download source code

NodeJS example

const { platform } = require('os');
const { exec } = require('child_process');
const axios = require('axios')

const masterServerIP = 'smarteditingonline.solveigmm.com' // should be changed according to your installation
const baseURL = `https://${masterServerIP}`

// import media url could be set via command line argument
let url = 'https://smarteditingonline.solveigmm.com/files/test_video.mp4'
const myArgs = process.argv.slice(2)
if (myArgs.length){
    url = myArgs[0]
}

const axiosInst = axios.create({ baseURL })

const openChrome = url => {
    let command
    const osPlatform = platform()
    if (osPlatform === 'win32') {
        command = `start chrome ${url}`;
    } else if (osPlatform === 'darwin') {
        command = `open -a "Google Chrome" ${url}`;
    } else {
        command = `google-chrome --no-sandbox ${url}`;
    }
    exec(command);
}

const registerUser = async () => {
    try {
        const user = {
            name: 'Example user',
        }
        const resp = await axiosInst.post('/api/users', user)
        return resp.data.id
    } catch (e) {
        throw Error(`Error registerUser: ${e}`)
    }
}

const importFromUrl = async userId => {
    try {
        const headers = { 'Authorization': userId }
        await axiosInst.post('/api/import/media/url', {url}, { headers })
    } catch (e) {
        throw Error(`Error registerUser: ${e}`)
    }
}

const main = async () => {
    const userId = await registerUser()
    await importFromUrl(userId)
    const chromeUrl = `${baseURL}?session_id=${userId}`
    console.log(`open chrome with the url:  ${chromeUrl}`)
    openChrome(chromeUrl)
}

main()

Python example

import sys
import platform
import subprocess
import requests

master_server_ip = 'smarteditingonline.solveigmm.com'  # should be changed according to your installation
base_url = f'https://{master_server_ip}'

# import media URL could be set via command line argument
url = 'https://smarteditingonline.solveigmm.com/files/test_video.mp4'
my_args = sys.argv[1:]
if my_args:
    url = my_args[0]

axios_inst = requests.Session()
axios_inst.headers.update({'Content-Type': 'application/json', 'Accept': 'application/json'})
axios_inst.headers.update({'Authorization': ''})

def open_chrome(url):
    os_platform = platform.system().lower()
    if os_platform == 'windows':
        command = f'start chrome {url}'
    elif os_platform == 'darwin':
        command = f'open -a "Google Chrome" {url}'
    else:
        command = f'google-chrome --no-sandbox {url}'
    subprocess.run(command, shell=True)

def register_user():
    try:
        user = {'name': 'Example user'}
        resp = axios_inst.post(f'{base_url}/api/users', json=user)
        resp.raise_for_status()
        return resp.json()['id']
    except requests.exceptions.RequestException as e:
        raise Exception(f'Error register_user: {e}')

def import_from_url(user_id):
    try:
        headers = {'Authorization': user_id}
        axios_inst.post(f'{base_url}/api/import/media/url', json={'url': url}, headers=headers).raise_for_status()
    except requests.exceptions.RequestException as e:
        raise Exception(f'Error import_from_url: {e}')

def main():
    user_id = register_user()
    import_from_url(user_id)
    chrome_url = f'{base_url}?session_id={user_id}'
    print(f'Open Chrome with the URL: {chrome_url}')
    open_chrome(chrome_url)

if __name__ == '__main__':
    main()

PHP example

<?php

$masterServerIP = 'smarteditingonline.solveigmm.com'; // should be changed according to your installation
$baseURL = "https://$masterServerIP";

// import media URL could be set via command line argument
$url = 'https://smarteditingonline.solveigmm.com/files/test_video.mp4';
$myArgs = array_slice($argv, 1);
if (!empty($myArgs)) {
    $url = $myArgs[0];
}

function openChrome($url) {
    $osPlatform = strtolower(php_uname('s'));
    $command = '';

    if (strpos($osPlatform, 'win') !== false) {
        $command = "start chrome $url";
    } elseif (strpos($osPlatform, 'darwin') !== false) {
        $command = "open -a 'Google Chrome' $url";
    } else {
        $command = "google-chrome --no-sandbox $url";
    }

    shell_exec($command);
}

function registerUser() {
    global $baseURL;
    try {
        $user = ['name' => 'Example user'];
        $headers = ['Content-Type: application/json'];
        $response = sendRequest("$baseURL/api/users", 'POST', json_encode($user), $headers);
        $responseData = json_decode($response, true);
        return $responseData['id'];
    } catch (Exception $e) {
        throw new Exception("Error registerUser: " . $e->getMessage());
    }
}

function importFromUrl($userId) {
    global $baseURL, $url;
    try {
        $headers = ['Authorization: ' . $userId,
                    'Content-Type: application/json'];
        sendRequest("$baseURL/api/import/media/url", 'POST', json_encode(['url' => $url]), $headers);
    } catch (Exception $e) {
        throw new Exception("Error importFromUrl: " . $e->getMessage());
    }
}

function sendRequest($url, $method = 'GET', $data = null, $headers = null) {
    $ch = curl_init();
    // Disable SSL certificate verification (not recommended for production)
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);

    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);

    if ($data !== null) {
        curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
    }

    if ($headers !== null) {
        curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
    }

    $response = curl_exec($ch);

    if ($response === false) {
        throw new Exception("cURL error: " . curl_error($ch));
    }

    curl_close($ch);

    return $response;
}

function main() {
    global $baseURL;
    $userId = registerUser();
    importFromUrl($userId);
    $chromeUrl = "$baseURL?session_id=$userId";
    echo "Open Chrome with the URL: $chromeUrl\n";
    openChrome($chromeUrl);
}

if (basename(__FILE__) === basename($_SERVER["SCRIPT_FILENAME"])) {
    main();
}

?>

React video editor component example

Download source code

Online demo

In this example the video editor component "SolveigEditor" is embedded into React application.

The media url is imported using "videoUrl" property.

The callback "onVideoAdded" is called after the video was put into timeline.

The callback "onRenderComplete" is called after the project rendering completed with the url of the output media.

To run example use:

npm install
npm start

SolveigEditor component description

<SolveigEditor> - tag defines the editor component

Properties:

src - defines the url of editor's minimized JS

css - define the url or editor's mimized CSS

width, height, hidden - The this size and visibility properties

apiUrl - defines the backend's API url to used by component.

authToken - set specific user's session

folderId - set specific folder at "Media" tab

projectId - load specific project

videoUrl - import media url with adding media to timeline

Callbacks:

onVideoAdded - Imported media is ready to be used in project

onRenderComplete - The project's redering completed

onProjectUpdate - The project was updated by the user

index.js

/* eslint-disable no-unused-expressions */
/* eslint-disable no-sequences */
import React from 'react';
import ReactDOM from 'react-dom/client';
import './index.css';
import App from './App';

const root = ReactDOM.createRoot(document.getElementById('root'));
root.render(
  <React.StrictMode>
    <App />
  </React.StrictMode>
);

App.js

import { useState } from "react";
import "./App.css";
import { SolveigEditor } from "solveigmm-video-editor";

// Files hosted from the public folder
const baseUrl = process.env.PUBLIC_URL || '';

const EDITOR_JS_SRC = `${baseUrl}/editor/main.chunk.js`;
const EDITOR_CSS_SRC = `${baseUrl}/editor/main.css`;

const INITIAL_BACK_URL = localStorage.getItem('editor-sample-backUrl') || 'https://smarteditingonline.solveigmm.com/'
const INITIAL_AUTH_TOKEN = localStorage.getItem('editor-sample-authToken') || ''
const INITIAL_FOLDER_ID = localStorage.getItem('editor-sample-folderId') || ''
const INITIAL_PROJECT_ID = localStorage.getItem('editor-sample-projectId') || ''

// In this example, we are using submittedUrl and submittedApi variables only for the form handling.
// You can change this approach
function App() {
  const [videoUrl, setVideoUrl] = useState(
    "https://smarteditingonline.solveigmm.com/files/test_video.mp4"
  );
  const [backUrl, setBackUrl] = useState(INITIAL_BACK_URL);
  const [authToken, setAuthToken] = useState(INITIAL_AUTH_TOKEN)
  const [folderId, setFolderId] = useState(INITIAL_FOLDER_ID)
  const [projectId, setProjectId] = useState(INITIAL_PROJECT_ID)
  const [submittedUrl, setSubmittedUrl] = useState("");
  const [showEditor, setShowEditor] = useState(true);

  const handleChangeShowEditor = (v) => {
    setShowEditor(v);
    localStorage.setItem("showEditor", v);
  };

  const handleRenderComplete = data => {
    console.log(`onRenderComplete(${JSON.stringify(data, null, 2)})`);
  };
  const handleProjectUpdate = data => {
    console.log(`onProjectUpdate(${JSON.stringify(data, null, 2)})`);
  };

  return (
    <div className="App">
      <header className="App-header">

        {/* Layout container: Form (left) + Editor (right) */}
        <div className="app-container">

          {/* Left Side - Forms */}
          <div className="form-container">

            <div className="form-group">
              <div className="form-field checkbox-field">
                <input
                  id="showeditor"
                  type="checkbox"
                  checked={showEditor}
                  onChange={(e) => handleChangeShowEditor(e.target.checked)}
                />
                <label htmlFor="showeditor">Show Editor Component</label>
              </div>
            </div>

            {/* Backend Configuration */}
            <div className="form-group">
              {[
                { label: "Backend URL:", value: backUrl, setValue: setBackUrl, placeholder: "Enter back URL" },
                { label: "Auth Token:", value: authToken, setValue: setAuthToken, placeholder: "Enter Auth Token" },
                { label: "Folder ID:", value: folderId, setValue: setFolderId, placeholder: "Enter Folder ID" },
                { label: "Project ID:", value: projectId, setValue: setProjectId, placeholder: "Enter Project ID" },
              ].map(({ label, value, setValue, placeholder }) => (
                <div className="form-field" key={label}>
                  <label>{label}</label>
                  <input
                    type="text"
                    value={value}
                    onChange={(e) => setValue(e.target.value)}
                    placeholder={placeholder}
                  />
                </div>
              ))}
              <button className="button-edit" onClick={() => {
                localStorage.setItem('editor-sample-backUrl', backUrl);
                localStorage.setItem('editor-sample-authToken', authToken);
                localStorage.setItem('editor-sample-folderId', folderId);
                localStorage.setItem('editor-sample-projectId', projectId);
                setTimeout(() => window.location.reload(), 100);
              }}>
                Initialize
              </button>
            </div>

            {/* Media URL */}
            <div className="form-group">
              <div className="form-field">
                <label>Media URL:</label>
                <input
                  type="text"
                  value={videoUrl}
                  onChange={(e) => setVideoUrl(e.target.value)}
                  placeholder="Enter video URL"
                />
              </div>
              <button className="button-edit" onClick={() => {
                setSubmittedUrl(videoUrl);
                setTimeout(() => setSubmittedUrl(''), 100);
              }}>
                Start Editing
              </button>
            </div>

          </div>

          {/* Right Side - Editor (Takes up remaining space) */}
          <div className="editor-container">
            <SolveigEditor
              src={EDITOR_JS_SRC}
              css={EDITOR_CSS_SRC}
              width="100%"
              height="100%"
              hidden={!showEditor}
              videoUrl={submittedUrl}
              apiUrl={backUrl}
              authToken={authToken}
              folderId={folderId}
              projectId={projectId}
              onVideoAdded={() => handleChangeShowEditor(true)}
              onRenderComplete={handleRenderComplete}
              onProjectUpdate={handleProjectUpdate}
            />
          </div>

        </div>

      </header>
    </div>
  );
}

export default App;

Custom media assets with prepared media info

The media asset's information will be stored at locally installed editor's database. The media asset's file will be downloaded to fetch media info and removed after operation completed.

The user's session will be created with the custom media assets.

Download source code

Configure .env docker variables:

MASTER_SERVER_IP - the editor's ip address

ADMIN_SECRETE - secrete to manage developer profiles.

Register developer profile

Use ADMIN_SECRETE as Authorization header

Replace MASTER_SERVER_IP to actual value

{
  "fetchFileUrl": "http://MASTER_SERVER_IP/api/mediafiles",
  "mediaTabContentUrl": "http://MASTER_SERVER_IP/api/mediafiles",
  "fetchThumbnailUrl":"http://MASTER_SERVER_IP/api/clip_thumbnail"
}

The result is DEV_API_KEY created.

Create media info task

Using DEV_API_KEY as Authorization header create media info task request /api/task/mediainfo and wait for import process completed.

Specify originalUrl to media file:

originalUrl: <string>

or aws bucket credentials as follow:

awsBucket: {
         "awsBucketName":<string>
         "awsAccessKey":<string>,
         "awsSecreteKey":<string>,
}

Register user's session

{
  "apiKey": DEV_API_KEY,
  "apiAuthToken": DEV_API_KEY
}

The result is USER_SESSION_ID created.

Open user's session

Open the user's session via url

http://MASTER_SERVER_IP?session_id=USER_SESSION_ID

NodeJS example

const axios = require('axios')

const originalUrl = 'https://smarteditingonline.solveigmm.com/files/test_video.mp4'
const masterServerIP = '172.16.1.121' // should be changed according to your installation
const adminSecrete = 'kCYqikHH5860' // should be changed according to your installation
const devProfileName = 'Developer'
const devProfileEmail = 'dev@example.com'
const baseURL = `http://${masterServerIP}`

const axiosInst = axios.create({ baseURL })
axiosInst.defaults.headers.common['Authorization'] = adminSecrete


const registerDeveloperProfile = async () => {
    try {
        const profile = {
            name: devProfileName,
            eMail: devProfileEmail,
            fetchFileUrl: `${baseURL}/api/mediafiles`,
            mediaTabContentUrl: `${baseURL}/api/mediafiles`,
            fetchThumbnailUrl: `${baseURL}/api/clip_thumbnail`
        }
        const resp = await axiosInst.post('/api/dev_profiles', profile)
        return resp.data.apiKey
    } catch (e) {
        throw Error(`Error registerDeveloperProfile: ${e}`)
    }
}

const processMediaInfo = async ({ apiKey, originalUrl }) => {
    try {
        const headers = { 'Authorization': apiKey }
        let resp = await axiosInst.post('/api/task/mediainfo', { originalUrl }, { headers })
        const id = resp.data.id
        do {
            resp = await axiosInst.get(`/api/task/mediainfo/${id}`, { headers })
        } while (resp.status === 202)
    } catch (e) {
        throw Error(`Error processMediaInfo: ${e}`)
    }
}

const registerUser = async ({ apiKey }) => {
    try {
        const user = {
            apiKey,
            apiAuthToken: apiKey
        }
        const resp = await axiosInst.post('/api/users', user)
        return resp.data.id
    } catch (e) {
        throw Error(`Error registerUser: ${e}`)
    }
}

const main = async () => {
    const apiKey = await registerDeveloperProfile()
    await processMediaInfo({ apiKey, originalUrl })
    const sessionId = await registerUser({apiKey})
    console.log(`open chrome with the url:  ${baseURL}?session_id=${sessionId}`)
}

main()

Python example

import requests

original_url = 'https://smarteditingonline.solveigmm.com/files/test_video.mp4'
master_server_ip = '172.16.1.121'  # should be changed according to your installation
admin_secret = 'kCYqikHH5860'  # should be changed according to your installation
dev_profile_name = 'Developer'
dev_profile_email = 'dev@example.com'
base_url = f'http://{master_server_ip}'

session = requests.Session()
session.headers.update({'Authorization': admin_secret})

def register_developer_profile():
    try:
        profile = {
            'name': dev_profile_name,
            'eMail': dev_profile_email,
            'fetchFileUrl': f'{base_url}/api/mediafiles',
            'mediaTabContentUrl': f'{base_url}/api/mediafiles',
            'fetchThumbnailUrl': f'{base_url}/api/clip_thumbnail'
        }
        resp = session.post(f'{base_url}/api/dev_profiles', json=profile)
        resp.raise_for_status()
        return resp.json()['apiKey']
    except requests.exceptions.RequestException as e:
        raise Exception(f'Error register_developer_profile: {e}')

def process_media_info(api_key, original_url):
    try:
        headers = {'Authorization': api_key}
        resp = session.post(f'{base_url}/api/task/mediainfo', json={'originalUrl': original_url}, headers=headers)
        task_id = resp.json()['id']

        while True:
            resp = session.get(f'{base_url}/api/task/mediainfo/{task_id}', headers=headers)
            if resp.status_code != 202:
                break
    except requests.exceptions.RequestException as e:
        raise Exception(f'Error process_media_info: {e}')

def register_user(api_key):
    try:
        user = {
            'apiKey': api_key,
            'apiAuthToken': api_key
        }
        resp = session.post(f'{base_url}/api/users', json=user)
        resp.raise_for_status()
        return resp.json()['id']
    except requests.exceptions.RequestException as e:
        raise Exception(f'Error register_user: {e}')

def main():
    api_key = register_developer_profile()
    process_media_info(api_key, original_url)
    session_id = register_user(api_key)
    print(f'Open Chrome with the URL: {base_url}?session_id={session_id}')

if __name__ == '__main__':
    main()

PHP example

<?php

$originalUrl = 'https://smarteditingonline.solveigmm.com/files/test_video.mp4';
$masterServerIP = '172.16.1.121'; // should be changed according to your installation
$adminSecret = 'kCYqikHH5860'; // should be changed according to your installation
$devProfileName = 'Developer';
$devProfileEmail = 'dev@example.com';
$baseURL = "http://$masterServerIP";

function sendRequest($url, $method = 'GET', $data = null, $headers = null) {
    $ch = curl_init();
    // Disable SSL certificate verification (not recommended for production)
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
    curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);

    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, $method);

    if ($data !== null) {
        curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
    }

    if ($headers !== null) {
        curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
    }

    $response = curl_exec($ch);

    if ($response === false) {
        throw new Exception("cURL error: " . curl_error($ch));
    }

    curl_close($ch);

    return $response;
}

function registerDeveloperProfile()
{
    global $baseURL,$adminSecret, $devProfileName, $devProfileEmail;

    $profile = [
        'name' => $devProfileName,
        'eMail' => $devProfileEmail,
        'fetchFileUrl' => "$baseURL/api/mediafiles",
        'mediaTabContentUrl' => "$baseURL/api/mediafiles",
        'fetchThumbnailUrl' => "$baseURL/api/clip_thumbnail",
    ];
    $headers = ['Authorization: ' . $adminSecret,
                'Content-Type: application/json'];
    $response = sendRequest("$baseURL/api/dev_profiles", 'POST', json_encode($profile), $headers);
    $data = json_decode($response, true);
    return $data['apiKey'];
}


function processMediaInfo($apiKey, $originalUrl)
{
    global $baseURL;

    $headers = ['Authorization: ' . $apiKey,
                'Content-Type: application/json'];

    $postData = [
        'originalUrl' => $originalUrl,
    ];
    $response = sendRequest("$baseURL/api/task/mediainfo", 'POST', json_encode(['originalUrl' => $originalUrl,]), $headers);
    $data = json_decode($response, true);
    $id = $data['id'];
    do {
        $response = sendRequest("$baseURL/api/task/mediainfo/$id", 'GET', null, $headers);
        $data = json_decode($response, true);
    } while (array_key_exists('status', $data));
}

function registerUser($apiKey)
{
    global $baseURL;

    $user = [
        'apiKey' => $apiKey,
        'apiAuthToken' => $apiKey,
    ];
    $headers = ['Content-Type: application/json'];
    $response = sendRequest("$baseURL/api/users", 'POST', json_encode($user), $headers);
    $data = json_decode($response, true);
    return $data['id'];
}

function main()
{
    try {
        $apiKey = registerDeveloperProfile();
        processMediaInfo($apiKey, $GLOBALS['originalUrl']);
        $sessionId = registerUser($apiKey);
        echo "open chrome with the url:  $GLOBALS[baseURL]?session_id=$sessionId";
    } catch (Exception $e) {
        echo 'Error: ' . $e->getMessage();
    }
}

main();

?>

Custom media assets using mediaFiles.json

The Custom media assets could be specified by mediaFiles.json list.

The path to json list specified by MEDIA_JSON_LIST config (default uses /var/www/smm_cloud_editor_v2/mediaFiles.json)

MEDIA_FILES_DIR configures root directory for media files (default uses /var/www/smm_cloud_editor_v2/shared/media_json_files)

Example mediaFiles.json:

In this example the 'media_json_video_mxf_xavc.mxf' will be available to all user the 'media_json_url.mxf' will be visible only to single user (1701244924069)

{
  "entries":[
    {
      "originalPath": "media_json_video_mxf_xavc.mxf",
      "proxyPath": "media_json_video_mxf_xavc.mp4"
    },
    {
      "originalUrl": "https://smarteditingonline.solveigmm.com/files/media_json_url.mxf",
      "proxyUrl": "https://smarteditingonline.solveigmm.com/files/media_json_url.mp4",
      "userId": "1701244924069"
    }
  ]
}

Fields Description:

originalPath - The absolute or relative path to original media file

proxyPath - The absolute or relative path to proxy media file (used for preview media files with format natively not supported by browser)

originalUrl - The url of the original media

proxyUrl - The proxy media file url

userId - set the user that will be able to see media file. if not specfied avaliable to all users

To switch user profile please use the following url:

 http://localhost?auth_token=1701244924069

The editor watches json list's changes and updates information on media files in database.

The import process and control it's status uses the same API as for audio library with the url /api/import/mediafiles_json

Output S3 bucket configuration

To upload output results to Amazon S3 Bucket outputS3Bucket should be configured with the following fields:

s3_bucket_object:

{
    "awsBucketName":<string>,
    "awsEndPoint":<string>,
    "awsAccessKey":<string>,
    "awsSecreteKey":<string>,
}

The example permission policy configuration for Amazon is the following:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::<bucketName>"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:GetObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::<bucketName>/*"
            ]
        }
    ]
}

CallBack events

The editor sends POST requests with JSON object body to the callback url that was specified during developer registration.

Common event structure

{
  "type": <string>,
  "userData": <string>,
  "apiAuthToken": <string>
  "payload": <object>
}

type - the event type

userData - the data that was passed at user's session registration

apiAuthToken - the apiAuthToken that was passed at user's session registration

payload- contains event dependant fields

Check event at callbackUrl

Each event's POST request contains the Authorization header that is set to the unique_user_session_id that was created by Registration api ("/api/users"). You can use it to restrict posting of unauthorised data to callback.

Project created

Called when the user created new project

type: project_create

payload:

{
  "projectId":<unique_project_id>,
  "thumbnail":<base64_image_string>
}

Project updated

Called when the user make some changes to project

type: project_update

payload:

{
  "projectId":<unique_project_id>,
  "thumbnail":<base64_image_string>
}

Render project started

Called when the user starts project rendering

type: render_start

payload:

{
  "projectId":<unique_project_id>,
}

Render project completed

Called when the project rendering completed

type: render_complete

payload:

{
  "projectId":<unique_project_id>,
  "downloadUrl":<output_file_url>,
  "error":<error_message>
}

HTML5 Video Editor - Backend Server API

Authorization

Authorization token is used to identify user in API requests,

Backend uses WebSocket connection associated with Authorization token in order to track if the user is online.

For the new connection Authorization token is created as follow (JS):

const client = new WebSocket('ws://backendHost:backendPort/websocket')
  client.onopen = () => {
    const authToken = Date.now().toString()
    client.send(JSON.stringify({
      type: 'user_event',
      payload: { authToken },
    }))
  }
  client.onmessage = message => {
    const dataFromServer = JSON.parse(message.data)
    if (dataFromServer.type === 'ping') {
      client.send(JSON.stringify({
        type: 'pong',
      }))
    }
  }

External authorization server

The backend server could be used along with external Authorization server.

Login form url configured via "EXTERNAL_LOGIN_URL" env variable and will be used to redirect if the user was not authorized or authorization token will expire.

External Authorization server should use "/api/tokens" editor's API to link permanent "userId" with the temporary "token". The "token" live time is configured via "TOKEN_LIVE_HOURS" env variable.

External Authorization server after successfull user's authorization should redirect to https://editor.com?token=<auth_token> to open the editor for the specific user.

Create userId token link

Headers:

"Content-Type": "application/json"

Request:

POST /api/tokens

Request Body:

{
  "userId": <string>,
  "token":<string>,
}

Description:

userId - permanent userId

token - temporary access token

Create user

Headers:

"Content-Type": "application/json"

Request:

POST /api/users

Request Body:

userObject:

{
  "name": <string>,
  "avatarUrl":<string>,
  "data": <json_object>,
  "apiKey":<string>,
  "apiAuthToken": <string>,
  "hotkeysProfiles":<string>
}

Description:

All properties are optional

name - name of the user

avatar - base64 svg or png image of user's profile avatar

data - custom user's data json_object

apiKey - associates the user with the the developer's profile

apiAuthToken - used as Authorization header with custom urls overriden by developer's profile (should be used with apiKey)

hotkeysProfiles - Contains hotkeys profiles definition

Response:

JSON encoded object:

{
  "id": <string>
}

Description:

id - users's id

Update existing user

Headers:

"Content-Type": "application/json"

Request:

PATCH /api/users/<id>

Request Body:

userObject as for create user request

Response:

JSON encoded object:

{
  "id": <string>
}

Description:

id - users's id

Get user data

Request:

GET /api/users/<id>

Parameters:

id - user's id

Response:

{
  "name": <string>
  "data": <string>
}

Create Developer's Profile

Managing developer's profile requires Authorization header to be set as specified in ADMIN_SECRETE config variable

Headers:

"Content-Type": "application/json"
"Authorization": <admin_secrete_token>

Request:

POST /api/dev_profiles

Request Body:

dev_profile_object:

{
   "name":<string>,
   "eMail":<string>,
   "callbackUrl":<string>,
   "fetchFileUrl":<string>,
   "mediaTabContentUrl":<string>,
   "fetchThumbnailUrl":<string>,
   "outputS3Bucket": {
      "awsBucketName":<string>,
      "awsEndPoint":<string>,
      "awsAccessKey":<string>,
      "awsSecreteKey":<string>,
   }
   "companyName": <string>,
   "appTitle": <string>,
   "appDescription":<string>,
   "svgLogo":<string>,
   "hideSVGLogo":<boolean>,
   "previewPlayerLogo":<string>,
   "hidePreviewPlayerLogo":<boolean>,
   "favIcon":<string>,
   "defaultUserAvatar":<string>,
   "hideDefaultUserAvatar":<boolean>,
   "customCSS": <string>,
   "customJS": <string>,
   "customLanguage": <string>,
   "imageLibSrcType": <string>,
   "pexelsApiKey":<string>,
   "unsplashApiKey": <string>,
   "unsplashApiSecrete":<string>,
   "unsplashAppName":<string>,
   "pexelsApiKey":<string>
   "defaultHotkeysProfile":<string>
}

Description:

Required fields:

name - The name of the profile

eMail - The profiles' e-mail

Optional fields:

callbackUrl - The url to recieve callbacks from the editor

fetchFileUrl, mediaTabContentUrl, fetchThumbnailUrl - Custom media files' urls

outputS3Bucket - Bucket to upload output media files

companyName - Displayed on preview player window

appTitle - Displayed at header and browser tab

appDescription - Meta description tag of the web page

svgLogo - Base64 string or url with the logo content( SVG and PNG formats supported) that should be displayed at header, preview player window (if not overriden by 'previewPlayerLogo' ), and browser tab icon (if not overriden by favIcon)

hideSVGLogo - Hide logo logo at header and preview

previewPlayerLogo - Base64 string or url with the logo content( SVG and PNG formats supported) that overrides preview player window logo

hidePreviewPlayerLogo Hide logo at preview player window

favIcon - Base64 string or url with the logo content( ICO formats supported) that overrides browser tab icon

defaultUserAvatar - Base64 string or url with the logo content(SVG and PNG formats supported) that should be displayed if no user avatar specified.

hideDefaultUserAvatar - Hide user's avatar logo

customCSS - plaintext CSS or url with overriden styles

customJS - plaintext JS or ulr with code

customLanguage - plaintext JSON or url with text definitions. See Language JSON format for details.

defaultHotkeysProfile - plaintext JSON or url with hotkeys definitions for the default profile. See Hotkeys JSON format for defails.

imageLibSrcType - coma separated image sources (could be json, unsplash, pexels)

unsplashApiKey, unsplashApiSecrete, unsplashAppName - credentials to use unsplash.com as source for images library

pexelsApiKey - credentials to use pexels.com as source for images library

In case of using external hosted resources to prevent browser CORS errors make sure the respose contains headers:

 'Access-Control-Allow-Origin' '*'
 'Access-Control-Allow-Headers' 'authorization,content-type'

Response:

JSON encoded object:

{
 "apiKey":<string>,
}

Language JSON format:

The language json template is the following:

{
  languageCode: {
  textId1: textValue1,
  textId2: textValue2,
  }
}

languageCode - Language code according to RFC 5664 Specification

textId1 - Unique id for the text. The full list of available values could be get from UI "Customization Editor" using "Edit Text values" button

textValue1 - The value of the text for the specific language

The language json example:

{
"en": {
  "EDITOR": "Editor",
  "PROJECTS": "Projects",
  "MY_MEDIA": "My Media",
  "MY_RECORDINGS": "My Recordings",
  "TRANSITIONS": "Transitions",
  "TEXT": "Text",
  "AUDIO": "Audio",
  "FILTERS": "Filters",
  "DESIGN": "Design",
  "IMAGES": "Images"
 }
}

Hotkeys JSON format: The hotkeys json template is the following:

{
  profileName:{
    functionId1: keyCodeValue1,
    functionId1: keyCodeValue1
  },
  active: profileName
}

profileName - the hotkeys profile name

functionId1 - the unique function id

keyCodeValue1 - the unique key code value

active - defines the current active hotkeys profile

Update existing developer's profile

Headers:

"Content-Type": "application/json"
"Authorization": <admin_secrete_token>

Request:

PATCH /api/dev_profiles/<api_key>

Request Body:

dev_profile_object as for create request

Response:

JSON encoded object:

{
  "apiKey": <string>
}

List dev profiles

Headers:

"Authorization": <admin_secrete_token>

Request:

GET /api/dev_profiles?start_number=0&count=10

Parameters:

start_number - profiles num to start with

count - count per page

Response:

{
  "totalCount":<number>,
  "profiles":[<dev_profile_object>]
}

Description:

totalCount - defines total count of profiles

profiles - An array of dev_profile_object

Delete dev profile

Headers:

"Authorization": <admin_secrete_token>

Request:

DELETE /api/dev_profiles/<api_key>

Get media files list

Headers:

"Authorization": <string>

Request:

GET /api/mediafiles?start_number=0&count=20&filter=video&sort_field=createTime&sort_order=asc&search=abc&folder=<id>

Parameters:

start_number- the start number of first element in requested media list

count - the max count of requested entries

filter - defines applied filter to all files by types could be video, audio, image, folder, voiceover(optional, could be comma separated combination of multiple items)

sort_field - defines sort field

name - by name of the file
createTime - by creation time (default)
size - by size
duration - by duration
startTimeCodeFrames - by media file's start TC

sort_order - defines order in which entries should be sorted(optional)

asc - ascending
desc - descending (default)

search - defines search string that will be applied to entries(optional)

folder - defines the folder with to get files from(optional)

Response:

{
 "total_count":4,
 "entries":[
    <media_file_entry_object>,
    <media_file_entry_object>,
    <media_file_entry_object>,
    <media_file_entry_object>,
  ]
}

Description:

total_count - defines total count of files entries - An array of media_file_entry_objects

Get information on media file

Used to get information on single media file

Headers:

"Authorization": <string>

valid_auth_token - was received on authorization

Request:

GET
/api/mediafiles/<id>

Parameters:

id- media file id

Response:

JSON encoded media_file_entry_object object:

{
  "id":<string>,
  "name":<string>,
  "type":<string>,
  "filetype":<string>,
  "mediaType":<string>,
  "duration":<number>,
  "size":<number>,
  "width":<number>,
  "height":<number>,
  "thumbnail":<base64_encoded_png_image>,
  "previewUrl":<string_url>,
  "originalUrl"::<string_url>,
  "audioTracks":[<string>,<string>],
  "status":<string>,
  "progress":<number>,
  "error": <media_error_description_object>,
  "taskId": <string>,
  "categoryId":<string>,
  "moodId":<string>
}

id - defines file id

name - defines file name of entry

type - defines entry type, could be "test", "uploaded", "url", "output"

test - test file available to all users
uploaded - files that was uploaded by user
url - file is imported from url
output - files that was exported/rendered

filetype - defines entry type, could be: "mp4", "mov", "mp3", "png","m4a","jpg","mxf", "folder", "webm"

mediaType - defines entry media type, could be: "video", "audio", "image"

duration - is duration in 100ns units,

size - file size in bytes,

width, height- is video resolution in pixels of source media

thumbnail- is png image converted to base64 encoded string.

previewUrl - url for playing media file

originalUrl - url for original media file. In case of using proxy file for previewUrl

audioTracks - array of audio tracks names

status- defines media file status if some operation is in progress, could be

render - status for project output file that is rendering now
upload - status for media file that user is uploading now
import - status for media file that is importing from url now
convert - status for media file that is converting now
prepare_thumbnails - status for media file that is involved in thumbnails building process.
ready- status indicates that on media file there will no further operations expected

progress- determine progress of the current stage

error - determine error description JSON object <media_error_description_object>

taskId - determine id of render task used for output files (optional)

categoryId - category id of the media file, used with audio library files (optional)

moodId - mood id of the media file, used with audio library files (optional)

Media error description object

media_error_description_object:

{
  "type": <error_type>,
  "message": <text>,
  "title": <error_title>,
}

error_type could be:

exceeded_file_size - Exceeded single file size limit
exceeded_per_user_size - Exceeded per user size limit
low_disk_space - low disk space on server
unsupported_media - media file is not supported
internal_server_error - Internal server error
not_found - media file not found
used_in_projects - media file is used in projects

text- contains error description

error_title - the error title

Get status on media file

Used to get status on media_file_entry_object

Headers:

"Authorization": <string>

valid_auth_token - was received on authorization

Request:

GET
/api/mediafile/status?&id="1233f2"

Parameters:

id - media file id

Response:

JSON encoded object

{
  "status":<media_status_string>,
  "progress":<number>,
  "error": <media_error_description_object>,
  "taskInfo":<task_info_object>
}

Description:

media_status_string - string description of current status

new - initial status for new files
upload - the media is now uploading by user
s3_upload - the media is uploading to s3 bucket by server
import - the media is downloading from url
convert - the media is converting
mediainfo - the server fetches media info
prepare_thumbnails - the prepare thumbnails is in progress
ready - the final status for media is ready to use
render - the media is in render status (used for output items in media tab)

progress - main progress in percent

error - see media_error_description_object

taskInfo - see task_info_object (optional):

task_info_object

Object provides current task information

{
  status:<task_status_string>,
  progress: <number>
}

status - current task's status

new - initial task status
running - task is running
completed - task was successful completed
aborted - task was aborted by user or error message
downloading - downloading media file
uploading - uploading media file

progress - current status progress in percent

Update media file properties

Used update media file properites

Headers:

"Authorization": <string>

valid_auth_token - was received on authorization

Request:

PATCH
/api/mediafiles/<id>

Parameters:

id- media file id

Request Body:

{
  "publicAccess": <number 0|1>
}

Description:

publicAccess - defines the file could be accessed by other users

Delete media file

Used to delete media_file_entry_object

Headers:

"Authorization": <string>

valid_auth_token - was received on authorization

Request:

DELETE
/api/mediafiles/<id>?force=1

Parameters:

id - media file id

force - ignore error

Response:

{
  "error": <media_error_description_object>
}

Working with folders

Create folder

Creates new folder with specified name

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST /api/folders

valid_auth_token - was received on authorization

Request Body:

{
  "name": "folder_name"
  "folder": "folder_id"
}

Description:

name - the name of the folder,

folder - the folder's id to create new folder in

Response:

JSON encoded object

{
  "id":"2sdv342s8834"
}

Delete folder

Removes folder with all it's content recursively with subfolders

Headers:

"Authorization": <string>

Request:

DELETE
/api/folders/<id>?force=1

Parameters:

id - folder's id

force - ignore error

Response:

{
  "error": <media_error_description_object>
}

Update folder name

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

PATCH
/api/folders/<id>

Parameters:

<id>- requested folder id

Request Body:

{
  "name": <folder_new_name>,
}

Description:

name- name of the folder. Optional

Response:

JSON encoded object:

{}

Move media file(s), folders to another folder

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

PUT
/api/folders/<id>/entries

Parameters:

<id>- requested folder id

Request Body:

{
  "files": [],
  "folders": [],
}

Description:

files - an array of file ids to move to folder ( Optional)

folders- an array of folder ids to move to folder ( Optional)

NOTE: in case case of moving files to root folder /api/folders/root/entries should be used

Audio library

Get audio library files list

Fetches list of audio library files. To initiate audio library files fetch process use /api/import/audiolib_json sort_field

Request:

GET /api/audiolib_files?type=music&start_number=0&count=20&sort_field=createTime&sort_order=asc&search=abc&category_id=<id>&mood_id=<id>

Parameters:

start_number - the start number of first element in requested media list

count - the max count of requested entries

type - selects the list of specific type could be:

music- audio tracks with music composition (default)
sound - audio samples of specific sounds or noises

category_id - optional field to filter by specific category

mood_id - optional field to filter by specific mood

sort_field - defines sort field

name - by name of the file (default)
createTime - by creation time

sort_order - defines order in which entries should be sorted(optional)

asc - ascending (default)
desc- descending

search - defines search string that will be applied to entries(optional)

Response:

Response is the same as for api /api/mediafiles

Get information on audio library file

Used to get information on audio library file

Request:

GET /api/audiolib_files/<id>

Response:

Response is the same as /api/mediafiles/

Get categories list for audio library files

Fetches list of existing categories

Request:

GET /api/audiolib_files/categories?type=music

Parameters:

type - field to define type of the list could be:

music - audio tracks with music composition (default)
sound - audio samples of specific sounds or noises

Response:

JSON array describing media_file_entry_object's:

{
  "entries":[
   <auidolib_category_object>,
             <auidolib_category_object>,
   <auidolib_category_object>,
   <auidolib_category_object>,
  ],
}

<auidolib_category_object> is JSON object

{
  "id": <unique_id>
  "name": <category_name>
  "entriesCount": 20
}

Get Moods list for audio library files

Fetches list of existing categories

Request:

GET /api/audiolib_files/moods?type=music

Parameters:

type - field to define type of the list could be:

music - audio tracks with music composition (default)
sound - audio samples of specific sounds or noises

Response:

JSON array describing media_file_entry_object's:

{
  "entries":[
    <auidolib_mood_object>,
             <auidolib_mood_object>,
    <auidolib_mood_object>,
    <auidolib_mood_object>,
  ],
}

<auidolib_mood_object> is JSON object

{
  "id": <unique_id>
  "name": <mood_name>
  "entriesCount": 20
}

Images library

The Image library api could use different sources:

json - locally hosted image library defined by imageLib.json file
unsplash - Unsplash image library. Credential could be configured via backend environment variables UNSPLASH_API_KEY, UNSPLASH_API_SECRETE or unsplashApiKey, unsplashApiSecrete - variables with /api/dev_profiles
pexels - Pexels image library. Credential could be configured via backend environment variable PEXELS_API_KEY or pexelsApiKey - variables with /api/dev_profiles

The "imageLibSources" payload variable of "user_info" websocket message contains the list of available sources for the current user session.

Get images library files list

Headers:

"Authorization": <string>

Request:

GET
/api/imagelib_files/<source>?page=<number>&count=<number>&search=<string>&sort_field=createTime&sort_order=asc&category_id=<string>

Parameters:

source - defines the image library source

page- the page number

count - the count of entries per page

category_id - optional field to filter by specific category

sort_field - defines sort field

name - by name of the file (default)
createTime - by creation time

sort_order - defines order in which entries should be sorted(optional)

asc - ascending(default)
desc - descending

search - defines search string that will be applied to entries(optional)

Response:

JSON encoded object

{
	entries: [<image_library_object>]
	total_count:<number>
}

entries - an array of image_library_objects

total_count - number of all entries matched to request

Get information on image library file

Headers:

"Authorization": <string>

Request:

GET
/api/imagelib_files/<source>/<id>

Parameters:

source - defines the image library source

id - defines image file id

Response:

JSON encoded image_library_object:

{
    "id":<string>,
    "name":<string>,
    "width":<number>,
    "height":<number>,
    "thumbnail":<string>,
    "previewUrl":<string>,
    "originalUrl":<string>,
    "webpageUrl":<string>,
    "userName":<string>,
    "size":<number>,
    "filetype":<string>,
}

name - the title of the image

width, height - is image resolution in pixels of source media

thumbnail - is png image converted to base64 encoded string or url

previewUrl - url for preview

originalUrl - original url for media

webpageUrl - web page of the image resource on external resource(optional)

userName - the user's name that image resource belongs to (optional)

size - the size of the original image in bytes (optional)

filetype - defines entry type (could be: "png", "jpg")

Headers:

"Authorization": <string>

Request:

GET /api/imagelib_files/<source>/categories?page=<number>&count=<number>search=<string>

Parameters:

source - defines the image library source

page- the page number

count - the count of entries per page

search - defines search string that will be applied to entries(optional)

Response:

JSON object:

{
 "entries":[<imagelib_category_object>],
 "total_count": <number>
}

entries - an array of image_library_category_objects

total_count - number of all entries matched to request

imagelib_category_object:

{
	"id": <sting>,
	"name": <string>,
	"entriesCount": <number>,
	"description":<string>,
	"parent": <string>,
}

id - unique category id

name - the category name

entriesCount - the entries count in this category

description - the description for category(optional)

parent - the parent category id (optional)

Thumbnails for clip on timeline

To get thumbnails for the clip on timeline from prepared database of thumbnails use

cached=true parameter in request

if cached=false thumbnails will be generated in real time

To get list of thumbnails for the media file with step and requested count

Request:

GET
/api/clip_thumbnails?id=0&start_time=10000000&step=5000000&count=20&cached=true

Parameters:

id- the media file's id ( see /api/mediafiles)

start_time - the time in 100ns units for the start time

step - the time in 100ns units time step

count - count of requested thumbnails

cached - should get thumbnails from prepared database

Response:

JSON encoded object

{
 "status":"",
 "thumbnails":[
    {
      time:0
      data:"base64_encoded_png_image"
    },
    {
      time:5000000
      data:"base64_encoded_png_image"
    }
  ]
}

Description:

status - contains error message, empty string means operation success,

thumbnails - an array of thumbnails objects,

time - thumbnail time in 100ns units,

data - base64_encoded_png_image

Get single thumbnail

Request:

GET
/api/clip_thumbnail?id=0&time=10000000

Parameters:

id- the media file's id ( see /api/mediafiles)

time - thumnbail's time in 100ns units

Response:

JSON encoded object

{
  "data":<base64_encoded_png_image>
}

Description:

data - is png image converted to base64 encoded string.

Editing task

Create new task

Editing Task is passed via POST request the request body should contain json object with XTL format task

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST
/api/task/editing

Request Body:

{
  "xtl_task": <valid_xtl_task_string>
}

Example of xtl_task:

<timelines version="2">
  <timeline>
    <group >
      <track video="1" audio="1" accuracy="frame">
        <clip src="media_file_id" start="00:00:01:00" stop="00:00:06:00" timeFormat="time10ms" />
        <clip src="media_file_id" start="00:00:07:00" stop="00:00:12:00" timeFormat="time10ms" />
      </track>
    </group>
  </timeline>
</timelines>

Response:

JSON encoded object

{
  "taskId":"2sdv342s8834"
  "fileId": "output_23234234234.mp4"
  "error":  <error_description_message>
}

Description:

taskId - an id of newly created task

fileId - identifies output Media File

error - contains error description (optional)

Editing task operation status

To get task's status use GET request

Headers:

"Authorization": <string>

Request:

GET
/api/task/editing?id=<task_id>

Parameters:

id- requested task's id

Response:

JSON encoded object

{
  "id":<task_id>,
  "progress":50,
  "status":"running",
  "error": <engine_error_description_object>,
  "outputUrls":[
    "http://35.234.23.3/output_files/43234/32343/name.mp4"
  ]
}

Description:

id - an id of task progress - task progress in percent status- string description of current task status could be:

running - task is running
completed - task was successful completed
aborted - task was aborted by user or error message
downloading - downloading media files before trimming (in case of "url" file types)
uploading - uploading saved result (e.g. to amazon bucket)

error - task's Engine error description object outputUrls - links to output media file(s)

Engine error description object

engine_error_description_object:

{
  "type": <type>,
  "message": <message>,
  "code": <code>,
}

type - could be:

engine_error - Edit engine error
download_error - unable to download input media
upload_error - upload error

message - contains error description code - internal engine error code

Stop editing task

To stop task use DELETE request

Headers:

"Authorization": <string>

Request:

DELETE
/api/task/editing?id=<task_id>

Parameters:

id- requested task's id

Response: JSON encoded object

{
  "status":"",
}

Projects

Create new project

Project passed via POST request the request body should contain json object with project's data

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST
/api/projects

Request Body:

{
  "name": <project_name>,
  "data": <project_data>,
  "thumbnail": <base_64_string>,
  "createTime": <unix_timestamp_ms>
}

Description:

name- name of the project.

data - actual project's data.

thumbnail - the project thumbnail

createTime - the project createTime (Optional). If not set the backend time will be used

Response:

JSON encoded object:

{
  "id": "B7pAdG3HJUZ25uJkwuuVcQH6Hkndx6gARcV76EV4"
}

Description:

id - project's id

Error Codes:

400 - bad request

404 - no user or no project with id found

500 - internal server error

Update existing project

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

PATCH
/api/projects/<id>

Parameters:

<id>- requested project id

Request Body:

{
  "name": <project_name>,
  "data": <project_data>,
  "thumbnail": <base64_string>,
  "modifTime":<unix_timestamp_ms>
}

Description:

name- name of the project. Optional

data - actual project's data. Optional

thumbnail - thumbnail for the project. Optional

modifTime - project's modification time. Optional. If not set the backend time will be used.

Response: JSON encoded object:

{}

Error Codes:

400 - bad request

404 - no user or no project with id found

500 - internal server error

Get existing project's data

To get project's data use GET request

Headers:

"Authorization": <string>

Request:

GET
/api/projects/<id>

Parameters:

<id> - requested project id

Response:

JSON encoded <project_object>:

{
  "id":<project_id>,
  "data":<project_data>,
  "name":<project_name>,
  "createTime":<unix_timestamp>,
  "modifTime":<unix_timestamp>,
  "thumbnail": <base64_string>
}

Description:

id - an id of the project

data - actual project's data

name - name of the project

createTime - project's creation time

modifTime - last project's modification time

thumbnail - thumbnail for the project

Error Codes:

400 - bad request

404 - no user or no project with id found

500 - internal server error

Get projects list

Headers:

"Authorization": <string>

Request:

GET /api/projects?start_number=<number>&count=<number>&data=<0|1>&search=<string>

Parameters:

start_number - the start number of first element in requested project list

count - the max count of requested entries

data - fill <project_object>'s data field for non zero value

search - defines search string that will be applied to entries(optional)

Response:

JSON array describing media_file_entry_object's:

{
 "total_count":<number>,
 "entries":[
        <project_object>,
        <project_object>,
        <project_object>,
        <project_object>,
        ],
}

Description:

total_count - defines total count of files The project_object description could be found in /api/project

Error Codes:

400 - bad request

404 - no user or no project with id found

500 - internal server error

Check media presence in projects

Headers:

"Authorization": <string>

Request:

GET /api/projects/presence?file_id=<id>

Parameters:

file_id - defines file's id to search

Response:

JSON array describing media_file_entry_object's:

{
 "presence":<boolean>,
}

Description:

total_count - defines total count of files The project_object description could be found in /api/project

Error Codes:

400 - bad request

404 - no user or no project with id found

500 - internal server error

Delete project

Used to delete project

Headers:

"Authorization": <string>

Request:

DELETE
/api/projects/<id>

Parameters:

id - project id

Response:

{}

Error Codes:

400 - bad request

404 - no user or no project with id found

500 - internal server error

### Actions history

The action history is saved at backend to keep project's changes between autosave intervals and ability to restore the changes if the user disconnected between autosave intervals. The order of POST requests will define the order of the entries returned by GET The total actions count for the project configured by ACTION_HISTORY_DEPTH (default value 1000) backend variable. Exceeding this count will remove earlier actions. The actions are linked to the project, removing the project removes all actions related to it.

Create new action

Action passed via POST request the request body should contain json object with action's data. On new action all action with canceled="true" field will be removed, before posting action that will remove canceled actions the project should be in sync.

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST
/api/actions

Request Body:

{
  "id": <action_id>,
  "project": <project_id>,
  "name": <action_name>,
  "data": <action_data>,
  "createTime": <unix_time_ms>,
}

Description:

id - action's id

projectId- project's id to which action belongs to

name - the name of the action

data - the action data

createTime- the action's createTime (Optional). If not set the backend time will be used

Response:

JSON encoded object:

{
  "id": "1632798732"
}

Description:

id - actions'id

Error Codes:

400 - bad request

404 - no user or no project with id found

500 - internal server error

Undo Redo action

In order to undo/redo action, the "canceled" field should be updated.

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

PATCH
/api/actions/<id>

Parameters:

<id>- requested action id

Request Body:

{
  "modifTime": <unix_time_ms>
  "canceled": true
}

Description:

canceled- true means action was undoed false action is active

modifTime - the time of modification. If not set backend time will be used

Response:

JSON encoded object:

{}

Error Codes:

400 - bad request

404 - no user or no project with id found

500 - internal server error

Get actions list

Headers:

"Authorization": <string>

Request:

GET /api/actions?project=<valid_project_id>

Parameters:

project - the project id that actions belongs to

Response:

JSON array describing media_file_entry_object's:

{
  "entries":[
      <action_object>,
      ],
}

Description:

The action object:

{
  "id": <action_id>,
  "name": <action_name>,
  "data": <action_data>,
  "canceled": <boolean>,
  "createTime": <unix_time_ms>
  "modifTime": <unix_time_ms>
}

id - action id.

name - name of the action.

data - actual action's data.

canceled - "true" - action was canceled (undo operation was done), "false" - action is active

createTime - action's creation time unix time in ms

modifTime - action's modification time unix time in ms

The entries are in the order that they were POSTed.

Error Codes:

400 - bad request

404 - no user or no project with id found

500 - internal server error

Overlay image upload

For editing with text overlays or image overlay it's required to put image data of rendered text or image to use it in editing task, it will be automatically deleted after user disconnected.

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST
/api/overlay_img

Request Body:

{
  "data":"base64_encoded_png_image"
}

Description:

data - base64 encoded_png_image

Response: JSON encoded object

{
  "status":"",
  "id":<overlay_img_id>,
}

status - contains error message, empty string means operation success,

id - an id of overlay_img

Local media upload

Create an upload session on response upload_session_id will be available. The upload session is valid within 24 hours. Upload media file parts with specifying upload_session_id. The progress of current session could be gotten via upload_session_id

Create an upload session

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST
/api/upload/media

Request Body:

{
  "size":12345
  "name":"my media.mp4"
  "folder":"folder_id"
}

Description:

valid_auth_token - User gets this token after authorization

size - is number of bytes of file data, which will be transferred in subsequent requests

folder - destination folder's id (optional)

Response:

JSON encoded object

{
  "id":<upload_session_id>,
}

id - an id of uploaded media

Error Codes:

400 - bad request

404 - no user found

500 - internal server error

507 - Upload limit exceeded error

514 - Unsupported media

Put media file parts to upload session

Headers:

"Authorization": <string>
"Content-Length": <chunk_size>
"Content-Range": bytes <range_start>-<range_end>/<total_size>
"Content-Type": "application/octet-stream"

Request:

PUT
/api/upload/media?id=<upload_session_id>

Request Body:

<binary_data>

Description:

chunk_size - number of bytes in the current chunk

<range_start>-<range_end>/<total_size> - show which bytes in the file you upload. For example, Content-Range: bytes 0-524287/2000000 shows that you upload the first 524,288 bytes (256 x 1024 x 2) in a 2,000,000 byte file ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Range )

Response:

Status 200 - the media file was uploaded Status 206 - more data expected

Use the Range header in the response to determine where to start the next chunk. The header contains information on which bytes the server has received. Do not assume that the server received all bytes sent in the previous request.

For example, Range bytes=0-524287 - means that server has received the first 524,288 bytes ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range )

Status 5xx - upload request was interrupted

Import media by URL

Initiates process of importing media by url at backend side.

Create an url import request

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST
/api/import/media/url

Request Body:

{
  "url":<string>
  "folder":<string>
}

Description:

url - url to import from

folder - destination folder's id (optional)

Response:

JSON encoded object

{
  "id":<imported_media_id>,
}

id - an id of imported media

Error Codes:

400 - bad request

404 - no user found

500 - internal server error

Import AWS bucket

Initiates process of importing media by aws bucket credentials at backend side.

Create an aws bucket import request

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST
/api/import/media/awsbucket

Request Body:

{
  "awsAccessKey":<key>,
  "awsSecreteKey":<secrete>,
  "awsBucketName":<bucket_name>,
}

Description:

awsAccessKey, awsSecreteKey, awsBucketName - Credentials to access aws bucket.

Response:

JSON encoded object

{
  "id":<awsbucket_id>,
  "error":<error description>,
}

id - an id of imported aws_bucket

error - description if present

Error Codes:

400 - bad request

404 - no user found

500 - internal server error

Import audio library files from json list

Initiates process of fetching audio library files using json list. The json list's absolute path is set by env variable AUDIOLIB_JSON_LIST (default uses /var/www/smm_cloud_editor_v2/audioLib.json) The media absolute path is configured by AUDIOLIB_FILES_DIR env variable

The example of audioLib.json:

{
    "entries": [
        {
            "name": "Bubble_G_Funk_Remastered.mp3",
            "relativePath": "Bubble_G_Funk_Remastered.mp3",
            "category": "Electro",
            "type": "music"
        },
        {
            "name": "cats.mp3",
            "relativePath": "cats.mp3",
            "category": "animals",
            "type": "sound"
        },
    ]
}

Create an audio library files import request

Initiate importing process. To get the progress and errors use GET

Request:

POST
/api/import/audiolib_json

Response:

JSON encoded object

{
}

Error Codes:

500 - internal server error

Get an audio library files import process status

Request:

GET
/api/import/audiolib_json

Response:

JSON encoded object

{
  "processedCount": 1
  "totalCount": 256
  "error":<error description>,
}

error - contains error description if present

processedCount - contains count of files already handled

total count - contains total number of files

Error Codes:

500 - internal server error

Import images library files from json list

The json list's absolute path is set by env variable IMAGELIB_JSON_LIST (default uses /var/www/smm_cloud_editor_v2/imageLib.json)

The media absolute path is configured by IMAGELIB_FILES_DIR env variable

The example of imageLib.json:

{
    "entries": [
        {
            "name": "Chair",
            "relativePath": "architecture1.jpg",
            "category": "Architecture"
        },
        {
            "name": "Blue Ocean",
            "url": "https://smarteditingonline.solveigmm.com/files/ocean.jpg",
            "category": "Nature",
            "subCategory":"Ocean",
            "width":1920,
            "height":1080,
            "thumbnail":"https://smarteditingonline.solveigmm.com/files/ocean.jpg",
            "size":241888,
            "filetype":"jpg",
            "userName":"SolveigMM",
            "userWebPage":"https://solveigmm.com"
        }
    ]
}

Fields Description:

name - The name of the image displayed in media

relativePath - The relative path to IMAGELIB_FILES_DIR of the image on local drive

url - The url of the original image

category - The root category name for the image

subCategory - The subcategory name for the image

width, height - The original image dimensions

thumbnail - The thumbnail url to be displayed in media (the default size required 174x113, other size will be scaled automatically)

size - The file size of the image displayed in popup

filetype - The file type of the image displayed in popup

userName, userWebPage - used to reference image author

The import process and control it's status uses the same API as for audio library with the url /api/import/imagelib_json

Import Project

Create an import session

Creates import project session on response the id of the new project will be available

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST
/api/upload/project

Request Body:

{
  "size":12345
  "name":"project.edl.zip"
  "fps":29.97
  "createFolder":true
}

Description:

size - is number of bytes of file data, which will be transferred in subsequent requests

name - the name of the uploaded project file (optional)

fps - the fps that will be used at the import stage e.g. for edl that is required(optional)

createFolder - import media files to the folder with the project name or to root media (optional)

Response:

JSON encoded object

{
  "id":<project_id>,
}

id - an id of the new project

Error Codes:

400 - bad request

500 - internal server error

Put project file parts to import session

Headers:

"Authorization": <string>
"Content-Length": <chunk_size>
"Content-Range": bytes <range_start>-<range_end>/<total_size>
"Content-Type": "application/octet-stream"

Request:

PUT
/api/upload/project?id=<project_id>

Request Body:

<binary_data>

Description:

valid_auth_token - User gets this token after authorization

chunk_size - number of bytes in the current chunk

<range_start>-<range_end>/<total_size> - show which bytes in the file you upload.

For example, Content-Range: bytes 0-524287/2000000 shows that you upload the first 524,288 bytes (256 x 1024 x 2) in a 2,000,000 byte file ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Range )

Response:

Status 200 - the media file was uploaded

Status 206 - more data expected

For example Range bytes=0-524287 - means that server has received the first 524,288 bytes ( https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range )

Status 5xx - upload request was interrupted

Get import project session progress and status

Headers:

"Authorization": <string>

Request:

GET
/api/upload/project?id=<project_id>

Response:

JSON encoded object

{
  "stage": <import_project_stage>,
  "progress": 0
  "error": "error description"
}

Description:

import_project_stage - the status of the import process, could be:

"upload" - the data is uploading now
"unpack" - the data is unpacking
"import_media" - import media files
"ready" - the import process completed

progress - the progress value of the current stage

error - error description if present

Cancel import project session

Headers:

"Authorization": <string>

Request:

DELETE
/api/upload/project?id=<project_id>

Response:

JSON encoded object

{}

Export Project

Create export session

Creates export project session on response the id of session will be available

Headers:

  "Authorization": <valid_auth_token>

Request:

POST
/api/task/export_project

Request Body:

{
   "projectId":<string>
   "includeMedia":<boolean>
   "exportType":<string>
}

Description:

valid_auth_token - User gets this token after authorization

id - is requested project id

includeMedia - include media files used in project

exportType - could have the following values:

edl - Edit decision list
json - json formated project

Response:

JSON encoded object

{
 "id":<string>
}

id - id of export session

Еxport project session status

Get export project session progress and status

Headers:

  "Authorization": <valid_auth_token>

Request:

GET
/api/task/export_project?id=<string>

Response:

JSON encoded object

{
 "progress": <number>
 "status": <string>
 "error": <error_description_object>
 "url": <string>
}

Description:

progress - the progress value

status- string description of current task status could be:

running - task is running
completed - task was successful completed
aborted - task was aborted by user or error message
downloading - downloading media files before trimming (in case of “url” file types)
uploading - uploading saved result (e.g. to amazon bucket)

error - error description object if present( <error_description_object>)

url - is url link to download exported project archive

Cancel export project session

Headers:

  "Authorization": <valid_auth_token>

Request:

DELETE
/api/task/export_project?id=<string>

Response:

JSON encoded object

{}

Voiceover recording

Voiceover is implemented using kurento media server

Create voiceover recording session

Creates webrtc connection and start sending voice data to server

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

POST
/api/voiceovers

Request Body:

{
  "sdpOffer":<sdp_offer>
  "iceCandidates":<array_of_local_ice_candidates>
}

Description:

offer - SDP offer string created by browser webRtcPeer.generateOffer

iceCandidates - local ice candidates array gathered by browser

Response:

JSON encoded object

{
  "id":<new_voiceover_id>
  "sdpAnswer": <remote_sdp_answer>
  "iceCandidates": <array_of_remote_ice_candidates>
}

Error Codes:

500 - internal server error

Start voiceover recording

Start record voiceover data to file

Headers:

"Content-Type": "application/json"
"Authorization": <string>

Request:

PUT
/api/voiceovers/<id>

Request Body:

{
  "name": <the file name for_recording>
}

Description:

id - recording id

name - new recording file name (optional) , the backend will set name automatically if missing

Response:

JSON encoded object

{
  error: <error description>
}

error - Error description if present

Error Codes:

500 - internal server error

Stop and delete voiceover recording session

Headers:

"Authorization": <string>

Request:

DELETE
/api/voiceovers/<id>?keepRecording=1

Description:

id - recording id

keepRecording - flag determines keeping media file recorded during the session

Response:

JSON encoded object

{
}

Error Codes:

500 - internal server error

Nablet Highlights detect

Create hld task

Headers:

"Authorization": <string>

Request:

POST
/api/task/nb_hld

Request Body:

{
  "fileId":<string>,
  "targetDurationSec":<number>,
  "onnxName":<string>
}

Description:

fileId - media file id

targetDurationSec - the required duration in seconds

onnxName - the neural network name to use for detection. To get available use /api/task/nb_hld/list_onnx

Response:

JSON encoded object

{
  "id":<string>
}

Get hld task status

Headers:

"Authorization": <string>

Request:

GET
/api/task/nb_hld/<id>

Response:

JSON encoded object

{
  "status": <string>,
  "progress":<number>,
  "cutLists": <array_of_cut_objects>,
  "error": <error_object>
}

Description:

status - current task's status

new - initial task status
running - task is running
completed - task was successful completed
aborted - task was aborted by user or error message
downloading - downloading media file

progress - current progress in percent

cutLists - Array of the labeled interval's start,stop in "100ns_units"

[{ "label": <string>, "intervals":[[<number>, <number>], ...]}, ...]

error - error description if present (optional)

Cancel hld task

Headers:

"Authorization": <string>

Request:

DELETE
/api/task/nb_hld/<id>

List of neural networks

Request:

GET
/api/task/nb_hld/list_onnx

Response:

JSON encoded object

{
  "list": [<string>,...],
}

Description:

list - Array of neural networks names

Statistics

Get backend live statistics.

Request:

GET
/api/stat

Response:

JSON encoded object

{
  "activeUsersCount":1,
  "activeBuildThumbsTasksCount":0,
  "activeRenderTasksCount":1,
  "activeImportFromURLTasksCount":0,
  "activeImportFromLocalDriveCount":0
}

Description:

activeUsersCount - Count of active user

activeBuildThumbsTasksCount - Count of active build timeline thumbnails tasks

activeRenderTasksCount - Count of active render tasks

activeImportFromURLTasksCount- Count of files that are downloading from url now

activeImportFromLocalDriveCount - Count of files that are uploading from local drive now

Backend version

Request:

GET
/api/version

Response:

JSON encoded object

{
  "version":"1.0.2006.18",
}

Description

Version string contains 1 - major, 0 - minor, 20 - year, 06 - month, 18 - day

HTML5 Video Editor Integration Guide

Deployment

Host requirements

Install docker compose

Ubuntu 22.04:

Windows:

MacOS:

Configuration notes

Chromakey requirements

G4ad instance GPU driver installation

External image library sources requirements

Customization

Override media assets information API

Generate media info

Create media info task

Get media info

Custom media assets examples

Import media by url example

NodeJS example

Python example

PHP example

React video editor component example

SolveigEditor component description

index.js

App.js

Custom media assets with prepared media info

Configure .env docker variables:

Register developer profile

Create media info task

Register user's session

Open user's session

NodeJS example

Python example

PHP example

Custom media assets using mediaFiles.json

Output S3 bucket configuration

CallBack events

Common event structure

Check event at callbackUrl

Project created

Project updated

Render project started

Render project completed

HTML5 Video Editor - Backend Server API

Authorization

External authorization server

Create userId token link

Create user

Update existing user

Get user data

Create Developer's Profile

Update existing developer's profile

List dev profiles

Delete dev profile

Get media files list

Get information on media file

Media error description object

Get status on media file

task_info_object

Update media file properties

Delete media file

Working with folders

Create folder

Delete folder

Update folder name

Move media file(s), folders to another folder

Audio library

Get audio library files list

Get information on audio library file

Get categories list for audio library files

Get Moods list for audio library files

Images library

Get images library files list

Get information on image library file

Get categories list for image library files

Thumbnails for clip on timeline

Get single thumbnail

Editing task

Create new task

Editing task operation status