Video Editing SDK 5 for Linux - main features - Blog

Linux-SDK-release

Video Editing SDK 5 for Linux – main features

24.05.2022 15 Minutes Olga Krovyakova 160 0 Comments

Video Editing SDK for Linux is a tool for those developers who work with video and audio and want to design smart video editing desktop or cloud applications that support key multimedia formats and codecs.
The SDK implements a SolveigMM Smart Rendering Technology that allows any processing of media files with different compression parameters according to the specified reference file’s parameters.
The heart of the SDK is a Video Editing Engine object that supports a variety of editing operations in smart mode. Those operations can be performed by the smm_batchsplit utility. It is a command-line utility for processing batches of media files. All editing configurations must be described within a batch file (*.xtl).

Check the available documentation on the Video Editing SDK for Linux. There is also a description of the HTML5 Video Editor API available.

And below is the list of operations that can be performed with help of the SDK and smm_batchsplit.

Trimming

This feature allows trimming, cutting out an arbitrary number of fragments from input pre-encoded files in fast and lossless modes with frame accuracy combined with the lossless media files merging.

For the batch processing you can use the following XTL code:


<timelines version="3">
  <timeline>
    <group output="../output/sample_mp4_0001.mp4">
      <track video="1" audio="1" accuracy="frame">
        <clip src="../mp4/test_file.mp4" start="00:00:01:00" stop="00:00:06:00" timeFormat="time10ms" />
        <clip src="../mp4/test_file.mp4" start="00:00:07:00" stop="00:00:12:00" timeFormat="time10ms" />
      </track>
    </group>
  </timeline>
</timelines>

As a result, the file “sample_mp4_0001.mp4” will be created out of the source file ‘test_file.mp4’ consisting of 2 intervals of the source file – seconds 1-6 and 7-12.

The output file for this project is shown here:

 

Insert effects/transitions to the output file

The Engine is capable of applying multiple effects and transitions to MP4 AVC and  MXF XAVC videos:

– fade-in / fade-out,
– dissolve

Dissolve

The sample shows the dissolve transition between 2 videos. The duration of the transition is 3 seconds.

Following XTL project can be used to demonstrate the transition:


<timelines version="3">
  <timeline>
    <group output="../output/transition_dissolve.mp4">
      <track video="1" audio="1" text="0" accuracy="frame">
        <clip src="../mp4/test_file.mp4" start="00:00:00:00" stop="00:00:15:00" />
        <clip src="../mp4/test_file.mp4" start="00:00:03:00" stop="00:00:15:00" />
        <effect type="transparency" start="00:00:12:00" stop="00:00:15:00">
          <param name="bkcolor" value="black" />
          <param name="alpha" value="255">
            <linear length="00:00:03:00" value="0"/>
          </param>
        </effect>
      </track>
      <track video="1">
        <clip src="../mp4/test_file.mp4" length="00:00:12:00" flags="blank" />
        <clip src="../mp4/test_file.mp4" start="00:00:00:00" stop="00:00:03:00"/>
        <clip src="../mp4/test_file.mp4" length="00:00:12:00" flags="blank" />
        <effect type="transparency" start="00:00:12:00" stop="00:00:15:00">
          <param name="bkcolor" value="transparent" />
          <param name="alpha" value="0">
            <linear length="00:00:03:00" value="255"/>
          </param>
        </effect>
        <transition type="mix" start="00:00:12:00" stop="00:00:15:00"/>
      </track>
    </group>
  </timeline>
</timelines>

The first track of the XTL is built to represent the entire output file structure and duration. It contains all files that will compose the output file.

It contains the “effect” tag, to describe the timings for each of the dissolves, which number should be the same as the number of dissolves. Each “effect” tag should describe the “fade out” part on the first out of two files affected by the current dissolve.

Additional track is required for each second file from the affected couple. Such tracks must specify the following info:

  1. Type and number of the stream to which the transition will be applied. Please make note that only one type of stream should be specified.
  2. “effect” tag, that describes the “fade in” on the second file
  3. “transition” tag, showing that the current track will be processed according to that tag and added to the output file.

As a result for this xtl the file transition_dissolve.mp4 should be created consisting of 2 equal videos and 1 dissolve transition between them starting from the 12th second for a duration of 3 seconds.

The output file for this project is shown here:

 

Fade-in and Fade-out

The sample shows the “fade in” and “fade out” effects on 2 combined videos. Each effect lasts for 3 seconds.

The description of the XTL for this kind of transitions looks as follows:


<timelines version="3">
   <timeline>
     <group output="../output/transition_fadeout_fadein.mp4">
       <track video="1" audio="1" text="0" accuracy="frame">
         <clip src="../mp4/test_file.mp4" start="00:00:00:00" stop="00:00:15:00" />
         <clip src="../mp4/test_file.mp4" start="00:00:00:00" stop="00:00:15:00" />
         <effect type="transparency" start="00:00:12:00" stop="00:00:15:00">
           <param name="bkcolor" value="black" />
           <param name="alpha" value="255">
             <linear length="00:00:03:00" value="0"/>
           </param>
         </effect>
         <effect type="transparency" start="00:00:15:00" stop="00:00:18:00">
           <param name="bkcolor" value="black" />
           <param name="alpha" value="0">
             <linear length="00:00:03:00" value="255"/>
           </param>
         </effect>
       </track>
     </group>
   </timeline>
</timelines>

The sample contains 2 files, described in the “clip” tags and 2 “effect” tags – the first one is for the “fade out” and the second one is for the “fade in”. Those tags also contain additional parameters in “param” tags – a background color and a transparency value and variation function.

As a result for this XTL project the file transition_fadeout_fadein.mp4 should be created which consists of 2 equal videos – the first video will have a “fade out” effect in its end and the second file will have a “fade in” effect in its beginning.

You can check the output file here:

 

Mute audio segments

The feature allows to mute audio on the specified parts of the media file.

The XTL project performing the “Mute audio” is shown below:


<timelines version="3">
  <timeline>
    <group output="../output/audio_mute.mp4">
      <track video="1" audio="1" text="0" accuracy="frame">
        <clip src="../mp4/test_file.mp4" start="00:00:00:00" stop="00:00:05:00" timeFormat="time10ms" flags="audio_silence"/>
        <clip src="../mp4/test_file.mp4" start="00:00:05:00" stop="00:00:10:00" timeFormat="time10ms" />
        <clip src="../mp4/test_file.mp4" start="00:00:10:00" stop="00:00:15:00" timeFormat="time10ms" flags="audio_silence"/>
      </track>
    </group>
  </timeline>
</timelines>

This project will create the “audio_mute.mp4” file. The first and the last 5 seconds of the output file are silent. See the result here:

 

Mix audio

The mixing feature allows you to overlay the sound from the separate M4A or MP3 audio files as well as the audio AAC, MP3 or PCM streams from the other MP4 files to your file. This can also be called a Voice-over.

Here is the XTL project example:


<timelines version="3">
  <timeline>
    <group output="../output/audio_mix.mp4">
      <track video="1" audio="0" text="0" accuracy="frame">
        <clip src="../mp4/test_file.mp4" start="00:00:00:00" stop="00:00:15:00" timeFormat="time10ms" />
      </track>
      <track video="0" audio="1" text="0" accuracy="frame">
        <clip src="../mp4/test_file.mp4" start="00:00:00:00" stop="00:00:15:00" timeFormat="time10ms" />
      </track>
      <track audio="1">
        <clip src="../mp3/mp3_single.mp3" length="00:00:02:00" flags="audio_silence" />
        <clip src="../mp3/mp3_single.mp3" start="00:00:00:00" stop="00:00:10:00" timeFormat="time10ms" />
        <clip src="../mp3/mp3_single.mp3" length="00:00:03:00" flags="audio_silence" />
        <transition type="mix" start="00:00:02:00" stop="00:00:12:00">
          <param name="audio" value="1"/>
        </transition>
      </track>
    </group>
  </timeline>
</timelines>

Remarks:

This project produces the file “audio_mix.mp4”. The file will contain a source file test_file.mp4 where a 10 seconds interval of the sound from the mp3_single.mp3 file will be mixed in.

 

Working with images

The SDK allows you to use images for your video. The following operations can be performed:

Overlaying pictures on a video

You can overlay a picture to your video file or add a watermark or logo. It is available for MP4 files with AVC video, MXF with XAVC.

The sample shows how to overlay images on video.


<timelines version="3">
  <timeline>
    <group output="../output/image_overlay.mp4">
      <track video="1" audio="1" text="0" accuracy="frame">
        <clip src="../mp4/test_file.mp4" start="00:00:00:00" stop="00:00:15:00" timeFormat="time10ms" />
      </track>
      <track video="1">
        <clip src="../logo.png" length="00:00:02:00" flags="blank" />
        <clip src="../logo.png" length="00:00:05:00" />
        <clip src="../logo.png" length="00:00:08:00" flags="blank" />
        <transition type="pip" start="00:00:02:00" stop="00:00:07:00">
          <param name="x" value="10"/>
          <param name="y" value="10"/>
          <param name="width" value="100"/>
          <param name="height" value="28"/>
        </transition>
      </track>
    </group>
  </timeline>
</timelines>

This XTL project makes an image_overlay.mp4 file and overlays the SolveigMM logo to a specified position starting at the 2 second and shows it for a duration of 10 seconds.

See the output file here:

 

Converting pictures into a video

You can compose a new MP4 (AVC) or MXF (XAVC) video from a set of the PNG/JPG  pictures.

The compression parameters must be set either by specifying the reference file or describing them directly:


<timelines version="3">
  <timeline>
    <group output="../output/image_to_video.mp4" out_type="mp4">
      <task type="transcoding">
        <param name="vcodec" value="type=h264,fps_n=25,fps_d=1,width=1280,height=720,profile=baseline,bitrate=1000"/>
      </task>
      <track video="1" audio="0" text="0">
        <clip src="../jpg/kitten.jpg" length="00:00:04:00" />
      </track>
    </group>
  </timeline>
</timelines>

The output video made with this XTL project consists of the kitten.jpg picture presenting for a duration of 4 seconds and is encoded with H264 codec.

Here is the output file for this project:

 

Common transitions

The sample demonstrates applying all possible effects in a single file.


<timelines version="3">
  <timeline>
    <group output="../output/transitions_avc_mp4.mp4">
<!--specify compression parameters for the transcoding-->
      <param name="reference_file" value="../mp4/test_video_1.mp4"/>
<!--add common output file structure, work only with the video stream-->
      <track video="1" audio="0" accuracy="frame">
        <clip src="../mp4/test_video_1.mp4" start="00:00:00:00" stop="00:00:02:00" />
        <clip src="../mp4/test_video_1.mp4" start="00:00:01:00" stop="00:00:02:00" />
        <--declare a fade-in on the beginning of the first file-->
        <effect type="transparency" start="00:00:00:00" stop="00:00:01:00" >
          <param name="bkcolor" value="black" />
          <param name="alpha" value="0">
            <linear length="00:00:01:00" value="255" />
          </param>
        </effect>
<!--prepare the fade-out part of the dissolve transition between 2 files, the rest part is in the separate track below-->
        <effect type="transparency" start="00:00:01:00" stop="00:00:02:00" >
          <param name="bkcolor" value="black" />
          <param name="alpha" value="255">
            <linear length="00:00:01:00" value="0" />
          </param>
        </effect>
<!--declare a fade-out on the ending of the second video file-->
        <effect type="transparency" start="00:00:02:00" stop="00:00:03:00" >
          <param name="bkcolor" value="black" />
          <param name="alpha" value="255">
            <linear length="00:00:01:00" value="0" />
          </param>
        </effect>
      </track>
<!--finalize the dissolve effect by specifying a fade-in part on the second file and declaring a mix transition-->
      <track video="1">
        <clip src="../mp4/test_video_1.mp4" length="00:00:01:00" flags="blank" />
        <clip src="../mp4/test_video_1.mp4" start="00:00:00:00" stop="00:00:01:00" />
        <clip src="../mp4/test_video_1.mp4" length="00:00:01:00" flags="blank" />
        <effect type="transparency" start="00:00:01:00" stop="00:00:02:00">
          <param name="bkcolor" value="transparent" />
          <param name="alpha" value="0">
            <linear length="00:00:01:00" value="255" />
          </param>
        </effect>
        <transition type="mix" start="00:00:01:00" stop="00:00:02:00"/>
      </track>
<!--declare a picture overlay-->
      <track video="1">
        <clip src="../png/attractive.png" length="00:00:01:00" flags="blank"/>
        <clip src="../png/attractive.png" length="00:00:01:00" />
        <clip src="../png/attractive.png" length="00:00:01:00" flags="blank"/>
        <transition type="pip" start="00:00:01:00" stop="00:00:02:00">
          <param name="x" value="0"/>
          <param name="y" value="0"/>
          <param name="width" value="184"/>
          <param name="height" value="112"/>
        </transition>
      </track>
<!--add audio from the same input files-->
      <track video="0" audio="1" accuracy="frame">
        <clip src="../mp4/test_video_1.mp4" start="00:00:00:00" stop="00:00:02:00" />
        <clip src="../mp4/test_video_1.mp4" start="00:00:01:00" stop="00:00:02:00" />
      </track>
<!--declare mixing of the additional audio from the separate mp3 file-->
      <track audio="1">
        <clip src="../mp3/mp3_single.mp3" length="00:00:00:50" flags="audio_silence"/>
        <clip src="../mp3/mp3_single.mp3" start="00:00:00:00" stop="00:00:02:00" />
        <clip src="../mp3/mp3_single.mp3" length="00:00:00:50" flags="audio_silence"/>
        <transition type="mix" start="00:00:00:50" stop="00:00:02:50"/>
      </track>
    </group>
  </timeline>
</timelines>

As an output for this XTL the file transitions_avc_mp4.mp4 will be created where there are different transitions demonstrated:

  • Fade-in on the first file for a duration of 1 second starting at 0 sec.
  • Dissolve between the first and the second files for a duration of 1 second starting at 1 sec.
  • Fade-out on the second files end for a duration of 1 second, starting at 2 sec.
  • Overlaying image for the duration of 1 second, starting at 1 sec.
  • Adding audio tracks separately from the same source files
  • Mixing audio from the external mp3 file for a duration of 2 seconds, starting at 0.5 sec.

A transcoding will take place to create an output file with the parameters, specified in the test_video_1.mp4 reference file.

You can check the output file for this project here:

 

HTML5 Online Video Editor as the sample

A description above shows that Video Editing SDK 5 for Linux incorporates a variety of useful functions. And those are not just potential features – we already use that functionality. The SDK includes the HTML5 Video Editor both as online and on premise solutions. HTML5 Video Editor uses the SDK engine at backend and supports all features provided by the SDK.

HTML5 Video Editor web demo:
https://smarteditingonline.solveigmm.com/

Please see this video to learn how to edit videos online with HTML5 Video Editor:

 

Utilities

Sometimes it might be less convenient to create the XTL project to perform a specific task. For such cases we added a couple of useful utilities, that perform features and don’t require the XTL:

smm_ts2mp4

It is a command-line utility that transmuxes the segment specified with start/end times from MPEG-TS file with AVC HEVC video and AAC audio to the MP4 file.
An example how to use the utility:

./smm_ts2mp4.sh -frameacc -source /samples/media/h264-ts/sample_h264_aac.ts -dest /tmp/output.mp4 -starttime 10000000 -endtime 20000000

-source – sets the name of the source MPEG-TS file
-dest – sets the name of the output MP4 file
-starttime and -endtime specify the start/stop time of the interval in 100 nanosecond units
-frameacc – enables the frame accuracy for trimming

smm_mp4transc

It is a command-line tool that transcodes an MP4 file with AVC or HEVC video and AAC audio with the exact same codec and container parameters specified by the reference MP4 (AVC video, AAC audio) file.
An example of usage:

./smm_mp4transc.sh -sourcemp4 ../samples/media/mp4/test_video_1.mp4 -refmp4 ../samples/media/mp4/sample_mp4_hd.mp4 -destmp4 /tmp/output.mp4

-sourcemp4 – sets the name of the source MP4 file
-refmp4 – specifies the reference file, whose parameters will be applied for encoding
-destmp4 – sets the name of the output MP4 file

The following picture shows the original video’s parameters
before transcoding

Here is a reference file parameters:
transcoding reference

 

And the picture below represents the transcoded video’s info

after transcoding

We can see that the transcoded video have different resolution, profile and level parameters etc.

5 / 5 - 1

leave a comment