About Products Support

Early Implementation of Mobile Real-time EVC Player MPEG Submission October 2020

Abstract

This contribution describes an early implementation of a Real-time Mobile Video Player designed to decode and playback bitstreams conformance with MPEG-5/Part 1: Essential Video Coding (EVC) International Standard under publication. The Player is based on ETM reference software version 6.1 and optimized for ARM architecture. The Player is developed and tested on Huawei P40 Pro smartphone and demonstrates 1080p real-time playback at 24 fps on a subset of EVC Main profile tools.

 

1. Introduction

The current early implementation is targeted to demonstrate feasibility of real-time playback of EVC bitstreams on top market available mobile devices. Huawei P40 Pro Smartphone with Kirin 990 SoC was chosen as a target device. It is operated under EMUI 10.1 based on Android 10 and has in total 8 CPU cores, among which:

  • 2 hi-end Cortex-A76 cores clocked at 2.86GHz;
  • 2 mid-end Cortex-A76 cores clocked at 2.36GHz;
  • 4 low-end Cortex-A55 cores clocked at 1.95GHz.

As an input material “Kimono1” and “ParkScene” test sequences from JVT-VC test set were selected. Both test sequences have 1080p resolution at 24 fps with 4:2:0 color format and 10 seconds duration.

As an encoder configuration default Random Access configuration of EVC Common Test Condition (CTC) was selected. Quantization Parameter (QP) 32 was chosen as a target QP.

 

2. Supported EVC tools

In order to achieve real-time playback speed on the target device a subset of EVC tools was selected based on profiling information, collected by Android Profiler on the test sequences encoded and decoded with ETM6.1 reference software. Figure 1 and Figure 2 demonstrate obtained profiling data on the target device for these two sequences correspondingly.

Kimono bitstream decoding profiling  

Figure 1. ETM6.1 Kimono bitstream decoding profiling

ParkScene bitstream decoding profiling

Figure 2. ETM6.1 ParkScene bitstream decoding profiling

 

Based on the profiling data a subset of EVC tools was selected as summarized in Table 1. Tools names and abbreviations in the table follow the standard connotations as defined in the EVC specification.

 

Table 1. Selected toolset for the early real-time player implementation

Tool short name Tool full name ETM6.1 CTC RA
default configuration
Selected toolset
ADMVP Advanced Motion Vectors Prediction 1 1
AFFINE Affine prediction 1 1
HTDF Hadamard Transform Domain Filter 1 1
DMVR Decoder side Motion Vectors Derivation 1 1
ADCC Advanced Coefficients Coding 1 1
ADDB Advanced Deblocking 1 1
ALF Adaptive Loop Filter 1 0
AMVR Adaptive Motion Vectors resolution 1 1
ATS Adaptive Transforms Selection 1 1
BTT Binary and Ternary Trees 1 0
CM_INIT Context Modeling Initialization 1 1
DBF Deblocking Filter 1 1
EIPD Enhanced Intra Prediction Directions 1 0
HMVP History Motion vectors Prediction 1 1
iQT Advanced Quantization and Transforms 1 1
MMVD Merge with Motion Vectors Difference 1 1
POCS Advanced Picture Order Count 1 1
RPL Reference Picture List 1 1
SUCO Split Unit Coding Order 1 0
IBC Intra Block Copy 0 0

 

As can be seen from Table 1 most of EVC tools were included into the selected toolset.  

3. Implementation details and playback

In order to achieve real-time playback on the selected toolset the following main modifications were performed on top ETM 6.1 reference SW:

  1. ARM SIMD implementation of the most critical functions in MC part
  2. Waterfront-like parallel processing of CTU lines inside one frame.
  3. Deblocking filter parallel implementation

For implementation simplicity the only 4 most powerful (hi-end and mid-end) cores of Kirin SoC were selected to be used in the Player.

As a result the Player demonstrates 24+ fps playback on the device. The average decoding speed over 10 second playback is summarized in Table 2.

 

Table 2. Average results of the selected toolset playback on the device 

Test sequence Decoding speed (fps)
ParkScene 24.39
Kimono1 25

In order to check playback speed objectively and subjectively the Player was deployed and tested on Huawei P40 Pro smartphone.

Figure 3 demonstrates a picture of the Player during Kimono bitstream playback on the device.

Picture of the Player working on Huawei P40 Pro

Figure 3. Picture of the Player working on Huawei P40 Pro

 4. Profiling of the optimized decoder

Optimized decoder Kimono bitstream profiling

Figure 4. Optimized decoder Kimono bitstream profiling

Optimized decoder ParkScene bitstream profiling

Figure 5. Optimized decoder ParkScene bitstream profiling

 

5. CPU Utilization

The current implementation of the Player is optimized for Huawei P40 Pro mobile phone in terms of CPU cores usage, more specificity the SW works only with 4 most powerful Kirin 990 CPU cores (hi-end and mid-end). Figure 6 and Figure 7 give CPU Utilization percentage for each of the 4 powerful cores during 10 second playback.

Hi-end and mid-end CPU cores utilization. Kimono playback.

Figure 6. Hi-end and mid-end CPU cores utilization. Kimono playback.

Hi-end and mid-end CPU cores utilization. ParkScene playback.

Figure 7. Hi-end and mid-end CPU cores utilization. ParkScene playback.

 

6. Power Consumption

In order to estimate power consumption an infinite playback loop of test bitstreams was launched at 100% charged device and the process was working until the device switched off due to energy insufficiency. As a result the Player was working for more than 4.5 hours with constant 24 fps speed. Figure 10 and 11 summarize obtained results.

Power consumption and playback speed during infinite playback of Kimono

Figure 10. Power consumption and playback speed during infinite playback of Kimono

Power consumption and playback speed during infinite playback of ParkScene

Figure 11. Power consumption and playback speed during infinite playback of ParkScene

 

7. Conclusion

The early version EVC software Player implemented by Solveig Multimedia demonstrates that real-time playback of EVC bitstreams is feasible and can be used on modern mobile devices with reasonable power consumption.