Early Implementation of Mobile Real-time EVC Player MPEG Submission October 2020
Автор: Ольга Кровякова - 3 ноября 2020 г.
This contribution describes an early implementation of a Real-time Mobile Video Player designed to decode and playback bitstreams conformance with MPEG-5/Part 1: Essential Video Coding (EVC) International Standard under publication. The Player is based on ETM reference software version 6.1 and optimized for ARM architecture. The Player is developed and tested on Huawei P40 Pro smartphone and demonstrates 1080p real-time playback at 24 fps on a subset of EVC Main profile tools.
The current early implementation is targeted to demonstrate feasibility of real-time playback of EVC bitstreams on top market available mobile devices. Huawei P40 Pro Smartphone with Kirin 990 SoC was chosen as a target device. It is operated under EMUI 10.1 based on Android 10 and has in total 8 CPU cores, among which:
- 2 hi-end Cortex-A76 cores clocked at 2.86GHz;
- 2 mid-end Cortex-A76 cores clocked at 2.36GHz;
- 4 low-end Cortex-A55 cores clocked at 1.95GHz.
As an input material “Kimono1” and “ParkScene” test sequences from JVT-VC test set were selected. Both test sequences have 1080p resolution at 24 fps with 4:2:0 color format and 10 seconds duration.
As an encoder configuration default Random Access configuration of EVC Common Test Condition (CTC) was selected. Quantization Parameter (QP) 32 was chosen as a target QP.
2. Supported EVC tools
In order to achieve real-time playback speed on the target device a subset of EVC tools was selected based on profiling information, collected by Android Profiler on the test sequences encoded and decoded with ETM6.1 reference software. Figure 1 and Figure 2 demonstrate obtained profiling data on the target device for these two sequences correspondingly.
Figure 1. ETM6.1 Kimono bitstream decoding profiling
Figure 2. ETM6.1 ParkScene bitstream decoding profiling
Based on the profiling data a subset of EVC tools was selected as summarized in Table 1. Tools names and abbreviations in the table follow the standard connotations as defined in the EVC specification.
Table 1. Selected toolset for the early real-time player implementation
|Tool short name||Tool full name||ETM6.1 CTC RA
|ADMVP||Advanced Motion Vectors Prediction||1||1|
|HTDF||Hadamard Transform Domain Filter||1||1|
|DMVR||Decoder side Motion Vectors Derivation||1||1|
|ADCC||Advanced Coefficients Coding||1||1|
|ALF||Adaptive Loop Filter||1||0|
|AMVR||Adaptive Motion Vectors resolution||1||1|
|ATS||Adaptive Transforms Selection||1||1|
|BTT||Binary and Ternary Trees||1||0|
|CM_INIT||Context Modeling Initialization||1||1|
|EIPD||Enhanced Intra Prediction Directions||1||0|
|HMVP||History Motion vectors Prediction||1||1|
|iQT||Advanced Quantization and Transforms||1||1|
|MMVD||Merge with Motion Vectors Difference||1||1|
|POCS||Advanced Picture Order Count||1||1|
|RPL||Reference Picture List||1||1|
|SUCO||Split Unit Coding Order||1||0|
|IBC||Intra Block Copy||0||0|
As can be seen from Table 1 most of EVC tools were included into the selected toolset.
3. Implementation details and playback
In order to achieve real-time playback on the selected toolset the following main modifications were performed on top ETM 6.1 reference SW:
- ARM SIMD implementation of the most critical functions in MC part
- Waterfront-like parallel processing of CTU lines inside one frame.
- Deblocking filter parallel implementation
For implementation simplicity the only 4 most powerful (hi-end and mid-end) cores of Kirin SoC were selected to be used in the Player.
As a result the Player demonstrates 24+ fps playback on the device. The average decoding speed over 10 second playback is summarized in Table 2.
Table 2. Average results of the selected toolset playback on the device
|Test sequence||Decoding speed (fps)|
In order to check playback speed objectively and subjectively the Player was deployed and tested on Huawei P40 Pro smartphone.
Figure 3 demonstrates a picture of the Player during Kimono bitstream playback on the device.
Figure 3. Picture of the Player working on Huawei P40 Pro
4. Profiling of the optimized decoder
Figure 4. Optimized decoder Kimono bitstream profiling
Figure 5. Optimized decoder ParkScene bitstream profiling
5. CPU Utilization
The current implementation of the Player is optimized for Huawei P40 Pro mobile phone in terms of CPU cores usage, more specificity the SW works only with 4 most powerful Kirin 990 CPU cores (hi-end and mid-end). Figure 6 and Figure 7 give CPU Utilization percentage for each of the 4 powerful cores during 10 second playback.
Figure 6. Hi-end and mid-end CPU cores utilization. Kimono playback.
Figure 7. Hi-end and mid-end CPU cores utilization. ParkScene playback.
6. Power Consumption
In order to estimate power consumption an infinite playback loop of test bitstreams was launched at 100% charged device and the process was working until the device switched off due to energy insufficiency. As a result the Player was working for more than 4.5 hours with constant 24 fps speed. Figure 10 and 11 summarize obtained results.
Figure 10. Power consumption and playback speed during infinite playback of Kimono
Figure 11. Power consumption and playback speed during infinite playback of ParkScene
The early version EVC software Player implemented by Solveig Multimedia demonstrates that real-time playback of EVC bitstreams is feasible and can be used on modern mobile devices with reasonable power consumption.
Improvements of Mobile Real-time EVC Decoder and Player
Ольга Кровякова - менеджер технической поддержки в комании Solveig Multimedia с 2010 года.
Она является автором многих текстовых и видео инструкций по программным продуктам компании: Video Splitter, HyperCam, WMP Trimmer Plugin, AVI Trimmer+ и TriMP4.
Посколько она работает с программами каждый день, то хорошо знает как они работают. Свяжитесь с Ольгой по почте firstname.lastname@example.org, если у вас возникнут вопросы и она с радостью вам поможет!