ᐈ How to validate HEVC streams with Zond 265

By Olga Krovyakova - modified October 9, 2023, created April 2, 2019.

What to check

When developing, tuning, and testing High Efficiency Video Coding (HEVC) codec components, researchers are faced with the task of video stream validation. To playback a file or a stream correctly in any player, it is not enough just to check whether it is correctly decoded by a special reference tool. Additionally, you need to determine whether the tested video stream conforms to HEVC specification [1].

For HEVC, there is a special ISO document that describes how to check video streams and decoders:

Information technology - High efficiency coding and media delivery in heterogeneous environments - Part 5: HEVC conformance testing and reference software [2].

The validation process can be divided into several stages. A negative test result of one of the conditions will result in an error. The condition types can be divided into three groups:

restrictions described in HEVC specification
to decode the stream by the utility of HEVC Test Model (HM) project [3], to write out all warnings
admissibility and changing basic parameters of the video: frame resolution and frame rate

Restrictions described in HEVC specification are as follows:

– maximum and minimum values of Network Abstraction Layer (NAL) unit parameters
– compatibility of each parameter with others
– validity of NAL units order, validity of the structure of the stream
– restrictions of specified level and profile
– availability of decoding parameters of a video sequence - Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and frames needed for reconstruction
– parameters of hypothetical reference decoder (HRD), affecting playback of the stream in real time

How to check by Zond 265

For example, file «tractor_1920x1080p_25fps.yuv» [5] is encoded by HM compressor, and then, in a compressed file, some parameters of NAL units are tuned in a HEX editor to cover part of the potential errors with the help of highlight feature of Zond 265. Then the result is validated. It is impossible to modify the parameters of the file so that all the restrictions are violated. So only several restrictions are selected.

Bitstream

It is better to use Bitstream tab content for a detailed analysis of errors found. Once a file has been opened, all the NAL units with all parameters should be listed in Bitstream tab. When decoding, Zond 265

1) verifies constraints described in the specification;
2) collects all warnings that have arisen in the decoding utility HM.

If something goes wrong, a notification will appear near the corresponding NAL unit. The notification includes the NAL unit name, its offset in the stream, the parameter’s bit offset where an error is found, and a link to the specification page (it will be opened in a new browser window) where you can find the details.

You can expand one after anothera number of lists of errors of Slice B (frame 0), SPS, Slice B (frame 4) by clicking the red icon with the exclamation mark.

Figure 1 shows the list of all the errors found in the SPS.

Figure 1. Window with error list of Bitstream tab

You can then expand the SPS list and explore parameter values and errors (figure 2).

Figure 2. Error message for a single parameter of SPS

The result of SPS parameters checking is described below.

Message	Description
The value of sps_max_sub_layers_minus1 shall be in the range of 0 to 6, inclusive. It is equal to 7	It is restriction of the specification. The type is checking maximal value
When vps_temporal_id_nesting_flag is equal to 1, sps_temporal_id_nesting_flag shall be equal to 1	It is restriction of the specification. The type is checking compatibility of each parameter with others
The value of pic_width_in_luma_samples shall be less than or equal to Sqrt( MaxLumaPs * 8 ), specified for general_level_idc. From the specification, the value of pic_width_in_luma_samples should not be greater than 1402 for level 2.1	It is restriction of the specification. The type is checking restrictions specified for given profile and level

Then you expand a slice of the first frame (figure 3).

Figure 3. Error list for a slice of frame 0

Errors of inaccessibility of decoding parameter are presented in an error list of the first frame.

Message	Description
There is no active VPS available There is no active SPS available	It is restriction of the specification. The type is checking accessibility of decoding parameters

You expand an error list of frame 4 (figure 4).

Figure 5. List of all errors of a file with a context menu

The result of frame 4 validating is described below.

Message	Description
Short-term reference picture with POC = 2 is lost or not correctly decoded	It is restriction of the specification. The type is checking availability of frames needed for reconstruction
decode CTU#46: exception decode CTU#49: exception decode CTU#50: exception	It is message got from the decoder. Reconstruction of the blocks has not been passed. They will be displayed incorrectly on a final picture

Errors tab

It is better to use Errors tab content when one needs to find all errors in a file and to sort them by their types. Error messages are accumulated for both the current frame and the entire stream in lists on the tab (figure 5).

Figure 6. CPB fullness graph drawn, based on HRD data available in SEI messages

Figure 7. CPB fullness graph drawn, based on data set manually

CPB buffer size analysis

Graphs of Coded Picture Buffer (CPB) fullness that are available on Hypothetical Reference Decoder (HRD) are needed to identify problems (e.g., delays) when playing video streams over a network with a small bandwidth. As a source of information, the data recorded in Supplemental enhancement information (SEI) messages (initial delay, bit rate, etc.) can be used, or these data can be set manually. Overflow or underflow areas (highlighted in red) indicate a problem in the interval (figure 6, figure 7). In case of problems when drawing graphs, the corresponding error message will be displayed in Bitstream and Errors tabs.

Figure 8. Error report imported to Google Spreadsheets

Generating reports via command line interface

Generating reports via command line interface is useful when a video stream should be validated automatically. To generate a report you should start Zond 265 with the parameters below.

Parameter	Description
-report	States that Zond 265 should work in report generating mode
t=errors	Sets report type
o=	Sets output CSV file path. It is equal to “out.csv” by default

For example, to analyze file «tractor.265» for errors and save them to file «tractor-errors.csv», you need to run the command:

zond265.exe tractor.265 -report t=errors o=”tractor-errors.csv”

As a result you get a CSV file that can be imported into table processor software. Figure 8 shows file “tractor-errors.csv” imported into Google Spreadsheets.

References

1. HEVC specification – http://www.itu.int/rec/T-REC-H.265
2. «Information technology - High efficiency coding and media delivery in heterogeneous environment - Part 5: HEVC conformance testing and reference software» – http://mpeg.chiariglione.org/standards/mpeg-h/hevc-conformance-testing/working-draft-hevc-conformance
3. HM project home page – https://hevc.hhi.fraunhofer.de/
4. Zond 265 home project – https://www.solveigmm.com/en/products/zond/
5. Test sequence URL – https://media.xiph.org/ldv/pub/test_sequences/1080p/tractor.yuv

TRY ZOND 265 FREE

https://www.solveigmm.com/en/howto/netflix-vmaf-how-to-measure-video-quality-with-vmaf-in-zond-265/
https://www.solveigmm.com/en/howto/zond-265-tutorial/

About the author

Olga Krovyakova is the Technical Support Manager in Solveig Multimedia since 2010.

She is the author of many text and video guidelines of company's products: Video Splitter, HyperCam, WMP Trimmer Plugin, AVI Trimmer+ and TriMP4.

She works with programs every day and therefore knows very well how they work. Сontact Olga via support@solveigmm.com if you have any questions. She will gladly assist you!