How to validate HEVC streams with Zond 265
When developing, tuning, and testing High Efficiency Video Coding (HEVC) codec components, researchers are faced with the task of video stream validation. To playback a file or a stream correctly in any player, it is not enough just to check whether it is correctly decoded by a special reference tool. Additionally, you need to determine whether the tested video stream conforms to HEVC specification .
For HEVC, there is a special ISO document that describes how to check video streams and decoders:
The validation process can be divided into several stages. A negative test result of one of the conditions will result in an error. The condition types can be divided into three groups:
- restrictions described in HEVC specification
- to decode the stream by the utility of HEVC Test Model (HM) project , to write out all warnings
- admissibility and changing basic parameters of the video: frame resolution and frame rate
Restrictions described in HEVC specification are as follows:
– maximum and minimum values of Network Abstraction Layer (NAL) unit parameters
– compatibility of each parameter with others
– validity of NAL units order, validity of the structure of the stream
– restrictions of specified level and profile
– availability of decoding parameters of a video sequence - Video Parameter Set (VPS), Sequence Parameter Set (SPS), Picture Parameter Set (PPS), and frames needed for reconstruction
– parameters of hypothetical reference decoder (HRD), affecting playback of the stream in real time
For example, file «tractor_1920x1080p_25fps.yuv»  is encoded by HM compressor, and then, in a compressed file, some parameters of NAL units are tuned in a HEX editor to cover part of the potential errors with the help of highlight feature of Zond 265. Then the result is validated. It is impossible to modify the parameters of the file so that all the restrictions are violated. So only several restrictions are selected.
It is better to use Bitstream tab content for a detailed analysis of errors found. Once a file has been opened, all the NAL units with all parameters should be listed in Bitstream tab. When decoding, Zond 265
1) verifies constraints described in the specification;
2) collects all warnings that have arisen in the decoding utility HM.
If something goes wrong, a notification will appear near the corresponding NAL unit. The notification includes the NAL unit name, its offset in the stream, the parameter’s bit offset where an error is found, and a link to the specification page (it will be opened in a new browser window) where you can find the details.
You can expand one after anothera number of lists of errors of Slice B (frame 0), SPS, Slice B (frame 4) by clicking the red icon with the exclamation mark.
Figure 1 shows the list of all the errors found in the SPS.
Figure 1. Window with error list of Bitstream tab
You can then expand the SPS list and explore parameter values and errors (figure 2).
Figure 2. Error message for a single parameter of SPS
The result of SPS parameters checking is described below.
|The value of sps_max_sub_layers_minus1 shall be in the range of 0 to 6, inclusive. It is equal to 7||It is restriction of the specification. The type is checking maximal value|
|When vps_temporal_id_nesting_flag is equal to 1, sps_temporal_id_nesting_flag shall be equal to 1||It is restriction of the specification. The type is checking compatibility of each parameter with others|
|The value of pic_width_in_luma_samples shall be less than or equal to Sqrt( MaxLumaPs * 8 ), specified for general_level_idc. From the specification, the value of pic_width_in_luma_samples should not be greater than 1402 for level 2.1||It is restriction of the specification. The type is checking restrictions specified for given profile and level|
Then you expand a slice of the first frame (figure 3).
Figure 3. Error list for a slice of frame 0
Errors of inaccessibility of decoding parameter are presented in an error list of the first frame.
|There is no active VPS available
There is no active SPS available
|It is restriction of the specification. The type is checking accessibility of decoding parameters|
You expand an error list of frame 4 (figure 4).
Figure 5. List of all errors of a file with a context menu
The result of frame 4 validating is described below.
|Short-term reference picture with POC = 2 is lost or not correctly decoded||It is restriction of the specification. The type is checking availability of frames needed for reconstruction|
|decode CTU#46: exception
decode CTU#49: exception
decode CTU#50: exception
|It is message got from the decoder. Reconstruction of the blocks has not been passed. They will be displayed incorrectly on a final picture|
It is better to use Errors tab content when one needs to find all errors in a file and to sort them by their types. Error messages are accumulated for both the current frame and the entire stream in lists on the tab (figure 5).
Figure 6. CPB fullness graph drawn, based on HRD data available in SEI messages
Figure 7. CPB fullness graph drawn, based on data set manually
Graphs of Coded Picture Buffer (CPB) fullness that are available on Hypothetical Reference Decoder (HRD) are needed to identify problems (e.g., delays) when playing video streams over a network with a small bandwidth. As a source of information, the data recorded in Supplemental enhancement information (SEI) messages (initial delay, bit rate, etc.) can be used, or these data can be set manually. Overflow or underflow areas (highlighted in red) indicate a problem in the interval (figure 6, figure 7). In case of problems when drawing graphs, the corresponding error message will be displayed in Bitstream and Errors tabs.
Figure 8. Error report imported to Google Spreadsheets
Generating reports via command line interface is useful when a video stream should be validated automatically. To generate a report you should start Zond 265 with the parameters below.
|-report||States that Zond 265 should work in report generating mode|
|t=errors||Sets report type|
|o=||Sets output CSV file path. It is equal to “out.csv” by default|
For example, to analyze file «tractor.265» for errors and save them to file «tractor-errors.csv» you need to run the command:
zond265.exe tractor.265 -report t=errors o=”tractor-errors.csv”
As a result you get a CSV file that can be imported into table processor software. Figure 8 shows file “tractor-errors.csv” imported into Google Spreadsheets.
1. HEVC specification – http://www.itu.int/rec/T-REC-H.265
2. «Information technology - High efficiency coding and media delivery in heterogeneous environment - Part 5: HEVC conformance testing and reference software» – http://mpeg.chiariglione.org/standards/mpeg-h/hevc-conformance-testing/working-draft-hevc-conformance
3. HM project home page – https://hevc.hhi.fraunhofer.de/
4. Zond 265 home project – http://www.solveigmm.com/en/products/zond/
5. Test sequence URL – https://media.xiph.org/ldv/pub/test_sequences/1080p/tractor.yuv