presentation on HEVC/H265 Codec

1. High Efficiency Video Coding
HEVC/H.265
Tejus Adiga M
Department of Electronics and Communication, NMAMIT, Nitte.
Presented By:

2. A New Standard for Video Coding
• Video is going richer, all growing the bit rate
• Frame size– 4K to 8K Ultra HD
• Temporal resolution - 24 FPS to 240 FPS
• Color resolution, bit depth - 8bpc to 12 bpc
• Stereo and 3D Video Technology
• Process intensive Algorithms in consumer electronics is now affordable
• Devices for Recording/Displaying HD to ultra HD video is affordable
• Multi Threaded multi Core CPUs and hardware Accelerators
• Hardware Video Decoders
• Necessary video data rate grows faster than feasible network transport capacities
• Better Video compression standard needed.
• New ITU-T standard H265 addressing needs of an efficient video coder of future
• Same Visual quality – 50% reduction in bitrate than H264/MPEG4 part 10.
• Better visual quality – Same bitrate as of H264/MPEG4 part 10.
28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte.
2

3. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 3
Block Diagram
Fig 1: Block diagram
of HEVC encoder
CABAC: Context Adaptive Binary
Arithmetic Coding.
CTU: Coding Tree Unit. 64x64
Luma block (Y) and two 32x32
Chroma block (U and V) as in YUV
420 format.
SAO: Sample Adaptive offset:
Non linear amplitude mapping to
better reconstruct original pixel
amplitudes.

4. 28 October 2015
Department of Electronics and Communications, NMAMIT,
Nitte.
4
Coding Modes
• Intra Frame Coding (I Frames)
• CTU to encode is predicted from previously encoded and decoded CTUs of upper
rows and left column of the same frame.
• Inter Frame Coding (P and B frame)
• Coded using Motion Estimation and Motion Compensation or Skip mode.
• P Frame: Coded with reference to Previously encoded and decoded frames.
• B Frame: Coded with reference to Previous and Future encoded and decoded
frames.

5. 28 October 2015
Block Partitioning
• Frame is partitioned into Non Overlapping 64x64 Coding
Tree Units (CTU)
• A CTU consists of 64x64 Luma Coding Tree Block (CTB)
and two 32 x 32 Chroma CTB for U and V components.
• Every CTB is further split into one or multiple Coding Unit
(CU) in Quad tree structure.
• Every CU has an associated Prediction Unit (PU) and
Transform Unit (TU).
• Prediction Unit (PU) is partitioning to perform Intra/Inter
prediction. It has root in CU and can be further split down
to 4x4 block including rectangle blocks like 8x4, 16x8 etc.
• Transform Unit (TU) is partitioning to perform Transform
and Entropy coding of Prediction Residual blocks down to
4x4 size.
Department of Electronics and Communications, NMAMIT, Nitte.
CTU
Luma (Y)
CTB
CTB
CTB
CB
CB
CB
CB
CB
CB
CB
CB
CBCB
5

6. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 6
Intra Frame Coding
• Transform Unit size can be from 4x4 upto 32x32
• 33 Directional Angular prediction with Planar and DC prediction
modes are used to predict a TB
• A Prediction Block is formed with previously decoded boundary
samples from spatially neighboring TBs.
• A PB with minimum prediction error is selected using the error
metric like SAD, MAD, SSE, MSE
𝑆𝑢𝑚 𝑂𝑓 𝐴𝑏𝑠𝑜𝑙𝑢𝑡𝑒 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑐𝑒 (𝑆𝐴𝐷) = 𝐼 𝑥, 𝑦 − 𝑃𝐵 𝑥, 𝑦
• Prediction Residual block is found by
𝑅 𝑥, 𝑦 = 𝐼 𝑥, 𝑦 − 𝑃𝐵(𝑥, 𝑦)

7. • Integer DCT is applied to Residual block with Butterfly computation.
• Integer DCT of size 4x4, 8x8, 16x16 and 32x32 computed with fast Separable Butterfly computation.
• Alternatively for only 4x4 residual block DST can be used as it better fit the statistical property that
the residual amplitudes tend to increase as the distance from the boundary samples that are used for
prediction becomes larger.
•
28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte.
7
Transform
4x4 transform
8x8 transform

8. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 8
Transform

9. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 9
Scalar Quantization and Scaling
• Integer DCT introduces Non Linearity.
• DCT Matrix is split into two integer
matrix with appropriate scaling and
combined with Quantization to form
Mf
𝑀𝑓 =
215 𝑆 𝑓
𝑄 𝑠𝑡𝑒𝑝
• Inverse Operation
𝑉𝑖= 𝑄𝑠𝑡𝑒𝑝. 26. 𝑆𝑖
Quantization
and Scaling
Inverse
Quantization
and Scaling

10. 28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 10
Scalar Quantization and Scaling
• Transformed Residual coefficients are quantized with Scalar
Quantization.
• Quantization Parameter (QP):
• Based on constraints like target bitrate, quality a value in between 0 to 51
need to be specified termed as Quantization Parameter.
• For every 6 QP value increase the Qstep doubles.
• Lower the QP, higher the video quality but less compression
• Quantization is done by multiplying Mf with Residual block.
𝑌 = 𝑀𝑓 𝑋
• Inverse Quantization at decoder side can be done by
𝑋 = 𝑉𝑖 𝑌

11. • The selection of the coefficient scanning order depends on the directionalities of the
intra picture prediction.
• Scanning methods
• Diagonal upright scanning: Default
• Vertical Scanning: Used when the prediction direction is close to horizontal
• Horizontal Scanning : Used when the prediction direction is close to vertical
• Scanned Coefficients are Entropy Coded using Context Adaptive Binary Arithmetic
Coding (CABAC) and formatted in bit stream format as specified by H265 NAL Unit
Syntax
28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 11
Entropy Coding

12. • Current frame is encoded using
previously decoded frame as
reference.
• Motion Estimation and Motion
Compensation is carried out at CB
level to get he best match of current
CB in the reference frame in the search
window.
• Offset (Δx, Δy) between current CB
and motion estimated CB referred to
as Motion Vector.
28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 12
Intra Picture Coding

13. • Difference between current CB and motion estimated CB is computed to
get the Prediction Block which is Transformed, quantized and entropy
coded along with Motion Vectors.
• Various Fast motion estimation algorithms are proposed to speedup the
motion estimation process like Three Step, Logarithmic, Hexagonal,
Diamond, Small Diamond etc.
• Optimizations:
• Motion Vector can be predicted spatially and temporally.
• Motion Vector Merge: Average of previously encoded motion vectors of current
frame.
• Skip Mode: MVs are copied from MVs of some other CB.
28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 13
Encoding of motion Estimated Parameters

14. • Due to block processing of CBs there occurs
discontinuities in adjacent blocks.
• To prevent excessive filtering filter are applied to
PBs greater than 4x4 blocks.
• Filter parameters β and tc are derived from QP.
• Types of filtering:
• Strong Filter: High variation in pixel values at boundary.
P0, P1, P2 and Q0, Q1, Q2 are filtered with
(1, 2, 2, 2, 1)/8, (1, 1, 1, 1)/4, and (2, 3, 1, 1, 1)/8
• Weak Filter: Slowly varying pixel values at boundary.
P0 and Q0 are filtered with (8, 19, −1, 9, −3)/32
• No Filter: Skip filtering for intra frame 4x4 block, inter
frame blocks whose MV derived from same reference
frame.
28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 14
In Loop De-Blocking Filter

15. 28 October 2015
Department of Electronics and Communications, NMAMIT,
Nitte.
15
Comparison of HEVC with H.264
Block Partitioning in H264 and HEVCSubjective quality comparison of H264 and HEVC

16. • High degree of parallelism in decoding can take advantage of SIMD
instruction sets like MMX, SSE, SSE2, 3D Now etc in Software Decoding.
• As decoding operation is modular wrt CTUs it is better suited to run on
Graphics processors
• Dedicated VLSI Decoders can take advantage of modularity of HEVC to
accelerate decoding.
• Can provide better video quality for Real time video communication
applications.
• Efficient storage of Video.
28 October 2015
Department of Electronics and Communications, NMAMIT,
Nitte.
16
Advantages

17. • Better suited for parallel processing.
• HEVC will accelerate Video services on Mobile devices.
• Competitors like VP9 from Google and Vorbis from XIPH.org provide
royalty free codecs that is slightly inferior to HEVC.
• HEVC Specification is not yet completely finalized. Opportunity to
improve efficiency further.
28 October 2015
Department of Electronics and Communications, NMAMIT,
Nitte.
17
Conclusion and Future

18. 1. Overview of the High Efficiency Video Coding (HEVC) Standard, G. J. Sullivan, J. R. Ohm, W. J. Han, and T. Wiegand, IEEE Transactions on Circuits and
Systems for Video Technology, Vol 22, no 12, Jan 2013.
2. HEVC Complexity and Implementation Analysis, Frank Bossen, Benjamin Bross, Karsten Suhring, and David Flynn, IEEE Transactions on Circuits
and Systems for Video Technology, vol. 22, no. 12, Jan 2013.
3. Comparison of the Coding Efficiency of Video Coding Standard – Including High Efficient Video Coding (HEVC), J.R. Ohm, G. J. Sullivan, H. Schwarz, T.
K. Tan, T. Wiegand, IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, Jan 2013.
4. HEVC Deblocking Filter, Andrey Norkin, Gisle Bjøntegaard, Arild Fuldseth, Matthias Narroschke, Masaru Ikeda, Kenneth Andersson, Minhua Zhou,
and Geert Van der Auwera, IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no. 12, Jan 2013.
5. Overview of H264/AVC video coding standard, T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra, IEEE Transactions on Circuits and Systems
for Video Technology, vol 13, no 7, July 2003.
6. Overview of the scalable video coding extension of the H.264/AVC standard, H. Schwarz, D. Marpe, and T. Wiegand, IEEE Trans. Circuits Systems for
Video Technology, vol. 17, no. 9, pp. 1103–1120, Sep. 2007.
7. HEVC: The New Gold Standard for Video Compression, Mahsa T. Pourazad, Colin Doutre, Maryam Azimi, and Panos Nasiopoulos, IEEE Consumer
Electronics Magazine, July 2012.
8. White Paper: 4x4 Transformation and Quantization in H264 AVC, Iain Richardson.
9. A High-Throughput VLSI Architecture for Deblocking Filter in HEVC, Weiwei Shen, Qing Shang, Sha Shen, Yibo Fan, Xiaoyang Zeng, IEEE Trans.
Circuits Systems for Video Technology, Vol 22, no 12, Jan 2013.
10. CE10: Core transform design for HEVC, A. Fuldseth, G. Bjøntegaard, M. Budagavi and V. Sze, JCTVC-G495, November 2011
11. ITU-T Recommendation H265, High efficiency video coding.
28 October 2015 Department of Electronics and Communications, NMAMIT, Nitte. 18
References