Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Pekka Astola: JPEG – Coding of digital representations of images | 30 years of success – image and multimedia standardization in 2021

Finnish Standards Association SFS held a webinar on image and multimedia standardization. The webinar 30 Years of Success – Image and Multimedia Standardization in 2021 took place on the 16th of January in 2021.

  • Be the first to comment

  • Be the first to like this

Pekka Astola: JPEG – Coding of digital representations of images | 30 years of success – image and multimedia standardization in 2021

  1. 1. 30 years of success – image and multimedia standardization in 2021 16.2.2021 Pekka Astola, Nokia
  2. 2. © 2021 Nokia 1 SFS Webinar: 30 years of success – image and multimedia standardization in 2021 16 Feb 2021 Pekka Astola Public JPEG – Coding of digital representations of images
  3. 3. © 2021 Nokia 2 Public Agenda • Introduction to SC29/WG1 (JPEG) • JPEG family of standards • Future JPEG standards
  4. 4. © 2021 Nokia 3 Public Presenter Pekka Astola D.Sc. (Tech), Senior Scientist, Nokia Technologies Active in JPEG standardization since 2017. Key contributor and co-editor of JPEG Pleno ISO/IEC 21794-2 Light field coding.
  5. 5. © 2021 Nokia 4 • Standardization of image coding. • Established in 1986. • Under the guidance of ITU-T SG16 and ISO/IEC JTC1/SC29. • Meets four times a year. • Experts from both industry and academia. Public SC29/WG1 Joint Photographic Experts Group (JPEG)
  6. 6. © 2021 Nokia 5 Public JPEG aims for royalty-free baseline https://upload.wikimedia.org/wikipedia/commons/thumb/5/57/Cumulus_Clouds_over_Yellow_Prairie2.jpg/1200px-Cumulus_Clouds_over_Yellow_Prairie2.jpg
  7. 7. © 2021 Nokia 6 Public JPEG family of standards JPEG XT LS 2000 XR JBIG2 AIC JPSearch Systems XS Pleno XL
  8. 8. © 2021 Nokia 7 Public Quality evaluation AIC
  9. 9. © 2021 Nokia 8 Public System level JPSearch Systems
  10. 10. © 2021 Nokia 9 Public Image coding JPEG XT LS 2000 XR JBIG2 XS Pleno XL
  11. 11. © 2021 Nokia 10 Nokia internal use ISO/IEC 10918 Published 1992 ISO/IEC 14495 Published 1999 ISO/IEC 18477 Published 2015 Main use case is lossy compression of continuous-tone images. Technology from late 1980s. Lossless and near-lossless coding of continuous-tone images. Low-complexity. Backward-compatible extensions to ISO/IEC 10918 JPEG. High-dynamic range imaging. The legacy JPEG suite Public JPEG LS XT
  12. 12. © 2021 Nokia 11 72.6% Public JPEG is used by 72.6% of all the websites. w3techs.com JPEG
  13. 13. © 2021 Nokia 12 Public It’s time to replace legacy JPEG with modern solutions
  14. 14. © 2021 Nokia 13 Nokia internal use ISO/IEC 15444 Published 2000 ISO/IEC 14495 Published 2009 ISO/IEC 18477 To be published 2021 Superior coding technology compared to legacy JPEG. Scalability in quality and resolution. Can be used in practically any image coding scenario. Coding performance between JPEG and JPEG 2000. Support for high bit-depths, alpha plane coding, lossless and lossy, low-complexity compressed-domain image manipulations. Significantly improved visual fidelity and support for modern applications. Lossy and lossless. Very wide variety of functionalities. JPEG transcoding. Successors to legacy JPEG Public 2000 XR XL
  15. 15. © 2021 Nokia 14 Public JPEG XL ISO/IEC 18477 Extensive functionality • Detail/texture preservation. • Mixed content (photo, synthetic). • State-of-the-art lossless. • Region decode (based on saliency). • Progressive decoding. • HDR, wide color gamut. • Parallel implementation. • Animations. • 360-degree images. XL Image: http://ds.jpeg.org/whitepapers/jpeg-xl-whitepaper.pdf
  16. 16. © 2021 Nokia 15 Public Technologies of the mainstream JPEG standards JPEG JPEG 2000 JPEG XR JPEG XL Discrete cosine transform (DCT) for 8x8 blocks. Huffman coding (HC) of quantized DCT coefficients, • arithmetic coding defined in standard but rarely used. Wavelet-transform. Embedded coding of wavelet coefficients, • scalability in both quality (PSNR) and spatial resolution. Binary arithmetic coding (BAC). Significantly higher computational cost than JPEG. Lapped biorthogonal transform. Adaptive HC of quantized transform coefficients. Complexity between JPEG and JPEG 2000 with image quality close to JPEG 2000. Combination of variable sized DCT and Haar-like wavelet transform. Asymmetric numeral systems for entropy coding, • offers speeds similar to HC but coding efficiency closer to BAC.
  17. 17. © 2021 Nokia 16 Public H.266 H.265 JPEG XL JPEG JPEG 2000 https://upload.wikimedia.org/wikipedia/commons/thumb/d/d6/NASCAR_practice.jpg/1200px-NASCAR_practice.jpg
  18. 18. © 2021 Nokia 17 Public Evolution of JPEG coding standards 1 Image coding Practical coding solutions for continuous-tone images, • DCT-based transform coding (JPEG) • Predictive coding (JPEG-LS) 2 Advanced image coding Highly flexible image coding solutions for wide variety of applications, • Wavelet transform and embedded coding (JPEG 2000) • Quality scalability, resolution scalability • Professional applications (JPEG XS) 3 New imaging modalities + AI Coding solutions for plenoptic modalities (JPEG Pleno - light field, holography, point cloud). Image coding using learning- based methods (JPEG AI). JPEG LS 2000 XR XL XT Pleno AI XS
  19. 19. © 2021 Nokia 18 Public JPEG standards are (usually) multi-part specifications JPEG Pleno ISO/IEC 21794-1 Specifies the JPEG Pleno framework and the interrelationships between the different components of the standard, i.e. representation of light-field, point-cloud and holographic modalities and system related aspects. ISO/IEC 21794-2 Specifies the coding technology for light field modalities ISO/IEC 21794-3 Defines conformance testing for the standardized technologies covered by the JPEG Pleno framework. ISO/IEC 21794-4 Provides reference implementations for the standardized technologies within the JPEG Pleno framework for purpose of reference for prospective implementers of the standard and compliance testing. Pleno
  20. 20. © 2021 Nokia 19 Public JPEG Pleno ISO/IEC 21794 Coding of emerging imaging modalities • Single file format for storing and transmitting of multiple modalities. • Light fields, holograms and point clouds. • Framework standard published in 2019 as ISO/IEC 21794-1. • Light field coding in most advanced state o International standard to be published in 2021 as ISO/IEC 21794-2/3/4. • Holography and point clouds still to gain more traction. Pleno Image: http://handle.itu.int/11.1002/pub/8153d79a-en
  21. 21. © 2021 Nokia 20 Public JPEG Pleno ISO/IEC 21794-2 Light field coding Two coding engines for light fields • 4D prediction mode for narrow and wide-baseline light fields • Disparity-based hierarchical view prediction. • Coding of reference views and prediction residual using standardized JPEG codecs. • 4D transform mode for narrow-baseline light fields • 4D DCT and coding of coefficients as bit planes using hexadeca trees and arithmetic coding. • To be published in 2021. Pleno
  22. 22. © 2021 Nokia 21 Public Timeline of JPEG Pleno ISO/IEC 21794-2 Light field coding 2021 2020 2019 2017 Call for proposals New project approved Committee draft Draft of international standard International standard Pleno Final draft of international standard
  23. 23. © 2021 Nokia 22 Throughput results for the 292 frame 12 bit/channel 4K 4:4:4 JPEG test video ARRI_AlexaDrums_3840x2160p_24_12b_P3_444. Public High-Throughput JPEG 2000 ISO/IEC 15444-15 Speed-up for JPEG 2000 • HTJ2K speeds-up JPEG 2000 by an order of magnitude at the expense of slightly reduced coding efficiency. • Replaces the Part 1 block coder (EBCOT) with an innovative high-throughput block coder for today's vectorized computing architectures. • Allows mathematically lossless transcoding to/from legacy JPEG 2000. • Published in 2019. Image: http://ds.jpeg.org/whitepapers/jpeg-htj2k-whitepaper.pdf 2000
  24. 24. © 2021 Nokia 23 Public Coding of discontinuous media ISO/IEC 15444-17 Extension for JPEG 2000 • Improved coding of media with hard discontinuities such as depth maps, disparity maps, optical flow or alpha channel. • Breakpoint-adaptive discrete wavelet transform. • Planned use case: depth coding in JPEG Pleno light field. • To be published in 2022. 2000 Pleno
  25. 25. © 2021 Nokia 24 Public JPEG XS ISO/IEC 21122 Visually transparent low latency image codec • Low-complexity and multi-platform. • Designed to replace uncompressed image/video in professional, industrial or commercial applications, • AR headsets, • Advanced driver-assistance systems, • Live broadcasting, cinema cameras, • Consumer cameras, drones, mobile devices, • Machine vision, computer vision. • Published in 2018. Images: Pexels.com and Getty XS
  26. 26. © 2021 Nokia 25 Nokia internal use JPEG AI ISO/IEC 6048 JPEG DNA JPEG Fake Media End-to-end learning-based image coding. Support for machine and computer vision applications through latent representations without decoding. Call for evidence last year with promising results. Re-thinking long term media storage. Currently in exploration stage. Meta-data standard for annotating media asset manipulations. Currently in exploration stage. Future JPEG standards Public AI DNA
  27. 27. © 2021 Nokia 26 Public JPEG AI ISO/IEC 6048 Scope • The scope of the JPEG AI is the creation of a learning-based image coding standard offering a single-stream, compact compressed domain representation, targeting both human visualization, with significant compression efficiency improvement over image coding standards in common use at equivalent subjective quality, and effective performance for image processing and computer vision tasks, with the goal of supporting a royalty- free baseline. AI
  28. 28. © 2021 Nokia 27 Public JPEG AI ISO/IEC 6048 Image: http://ds.jpeg.org/whitepapers/jpeg-ai-white-paper.pdf AI
  29. 29. © 2021 Nokia 28 Public JPEG AI ISO/IEC 6048 Relevant image processing tasks: • Super-resolution • Denoising • Low-light enhancement • Color correction • Exposure compensation • Inpainting AI
  30. 30. © 2021 Nokia 29 Public JPEG AI ISO/IEC 6048 Relevant computer vision tasks: • Image retrieval and classification • Object detection, recognition and identification • Semantic segmentation • Event detection and action recognition • Face detection and recognition AI
  31. 31. © 2021 Nokia 30 Public JPEG website More information about JPEG • www.jpeg.org • Up-to-date information on JPEG meetings, workshops and detailed summaries of the standards. • Mailing-lists. • Many of the ad-hoc groups are public and can be attended even without representing a national body.
  32. 32. Public

×