API Build-data JSON Resources
Theme

Choose how MSRBot.io looks on this device.

Auto (match system) Light mode Dark mode

Preference is stored in this browser only.

SMPTE Meetings and Conferences ( October 2022)
[ACTIVE]

Video Compression Using Convolutional Neural Networks of Video with Chroma Subsampling

Metadata

Publisher
SMPTE — White Plains, NY, USA
Doc Type
Conference Paper
Content Type
Original Research
Volume
2022, No. 10, pp. 1–16
Abstract
In the context of Convolutional Neural Networks based video compression, motivated by the lower acuity of the human visual system for color differences as compared with luma, we investigate a video compression framework using autoencoder networks that encode and decode videos by using less chroma information than luma information. For this purpose, instead of converting Y'CbCr 4:2:2/4:2:0 videos to and from RGB 4:4:4 as per the current state-of-the-art, we have kept the video in Y'CbCr 4:2:2/4:2:0 and merged the luma and chroma channels after the luma is downsampled to match the chroma size. We have performed an inverse function for the decoder. The performance of our models against the 4:4:4 baseline is evaluated by using CPSNR, MS-SSIM, and VMAF metrics. Our experiments reveal that, as compared to video compression involving conversion to and from RGB 4:4:4, the proposed method increases the video quality by about 5% for Y'CbCr 4:2:2 and 6% for Y'CbCr 4:2:0 while reducing the amount of computation by nearly 37% for Y'CbCr 4:2:2 and 40% for Y'CbCr 4:2:0. These results point us to optimization for 4:2:2 and 4:2:0 video of the current state-of-the-art autoencoder.
Publication Date
2022-10-24
DOI
10.5594/M001957
ISBN
Print: 978-1-61482-963-8
Link
https://doi.org/10.5594/M001957
Author(s)
Vahid Khorasani GhassabPhD, Avid Technology and Concordia University
Rob GonsalvesAvid Technology
Shailendra MathurAvid Technology
Nizar BouguilaPhD, Concordia University
Keyword(s)
Video Compression, Neural Network, Machine Learning, CNN, Chroma Subsampling
Copyright
© 2022 Society of Motion Picture and Television Engineers, Inc.

Bibliographic Reference(s)

  • Ren Yang, Fabian Mentzer, Luc Van Gool, and Radu Timofte. Learning for video compression with hierarchical quality and recurrent enhancement. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6628–6637, 2020. EXTERNAL
  • Shailendra Mathur and Al Kovalick. Vc-3 codec updates for handling better, faster, and more pixels. SMPTE Motion Imaging Journal, 125(5):1–11, 2016. EXTERNAL
  • Byung Cheol Song, Yun Gu Lee, and Nak Hoon Kim. Block adaptive inter-color compensation algorithm for rgb 4: 4: 4 video coding. IEEE transactions on circuits and systems for video technology, 18(10):1447–1451, 2008. EXTERNAL
  • Adam Wieckowski, Gabriel Hege, Christian Bartnik, Christian Lehmann, Christian Stoffers, Benjamin Bross, and Detlev Marpe. Towards a live software decoder implementation for the upcoming versatile video coding (vvc) codec. In 2020 IEEE International Conference on Image Processing (ICIP), pages 3124–3128. IEEE, 2020. EXTERNAL
  • Thomas Wiegand, Gary J Sullivan, Gisle Bjontegaard, and Ajay Luthra. Overview of the h.264/avc video coding standard. IEEE Transactions on circuits and systems for video technology, 13(7):560–576, 2003. EXTERNAL
  • Netflix public database. https://github.com/Netflix/vmaf/blob/master/resource/dataset. EXTERNAL
  • Charles Poynton. Digital video and HD: Algorithms and Interfaces. Elsevier, 2012. EXTERNAL
  • Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009. EXTERNAL
  • Yue Chen, Debargha Mukherjee, Jingning Han, Adrian Grange, Yaowu Xu, Sarah Parker, Cheng Chen, Hui Su, Urvang Joshi, Ching-Han Chiang, et al. An overview of coding tools in av1: the first video codec from the alliance for open media. APSIPA Transactions on Signal and Information Processing, 9, 2020. EXTERNAL
  • Tong Chen, Haojie Liu, Qiu Shen, Tao Yue, Xun Cao, and Zhan Ma. Deepcoder: A deep neural network based video compression. In 2017 IEEE Visual Communications and Image Processing (VCIP), pages 1–4. IEEE, 2017. EXTERNAL
  • Debargha Mukherjee, Shunyao Li, Yue Chen, Aamir Anis, Sarah Parker, and James Bankoski. A switchable loop-restoration with side-information framework for the emerging av1 video codec. In 2017 IEEE International Conference on Image Processing (ICIP), pages 265- EXTERNAL
  • Fabian Mentzer, George Toderici, Michael Tschannen, and Eirikur Agustsson. Highfidelity generative image compression. arXiv preprint arXiv:2006.09965, 2020. EXTERNAL
  • Sang Heon Lee and Nam Ik Cho. Intra prediction method based on the linear relationship between the channels for yuv 4: 2: 0 intra coding. In 2009 16th IEEE International Conference on Image Processing (ICIP), pages 1037–1040. IEEE, 2009 EXTERNAL
  • Yi-Hsuan Tsai, Ming-Yu Liu, Deqing Sun, Ming-Hsuan Yang, and Jan Kautz. Learning binary residual representations for domain-specific video streaming. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 32, 2018. EXTERNAL
  • Oren Rippel, Sanjay Nair, Carissa Lew, Steve Branson, Alexander G Anderson, and Lubomir Bourdev. Learned video compression. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 3454–3463, 2019. EXTERNAL
  • Zhenyu Liu, Xianyu Yu, Yuan Gao, Shaolin Chen, Xiangyang Ji, and Dongsheng Wang. Cu partition mode decision for hevc hardwired intra encoder using convolution neural network. IEEE Transactions on Image Processing, 25(11):5088–5103, 2016. EXTERNAL
  • Shiqi Wang, Ke Gu, Siwei Ma, and Wen Gao. Joint chroma downsampling and upsampling for screen content image. IEEE Transactions on Circuits and Systems for Video Technology, 26(9):1595–1609, 2015. EXTERNAL
  • Fabian Mentzer, George Toderici, David Minnen, Sung Jin Hwang, Sergi Caelles, Mario Lucic, and Eirikur Agustsson. VCT: A Video Compression Transformer. arXiv preprint arXiv:2206.07307, 2022. EXTERNAL
  • Rui Song, Dong Liu, Houqiang Li, and Feng Wu. Neural network-based arithmetic coding of intra prediction modes in hevc. In 2017 IEEE Visual Communications and Image Processing (VCIP), pages 1–4. IEEE, 2017. EXTERNAL
  • Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Zhiyong Gao, and Ming-Ting Sun. Deep kalman filtering network for video compression artifact reduction. In Proceedings of the European Conference on Computer Vision (ECCV), pages 568–584, 2018. EXTERNAL
  • Zhibo Chen, Tianyu He, Xin Jin, and Feng Wu. Learning for video compression. IEEE Transactions on Circuits and Systems for Video Technology, 30(2):566–576, 2019. EXTERNAL
  • Sergey Ioffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning, pages 448–456. PMLR, 2015. EXTERNAL
  • Video multi-method assessment fusion (vmaf). https://github.com/Netflix/vmaf. EXTERNAL
  • Tao Lin, Peijun Zhang, Shuhui Wang, Kailun Zhou, and Xianyi Chen. Mixed chroma sampling-rate high efficiency video coding for full-chroma screen content. IEEE Transactions on Circuits and Systems for Video Technology, 23(1):173–185, 2012. EXTERNAL
  • Gary J Sullivan, Jens-Rainer Ohm, Woo-Jin Han, and Thomas Wiegand. Overview of the high efficiency video coding (hevc) standard. IEEE Transactions on circuits and systems for video technology, 22(12):1649–1668, 2012. EXTERNAL
  • BT Series. Parameter values for ultra-high definition television systems for production and international programme exchange. 2012. EXTERNAL
  • Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. Conditional probability models for deep image compression. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4394–4402, 2018. EXTERNAL
  • Amirhossein Habibian, Ties van Rozendaal, Jakub M Tomczak, and Taco S Cohen. Video compression with rate-distortion autoencoders. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7033–7042, 2019. EXTERNAL
  • Chao-Yuan Wu, Nayan Singhal, and Philipp Krahenbuhl. Video compression through image interpolation. In Proceedings of the European Conference on Computer Vision (ECCV), pages 416–431, 2018. EXTERNAL
  • Shuyuan Zhu, Chang Cui, Ruiqin Xiong, Yuanfang Guo, and Bing Zeng. Efficient chroma sub-sampling and luma modification for color image compression. IEEE Transactions on Circuits and Systems for Video Technology, 29(5):1559–1563, 2019. EXTERNAL
  • ST 2019-1:2016 - SMPTE Standard - VC-3 Picture Compression and Data Stream Format EXTERNAL
  • Guo Lu, Wanli Ouyang, Dong Xu, Xiaoyun Zhang, Chunlei Cai, and Zhiyong Gao. Dvc: An end-to-end deep video compression framework. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11006–11015, 2019. EXTERNAL
  • Douglas A Kerr. Chrominance subsampling in digital images. The Pumpkin, 2012(3): 1–15, 2012. EXTERNAL
  • Zhou Wang, Eero P Simoncelli, and Alan C Bovik. Multiscale structural similarity for image quality assessment. In The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003, volume 2, pages 1398–1402. Ieee, 2003. EXTERNAL
  • Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. EXTERNAL
  • Kuo-Liang Chung, Chi-Chao Huang, and Tsu-Chun Hsu. Adaptive chroma subsamplingbinding and luma-guided chroma reconstruction method for screen content images. IEEE Transactions on Image Processing, 26(12):6034–6045, 2017. EXTERNAL
  • Walter Gish and Scott Miller. Unambiguous video pipeline description motivated by hdr. In 2016 IEEE International Conference on Image Processing (ICIP), pages 909–912. IEEE, 2016. EXTERNAL
Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001957.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree
Reference this Doc

Plain text (ISO 690 compliant)

Preview:
Vahid Khorasani Ghassab, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; Video Compression Using Convolutional Neural Networks of Video with Chroma Subsampling, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001957
Snippet:
Vahid Khorasani Ghassab, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; Video Compression Using Convolutional Neural Networks of Video with Chroma Subsampling, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001957

HTML (ISO 690 compliant)

Preview:
Vahid Khorasani Ghassab, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; Video Compression Using Convolutional Neural Networks of Video with Chroma Subsampling, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001957
Snippet:
<span class="citation">Vahid Khorasani Ghassab, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; <cite>Video Compression Using Convolutional Neural Networks of Video with Chroma Subsampling</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at <a href="https://doi.org/10.5594/M001957" target="_blank" rel="noopener">https://doi.org/10.5594/M001957</a></span>

SMPTE Icon SMPTE's HTML Pub

Preview:
Vahid Khorasani Ghassab, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; Video Compression Using Convolutional Neural Networks of Video with Chroma Subsampling, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8
doi: 10.5594/M001957
url: https://doi.org/10.5594/M001957
Snippet:
<li>
Vahid Khorasani Ghassab, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; <cite id="bib-10-5594-m001957">Video Compression Using Convolutional Neural Networks of Video with Chroma Subsampling</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8
<span class="doi">10.5594/M001957</span>
</li>

Referenced By