Large Multimodal Model-Based Video Encoding Optimization

Bibliographic Reference(s)

• T. Stockhammer, “Dynamic Adaptive Streaming over HTTP: Standards and Design Principles,” Proc. ACM Multimedia Syst. Conf., pp. 133–144, San Jose, CA, Feb. 2011. EXTERNAL
• A. Aaron, Z. Li, M. Manohara, D. J. Cock, and D. Ronca, “Per-Title Encode Optimization,” 2015. [Online]. Available: https://medium.com/netflix-techblog/per-title-encode-optimization-7e99442b62a2. EXTERNAL
• Apple, “HTTP Live Streaming (HLS) authoring specification for Apple devices,” 2016. [Online]. Available: http://is.gd/LBOdpz. EXTERNAL
• G. Michael, T. Christian, H. Hermann, C. Wael, N. Daniel, and B. Stefano, “Combined bitrate suggestions for multirate streaming of industry solutions,” 2013. [Online]. Available: http://alicante.itec.aau.at/am1.html. EXTERNAL
• J. Dahl,”Instant Per-title Encoding, 2018. [Online]. Available: https://www.mux.com/blog/instant-per-title-encoding. EXTERNAL
• V. V. Menon, C. Feldmann, H. Amirpour, M. Ghanbari, and C. Timmerer, “VCA: Video complexity analyzer,” Proc. ACM Multimedia Syst. Conf., pp. 259–264, New York, NY, Aug. 2022. EXTERNAL
• H. Amirpour, P. T. Rajendran, V. V. Menon, M. Ghanbari, and C. Timmerer, “Light-weight Video Encoding Complexity Prediction using Spatio-temporal Features,” Proc. IEEE Int. Workshop Multimedia Signal Process., pp. 1–6, Shanghai, China, Sep. 2022. EXTERNAL
• A. V. Katsenou, M. Afonso, D. Agrafiotis, and D. R. Bull, Predicting video rate-distortion curves using textural features, Proc. Picture Coding Symp, pp. 1–5, Nuremberg, Germany, Dec. 2016. EXTERNAL
• A. Telili, W. Hamidouche, S. A. Fezza, and L. Morin, “Benchmarking Learning-based Bitrate Ladder Prediction Methods for Adaptive Video Streaming,” Proc. Picture Coding Symp., pp. 325–329, San Jose, CA, Dec. 2022. EXTERNAL
• K. S. Durbha, H. Tmar, C. Stejerean, I. Katsavounidis, and A. C. Bovik. Bitrate ladder construction using visual information fidelity. arXiv preprint arXiv:2312.07780. Dec. 2023. EXTERNAL
• A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and L. Sutskever, “Learning Transferable Visual Models from Natural Language Supervision,” Proc. Int. Conf. Mach. Learn., pp. 8748–8763. Jun. 2021. EXTERNAL
• C. Sun, A. Myers, C. Vondrick, K. Murphy, and C. Schmid, Video BERT: A Joint Model for Video and Language Representation Learning, Proc. Int. Conf. Comput. Vis., pp. 7464–7473, Long Beach, CA, Jun. 2019. EXTERNAL
• J. Lu, D. Batra, D. Parikh, and S. Lee, “ViLBERT: Pretraining Task-agnostic Visio-linguistic Representations for Vision-and-language Tasks,” Proc. Adv. Neural Inf. Process. Syst., pp. 13–23, Vancouver, BC, Canada, Dec. 2019. EXTERNAL
• C. Chen, S. Inguva, A. Rankin, and A. Kokaram, “A subjective study for the design of multi-resolution ABR video streams with the VP9 codec,” Proc. Electron. Imag., pp. 1–5, Nov. 2016. EXTERNAL
• C. Kreuzberger, B. Rainer, H. Hellwagner, L. Toni, and P. Frossard, “A comparative study of DASH representation sets using real user characteristics,” Proc. Int. Work-shop Netw. Oper. Syst. Support Digit. Audio Video, pp. 1–6, May 2016. EXTERNAL
• Z. Duanmu, W. Liu, Z. Li, and Z. Wang, “Modelling generalized rate-distortion functions,” IEEE Trans. Image Process., 23(29): 7331–7344, Jun. 2020. EXTERNAL
• Z. Duanmu, W. Liu, Z. Li, K. Ma, and Z. Wang, “Characterizing generalized rate-distortion performance of video coding: An eigen analysis approach,” IEEE Trans. Image Process., 29: 6180–6193, Apr. 2020. EXTERNAL
• K. Li, Y. Wang, Y. Li, Y. Wang, Y. He, L. Wang, and Y. Qiao, “Unmasked teacher: Towards Training-efficient Video Foundation Models,” arXiv preprint, arXiv:2303.16058, Mar. 2023. EXTERNAL
• Z. Wang, K. Kuan, M. Ravaut, G. Manek, S. Song, Y. Fang, S. Kim, N. Chen, L. F. D'Haro, L. A. Tuan, and H. Zhu, “Truly multimodal YouTube-8M Video Classification with Video, Audio, and Text,” arXiv preprint, arXiv:1706.05461, Jun. 2017. EXTERNAL
• S. S. Kalakonda, S. Maheshwari, and R. K. Sarvadevabhatla, “Action-GPT: Leveraging Large-scale Language Models for Improved and Generalized Action Generation,” Proc. IEEE Int. Conf. Multimedia Expo, Brisbane, Australia, pp. 31–36, Jul. 2023. EXTERNAL
• G. Delétang, A. Ruoss, P. A. Duquenne, E. Catt, T. Genewein, C. Mattern, J. Grau-Moya, L. K. Wenliang, M. Aitchison, L. Orseau, and M. Hutter, “Language Modeling is Compression,” arXiv preprint, arXiv:2309.10668, Sep. 2023. EXTERNAL
• Z. Duanmu, K. Zeng, K. Ma, A. Rehman, and Z. Wang, “A Quality-of-experience Index for Streaming Video,” IEEE J. Sel. Topics Signal Process., 11(1): 154–166, Sep. 2016. EXTERNAL
• S. Wang, A. Rehman, Z. Wang, S. Ma, and W. Gao, “SSIM-motivated rate-distortion optimization for video coding,” IEEE Trans. Circuits Syst. Video Technol., 22(4): 516–529, Sep. 2011. EXTERNAL
• Z. Li, Z. Duanmu, W. Liu, and Z. Wang, “AVC, HEVC, VP9, AVS2 or AV1? - A comparative study of state-of-the-art video encoders on 4K videos,” Proc. Int. Conf. Image Anal. Recognit., Waterloo, ON, Canada, pp. 162–173, Aug. 2019. EXTERNAL
• A. Rehman, K. Zeng, and Z. Wang, “Display device-adapted video Quality-of-Experience assessment,” Proc. SPIE, San Francisco, CA, vol. 9394, pp. 939406.1–11, Mar. 2015. EXTERNAL
• T-REC-P.910 — ITU-T Recommendation P.910, “Subjective Video Quality Assessment Methods for Multimedia Applications,” International Telecommunication Union, Geneva, Switzerland, Apr. 2008. EXTERNAL