SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023)
[ACTIVE]
Semantic Regions of Interest Applied to Adaptive Image Compression, Color Enhancement, and Automated Pan-and-Scan
Metadata
- Publisher
- SMPTE — White Plains, NY, USA
- Doc Type
- Journal Article
- Content Type
- Original Research
- Abbreviated Title
- SMPTE Mot. Imag. J
- Volume
- 132, No. 3, pp. 39–47
- Abstract
- One of the most challenging problems in computer vision and image processing is the detection of the semantic regions of interest (SRoIs). In this article, we propose a method using OpenAI’s contrastive language-image pretraining (CLIP) model to find SRoIs by performing a semantic search for objects in the image that are detected by an object detection model called generic RoI extractor (GRoIE). Finding the SRoI can be used in different image-processing tasks such as image and video compression, enhancement, and reformatting. By knowing the SRoI within images, we can improve the visual quality of images by compressing the more important parts with higher quality and the less important parts, such as the background, with a lower quality. This operation can be achieved without changing the overall compression ratio and the peak signal-to-noise ratio (PSNR) quality metric. Finding the SRoI can make the processes of image enhancement and color correction more accurate by focusing only on the important parts. Moreover, in the image reformatting process, the important parts of the image may be lost. But by using the SRoI, we can reformat the image in a better way by keeping the regions within a frame that draw the greatest audience attention.
- Publication Date
- 2023-04-01
- DOI
10.5594/JMI.2023.3246764- ISSN
- Print:
1545-0279| Electronic:2160-2492 - Link
- https://doi.org/10.5594/JMI.2023.3246764
- Author(s)
- Zahra Montajabi
bio
Zahra Montajabi received a BSc degree in computer engineering from the Sharif University of Technology, Tehran, Iran, in 2019, and an MSc degree in quality system engineering from Concordia University, Montreal, QC, Canada, in 2022. She was an AI researcher intern with Avid Technology, Montreal. She is currently an AI/ML software developer with Avid Technology. Her research interests are computer vision, image processing, and machine learning. She is a member of IEEE.Rob Gonsalvesbio
Rob Gonsalves joined Avid Technology as their 15th employee in 1989. He helped to develop the industry’s preeminent nonlinear editing system Avid Media composer, specializing in programming video effects and color correction. His work on multicamera editing software led to Avid winning the Technology and Engineering Emmy Award. He later spearheaded the development of the architecture of Avid’s media enterprise products. He is currently researching the use of AI for media production. He holds more than 50 patents in the media and entertainment industry.Shailendra Mathurbio
Shailendra Mathur is responsible for the technology and architecture of Avid’s products in the role of vice president and chief architect. He has been awarded several patents and presented papers at numerous conferences, panels, and journals on subjects such as AI, computer vision, medical imaging, media creation tools, security, immersive entertainment, asset management, 4K/HDR, high-performance computing, and cloud technologies. With more than 25 years of innovation experience in the media industry, he was privileged to be one of the team members who received the Philo T. Farnsworth Prime Time Engineering Emmy Award on behalf of Avid in 2018.Nizar Bouguilabio
Nizar Bouguila received an engineering degree from the University of Tunis, Tunis, Tunisia, in 2000, and MSc and PhD degrees in computer science from Sherbrooke University, Sherbrooke, QC, Canada, in 2002 and 2006, respectively. He is currently a professor at the Concordia Institute for Information Systems Engineering (CIISE), Concordia University, Montreal, QC, Canada. His research interests include image processing, machine learning, data mining, computer vision, pattern recognition, smart buildings, and energy. He is a licensed professional engineer registered in Ontario and a Senior Member of the IEEE for 12 years. - Keyword(s)
- Artificial Intelligence, Image Compression, Image Enhancement, Image Reformatting, Machine Learning, Object Detection, CLIP, Region of Interest, Semantic Region Of Interest (SRoI), Semantic Search
- Copyright
- © 2023 SMPTE
Bibliographic Reference(s)
- Using ML to Find the Semantic Region of Interest [Active]
- G. Salton and C. Buckley, “Term-Weighting Approaches in Automatic Text Retrieval,” Inform. Process. Manag., 24(5): 513–523, Jan. 1988. EXTERNAL
- G. K. Wallace, “The JPEG Still Picture Compression Standard,” IEEE Trans. Consumer Electron., 38(1):xviii–xxxiv, Feb. 1992, doi: 10.1109/30.125072. EXTERNAL
- I. S. Amrutha, S. S. Shylaja, S. Natarajan, and K. N. Balasubramanya Murthy, “A Smart Automatic Thumbnail Cropping Based on Attention Driven Regions of Interest Extraction,” ICIS ‘09: Proc. of the 2nd Int. Conf. on Interaction Sci.: Inform. Technol., Culture and Human, pp. 957–962, Nov. 2009, doi: 10.1145/1655925.1656099. EXTERNAL
- A. Radford et al., “Learning Transferable Visual Models From Natural Language Supervision,” Feb. 2021, arXiv:2103.00020. EXTERNAL
- Azure Documentation. Accessed: Feb. 2023. [Online]. Available: https://docs.microsoft.com /en-us/azure/cognitive-ser vices/ computer-vision/concept-generating-thumbnails EXTERNAL
- F. Cheng and S. Huang, “Efficient Histogram Modification Using Bilateral Bezier Curve for the Contrast Enhancement,” J. Display Technol., 9(1):44–50, Jan. 2013, doi: 10.1109/ EXTERNAL
- K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” IEEE Trans. Pattern Anal. Mach. Intell., 42(2):386–397, Feb. 2020, doi: 10.1109/TPAMI.2018.2844175. EXTERNAL
- Z. Cai and N. Vasconcelos, “Cascade R-CNN: High Quality Object Detection and Instance Segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., 43(5):1483–1498, May 2021, doi: 10.1109/ EXTERNAL
- L. Rossi, A. Karimi, and A. Prati, “A Novel Region of Interest Extraction Layer for Instance Segmentation,” 2020 25th Int. Conf. on Pattern Recognit. (ICPR), Apr. 2020. doi: 10.1109/ EXTERNAL
- B. Subramani, A. K. Bhandari, and M. Veluchamy, “Optimal Bezier Curve Modification Function for Contrast Degraded Images,” IEEE Trans. Instrum. Meas., 70:1–10, Jan. 2021, doi: 10.1109/TIM.2021.3073320. EXTERNAL
- S. M. Pizer et al., “Adaptive Histogram Equalization and Its Variations,” Comput. Vision Graph. Image Process., 39(3):355–368, 1987. EXTERNAL
- A. Prakash, N. Moran, S. Garber, A. Dilillo, and J. Storer, “Semantic Perceptual Image Compression Using Deep Convolution Networks,” Proc. 2017 Data Compression Conf. (DCC), pp. 250–259, 2017, doi: 10.1109/DCC.2017.56. EXTERNAL
- R. Girshick, “Fast R-CNN,” 2015 IEEE Int. Conf. on Comput. Vision (ICCV), pp. 1440–1448, 2015, doi: 10.1109/ICCV.2015.169. EXTERNAL
- S. E. Umbaugh, Computer Vision and Image Processing, p. 209, Prentice Hall: Englewood Cliffs, NJ, 1998. EXTERNAL
Source Data (JSON)
Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2023.3246764.json
Reference Tree
Explore all references and references to this document, as a navigable tree.
Open Reference TreeReference this Doc
Plain text (ISO 690 compliant)
Preview:
Zahra Montajabi, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; Semantic Regions of Interest Applied to Adaptive Image Compression, Color Enhancement, and Automated Pan-and-Scan, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023. Available at https://doi.org/10.5594/JMI.2023.3246764
Snippet:
Zahra Montajabi, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; Semantic Regions of Interest Applied to Adaptive Image Compression, Color Enhancement, and Automated Pan-and-Scan, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023. Available at https://doi.org/10.5594/JMI.2023.3246764
HTML (ISO 690 compliant)
Preview:
Zahra Montajabi, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; Semantic Regions of Interest Applied to Adaptive Image Compression, Color Enhancement, and Automated Pan-and-Scan, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023. Available at https://doi.org/10.5594/JMI.2023.3246764
Snippet:
<span class="citation">Zahra Montajabi, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; <cite>Semantic Regions of Interest Applied to Adaptive Image Compression, Color Enhancement, and Automated Pan-and-Scan</cite>, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023. Available at <a href="https://doi.org/10.5594/JMI.2023.3246764" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2023.3246764</a></span>
SMPTE's HTML Pub
Preview:
Zahra Montajabi, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; Semantic Regions of Interest Applied to Adaptive Image Compression, Color Enhancement, and Automated Pan-and-Scan, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023
doi: 10.5594/JMI.2023.3246764
url: https://doi.org/10.5594/JMI.2023.3246764
doi: 10.5594/JMI.2023.3246764
url: https://doi.org/10.5594/JMI.2023.3246764
Snippet:
<li> Zahra Montajabi, Rob Gonsalves, Shailendra Mathur, and Nizar Bouguila; <cite id="bib-10-5594-jmi-2023-3246764">Semantic Regions of Interest Applied to Adaptive Image Compression, Color Enhancement, and Automated Pan-and-Scan</cite>, SMPTE Motion Imaging Journal ( Volume: 132, Issue: 3, April 2023); SMPTE, 2023 <span class="doi">10.5594/JMI.2023.3246764</span> </li>