SMPTE Meetings and Conferences ( October 2022)
[ACTIVE]
Using ML to Find the Semantic Region of Interest
Metadata
- Publisher
- SMPTE — White Plains, NY, USA
- Doc Type
- Conference Paper
- Content Type
- Original Research
- Volume
- 2022, No. 10, pp. 1–11
- Abstract
- One of the most challenging problems in computer vision and image processing is the detection of the semantic regions of interest (SRoI). In this paper, we propose a method using OpenAI's CLIP model to find SRoI by performing semantic search for objects in the image which are detected by an object detection model called Generic RoI Extractor (GRoIE). Finding the semantic regions of interest can be used in different image processing tasks such as image and video compression, enhancement, and reformatting. By knowing the semantic region of interest within images, we can improve the visual quality of images by compressing the more important parts with higher quality and the less important parts, such as the background, with a lower quality. This operation can be achieved without changing the overall compression ratio and the Peak Signal-to-noise Ratio (PSNR) quality metric. Finding the SRoI can make the processes of image enhancement and color correction more accurate by focusing only on the important parts. Moreover, for the image reformatting process, the important parts of the image may be lost. But by using the SRoI, we can reformat the image in a better way by keeping the most important regions in the frame.
- Publication Date
- 2022-10-24
- DOI
10.5594/M001971- ISBN
- Print:
978-1-61482-963-8 - Link
- https://doi.org/10.5594/M001971
- Author(s)
- Zahra MontajabiConcordia University, Avid TechnologyRob GonsalvesEngineering Fellow, Avid TechnologyNizar BouguilaPhD, Concordia University
- Keyword(s)
- Artificial Intelligence, Machine Learning, Semantic Search, OpenAI CLIP, Object Detection, Image Compression, Image Reformatting, Image Enhancement, Region of Interest, Semantic Region Of Interest (SRoI)
- Copyright
- © 2022 Society of Motion Picture and Television Engineers, Inc.
Bibliographic Reference(s)
- A. Radford, et al., “Learning Transferable Visual Models from Natural Language Supervision,” arXiv:2103.00020, Feb. 2021 EXTERNAL
- G. Salton and C. Buckley, “Term-weighting Approaches in Automatic Text Retrieval,” Information Processing and Management, 1;24(5):513-23, Jan. 1988 EXTERNAL
- Z. Cai and N. Vasconcelos, "Cascade R-CNN: High Quality Object Detection and Instance Segmentation," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 43, no. 5, pp. 1483-1498, 1 May 2021, doi: 10.1109/TPAMI.2019.2956516. EXTERNAL
- Rossi, Leonardo & Karimi, Akbar & Prati, Andrea. (2021). A Novel Region of Interest Extraction Layer for Instance Segmentation. 10.1109/ICPR48806.2021.9412258. EXTERNAL
- https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/concept-generating-thumbnails EXTERNAL
- F. Cheng and S. Huang, "Efficient Histogram Modification Using Bilateral Bezier Curve for the Contrast Enhancement," in Journal of Display Technology, vol. 9, no. 1, pp. 44-50, Jan. 2013, doi: 10.1109/JDT.2012.2226234. EXTERNAL
- Prakash, Aaditya & Moran, Nick & Garber, Solomon & Dilillo, Antonella & Storer, James. (2017). Semantic Perceptual Image Compression Using Deep Convolution Networks. 250-259. 10.1109/DCC.2017.56. EXTERNAL
- I. S. Amrutha, S. S. Shylaja, S. Natarajan, K. N. Balasubramanya Murthy, “A smart automatic thumbnail cropping based on attention driven regions of interest extraction”, ICIS '09: Proceedings of the 2nd International Conference on Interaction Sciences: Information Technology, Culture and Human, November 2009 Pages 957–962, https://doi.org/10.1145/1655925.1656099 EXTERNAL
- R. Girshick, "Fast R-CNN," 2015 IEEE International Conference on Computer Vision (ICCV), 2015, pp. 1440-1448, doi: 10.1109/ICCV.2015.169. EXTERNAL
- S. M. Pizer, E. P. Amburn, J. D. Austin, et al.: Adaptive Histogram Equalization and Its Variations. Computer Vision, Graphics, and Image Processing 39 (1987) 355-368. EXTERNAL
- B. Subramani, A. K. Bhandari and M. Veluchamy, "Optimal Bezier Curve Modification Function for Contrast Degraded Images," in IEEE Transactions on Instrumentation and Measurement, vol. 70, pp. 1-10, 2021, Art no. 5008910, doi: 10.1109/TIM.2021.3073320. EXTERNAL
- Scott E Umbaugh, Computer Vision and Image Processing, Prentice Hall: New Jersey 1998, pp. 209. EXTERNAL
- K. He, G. Gkioxari, P. Dollár and R. Girshick, "Mask R-CNN," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 2, pp. 386-397, 1 Feb. 2020, doi: 10.1109/TPAMI.2018.2844175. EXTERNAL
- G. K. Wallace, "The JPEG still picture compression standard," in IEEE Transactions on Consumer Electronics, vol. 38, no. 1, pp. xviii-xxxiv, Feb. 1992, doi: 10.1109/30.125072. EXTERNAL
Source Data (JSON)
Full registry record with provenance metadata. Open directly: /api/doc/10.5594-M001971.json
Reference Tree
Explore all references and references to this document, as a navigable tree.
Open Reference TreeReference this Doc
Plain text (ISO 690 compliant)
Preview:
Zahra Montajabi, Rob Gonsalves, and Nizar Bouguila; Using ML to Find the Semantic Region of Interest, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001971
Snippet:
Zahra Montajabi, Rob Gonsalves, and Nizar Bouguila; Using ML to Find the Semantic Region of Interest, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001971
HTML (ISO 690 compliant)
Preview:
Zahra Montajabi, Rob Gonsalves, and Nizar Bouguila; Using ML to Find the Semantic Region of Interest, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at https://doi.org/10.5594/M001971
Snippet:
<span class="citation">Zahra Montajabi, Rob Gonsalves, and Nizar Bouguila; <cite>Using ML to Find the Semantic Region of Interest</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8. Available at <a href="https://doi.org/10.5594/M001971" target="_blank" rel="noopener">https://doi.org/10.5594/M001971</a></span>
SMPTE's HTML Pub
Preview:
Zahra Montajabi, Rob Gonsalves, and Nizar Bouguila; Using ML to Find the Semantic Region of Interest, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8
doi: 10.5594/M001971
url: https://doi.org/10.5594/M001971
doi: 10.5594/M001971
url: https://doi.org/10.5594/M001971
Snippet:
<li> Zahra Montajabi, Rob Gonsalves, and Nizar Bouguila; <cite id="bib-10-5594-m001971">Using ML to Find the Semantic Region of Interest</cite>, SMPTE Meetings and Conferences ( October 2022); SMPTE, 2022, ISBN: 978-1-61482-963-8 <span class="doi">10.5594/M001971</span> </li>