Is Seeing Still Believing: Factors That Allow Humans and Machines to Discriminate Between Real and Generated Images

Metadata

Publisher: SMPTE — White Plains, NY, USA
Doc Type: Journal Article
Content Type: Original Research
Abbreviated Title: SMPTE Mot. Imag. J
Volume: 127, No. 9, pp. 70–78
Abstract: The growing photorealistic capabilities of 3D animation and computer-generated imagery (CGI) poses potential issues for broadcasters, regulators, governments, and viewers. What happens if a news story is broadcast, or available on a website, that is CGI or a combination of CGI and camera-acquired images? There are already instances of video where there is polarized debate as to the degree of CGI present in the published content, a prime example being the Islamic State of Iraq and Syria (ISIS) video of the burning of a person. An analysis of the artefacts seen in the video that gave rise to the truthfulness debate is presented. The factors that humans use to discriminate or to evaluate the realness of photographs include shadow softness, surface smoothness, scene complexity and composition, and the number of light sources. Initial studies showed a human ability to discriminate; however, recent studies have shown that the ability to discriminate is getting more difficult, although with training the ability improves. Computer Vision research has evaluated image features and descriptors that discriminate between photographs and CGI. No such studies with quantitative data or experimental methodologies seem to exist for evaluating the CGI moving image by humans or computers. Possible temporal assessment factors for human discrimination (motion parameters), for computer vision (optical flow), and for artificial intelligence (semantic scene analysis) are presented.
Publication Date: 2018-10-01
DOI: 10.5594/JMI.2018.2861438
ISSN: Print: 1545-0279 | Electronic: 2160-2492
Link: https://doi.org/10.5594/JMI.2018.2861438
Author(s): Martyn Gates
Keyword(s): Artificial Intelligence, Computer-generated Imagery (CGI), CGI Artifacts, Compression Artifacts, Computer Vision, Faked News, Visual Perception, ISIS, Optic Flow, Photorealism, Semantic Scene Analysis
Copyright: © 2018 SMPTE

Bibliographic Reference(s)

O. Holmes, M. S. Banks, and H. Farid, “Assessing and Improving the Identification of Computer-Generated Portraits,” ACM Trans. Appl. Percept., 13(2):1–12, 2016. EXTERNAL
M. Szummer and R. W. Picard, “Indoor-Outdoor Image Classification,” IEEE International Workshop on Content-Based Access of Image and Video Databases, in Conjunction With CAIVD’98, pp. 42–51, 1998. EXTERNAL
AT&T, “1924: Fax Service,” 2002. [Online]. Available: http://www.corp.att.com/attlabs/reputation/timeline/24fax.html EXTERNAL
A. Wedel, U. Franke, J. Klappstein, T. Brox, and D. Cremers, Realtime Depth Estimation and Obstacle Detection from Monocular Video,” Lecture Notes Comp. Sci., 4174:475–484, 2006. EXTERNAL
B. Walski, “British Soldier in Basra,” 2003. [Online]. Available: http://www.museumofhoaxes.com/hoax/photo_database/image/british_soldier_in_basra EXTERNAL
K. Masaoka, M. Nishida, M. Sugawara, E. Nakasu, and Y. Nojiri, “Sensation of Realness From High-Resolution Images of Real Objects,” IEEE Trans. Broadcasting, 59(01):72–83, Mar. 2013. EXTERNAL
“New ISIS Video Shows Jordanian Pilot Being Burned Alive,” February 3, 2015. [Online]. Available: www.foxnews.com: http://nation.foxnews.com/2015/02/03/new-isis EXTERNAL
A. Vailaya, A. K. Jain, and H.-J. Zhang, “On Image Classification: City vs. Landscapes,” Int. J. Pattern Recogn., 31:1921–1936, 1998. EXTERNAL
W. Loman, “The Muath Al-Kaseasbeh Immolation Video: ISIS™ Uniforms Look a Lot Like Jordanian Military,” February 4, 2015. [Online]. Available: https://willyloman.wordpress.com/2015/02/04/the-muath-al-kaseasbeh-immolation-video-isis-uniforms-look-a-lot-like-jordanian-military/ EXTERNAL
B. Kaneva, A. Torralba, and W. T. Freeman, “Evaluation of Image Features Using a Photorealistic Virtual World,” Int. Conf. Comp. Vision, 2011. EXTERNAL
T. Wictor, “The Jordanian Pilot was not Burned Alive,” February 5, 2015. [Online]. Available: http://www.thomaswictor.com/the-jordanian-pilot-was-not-burned-alive/ EXTERNAL
F. Cutzu, R. Hammond, and A. Leykin, “Estimating the Photorealism of Images: Distinguishing Paintings from Photographs,” Bloomington, IN, 2003. EXTERNAL
P. Rademacher, J. Lengyel, E. Cutrell, and T. Whitted, “Measuring the Perception of Visual Realism in Images,” in Rendering Techniques. S. G. Alia Ed. 2001. EXTERNAL
“British Soldier in Basra,” 2015. [Online]. Available: http://hoaxes.org/photo_database/image/british_soldier_in_basra EXTERNAL
K. Jenkins, “The 85 Pentagon Area Surveillance Cameras,” 2006. [Online]. Available: http://www.9-11tv.org/the-pentagon-plane-puzzle/85-pentagon-area-surveillance-cameras EXTERNAL
A. Hanson, E. Learned-Miller, and M. Narayana, “Coherent Motion Segmentation in Moving Camera Videos,” ICCV, 2013. EXTERNAL
S. Lim, J. G. Apostolopoulos, and A. E. Gamal, “Optical Flow Estimation using Temporally Oversampled Video,” IEEE Trans. Image Proc., 14(8):1074–1087, 2006. EXTERNAL
The Max Planck Institute for Intelligent Systems, “Stereo and Optical Flow,” 2017. [Online]. Available: https://ps.is.tuebingen.mpg.de/research_fields/stereo-and-flow EXTERNAL
O. Haines, “An Introduction to Simultaneous Localization and Mapping,” Kudan Computer Vision, May 13, 2016. [Online]. Available: https://www.kudan.eu/kudan-news/an-introduction-to-slam/ EXTERNAL
V. Athitsos, M. J. Swain, and C. Frankel, “Distinguishing Photographs and Graphics on the World Wide Web,” Workshop on Content-Based Access of Image and Video Libraries (CBAIVL ’97). Puerto Rico, 1997. EXTERNAL
M. Pinson and S. Wolf, “Comparing Subjective Video Quality Testing Methodologies,” July 11, 2003. [Online]. Available: http://www.videoclarity.com/PDF/WPSubjectiveTestingMeth-odsReviewed.pdf EXTERNAL
R. Salmon, “Higher Frame Rates for Television (M. Gates, Interviewer),” September 12, 2013. EXTERNAL
Liveleak, “Jordanian Pilot Fake Burning Debunked!!” February, 2015. [Online]. Available: http://www.liveleak.com/view?i=df2_1423081561&comments=1 EXTERNAL
A. Chalmers, “Warwick Leader in High Dynamic Range Video to Demo at World’s Biggest Media Tradeshow,” February 2012. [Online]. Available: http://www2.warwick.ac.uk/fac/sci/wmg/mediacentre/wmgnews/?newsItem=094d43a235632b2d013580734c1551c3 EXTERNAL
MSU Graphics & Media Lab (Video Group), “MSU Subjective Comparison of Modern Video Codecs,” 2014. [Online]. Available: http://www.compression.ru/video/codec_comparison/01_sub-ject_codec_comparison/subject_codec_comparison_part1.htm EXTERNAL
E. Selmanovic, “Stereoscopic High Dynamic Range Imaging,” 2013. [Online]. Available: http://wrap.warwick.ac.uk/58125/ EXTERNAL
H. Kalva and J.-B. Lee, “Video Coding Techniques and Standards,” in Encyclopedia of Multimedia, 2008. [Online]. Available: http://www.springerreference.com/docs/html/chapterdbid/73083.html EXTERNAL
N. Narita and M. Kanazawa, “Analysis of Psychological Factors for HDTV/3-D HDTV and Evaluation Method of Their Overall Impressions,” IEEE Trans. Broadcasting, 59(01):13–19, 2013. EXTERNAL
J. Flybring, “The 2003 Brian Walski Scandal,” November 4, 2009. [Online]. Available: https://johanflybringcmp.wordpress.com/page/2/video-purportedly-shows-jordanian-pilot-being-burned-alive EXTERNAL
H. Ackerman, W. Liao, and B. Rosenhahn, “Semantic Scene Analysis,” 2017. [Online]. Available: http://www.tnt.uni-hannover.de/project/Semantic_Scene_Analysis/ EXTERNAL
R-REC-BT.500-13 — Recommendation ITU-R BT.500-13, “Methodology for the Subjective Assessment of the Quality of Television Pictures,” International Telecommunication Union, Radiocommunication Sector, Geneva, 2012. EXTERNAL

Source Data (JSON)

Full registry record with provenance metadata. Open directly: /api/doc/10.5594-JMI.2018.2861438.json

Reference Tree

Explore all references and references to this document, as a navigable tree.

Open Reference Tree

Reference this Doc

Plain text (ISO 690 compliant)

Preview:

Martyn Gates; Is Seeing Still Believing: Factors That Allow Humans and Machines to Discriminate Between Real and Generated Images, SMPTE Motion Imaging Journal ( Volume: 127, Issue: 9, October 2018); SMPTE, 2018. Available at https://doi.org/10.5594/JMI.2018.2861438

Snippet:

Martyn Gates; Is Seeing Still Believing: Factors That Allow Humans and Machines to Discriminate Between Real and Generated Images, SMPTE Motion Imaging Journal ( Volume: 127, Issue: 9, October 2018); SMPTE, 2018. Available at https://doi.org/10.5594/JMI.2018.2861438

HTML (ISO 690 compliant)

Preview:

Snippet:

<span class="citation">Martyn Gates; <cite>Is Seeing Still Believing: Factors That Allow Humans and Machines to Discriminate Between Real and Generated Images</cite>, SMPTE Motion Imaging Journal ( Volume: 127, Issue: 9, October 2018); SMPTE, 2018. Available at <a href="https://doi.org/10.5594/JMI.2018.2861438" target="_blank" rel="noopener">https://doi.org/10.5594/JMI.2018.2861438</a></span>

SMPTE's HTML Pub

Preview:

Martyn Gates; Is Seeing Still Believing: Factors That Allow Humans and Machines to Discriminate Between Real and Generated Images, SMPTE Motion Imaging Journal ( Volume: 127, Issue: 9, October 2018); SMPTE, 2018
doi: 10.5594/JMI.2018.2861438
url: https://doi.org/10.5594/JMI.2018.2861438

Snippet:

<li>
Martyn Gates; <cite id="bib-10-5594-jmi-2018-2861438">Is Seeing Still Believing: Factors That Allow Humans and Machines to Discriminate Between Real and Generated Images</cite>, SMPTE Motion Imaging Journal ( Volume: 127, Issue: 9, October 2018); SMPTE, 2018
<span class="doi">10.5594/JMI.2018.2861438</span>
</li>