The 'unreal' images change the 'real' world

There have been a few decades since computer graphics made great impacts on our life. Computer generated imagery (CGI) is now being used widely in different areas such as the movie and the design industry. As we all know, these images are unrealistic – they are just polygons, vectors and programs. But who cares about the factuality of the graphics? CGI can really help us in many aspects so we ought to make good use of this tool. In this essay, I will investigate how the CGI technology aids us and reveal its limitations and anticipation.

Computer-generated imagery (CGI) is the application of the field of computer graphics, or, more specifically, 3D computer graphics to special effects in films, television programs, commercials, simulators and simulation generally, and printed media [1].

Indeed, there are a myriad of examples of CGI technology used in our daily life. Computer aided design (CAD) is one of the applications of CGI [2]. In the field of design, designers wish to visualize the thoughts in their mind so they traditionally need paper and pen to draw what they want. Nowadays, CAD software packages enable them to make a detailed 3D objects which is all-angle visible. It makes the process of engineering and manufacturing of the products become easier and more convenient because 3D models are easier to be understood by manufacturer than hand-drawn 3D-pretending picture.

 Figure 1 3D designs by using CAD software [3]

Moreover, the CGI technology typically benefits the movie industry. As an example, the movie, ‘Toy Story’ is an animation generated purely by the CGI technology. The characters and all the background in the animation are all 3D models which can be treated as some virtual objects. All the motions and movements of the objects are manipulated based on computer programs.

Figure 2 A scene from the movie Toy Story [4]

The above ‘Toy Story’ example is an animation that people can intuitively figure out the graphics involved are not real. But what if some ‘realistic elements’ are added in the movie so that people would intuitively think that ‘the movie is so real’? Let’s see the movie Avatar as a further example.

Avatar is a 2009 American epic science fiction film which mainly consists of the CGI technology to visualize and simulate the imaginary amazing world in the movie [5]. Owing to the state-of-the-art CGI technology, not only images are generated, but also a tremendous income is also generated. Until May 4 2010, worldwide revenue of $2,777,490,390 USD has already been earned from the movie [6].

The graphical models in the CGI technology in Avatar are very similar to the things we encountered in the real world, or even more gorgeous than what we observe in the real world. Nevertheless, not like Toy Story, we think that the movie Avatar is very ‘realistic’ because the images used in the movie fit comfortably to our intuition, without a diametrical conflict to what we observe and experience in our daily life. In other words, we will ‘believe’ the unreal graphics in Avatar but not in Toy Story.

So, something wield springs up in my mind. In what mentioned above, it seems that the CGI technology is so powerful that we can purely generate a ‘realistic’ movie without any human interventions. However, something contradictory to our thought is that the movie Avatar still needs actors and actresses to generate 3D models! Without anyone acting in the movie, the CGI technology is incapable of creating creatures that moves so smooth like humans. What the directors of the movie Avatar have used to capture the motion of humans in an attempt to create human-like creatures is that they install motion sensors in every single joint of the actors and actresses [7], and meanwhile transfer the sensors’ signal to the virtual camera in order to perform motions of the computer 3D models [7]. In fact, generating a series of human actions does require a lot of human resources. Without capturing the motions of human in the real world, it seems that it is extremely difficult to generate some complicated movements of a 3D model purely by manipulating the movements of the 3D components and adjusting the positions of the joints of the 3D model. Knowing that these kinds of movies still involves human manipulation is very important because people will feel bad if we intuitively ‘believe’ that the whole displayed world in the movie Avatar is real, but the movie actually contains no ‘real’ things. We are pleased to know that there is still a little proportion of ‘real things’ (the motions and gestures of the  haracters) in the movie, corresponding to what we ‘believe’.

Figure 3 The tricks behind Avatar [8]

This fact inspires us to think of a very thought-provoking question: Can the CGI technology generate a perfectly unblemished video (or animation) containing smooth and human-like motions? In my own perspective, this dream is nearly impossible in the near future. In the field of computing, it seems like we are continuously using computer graphics to imitate the real world. In fact, I think the machine cannot perfectly simulate the real world because the world is perfect due to its imperfectness. What I mean is that there exist some blemishes in the real world as some random noises and flaws will emerge in the real world. The images generated purely by the CGI technology are highly imitative and with regular pattern, but thanks to these seem-like merits, the images cannot present the imperfect random events naturally. So, we can observe that CGI is more commonly used in the movies having imaginary characters which are less likely to be contradictory to our intuition, in order to make the character more ‘realistic’. Iron man, Spiderman and Transformer are some of the typical examples substantiating the idea that the CGI technology aids mainly the imaginary characters in the movies.

Based on the computer technology now, I think it is too soon to say that computers can one day perfectly simulate a virtual world that the unreal graphics can make us ‘believe’ the ‘unreal’ is ‘real’. Believing what we are seeing still seems agreeable in the sense that if we see something realistic, there must be at least some human manipulations involved (e.g. the motion generating aspect in the movie Avatar).

Are people going to endeavor to make the ‘realistic’ video described above? May or may not be. But at least, it’s clear that amazing images can be made without human intervention if human-like motions are not required. Believe it or not, both of the following images are based on CGI that they are purely vectors and polygons drawn by using graphical software. In the first glance, or even deeper examination, we will conclude that the two girls are real if we are not allowed to use any computer program to examine the factuality of the images.

Figure 4 The realistic women created by CGI [9]

But what if the CGI technology eventually advanced to a level that it is capable of simulating human and even the whole real world? If the CGI technology can really simulate all movements and motions that humans can do, a very interesting phenomenon may appear – the virtual idol. What I mean is a tailor-made idol who can act, sing and even do anything that human can do, but can only be seen on videos. He or she may be very handsome or very pretty, having multitudes of talents in different areas. He or she (or should I say ‘it’?) can be the main character of a movie and interact with other movie stars. The essence of this issue is in the question – would people still like to become fans of the virtual idol even they know it is not real at all?

Would people choose to ‘believe’ in a very realistic virtual object even when they are told that the object is completely made by CGI? This is really an intriguing question aiming to examine the reaction of people dealing with something which is completely unreal. But sadly, the answer will only be released at the day when CGI can impeccably simulate the real world.

Reference

1. Wikipedia Computer-generated imagery. [online], available at: http://en.wikipedia.org/wiki/Computer-generated_imagery
2. Wikipedia Computer-aided design. [online], available at: http://en.wikipedia.org/wiki/Computer-aided_design 
3. John S 2010, designer. The 3rd and 4th assignments’ illustration of the course ‘Introduction to 3D Modelling’. Course number CD 85-215-56-01 (01). HKU Space.
4. Photos: Top ten CGI films to keep the kids quiet over easter. [online], available at: http://www.computerweekly.com/galleries/235560-2/Toy-Story-Top-ten-CGI-films.htm 
5. Wikipedia Avatar (2009 film). [online], available at: http://en.wikipedia.org/wiki/Avatar_(2009_film) 
6. Total Lifetime Grosses of Avatar. Boxofficemojo. [online], available at: http://boxofficemojo.com/movies/?id=avatar.htm 
7. Taking movies beyond Avatar -- for under $150. University of Abertay Dundee 2010. [online], available at: http://www.photoxels.com/tag/avatar-movie/ 
8. Avatar Showtimes Clips and Trailers. [online], available at: http://www.youtube.com/watch?v=xq_nRfoBSm0&feature=fvw 
9. Brain 2008. Spring Fling. [online], available at: http://librarybrainblog.wordpress.com/2008/04/07/spring-fling/

Seeing is not believing: Can we trust a photo anymore?

As technology advances, many things in our daily lives can be digitized, including our photos. Digitization is an implication that things can be processed by using computers, so photos are not exceptions. In the following paragraphs, the concept of “photo” would focus on the digital images only.

An image processing software called Adobe Photoshop, or simply Photoshop, is very popular for its powerful tools to modify photos. How powerful could it be? Here we will see a few examples.

There is a tool in Photoshop called “Quick Selection Tool”, which can conveniently crop a distinct figure on the photo without including the background. This is basically done by edge detection technique, i.e., Photoshop attempts to check those pixels that have contrasting color comparing with the neighboring pixels.

Another commonly used function is to smooth out rough lines or edges in a photo. This can be done by selecting the desired edges, and then blur it by using any options like Gaussian Blur. Simply speaking, the principle used in Gaussian Blur is that for each selected pixel, we perform averaging of neighboring pixels with the weightings in Gaussian matrix. The weightings are calculated by means of a complicated Gaussian function.

An even more powerful tool in Photoshop is called “Content-Aware Fill”. It can remove any image detail on a photo and automatically fill in the space left behind from the background so it looks as if the removed content never existed. Briefly speaking, the mechanism behind this tool is randomly choosing small regions around the image detail we want to get rid of, and using these small regions to fill in the empty spaces after removal.

One can imagine that using only these three tools in Photoshop to perform a lot of modifications, like adding a new object into a photo followed by smoothing edges, or removing an extra object and use “Content-Aware Fill”. Therefore, no matter we are professionals or laymen, it is not a difficult job for us to change any details in a photo. The problem arises: should we trust a photo anymore? In fact, it depends on how we use this two-edged knife.

If we have taken a photo of nice scenery in a foreign country, and we want to use it as our desktop background, unfortunately there are a few visitors on the photo which obstructed this beautiful scenery, what could we do? In the past, we seem to have no solution except taking the photo by going to that country once again. But with the above tools, we can simply select the visitors, and delete them with the use of “Content-Aware Fill” (see Figure 1). These tools undoubtedly bring us more convenience and in turn save a lot of our time.

Figure 1: before (left) and after (right) the use of “Content-Aware Fill”.

Nevertheless, some people attempt to use these powerful tools to conceal certain facts revealed from the photos. Such act is known as “image tampering”. There can be many reasons behind for doing this, e.g. exaggeration of the facts in news in order to arouse public’s interest (see Figure 2), or even modification of people’s political memories (see Figure 3), etc. Undoubtedly this is not a good phenomenon because if the processing tools are abused, the mass media would not bear its responsibility to deliver the truth.

Figure 2: In this Reuters photo from August 2006 (left), thick black smoke rises above the capital of Lebanon after an Israeli air raid. But in the original (right), the smoke is neither as thick nor as black. Reuters subsequently removed all of photographer Adnan Hajj’s work from its Web

Figure 3: An original photo of Tiananmen Square (top) and a modified photo of the same scene (bottom). Compared with subjects who saw the real photo of Tiananmen, those who saw the modified photo were twice as likely to estimate that more than 500,000 people had participated.

We come back to the problem of whether we should trust the photos. In my opinion, as long as the modified photos are not doing any harm to any parties, we do not need to bother the originality of the photos. Just like the first imagined situation mentioned above, sometimes the “truths” (obstacles of scenery) are not what we want to see. Getting rid of them may be beneficial to us, since we improve the photos’ quality in the sense of art. However, if the photos are related to some critical issues, such as some historical events or scenes of current news, we should be more sensitive to image tampering.

But how could we know when we should trust a photo? To deal with image tampering, researchers have done a lot in a field called “digital forensic”. Digital forensic is mainly about different techniques that enable us to check whether a photo is modified, such as agreement of lighting environments, detection of chromatic aberration, etc. But one may wonder: how about the laymen? Although we cannot perform those verifications like the researchers, we can still try to make critical judgments on the issues related to the photos, and think about if there could be reasons for someone to modify those photos.

References

1. http://www.adobe.com/products/photoshop/photoshop/whatsnew/index_rr.html?segment=design
2. http://www.slate.com/id/2255276/
3. http://www.cs.dartmouth.edu/farid/research/tampering.html
4. http://spectrum.ieee.org/slideshow/computing/software/slide-show-a-tour-of-photo-tampering

Seeing is not believing: Can we trust a photo anymore? --- A real photo is really “real”?

There are two common types of equipment for taking photos, first is the traditional film camera which has a long history. The second is the digital camera which was developed in the last quarter of the 20^th century. Both of them are designed to produce images. Throughout the history of mankind, writing, drawing and spoken language have been the only three ways to record information until the camera was invented which became an extra option. Photos are proved to be a lot more effective for recording in certain areas. However, photo is also a double edged sword unfortunately.

We are living in the age of information. I would not be wrong if I say that we all are surrounded by photos everywhere in our daily life. However, there is one thing people need to take notice is that photo doesn’t necessary equal to truth because there exists image tampering. Thanks to technology, we have software like “Photoshop” which helps us easily modify an image in any way you wish. Depends on the amount of modification, the original message or the impact of an image can be changed completely. As a result, we may be easily misled by a photo if we trust it entirely. Therefore, we should not regard photo as physical evidence, instead we should use it as a reference or visual aid. This is similar to the situation of Wikipedia where anyone can edit the content of articles.

I once heard of an interesting thought saying that every photo cannot be “real” simply because it is a photo, no matter it is taken by a film or a digital camera. The question is how do we define the word “real”?  We normally distinguish a real image from a fake one by judging whether the objects in the photo really exist and the appearance is identical in the real world. If we just focus on the photo itself, regardless of the meaning or the impact it brought. For a photo without any modification, either it is directly developed from a film or captured in digital format, two out of the three primary elements of an image, line and shape, can look almost the same as the original object. However, for the last element, colour, is a lot more difficult to achieve. In terms of colour accuracy, photos are sometimes much different from what your eyes actually see unless you take pictures with the most high end camera and view the photo using a professional computer display. For most of us, the different brands and models of camera we use can produce varied images with the same scene, even in a controlled situation, where lighting, angle, position are the same. This may be the reason that people sometimes ask why the photo doesn’t seem to be real even though they know it is not a fake one. Therefore, can we decide which image is really presenting the true colours of the object? If not, can we still trust the appearance of an object in a photo?

To enrich the idea of a “real” photo, the two examples below is an experiment in photography I’ve done in the past as a hobby. The photo on the left hand side is the original without any modification, came out straight from a memory card. The photo on the right hand side is modified by making adjustments carefully to improve colour accuracy. The goal of this experiment is to capture a truly real photo which means it is very close to the reality. This is done by using naked eyes to calibrate the photo in detail at the same place as where the photo is taken immediately after it is taken.

  

The result is a photo which is almost identical to what you actually see in reality. No professional equipment like DSLR was used, but only a simple digital camera which is widely available in the consumer market.

People nowadays have different purposes to modify a photo. Some of them only want to remove the impurities from a photo so that they can have a good looking photo to share with friends. This kind of modification is generally acceptable. However, sometimes image tampering can cause a lot of troubles. Even large corporation like Microsoft use fake photo on their website. The following photo was reported on the Microsoft’s Polish website in August last year.

Although Microsoft didn’t give out any official explanation and the reason remained unknown, people suspect of racism and they feel bad about this because they wouldn’t expect such a large company would use this rather unpleasant technique.

In conclusion, I personally would advise we treat a photo just like a piece of art. Artist can do paintings freely without any restriction. They won’t be penalized because of presenting something not real. When people look at a painting, they simply focus only on the beauty and seldom have a thought of trusting anything from the painting. We all should remind ourselves that a computer only provide us a virtual platform. For example, a folder in a hard disk is not an actual folder made of paper we use for filing paperwork and the recycling bin on your desktop is not the same as a one in the real world, it is just a symbol indicating where we moved the unwanted data to. We are only looking at a screen and that’s all. For a photo print, it is just a piece of paper filled with ink, similar to a canvas with paints. What’s the point of caring about whether some virtual stuff is true or not?

There is another very good example I can think of. The movie “Matrix” expressed the idea “Seeing (or even feeling) is not believing” thoroughly. How can we prove that the world we are living, the world we know is actually real, but not a simulated reality? This brings the whole question to a new and more complex level. To keep our life simple, my suggestion would be trust a photo only if necessary. Otherwise, it wouldn’t be worthwhile having our life disturbed by over focusing on the trueness of a photo.