Title of Thesis

"Evaluating Perceptual Effects of Non-photorealistic Imagery"
(subject to change)

>> Now that we have pretty images - what do we do with them? <<

>> The following is my original abridged research proposal. A pdf version can be found here. If you require the longer version, please contact me (see cv page for details). As anyone who has been involved in a long-term research project knows, ideas, concepts and objectives tend to change and develop over time. If you would like to know more about how things have evolved in the meantime please feel free to contact me.<<

Introduction

Historically, the field of Computer Graphics has evolved around the idea of producing images that are as realistic as possible [24], coining the term photorealism. Great advances have been made in this respect and, as technology improves or becomes available, will be made for some time to come. We believe that in most cases (apart from very few simulation applications) such realistic images are not an end in themselves. They were created for a specific audience, a particular viewer. They were created for architects, designers, medical staff, movie-goers, etc. In other words, they were created to be looked at, scrutinised or enjoyed by humans. For a very long time this fact has been largely ignored, one argument being that if an image is as real as possible, it resembles reality and thus requires no tailoring for a given observer (we consider this a fallacy, as the mere act of choosing a particular scene, subject, view-point, lighting, etc. already customises the content and visual composition of an image to a large degree).

The other side of realism

One recent development which is trying to complement the merits of existing realistic graphics, and which is quickly gaining momentum in the scientific community is that of Non-photorealistic Rendering (NPR). NPR has been attributed with many desirable qualities, chiefly its abilities of representing abstract data and ideas ([23], [15], [3]), of realizing expressive styles ([16], [17], [14], [1], [13]), of evoking emotional responses ([19], [28], [21]), and of pertaining to low-level perceptual processes ([22], [5], [6]).

It is often argued, that the abovementioned qualities might be somewhat interlinked (e.g. the clarity/economy of line in sketches which is believed to appeal to low-level visual processes helps in abstracting from a concrete form), but not much is known about the correlations between these qualities. Recently, we see computer-generated NPR being employed in ever increasing form, including the latest PIXAR releases, computer-games such as XIII and Teenage Mutant Ninja Turtles, 3D modelling aids such as Teddy and Jot ([11], [12]) to name but a few. This trend is likely to continue for some time to come, as technical and theoretical advances in NPR and related fields are made.

Enabling Technologies

One of the most exciting developments with respect to NPR is the advent of user-programmable graphics processing units (GPUs). This new technology allows for (limited, but expanding) access to the graphics pipeline, thus permitting the creation of endless variations of shaders and renderers. Many special NPR effects which used to be difficult to implement (e.g. [18], [15], [20], [9], [26], [27]) can now be implemented in a few lines of portable, specialized graphics languages (CG, HLSL), even though speed and versatility are still issues on lower end graphics cards. We anticipate these problems to be overcome in the near future. One upshot of this development will be an increased ease of creating and implementing NPR renderers and effects. As a result, NPR will increasingly find its way onto desktop computers in numerous guises.

Our Goal

We therefore return to our initial statement; that computer generated images are generally created for humans. This is especially true for NPR images (we find it difficult to conceive of a situation where, say, a water-color rendition of network-traffic would be analyzed by a secondary program). Currently, only very little research has been undertaken into the perceptual effects of NPR, i.e. how NPR images are actually perceived and how this affects interaction with NPR environments ([6], [2], [23]). It is our aim to address this issue.

Our Approach

Using our background in NPR research [25] and combining this with work in human perception (e.g. [4], [8], [7], [10], [29]) we are in the process of building specialised renderers, which address perceptual modules of the human visual system. User-studies are then to be employed to test various virtual environment (VE) settings and evaluate them with respect to interactivity measures as compared to more traditionally rendered VEs. The analysis of our tests will help identify areas of perception and interaction differences on various levels. This in turn will assist in constructing guidelines for use with NPR environments.

Conclusion

After significant advances in NPR theory and technology have been brought forward in all the recent major graphics-related conferences, we believe it is time to research meta-aspects of NPR, such as perception and interaction with NPR environments. We hope that by doing this, we are able to contribute to the understanding of why and where NPR can be gainfully employed and what problems are associated with its use.

References

[1] Cohen, J. M ., Hughes, J. F., and Zeleznik, R. C. Harold: A World Made of Drawings. In Proceedings of NPAR 2000, Symposium on Non-Photorealistic Animation and Rendering (Annecy, France, June 2000) (New York, 2000), ACM, pp. 83­90.

[2] DeCarlo, D., Finkelstein, A ., Rusinkiewicz, S., and Santella, A . Suggestive contours for conveying shape. In Proceedings SIGGRAPH (2003).

[3] Durand, F. An invitation to discuss computer depiction. In Proc. 2nd Intern. Symposium on Non-Photorealistic Animation and Rendering (June 2002).

[4] Ferweda, J. A . Fundamentals of spatial vision. In Applications of Visual Perception (1998), Siggraph '98 Course Notes.

[5] Gooch, A . A ., and Gooch, B. Using Non-Photorealistic Rendering to Communicate Shape. In SIGGRAPH'99 Course Notes. Course on Non-Photorelistic Rendering, S. Green, Ed. New York, 1999, ch. 8.

[6] Gooch, A . A ., and Willemsen, P. Evaluating space perception in NPR immersive environments. In Proceedings of the Second International Symposium on Non-Photorealistic Animation and Rendering (Annecy, France, June 2002), pp. 105­110.

[7] Gordon, I. E. Theories of Visual Perception. John Wiley & Sons Ltd., 1989.

[8] Gregory, R. Eye and Brain - The Psychology of Seeing, 4th ed. Oxford University Press, 1994.

[9] Hall, P. Comic-strip Rendering. Tech. Rep. CS-TR-95/ 2, Department of Computer Science, Victoria University of Wellington, New Zealand, 1995.

[10] Hoffman, D. D. Visual Intelligence: How We Create What We See. W.W. Norton & Company, NY, 2000.

[11] Igarashi, T., Matsuoka, S., and Tanaka, H. Teddy: A Sketching Interface for 3D Freeform Design. In Proceedings of SIGGRAPH'99 (Los Angeles, Aug. 1999) (New York, 1999), Computer Graphics Proceedings, Annual Conference Series, ACM Siggraph, pp. 409­416.

[12] Kalnins, R. D., Davidson, P. L., Markosian, L., and Finkelstein A . WYSIWYG NPR: Drawing strokes directly on 3D models. In ACM Transactions on Graphics (July 2003), vol. 22, pp. 856­861.

[13] Klein, A. W., Li, W. W., K azhdan, M. M., Cor r ea, W. T., Finkelstein, A., and Funkhouser , T. A. Non-Photorealistic Virtual Environments. In Proceedings of SIGGRAPH 2000 (New Orleans, July 2000) (New
York, 2000), K. Akeley, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, pp. 527­534.

[14] Kowalski, M. A., Markosian, L., Northrup, J. D., Bourdev, L., Barzel , R., Holden, L. S., and Hughes, J. F. Art-Based Rendering of Fur, Grass, and Trees. In Proceedings of SIGGRAPH'99 (Los Angeles, Aug. 1999) (New York, 1999), Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, pp. 433­438.

[15] Lake, A., Marshal l , C., Harris, M., and Blackstein, M. Stylized Rendering Techniques for Scalable Real-Time 3D Animation. In Proceedings of NPAR 2000, Symposium on Non-Photorealistic Animation and Rendering (Annecy, France, June 2000) (New York, 2000), ACM, pp. 13­20.

[16] Lansdown, J., and Schofield, S. Expressive Rendering: A Review of Nonphotorealistic Techniques. IEEE Computer Graphics and Applications 15, 3 (May 1995), 29­37.

[17] Markosian, L. Art-based Modeling and Rendering for Computer Graphics. PhD thesis, Department of Computer Science at Brown University, 2000.

[18] Markosian, L., Kowalsk i, M . A ., Trychin, S. J., Bourdev, L. D., Goldstein, D., and Hughes, J. F. Real-Time Nonphotorealistic Rendering. In Proceedings of SIGGRAPH'97 (Los Angeles, Aug. 1997) (1997),
T. Whitted, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH, pp. 415­420.

[19] Mc Cloud, S. Understanding Comics - The Invisible Art. Harper Perennial, 1993.

[20] Meier, B. J. Painterly Rendering for Animation. In Proceedings of SIGGRAPH'96 (New Orleans, Aug. 1996) (New York, 1996), H. Rushmeier, Ed., Computer Graphics Proceedings, Annual Conference Series, ACM
SIGGRAPH, pp. 477­484.

[21] Reitberger , R., and Fuchs, W . Comics - Anatomy of a Mass Medium. Little, Brown and Company, Boston, Toronto, 1972.

[22] Saito, T ., and Takahashi, T . Comprehensible Rendering of 3-D Shapes. In Proceedings of SIGGRAPH'90 (Dallas, Aug. 1990) (New York, Aug. 1990), F. Baskett, Ed., Computer Graphics Proceedings, Annual Confer-
ence Series, ACM SIGGRAPH, pp. 197­206.

[23] Strothotte, T ., and Schlechtweg, S. Non-Photorealistic Computer Graphics: Modeling, Rendering and Animation. Morgan Kaufmann, 2002.

[24] Watt, A., and Watt , M. Advanced Animation and Rendering Techniques. Addison Wesley, ACM Press, 1992.

[25] Winnemöller, H. Implementing Non-photorealistic Rendering Enhancements with Real-Time Performance. Master's thesis, Rhodes University, South Africa, 2002.

[26] Winnemöller, H., and Bangay, S. Geometric approximations towards free specular comic shading. Computer Graphics Forum 21, 3 (September 2002), 309­316.

[27] Winnemöller, H., and Bangay, S. Rendering Optimisations for Stylised Sketching. In ACM Afrigraph 2003: 2nd International Conference on Computer Graphics, Virtual Reality and Visualization in Africa (February 2003), ACM, ACM SIGGRAPH, pp. 117­122.

[28] Wood, D. N., Finkelstein, A., Hughes, J. F., Thayer , C. E., and Salesin, D. H. Multiperspective panoramas for cel animation. Computer Graphics 31, Annual Conference Series (1997), 243­250.

[29] Zeki, S. A Vision of the Brain. Blackwell Scientific Publications, Oxford, 1993.