THAT’S A RELIEF: ASSESSING BEAUTY, REALISM, AND LANDFORM CLARITY IN TERRAIN MAPS by NATHANIEL ALEXANDER DOUGLASS A THESIS Presented to the Department of Geography and the Division of Graduate Studies of the University of Oregon in partial fulfillment of the requirements for the degree of Master of Science September 2021 THESIS APPROVAL PAGE Student: Nathaniel Alexander Douglass Title: That’s a Relief: Assessing Beauty, Realism, and Landform Clarity in Terrain Maps This thesis has been accepted and approved in partial fulfillment of the requirements for the Master of Geography degree in the Department of Geography by: Carolyn Fish Chairperson Amy Lobben Member James Meacham Member and Andrew Karduna Interim Vice Provost for Graduate Studies Original approval signatures are on file with the University of Oregon Division of Graduate Studies. Degree awarded September 2021 ii © 2021 Nathaniel Alexander Douglass This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike (United States) License iii THESIS ABSTRACT Nathaniel Alexander Douglass Master of Geography Department of Geography September 2021 Title: That’s a Relief: Assessing Beauty, Realism, and Landform Clarity in Terrain Maps Terrain maps are often composed of shaded relief along with other thematic layers to create aesthetically pleasing and clear maps of the physical landscape. Despite that the interplay of layers is of primary concern to a cartographer, much of the research on terrain mapping has focused on testing individual layers. This research aimed to fill the gap by testing the perceived aesthetics of beauty and realism, in combination with landform clarity of terrain maps when combining shaded relief with common thematic terrain layers using an online user study. Ultimately, neither shaded relief nor thematic terrain layers were the sole contributors to aesthetics or clarity rating scores. Given the results, I argue that a successful terrain map, that accounts for the aesthetics beauty, realism, and visual clarity of specific landforms, is created through a combination of layers, not a single dataset. iv CURRICULUM VITAE NAME OF AUTHOR: Nathaniel Alexander Douglass GRADUATE AND UNDERGRADUATE SCHOOLS ATTENDED: University of Oregon, Eugene Humboldt State University, Arcata, CA DEGREES AWARDED: Master of Arts, Geography, University of Oregon Bachelors of Arts, Geography, Humboldt State University AREAS OF SPECIAL INTEREST: Cartography PROFESSIONAL EXPERIENCE: Graduate Employee, UO: InfoGraphics Lab, Jan 2020 – Jun 2021 Cartographic production and GIS analysis Graduate Employee, UO: Department of Geography, Aug 2019 – June 2021 GEOG 481: GIScience I Fall 2019 GEOG 493: Advanced Cartography Winter 2020 GEOG 181: Our Digital Earth Spring 2020 GEOG 481: GIScience I Summer 2020 GEOG 481: GIScience I Fall 2020 GEOG 493: Advanced Cartography Winter 2021 GEOG 490: Web Mapping Spring 2021 Cartographer, Green Trails Maps, Oct 2018 – Present Cartographic production and field mapping Cartographer, Apple Maps, Jan 2018 – Jun 2019 Cartographic quality and evaluation Geospatial Tutor, Humboldt State University, Jan 2016 – Dec 2017 Intern, Mapzen, Jun 2017 – Aug 2017 v GRANTS, AWARDS, AND HONORS: Rippey Graduate Student Grant, University of Oregon, 2021 Bill Loy Award for Excellence in Cartography, University of Oregon, 2020 Promising Scholar Award, University of Oregon, 2019 Student Map Competition Winner for Design, NACIS, 2017 David Lantis Scholarship, California Geographic Society, 2017 Tom McKnight Student Paper Award, 2017 Student Scholarship Program, NorCal URISA, 2017 PRESENTATIONS: Douglass, N. (2021). That’s a Relief: Assessing Landform Clarity and Aesthetic Preference in Terrain Maps. American Association of Geographers, Seattle, WA. April, 2021. Douglass, N. & Fish, C. (2020). Revamping an Introductory GIS Course for Online Delivery. North American Cartographic Information Society. October, 2021 Douglass, N. (2019). Green Trails Maps. North American Cartographic Information Society, Tacoma, WA. October, 2019. Douglass, N. (2018). Mapping the Sierra Nevada, Snow Cover 2017. North American Cartographic Information Society, Norfolk, VA. October, 2018. Douglass, N. (2017). Mapping Arcata Neighborhoods and Perceptions. California Geographic Society, San Diego, CA. April, 2017. PUBLICATIONS: Douglass, N. & Perdue, N. (2018). Mapping Arcata Neighborhoods and Perceptions. California Geographer, Vol. 57, 109-138. vi ACKNOWLEDGMENTS I would like to first and foremost thank Dr. Carolyn Fish for her financial support, guidance, and encouragement in picking this research topic. One day after proposing yet another unsuccessful project idea, Carolyn said, “I don’t know much about you Nate, but I know you love making beautiful maps”. She constantly urged me to research something I was passionate about, so that’s exactly what I did! Special thanks are due to my committee members, Dr. Amy Lobben and Jim Meacham, whose feedback and suggestions were insightful and helpful. Additional thanks to Jim for his mentorship and wisdom as I pursue cartography as a career. To Cameron Mulder from the UO Libraries Data Services for helping me understand what statistical analysis to use. To Ty McCarthy for letting me test my survey on his students, and for being a great friend. Thank you to the Geography graduate student body, whose support and sense of community was invaluable while we navigated graduate school during the COVID-19 pandemic. Specifically, I’d like to extend a very special thanks to my fellow AGSG graduate representatives: Bill Limpisathian, Devin Lea, Nicole Merrill, and Sam McLaughlin. I want to thank Michala Garrison for her constant feedback, graphic input, and knowledge sharing. Thanks also to Stuart Steidle-Nix for helping me survive many late nights in Condon Hall. I must thank Claire Roth for being an incredible partner, a pillar of support, and reminding me to have fun and take breaks. Last, thank you to my father, Kevin Douglass, for instilling me with his courage, determination, and tenacity. You are dearly missed. vii To my family, who support and love me unconditionally. I’m so grateful for you. Aloha no! Na ohana, mahalo piha loa! An meine Familie, die mich bedingungslos unterstützt und liebt. Ich bin dankbar. viii TABLE OF CONTENTS Chapter Page I. INTRODUCTION ........................................................................................................... 1 II. BACKGORUND ............................................................................................................ 4 2.1 Shaded Relief ...................................................................................................... 4 2.1.1 Early History ................................................................................................... 4 2.1.2 Manual Relief Shading ................................................................................... 8 2.1.3 Analytical and Multidirectional Relief Shading ........................................... 10 2.1.4 Blender Relief Shading ................................................................................. 11 2.2 Thematic Terrain Layers ................................................................................... 13 2.2.1 Hypsometric Tinting ..................................................................................... 13 2.2.2 Landcover ..................................................................................................... 14 2.2.3 Orthographic Imagery ................................................................................... 15 2.3 Aesthetics .......................................................................................................... 16 2.3.1 Map Design, Aesthetics, and Perception ...................................................... 17 2.3.2 Terrain Maps and Aesthetics ........................................................................ 17 III. METHODOLOGY ..................................................................................................... 19 3.1 Participants ........................................................................................................ 19 3.2 Materials ........................................................................................................... 19 3.2.1 User Study Design ........................................................................................ 19 3.2.2 Stimuli ........................................................................................................... 24 3.2.3 Shaded Relief Development ......................................................................... 26 3.2.4 Thematic Terrain Layer Development .......................................................... 28 ix Chapter Page 3.3 Procedure .......................................................................................................... 29 IV. RESULTS ................................................................................................................... 31 4.1 User Study Participants ..................................................................................... 31 4.2 Descriptive Statistics ......................................................................................... 32 4.2.1 Overall Mean Comparison by Map ............................................................... 33 4.2.2 Overall Mean Comparison by Shaded Relief ............................................... 34 4.2.3 Overall Mean Comparison by Thematic Terrain Layer ................................ 34 4.3 Beauty and Realism Rating Tasks .................................................................... 35 4.3.1 Effects of Shaded Relief and Thematic Terrain Layers on Beauty Rating Scores 36 4.3.2 Effects of Shaded Relief and Thematic Terrain Layers on Realism Rating Scores 39 4.3.3 Correlation Between Beauty and Realism Scores ........................................ 41 4.4 Landform Clarity Rating Tasks ........................................................................ 43 4.4.1 Effects of Shaded Relief and Thematic Terrain Layers on Landform Clarity Scores: Wizard Island ............................................................................................... 43 4.4.2 Effects of Shaded Relief and Thematic Terrain Layers on Landform Clarity Scores: Mount Scott .................................................................................................. 46 4.4.3 Effects of Shaded Relief and Thematic Terrain Layers on Landform Clarity Scores: Grouse Hill ................................................................................................... 49 4.5 Geographic Familiarity ..................................................................................... 51 4.5.1 Combined Mean Rating Scores .................................................................... 52 x Chapter Page 4.5.2 Individual Sample Rating Scores .................................................................. 54 V. DISCUSSION .............................................................................................................. 56 5.1 Beauty and Realism Rating Tasks .................................................................... 56 5.2 Landform Clarity Rating Tasks ........................................................................ 59 5.3 Geographic Familiarity ..................................................................................... 63 5.4 Qualitative Findings from Open-ended Post-questionnaire .............................. 64 VI. CONCLUSION ........................................................................................................... 65 6.1 Overview of the Findings .................................................................................. 66 6.2 Limitations and Further Research ..................................................................... 67 6.3 Concluding Thoughts ........................................................................................ 69 APPENDICIES ……………………………………………………………………………………………………………. 74 A. INFORMED CONSENT ..................................................................................... 70 B. PRE-TEST QUESTIONNAIRE .......................................................................... 76 C. TUTORIAL .......................................................................................................... 78 D. USER STUDY ..................................................................................................... 89 E. POST-TEST QUESTIONNAIRE ...................................................................... 116 F. STIMULI CREATION ....................................................................................... 117 G. SUMMARY STATISTICS ................................................................................ 119 REFERENCES CITED ................................................................................................... 123 xi LIST OF FIGURES Figure Page 2.1. Early Mixtec depictions of mountains (digitally traced from Wood, 1995). ............... 5 2.2. Horizontal rows of hills from the 13th century (Peutinger, 2021). ............................... 5 2.3. Shaded hills from the 14th century (Waldseemüller, 1513). ....................................... 6 2.4. Late 17th century map using hachuring technique (Landesaufnahme, 1872). ............ 7 2.5. Oberengadin by Eduard Imhof (Imhof, 1958) ............................................................. 7 2.6. Manual shaded relief map of Rocky Mountain National Park, Colorado, USA by Bill von Allmen, U.S. National Park Service. Available at shadedreliefarchive.com. ...... 8 2.7. Azimuth (sun angle) and altitude (angle of illumination). Graphic by ESRI. ........... 10 2.8. Visual comparison of a standard single light source shaded relief (left) and multidirectional shaded relief (right) of Churfiisten, Switzerland. Created by author. ................................................................................................................................... 11 2.9. 3D perspective and 2D perspective relief rendered in Blender. Created by author. .. 12 2.10. Hypsometric tint colorizing low elevations with pale yellow color and lower elevations in darker pale green. Created by author. ................................................... 14 2.11. National Land Cover Dataset with default colors (left). Reclassified data with natural colors applied (right). Created by author. ...................................................... 15 2.12. Orthographic imagery gives readers a sense of realism in terrain maps. Imagery from Google. .............................................................................................................. 16 3.1. Tutorial maps of Churfisten, Switzerland using multidirectional relief (left), Blender relief (middle), and manual relief (right). .................................................................. 21 3.2. Aesthetic rating task for beauty and realism. ............................................................. 22 3.3. Landform clarity rating task for selected features ..................................................... 23 3.4. Crater Lake, OR, USA. Created by author. ............................................................... 24 3.5. Nine variations of Crater Lake, OR, USA created by combining three shaded relief maps (Manual relief, multidirectional relief, and Blender relief) with three thematic terrain layers (Hypsometric tint, landcover, orthoimagery). ..................................... 25 xii Figure Page 3.6. Three shaded relief map stimuli before incorporating thematic terrain layers. ......... 27 3.7. Three thematic terrain layers used for stimuli maps. ................................................. 29 4.1. Plotted results of the two-way ANOVA. Estimated marginal means for beauty rating scores. ........................................................................................................................ 37 4.2. Bar graphs and images of estimated marginal means for beauty ratings. .................. 37 4.3. Plotted results of the two-way ANOVA. Estimated marginal means for realism rating scores. ........................................................................................................................ 39 4.4. Bar graphs and images of estimated marginal means for realism ratings. ................. 40 4.5. Heatmap of all beauty and realism rating scores. ...................................................... 42 4.6. Plotted results of the two-way ANOVA. Estimated marginal means for Wizard Island landform clarity rating scores ......................................................................... 44 4.7. Bar graphs and images of estimated marginal means for Wizard Island landform clarity ratings. ............................................................................................................ 44 4.8. Plotted results of the two-way ANOVA. Estimated marginal means for Mount Scott landform clarity rating scores. ................................................................................... 47 4.9. Bar graphs and images of estimated marginal means for Mount Scott clarity rating scores. ........................................................................................................................ 47 4.10. Bar graphs and images of estimated marginal means for Grouse Hill clarity rating scores. ........................................................................................................................ 50 5.1. Comparison of estimated marginal means for beauty and realism rating scores by map. Scores were statistically similar between beauty and realism. ......................... 59 xiii LIST OF TABLES Table Page 4.1. Breakdown of participant's reported gender .............................................................. 31 4.2. Breakdown of participant’s highest education level .................................................. 32 4.3. Breakdown of participant’s reported knowledge of cartographic design .................. 32 4.4. Breakdown of participant’s reported familiarity with Crater Lake, OR, USA .......... 32 4.5. Combined mean rating scores for beauty, realism, and clarity. N = 855 (95 participants x 9 maps) ................................................................................................ 33 4.6. Mean rating scores by map type for beauty, realism, and clarity. Highest values shaded in the table ..................................................................................................... 34 4.7. Mean rating scores by shaded relief for beauty, realism, and landform clarity. Highest values shaded in the table ............................................................................. 34 4.8. Mean rating scores by thematic layer for beauty, realism, and landform clarity. Highest values shaded in the table ............................................................................. 35 4.9. Beauty results of the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on beauty ratings. ............................. 38 4.10. Beauty rating results of the Tukey’s HSD post-hoc test for the relationship of shaded relief. .............................................................................................................. 38 4.11. Beauty rating results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on. .......................................................................................... 38 4.12. Realism results for the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on realism ratings ............................. 40 4.13. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on realism ratings. ........................................................................................... 40 4.14. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on realism ratings. ........................................................................................... 41 4.15. Results of Spearman's rank-order correlation between beauty and realism rating scores. ........................................................................................................................ 42 4.16. Wizard Island results for the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on clarity ratings ........................ 45 xiv Table Page 4.17. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Wizard Island ................................................................ 45 4.18. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Wizard Island ................................................................ 46 4.19. Mount Scott results for the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on clarity ratings. ....................... 48 4.20. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Mount Scott ................................................................... 48 4.21. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Mount Scott ................................................................... 48 4.22. Grouse Hill results for the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on clarity ratings ............................... 50 4.23. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Grouse Hill .................................................................... 51 4.24. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Grouse Hill .................................................................... 51 4.25. No significance was found for the combined mean rating scores between each of the three groups. Independent Samples Mann-Whitney U Test summary ...................... 53 4.26. Overall mean rank comparison between groups of geographic familiarity ............. 53 4.27. Significance was found amongst four individual samples across the three familiarity questions. Independent Samples Mann-Whitney U Test summary ........................... 55 4.28. Significant samples and their mean rank comparison between groups of geographic familiarity. ................................................................................................................. 55 xv CHAPTER I INTRODUCTION Shaded relief, relief shading, and hillshading are terms often used interchangeably that refer to the cartographic technique of creating intuitive, aesthetic, and realistic configurations of topographic features on maps by mimicking the shadows cast by a light source. Initially created manually with airbrushed or sketched manual techniques, creating shaded relief is much easier today with digital tools. These algorithmic techniques have simplified the ability to create realistic relief in terrain maps. We commonly access these algorithms in geographic information systems (GIS), but novel methods are increasing in popularity some of which use 3D rendering software (Huffman, 2017). Such tools allow cartographers to create shaded relief that mimics the expressiveness and beauty of early manual shaded relief but is more easily reproducible and time efficient. Empirical research on relief shading has increased over the past decade, providing insight into new digital techniques to represent the topographic relief on maps (Marston and Jenny, 2015; Jenny et al., 2020; Jenny, 2021; Kennelly and Stewart, 2014), user perception of illumination angles (Billand and Çöltekin, 2017), and perceived effectiveness of relief shading techniques (Farmakis-Serebryakova and Hurni, 2020). However, studies that investigated user perceptions of relief often tested maps that only depicted a grey-toned hillshade, when, in fact, cartographers typically pair terrain maps with additional layers to mimic the natural environment (Imhof, 1982; Imus, 2012). 1 To understand which techniques are used by cartographers today, I examined maps from two volumes of the North American Cartographic Information Society (NACIS) Atlas of Design (NACIS, 2018; NACIS, 2020) and a collection of award- winning maps from the NACIS conference. Among these maps, I identified three shaded relief techniques that were the most popular: 1) manual shaded relief, 2) multidirectional hillshading, and 3) Blender generated relief. Traditional manual shaded relief is often hand-drawn or created with airbrushed techniques, providing more control over landform representation. Though it is the least commonly used today, manual relief has an appealing aesthetic quality and is still a relevant method amongst contemporary cartographers (e.g., Bell, 2018; Hauser, 2017). Multidirectional hillshading is an automated relief rendering technique that uses multiple light sources to better highlight landforms, that attempts to mimic manual relief lighting. Blender is a 3D rendering software that has recently gained popularity in the cartographic community for its ability to generate incredibly realistic shaded relief. While shaded relief is often the key element in representing the physical landscape in terrain maps, cartographers often pair shaded relief with additional terrain layers to provide context to the underlying landscape. Across the selection of maps, many included one of three frequently used thematic terrain layers 1) hypsometric tinting, 2) landcover, or 3) orthoimagery. Hypsometric tinting helps reinforce changes in elevation by colorizing elevation ranges using a continuous or classed color scheme. Landcover is a discrete dataset that provides descriptive context to land surface characteristics and can add texture and a naturalistic aesthetic to the map. Orthoimagery is an aerial photo or 2 satellite image and can provide instant recognition of place, the landforms, and the texture of the underlying topography (Peterson, 2012; 2020). The frequent combination of shaded relief and thematic terrain layers suggests that the cartographer’s choice in hillshading is not the sole contributor to a successful terrain map, but an interaction between shaded relief and thematic terrain layers to create beauty, realism, and clarity of topographic features. Cartography is as much an art as it is a science (Cosgrove, 2005), and we often leave the artistry to trial and error on part of the cartographer. As a result, mapmakers must experiment with the many. The task is even more difficult today when there are a myriad of tools to render shaded relief. Cartographers have long touted the supremacy of manual shaded relief techniques, leaving some modern cartographers to wonder if the analytical methods are good enough. Thus, this research examines the use of both traditional and modern shaded relief techniques when combined with thematic terrain layers, to evaluate map reader perception regarding beauty, realism, and landform clarity. The following research question drives this study: How do manual and analytical shaded relief techniques influence reader perceptions of beauty, realism, and landform clarity in terrain maps that incorporate hypsometric tinting, landcover, and orthoimagery? 3 CHAPTER II BACKGROUND AND LITERATURE REVIEW This chapter reviews the previous research on terrain representation in cartography. In this thesis I define a terrain map as any map that combines a shaded relief layer and a thematic terrain layer. Section 2.1 defines shaded relief, gives a brief history of its evolution, and explains the difference between manual, multidirectional, and Blender generated relief. Section 2.2 defines the use of thematic terrain layers in terrain mapping. Finally, Section 2.3 inspects arguments made in cartography regarding aesthetics and map design in terrain mapping. 2.1 Shaded Relief Shaded relief is the cartographic technique of representing landforms and changes in topography in an intuitive and aesthetically pleasing way. A landform is any geomorphic feature on earth that appears on a map (hills, islands, plains, plateaus, valleys, mountains, etc.). Initially, shaded relief was created manually with an airbrush or other hand drawn techniques by master cartographers. Today, many analytical and digital tools exist to create shaded relief. This section briefly explains the history of the art and details some modern techniques developed since the early origins. 2.1.1 Early History The techniques used to represent mountains, hills, valleys, and other landforms on maps have evolved throughout human history. During this time there has been a transition from oblique representations to the contemporary areal perspectives. Some of 4 the earliest maps that depict mountains are from 2400 BC by early Mixtec peoples of pre- Hispanic Mexico (Figure 2.1). Such depictions represented hills as ornate symbols imbued with power and wealth for those who lived upon them (Wood, 1995). Figure 2.1. Early Mixtec depictions of mountains (digitally traced from Wood, 1995). Such representations used an oblique perspective, where instead of depicting the topography from an angle directly above, these maps illustrated the terrain from a side angle. 13th century European maps represented mountain ranges as clusters and strings of hills. This representation often depicted geologic landforms on a plane going east to west and with a regularly repeating dome shaped pattern (Figure 2.2). Figure 2.2. Horizontal rows of hills from the 13th century (Peutinger, 2021). 5 The 14th century maps look almost identical, but the major difference was the varied size and shape of mountains and hills (Figure 2.3). Cartographers also incorporated shading on the shadowed side of the mountain, which brings a sense of dimensionality to the drawing. Figure 2.3. Shaded hills from the 14th century (Waldseemüller, 1513). The late 17th century marked a shift away from oblique symbols to planimetric areal perspectives that are commonly seen in maps today. Hachuring techniques incorporated short, disconnected lines drawn on the hill's slope direction (Figure 2.4). The Industrial Revolution, which spurred mining efforts and required the claiming, naming, and climbing of the land, required more precise methods for representing changes in topography. Cartographers of this time relied heavily on hachuring, shading, and contour lines to represent elevation change more precisely (Fryer, 1958). 6 Figure 2.4. Late 17th century map using hachuring technique (Landesaufnahme, 1872). European cartographers of the early 20th century refined map making techniques and created more naturalistic, aesthetic, and expressive maps. Cartographers such as Eduard Imhof, Tibor Toth, and Fridolin Becker incorporated rock/scree drawings into maps and used colors that would mimic the natural environment. One of the major accomplishments of these cartographers was their development of manual shaded relief, which gave more dimensionality to terrain maps and accurately represented landforms using localized light sources (Figure 2.5). Figure 2.5. Oberengadin by Eduard Imhof (Imhof, 1958). 7 2.1.2 Manual Relief Shading In the same way that master painters depict shadows on their subjects from a light source, cartographers used manual shaded relief through airbrushed techniques to represent shaded areas on the map’s landscape (Figure 2.6). This technique was revolutionary in terrain mapping and is an artistically engaging method for representing topography. Figure 2.6. Manual shaded relief map of Rocky Mountain National Park, Colorado, USA by Bill von Allmen, U.S. National Park Service. Available at shadedreliefarchive.com. Manual shaded relief offers many benefits to cartographers when depicting terrain. One benefit is the control in which the artist has in representing landforms by locally adjusting the illumination of features based on its spatial orientation (Marston and Jenny, 2015). The cartographer creates modulated light and shadows that gave a vivid 3D effect on a 2D surface and enhance prominent features at their discretion. The result is a map that visualizes the topography in both an artistic and a realistic fashion (Imhof, 1982; Brassel, 1974; Collier et al., 2003). 8 There are, however, some limitations to manual shaded relief, including time, generalization, and reproducibility. Creating a shaded relief map by hand is extremely time intensive. Cartographers can take a month or longer to finish these maps, depending on the desired level of detail. Since perfecting the artistry of manual shaded relief is painstaking and time-consuming, generalization was common among many manual drawings (Patterson, 2015). Reproducibility is also a major limitation of manual shaded relief, since any two cartographers are likely to draw very different representations of the same location because of their own personal style, skill level, and interpretation of the landscape. This creates a fundamental issue in producing relief that represents landforms with equal levels of detail and clarity. Despite the time intensiveness of manual relief, it is still an accessible means of adding shaded relief to maps, and there is a reinvigorated interest in exploring manual techniques amongst contemporary cartographers. Bell (2018) created a time-lapse video of a shaded relief drawing done completely by hand and shared an in-depth tutorial of her process (https://www.sarahbellmaps.com/drawing-color-hillshade-a-tutorial-with-time- lapse-videos/). If creating a manual relief on your own is still too time-consuming, Patterson and Jenny (2012) created a publicly available repository of shaded relief maps depicting a wide variety of geographies (http://www.shadedreliefarchive.com/). Researchers have also attempting to replicate the aesthetic quality of manual relief through algorithmic processes, allowing the style to be more available to digital map makers (Jenny et al., 2020). 9 2.1.3 Analytical and Multidirectional Relief Shading Analytical methods provide a more efficient and consistent method for representing terrain. Analytical relief shading refers to the algorithmic process of creating a grey-scale representation of a terrain surface with an artificial light source. The algorithms derive the relief from a Digital Elevation Model (DEM) given a relative sun angle and altitude (Figure 2.7). Figure 2.7. Azimuth (sun angle) and altitude (angle of illumination). Graphic by ESRI. Multidirectional relief provides a solution to some of the common limitations by standard analytical hillshade algorithms. Since a standard hillshade model uses a global light source, all landforms are treated equally regardless of the direction from which the shadow is being cast. A major challenge with analytical shading is the lack of clarity of minor landforms, especially within the shadowed slope of a larger landform (Zakšek et al., 2011). Some remedies include adjusting the light angle so that it better captures most landforms (Biland and Çöltekin, 2017). However, multidirectional relief provides a solution to the aforementioned disadvantages with less effort than manual techniques. By casting multiple virtual light sources from different sun angle directions, multidirectional 10 relief provides more structure and clarity to landforms, especially minor landform features that fall within the shaded areas (Loissios et al. 2021). Figure 2.8 provides an example of a standard shaded relief compared to a multidirectional relief. Figure 2.8. Visual comparison of a standard single light source shaded relief (left) and multidirectional shaded relief (right) of Churfiisten, Switzerland. Created by author. 2.1.4 Blender Relief Shading Blender (version 2.92, Community, B.O., 2018) is an open-sourced 3D rendering software that has become increasingly popular for creating both 3D and 2D perspective relief maps (Figure 2.9). This method has gained some publicity for its novel approach and highly realistic aesthetic appeal. Blender can render complex shapes and create a variety of surface textures, which makes it a unique tool for depicting topographic relief. Much like other analytical methods, Blender can render relief shading from Digital Elevation Models (DEMs) and produce a grey scale relief hillshade using a given light source. The major difference is that Blender offers more complex texture 11 rendering capabilities and light source adjustments. For example, rather than shaded areas in a hillshade being calculated by a single light source, Blender incorporates shaders which alter the way light interacts with the adjacent surfaces. The algorithm mimics natural lighting and is designed to achieve a more realistic 3D rendering. Blender has recently become a popular tool in cartography for creating shaded relief maps and is an accessible option for those who lack in-depth knowledge of 3D rendering. The software has undergone significant improvements and is great for introductory users. A handful of cartographers have also openly shared tools, tutorials, and blogs about their process, making Blender generated relief more reproducible and readily available (Huffman, 2017; Powell, 2016; Underwood, 2019, Larsen, 2019; Atwood, 2020). Due to this, Blender is being used more often for cartographic purposes with many people being attracted to the aesthetic qualities the software has to offer. Figure 2.9. 3D perspective and 2D perspective relief rendered in Blender. Created by author. 12 2.2 Thematic Terrain Layers While shaded relief is often the key element in representing the physical landscape in terrain maps, relief maps are typically paired with additional terrain layers to provide context to the underlying landscape (Imhof, 1982; Imus and Loftin, 2012). Imhof (1982) refers to some of these layers as “abstractions” that provide a corresponding link to elevation change such as hypsometric tinting, hachuring, and contour lines. While abstraction layers don’t necessarily mimic the natural environment in terms of visual depiction, other layers, such as landcover and orthoimagery, can represent variation in vegetation and surface cover. In this thesis, I define thematic terrain layers as a single raster layer on a map used to aid in representing the surface elevation, texture, or landcover. The sections below review three thematic terrain layers commonly used in terrain mapping: hypsometric tinting, landcover, and orthoimagery. 2.2.1 Hypsometric Tinting Hypsometric tinting, also known as the color stereoscopic effect, is a method of colorizing elevation values using a continuous or classed single hue, multi-hue, or spectral color scheme (Figure 2.10). This combined with shaded relief reinforces elevation heights through color cues (Eyton, 1990). Two major limitations have been discussed regarding hypsometric tinting in the past decade. The first limitation is that using an evenly stretched scheme on small scale global world maps causes a perceived loss of lowland and highland detail, which can be remedied by creating locally enhanced hypsometric tinting (Huffman and Patterson, 2013). The second limitation is the confusion that the colors applied to the tinting directly represent the “environmental phenomena such as vegetation, landcover, or climate” (Patton and Crawford, 1977; 13 Patterson and Jenny, 2011). While this issue is one of user perception, Imhof (1982) argues that to remedy any misunderstanding, the colors in elevation tinting should reflect hues one might see in the natural landscape. Figure 2.10. Hypsometric tint colorizing low elevations with pale yellow color and lower elevations in darker pale green. Created by author. 2.2.2 Landcover Landcover offers a method of representing the landscape and has the potential to offer a natural aesthetic. Landcover is a discrete dataset that provides descriptive context to land surface characteristics and is commonly used by cartographers to create realistic maps by incorporating real data about the landscape and representing those landscapes with natural colors (Patterson, 2015). The National Land Cover Dataset (NLCD) is a satellite derived raster dataset that provides a landcover classification for the United States. While the default colors used in the NLCD are intuitive, they may not always be 14 suitable, depending on the preference of the cartographer and the goal of the map. Applying a different color pallet to landcover data can aid in reinforcing subtle changes in the landscape and offer a natural aesthetic (Figure 2.11). Figure 2.11. National Land Cover Dataset with default colors (left). Reclassified data with natural colors applied (right). Created by author. 2.2.3 Orthographic Imagery Orthographic imagery is an aerial photo or satellite image that has been geometrically corrected to accurately represent the Earth's surface. When used in maps, orthoimagery provides context and can make maps easier for general map audiences to interpret. Much like shaded relief, orthographic imagery allows for instant recognition of place, the landforms, and the texture of the underlying topography (Peterson, 2012; 2020). The depiction gives map readers the perspective as if they are in an airplane overhead and is the least abstract of the thematic terrain layers (Figure 2.12). Hoarau and Sidonie (2017) improved image-based representation using imagery to enhance photo- 15 realism perception in maps. They state that the use of orthoimagery in maps adds realism and context to the representation (Hoarau and Sidonie, 2017). While orthoimagery provides many benefits, other research has shown that when paired with shaded relief, orthoimagery can negatively affect landform perception (Çöltekin and Biland, 2019). Figure 2.12. Orthographic imagery gives readers a sense of realism in terrain maps. Imagery from Google. 2.3 Aesthetics The very nature of relief representation is a devotion to artistic qualities and requires careful thought of the aesthetic representation (Imhof, 1982). Cartographers highly debate aesthetics, despite close ties to art and visual representation. As Kent et al. (2012) states “opinions are strong and varied [regarding aesthetics] and there are no universal rules, even though when we say a map is ‘beautiful’ we believe others ought to agree with us”. Regardless of the data conveyed and message being shared, the map must 16 appeal in its representation (Field & Demaj, 2012). While learning a set of rules and conventions for representing map features is conceivable, a cartographer’s goal with a map is to not only be informationally effective but also aesthetically pleasing (Dent, 2008). 2.3.1 Map Design, Aesthetics, and Perception In the last decade, a handful of researchers have attempted to quantify the aesthetic response of map readers on several fronts. Limpisathian (2017) tested the visual contrast of maps at multiple scales and asked map readers to rank a series of color and contrast schemes based on their clarity and aesthetic qualities. Similarly, Fabrikant et al. (2012) tested a small group of map reader’s arousal levels while reading several design iterations of the same map to test aesthetic preferences. Cartographers have also examined the micro-aesthetics in map typefaces (Guidero, 2016). Contemporary cartographic researchers are answering a call to bring aesthetics to the center of cartographic theory and critique the factors that influence aesthetic decision making in cartography (Kent, 2005). 2.3.2 Terrain Maps and Aesthetics The success of a terrain map is not dependent purely on how efficiently it conveys the information, but also on how it looks aesthetically. The function of the map and its graphical appearance are intertwined, and the visual effect of a map is constructed from the interplay of its elements (Kent et al., 2012). Creating an effective terrain map takes time, artistry, and aesthetic sensitivity (Imhof, 1982). While mapmakers today have many digital tools to create shaded relief quickly and easily, many terrain cartographers refine, 17 adjust, and perfect the relief in post processing software (Patterson, 1997; 2002; Tait, 2002; Jenny and Patterson, 2007; Imus, 2012). For example, by incorporating orthoimagery in terrain maps, the cartographer can achieve an appealing design aesthetic that adds complexity, texture, and realism to the representation (Raposo and Brewer, 2014). Some researchers have conducted studies that test terrain maps to understand the design and aesthetic preferences (Raposo and Brewer, 2014; Jenny et al., 2020). However, there is still minimal empirical research attempting to classify the aesthetic qualities between digital and manual shaded relief techniques. 18 CHAPTER III METHODOLOGY To answer my research question, I designed a user study to examine: 1) beauty, 2) realism, and 3) landform clarity across a set of maps with variations in shaded relief and thematic terrain layers. The following chapter outlines the participants, materials, and procedures in a user study created to assess aesthetic preference and to landform clarity in terrain maps. Section 3.1 describes my study participants. Section 3.2 discusses the user study design and the creation of the stimuli. Lastly, section 3.3 details the procedure of the user study. 3.1 Participants I solicited 105 participants for the study from the recruitment site Prolific. Participants were only able to participate if they were using a desktop computer and their Prolific profile indicated they were 18 years of age or older, a U.S. resident, and fluent in English. Each participant was paid $4.97 USD through Prolific for 15 minutes of their time after they completed the user study, and I approved their answers. 3.2 Materials 3.2.1 User Study Design The user study consisted of 6 sections: 1) finding the study on Prolific, 2) being redirected to Qualtrics and to the informed consent form, 3) a pre-test questionnaire, 4) a tutorial, 5) the main user study, and 6) a final feedback question. 19 3.2.1.1 Finding the Study on Prolific The user study was posted on Prolific. Crowdsourced survey design is a popular route for perceptual studies that measure affective visualization design using graphic stimuli (Mylavarapu et al., 2019; Bartram et al., 2017; Lin et al., 2013; Heer and Bostock, 2010). There are many advantages to using crowdsourced surveying platforms like Prolific and Amazon Mechanical Turk (MTurk) including access to willing participants and quick response times. Prolific requires users to maintain a profile and answer general demographics questions to participate. 3.2.1.2 Qualtrics and Informed Consent The Prolific listing of this user study redirected participants to a Qualtrics study where the first question asked participants to enter their Prolific ID to cross-reference with their Prolific profile information to receive compensation. The second page of the Qualtrics study provided the participants with the consent form (APPENDIX A). 3.2.1.3 Pre-test Questionnaire The pre-test questionnaire consisted of six questions: Two demographics questions on gender and education level, a question about their knowledge of cartography and map design, and three questions about their familiarity with Crater Lake, Oregon which served as the geographic focus of the stimuli. The three geographic familiarity questions asked whether they had ever 1) heard of, 2) seen pictures or maps of, 3) or visited Crater Lake, Oregon (APPENDIX B). 20 3.2.1.4 Tutorial The tutorial section consisted of three maps of a different location than used in the main study and the five questions which participants would see during the main user study. Instead of Crater Lake, OR, participants were shown map designs of Churfisten, Switzerland (Figure 3.1) with just three variations (as opposed to the nine they would see in the main user study) of terrain and thematic layers. They were then asked the same five questions that I elaborate on in the next section (see APPENDIX C). Figure 3.1. Tutorial maps of Churfisten, Switzerland using multidirectional relief (left), Blender relief (middle), and manual relief (right). 3.2.1.5 Main User Study and Stimuli The main user study consisted of a total of 45 questions (five questions for each of the nine stimuli maps). The nine maps were a combination of three shaded relief methods (manual shaded relief, multidirectional hillshade, and Blender shaded relief) and three thematic terrain layers (hypsometric tint, landcover, and orthoimagery). I detail the design of the nine maps in Section 3.2.2. The five questions the participants answered were shown to participants in two sets. The first set of questions asked the participants to look at one of the nine maps 21 without labels and rate the map on its 1) beauty and 2) realism on a five-point Likert scales (Figure 3.2). For the second set of questions participants were shown the same map, but with labels of physical geographic features. They were asked to rate the clarity of a selection of three landforms (Wizard Island, Mount Scott, and Grouse Hill) on a five- point scale from “Very unclear” to “Very clear” (Figure 3.3). See APPENDIX D for the full user study. Figure 3.2. Aesthetic rating task for beauty and realism. 22 Figure 3.3. Landform clarity rating task for selected features. 23 3.2.1.6 Post-test Questionnaire The final question of the user study asked the participants to “Please provide any comments or feedback on your experience while taking part in this study” to gain qualitative insights on the stimuli design and user experience, as well as to solicit overall feedback on the study design (APPENDIX E). 3.2.2 Stimuli The stimuli for the user study consisted of nine different terrain maps of Crater Lake, OR, USA. I chose Crater Lake as the location for the primary stimuli because of the variation in physical landforms, overall symmetry, and availability of both digital and manual relief (Figure 3.4). Figure 3.4. Crater Lake, OR, USA. Created by author. 24 The maps were made up of a combination of three shaded relief maps and three thematic terrain layers (Figure 3.5). The following sections give a general overview of the data collection, processing, development, and design of the stimuli maps. A more detailed breakdown of the data sources, design choices, and colors used in the stimuli development can be found in APPENDIX F. Figure 3.5. Nine variations of Crater Lake, OR, USA created by combining three shaded relief maps (Manual relief, multidirectional relief, and Blender relief) with three thematic terrain layers (Hypsometric tint, landcover, orthoimagery). 25 3.2.3 Shaded Relief Development In this section I detail how I created three shaded relief layers: 1) a manual shaded relief map, 2) a multidirectional shaded relief, and 3) a Blender shaded relief. I downloaded a manual relief map from the shaded relief archive (Patterson and Jenny, 2012) that was published by the National Park Service (Allmen, 1988). Allmen (1988) manually created the relief by referencing orthoimagery and contour lines, then drew the relief to incorporate both land surface terrain and bathymetry (Figure 3.6). I downloaded the manual relief as a non-geo-referenced TIFF, brought it into Adobe Photoshop (version 21.0.1, Adobe Inc., 2021), and manually aligned it to “best fit” the digital relief. The algorithmically generated shaded relief maps (multidirectional hillshade and Blender) were derived from a 3.33-meter resolution DEM from a set of sample elevation models provided by Kennelly et al. (2021). The DEM did not capture bathymetric elevations, so a subsequent 1-meter ASCII XYZ grid was downloaded from the USGS (Gardner and Dartnell, 2001). Since the manual shaded relief included bathymetry, this was a necessary requirement for my digital shaded relief maps. The bathymetric grid data was re-sampled to 3.33 meters and combined with the DEM to produce a complete elevation model that included bathymetric elevation. The multidirectional hillshade was created in QGIS (version 3.16.3-Hannover) using the GDAL hillshade tool (Figure 3.6). The following parameters were set for the hillshade tool: vertical exaggeration (Z factor) was set to 3.0, the azimuth was set to 26 337.5 as suggested by Biland and Çöltekin (2017), the altitude was set to 45.0 degrees, and I selected the multidirectional shading option. The Blender relief map was created with Blender version 2.82a following Huffman’s (2017) relief tutorial (Figure 3.6). Using Blender for relief modeling requires the DEM to be converted to a 16-bit unsigned integer, which I did using a GDAL Warp command (Larson, 2019). A challenge with Blender relief is capturing low elevation areas with similar detail, so I adjusted the color space from sRGB to Linear (Huffman, 2017). I set vertical exaggeration to 3.0 with a Midlevel of 0.50. I used the sun method for light sourcing and the angle used was 90 degrees. Finally, the surface displacement method was set to “Displacement and Bump” which combines both displacement and bump mapping option, this allows for larger amounts of displacement in the 3D model, and the bump option, which preserved finer details and textures in the rendering. Combining the two methods can provide a good balance and reduce memory usage (Blender Documentation Team, 2021). Last, I adjusted each shaded relief map in Photoshop to ensure equal levels of contrast and brightness. Figure 3.6. Three shaded relief map stimuli before incorporating thematic terrain layers. 27 3.2.4 Thematic Terrain Layer Development To create nine maps, the three shaded relief maps were combined with three thematic terrain layers: 1) hypsometric tinting, 2) landcover, and 3) orthoimagery (Figure 3.7). I detail how these were created in this section. The hypsometric tint layer was created from the DEM and used two separate color schemes, one for land surface elevations and a second color scheme for bathymetric elevations (Figure 3.7). Since hypsometric tinting does not show the terrain’s vegetation or landcover, a common practice is to mimic colors of the climatic region to avoid confusion for the map reader (Patterson and Jenny, 2011) which I did here. The colors specs used for the hypsometric tint layer can be found in APPENDIX F. The landcover layer used raster data collected from the National Land Cover Dataset (NLCD) to create a thematic landcover map of Crater Lake (Figure 3.7). Three landcover classifications were created for the map: 1) tree cover, 2) shrub/grass, and 3) water. The result was exported from QGIS as a TIFF and adjusted in Adobe Photoshop. The Select by Color Range tool was used to highlight each classification and separated into its own layer then colorized to create a realistic looking image of landcover. The orthographic image layer (Figure 3.7) was derived from Google Maps. The Google Map satellite basemap was loaded into QGIS and exported as a TIFF, then brought into Adobe Photoshop for image processing. A common issue when using orthographic imagery is that the angle of shadows cast from the time of capture may differ from the shaded relief shadows. This can cause over saturation in the map when the two layers are blended and result in a misrepresentation of landform features. To avoid 28 this problem, I used the content-aware fill option in Photoshop on areas of the orthoimage that appeared to be shaded so they did not conflict with the shaded relief shadows. Figure 3.7. Three thematic terrain layers used for stimuli maps. 3.3 Procedure A user study experiment was conducted and approved by the University of Oregon Institutional Review Board (STUDY00000080). Participants found the study through the Prolific recruitment site. Once they clicked on the study, they were redirected to a Qualtrics site to take part in the user study. The first page of the user study asked participants to enter their Prolific ID and next asked them to read through the consent form (APPENDIX A). If participants agreed, they would be moved on to the next sections of the user study. If they disagreed, the study would end, and they would not be compensated for their time. If they moved on, they were then presented with the pre-test questionnaire. Following the pre-test, participants navigated through the tutorial where they were introduced to the five questions they would answer in the main user study, as well as the concepts, and flow of the main user study. Following the tutorial section, 29 participants started the main user study. Participants answered the five questions for each of the nine randomized maps of Crater Lake, OR. Two attention check questions were presented to participants to ensure they were actively engaging in the survey. The first attention check question was presented after the tutorial section and the other was presented after they completed the user study. Finally, participants were given the opportunity to leave any comments or feedback in a final open-ended question. They were then redirected back to Prolific and compensated for their time once I approved their answers. 30 CHAPTER IV RESULTS In this chapter I review the results from the user study. Section 4.1 details a series of descriptive statistics. Section 4.2 reports on the results of a two-way ANOVA conducted on aesthetic rating preferences for beauty and realism. 4.3 shares the results of a two-way ANOVA for clarity ratings for each landform. Finally, section 4.4 details the results of a non-parametric Mann-Whitney test on the effects that geographic familiarity had on user rating scores. 4.1 User Study Participants Of the initial 105 respondents, I removed 8 responses because of incomplete answers and failed attention checks. I also removed two responses that reported they had expert knowledge of cartographic design, ultimately leaving 95 total responses for analysis. Table 4.1 and Table 4.2 report on the demographic characteristics of the study participants. Table 4.3 reports the breakdown of participant’s reported cartographic knowledge and Table 4.4 reports the breakdown of participant’s reported geographic familiarity with Crater Lake, OR. Table 4.1. Breakdown of participant's reported gender. N % Male 46 48.4% Gender Female 48 50.5% Non-Binary 1 1.1% 31 Table 4.2. Breakdown of participant’s highest education level. N % Less than Highschool 2 2.1% High school graduate 14 14.7% Education Some college 19 20.0% 2-year degree 5 5.3% 4-year degree 36 37.9% P rofessional degree 2 2.1% Master's 5 5.3% Doctorate 12 12.6% Table 4.3. Breakdown of participant’s reported knowledge of cartographic design. N % Very knowledgeable 3 3.1% Cartographic Moderately knowledgeable 25 26.3% Knowledge Slightly knowledgeable 44 46.3% Not knowledgeable at all 23 24.2% Table 4.4. Breakdown of participant’s reported familiarity with Crater Lake, OR, USA. 2.) “Have you 1.) “Have you ever heard of Crater ever seen photos 3.) “Have you ever visited Crater Lake?” or maps of Crater Lake?” Lake?” Yes No Yes No Yes No N 61 34 40 55 10 85 % 64.2% 35.8% 42.1% 57.9% 10.5% 89.5% 4.2 Descriptive Statistics I used the SPSS Statistics software (version 27) to process and analyze the results of the user study. First, I filtered the raw data and restructured it in SPSS to ensure proper grouping of the variables. Each dependent variable (beauty, realism, landform clarity) and independent variable (shaded relief, and thematic layer) was explored in SPSS for 32 normality and equality of variance before conducting the analysis. You can find a detailed list of mean distributions, standard deviation, and standard error in Appendix G. The following section details the mean descriptive statistic results for each rating task categorized by map, shaded relief, and thematic layer. 4.2.1 Overall Mean Comparison by Map Since each participant (N = 95) responded to all the stimuli maps (N = 9) in random order, the final entries equaled the multiplication of the two (N = 855) which would then serve as the final N value for analysis. Table 4.5 shows the descriptive mean, standard deviation, minimum and maximum values for each rating task. Table 4.6 shows that the map using Blender generated relief combined with orthographic imagery had the highest mean score for both beauty and realism. The map that used a multidirectional hillshade combined with landcover had the highest mean score for landform clarity. Lastly, the map that used a multidirectional hillshade combined with orthoimagery had the highest mean score for the total average beauty, realism, and landform clarity scores (Table 4.6). Table 4.5. Combined mean rating scores for beauty, realism, and clarity. N = 855 (95 participants x 9 maps). N Mean Std. Minimum Maximum Deviation Beauty 855 3.18 1.099 1 5 Realism 855 3.24 1.130 1 5 Clarity: Wizard Island 855 3.90 1.036 1 5 Clarity: Mount Scott 855 3.74 1.087 1 5 Clarity: Grouse Hill 855 3.62 1.142 1 5 33 Table 4.6. Mean rating scores by map type for beauty, realism, and clarity. Highest values shaded in the table. Landform Clarity Aesthetic Rating Rating Thematic Terrain Wizard Mount Grouse Shaded Relief Layer Beauty Realism Island Scott Hill Total Manual Hypsometric Tint 2.41 2.29 3.22 3.05 3.00 2.794 Multidirectional Hypsometric Tint 3.06 2.98 3.96 3.86 4.06 3.584 Blender Hypsometric Tint 2.81 2.69 3.76 3.59 3.66 3.302 Manual Landcover 2.68 2.79 3.53 3.43 3.11 3.108 Multidirectional Landcover 3.15 3.33 4.34 4.15 4.18 3.83 Blender Landcover 3.01 3.17 3.94 3.94 3.56 3.524 Manual Orthoimagery 3.72 3.84 4.05 3.76 3.46 3.766 Multidirectional Orthoimagery 3.83 3.97 4.26 3.91 3.88 3.97 Blender Orthoimagery 3.92 4.07 4.06 3.95 3.68 3.936 4.2.2 Overall Mean Comparison by Shaded Relief Table 4.7 shows the descriptive mean values for each rating score by shaded relief. The results of the scores show that the maps using a multidirectional hillshade scored highest in beauty, realism, and landform clarity. Table 4.7. Mean rating scores by shaded relief for beauty, realism, and landform clarity. Highest values shaded in the table. Aesthetic Rating Landform Clarity Rating Shaded Relief Beauty Realism Wizard Mount Grouse Island Scott Hill Manual 2.94 2.98 3.60 3.41 3.19 Multidirectional 3.35 3.42 4.19 3.97 4.04 Blender 3.25 3.31 3.92 3.82 3.64 4.2.3 Overall Mean Comparison by Thematic Terrain Layer Table 4.8 shows the descriptive mean values for each rating score by thematic layer. The results show that the maps using an orthographic image scored highest in beauty, realism, as well as landform clarity. 34 Table 4.8. Mean rating scores by thematic layer for beauty, realism, and landform clarity. Highest values shaded in the table. Aesthetic Rating Landform Clarity Rating Thematic Layer Beauty Realism Wizard Mount Grouse Island Scott Hill Hypsometric Tint 2.76 2.66 3.65 3.50 3.58 Landcover 2.95 3.09 3.93 3.84 3.61 Orthoimagery 3.82 3.96 4.13 3.87 3.68 4.3 Beauty and Realism Rating Tasks I used a two-way ANOVA to test the significance of each independent variable (shaded relief and thematic terrain layers) and its effect on the dependent variables (beauty and realism rantings) to answer my research question. How do manual and analytical shaded relief techniques influence reader perceptions of beauty, realism, and landform clarity in terrain maps that incorporate hypsometric tinting, landcover, and orthoimagery? The two-way ANOVA also determines if there is a significant interaction effect between the two independent variables (shaded relief and thematic terrain layers) and if that affected the dependent variable scores. This interaction effect makes the two-way ANOVA a more robust and valuable analysis for studies with two categorical independent variables (Norušis, 2008). Essentially, an interaction occurs when “the effect of one independent variable is not the same for all levels of the other independent variable.” (Rahman, 2019). Determining approximate normal distribution is crucial before a researcher decides what statistical analysis to use (Kent, 2001). I ran a Shapiro-Wilk’s test (p > .05) for each set of variables. The result (p < .0001) suggested that there was not a normal 35 distribution for some samples and could affect results from tests such as ANOVA (Norušis, 2008). However, after calculating both skewness and kurtosis z-values, as well as a visual inspection of each Q-Q plot and histogram, I determined there was in fact an approximate normal distribution for most of the samples within each set of variables and warrants the use of a parametric ANOVA. I also conducted a Lavine’s test to determine homogeneity of variances (p < .0001) and violated another assumption of the ANOVA. I converted the data to a logarithmic scale and still did not remedy the violation. To test the validity of the ANOVA results, I ran non-parametric Welsh and Brown-Forsythe tests for each set that failed the Shapiro-Wilk’s test and Levine’s test. This allowed me to double check the significance values from the ANOVA. 4.3.1 Effects of Shaded Relief and Thematic Terrain Layers on Beauty Rating Scores Figure 4.1 and Figure 4.2 show the estimated marginal means plotted for perceived beauty for each thematic layer across the three shaded relief techniques. The results of a two-way ANOVA (Table 4.9) showed a significant main effect for both shaded relief (F(2,4) = 13.495, p < .001), and thematic layer (F(2,4) = 94.520, p < .001), but no significant interaction between the two (F(2,4) = 1.799, p = .115) on beauty rating scores. A Tukey HSD post-hoc test showed that multidirectional relief and manual relief differed for beauty (p < .001); manual relief differed significantly from the other two shaded relief (p < .001), but multidirectional relief and Blender relief were not significantly different (Table 4.10). The HSD post-hoc test for thematic terrain layers found significant effect for all thematic terrain layers and their comparisons (p < .001) except for the hypsometric tint when compared to landcover (Table 4.11) on the beauty scores. 36 Figure 4.1. Plotted results of the two-way ANOVA. Estimated marginal means for beauty rating scores. Figure 4.2. Bar graphs and images of estimated marginal means for beauty ratings. 37 Table 4.9. Beauty results of the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on beauty ratings. Type III Sum of Mean Eta Variable Squares df Square F Sig. Squared Shaded Relief 26.051 2 13.026 13.495 .000* .031 Thematic Layer 182.473 2 91.236 94.520 .000* .183 Shaded Relief ‡ Thematic Layer 7.198 4 1.799 1.864 .115 .009 Table 4.10. Beauty rating results of the Tukey’s HSD post-hoc test for the relationship of shaded relief. 95% Confidence Mean Interval (I) Shaded (J) Shaded Difference Std. Lower Upper Relief Relief (I-J) Error Sig. Bound Bound Manual Multidirectional -.41* .082 .000* -.60 -.22 Blender -.31* .082 .001* -.50 -.12 Multidirectional Manual .41* .082 .000* .22 .60 Blender .10 .082 .432 -.09 .29 Blender Manual .31* .082 .001* .12 .50 Multidirectional -.10 .082 .432 -.29 .09 Based on observed means. The error term is Mean Square (Error) = .965. *. The mean difference is significant at the .05 level. Table 4.11. Beauty rating results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on. 95% Confidence Mean Interval (I) Thematic (J) Thematic Differenc Std. Lower Upper Terrain Layer Terrain Layer e (I-J) Error Sig. Bound Bound Hypsometric Tint Landcover -.19 .082 .062 -.38 -.22 Orthoimagery -1.06* .082 .000* -1.25 -.12 Landcover Hypsometric Tint .19 .082 .062 -.01 .60 Orthoimagery -.87* .082 .000* -1.07 .29 Orthoimagery Hypsometric Tint 1.06* .082 .000* .87 .50 Landcover .87* .082 .000* .68 .09 Based on observed means. The error term is Mean Square (Error) = .965. *. The mean difference is significant at the .05 level. 38 4.3.2 Effects of Shaded Relief and Thematic Terrain Layers on Realism Rating Scores Figure 4.3 and Figure 4.4 show the estimated marginal means plotted for perceived realism for each thematic layer across the three shaded relief techniques. The results of a two-way ANOVA (Table 4.12) showed a significant main effect for both shaded relief (F(2,4) = 16.46, p < .001), and thematic layer (F(2,4) = 132.996, p < .001), but no significant interaction between the two (F(2,4) = 2.202, p = .067) on realism rating scores. Figure 4.3. Plotted results of the two-way ANOVA. Estimated marginal means for realism rating scores. 39 Figure 4.4. Bar graphs and images of estimated marginal means for realism ratings. A Tukey HSD post-hoc test showed that multidirectional relief and manual relief differed (p < .001); manual relief differed significantly from the other two shaded relief (p < .001), but multidirectional relief and Blender relief were not significantly different for realism ratings (Table 4.13). The HSD post-hoc test for thematic terrain layers found a significant effect for all thematic terrain layers and their comparisons (p < .001) for hypsometric tint, landcover, and orthoimagery (Table 4.14) for realism. Table 4.12. Realism results for the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on realism ratings Type III Sum of Mean Eta Variable Squares df Square F Sig. Squared Shaded Relief 31.139 2 15.570 16.468 .000* .037 Thematic Layer 251.483 2 125.742 132.996 .000* .239 Shaded Relief ‡ Thematic Layer 8.327 4 2.082 2.202 .067 .010 40 Table 4.13. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on realism ratings. 95% Confidence Mean Interval (I) Shaded (J) Shaded Difference Std. Lower Lower Relief Relief (I-J) Error Sig. Bound Bound Manual Multidirectional -.45* .081 .000* -.64 -.64 Blender -.34* .081 .000* -.53 -.53 Multidirectional Manual .45* .081 .000* .26 .26 Blender .11 .081 .353 -.08 -.08 Blender Manual .34* .081 .000* .15 .15 Multidirectional -.11 .081 .353 -.30 -.30 Based on observed means. The error term is Mean Square(Error) = .945. *. The mean difference is significant at the .05 level. Table 4.14. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on realism ratings. 95% Confidence Mean Interval (I) Thematic (J) Thematic Differenc Std. Lower Upper Terrain Layer Terrain Layer e (I-J) Error Sig. Bound Bound Hypsometric Tint Landcover -.44* .081 .000* -.63 --- Orthoimagery -1.31* .081 .000* -1.50 -.87 Landcover Hypsometric Tint .44* .081 .000* .25 -- Orthoimagery -.87* .081 .000* -1.06 -.68 Orthoimagery Hypsometric Tint 1.31* .081 .000* 1.11 --- Landcover .87* .081 .000* .68 1.07 Based on observed means. The error term is Mean Square(Error) = .945. *. The mean difference is significant at the .05 level. 4.3.3 Correlation Between Beauty and Realism Scores Figure 4.5 charts the frequency of beauty and realism scores for each rank-order value. The figure shows a positive relationship between beauty and realism scores across the five-point scale for each rating task and indicated that map readers found the 41 most/least beautiful maps were also the most/least realistic. To confirm the relationship between beauty and realism scores, I ran a Spearman's rank-order correlation. The results of the Spearman’s correlation (Table 4.15) showed a positive correlation between beauty and realism rating scores, which was statistically significant (rs(8) = .653, p < .001). Figure 4.5. Heatmap of all beauty and realism rating scores. Table 4.15. Results of Spearman's rank-order correlation between beauty and realism rating scores. Beauty Realism Correlation Coefficient 1 .653 Beauty Sig. (2-tailed) .000** Spearma n's rho N 855 855 Correlation Coefficient .653 1 Realism Sig. (2-tailed) .000** N 855 855 **. Correlation is significant at the 0.01 level (2-tailed). 42 4.4 Landform Clarity Rating Tasks I used a two-way ANOVA to test the significance of each independent variable (shaded relief and thematic terrain layers) and its effect on the dependent variables (landform clarity for Wizard Island, Mount Scott, and Grouse Hill) to answer my research question. The two-way ANOVA also determines if there is a significant interaction effect between the two independent variables (shaded relief and thematic terrain layers) and if that affected the dependent variable scores. This interaction effect makes the two-way ANOVA a more robust and valuable analysis for studies with two categorical independent variables (Norušis, 2008). Essentially, an interaction occurs when “the effect of one independent variable is not the same for all levels of the other independent variable.” (Rahman, 2019). If an interaction effect is detected by the analysis, it is ideal to then run separate one-way ANOVA tests on the independent variables. I used the non-parametric Kruskal–Wallis one-way ANOVA to cross examine the results when there was a significant interaction effect from the two-way ANOVA. 4.4.1 Effects of Shaded Relief and Thematic Terrain Layers on Landform Clarity Scores: Wizard Island Figure 4.6 and Figure 4.7 show the estimated marginal means plotted for perceived clarity of Wizard Island for each shaded relief across the three thematic layers. For Wizard Island, the results of a two-way ANOVA (Table 4.16) showed a significant main effect for both shaded relief (F(2,4) = 25.28, p < .001), and thematic layer (F(2,4) = 17.187, p < .001), and a significant interaction between the two (F(2,4) = 3.196, p = .013) on clarity rating scores. 43 Figure 4.6. Plotted results of the two-way ANOVA. Estimated marginal means for Wizard Island landform clarity rating scores. Figure 4.7. Bar graphs and images of estimated marginal means for Wizard Island landform clarity ratings. 44 A Tukey HSD post-hoc test showed that multidirectional relief and Blender relief differed (p = .004); manual relief differed significantly from the other two shaded relief (p < .001) (Table 4.17). The HSD post-hoc test for thematic terrain layers found significant effects for hypsometric tint and landcover (p = .001); hypsometric tint and orthoimagery were significantly different (p < .001); but orthoimagery and landcover were not significantly different (p = .051) (Table 4.16) Table 4.16. Wizard Island results for the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on clarity ratings. Type III Sum of Mean Eta Variable Squares df Square F Sig. Squared Shaded Relief 49.060 2 24.530 25.279 .000* .056 Thematic Layer 33.354 2 16.677 17.187 .000* .039 Shaded Relief ‡ Thematic Layer 12.407 4 3.102 3.196 .013* .015 Table 4.17. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Wizard Island. 95% Confidence Mean Interval (I) Shaded (J) Shaded Difference Std. Lower Upper Relief Relief (I-J) Error Sig. Bound Bound Manual Multidirectional -.59* .083 .000* -.78 --- Blender -.32* .083 .000* -.51 1.07 Multidirectional Manual .59* .083 .000* .39 -.87 Blender .27* .083 .004* .07 -- Blender Manual .32* .083 .000* .13 -.68 Multidirectional -.27* .083 .004* -.46 -- Based on observed means. The error term is Mean Square(Error) = .970. *. The mean difference is significant at the .05 level. 45 Table 4.18. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Wizard Island. 95% Confidence Mean Interval (I) Thematic (J) Thematic Differenc Std. Lower Upper Terrain Layer Terrain Layer e (I-J) Error Sig. Bound Bound Hypsometric Tint Landcover -.29* .083 .001* -.48 -.09 Orthoimagery -.48* .083 .000* -.67 -.29 Landcover Hypsometric Tint .29* .083 .001* .09 .48 Orthoimagery -.19 .083 .051 -.39 .00 Orthoimagery Hypsometric Tint .48* .083 .000* .29 .67 Landcover .19 .083 .051 .00 .39 Based on observed means. The error term is Mean Square(Error) = .970. *. The mean difference is significant at the .05 level. 4.4.2 Effects of Shaded Relief and Thematic Terrain Layers on Landform Clarity Scores: Mount Scott Figure 4.8 and Figure 4.9 show the estimated marginal means plotted for perceived clarity of Mount Scott for each shaded relief across the three thematic layers. For Mount Scott, the results of a two-way ANOVA (Table 4.19) showed a significant main effect for both shaded relief (F(2,4) = 21.76, p < .001), and thematic layer (F(2,4) = 10.86, p < .001), and a significant interaction between the two (F(2,4) = 2.798, p = .025) on clarity rating scores. A Tukey HSD post-hoc test showed that shaded relief methods were not all significantly different from one another. Multidirectional relief and Blender relief were not significantly affected by each other (p = .213), but manual relief differed significantly from the other two shaded relief (p < .001) (Table 4.20). The HSD post-hoc test for thematic terrain layers for Mount Scott found that orthoimagery and landcover were not significantly different (p = .931) (Table 4.21). 46 Figure 4.8. Plotted results of the two-way ANOVA. Estimated marginal means for Mount Scott landform clarity rating scores. Figure 4.9. Bar graphs and images of estimated marginal means for Mount Scott clarity rating scores. 47 Table 4.19. Mount Scott results for the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on clarity ratings. Type III Sum of Mean Eta Variable Squares df Square F Sig. Squared Shaded Relief 47.642 2 23.821 21.760 .000* .049 Thematic Layer 23.768 2 11.884 10.856 .000* .025 Shaded Relief ‡ Thematic Layer 12.253 4 3.063 2.798 .025* .013 Table 4.20. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Mount Scott. 95% Confidence Mean Interval (I) Shaded (J) Shaded Difference Std. Lower Upper Relief Relief (I-J) Error Sig. Bound Bound Manual Multidirectional -.56* .088 .000* -.76 -.35 Blender -.41* .088 .000* -.62 -.20 Multidirectional Manual .56* .088 .000* .35 .76 Blender .15 .088 .213 -.06 .35 Blender Manual .41* .088 .000* .20 .62 Multidirectional -.15 .088 .213 -.35 .06 Based on observed means. The error term is Mean Square(Error) = 1.095. *. The mean difference is significant at the .05 level. Table 4.21. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Mount Scott. 95% Confidence Mean Interval (I) Thematic (J) Thematic Difference Std. Lower Upper Terrain Layer Terrain Layer (I-J) Error Sig. Bound Bound Hypsometric Tint Landcover -.29* .083 .000* -.48 -.09 Orthoimagery -.48* .083 .000* -.67 -.29 Landcover Hypsometric Tint .29* .083 .000* .09 .48 Orthoimagery -.19 .083 .931 -.39 .00 Orthoimagery Hypsometric Tint .48* .083 .000* .29 .67 Landcover .19 .083 .931 .00 .39 Based on observed means. The error term is Mean Square(Error) = .970. *. The mean difference is significant at the .05 level. 48 4.4.3 Effects of Shaded Relief and Thematic Terrain Layers on Landform Clarity Scores: Grouse Hill and Figure 4.11 show the estimated marginal means plotted for perceived clarity of Grouse Hill for each shaded relief across the three thematic layers. For Grouse Hill, the results of a two-way ANOVA (Table 4.22) showed a significant main effect for shaded relief (F(2,4) = 44.16, p < .001), but not for thematic layer (F(2,4) = .641, p = .527). However, there was significant interaction between the two (F(2,4) = 2.798, p = .014) on clarity rating scores for Grouse Hill. A Tukey HSD post-hoc test showed that there was a significant difference between multidirectional, Blender, and manual shaded relief (Table 4.23). Figure 4.10. Plotted results of the two-way ANOVA. Estimated marginal means for Mount Scott landform clarity rating scores. 49 Figure 4.11. Bar graphs and images of estimated marginal means for Grouse Hill clarity rating scores. The HSD post-hoc test for thematic terrain layers found the inverse was true compared to shaded relief. There were no significant effects for hypsometric tint compared to landcover (p = .91), hypsometric tint compared to orthoimagery (p = .50), or orthoimagery compared to landcover (p = .78) (Table 4.24). Table 4.22. Grouse Hill results for the two-way ANOVA. Multiple comparison of the effect that shaded relief and thematic terrain layers have on clarity ratings. Type III Sum of Mean Eta Variable Squares df Square F Sig. Squared Shaded Relief 103.665 2 51.833 44.157 .000* .095 Thematic Layer 1.504 2 .752 .641 .527 .002 Shaded Relief ‡ Thematic Layer 14.756 4 3.689 3.143 .014* .015 50 Table 4.23. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Grouse Hill. 95% Confidence Mean Interval (I) Shaded (J) Shaded Difference Std. Lower Upper Relief Relief (I-J) Error Sig. Bound Bound Manual Multidirectional -.85* .091 .000* -1.07 -.64 Blender -.45* .091 .000* -.66 -.23 Multidirectional Manual .85* .091 .000* .64 1.07 Blender .41* .091 .000* .19 .62 Blender Manual .45* .091 .000* .23 .66 Multidirectional -.41* .091 .000* -.62 -.19 Based on observed means. The error term is Mean Square(Error) = 1.095. *. The mean difference is significant at the .05 level. Table 4.24. Results of the Tukey’s HSD post-hoc test for the relationship of thematic terrain layers on clarity ratings for Grouse Hill. 95% Confidence Mean Interval (I) Thematic (J) Thematic Difference Std. Lower Upper Terrain Layer Terrain Layer (I-J) Error Sig. Bound Bound Hypsometric Tint Landcover -.04 .091 .905 -.25 .17 Orthoimagery -.10 .091 .501 -.31 .11 Landcover Hypsometric Tint .04 .091 .905 -.17 .25 Orthoimagery -.06 .091 .766 -.28 .15 Orthoimagery Hypsometric Tint .10 .091 .501 -.11 .31 Landcover .06 .091 .766 -.15 .28 Based on observed means. The error term is Mean Square(Error) = .970. *. The mean difference is significant at the .05 level. 4.5 Geographic Familiarity At the beginning of the user study, participants answered “yes” or “no” to three questions pertaining to their familiarity with Crater Lake, OR: 1.) “Have you heard of Crater Lake before?” 2.) “Have you seen photos or maps of Crater Lake before?” 3.) “Have you visited Crater Lake before?” 51 To assess if a participant’s geographic familiarity would have a confounded impact on the user study results, I ran two separate Independent Samples Mann-Whitney U Tests for each of the self-reported geographic familiarity questions. The first analysis I ran was between groups for each familiarity question and the combined mean rating scores for the entire study. To do this, I averaged each of the participant’s beauty, realism, and clarity scores and used this mean value as the dependent variable for the independent samples test. Next, I ran a second set of tests on the participant’s independent rating scores for beauty, realism, and landform clarity tasks across the nine maps. Since some samples were not normally distributed, or equal in variance, I used the non-parametric Mann- Whitney U test. However, some samples did in fact meet the assumptions for parametric tests. For these, I used a parametric independent t-test as a validation process to confirm or reject the significance values concluded from the non-parametric test results. 4.5.1 Combined Mean Rating Scores The initial test found there to be no significant difference between groups of each familiarity question for the combined mean rating scores of beauty, realism, and the three clarity tasks (Table 4.25). There was no significant difference (U = 1104, p = 0. 603) between those who had heard of Crater Lake (N = 61) and those who had not heard of Crater Lake (N = 34). There was no significant difference (U = 1141.5, p = 0. 754) between those who had seen photos or maps of Crater Lake (N = 40) and those who had not seen photos or maps of Crater Lake (N = 55). Last, there was no significant difference (U = 356.5, p = 0. 406) between those who reported they had visited Crater Lake (N = 10) and those who had not visited Crater Lake before (N = 85). 52 Table 4.26 compares the overall mean rank for beauty, realism, and clarity between groups for each level of geographic familiarity. This showed that regardless of whether a participant had heard of Crater Lake, seen photos or maps of Crater Lake, or had visited Crater Lake, there was no impact on their combined mean score for beauty, realism, and landform clarity ratings. Table 4.25. No significance was found for the combined mean rating scores between each of the three groups. Independent Samples Mann-Whitney U Test summary. Combined Mean Rating Scores for Beauty, Realism, and Clarity 1.) “Have you 2.) “Have you ever 3.) “Have you ever heard seen photos or ever visited…?” of…?” maps of…?” Total N 95 95 95 Mann-Whitney U 1104.000 1141.5 356.5 Wilcoxon W 1699.000 2681.5 4011.5 Test Statistic 1104.000 1141.5 356.5 Standard Error 128.764 132.618 82.433 Standardized Test Statistic .520 0.313 -0.831 Asymptotic Sig. (2-sided test) .603 0.754 0.406 Table 4.26. Overall mean rank comparison between groups of geographic familiarity. 1.)“Have you ever 2.)“Have you ever seen photos or 3.)“Have you ever heard of…?” maps of…?” visited…?” Yes No Yes No Yes No Total N 61 34 40 55 10 85 Mean Rank 46.9 49.97 46.96 48.75 54.85 47.19 53 4.5.2 Individual Sample Rating Scores Four individual rating tasks were significant when testing the relationship between groups for each familiarity question (Table 4.27). The first familiarity question found one significant sample between groups for the clarity rating task of Wizard Island on the map that used multidirectional hillshading combined with hypsometric tinting (U = 1343, p = 0.009) (Table 4.27). Though more participants had heard of Crater Lake (N = 61, M = 42.98), those who had not heard of Crater Lake (N = 32, M = 57) contributed to a higher mean rank (Table 4.28). The second familiarity question found two significant samples between groups for the realism rating task on the map that used multidirectional shaded relief combined with hypsometric tinting (U = 762.5, p = 0.008) and for the clarity rating task for Wizard Island on the map that used multidirectional shaded relief combined with landcover (U = 1386, p = 0.017) (Table 4.27). Though more participants had not seen maps or photos of Crater Lake (N = 55, M = 41.86), those who had seen photos or maps of Crater Lake (N = 40, M = 56.44) contributed to a higher mean rank. Additionally, though more participants had not seen maps or photos of Crater Lake (N = 55, M = 46.01), those who had seen photos or maps of Crater Lake (N = 40, M = 64.90) contributed to a higher mean rank (Table 4.28). The third familiarity question found one significant sample between groups for the clarity rating task for Mount Scott on the map that used manual shaded relief combined with orthoimagery (U = 256, p = 0.031) (Table 4.27). Though more participants had not visited Crater Lake (N = 85, M = 46.01), those who had visited Crater Lake (N = 10, M = 64.90) contributed to a higher mean rank (Table 4.28). 54 Table 4.27. Significance was found amongst four individual samples across the three familiarity questions. Independent Samples Mann-Whitney U Test summary. 1.) “Have you 2.) “Have you ever seen photos or 3.) “Have you ever heard maps of…?” ever of…?” visited…?” Clarity: Clarity: Clarity: Wizard Is. Realism Wizard Is. Mt. Scott Multidirectional Multidirectional Multidirectional Manual Hypsometric Hypsometric Landcover Orthoimagery Tint Tint Total N 95 95 95 95 Mann-Whitney U 1343.000 762.500 1386.000 256 Wilcoxon W 1938.000 2302.500 2926.000 3911 Test Statistic 1343.000 762.500 1386.000 256 Standard Error 116.649 127.792 120.140 78.577 Standardized Test Stat. 2.623 -2.641 2.381 -2.151 Asymptotic Sig. (2-sided test) .009* .008* .017* 0.031* Table 4.28. Significant samples and their mean rank comparison between groups of geographic familiarity. 1.)“Have you ever heard 2.)“Have you ever seen photos or 3.)“Have you of…?” maps of…?” ever visited…?” Clarity: Clarity: Clarity: Mt. Wizard Is. Realism Wizard Is. Scott Multidirectional Multidirectional Multidirectional Manual Hypsometric Hypsometric Tint Tint Landcover Orthoimagery Yes No Yes No Yes No Yes No Total N 61 34 40 55 40 55 10 85 Mean Rank 42.98 57.00 56.44 41.86 64.90 46.01 64.90 46.01 55 CHAPTER V DISCUSSION In this chapter, I discuss the implications of the results detailed in Chapter 4. In Section 5.1, I discuss the results of the aesthetic preference rating scores for beauty and realism. In Section 5.2, I discuss the results of the clarity rating scores for Wizard Island, Mount Scott, and Grouse Hill. Then, in section 5.3, I explain the relevance that geographic familiarity had on user responses. Finally, I conclude this chapter in Section 5.4 with a short overview of the open-ended concluding question in the user study. 5.1 Beauty and Realism Rating Tasks Overall, there was a wide variety of responses to the beauty and realism rating tasks and the ratings depended on the shaded relief and thematic terrain layers used. The ratings were more influenced by the thematic terrain layers than shaded relief designs, although they were both significant. And while both shaded relief and thematic terrain layers were significant, the scores were not dependent on the combination of thematic terrain layer and shaded relief, rather the two variables had unique outcomes on user perceptions. Finally, the outcomes from both tasks showed there to be a correlation between beauty and realism. The effect of shaded relief on the beauty and realism ratings showed that participants consistently rated manual relief as the least beautiful and realistic of the three shaded relief methods. This is a surprising finding because the cartographic community time and time again has pointed to manual relief as the most artistic and realistic technique for representing terrain (Imhof, 1982; Brassel, 1974; Collier et al., 2003; 56 Marston and Jenny, 2015). I found this to be more surprising for beauty than for realism. In fact, the realism results confirmed my expectation that manual relief would lead to lower realism rating scores since manual relief inherently is tied to the cartographer’s interpretation of the landscape through generalization (Patterson, 2015). This could have contributed to participants finding the manual relief to be less realistic, especially when compared to the two analytical relief methods. Blender relief and multidirectional hillshading were both rated highly for perceived beauty and realism; however, the ratings were not significantly different between these two analytical shaded relief methods, even though multidirectional relief had the highest mean score compared to Blender relief. This finding reveals that I cannot conclude which of the two analytical methods were the most beautiful or realistic in this study. For the thematic terrain layers, map readers found orthographic imagery to be the most beautiful and realistic. Again, it surprised me to find that orthoimagery was most beautiful since cartographers use orthoimagery sparingly, because it adds visual complexity and has darker color distributions, making it a challenging layer to pair with overlaying vector data (Hoarau et al., 2013). It was, however, not surprising that orthoimagery was the highest rated for the realism task since orthoimagery in maps is a photo, or combination of several photos, which understandably would add realism and context (Hoarau and Sidonie, 2017). This confirms Peterson’s (2012) notion that incorporating satellite imagery into maps, such as with Google Maps, Google Earth, and other online navigation maps, provides context and an aesthetic that is more relatable to average map users. Further, web applications rarely use hypsometric tinting and 57 landcover, which could have contributed to orthoimagery’s success in the beauty and realism rating tasks, since average map readers are now quite familiar with imagery. Landcover and hypsometric tinting were both rated low for perceived beauty and realism; however, for beauty ratings, the two thematic terrain layers were not significantly different, but for realism ratings, the results confirmed that hypsometric tinting was the least realistic. This means that while we cannot conclude that hypsometric tinting was the least beautiful of the three techniques compared to landcover in this study, this was not the case for realism. This finding may have some merit, since hypsometric tinting is an abstraction of reality and, depending on the design, it is not always a representative visualization of how the landscape looks. Depending on the location, colors used, represents it has the potential to convey inaccurate information about vegetation, rainfall, or temperature (Patton and Crawford, 1977). It is possible that for an average map reader, the combination of manual shaded relief and hypsometric tinting may not elicit a strong sense of realism, especially when compared to other depictions that use combinations of orthoimagery. One of the most surprising findings was the direct correlation between beauty and realism scores. Cartographers have alluded to a connection between beauty and realism, claiming that a realistic cartographic representation of the landscape can aid in an appealing aesthetic quality for readers (Harvey, 1980; Robinson, 1989). In other words, the literature has suggested that a beautiful terrain map is a realistic terrain map. A Spearman’s correlation statistic confirmed this relationship and showed that map readers found the most/least beautiful maps were also the most/least realistic ones (Figure 5.1). 58 Figure 5.1. Comparison of estimated marginal means for beauty and realism rating scores by map. Scores were statistically similar between beauty and realism. 5.2 Landform Clarity Rating Tasks The results of the user study experiment showed that landform clarity ratings differed from beauty and realism ratings. Unlike the beauty and realism ratings, the landform clarity ratings showed that shaded relief was more influential on the clarity ratings than the thematic terrain layers. In addition, there was a significant interaction effect between the shaded relief layers and the thematic layers for landform clarity ratings for all three landforms: Wizard Island, Mount Scott, and Grouse Hill. This meant that there were unique outcomes on user perceptions of landform clarity depending on the pairing of shaded relief and thematic layers. Essentially, while shaded relief was more influential than thematic terrain layers, the combination of the two was also significant; which differed from the beauty and realism rating tasks, where there was no interaction 59 effect. This is an interesting finding because it suggests that shaded relief, and specific thematic terrain layer pairings, were more important for landform clarity than they were for beauty and realism. This confirms that neither shaded relief nor thematic terrain layers are the sole contributor to a beautiful, realistic, or clear terrain map, but that the layers provide distinct qualities and the interaction between shaded relief and thematic terrain is of important consideration for landform clarity specifically. In terms of the effect of shaded relief on landform clarity, I found different outcomes depending on the layers used and the specific landform I was asking about. Only two results were consistent across all three of the landforms in the study (Wizard Island, Grouse Hill, and Mount Scott): 1) shaded relief was more influential in landform clarity rating than the thematic terrain layers, and 2) manual relief was the least clear of the three shaded relief techniques. Grouse Hill and Wizard Island, were similar in that there was a statistically significant order of the shaded relief layers from most to least clear. For those two landforms, multidirectional hillshade was the highest rated, Blender was second, and manual was the least clear. While for Mount Scott, manual relief was the least clear, but there was no statistically significant difference between the two analytical shaded relief methods. The effect of the thematic terrain layers on landform clarity ratings was also inconsistent across the three landforms and not always a significant influencing factor. For example, thematic terrain layers had an influence on clarity ratings for Wizard Island and Mount Scott, but not for Grouse Hill. This finding confirms that shaded relief had a stronger impact on each landform, since statistically I did not find that the thematic terrain layers played a significant role in influencing landform clarity on their own. 60 The results of statistically testing the interaction of combining shaded relief and thematic terrain layers was significant and provided insights into designing terrain maps with the goal of more clearly depicting landforms to map readers. In short, depending on how the two types of layers were combined, had different impacts on the landform clarity ratings, and this was dependent on the specific landform. For instance, with Mount Scott, landform clarity was highest when multidirectional relief was paired with landcover, but when multidirectional relief was paired with orthoimagery the clarity was negatively affected. However, this was not the case for Grouse Hill and Wizard Island. For those two landforms, while the interaction effect was still statistically significant, the pairing of the thematic terrain layers with multidirectional relief did not lead to such a dramatic effect on landform clarity. In general, this interaction effect was interesting because shaded relief is rarely used in isolation and it was clear that the thematic terrain layers, while not always individually influential, did have an effect depending on how they were paired with the different shaded relief layers. The interaction effect also showed that the pairing of orthoimagery consistently improved rating scores for all three landforms when combined with manual relief. Manual relief led to the lower clarity ratings, and this confirms that the combination of certain shaded relief and thematic terrain layers makes a significant difference in user perception. I found this to be interesting because, again, cartographers rarely use orthoimagery in combination with any type of shaded relief, especially manual relief. It is possible that while highly generalized relief (e.g. manual relief) may not be the clearest at representing individual landforms, the detail and texture of orthoimagery could enhance its. 61 While statistically I saw a significant interaction effect for landform clarity, my data would have been better suited for a non-parametric test given that the data was not normally distributed and it failed a homogeneity of variance test; however, there is no non-parametric version of the two-way ANOVA test. In addition, there is much debate over to what degree the violation of these two assumptions has significant impacts on the statistical results (Scariano & Davenport, 1987; Lix et al. 1996). To ensure these results were credible, I ran subsequent non-parametric tests to corroborate the results of the independent ANOVA for shaded relief and thematic terrain layers on each rating task. Finally, the results of the landform clarity section of the user study experiment imply that figure-ground may influence how the different layers affect landform clarity. For example, Wizard Island had the highest grand mean value for perceived clarity (M = 3.90), with Mount Scott following (M = 3.74), and Grouse Hill with the lowest (M = 3.62), suggesting that map readers on average found Wizard Island to be clearer than the other landforms. It is easy to see when looking at a map of Crater Lake, Oregon that, indeed, Wizard Island is a very prominent feature in the water-filled crater, followed by Mount Scott, the largest mountain landform in the vicinity, followed by Grouse Hill which is far less visually prominent. It’s understood that there is a special relationship between land and water regarding map interpretation (Head, 1972) and that foreground features move up higher in the visual order (Haber & Hershenson, 1973). This is especially true when foreground features have greater visual contrast (like the contrast between Wizard Island and the water of Crater Lake) which can help them stand out, become more distinguishable, and create a perceptually distinct sense of hierarchy (MacEachren & 62 Mistrick, 1992). The lower clarity scores for Grouse Hill, on the other hand, can be explained by these same theories. Indeed, one participant said that Grouse Hill “was [the] hardest to make clear” which was perhaps because of its lower elevation, minimal vertical height difference, and less contrast between the landform and the base of the hill. While the aim of this research was not to understand the connection between landform clarity and contrast, this result indicates that certain landforms (such as islands, coastal cliffs, and peninsulas) might assume higher landform clarity ratings, regardless of their shaded relief and thematic terrain pairing, because their figure-ground relationship. 5.3 Geographic Familiarity It is known that landscape preference (Dearden, 1984; Herzog et al., 2000) and map reading cognition lead to different responses depending on the reader’s level of geographic familiarity (Kaplan, 1987). In this study, geographic familiarity with Crater Lake, Oregon did not have an effect on the overall combined ratings scores for all nine maps, but it did have a statistically significant impact in four specific instances in the user study. Participants who had never heard of Crater Lake were more likely to find Wizard Island to be most clear when multidirectional shaded relief and hypsometric tinting were combined. Interestingly, those who had seen photos and maps of Crater Lake rated Wizard Island as most clear when multidirectional shaded relief was combined with landcover instead. Those same participants who had seen photos and maps of Crater Lake were also more likely to rate the combination of multidirectional shaded relief and hypsometric tinting as most realistic. And finally, those who had visited Crater Lake were more likely to rate Mount Scott as most clear when manual relief was combined with orthoimagery. So, while there was no consistent pattern in regards to geographic 63 familiarity, these results still suggest that different levels of familiarity have an effect on reader perceptions depending on the question being asked and the specific combinations of shaded relief and thematic terrain layers. 5.4 Qualitative Findings from Open-ended Post-questionnaire The last question in the user study asked participants to: “Please provide any comments or feedback on your experience while taking part in this study”. Only 22 participants of the 105 who took the user study answered this question. Overall, most participants stated they enjoyed the study and found the map rating tasks to be interesting and enjoyable. The rest of the comments either confirmed results from the statistical analysis or provided intriguing insight into confounding effects from the stimuli design or study design, which I discuss the implications of a few more comments in the limitations section of Chapter 6. 64 CHAPTER VI CONCLUSION This research set out to understand the effect of shaded relief and thematic terrain layers on map reader perceptions of beauty, realism, and landform clarity in terrain maps. Cartographers often combined shaded relief with thematic terrain layers when creating terrain maps, thus the primary aim of this research was to understand the relationship between these two layers regarding map reader perceptions. To do this, I conducted an online user study where participants rated beauty, realism, and landform clarity on a series of terrain maps of Crater Lake, Oregon. Specifically, I tested the popular shaded relief techniques: manual relief, multidirectional relief, and Blender generated relief and three common thematic terrain layers: hypsometric tinting, landcover, orthoimagery. I used a two-way ANOVA to test the significance of each shaded relief technique and thematic terrain layer and their effect on participant’s ratings of beauty, realism, and the clarity of three landforms (Wizard Island, Mount Scott, and Grouse Hill). Ultimately, neither shaded relief nor thematic terrain layers were the sole contributors to beauty, realism, or landform clarity scores. Instead, first, I found that beauty and realism were highly correlated and that shaded relief, more than the thematic terrain layers, led to differences in these ratings. Second, I found that landform clarity had nearly opposite ratings from beauty and realism, and these ratings may have been more affected by the type of landform itself rather than the layers used to map it. This research contributes to a body of knowledge aimed at understanding and improving the perceived aesthetic qualities and clarity of landforms in terrain representation. 65 6.1 Overview of the Findings In this section, I outline some of the more important findings from this research. First, I found there to be a direct correlation between beauty and realism scores, which were more influenced by thematic terrain layers than by shaded relief. The interaction between thematic layers and shaded relief was not significant, meaning that shaded relief and thematic terrain layers were independently influential. Participants rated manual relief lowest for beauty and realism, while participants rated orthoimagery as the most beautiful and realistic. This was surprising given that many professional cartographers believe manual shaded relief is the most beautiful shaded relief style and rarely use orthoimagery because it is difficult to combine with other vector overlay layers. Second, and conversely to beauty and realism, landform clarity ratings showed that shaded relief was more influential than the thematic terrain layers, and the combination of the two layers affected clarity outcomes. However, similar to beauty and realism, manual relief was the least clear of the three shaded relief techniques. The results could not establish that any specific thematic terrain layers played a significant role in influencing landform clarity, which supports the conclusion that shaded relief had a stronger impact on landform clarity. The results also implied that figure-ground may have influenced how the different layers affected landform clarity. This was most apparent for Wizard Island, which had the highest visual contrast because of the land and water relationship, and Grouse Hill, which was the least visually prominent because of its lower elevation, minimal vertical height difference, and less contrast between the landform and the base of the hill. Future research would be well served to control for or test the implications of contrast and figure-ground on ratings of beauty, realism, and landform clarity. 66 The results of the analysis ultimately showed that neither shaded relief nor thematic terrain layers were the sole contributors to map reader perceptions of beauty, realism, or landform clarity. Instead, these layers provided distinct qualities depending on their pairing by confirming that the interaction between shaded relief and thematic terrain is of important consideration when creating terrain maps. Last, a participant’s geographic familiarity with Crater Lake, Oregon did not influence the overall combined rating scores for all nine maps, but only had a statistically significant impact in four specific instances in the user study. The findings suggest that different levels of familiarity influence reader perceptions depending on the question being asked and the specific combinations of shaded relief and thematic terrain layers. 6.2 Limitations and Further Research While these results have substantive implications for perceived beauty, realism, and landform clarity in terrain maps when combining shaded relief with thematic terrain layers, the findings have some limitations which I elaborate on here: 1.) First, while participants consistently rated manual relief as the least beautiful, realistic, and clear in depicting landforms, the manual relief in this study was just one example of this type of shaded relief. Manual relief commonly varies depending on the cartographer’s style, skill level, and interpretation of the landscape. Further research should verify my findings by testing multiple manual shaded relief drawings (Jenny et al., 2020), multiple geographies and landform types (Farmakis- Serebryakova and Hurni, 2020), varying spatial scales, and different levels of detail. 2.) Second, this research did not consider contrast, lightness, or saturation, which may have had confounding effects on the results. The analysis showed that 67 participants felt orthoimagery was the most realistic and beautiful, which was also the darkest, had the highest contrast, and high degrees of saturation. One participant commented: “I noticed I liked the darker contrasted maps better than the lighter colored ones”. Inversely, maps that had the lowest ratings used hypsometric tinting, which were significantly lighter and less saturated. Other cartographic research has shown that map readers respond favorably to images with high degrees of contrast, regardless of the aesthetic rendering (Fabrikant et al., 2012), and different levels of contrast can yield different responses for map reading (Limpisathian, 2017; Brewer, 1992). Future research should emphasize consistency in contrast, lightness, and saturation in terrain representation to avoid this potentially confounding variable. 3.) While my research showed that geographic familiarity had little impact on the results of the study, this was in contrast to Raposo and Brewer’s (2014) conclusions, which found that geographic location played a significant role in influencing aesthetic preference. Future research could well test multiple geographies to either corroborate or challenge the findings of my research. 4.) Showing one map at a time caused participants to have a hard time judging the beauty and realism of all nine stimuli maps. I received comments that suggested having multiple maps side by side would help participants to better compare between map designs. Future research might try allowing participants to see all or some of the map designs side-by-side when rating beauty and realism. 5.) Since this research eliminated participants with expert knowledge in cartographic design, future research should test the differences between the pool of participants in this study with experts. 68 6.) Finally, this research focused on just three shaded relief layers and three thematic terrain layers. Future research could expand on this by testing a wider range of different layers or parameters to create these layers. Some examples of terrain map representation techniques to include in a future analysis could comprise: texture shading (Brown, 2014), neural network relief shading (Jenny et al., 2020), sky-view factor hillshading (Zakšek et al., 2011), aerial perspective (Jenny and Patterson, 2021), 3D versus 2D relief representations (Taveras, 2018), and applying different hues to shaded and highlighted areas of a relief drawings (Imhof, 2982). 6.3 Concluding Thoughts Maps are not just tools for navigating when we get lost, but also works of art worth getting lost in (Ribeiro and Caquard, 2018; Harmon et al., 2009). The perceived aesthetics of beauty, realism, in combination with assuring that map readers can accurately see specific landforms, is essential for producing a successful terrain map. With this in mind, this thesis provides cartographers with some best practices for designing aesthetically pleasing and clear terrain maps with mindful consideration to both traditional and contemporary shaded relief methods when combined with thematic terrain layers. For creating a beautiful and realistic map, consider including orthoimagery in the representation. When designing a map that clearly represents a majority of the landforms, I suggest using multidirectional relief or Blender generated relief with thematic landcover. For those cartographers trying to balance aesthetics and clarity, I suggest using multidirectional relief and orthoimagery based on the results of my research. 69 APPENDIX A INFORMED CONSENT You are being asked to participate in a research study. The section below highlights key information about this research for you to consider when making a decision whether or not to participate. Carefully consider this information and the more detailed information provided below the box. Please ask questions about any of the information you do not understand before you decide whether to participate. Voluntary Consent. You are being asked to volunteer for a research study. It is up to you whether you choose to participate or not. There will be no penalty or loss of benefits to which you are otherwise entitled if you choose not to participate or discontinue participation. Purpose. The purpose of this research is to assess aesthetic responses to different terrain map designs. Up to 300 people will take part in this research. Duration. It is expected that your participation will last no longer than 20 minutes Procedures and Activities. You will be asked to view maps and rate them on how clearly each represents certain geographic features and how they appear aesthetically. Risks. There are no foreseeable risks in participating. 70 Financial Compensation. You will be paid $4.97 for your participation in the study. Benefits. There is no direct benefit to you beyond financial compensation, but the researcher hopes to learn about which terrain mapping techniques rate higher in geographic landform clarity and aesthetic preference. Alternatives. Participation is voluntary and the only alternative is to not participate. Who is conducting this research? The researchers Nathaniel Douglass and Dr. Carolyn Fish from the University of Oregon are asking for your consent to this research. What happens if I agree to participate in this research? If you agree to be in this research, your participation will include participating in a questionnaire. The questionnaire will be conducted in an online survey tool called Qualtrics. During the questionnaire, you will answer demographic questions about your gender, education level, familiarity with cartographic design, and familiarity with the geographic location in question. After these questions, you will complete a brief pre- study tutorial where you will learn about the questions asked of you. Then you will see nine different map designs depicting Crater Lake National Park, Oregon, US. You will be asked to rate how clearly the map represents a variety of geographic features, as well as the map's overall beauty and realism. A final post-survey 71 question will ask you if there were any comments you had on the maps and if one stood out to you as most effective or more aesthetically pleasing. Once you have completed the questionnaire you will follow the link back to Prolific so that we can send you your payment for participating. What happens to the information collected for this research? Information collected for this research will be used to answer a set of research questions. The goal is to understand how certain terrain mapping techniques influence the map reader's perceived clarity and aesthetic preference. The results, after statistical analysis, will be written into an academic journal article and presented at conferences. How will my privacy and data confidentiality be protected? We will not collect any identifying information in the questionnaire. All data collected in the questionnaire will be stored on a password-protected laptop or password-protected server. What are the benefits of participating in this research? You may or may not benefit from participating in this research. There is no direct benefit to you beyond financial compensation. What other choices do I have besides participation in this research? It is your choice to participate or not to participate in this research. 72 What if I want to stop participating in this research? Taking part in this research study is your decision. Your participation in this study is voluntary. You do not have to take part in this study, but if you do, you can stop at any time. You have the right to choose not to participate in any study activity or completely withdraw from continued participation at any point in this study without penalty or loss of benefits to which you are otherwise entitled. Your decision whether or not to participate will not affect your relationship with the researchers or the University of Oregon. What if I am injured because of participating in this research? If you experience harm because of the project, you can ask the State of Oregon to pay you. If you have been harmed, there are two University representatives you need to contact. Here are their addresses and phone numbers: General Counsel/ Office of the Research Compliance President Services 1226 University of Oregon 5237 University of Oregon Eugene, OR 97403-1226 Eugene, OR 97403-5237 (541) 346-3082 (541) 346-2510 A law called the Oregon Tort Claims Act may limit the amount of money you can receive from the State of Oregon if you are harmed. 73 Will I be paid for participating in this research? Each participant will be paid $4.97 for their participation in the study. Who can answer my questions about this research? If you have questions, concerns, or have experienced a research-related injury, contact the research team at: Nathaniel Douglass 707-362-5175 Ndougla7@uoregon.edu An Institutional Review Board (IRB) is overseeing this research. An IRB is a group of people who perform an independent review of research studies to ensure the rights and welfare of participants are protected. UO Research Compliance Services is the office that supports the IRB. If you have questions about your rights or wish to speak with someone other than the research team, you may contact: 5237 University of Oregon Eugene, OR 97403-5237 (541) 346-2510 74 STATEMENT OF CONSENT I have had the opportunity to read and consider the information in this form. I have asked any questions necessary to make a decision about my participation. I understand that I can ask additional questions throughout my participation. I understand that by signing below, I volunteer to participate in this research. I understand that I am not waiving any legal rights. I have been provided with a copy of this consent form. I understand that if my ability to consent or assent for myself changes, either I or my legal representative may be asked to re-consent prior to my continued participation in this study. Do you consent to participate in this study? o Yes o No 75 APPENDIX B PRE-TEST QUESTIONNAIRE Please answer the following questions to the best of your abilities. 1. Please indicate your gender. o Male o Female o Non-binary o Prefer not to say 2. Please indicate your highest education level. o Less than high school o High school graduate o Some college o 2 year degree o 4 year degree o Professional degree o Master's o Doctorate 3. Please indicate how knowledgeable you are with cartography and map design. o Extremely knowledgeable o Very knowledgeable o Moderately knowledgeable 76 o Slightly knowledgeable o Not knowledgeable at all 4. Have you ever heard of Crater Lake, OR, USA? o Yes o No 5. Please indicate if you have ever seen pictures or maps of Crater Lake? o Yes o No 6. Have you ever visited Crater Lake, OR, USA? o Yes o No 77 APPENDIX C TUTORIAL Before you begin the user study, please be sure to carefully read through the tutorial section and complete the three practice questions. You will take an example user study where you will be shown three sample maps of Churfisten, Switzerland. Click "Next" to begin. -End Page- In this study, you will be answering questions about terrain maps. These maps use shading techniques to represent topography as if the map were three dimensional. Terrain maps are effective because they accurately represent the earth's surface. Terrain maps also provide a pleasing aesthetic appeal and are often known for being both beautiful and realistic. The example bellow shows a terrain map without any labels. You are being asked to comment on the overall aesthetic appeal of this map. 78 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o 79 Nice work! Terrain maps also allow map readers to more accurately see landforms like mountains, valleys, and plains. In this next section, you are being shown the same map, but with landform labels. Please comment on the numbered landforms and how clearly they are represented in this map. 80 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Churfisten o o o o o Alp Sol o o o o o Vilan o o o o o -End of Page- 81 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 82 83 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Churfisten o o o o o Alp Sol o o o o o Vilan o o o o o -End Page- Awesome! One more map to go in this tutorial. Again, this is a slightly different map design. Please rate the overall aesthetic appeal of this final map. You will then be shown the same map with landform labels. 84 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 85 86 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Churfisten o o o o o Alp Sol o o o o o Vilan o o o o o -End of Page- 87 You were just shown a series of maps of Churfisten, Switzerland. Please indicate the location of the maps you were just shown. o Los Angeles, CA, USA o Churfisten, Switzerland o Ontario, Canada -End of Page- Great job! You have completed the tutorial. You are now ready to move on to the rest oft he user study. On the following section you will complete the same tasks but for 9 different maps of Crater Lake, OR, USA. Please read the information below before starting: - Please be sure to set your browser window to fullscreen. - Allow for at least 20 minutes to complete the survey. - Please do not rush and be sure to attempt to answer every question. - Depending on your internet connection, please allow each map to fully load before completing each task. Click "Next" to begin. -End of Page- 88 APPENDIX D USER STUDY Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. 89 Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Wizard Island o o o o o 90 Mount Scott o o o o o Grouse Hill o o o o o -End of Page- 91 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 92 93 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Wizard Island o o o o o Mount Scott o o o o o Grouse Hill o o o o o -End of Page- 94 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 95 96 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Wizard Island o o o o o Mount Scott o o o o o Grouse Hill o o o o o -End of Page- 97 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 98 99 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Wizard Island o o o o o Mount Scott o o o o o Grouse Hill o o o o o -End of Page- 100 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 101 102 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Wizard Island o o o o o Mount Scott o o o o o Grouse Hill o o o o o -End of Page- 103 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 104 105 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Wizard Island o o o o o Mount Scott o o o o o Grouse Hill o o o o o -End of Page- 106 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 107 108 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Wizard Island o o o o o Mount Scott o o o o o Grouse Hill o o o o o -End of Page- 109 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 110 111 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Wizard Island o o o o o Mount Scott o o o o o Grouse Hill o o o o o -End of Page- 112 Rate the map based on its overall beauty. Somewhat Very Not at all beautiful Not so beautiful beautiful Beautiful beautiful o o o o o Rate the map based on its overall realism. Not so Somewhat Very Not at all realistic realistic realistic Realistic realistic o o o o o -End of Page- 113 114 Rate the clarity of each landform in this map. Neither Very clear nor Very unclear Unclear unclear Clear clear Wizard Island o o o o o Mount Scott o o o o o Grouse Hill o o o o o -End of Page- 115 APPENDIX E POST-TEST QUESTIONNAIRE You were just shown a series of maps of Crater Lake, OR, USA. Please indicate thel ocation of the maps you were just shown. o Yellowstone, WY, USA o Yosemite, CA, USA o Crater Lake, OR,USA o Great Sand Dunes, CO, USA -End of Page- Please provide any comments or feedback on your experience while taking part in this study. ______________________________________________________________________ -End of Page- 116 APPENDIX F STIMULI CREATION This section provides a breakdown of the data sources, design choices, and colors used in the development of the stimuli maps. I’ve also created a public repository containing the Blender, Illustrator, Photoshop, merged DEMs, and final map files: https://github.com/nadouglass7/Terrain-Map-User-Study. I. STIMULI MAP CODES ID Shaded Relief Thematic Layer Location CL_1A Multidirectional Hypsometric Tint Crater Lake, OR CL_1B Blender Hypsometric Tint Crater Lake, OR CL_1C Manual Hypsometric Tint Crater Lake, OR CL_2A Multidirectional Landcover Crater Lake, OR CL_2B Blender Landcover Crater Lake, OR CL_2C Manual Landcover Crater Lake, OR CL_3A Multidirectional Orthoimagery Crater Lake, OR CL_3B Blender Orthoimagery Crater Lake, OR CL_3C Manual Orthoimagery Crater Lake, OR CH_1A Multidirectional Hypsometric Tint Churfisten, Switzerland CH_1B Blender Hypsometric Tint Churfisten, Switzerland CH_1C Manual Hypsometric Tint Churfisten, Switzerland Geographic Location: CL = Crater Lake, OR; CH = Churfisten, Switzerland Thematic Later: 1 = Hypsometric Tint; 2 = Landcover; 3 = Orthoimagery Shaded Relief: A = Multidirectional; B = Blender; C = Manual II. STIMULI MAP DATA SOURCES Shaded Relief Data Source Multidirectional http://shadedrelief.com/SampleElevationModels/ and Blender https://pubs.usgs.gov/dds/dds-72/site/data.htm Manual http://www.shadedreliefarchive.com/ Thematic Terrain Data Source Hypsometric Tint http://shadedrelief.com/SampleElevationModels/ Landcover https://www.mrlc.gov/ Orthoimagery https://www.google.com/maps/ 117 III. APPLIED STIMULI MAP COLORS Thematic Terrain Colors Hypsometric Tint (Land) #a2c2b1, #a8cebb, #eef6e4, #ffffff Hypsometric Tint (Water) #accbee, # accbee, #e7f0fd Landcover #c5d8ee, #ffffe8, #eaf4e6, #548154 Orthoimagery NA 118 APPENDIX G SUMMARY STATISTICS DESCRIPTIVES FOR RATING SCORES BY MAP TYPE 95% Confidence Interval for Mean Std. Std. Lower Upper Map ID N Mean Deviation Error Bound Bound Min Max CL_1A 95 3.06 1.08 0.111 2.84 3.28 1 5 CL_1B 95 2.81 1.014 0.104 2.6 3.02 1 5 CL_1C 95 2.41 1.116 0.114 2.18 2.64 1 5 CL_2A 95 3.15 1 0.103 2.94 3.35 1 5 Beauty CL_2B 95 3.01 1.037 0.106 2.8 3.22 1 5 CL_2C 95 2.68 0.97 0.1 2.49 2.88 1 5 CL_3A 95 3.83 0.859 0.088 3.66 4.01 1 5 CL_3B 95 3.92* 0.794 0.082 3.75 4.08 2 5 CL_3C 95 3.72 0.93 0.095 3.53 3.91 1 5 Total 855 3.18 1.099 0.038 3.1 3.25 1 5 CL_1A 95 2.98 1.062 0.109 2.76 3.2 1 5 CL_1B 95 2.69 0.957 0.098 2.5 2.89 1 5 CL_1C 95 2.29 0.977 0.1 2.1 2.49 1 5 CL_2A 95 3.33 1.026 0.105 3.12 3.54 1 5 Realism CL_2B 95 3.17 0.975 0.1 2.97 3.37 1 5 CL_2C 95 2.79 1.009 0.104 2.58 3 1 5 CL_3A 95 3.97 0.928 0.095 3.78 4.16 1 5 CL_3B 95 4.07* 0.815 0.084 3.91 4.24 1 5 CL_3C 95 3.84 0.982 0.101 3.64 4.04 1 5 Total 855 3.24 1.13 0.039 3.16 3.31 1 5 CL_1A 95 3.96 1.02 0.105 3.75 4.17 1 5 CL_1B 95 3.76 0.986 0.101 3.56 3.96 1 5 CL_1C 95 3.22 1.213 0.124 2.97 3.47 1 5 Clarity: CL_2A 95 4.34* 0.694 0.071 4.2 4.48 2 5 Wizard CL_2B 95 3.94 0.987 0.101 3.74 4.14 1 5 Island CL_2C 95 3.53 1.128 0.116 3.3 3.76 1 5 CL_3A 95 4.26 0.802 0.082 4.1 4.43 1 5 CL_3B 95 4.06 0.976 0.1 3.86 4.26 1 5 CL_3C 95 4.05 0.961 0.099 3.86 4.25 1 5 Total 855 3.9 1.036 0.035 3.83 3.97 1 5 119 DESCRIPTIVES FOR RATING SCORES BY MAP TYPE (Continued) 95% Confidence Interval for Mean Std. Std. Lower Upper Map ID N Mean Deviation Error Bound Bound Min Max CL_1A 95 3.86 1.078 0.111 3.64 4.08 1 5 CL_1B 95 3.59 1.125 0.115 3.36 3.82 1 5 CL_1C 95 3.05 1.224 0.126 2.8 3.3 1 5 Clarity: CL_2A 95 4.15* 0.934 0.096 3.96 4.34 2 5 Mount CL_2B 95 3.94 1.009 0.103 3.73 4.14 1 5 Scott CL_2C 95 3.43 1.028 0.105 3.22 3.64 1 5 CL_3A 95 3.91 0.99 0.102 3.7 4.11 2 5 CL_3B 95 3.95 0.982 0.101 3.75 4.15 1 5 CL_3C 95 3.76 1.018 0.104 3.55 3.97 1 5 Total 855 3.74 1.087 0.037 3.66 3.81 1 5 CL_1A 95 4.06 0.954 0.098 3.87 4.26 1 5 CL_1B 95 3.66 1.107 0.114 3.44 3.89 1 5 CL_1C 95 3 1.296 0.133 2.74 3.26 1 5 Clarity: CL_2A 95 4.18* 0.863 0.089 4 4.35 2 5 Grouse CL_2B 95 3.56 1.099 0.113 3.33 3.78 1 5 Hill CL_2C 95 3.11 1.189 0.122 2.86 3.35 1 5 CL_3A 95 3.88 0.909 0.093 3.7 4.07 2 5 CL_3B 95 3.68 1.104 0.113 3.46 3.91 1 5 CL_3C 95 3.46 1.156 0.119 3.23 3.7 1 5 Total 855 3.62 1.142 0.039 3.55 3.7 1 5 120 DESCRIPTIVES FOR RATING SCORES BY SHADED RELIEF 95% Confidence Interval for Mean Shaded Relief N Mean Std. Std. Lower Upper Deviation Error Bound Bound Min Max Multidirectional 285 3.35* 1.039 0.062 3.23 3.47 1 5 Beauty Blender 285 3.25 1.066 0.063 3.12 3.37 1 5 Manual 285 2.94 1.152 0.068 2.8 3.07 1 5 Total 855 3.18 1.099 0.038 3.1 3.25 1 5 Multidirectional 285 3.42* 1.084 0.064 3.3 3.55 1 5 Realism Blender 285 3.31 1.08 0.064 3.19 3.44 1 5 Manual 285 2.98 1.179 0.07 2.84 3.11 1 5 Total 855 3.24 1.13 0.039 3.16 3.31 1 5 Clarity: Multidirectional 285 4.19* 0.862 0.051 4.09 4.29 1 5 Wizard Blender 285 3.92 0.988 0.059 3.8 4.03 1 5 Island Manual 285 3.6 1.154 0.068 3.47 3.73 1 5 Total 855 3.9 1.036 0.035 3.83 3.97 1 5 Clarity: Multidirectional 285 3.97* 1.007 0.06 3.85 4.09 1 5 Mount Blender 285 3.82 1.05 0.062 3.7 3.95 1 5 Scott Manual 285 3.41 1.128 0.067 3.28 3.55 1 5 Total 855 3.74 1.087 0.037 3.66 3.81 1 5 Clarity: Multidirectional 285 4.04* 0.914 0.054 3.94 4.15 1 5 Grouse Blender 285 3.64 1.101 0.065 3.51 3.76 1 5 Hill Manual 285 3.19 1.227 0.073 3.05 3.33 1 5 Total 855 3.62 1.142 0.039 3.55 3.7 1 5 121 DESCRIPTIVES FOR RATING SCORES BY THEMATIC TERRAIN LAYERS 95% Confidence Interval for Mean N Mean Std. Std. Lower Upper Deviation Error Bound Bound Min Max Hypsometric Tint 285 2.76 1.1 0.065 2.63 2.89 1 5 Beauty Landcover 285 2.95 1.018 0.06 2.83 3.07 1 5 Orthoimagery 285 3.82* 0.864 0.051 3.72 3.92 1 5 Total 855 3.18 1.099 0.038 3.1 3.25 1 5 Hypsometric Tint 285 2.66 1.035 0.061 2.54 2.78 1 5 Realism Landcover 285 3.09 1.025 0.061 2.98 3.21 1 5 Orthoimagery 285 3.96* 0.913 0.054 3.85 4.07 1 5 Total 855 3.24 1.13 0.039 3.16 3.31 1 5 Hypsometric Clarity: Tint 285 3.65 1.118 0.066 3.52 3.78 1 5 Wizard Landcover 285 3.93 1.007 0.06 3.82 4.05 1 5 Island Orthoimagery 285 4.13* 0.918 0.054 4.02 4.23 1 5 Total 855 3.9 1.036 0.035 3.83 3.97 1 5 Hypsometric Clarity: Tint 285 3.5 1.189 0.07 3.36 3.64 1 5 Mount Landcover 285 3.84 1.032 0.061 3.72 3.96 1 5 Scott Orthoimagery 285 3.87* 0.997 0.059 3.75 3.99 1 5 Total 855 3.74 1.087 0.037 3.66 3.81 1 5 Hypsometric Clarity: Tint 285 3.58 1.207 0.071 3.43 3.72 1 5 Grouse Landcover 285 3.61 1.144 0.068 3.48 3.75 1 5 Hill Orthoimagery 285 3.68* 1.072 0.063 3.55 3.8 1 5 Total 855 3.62 1.142 0.039 3.55 3.7 1 5 122 REFERENCES CITED Adobe Inc. (2021). Adobe Illustrator. Retrieved from https://adobe.com/products/illustrator Adobe Inc. (2021). Adobe Illustrator. Retrieved from https://adobe.com/products/photoshop Allmen, Bill von. Map. 1988. Crater Lake. http://www.shadedreliefarchive.com/Crater- Lake.html. Atwood, Peter. “Relief Shading in Blender - Tips and Tricks with Peter Atwood.” Geospatial Frontier. Geospatial Frontier, June 15, 2020. https://www.geospatialfrontier.com/medialibrary/relief-shading-in-Blender-tips- and-tricks-with-peter-atwood. Bartram, Lyn, Abhisekh Patra, and Maureen Stone. "Affective color in visualization." In Proceedings of the 2017 CHI conference on human factors in computing systems, pp. 1364-1374. 2017. Bell, Sarah. 2018. Review of Drawing Hillshade: A Tutorial (with Time Lapse Videos). Petrichor. 2018. https://petrichor.studio/2018/10/22/drawing-hillshade-shaded- relief-with-time-lapse-video-demos/ Biland, Julien, and Arzu Çöltekin. "An empirical assessment of the impact of the light direction on the relief inversion effect in shaded relief maps: NNW is better than NW." Cartography and Geographic Information Science 44, no. 4 (2017): 358- 372. Blender Documentation Team. “The Blender 2.93 Manual,” 2021. https://docs.blender.org/manual/ Brassel, Kurt. "A model for automatic hill-shading." The American Cartographer 1, no. 1 (1974): 15-27. Brewer, Cynthia A. "Review of colour terms and simultaneous contrast research for cartography." Cartographica: The International Journal for Geographic Information and Geovisualization 29, no. 3-4 (1992): 20-30. Brown, Leland. "Texture shading: A new technique for depicting terrain relief." In ICA Mountain Cartography Workshop, vol. 24. 2014. Collier, Peter, David Forrest, and Alastair Pearson. "The representation of topographic information on maps: the depiction of relief." The Cartographic Journal 40, no. 1 (2003): 17-26. 123 Çöltekin, Arzu, and Julien Biland. "Comparing the terrain reversal effect in satellite images and in shaded relief maps: an examination of the effects of color and texture on 3D shape perception from shading." International Journal of Digital Earth 12, no. 4 (2019): 442-459. Community, B.O., 2018. Blender - a 3D modelling and rendering package, Stichting Blender Foundation, Amsterdam. Available at: http://www.Blender.org. Cosgrove, Denis. "Maps, mapping, modernity: Art and cartography in the twentieth century." Imago Mundi 57, no. 1 (2005): 35-54. Dearden, Philip. "Factors influencing landscape preferences: an empirical investigation." Landscape planning 11, no. 4 (1984): 293-306. Dent, Borden, J. Torguson, and T. Hodler. Thematic map design. New York, New York, NY: McGraw-Hill, 2008. Eyton, J. Ronald. “Color stereoscopic effect cartography”. Cartographica: The International Journal for Geographic Information and Geovisualization 27, no. 1 (1990): 20-29. Fabrikant, Sara I., Sidonie Christophe, Georgios Papastefanou, Sara Lanini-Maggi, Ningchuan Xiao, Mei-Po Kwan, and Hui Lin. "Emotional response to map design aesthetics." (2012). Farmakis-Serebryakova, Marianna, and Lorenz Hurni. "Comparison of relief shading techniques applied to landforms." ISPRS International Journal of Geo- Information 9, no. 4 (2020): 253. Field, Kenneth, and Damien Demaj. "Reasserting design relevance in cartography: some concepts." The Cartographic Journal 49, no. 1 (2012): 70-76. Gardner, James V., and Peter Dartnell. 2000 Multibeam Sonar Survey of Crater Lake, Oregon-Data, GIS, Images, and Movies. No. 72. US Geological Survey, 2001. Guidero, Elaine Meredith. Where cartography meets typography: Choosing typefaces and semantic effects for maps using microaesthetics. The Pennsylvania State University, 2016. Haber, Ralph N., and Maurice Hershenson. The psychology of visual perception. Holt, Rinehart & Winston, 1973. Harmon, Katharine, Gayle Harmon, and Gayle Clemans. Map As Art, The: Contemporary Artists Explore Cartography. Princeton Architectural Press, 2009. 124 Harvey, Paul Dean Adshead. The history of topographical maps: symbols, pictures and surveys with 116 illustrations 10 in colour. Thames and Hudson, 1980. Hauser, Markus. Map. Asahi-Dake: the Heart of the Daisetsuzan National Park. Gecko Maps, 2017. Head, C. Grant. "Land-water differentiation in black and white cartography." Cartographica: The International Journal for Geographic Information and Geovisualization 9, no. 1 (1972): 25-38. Heer, Jeffrey, and Michael Bostock. "Crowdsourcing graphical perception: using mechanical turk to assess visualization design." In Proceedings of the SIGCHI conference on human factors in computing systems, pp. 203-212. 2010. Herzog, Thomas R., Eugene J. Herbert, Rachel Kaplan, and C. L. Crooks. "Cultural and developmental comparisons of landscape perceptions and preferences." Environment and behavior 32, no. 3 (2000): 323-346. Hoarau, Charlotte, and Sidonie Christophe. “Cartographic Continuum Rendering Based on Color and Texture Interpolation to Enhance Photo-Realism Perception.” ISPRS Journal of Photogrammetry and Remote Sensing, Geospatial Week 2015, 127 (May 1, 2017): 27–38. https://doi.org/10.1016/j.isprsjprs.2016.09.012. Hoarau, Charlotte, Sidonie Christophe, and Sébastien Mustière. “Mixing, blending, merging or scrambling topographic maps and orthoimagery in geovisualizations.” In Proceedings of 26th International Cartographic Conference (ICC’13), Dresden, Germany. 2013. Horn, Berthold KP. "Hill shading and the reflectance map." Proceedings of the IEEE 69, no. 1 (1981): 14-47. Huffman, Daniel P., and Tom Patterson. "The design of gray earth: A monochrome terrain dataset of the world." Cartographic Perspectives 74 (2013): 61-70. Huffman, D. 2017. “Creating shaded relief in Blender.” Somethingaboutmaps.com. November 16, 2017. https://somethingaboutmaps.wordpress.com/2017/11/16/creatingshaded- relief-in-Blender/ Imhof, Eduard. "Cartographic relief presentation, ed." Harry J. Stewart. Berlin: Walter de Gruyter&co (1982). Imhof, Eduard. “Oberengadin.” 1958. Map. Schweizerischer Mittelschulatlas. http://codex99.com/cartography/images/imhof/atlas_2_lg.jpg. 125 Imus, David, and Paula Loftin. "The Beauty of Clear Communication." Cartographic Perspectives 73 (2012): 103-106. Jenny, Bernhard, Magnus Heitzler, Dilpreet Singh, Marianna Farmakis-Serebryakova, Jeffery Chieh Liu, and Lorenz Hurni. "Cartographic relief shading with neural networks." IEEE Transactions on Visualization and Computer Graphics (2020). Jenny, Bernhard. "Terrain generalization with line integral convolution." Cartography and Geographic Information Science 48, no. 1 (2021): 78-92. Jenny, Bernhard, and Tom Patterson. "Aerial perspective for shaded relief." Cartography and Geographic Information Science 48, no. 1 (2021): 21-28. Jenny, Bernhard, and Tom Patterson. "Introducing plan oblique relief." Cartographic Perspectives 57 (2007): 21-40. Kaplan, Stephen. "Aesthetics, affect, and cognition: Environmental preference from an evolutionary perspective." Environment and behavior 19, no. 1 (1987): 3-32. Kennelly, Patrick J., and A. James Stewart. "General sky models for illuminating terrains." International Journal of Geographical Information Science 28, no. 2 (2014): 383-406. Kennelly, Patrick J., Tom Patterson, Bernhard Jenny, Daniel P. Huffman, Brooke E. Marston, Sarah Bell, and Alexander M. Tait. "Elevation models for reproducible evaluation of terrain representation." Cartography and Geographic Information Science 48, no. 1 (2021): 63-77. Kent, Alexander J. "Aesthetics: a lost cause in cartographic theory?." The Cartographic Journal 42, no. 2 (2005): 182-188. Kent, A., Kenneth Field, Bernhard Jenny, and Anja Hopfstock. "Cartographic design and aesthetics “FAQ”." Cartographic Perspectives 73 (2012). Kent, Raymond. Data construction and data analysis for survey research. Macmillan, 2001. Landesaufnahme. “Aufnahmeblätter 4856-1b Weißenbach an Der Triesting, Neuhaus.” 1872. Map. Wikipedia. https://de.wikipedia.org/wiki/Datei:Aufnahmeblatt_4856- 1b_Wei%C3%9Fenbach_an_der_Triesting,_Neuhaus.jpg. Larson, Joe. 2019. “Yet Another Blender Hillshade Tutorial.” Joelarson.com. April 7, 2019. https://joelarson.com/2019/04/07/Blender-hillshades.html. 126 Lin, Sharon, Julie Fortuna, Chinmay Kulkarni, Maureen Stone, and Jeffrey Heer. "Selecting semantically‐resonant colors for data visualization." In Computer Graphics Forum, vol. 32, no. 3pt4, pp. 401-410. Oxford, UK: Blackwell Publishing Ltd, 2013. Limpisathian, Pongpichaya. "Evaluating Visual Contrast and Hierarchy Relations of Cartographic Features Across Multi-Scale Map Displays." (2017). Lix, Lisa M., Joanne C. Keselman, and Harvey J. Keselman. "Consequences of assumption violations revisited: A quantitative review of alternatives to the one- way analysis of variance F test." Review of educational research 66, no. 4 (1996): 579-619. Loissios, Dimitrios, Nikolaos Tzelepis, and Byron Nakos. "A methodology for creating analytical hill-shading by combining different lighting directions." (2021). Lucchesi, Annita Hetoevėhotohke'E. "“Indians don't make maps”: Indigenous cartographic traditions and innovations." American Indian Culture and Research Journal 42, no. 3 (2018): 11-26. MacEachren, Alan M., and T. A. Mistrick. "The role of brightness differences in figure- ground: is darker figure?." The Cartographic Journal 29, no. 2 (1992): 91-100. Marston, Brooke E., and Bernhard Jenny. "Improving the representation of major landforms in analytical relief shading." International Journal of Geographical Information Science 29, no. 7 (2015): 1144-1165. Mylavarapu, Pranathi, Adil Yalcin, Xan Gregg, and Niklas Elmqvist. "Ranked-list visualization: A graphical perception study." In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp. 1-12. 2019. North American Cartographic Information Society. Atlas of Design. Volume Five. Golden Valley, MN: North American Cartographic Society, 2020. North American Cartographic Information Society. Atlas of Design. Volume Four. Golden Valley, MN: North American Cartographic Society, 2018. Norušis, Marija J. SPSS statistics 17.0: Statistical procedures companion. Prentice Hall/Pearson Education, 2008. Patterson, Tom. "A desktop approach to shaded relief production." Cartographic Perspectives 28 (1997): 38-39. Patterson, Tom. "Getting real: Reflecting on the new look of National Park Service maps." Cartographic perspectives 43 (2002): 43-56. 127 Patterson, Tom, and Bernard Jenny. Shaded Relief Archive, 2010. https://www.shadedreliefarchive.com/. Patterson, Tom. “Relief Presentation on Us National Park Service Maps.” Proceedings of the ICA 1 (2018): 1–5. https://doi.org/10.5194/ica-proc-1-90-2018. Patterson, Tom, and Bernhard Jenny. "The development and rationale of cross-blended hypsometric tints." Cartographic Perspectives 69 (2011): 31-46. Patterson, Tom, and Bernhard Jenny. "ShadedReliefArchive. com–a Digital Repository for Cartographic Relief Art." World Museums Magazine 297 (2012): 46-50. Patton, Jeffrey C., and Paul V. Crawford. "The perception of hypsometric colours." The Cartographic Journal 14, no. 2 (1977): 115-127. Peterson, Gretchen N. Cartographer’s Toolkit: Colors, Typography, Patterns. PetersonGIS, 2012. Peterson, Gretchen N. GIS Cartography: A Guide to Effective Map Design. CRC Press, 2020. Peutinger, Conradus, Scheyb von Franciscus Christophorus, Conradus Mannertus, and Salomon Kleiner. “Tabula Peutingeriana.” Map. Wikipedia. Accessed August 30, 2021. https://en.wikipedia.org/wiki/Tabula_Peutingeriana. Powell, Owen. “Owen Powell - Maps and Terrain Models.” BlenderNation. BlenderNation, September 3, 2016. https://www.Blendernation.com/2016/09/03/owen-powell-maps-terrain-models/. Rahman, Akm Samsur. "Effects of nanofibers on properties of geopolymer composites." In Nanotechnology in Eco-efficient Construction, pp. 123-140. Woodhead Publishing, 2019. Raposo, Paulo, and Cynthia A. Brewer. "Landscape preference and map readability in design evaluation of topographic maps with an orthoimage background." The Cartographic Journal 51, no. 1 (2014): 25-37. Ribeiro, Daniel Melo, and Sébastien Caquard. "Cartography and art." The Geographic Information Science & Technology Body of Knowledge (2018). Robinson, Artur. "Cartography as an Art." Cartography Past, Present, and Future: A Festschrift for FJ Ormeling (1989): 91-102. Samsonov, Timofey. "Multiscale hypsometric mapping." In Advances in Cartography and GIScience. Volume 1, pp. 497-520. Springer, Berlin, Heidelberg, 2011. 128 Scariano, Stephen M., and James M. Davenport. "The effects of violations of independence assumptions in the one-way ANOVA." The American Statistician 41, no. 2 (1987): 123-129. Tait, Alex. "Photoshop 6 tutorial: How to create basic colored shaded relief." Cartographic Perspectives 42 (2002): 12-17. Taveras, Aaron M. "The performance of 2D and 3D tsunami evacuation maps: A case study for Seaside, Oregon." (2018). Underwood, Nick. “Beyond Hillshading in Blender- Breaking Cartographic Convention with Care” Presentation, North American Cartographic Information Society, Tacoma, Washington, October 16, 2019. Beyond Hillshading in Blender- Breaking Cartographic Convention with Care Waldseemüller, Martin. “Tabula Moderna Germanie.” Map. Tabula Moderna Germanie. Straßburg: Johannes Schott, 1513. Zakšek, Klemen, Kristof Oštir, and Žiga Kokalj. "Sky-view factor as a relief visualization technique." Remote sensing 3, no. 2 (2011): 398-415. 129