MASARYK UNIVERSITY
FACULTY OF SCIENCE
DEPARTMENT OF GEOGRAPHY
EXPERIMENTAL RESEARCH IN
CARTOGRAPHIC VISUALIZATION
HABILITATION THESIS
ZDENĚK STACHOŇ
BRNO 2022
The good cartographer is both a scientist and an
artist. He must have a thorough knowledge of his
subject and model, the Earth.
Erwin Raisz
Acknowledgements
At this point, I would like to thank the supervisor of my Ph.D. studies, Professor Konečný, for
his guidance in my professional and personal life. Furthermore, to Professor Kubíček, the
Director of the Department of Geography, Faculty of Science, Masaryk University for his
support and professional feedback and Professor Dobrovolný for the conditions created for
my professional activities. To Dr. Šašinka for inspiring discussions over hectolitres of coffee
and to his wife Alžběta for her understanding.
To the co-authors of the published articles, without whom, this work would not have been
possible. I would also like to thank my colleagues from Masaryk University and other Czech
and foreign universities for their cooperation. I thank the Czech Science Foundation, the
Technology Agency of the Czech Republic, the Ministry of Education, Youth and Sports of the
Czech Republic, and the Grant Agency of Masaryk University for providing funds for our
research.
I also thank all current and former hiking club members for their outstanding physical
performances and contribution to my normality.
Last but not least, I thank my wife Barbara for her patience and support and my parents, Eva
and Zdeněk, for their support.
commentary
This habilitation thesis contains twelve research articles which represent long-term research
in the field of cartographic visualization. The research component of the thesis is divided into
two parts, consisting in (i) the design and configuration of tools for use in experimental
cartographic research and publication of research studies in cartographic visualization, and (ii)
studies on differences in visual perception, differences in bivariate cartographic visualization,
and complex spatial tasks. The results were subsequently applied to advance theoretical
knowledge in cartography, more generally for practical application in creating maps, and
specifically for application in geographic support during crisis management. Based on the
results, the thesis presents a discussion and proposal for a modified scheme of cartographic
communication and the forms which potentially affect this process in theoretical cartography.
Possible applications lie in generating cartographic visualizations according to the user’s
characteristics (map literacy, familiarity with the environment, cultural background, etc.) or
applying recommendations to geographic visualizations designed to support crisis
management, where time is a significant factor. Detailed results and discussions are included
in the attached research articles.
The following list of publications specifies the author’s role in the experimental work,
supervision, writing and editing of the manuscript and research direction.
Paper 1
Popelka, S., Stachoň, Z., Šašinka, Č., & Doležalová, J. (2016). Eyetribe Tracker Data Accuracy
Evaluation and Its Interconnection with Hypothesis Software for Cartographic Purposes.
Computational Intelligence and Neuroscience, Vol. 2016, Special Issue, February, Article ID 9172506.
https://doi.org/10.1155/2016/9172506. (WoS JIF QUARTILE: Q2)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
25 25 25 25
Paper 2
Šašinka, Č., Morong, K., & Stachoň, Z. (2017). The Hypothesis Platform : An Online Tool for
Experimental Research into Work with Maps and Behavior in Electronic Environments. ISPRS
International Journal of Geo-Information, 6(12). https://doi.org/10.3390/ijgi6120407. (WoS JIF
QUARTILE: Q2)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
40 30 40 40
Paper 3
Šašinka, Č., Stachoň, Z. (corresponding author), Sedlák, M., Chmelík, J., Herman, L., Kubíček, P.,
Strnadová, A., Doležal, M., Tejkl, H., Urbánek, T., Svatoňová, H., Ugwitz, P., & Juřík, V. (2019).
Collaborative Immersive Virtual Environments for Education in Geography. ISPRS International
Journal of Geo-Information, 8(1). https://doi.org/10.3390/ijgi8010003. (WoS JIF QUARTILE: Q2)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
30 45 10 50
Paper 4
Konečný, M., Kubíček, P., Stachoň, Z. (corresponding author), & Šašinka Č. (2011). The usability of
selected base maps for crises management: users' perspectives. SpringerLink, Applied Geomatics.
Springer, 2011/3, pp. 189–198. ISSN 1866-9298. https://doi.org/10.1007/s12518-011-0053-1. (WoS
JIF QUARTILE: Q3)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
70 40 70 60
Paper 5
Stachoň, Z., Šašinka, Č., Čeněk, J., Angsüesser, S. Kubíček, P., Štěrba, Z., Bilíková, M. (2018). Effect of
Size, Shape and Map Background in Cartographic Visualization: Experimental Study on Czech and
Chinese Populations. ISPRS International Journal of Geo-Information, 7/427, pp. 1-15. ISSN 2220-
9964. https://doi.org/10.3390/ijgi7110427. (WoS JIF QUARTILE: Q2)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
70 95 60 80
Paper 6
Stachoň, Z., Šašinka, Č., Čeněk, J., Štěrba, Z., Angsüesser, S., Fabrikant, S., I., Štampach, R., & Morong,
K. (2019). Cross-cultural differences in figure–ground perception of cartographic stimuli. Cartography
and Geographic Information Science, 46(1), 82-94. https://doi.org/10.1080/15230406.2018.1470575.
(WoS JIF QUARTILE: Q3)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
50 50 70 50
Paper 7
Kubíček, P., Šašinka, Č., Stachoň, Z., Štěrba, Z., Apeltauer, J., & Urbánek, T. (2017). Cartographic
Design and Usability of Visual Variables for Linear Features. Cartographic Journal, 54(1), 91-102.
https://doi.org/10.1080/00087041.2016.1168141. (WoS JIF QUARTILE: Q4)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
50 30 30 50
Paper 8
Šašinka, Č., Stachoň, Z., Kubíček, P., Tamm, S., Matas, A., & Kukaňová, M. (2018). The Impact of
Global/Local Bias on Task-solving in Map-related Tasks Employing Extrinsic and Intrinsic Visualization
of Risk Uncertainty Maps. The Cartographic Journal, ISSN 0008-7041.
https://doi.org/10.1080/00087041.2017.1414018. (WoS JIF QUARTILE: Q4)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
70 40 40 60
Paper 9
Šašinka, Č., Stachoň, Z. (corresponding author), Čeněk, J., Šašinková, A., Popelka, S., Ugwitz, P., &
Lacko, D. (2021). A Comparison of the Performance on Extrinsic and Intrinsic Cartographic
Visualizations through Correctness, Response Time and Cognitive Processing. PLoS ONE, 16(4).
https://doi.org/10.1371/journal.pone.0250164. (WoS JIF QUARTILE: Q2)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
40 20 40 40
Paper 10
Lacko, D., Šašinka, Č., Čeněk, J., Stachoň, Z., & Lu W.-L. (2020). Cross-Cultural Differences in Cognitive
Style, Individualism/Collectivism and Map Reading between Central European and East Asian
University Students. Studia Psychologica, 62(1), 23-42. http://dx.doi.org/10.31577/sp.2020.01.789.
(WoS JIF QUARTILE: Q4)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
30 50 10 40
Paper 11
Kubíček, P., Šašinka, Č., Stachoň, Z., Herman, L., Juřík, V., Urbánek, T., & Chmelík, J. (2019).
Identification of altitude profiles in 3D geovisualizations: the role of interaction and spatial abilities.
International Journal of Digital Earth, 12(2), 156-172.
https://doi.org/10.1080/17538947.2017.1382581. (WoS JIF QUARTILE: Q1)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
60 60 30 50
Paper 12
Zhu, L., Shen, J., Zhou, J., Stachoň, Z., Hong, S., & Wang, X., (2022). Personalized landmark adaptive
visualization method for pedestrian navigation maps: Considering user familiarity. Transactions in
GIS. Wiley, 26/2, pp. 669-690. ISSN 1361-1682. https://doi.org/10.1111/tgis.12877. (WoS JIF
QUARTILE: Q3)
Author’s contribution:
Experimental work (%) Supervision (%) Manuscript (%) Research direction (%)
10 50 10 60
8
Contents!
1. Introduction..................................................................................................................................... 9
1.1 Concept ................................................................................................................................. 10
1.2 Motivation............................................................................................................................. 11
2. Theoretical background................................................................................................................. 14
3. Research Methods......................................................................................................................... 18
3.1 Terminology........................................................................................................................... 18
3.2 Methods ................................................................................................................................ 21
3.3. Socio-Cultural Context........................................................................................................... 24
4. Results ........................................................................................................................................... 26
4.1 Part 1 – Aim 1 Research Tools............................................................................................... 26
Eye-Tracking for research in Cartography (Paper 1) ..................................................................... 27
Hypothesis software for research in Cartography (Paper 2)......................................................... 28
Tool for Research on Immersive Environments (Paper 3) ............................................................ 29
4.2 Part 2 – Aim 1 Experimental Research on Visual Perception................................................ 31
The effect of topographic backgrounds in the visual search process (Paper 4) ........................... 31
The effect of the size and shape of map symbols on perception (Paper 5).................................. 32
Experimental research on figure-ground perception in cartography (Paper 6)............................ 33
4.3 Part 2 – Aim 2 Experimental Research on Bivariate Mapping .............................................. 35
Experimental research on bivariate visualization for linear features (Paper 7)............................ 35
Experimental research on intrinsic and extrinsic bivariate visualization A (Paper 8) ................... 36
Experimental research on intrinsic and extrinsic bivariate visualization B (Paper 9) ................... 36
4.4 Part 2 – Aim 3 Experimental Research on Complex Spatial Tasks......................................... 38
Effect of clustering on Cartographic Visualization (Paper 10)....................................................... 38
Role of interaction in perception of 3D visualizations (Paper 11)................................................. 39
Association and preference of landmark visualization during pedestrian navigation (Paper 12) 40
5. Discussion, Conclusions and Future Work .................................................................................... 41
Discussion.......................................................................................................................................... 41
Conclusions and Future Work ........................................................................................................... 44
References............................................................................................................................................. 46
List of related publications.................................................................................................................... 50
List of Figures......................................................................................................................................... 51
Supplements.......................................................................................................................................... 52
9
1. Introduction!
Cartographic visualizations can potentially play a key role in many everyday human
experiences such as emergency management, military purposes, landscape management,
regional planning, education, and so on. Some methods are suboptimal or misleading in
specific contexts or not suitable for certain users or technology (Kraak et al., 2020b). A detailed
understanding of the fundamental principles of cartographic visualization is therefore vital.
The fundamentals employed in our research are outlined by the International Cartographic
Association (ICA), which is a key umbrella organization for cartographers at the international
level. In 2009, the ICA published its Research Agenda (RA) for Cartography and GI Science
(Virrantaus et al., 2009), specifying ten key future research areas in cartography. Two main
areas relate significantly to the research presented here: (i) the usability of maps and GI and
(ii) cartographic Theory (Figure 1). Both areas contain a number of subtopics which link to the
remainder of the defined research areas.
Figure 1: Selected research areas (yellow) and subtopics (grey) in the ICA’s research agenda
(adapted from Virrantaus et al., 2009).
These two cartographic research areas relate to the cartographic communication process in
that experimental studies are able to verify theoretical background and vice versa; knowledge
of the principles of cartographic communication is able to improve the methods and
approaches applied in experimental studies.
10
Empirical research which investigates the knowledge contained within the principles of
cartographic communication has been the subject of cartographic research since the
publication of Robinson’s work – The Look of Maps: An Examination of Cartographic Design in
1952 (Robinson, 1952). Initially, only cartographers (see below) attempted to describe and
explain the processes of cartographic communication. Cartography and other research
focused on cartographic visualization as stimuli is highly multidisciplinary, for example Worth
(1989), Konečný & Švancara (1996), MacEachren (1995), and Montello & Freundschuh (2004);
collaboration with experts in fields such as psychology, cognitive sciences, neurosciences and
related disciplines mentioned by MacEachren (1995) and Griffin et al. (2017) is therefore
essential. The importance of this approach is already mentioned in the work by Eckert – Die
Kartenwischenschaft (1925) – and emphasized by key organizations such as the ICA, first in
the Commission on User Experience (before 2019, the Commission on Use, User and Usability
Issues) – http://cartogis.ugent.be/kooms/UUI/), http://use.icaci.org – which has been active
since 1984 (Roth, 2021), and then in the Commission on Visualization, established in 1995
(MacEachren & Kraak, 1997), and the contemporary Commission on Cognitive Issues in
Geographic Information Visualization – https://cogvis.icaci.org/.
The ICA’s commission helps promote research directions in the above-mentioned areas
through the publication of relevant books (e.g., Mapping for a Sustainable World by
Kraak et al., 2020b), establishment of research agendas (e.g., Roth et al., 2017; Griffin et al.,
2017; Roth et al., 2022), creation of educational materials (e.g., chapters in the GIS Body of
Knowledge: Roth, 2017); Ooms & Skarlatidou, 2018; Stachoň et al., 2020), organization of
themed workshops (e.g., Stachoň & Kubíček, 2021; Griffin et al., 2021; Stachoň, 2022) and
other related activities.
1.1 Concept!
The scope of this work is based on several goals highlighted in the ICA’s commission materials:
the Commission on Cognitive Issues in Geographic Information Visualization
(https://cogvis.icaci.org/) promotes “the development of human-centred cartographic theory
and practice based on sound empirical findings on the use of cartographic displays for
spatiotemporal inference and decision-making”; the Commission on User eXperience
(http://use.icaci.org) emphasizes “the map user’s importance and involvement in evaluating
cartographic products for the improvement of usability”. The research presented here was
framed according to these goals and contains twelve studies which have two distinct research
areas according to specific content (Figure 2). The selected content aligns with the need to
converge cartographic theory and practice, emphasized by Kent (2018).
11
Figure 2: Schema of general research.
The research component consists of two parts. PART 1 discusses the aim to develop new
research tools suitable for collecting experimental data in user studies which apply a variety
of methodological approaches with respect to the unique nature of cartography and related
disciplines in cartographic visualization (Paper 1, Paper 2) and visualization of spatial
information for immersive virtual environments (IVEs) (Paper 3).
PART 2 pursues three main aims. Aim 1 – experimental research on the visual perception of
maps (Paper 4, Paper 5, Paper 6); Aim 2 – experimental research on bivariate mapping (Paper
7, Paper 8, Paper 9); Aim 3 – experimental research on complex spatial tasks focused on
individual user and group performance using 2D or 3D cartographic visualization methods and
associations (Paper 10, Paper 11, Paper 12). These three aims were not worked towards in
isolation but linked through several aspects.
The structure illustrated in Figure 2 is discussed in subsequent chapters.
1.2 Motivation!
Research in this area is highly interdisciplinary and aligned with increasing interest in
experimental approaches to maps and cartography from cartographers, psychologists,
geographers, designers, information scientists, data specialists and other researchers. This is
12
evident from the increasing number of papers published in reputable research databases –
Web of Science and Scopus (Figure 3).
Figure 3: Number of papers featuring experiments in cartography, published in Web of
Science (blue) and Scopus (yellow) since 1990.
Experimental research is just one of the possible approaches of studying usability in
cartography. Roth & Harrower (2008) presented a continuum of controlled experimentation
and usability testing. The research presented in this thesis focuses mainly on the controlled
experimentation area of the schema.
Experimental research in cartographic visualization primarily investigates the perceptual and
cognitive aspects of human interaction with cartographic visualizations (Konečný & Švancara,
1996). This area falls under the cognitive cartography research subdiscipline (Fabrikant &
Lobben, 2009). Montello (2002) conceptualized three main cognitive cartography research
areas: map design research (improving maps), map-psychology research (using simplified
cartographic visualizations as stimuli), and map-education research. To a large extent,
cognitive cartography adopts similar approaches and designs to psychology.
The research team at the Masaryk University (MUNI) Department of Geography, Faculty of
Science has been cooperating for a long time in this area with staff at the Department of
Psychology at the Faculty of Arts, Cabinet of Information Sciences and Librarianship (KISK),
Faculty of Informatics, and Department of Visual Computing. The team is also involved in
international collaboration with University of Zurich, Freie University of Berlin, University of
Wageningen, and Palacký University Olomouc. Work with these external partners has brought
together several original research projects and solutions, which are presented and discussed
in this text.
Linked to several international and national research activities, the research was conducted
under the national Research plan (2005–2011) – Dynamic Geovisualization in Crisis
0
20
40
60
80
100
120
140
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
13
Management (MSM0021622418), led by principal investigator (PI) prof. RNDr. Milan Konečný,
CSc., followed by the international project Dynamic Mapping Methods for Risk and Disaster
Management in the Era of Big Data (LTACH-17002, 2017–2019, PI prof. Konečný) and The
Effects of Socio-Cultural Factors and Writing Systems on Perception and Cognition of
Complex Visual Stimuli (GC19-09265J, 2018–2021, PI dr. Šašinka). Two projects focused on
education were conducted at the national level – Effects of Cartographic Visualizations in
Solving Practical and Instructional Spatial Tasks (MUNI/M/0846/2015, 2016–2018, PI assoc.
prof. Svatoňová) and Education in Collaborative Immersive Virtual Reality (TL03000346,
2020–2023, PI dr. Šašinka).
14
2. Theoretical!background!
Controlled experimentation delivers repeatable, generalizable findings in various scientific
fields and is associated more with theoretical research. The theoretical background for
cartography (and cartographic visualization) can be traced to the work by German geographer
Max Eckert (Eckert, 1925), yet to date it has not been fully explored. Even the general
theoretical framework of six visual variables postulated by Bertin (1967/1983) has not been
fully verified, and sometimes it has even been misinterpreted. Research into the process of
communicating spatial information to the map user from a map (cartographic visualization in
general) is a significant area of theoretical cartography. The work by Koláčný (1969) and
Ratajski (1972) established a new paradigm for cartographic communication/transmission
(Figure 4).
Figure 4: Theoretical model of the cartographic communication process in cartography
(adapted from Koláčný, 1969).
15
Figure 5: Cognitive elements in cartographic communication (Robinson & Petchenik, 1976)
A and B represent the map maker’s and map user’s general idea of reality, respectively.
The square area M represents the cartographer’s map viewed by the percipient.
M1 is the fraction of M already known to the map user.
M2 is the fraction of M that is newly comprehended by the map user.
M3 is the fraction that is not comprehended by the map user.
U is the increase in the map user’s notion of reality, unintended by the map maker.
The approach was later developed by Morrison (1976), Robinson & Petchenik (1976) (Figure
5), Board (1978), MacEachren (1995), Ormeling (1997) (Figure 7) and Morita (2004) and also
modified, discussed and criticized by Keates (1982) (Figure 6), Morrison (1976), MacEachren
(1995), Griffin (2021) and others.
Figure 6: Map communication model (Keates, 1982).
16
Figure 7: Model of spatial information transfer procedure published by (Ormeling, 1997).
The approaches by Konečný et al. (2006) and Griffin et al. (2017) incorporate the effects of
context in all parts of the process (Figure 8); Kent (2018) focuses on user feedback (Figure 9);
Kitchin et al. (2012) shift the attention from the map to the mapping process itself.
Figure 8: Model for operationalizing context in map use (adapted from Griffin et al., 2017).
17
Figure 9: Map communication model conceived by Kent (2018).
The most recent approaches explore new perspectives, but they cover only specific aspects,
not the full spectrum of cartographic characteristics. A detailed overview and classification of
theoretical approaches in cartography is presented by Griffin (2021).
One irrefutable aspect of cartography is its interdisciplinary nature. Cartographers address
matters linked to geography, physics, mathematics, statistics, informatics, psychology and
other fields of science. Cartography could be viewed as a behavioural science since it involves
the human aspect (Worth, 1989). Experimental cartography uses many methods and
techniques (Bandrova, T., Konečný, M., Zlatanova, 2014) which are normally applied in
sociology, human geography, and psychology (especially cognitive psychology) to investigate
specific information about human perception and cognition.
Despite drawing some criticisms, the communication model above is a universal and robust
theoretical concept. Its lack of experimental evidence, however, is its main weakness (Keates,
1982), although experimental research and development would be able to bring this approach
up to date.
18
3. Research!Methods!
To achieve our aims in the research, we applied a range of methods. This chapter first defines
the basic terminology of cartographic research, then describes the methods we applied and
any cultural aspects which affected our research results.
3.1 Terminology!
Contemporary cartographic literature has an extensive terminology for traditional and
commonly applied methods and approaches. However, cartography must respond to new
challenges posed mainly by developments in technology. New technologies introduce new
environments that require modification or extension of the existing terminology. 3D
visualizations and immersive virtual environments are areas of great interest to cartographers
and entail cartographic challenges in presenting spatial information.
The text here uses various terms, some which are still under consideration in the
(cartographic) community. The main source for cartographic terminology is the Cartography
& Visualization section in the GIS&T Body of Knowledge (https://gistbok.ucgis.org/knowledgearea/cartography-and-visualization).
However, selected terms are presented in a form which
is commonly used and understood.
Cartographic communication
The cartographic communication/transmission paradigm established in the 1960s is often
considered obsolete (see Chapter 2). Nevertheless, this paradigm is considered the most
successful in terms of unifying cartographic theory and practice (Kent, 2018), and it is still
mentioned in expert publications.
Cartographic communication/transmission of spatial information in our research is
understood in the broader sense noted by Ratajski (1972), constituting a process of
transmitting information about the spatial distribution of phenomena.
Cartographic visualization
Cartographic visualization generally deals with the visualization of spatial data. Alternative
terms such as geographic visualization, geovisualization (MacEachren & Kraak, 2001) and
others are frequently seen. For cartographic visualization, the definition of the term is
sufficient. The term geovisualization may be understood as a general umbrella term that
covers all aspects of the “visualization of geographic information (spatial, temporal or
attribute or a combination of all three), the process of creating interactive visualizations for
geographic analysis, using maps, map-like displays, multimedia, plots and graphs (also in
19
combination) to aid visual thinking and insight/hypotheses generation, and a perspective on
cartography” (Çöltekin et al., 2018). The term geovisualization is understood in the broader
sense described by Goodchild (2009), and the terms geographic visualization, geovisualization,
and cartographic visualization are considered synonymous.
Immersive Virtual Environments
The matter of terminology in immersive virtual environments in cartography is addressed in
Chapter CV-17 of the GIS Body Of Knowledge (BOK) (Stachoň et al., 2020) and the paper
Extended Reality in Spatial Sciences: A Review of Research Challenges and Future Directions by
Çöltekin et al. (2020). These publications are used as a terminological base for our current
research (Figure 10).
Figure 10: Definition of a Virtual Immersive Environment (Stachoň et al., 2020).
Maps / map-like visualizations
The term map has existed for centuries. However, its exact definition is continually evolving,
and most of the variations of this definition are not able to capture the full scope of
cartographic visualization and cartography as a scientific discipline. Most of the newly
established terms are reactions to technological developments – see the image map by
Vozenilek & Belka (2016) or 3D map by Herman (2018). We examined these definitions of
maps and 3D maps as stimuli for the current research.
20
A variety of definitions and classifications of maps serve as the primary output and means of
research in cartography. A comprehensive analysis by Andrews (1996) describes a continuum
extending from traditional detailed definitions:
“A map is a symbolized representation of geographical reality, representing selected features
or characteristics and resulting from the creative effort of its author’s execution of choices, and
is designed for use when spatial relationships are of primary relevance.” (Rystedt et al., 2011)
to broader definitions:
“map: a representation of spatial relationships” (Griffin, 2021),
or
“Maps are media designed to communicate generalized spatial information and relationships”
(Lapaine et al., 2021)
An interesting approach which considers real and virtual maps is presented by Moellering
(2007). Montello (2009) describes maps from a cognitive perspective, “maps and other
representations of geographic information serve the purpose of communicating information
about space, place and environment to people, or otherwise elicit ideas and thoughts”.
Discussion about the various aspects of map definition was published by Fairbairn et al.,
(2021).
3D maps
The term 3D map is even more actively debated (Herman, 2018). A very promising definition
was published by Bandrova & Yonov (2018): “A 3D map is a digital, mathematically defined,
three-dimensional virtual representation of the Earth and surfaces (e.g., luminous body),
objects and phenomena occurring in nature and society”.
This broad definition covers the majority of existing 3D models, and therefore, it has been
modified to include spatial reference: “A 3D map is a mathematically defined, threedimensional
representation of the surfaces, objects and phenomena occurring in nature and
society and uses a spatial reference system”.
Symbolization
Symbolization in cartography is generally defined as the visual encoding of data in a map or
diagram (Kraak et al., 2020a). Terms such as symbolization and symbol have been commonly
used in cartography since cartographers started developing their interest in cartographic
theory (see Robinson et al., 1995); these terms are linked to the semiological approach by
Bertin (1983). A cognitive-semiotic explanation of how maps work was later elaborated by
MacEachren (1995). In a classification designed by the scientist Charles Sanders Peirce, the
term symbol represents a meaning according to a rule, law or convention; however, this is not
true of most symbols in cartography. Keates (1982), MacEachren (1995) and other
21
cartographers refer to and discuss this confusing matter. In addition, both the map symbol
and map itself can be understood as signs.
General classifications use geometry to classify map symbols. For example, Morita (2011)
distinguishes three types of sign (points, lines and zones). Pravda & Kusendová (2004) apply a
different approach by classifying map signs as simple or complex according to the signified
meaning in geometric figures, lines and areas. Contrary to the classification designed and used
by Keates (1982), the categories of conventional signs and iconic signs appear to be universal
for use in cartography.
In our research, we understand a map symbol as the definition of map sign according to
Schlichtmann (2006) “A (map) sign consists of a conceptual item – a (sign) content or meaning
– and a perceivable item – an expression or sign”.
!
3.2 Methods!
Cartography is considered a visual language (Kraak et al., 2020a), and therefore, visual
perception is a fundamental component in most cartographic visualization methods. This is
highly influenced by the physiological nature of the visual organ and its perceptual limitations
(see chapter How Maps are Seen in MacEachren, 1995) and the cognitive processes which
enable the comprehension of presented information (see chapter How Maps are Understood
in MacEachren, 1995).
In the current research, limitations at the perceptual level are represented mainly by the
ability to capture only the visible part of the spectrum limited to the observer’s field of view,
colour spectrum shifts, and the asymmetry of visual searching (Figure 11; Wolfe, 2001).
Figure 11: Example of one possible relevant difference at the perceptual level in cartographic
visualization.
At the cognitive level, the user’s performance is affected by their ability to comprehend
abstraction, their mental rotation abilities and experience with the presented topic, and sociocultural
aspects.
The articles published for the first part of our research attempted to identify methods which
are crucial to experimental cartographic research and related fields. Most of these methods
were later incorporated into research tools (Paper 1, Paper 2, Paper 3).
22
An overview of methods for researching usability is found in the works by Nielsen (1993) and
Štěrba et al. (2015). In these works, we encounter two mixed research design approaches:
· An eye-tracking system used in a pilot study to examine the quality of an experimental
design, and application of the results to improve the design before incorporation into
large-scale data collection.
· Software (e.g., Hypothesis – Paper 2) used with a large-scale quantitative approach
and an eye-tracking method for subsequent analysis of results and adjustment or
modification of tasks.
Figure 12: Methods and approaches applied in the current research (adapted from Šterba et
al., 2014).
The experimental studies in the second part of our research investigated three essential
aspects of usability in cartographic visualization: effectiveness, efficiency and satisfaction.
Along with methods for measurement, these aspects are defined in ISO standards (Bevan et
al., 2016).
Throughout our research, we have applied objective and subjective methods (Figure 12),
mainly according to mixed research designs. The course of studies involved a combination of
large-scale experiments aimed at generalizable findings and discovering the differences in a
large sample of participants, and small-scale studies designed for more detailed investigation
of specific topics. The small-scale studies worked with small participant samples but employed
highly specific methods (e.g., eye-tracking) for detailed insight into observed phenomena and
clarification of results.
Quantitative methods were defined (PART 1) and applied (PART 2) to measure effectiveness
(error rate), efficiency (response time), and satisfaction (Likert scale) using the software
Hypothesis (Paper 2) and CIVE (Paper 3). Eye-tracking was used to identify in greater detail
the differences in cognitive processes (e.g., search strategies) involved in map-reading (Paper
23
1). Eye-tracking provides objective, detailed data on user eye-movements during exposure to
specific cartographic stimuli (e.g., Popelka & Brychtová, 2013). Subjective evaluation was
obtained from semi-structural interviews and focus groups with the participants of selected
experiments.
The second part is divided into three subsections based according to task complexity. Kubíček
(2012) indicates that cartographic literature offers multiple taxonomies of tasks involving
maps, outlining the ideas of Wehrend & Lewis (1990), who offer one of the most extensive
taxonomies of map tasks regarding number of items. Tasks which have much potential for use
in research are mentioned in the list below, ranked according to degree of difficulty (Wehrend
& Lewis, 1990):
• Identify – identify a general (visual) feature distinguished by individual
objects.
• Locate – locate items of certain value and recognizing absolute or relative
location. Where is it? Is it on the left or the right?
• Distinguish – recognize or distinguish two separate elements. This entails
discriminating the map field’s individual elementary visual elements
(between each other or from the underlying base), which should be easily
distinguishable by the user.
• Categorize – create categories or classifications, for example, according to
the colour, size, location and type (shape) of an object.
• Cluster – create of cluster groups according to the same, similar or related
type of graphical quality.
• Spatial distribution – describe the entire spatial pattern. This task is
closely related to cluster, also to locate and identify, but it is more
complex in character. The aim of locating clusters is to detect individual
cluster groups, whereas spatial distribution also requires a detailed
description of overall cluster organization.
• Rank – requires that objects be ordered or positioned according to similar
type (with respect to a pre-defined set of attributes).
• Compare – the process of finding similarities, differences or order.
• Associate – the process of linking graphical elements according to certain
relationships.
• Correlate – finding a direct connection between elements.
The experimental research in Papers 4, 5 and 6 focused mainly on identify and locate tasks.
The research in Papers 7, 8 and 9 added the distinguish task; Paper 10 added cluster; Paper
11 added compare; Paper 12 added associate.
24
3.3. Socio-Cultural!Context!
The importance of cultural background in cartography is mentioned by several authors, for
example Wood (1984) and MacEachren (1995), and is linked with the long-term cartographic
research goal of unifying cartographic visualization. Cartographic communication is
considered potentially universal (Modley, 1970), but obstacles exist in the different languages,
writing systems, and so on, used in cultures around the world.
Research into the socio-cultural differences of perception has a relatively long tradition in the
field of psychology. Various published works highlight the differences in basic perception of
visual materials (Deregowski, 1972) or the significant influence of language on segmentation
of the user’s world, which includes an understanding of terms to describe colour,
categorization of existing objects and phenomena, and methods for determining the relative
positions of objects in space (Imai, 2010). These differences are perhaps due to understanding
information through verbalization (Imai, 2010). Depending on the individual user, the same
graphic sign is possible to verbalize in various ways. This assumption is based on the
hypothesis of linguistic relativity, otherwise known as the Sapir–Whorf hypothesis. From
studies of different languages, American linguist Benjamin Lee Whorf concluded that thought
cannot be separated from language and that the categories expressed in the mother tongue
are synonymous with the concepts generated in thought (Imai, 2010), (Whorf, 1956). This
approach was later criticized, however, and cannot be considered universal; it is rather a
stimulus for investigating the effects of language on the use of maps by certain speakers,
especially concerning the general validity of mechanisms associated with the perception of
visual variables.
We assume that these principles and effects might also be widely applied or encountered
during the process of cartographic communication, for example in designing coding systems
and individual map symbols and in map use. Even cartographers are aware of this and argue
that the meanings of signs differ in space and time (MacEachren, 1995, p. 329). This has also
been documented and highlighted in comparative studies of cross-cultural topics in the
context of mapping (Morita, 2009) and articles which have illustrated the differences in map
characters used on German and Chinese maps (Angsüsser, 2014). Differing writing system
structures should also be highlighted in connection with the perception of signs by people of
different cultural backgrounds; the most striking example is Chinese script (the evolution of
Chinese characters over is well documented, e.g., Wei, 2014).
Chinese script comprises a system of characters which each carry a meaning. In the case of
Chinese maps, this often led to approaches where maps were created with no map legend, as
the characters in the map field already clarified intended meanings (Figure 13).
25
Figure 13: Example of a Chinese map – 12th century Song dynasty territorial map (retrieved
from https://www.viewofchina.com/ancient-chinese-maps/)
The current Anglo-European approach to creating maps for China and other countries often
leads, especially in descriptions, to a significant redundancy of information in the map field
and thus may disadvantage Chinese and other map users. We therefore employed participants
from European and Chinese populations in the research since we could expect that the
differences between people of these population groups would be significant.
26
4. Results!
The experimental research presented in this thesis is based on theoretical works in
cartography (e.g., system of visual variables described by Bertin, 1983), studies in
communication of cartographic information and utilitarian cartography (Koláčný, 1969;
MacEachren, 1995; others), experimental studies which used cartographic visualizations
(Flannery, 1971; Kimerling, 1975; Muller, 1979; Gilmartin, 1981; others), the research agendas
of the above-mentioned ICA Commissions, and theories from the fields of cognitive
psychology (Wagemans et al., 2012) and cross-cultural psychology (Nisbett, 2003; Imai, 2010).
The following points summarize the research activities:
̵ Cartographic visualizations are used as visual stimuli.
̵ Research progresses from atomized perceptual tasks to more complex cognitive
operations.
̵ Experiments are analysed according to selected user characteristics, such as
gender, age, domain knowledge, cultural background, etc.
̵ Tools and studies are designed for reproducibility.
The second part discusses and considers the primary research questions from a general and a
socio-cultural level.
4.1 Part!1!–!Aim!1!Research!Tools!
Cartographic visualizations are complex stimuli (Thorndyke & Stasz, 1979; Edler et al., 2014;
others) which complicate the possibility to maintain an experiment (Worth, 1989). Specific
research tools are therefore needed to not limit the research with various constraints. These
constraints are found in the large variety of stimuli (static vs. interactive, 2D vs. 3D, etc.),
methods and combinations of sensors available for research purposes. New
challenges/opportunities in experimental cartographic research have been reported in
various research papers and research agendas (Griffin, Robinson, et al., 2017; Roth, 2017;
Griffin, White, et al., 2017). To explore a complex set of aims, suitable tools must be created
for experimental research in cartography. Identifying these needs establishes a framework for
researching and developing the software tools that enable a wide range of methodological
approaches for use in experimental cartographic research and related fields. The necessity for
a combination of research methods is indicated by several researchers (Štěrba et al., 2015).
Roth et al. (2017) outline recommendations for expanding qualitative and mixed research
methods are outlined in their paper User Studies in Cartography: Opportunities for Empirical
Research on Interactive Maps and Visualizations.
27
The tools for research in cartography should support:
· Use of a range of stimuli with different characteristics relevant to specific tasks and
contexts.
· Recording of data from a large variety of methods and inputs.
· Collection and storage of a relatively large volume of data suitable for subsequent
statistical analysis. (Valid and reliable results can only be obtained with sufficient
participant sample sizes and specific research methods).
· Application of procedures in a controlled environment to enable the replicability of
experiments.
Eye-Tracking!for!research!in!Cartography!(Paper!1)!
Eye-tracking is a unique method for obtaining quantitative data which records the movements
of the human eye, i.e., for use in the analysis of user performance with visual stimuli. The
method is widely applicable, and the beginnings of its use date to the nineteenth century
(Popelka, 2018). Its application in cartography begins around the second half of the twentieth
century (e.g., Enoch, 1959; Morita, 1987), but greater use has occurred more recently in the
twenty-first century through technological developments that have simplified devices and
improved their availability. These observations inspired Paper 1, which focused on verifying
the accuracy of the inexpensive EyeTribe eye-tracking system (vs. the professional SMI RED
250 system, Figure 14) and its suitability with the Hypothesis software (see below). The results
demonstrated the usability of the EyeTribe system for experimental research in general and
verified the possibility to link the data recorded by the EyeTribe with records stored in the
Hypothesis system.
Figure 14: Comparison of recorded eye-movement data from EyeTribe (blue) and SMI RED
(red) (Paper 1).
28
Hypothesis!software!for!research!in!Cartography!(Paper!2)!!!!
To overcome or minimize any constraints on experimental research, an experimental
platform, called Hypothesis, was designed and developed at Masaryk University. Hypothesis
was developed in close cooperation with cartographers and psychologists at Masaryk
University and other partnered universities (details in Papers 1 and 2) to satisfy the demand
for tools suitable for experimental cartographic research and related disciplines. The platform
was designed as a universal and open environment which enables the use of stimuli with
different characteristics, linking of various data inputs, and a combination of methodological
approaches. The Hypothesis software allows researchers to conduct experiments at various
scales with a variety of stimuli in a controlled environment. It also enables large-scale studies
with small-scale approaches through specific methods such as eye-tracking or EEG to explore
certain research questions.
The development of the platform started under the Research Plan Geokrima
(MSM0021622418), which was developed provide geographic support for crisis management.
From an analysis of the research plan’s requirements, the use of proprietary tools was
abandoned and the development of a universal solution more suitable for user studies in
various fields commenced. The pilot MuTeP software was developed in the first phase (Šterba
et al., 2014). The Hypothesis software functions on two fundamental principles of sharing and
accumulation (Paper 2). The basic premise behind the software is the ability to store test
batteries, functionalities and research design templates created by researchers and to share
these with other research teams (Figure 15). Resources, tools and other created content are
therefore highly distinguishable and can also be developed gradually while their availability is
maintained.
Figure 15: Basic functional components of the Hypothesis platform (Paper 2).
The system has significant potential for use in setting up experimental cartographic studies. It
has been used several times for experimental studies conducted by local and international
29
authors; research groups from Norwegian University of Science and Technology (Opach et al.,
2017) and University of Zurich (Credé et al., 2019) have used the platform specifically for their
cartographic studies.
Tool!for!Research!on!Immersive!Environments!(Paper!3)!
To address the parallel matter of wider availability of systems for generating immersive virtual
environments, an experimental testing environment was developed. The tools for
experiments performed in this environment enable the use and handling of existing
geographic data and the recording of user interaction within a virtual geographic
environment. Virtual geographic environments provide a resource with significant added
value in several research areas, for example crisis management, landscape planning and
education, at both individual and multiple user levels; virtual geographic environments enable
the presentation of complex topics in multiple dimensions and areas. To leverage these
benefits, the Collaborative Immersive Virtual Environment – CIVE was created (PAPER 3), using
the Unity programming environment (Figure 16). Users of the CIVE environment have
reported many advantages in using the collaborative environment, one being the possibility
to “enter” or immerse themselves in the presented topic. The advantages mentioned by users
are partially limited by constraints resulting from cybersickness and few options for
personalizing avatars in the environment, which led to some drawbacks in communication.
Figure 16: Comparison of person in objective reality (left corner) and an avatar in a virtual
room (Paper 3).
An indisputable benefit of this environment is the transferability of conclusions drawn from
the virtual environment to the real environment. This quality of transferability is especially
30
beneficial in fields which carry inherent risk, such as crisis management, where few and limited
options for simulation and exercises in a real environment are feasible, in addition to being
expensive and demanding to organize. CIVE enables the recording of user activity and easy
collection of user feedback.
31
4.2 !Part!2!–!Aim!1!Experimental!Research!on!Visual!Perception!!
The first stage of the research explored the visual search process with simple tasks and visual
stimuli. We compared the different cartographic visualization methods for topographic
backgrounds (figure and effects of the background – Paper 4) and map symbols (effect of the
size and shape of the map symbol – Papers 5 and 6). The main research questions probed two
levels:
General level:
̵ How does the map background’s complexity affect the speed of performing various
tasks with map stimuli? (Paper 4)
̵ What is the limiting size that significantly increases the speed of performing search
tasks with map stimuli? (Paper 5)
̵ How does the shape of a point symbol affect the speed of performing search tasks
with map stimuli? (Paper 6)
Cultural level:
̵ Do any differences exist at the cross-cultural level in the speed of performing
various tasks with map stimuli? (Papers 5 and 6)
The!effect!of!topographic!backgrounds!in!the!visual!search!process!(Paper!4)!
The study (Paper 4) was originally prepared for pilot verification of the tool created for the
collection of user data. We used an orthophoto map and generalized topographic map as
stimuli because we expected perceptual differences between visual material which contains
generalized information (topographic map) and visual material which contains nongeneralized
information (aerial photograph). Although it was a pilot study, we observed
significant differences between the stimulus material with a topographic map and the
orthophoto as a topographic background (Figure 17); we also observed differences between
participants who had different levels of map literacy.
32
Figure 17: Different topographic backgrounds used in a study (Paper 4).
The results supported our hypothesis of the advantages of using generalized information
versus the non-generalized information in an aerial photograph. The second major finding was
that participants with greater map literacy performed better in the tasks.
The!effect!of!the!size!and!shape!of!map!symbols!on!perception!(Paper!5)!
Bertin (1983) established a set of consistent expectations regarding visual variables (e.g.,
larger = more) in cartography. These expectations were discussed but still need verification.
Paper 5 studied the two most prominent variables – size and shape. Generally, the most
efficient searches are those where a single basic feature defines the target and the distractors
are homogeneous (Wolfe, 2001). However, differences between certain visual variables are
expected.
The research followed on from some studies in psychology and a cartographic study by
Koláčný (1969), who established minimum sizes for map symbols at a given reading distance
on a large participant sample and the dependence of the number of map symbols on search
speed. The proposed experimental study measured the differences in perception of three
basic shapes of different sizes in a sample of European and Chinese participants. The study
contained two sub-parts, the first using no map background and the second using a map
background as a distractor with more complex scene/stimulus material (Figure 18).
33
Figure 18: Differences in the stimuli with and without a map background (Paper 5).
The results identified differences in the perception of basic shapes within and between each
studied population and with and without the use of a topographic background.
Experimental!research!on!figure-ground!perception!in!cartography!(Paper!6)!
The effect of figure and background is part of the system of visual hierarchy in each scene,
even in cartographic visualizations. It plays a role in all maps, but it is critical in thematic maps,
where themed information is prioritized over the topographic background. To experimentally
verify this effect, we designed two sets of symbols (Figure 19) and arranged them on a unified
map base with a location name in Greek to limit any effect of familiarity with a writing system.
Participants in sample groups from the Czech Republic and China then completed visual search
tasks focused on figures (symbols) and objects in the background (street crossings, etc.).
Differences were observed at both at the level of the figure-background (participants were
able to identify figures much more quickly) and the level of cultural background, being more
obvious with objects in the background. The existence of these differences was verified with
a cognitive style FLT (framed-line test) included in the experimental design. The results of the
study verified that significant differences between participants from different cultural
backgrounds exist.
34
Figure 19: Examples of stimuli used in an experiment (Paper 6).
35
4.3 !Part!2!–!Aim!2!Experimental!Research!on!Bivariate!Mapping!!
Bivariate mapping is defined as “a form of multivariate mapping specific to encoding two data
variables into a single product, for the purposes of investigating a relationship.”(Nelson, 2020)
and is the simplest and most used multivariate mapping method. Cartography has various
approaches to bivariate mapping. We investigated bivariate mapping for linear features
(Paper 7) and bivariate mapping for areal features (Papers 8 and 9).
The proposed research questions investigated two levels:
General level:
̵ Are the informational equivalent methods of bivariate cartographic visualization
equivalent at the level of perception? (Paper 7)
̵ Do any differences exist between bivariate cartographic visualizations which use
separable or integral encoding? (Paper 8, Paper 9)
Cultural level:
̵ Do any differences exist at the cross-cultural level in the speed of performing
different tasks with bivariate map stimuli? (Paper 9)
Experimental!research!on!bivariate!visualization!for!linear!features!(Paper!7)!
The study investigated the differences in cognitive load during tasks where bivariate variables
were used to visualize line data. Spatial data was represented with a simplified geographical
scenario simulating a map. Based on previous research, we compared the line element
variables for qualitative information (hue and shape) and quantitative information (size and
intensity). The final combinations were hue and size vs. shape and intensity (Figure 20). The
results highlighted the differences between visual variables used to visualize data and thus
indicated the importance of research into cognitive functions involved in performing map
tasks. The results also showed that visualization A (combination of colour hue and size)
compared to visualization B (colour value and symbol shape) was more effective in all tasks,
but only partially significant.
36
Figure 20: Examples of bivariate visualizations (Paper 7).
Experimental!research!on!intrinsic!and!extrinsic!bivariate!visualization!A!(Paper!8)!
Paper 8 presented a follow-up study to Paper 7 but instead focused on bivariate visualization
methods used for the purpose of visualizing values and uncertainty in the presented
phenomenon. We employed two bivariate cartographic visualization methods: extrinsic
visualization (shape and hue) and visualization (hue and intensity) (Figure 21). The results
indicated that participants performed worse with intrinsic bivariate visualizations at the
beginning of the test battery. The differences then stabilized to a minimum. The results from
eye-tracking revealed a higher cognitive load with the intrinsic legend, which the participants
were able to access during the test. The use of intrinsic bivariate visualization therefore
assumes an experienced target group of users and is limited to the visualization of a
phenomenon and its uncertainty. In the case of a lay user group, the concept seems rather
complicated. This effect would be removed in the follow-up study (Paper 9).
Figure 21: Bivariate visualizations used in a study (Paper 8).
Experimental!research!on!intrinsic!and!extrinsic!bivariate!visualization!B!(Paper!9)!
The follow-up study explored analogous methods of bivariate cartographic visualization
(separable and integral, Figure 22) but with the difference of visualizing two correlated
phenomena. Separable bivariate visualization proved faster and more accurate. A detailed
analysis using eye-tracking showed that reading the legend is more cognitively demanding
with intrinsic visualization. However, the observed differences were reduced significantly in
37
tasks which focused on parallel processing of two variables. Our findings, however, were only
verified on the target lay population. We could therefore assume that the results would be
different in a professional population sample.
Figure 22: Bivariate visualizations used in a study (Paper 9).
38
4.4 !Part!2!–!Aim!3!Experimental!Research!on!Complex!Spatial!Tasks!!
The final aim investigated more complex tasks linked to categorizing multivariate point
symbols (Paper 10), comparing 3D profiles using real-3D and pseudo-3D cartographic
visualizations (Paper 11), and associating landmark symbols for navigation purposes (Paper
12). The proposed research questions investigated two levels:
General level:
̵ Are there any differences in the perception of real-3D and pseudo-3D cartographic
visualizations? (Paper 11)
̵ What is the role of interaction in the perception of 3D visualizations? (Paper 11)
̵ Are there any differences in association and preferences between imagery and
iconic, pictorial and textual symbols? (Paper 12)
Cultural level:
̵ Are there any differences at the cross-cultural level in the preferred manner of
categorizing point multivariate symbols? (Paper 10)
Effect!of!clustering!on!Cartographic!Visualization!(Paper!10)!
For the experiment, figural multivariate map symbols were used to measure clustering, one
of the components of categorization (McCleary, 1975). The maps were intentionally created
to provoke categorization (Figure 23) in relation to the cognitive styles of participants from
Europe and China and the differences expected as a consequence of dissimilarity in culture.
Asian cultures tend to be more holistic, while Anglo-European cultures tend to be more
analytic. The inherent differences of holistic and analytic approaches were expected to have
an effect on clustering the map stimulus. The results highlighted the differences between the
research sample groups in performing the categorization process.
39
Figure 23: Example of regions (analytic – left, holistic – right) for different approaches to
clustering (Paper 10).
Role!of!interaction!in!perception!of!3D!visualizations!(Paper!11)!
Paper 10 examined the role of interaction in the perception of real-3D and pseudo-3D
cartographic visualizations. 3D terrain models (Figure 24) were presented as stimuli,
supplemented by a mental rotation test (MRT). The task was involved comparing a 2D terrain
profile with the 3D visualization.
The results indicated a minimal advantage of real-3D visualizations over pseudo-3D
visualizations. The main benefit in correct comparison in the 3D profile was interactivity,
suggesting that pseudo-3D visualization was sufficient for the given task.
Figure 24: Example of 3D visualization used in the study (Paper 11).
40
Association!and!preference!of!landmark!visualization!during!pedestrian!navigation!
(Paper!12)!
The paper studied the use of adaptive navigation landmark visualizations based on the user
familiarity with the environment. Four types of landmark map symbols were investigated
(Figure 25). The results provided strong evidence of preferences in users for associative imagebased
visualizations in the case of unfamiliar environments; in a familiar environment, users
preferred text-based visualizations. The results could be applied, for example, in mobile map
applications.
Figure 25: Example of the variation of landmark symbols used in the study (Paper 12).
41
5. Discussion,!Conclusions!and!Future!Work!
The development of experimental research in cartography is an opportunity for an objective
shift of theoretical knowledge in cartography and has significant application potential in
cartographic practice.
The aims of both parts of the research were accomplished:
̵ PART 1 – Experimental tools were created and applied successfully in the research.
̵ PART 2 – The experimental studies of selected aspects of cartographic visualizations
were accomplished and many differences at the perceptual and cultural levels were
observed in the results.
Discussion!
The results of the studies contribute to the creation of new ideas and redefinition of
established concepts in cartography in direct response to dynamically changing technologies
and cartographic communication environments. From the presented facts, the approach to
the process of cartographic communication requires both minor modifications and significant
changes as a consequence of developments in technology, greater understanding of
differences in socio-cultural aspects, and the expansion of knowledge in the functioning of
cognitive processes (Figure 26).
The results of the experimental studies indicate that understanding of the process of
cartographic communication according to the concept of Koláčný and subsequently published
works requires adjustments at several levels. Redefining the position of the cartographer as a
map creator is therefore also a consideration, as map creation is widely transferred to the
population beyond the geo-community and includes the means and options to interact with
the mapping process. Furthermore, the influence of technology on most of the processes
involved in the process of cartographic communication is significant and permits the
incorporation of feedback from map users. Therefore, a modified user-centered cartographic
communication schema was proposed.
42
Figure 26: User-centered schema of cartographic communication.
In the diagram (Figure 26), the map maker (S1) is influenced by certain factors (overlap with
map user S2 because through the availability of advanced tools, anyone today can create a
map) as they observe (i) and map (m) a real or fictitious phenomenon (U1) from reality (U) or
fiction (Uv). The technology (T) required to acquire the input data affects these observations.
Information is encoded (e) with the map language (L), and a cartographic visualization (M) in
real or virtual environment is created (v) using certain technology (T). The cartographic
visualization represents a part of reality which exists independently of the displayed
object/phenomenon and is ideally an image of its significant properties. The user (S2), who is
influenced by certain factors and socio-cultural aspects while using and understand (u) the
map language (L) and technology (T), studies and interprets (s) the visualization for distant
phenomena. The user then acts (a) according to their perceptions and may create an
alternative interpretation (U2, Un) of the visualized phenomenon. For the user, the difference
(e.g., U1 - U2 = d) in the resulting representation of the phenomenon may represent a loss or
distortion of information, but it might also create new information or add value. Actions can
be also performed only on the cartographic visualization.
43
In this process, interaction or feedback (f) between the map maker and the user may or may
not necessarily occur, and the user may participate in the map maker’s mapping or creative
process. The user can also use the map without understanding the map language, for example
as a picture on a wall. In this case, however, the user is not acting according to their
understanding. The context of map use (C) then includes the user, their characteristics and
socio-cultural background, the map language, the technology used, and finally, the influences
of the external environment and reality.
The scheme below (Figure 27) contains a classification and list of possible factors to
demonstrate the complexity of variables which influence the process of cartographic
communication. It was created based on application of the approach from Visual
Methodologies authored by Rose (2001).
Figure 27: Factors of Cartographic Communication.
The above diagram (Figure 27) indicates four levels of factors influencing process of
cartographic communication, i.e., technology, environment, performed activity (task), and
socio-cultural aspects. This definition was produced from work by Morita (2009), who studied
map use and mapping in the context of wayfinding, Konečný et al. (2006), who researched
adaptive visualization in cartography, Hanus (2019), who explored the factors which influence
44
map skills in education, and Wehrend & Lewis (1990), who authored one of the most extensive
taxonomies of research tasks for maps and number of items contained in maps.
This list is certainly not exhaustive, but it demonstrates the complex process of creating and
using cartographic visualizations and therefore the need for detailed comprehension of the
processes involved.
Conclusions!and!Future!Work!
The results of the research can potentially contribute to advancing the theoretical knowledge
of cartography at several levels:
1. New tools (Papers 1, 2 and 3) provide a catalyst for many empirical studies and aid the
verification of theoretical concepts and application of these tools in practice.
2. Observations from the studies introduce a new perspective on how cartographic
communication functions, especially in the differences during visual searches of map stimuli
(Papers 4, 5 and 6), the differences between informationally equivalent methods of bivariate
cartographic visualizations (Papers 7, 8 and 9), and the differences in complex processes
involved in using maps (Papers 10, 11 and 12).
3. The differences observed in population samples with different cultural backgrounds
(Papers 5, 6 and 10) support the significance of this factor in designing suitable maps keys and
cartographic visualization methods. In addition to thorough knowledge of the mapped topic,
map makers must also have thorough knowledge of the target user group; this is an integral
part of the data acquisition methodology, data processing method, and choice of cartographic
visualization method.
Both the theoretical research we performed and the results from our partial studies are
immediately applicable in practice. The tools (Papers 1, 2 and 3) designed for the research are
available for further study and development, and the experimental results can also be applied;
for example, we investigated the principles of perception (Papers 4 and 8) and their possible
effects in general map use and other areas, such as geographic support for crisis management.
At the general level, personalized cartographic visualizations can be generated according to
the user’s map literacy (Paper 9), familiarity with the environment (Paper 12), cultural
background (Papers 5, 6 and 10).
̵ A generalized map base is preferred over orthophotos in visual search tasks
(Paper 4).
̵ In designing a map key, the differences in mean detection time of shapes
(Paper 6) are a major consideration. Significant objects or events are often
represented by diamond-shaped symbols (e.g., Dymon & Mbobi, 2005), which has an
evident disadvantage during visual searches (Figure 28).
45
Figure 28: Mean detection time of shapes (Paper 5).
Six basic pillars emerge from this research on cartographic visualization:
1. Clear terminological anchoring of cartographic concepts in the existing
terminological apparatus of related fields and modern technologies (e.g., virtual
geographic environments).
2. The need to expand interdisciplinary cooperation, especially with experts in
cognitive psychology, applied informatics and other related fields.
3. Research into the processes of effective perception of generalized
information.
4. Research into the effective use of multivariate mapping methods.
5. Expansion of research into the basic mechanisms of 3D geovisualizations and
immersive virtual geographic environments.
6. Research into geography education.
In conclusion, cartographic visualization remains a key output of cartography as a science. The
results of the research demonstrate that different methods of cartographic visualization are
suitable in a variety of ways for certain tasks and users with differences in experience, cultural
background and other aspects. Cartographers should not therefore abandon experimental
research or studies to verify the efficiency and effectiveness of maps and satisfaction with
their use.
46
References!
Andrews,!J.!H.!(1996).!What!Was!a!Map?!The!Lexicographers!Reply.!Cartographica,!33,!1–11.!
Angsüsser,!S.!(2014).!Possible!Reasons!for!Size!Differences!Between!Point!Symbols!in!German!
and!Chinese!City!Maps.!In!T.!Bandrova!&!M.!Konecny!(Eds.),!Proceedings of the 5th
International Conference on Cartography & GIS!(pp.!268–277).!
Bandrova,!T.,!Koneèný,!M.,!Zlatanova,!S.!(2014).!Thematic!Cartography!for!the!Society.!In!Lecture
Notes in Geoinformation and Cartography.!(p.!357).!Springer.!https://doi.org/10.1007/978-
3-319-08180-9!
Bandrova,!T.,!&!Yonov,!N.!(2018).!3d!Maps!–!Cartographical!Aspects!3d!Maps!–!Cartographical!
Aspects.!In!T.!Bandrova!&!M.!Koneèný!(Eds.),!7th International Conference on Cartography
and GIS!(pp.!452–463).!
Bertin,!J.!(1983).!Semiology of graphics.!University!of!Wisconsin!Press.!
Bevan,!N.,!Carter,!J.,!Earthy,!J.,!Geis,!T.,!&!Harker,!S.!(2016).!New!ISO!Standards!for!Usability!,!
Usability!Reports!and!Usability!Measures!New!ISO!Standards!for!Usability!,!Usability!
Reports!and!Usability!Measures.!HCI 2016,!July 2016,!268–278.!
https://doi.org/10.1007/978-3-319-39510-4!
Board,!C.!(1978).!How!can!theories!of!cartographic!communication!be!used!to!make!maps!more!
effective?!International Yearbook of Cartography,!18,!41–49.!
Coltekin,!A.,!Janetzko,!H.,!&!Fabrikant,!I.!S.!(2018).!CV-35!-!Geovisualization.!In!he Geographic
Information Science & Technology Body of Knowledge.!University!Consortium!of!Geographic!
Information!Science.!https://doi.org/10.22224/gistbok/2018.2.6.!
Çöltekin,!A.,!Lochhead,!I.,!Madden,!M.,!Christophe,!S.,!Devaux,!A.,!Pettit,!C.,!Lock,!O.,!Shukla,!S.,!
Herman,!L.,!Stachoò,!Z.,!Kubíèek,!P.,!Snopková,!D.,!Bernardes,!S.,!&!Hedley,!N.!(2020).!
Extended!reality!in!spatial!sciences:!A!review!of!research!challenges!and!future!directions.!
ISPRS International Journal of Geo-Information,!9(7).!https://doi.org/10.3390/ijgi9070439!
Credé,!S.,!Thrash,!T.,!Hölscher,!C.,!&!Fabrikant,!S.!I.!(2019).!The!acquisition!of!survey!knowledge!
for!local!and!global!landmark!configurations!under!time!pressure.!Spatial Cognition &
Computation,!00(00),!1–30.!https://doi.org/10.1080/13875868.2019.1569016!
Deregowski,!J.!B.!(1972).!Pictorial!perception!and!culture.!Scientific American,!227(5),!82–88.!
Eckert,!M.!(1925).!Die!Kartenwissenschaft.!Vereinigung Wissenschaftlicher Verleger,!2.!
Edler,!D.,!Bestgen,!A.!K.,!Kuchinke,!L.,!&!Dickmann,!F.!(2014).!Grids!in!topographic!maps!reduce!
distortions!in!the!recall!of!learned!object!locations.!PLoS ONE,!9(5).!
https://doi.org/10.1371/journal.pone.0098148!
Enoch,!J.!(1959).!Effect!of!the!Size!of!a!Complex!Display!upon!Visual!Search.!JOURNAL OF THE
OPTICAL SOCIETY OF AMERICA,!49(3).!
Fabrikant,!S.!I.,!&!Lobben,!A.!(2009).!Cognitive!Issues!in!Geographic!Information!Visualization.!
Cartographica,!44(3),!139–143.!https://doi.org/10.3138/carto.44.3.iii!
Fairbairn,!D.,!Gartner,!G.,!&!Peterson,!M.!P.!(2021).!Epistemological!thoughts!on!the!success!of!
maps!and!the!role!of!cartography.!International Journal of Cartography,!7(3),!317–331.!
https://doi.org/10.1080/23729333.2021.1972909!
Flannery,!J.!J.!(1971).!THE!RELATIVE!EFFECTIVENESS!OF!SOME!COMMON!GRADUATED!POINT!
SYMBOLS!IN!THE!PRESENTATION.!Cartographica: The International Journal for Geographic
Information and Geovisualization,!8(2),!96–109.!https://doi.org/doi:10.3138/j647-1776-
745h-3667!
Gilmartin,!P.!P.!.!(1981).!Influences!of!Map!Context!on!Circle!Perception.!Annals of the Association
of American Geographers,!71(2),!253–258.!https://www.jstor.org/stable/2562795!
Goodchild,!M.!F.!(2009).!CARTOGRAPHY : GIS AND CARTOGRAPHY ( RELATIONSHIP BETWEEN ).!
https://people.geog.ucsb.edu/~good/papers/474B.pdf!
Griffin,!A.!(2021).!CV-01!-!Cartography!and!Science.!In!The Geographic Information Science &
Technology Body of Knowledge.!University!Consortium!of!Geographic!Information!Science.!
https://gistbok.ucgis.org/bok-topics/cartography-and-science!
47
Griffin,!A.,!Kubíèek,!P.,!Kettunen,!P.,!Roth,!R.!E.,!Delazari,!L.,!&!Stachoò,!Z.!(2021).!Workshop on
Adaptable Research Methods For Empirical Rresearch with Map Users.!
https://icaci.org/workshop-on-adaptable-research-methods-for-empirical-research-with-
map-users/!
Griffin,!A.!L.,!Robinson,!A.!C.,!&!Roth,!R.!E.!(2017).!Envisioning!the!future!of!cartographic!
research.!International Journal of Cartography,!3(sup1),!1–8.!
https://doi.org/10.1080/23729333.2017.1316466!
Griffin,!A.!L.,!White,!T.,!Fish,!C.,!Tomio,!B.,!Huang,!H.,!Sluter,!C.!R.,!Bravo,!J.!V.!M.,!Fabrikant,!S.!I.,!
Bleisch,!S.,!Yamada,!M.,!&!Picanço,!P.!(2017).!Designing!across!map!use!contexts:!a!research!
agenda.!International Journal of Cartography,!3(sup1),!90–114.!
https://doi.org/10.1080/23729333.2017.1315988!
Hanus,!M.!(2019).!Map Skills in Education : A Systematic Review of Terminology , Methodology ,
Map Skills in Education : A Systematic Review of Terminology , Methodology and Influencing
Factors!(Issue!August).!https://doi.org/10.33403/rigeo.591094!
Herman,!L.!(2018).!User Issues of Interactive 3D Geovisualizations.!Masaryk!University.!
Imai,!M.!(2010).!Jazyk a myšlení.!
Keates,!J.!S.!(1982).!Understanding Maps.!Longman.!
Kent,!A.!J.!(2018).!Form!Follows!Feedback:!Rethinking!Cartographic!Communication.!
Westminster Papers in Communication and Culture,!13(2),!96–112.!
https://doi.org/10.16997/wpcc.296!
Kimerling,!A.!J.!(1975).!A!Cartographic!Study!of!Equal!Value!Gray!Scales!for!Use!With!Screened!
Gray!Areas!A!Cartographic!Study!of!Equal!Value!Gray!Scales!for!Use!With!Screened!Gray!
Areas.!The American Cartographer,!2(2),!119–127.!
https://doi.org/10.1559/152304075784313313!
Kitchin,!R.,!Gleeson,!J.,!&!Dodge,!M.!(2012).!Unfolding!mapping!practices:!a!newepistemology!for!
cartography.!Transactions of the Institute of British Geographers.!
https://doi.org/0.1111/j.1475-5661.2012.00540.x!
Koláèný,!A.!(1969).!Utilitární!kartografie,!cesta!k!optimální!úèinnosti!kartografické!informace.!
Geodetický a Kartografický Obzor,!12,!301–308.!
Koneèný,!M.,!Friedmannová,!L.,!&!Stanìk,!K.!(2006).!An adaptive cartographic visualization for
support of the crisis management.!100–105.!
Koneèný,!M.,!&!Švancara,!J.!(1996).!(A)perception!of!the!Maps!by!Czech!School!Children.!
Proceedings of the ICA Seminar On,!8–10.!
Kraak,!M.-J.,!Roth,!R.!E.,!Ricker,!B.,!Kagawa,!A.,!&!Le!Sourd,!G.!(2020a).!MAPPING for a Sustainalble
World.!United!Nations,!ICA.!
Kraak,!M.-J.,!Roth,!R.,!Ricker,!B.,!Kagawa,!A.,!&!Le!Sourd,!G.!(2020b).!Mapping for a Sustainable
World.!the!United!Nations!and!the!International!Cartographic!Association.!
Kubíèek,!P.!(2012).!Habilitation Thesis.!University!of!Defence.!
Lapaine,!M.,!Midtb",!T.,!Gartner,!G.,!Bandrova,!T.,!&!Wang,!T.!(2021).!Definition!of!the!Map.!In!
Advances in Cartography and GIScience of the International Cartographic Association!(Issue!
December,!pp.!14–18).!
MacEachren,!A.,!&!Kraak,!M.-J.!(1997).!Exploratory!Cartographic!visualization.!In!Computers &
Geosciences!(Vol.!23,!Issue!4,!pp.!335–343).!Elsevier!Science!Ltd.!
MacEachren,!A.!M.!(1995).!How Maps Work: Representation, Visualization, and Design.!Guilford!
Press.!
MacEachren,!Alan!M,!&!Kraak,!M.!(2001).!Research!Challenges!in!Geovisualization.!Science,!28(1),!
1–11.!
McCleary,!G.!F.!(1975).!In!pursuit!of!the!map!user.!Proceedings, AUTO-CARTO 11,!238–250.!
Modley,!R.!(1970).!Universal!Symbols!and!Cartography.!Symposium on the Influence of the Map
User on Map Design.!https://files.eric.ed.gov/fulltext/ED045479.pdf!
Moellering,!H.!(2007).!EXPANDING!THE!ICA!CONCEPTUAL!DEFINITION!OF!A!MAP.!XXIII
International Cartographic Conference.!
https://icaci.org/files/documents/ICC_proceedings/ICC2007/html/Proceedings.htm!
Montello,!D.!R.!(2002).!Cognitive!map-design!research!in!the!twentieth!century:!Theoretical!and!
48
empirical!approaches.!Cartography and Geographic Information Science,!29(3),!283–304.!
https://doi.org/10.1559/152304002782008503!
Montello,!D.!R.!(2009).!Cognitive Geography.!Elsevier!Ltd.!
https://doi.org/https://doi.org/10.1016/B978-0-444-64046-8.00067-7!
Montello,!D.!R.,!&!Freundschuh,!S.!(2004).!Cognition!of!geographic!information.!A Research
Agenda for Geographic Information Science,!61–91.!
Morita,!T.!(2004).!Ubiquitous!Mapping!in!Tokyo.!Proceedings of the First International Joint
Workshop on Ubiquitous, Pervasive and Internet Mapping.!
Morita,!T.!(2011).!Reflections!on!the!Works!of!Jacques!Bertin#:!From!Sign!Theory!to!Cartographic!
Discourse.!The Cartographic Journal,!48(2),!86–91.!
https://doi.org/10.1179/000870411X13038059668604!
Morita,!T.!(2009).!CROSS-CULTURAL!ISSUES!IN!CONTEXT!BASED!MAPPING.!ICC 2009
Proceedings,!i.!
Morita,!T.!(1987).!Measurement!of!eye!movements!for!the!map!design!evaluation.!13th ICA
Conference, Vol. 1,!578–590.!
Morrison,!J.!(1976).!The!science!of!cartography!and!its!essential!processes.!International
Yearbook of Cartography.!
Muller,!J.!(1979).!Perception!of!continuously!shaded!maps.!Als of the Association of American
Geographers,!2(69),!240–249.!https://doi.org/10.1111/j.1467-8306.1979.tb01254.x!
Nelson,!J.!(2020).!CV-12!-!Multivariate!Mapping.!In!The Geographic Information Science &
Technology Body of Knowledge.!University!Consortium!of!Geographic!Information!Science.!
https://doi.org/10.22224/gistbok/2020.1.5!(link!is!external)!
Nielsen,!J.!(1993).!Usability Engineering.!Morgan!Kaufmann.!
Nisbett,!R.!(2003).!The Geography of thought: How Asians and Westerners think differently… and
why.!
Ooms,!K.,!&!Skarlatidou,!A.!(2018).!Usability!engineering!and!evaluation.!In!The Geographic
Information Science & Technology Body of Knowledge.!University!Consortium!of!Geographic!
Information!Science.!
Opach,!T.,!Popelka,!S.,!Dolezalova,!J.,!&!R"d,!J.!K.!(2017).!Star!and!polyline!glyphs!in!a!grid!plot!
and!on!a!map!display#:!which!perform!better#?!Cartography and Geographic Information
Science,!00(00),!1–20.!https://doi.org/10.1080/15230406.2017.1364169!
Ormeling,!F.!(1997).!Mass!media!maps!as!objects!of!cartographic!thought!and!theory.!In!W.!
Scharfe!(Ed.),!International Conference on Mass Media Maps.!Berliner!Geowissenschaftliche!
Abhandlungen!Reihe!C,!Kartographie,!Band!16.!
Popelka,!S.!(2018).!EYE-TRACKING ( NEJEN ) V KOGNITIVNÍ KARTOGRAFII Praktický průvodce
tvorbou.!Univerzita!Palackého!v!Olomouci.!https://doi.org/10.5507/prf.18.24453132!
Popelka,!S.,!&!Brychtová,!A.!(2013).!Eye-tracking!Study!on!Different!Perception!of!2D!and!3D!
Terrain!Visualisation.!Cartographic Journal,!50(3),!240–246.!
https://doi.org/10.1179/1743277413Y.0000000058!
Pravda,!J.,!&!Kusendová,!D.!(2004).!Počítačová tvorba tematických máp.!Prírodovedecká!fakulta!
Univerzity!Komenského!v!Bratislave.!
Ratajski,!L.!(1972).!The!Research!Structure!of!Theoretical!Cartography.!Geographia Polonica,!21,!
63–78.!
Robinson,!A.!H.!(1952).!The!Look!of!Maps:!An!Examination!of!Cartographic!Design.!University of
Wisconsin Press,!130.!
Robinson,!Arthur!H.,!Morrison,!J.!L.,!Muehrcke,!P.!C.,!Kimerling,!J.!A.,!&!Guptil,!S.!C.!(1995).!
Elements of Cartography!(6!th).!John!Wiley!and!Sons,!Inc.!
Robinson,!Arthur!H.,!&!Petchenik,!B.!B.!(1976).!The Nature of Maps.!University!of!Chicago!Press.!
Rose,!G.!(2001).!Visual Methodologies.!SAGE!Publication,!Ltd.!
Roth,!R.!E.!(2017).!User!Interface!&!User!Experience!Design.!In!The Geographic Information
Science & Technology Body of Knowledge.!University!Consortium!of!Geographic!Information!
Science.!
Roth,!R.!E.!(2021).!2021 UX Commission Business Meeting.!https://use.icaci.org/2021-ux-
business-meeting/!
49
Roth,!R.!E.,!Coltekin,!A.,!Delazari,!L.,!Denney,!B.,!Mendonça,!A.,!Shen,!J.,!Stachoò,!Z.,!&!Mingguang,!
W.!(2022).!Making!Maps!&!Visualizations!for!Mobile!Devices:!Research!Challenges!for!
Mobile-First!and!Responsive!Cartographic!Design.!Under Consideration.!
Roth,!R.!E.,!Çöltekin,!A.,!Delazari,!L.,!Filho,!H.!F.,!Griffin,!A.,!Hall,!A.,!Korpi,!J.,!Lokka,!I.,!Mendonça,!
A.,!Ooms,!K.,!&!van!Elzakker,!C.!P.!J.!M.!(2017).!User!studies!in!cartography:!opportunities!
for!empirical!research!on!interactive!maps!and!visualizations.!International Journal of
Cartography,!3(sup1),!61–89.!https://doi.org/10.1080/23729333.2017.1288534!
Roth,!R.!E.,!&!Harrower,!M.!(2008).!Addressing!map!interface!usability:!Learning!from!the!
Lakeshore!Nature!Preserve!Interactive!Map.!Cartographic Perspectives,!60,!46–66.!
https://doi.org/10.14714/CP60.231!
Rystedt,!B.,!Konecny,!M.,!Li,!L.,!Liebenberg,!E.,!Mcmaster,!R.,!Morita,!T.,!Virrantaus,!K.,!Wood,!M.,!&!
Ormeling,!F.!(2011).!A Strategic Plan for the International Cartographic Association.!
Schlichtmann,!H.!(2006).!rocesses!of!sign!production!in!map!making.!INTELEQTI Publishers,!19–
38.!
Stachoò,!Z.!(2022).!Workshop - Cartography for Immersive Environments.!
https://use.icaci.org/wp-content/uploads/2022/07/Workshop_8ICCGIS_web.pdf!
Stachoò,!Z.,!&!Kubíèek,!P.!(2021).!Workshop Immersive Cartography.!https://use.icaci.org/joint-
pre-conference-workshop-immersive-cartography-program/!
Stachoò,!Z.,!Kubíèek,!P.,!&!Herman,!L.!(2020).!CV-16!-!Virtual!and!Immersive!Environments.!In!
The Geographic Information Science & Technology Body of Knowledge!(3rd!Quarte).!
University!Consortium!of!Geographic!Information!Science.!
https://doi.org/10.22224/gistbok/2020.3.9!
Šterba,!Z.,!Šašinka,!È.,!Stachoò,!Z.,!Kubíèek,!P.,!&!Sascha,!T.!(2014).!Mixed!Research!Design!in!
Cartography#:!A!Combination!of!Qualitative!and!Quantitative!Approaches.!Kartographische
Nachrichten,!4(November!2015),!13.!
Štìrba,!Z.,!Šašinka,!È.,!Stachoò,!Z.,!Štampach,!R.,!&!Morong,!K.!(2015).!Selected!Issues!of!
Experimental!Testing!in!Cartography.!In!Selected Issues of Experimental Testing in
Cartography.!https://doi.org/10.5817/cz.muni.m210-7893-2015!
Thorndyke,!P.!W.,!&!Stasz,!C.!M.!(1979).!Individual!differences!in!knowledge!acquisition!from!
maps.!Rand Corporation, Santa Monica, Cal., Report,!175(R-2375-ONR),!137–175.!
Virrantaus,!K.,!Fairbairn,!D.,!&!Kraak,!M.!J.!(2009).!ICA!research!agenda!on!cartography!and!GI!
science.!Cartographic Journal,!46(2),!63–75.!https://doi.org/10.1179/000870409X459824!
Vozenilek,!V.,!&!Belka,!L.!(2016).!The!cartographic!concept!of!the!image!map.!International
Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences - ISPRS
Archives,!41(July),!605–610.!https://doi.org/10.5194/isprsarchives-XLI-B4-605-2016!
Wagemans,!J.,!Elder,!J.!H.,!Kubovy,!M.,!Palmer,!S.!E.,!Peterson,!M.!A.,!Singh,!M.,!&!von!der!Heydt,!R.!
(2012).!A!century!of!Gestalt!psychology!in!visual!perception:!I.!Perceptual!grouping!and!
figure–ground!organization.!Psychological Bulletin,!138(6),!1172–1217.!
https://doi.org/10.1037/a0029333!
Wehrend,!S.,!&!Lewis,!C.!(1990).!A!Problem-oriented!Classification!of!Visualization!Techniques.!
Proceedings of the 1st Conference on Visualization ‘,!139–143.!
Wei,!B.!(2014).!The!Origin!and!Evolvement!of!Chinese!Characters.!Gdańskie Studia Azji
Wschodniej,!05,!33–44.!https://doi.org/10.4467/23538724GS.14.003.2206!
Whorf,!B.!L.!(1956).!Language, Thought, and Reality!(J.!B.!Carroll!(ed.)).!
Wolfe,!J.!M.!(2001).!Asymmetries!in!visual!search:!an!introduction.!Perception & Psychophysics,!
63(3),!381–389.!https://doi.org/10.3758/BF03194406!
Wood,!D.!(1984).!Thoughts!on!the!cultural!context!of!cartographic!symbols.!Cartographica,!
483(21),!9–37.!http://doi.org/10.3138/B88T-68X3-L0J1-0226!
Worth,!C.!(1989).!Problems!with!experimental!research!in!map!design!-!a!case!study!Problems!
with!experimental!research!in!map!design!-!a!case!study.!The Cartographic Journal,!26(2),!
148–153.!https://doi.org/10.1179/caj.1989.26.2.148!
!
!
! !
50
List!of!related!publications!
!
Major publications which were published on the subject of cartographic visualization.
· Lang, M., Šašinka, Č., Stachoň, Z., & Chmelík, J., (2022). Brýle pro rozšířenou realitu: letová
podpora pilotů ultralehkých letadel - funkční prototyp.
· Snopková, D., Ugwitz, P., Stachoň, Z., Hladík, J., Juřík, J., Kvarda, O., & Kubíček, P. (2022).
Retracing evacuation strategy: A virtual reality game-based investigation into the influence of
building’s spatial configuration in an emergency. Spatial Cognition & Computation. Taylor &
Francis, 22, 1-2, pp. 30-50. ISSN 1387-5868. doi:10.1080/13875868.2021.1913497.
· Herman, L., Juřík, V., Snopková, D., Chmelík, J., Ugwitz, P., Stachoň, Z., Šašinka , Č., & Řezník,
T. (2021). A Comparison of Monoscopic and Stereoscopic 3D Visualizations: Effect on Spatial
Planning in Digital Twins. Remote Sensing. MDPI, 13, n. 15, pp. 1-21. ISSN 2072-4292.
doi:10.3390/rs13152976.
· Šašinka, Č., Chmelík, J., Šašinková, A., & Stachoň, Z. (2021). Metodika vzdělávání pomocí
aplikace VR-Pilots pro potřeby základního kurzu a piloty začátečníky.
· Ugwitz, P., Šašinková, A., Šašinka, Č., Stachoň, Z., & Juřík, V. (2021) Toggle Toolkit: A Tool for
Conducting Experiments in Unity Virtual Environments. Behavior Research Methods. Springer
New York LLC, 53, n. 4, pp. 1581-1591. ISSN 1554-351X. doi:10.3758/s13428-020-01510-4.
· Ugwitz, P., Herman, L., & Stachoň, Z. (2021). 3d Visualization of Historical Buildings: Methods,
Their Utility in the Tourism Industry and Beyond. Regionální rozvoj mezi teorií a praxí. Hradec
Králové: Civitas per Populi, o.p.s., 1, pp. 43-62. ISSN 1805-3246.
· Çöltekin, A., Lochhead, I., Madden, M., Christophe, S., Devaux, A., Pettit, C., Lock, O., Shukla,
S., Herman, L., Stachoň, Z., Kubíček, P., Snopková, D., Bernardes, S., & Hedley, N. (2020).
Extended Reality in Spatial Sciences: A Review of Research Challenges and Future Directions.
ISPRS International Journal of Geo-Information. Basel: MDPI, 9, n. 7, pp. 1-29. ISSN 2220-
9964. doi:10.3390/ijgi9070439.
· Juřík, V., Herman, L., Snopková, D., Galang, AJ., Stachoň, Z., Chmelík, J., Kubíček P., & Šašinka,
Č. (2020). The 3D hype : Evaluating the potential of real 3D visualization in geo-related
applications. PLoS ONE. San Francisco: Public Library of Science, 15, n. 5, pp. 1-18. ISSN 1932-
6203. doi:10.1371/journal.pone.0233353.
· Snopková, D., Švedová, H., Kubíček P., & Stachoň (2019). Navigation in Indoor Environments:
Does the Type of Visual Learning Stimulus Matter? ISPRS International Journal of GeoInformation.
Basel: MDPI, 8, n. 6, pp. 1-26. ISSN 2220-9964. doi:10.3390/ijgi8060251.
· Ugwitz, P., Juřík, V., Herman, L., Stachoň, Z., Kubíček P., & Šašinka, Č. (2019). Spatial Analysis
of Navigation in Virtual Geographic Environments. Applied Sciences. Basel, Switzerland:
MDPI, 9, n. 9, pp. 1-22. ISSN 2076-3417. doi:10.3390/app9091873.
· Herman, L., Kvarda, O., Stachoň, Z. (2018). Cheap and Immersive Virtual Reality: Application
in Cartography. In Zlatanova, S., Dragicevic, S., Sithole, G. ISPRS Archives of the
Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. XLII-4. Gottingen:
Copernicus GmbH, pp. 329-334. ISSN 1682-1750.
51
List!of!Figures!
Figure 1: Selected research areas (yellow) and subtopics (grey) in the ICA’s research agenda (adapted
from Virrantaus et al., 2009)................................................................................................................... 9
Figure 2: Schema of general research................................................................................................... 11
Figure 3: Number of papers featuring experiments in cartography, published in Web of Science (blue)
and Scopus (yellow) since 1990. ........................................................................................................... 12
Figure 4: Theoretical model of the cartographic communication process in cartography (adapted
from Koláčný, 1969). ............................................................................................................................. 14
Figure 5: Cognitive elements in cartographic communication (Robinson & Petchenik, 1976) ............ 15
Figure 6: Map communication model (Keates, 1982)........................................................................... 15
Figure 7: Model of spatial information transfer procedure published by (Ormeling, 1997)................ 16
Figure 8: Model for operationalizing context in map use (adapted from Griffin et al., 2017). ............ 16
Figure 9: Map communication model conceived by Kent (2018). ........................................................ 17
Figure 10: Definition of a Virtual Immersive Environment (Stachoň et al., 2020)................................ 19
Figure 11: Example of one possible relevant difference at the perceptual level in cartographic
visualization........................................................................................................................................... 21
Figure 12: Methods and approaches applied in the current research (adapted from Šterba et al.,
2014)...................................................................................................................................................... 22
Figure 13: Example of a Chinese map – 12th
century Song dynasty territorial map (retrieved from
https://www.viewofchina.com/ancient-chinese-maps/) ..................................................................... 25
Figure 14: Comparison of recorded eye-movement data from EyeTribe (blue) and SMI RED (red)
(Paper 1)................................................................................................................................................ 27
Figure 15: Basic functional components of the Hypothesis platform (Paper 2). .................................. 28
Figure 16: Comparison of person in objective reality (left corner) and an avatar in a virtual room
(Paper 3)................................................................................................................................................ 29
Figure 17: Different topographic backgrounds used in a study (Paper 4). ........................................... 32
Figure 18: Differences in the stimuli with and without a map background (Paper 5).......................... 33
Figure 19: Examples of stimuli used in an experiment (Paper 6).......................................................... 34
Figure 20: Examples of bivariate visualizations (Paper 7)..................................................................... 36
Figure 21: Bivariate visualizations used in a study (Paper 8). ............................................................... 36
Figure 22: Bivariate visualizations used in a study (Paper 9). ............................................................... 37
Figure 23: Example of regions (analytic – left, holistic – right) for different approaches to clustering
(Paper 10).............................................................................................................................................. 39
Figure 24: Example of 3D visualization used in the study (Paper 11). .................................................. 39
Figure 25: Example of the variation of landmark symbols used in the study (Paper 12). .................... 40
Figure 26: User-centered schema of cartographic communication...................................................... 42
Figure 27: Factors of Cartographic Communication. ............................................................................ 43
Figure 28: Mean detection time of shapes (Paper 5)............................................................................ 45
52
Supplements!!
The list of attached publications:
Paper 1
Popelka, S., Stachoň, Z., Šašinka, Č., & Doležalová, J. (2016). Eyetribe Tracker Data Accuracy
Evaluation and Its Interconnection with Hypothesis Software for Cartographic Purposes.
Computational Intelligence and Neuroscience, Vol. 2016, Special Issue, February, Article ID 9172506.
https://doi.org/10.1155/2016/9172506. (WoS JIF QUARTILE: Q2)
Abstract
The mixed research design is a progressive methodological discourse that combines the advantages of
quantitative and qualitative methods. Its possibilities of application are, however, dependent on the efficiency
with which the particular research techniques are used and combined. The aim of the paper is to introduce the
possible combination of Hypothesis with EyeTribe tracker. The Hypothesis is intended for quantitative data
acquisition and the EyeTribe is intended for qualitative (eye-tracking) data recording. In the first part of the paper,
Hypothesis software is described. The Hypothesis platform provides an environment for web-based computerized
experiment design and mass data collection. Then, evaluation of the accuracy of data recorded by EyeTribe
tracker was performed with the use of concurrent recording together with the SMI RED 250 eye-tracker. Both
qualitative and quantitative results showed that data accuracy is sufficient for cartographic research. In the third
part of the paper, a system for connecting EyeTribe tracker and Hypothesis software is presented. The
interconnection was performed with the help of developed web application HypOgama. The created system uses
open-source software OGAMA for recording the eye-movements of participants together with quantitative data
from Hypothesis. The final part of the paper describes the integrated research system combining Hypothesis and
EyeTribe.
Paper 2
Šašinka, Č., Morong, K., & Stachoň, Z. (2017). The Hypothesis Platform : An Online Tool for
Experimental Research into Work with Maps and Behavior in Electronic Environments. ISPRS
International Journal of Geo-Information, 6(12). https://doi.org/10.3390/ijgi6120407. (WoS JIF
QUARTILE: Q2)
Abstract
The article presents a testing platform named Hypothesis. The software was developed primarily for the purposes
of experimental research in cartography and psychological diagnostics. Hypothesis is an event-logger application
which can be used for the recording of events and their real-time processing, if needed. The platform allows for
the application of Computerized Adaptive Testing. The modularity of the platform makes it possible to integrate
various Processing.js-based applications for creation and presentation of rich graphic material, interactive
animations, and tasks involving manipulation with 3D objects. The Manager Module allows not only the
administration of user accounts and tests but also serves as a data export tool. Raw data is exported from the
central database in text format and then converted in the selection module into a format suitable for statistical
analysis. The platform has many functions e.g., the creation and administration of tasks with real-time interaction
between several participants (“multi-player function”) and those where a single user completes several tests
simultaneously (“multi-task function”). The platform may be useful e.g., for research in experimental economics
53
or for studies involving collaborative tasks. In addition, connection of the platform to an eye-tracking system is
also possible.
Paper 3
Šašinka, Č., Stachoň, Z. (corresponding author), Sedlák, M., Chmelík, J., Herman, L., Kubíček, P.,
Strnadová, A., Doležal, M., Tejkl, H., Urbánek, T., Svatoňová, H., Ugwitz, P., & Juřík, V. (2019).
Collaborative Immersive Virtual Environments for Education in Geography. ISPRS International
Journal of Geo-Information, 8(1). https://doi.org/10.3390/ijgi8010003. (WoS JIF QUARTILE: Q2)
Abstract
Immersive virtual reality (iVR) devices are rapidly becoming an important part of our lives and forming a new way
for people to interact with computers and each other. The impact and consequences of this innovative technology
have not yet been satisfactory explored. This empirical study investigated the cognitive and social aspects of
collaboration in a shared, immersive virtual reality. A unique application for implementing a collaborative
immersive virtual environment (CIVE) was developed by our interdisciplinary team as a software solution for
educational purposes, with two scenarios for learning about hypsography, i.e., explanations of contour line
principles. Both scenarios allow switching between a usual 2D contour map and a 3D model of the corresponding
terrain to increase the intelligibility and clarity of the educational content. Gamification principles were also
applied to both scenarios to augment user engagement during the completion of tasks. A qualitative research
approach was adopted to obtain a deep insight into the lived experience of users in a CIVE. It was thus possible
to form a deep understanding of very new subject matter. Twelve pairs of participants were observed during their
CIVE experience and then interviewed either in a semistructured interview or a focus group. Data from these three
research techniques were analyzed using interpretative phenomenological analysis, which is research method for
studying individual experience. Four superordinate themes—with detailed descriptions of experiences shared by
numerous participants—emerged as results from the analysis; we called these (1) Appreciation for having a
collaborator, (2) The Surprising “Fun with Maps”, (3) Communication as a challenge, and (4) Cognition in two
realities. The findings of the study indicate the importance of the social dimension during education in a virtual
environment and the effectiveness of dynamic and interactive 3D visualization.
Paper 4
Konečný, M., Kubíček, P., Stachoň, Z. (corresponding author), & Šašinka Č. (2011). The usability of
selected base maps for crises management: users' perspectives. SpringerLink, Applied Geomatics.
Springer, 2011/3, pp. 189–198. ISSN 1866-9298. https://doi.org/10.1007/s12518-011-0053-1. (WoS
JIF QUARTILE: Q3)
Abstract
Cartography has become an important tool for supporting decision-making processes in the field of crisis
management. Maps (or GIS) can be used for solving various problems, e.g. the localization of accident site, the
delimitation of endangered areas, the formulation of evacuation plans and others. People involved in decision
making processes use various procedures to solve these problems. However, a suitable and efficient form of
cartographic support for particular situations and crisis management cycle stages is still missing. The main goal
of the experiment was twofold. First, we wanted to assess the interdisciplinary (cartography–psychology) webbased
testing environment and achieve the first usability results. Second, the use of different cartographic base
map representations was analysed in order to judge the efficiency for specific situations. Testing was focused on
various types of tasks, e.g. simple sign selection, the possibility of memorizing important information from the
map and the choice of the optimal evacuation route. The overall testing was performed within the interactive
web environment, based on predefined templates, automatically recorded and calibrated against the evaluation
54
of the pretest users’ abilities. Preliminary testing results provide valuable inputs concerning the usability of
selected base maps for supporting decision-making processes during various crisis situations.
Paper 5
Stachoň, Z., Šašinka, Č., Čeněk, J., Angsüesser, S. Kubíček, P., Štěrba, Z., Bilíková, M. (2018). Effect of
Size, Shape and Map Background in Cartographic Visualization: Experimental Study on Czech and
Chinese Populations. ISPRS International Journal of Geo-Information, 7/427, pp. 1-15. ISSN 2220-
9964. https://doi.org/10.3390/ijgi7110427. (WoS JIF QUARTILE: Q2)
Abstract
This paper deals with the issue of the perceptual aspects of selected graphic variables (specifically shape and size)
and map background in cartographic visualization. The continued experimental study is based on previous
findings and the presupposed cross-cultural universality of shape and size as a graphic variable. The results bring
a new perspective on the usage of shape, size and presence/absence of background as graphic variables, as well
as a comparison to previous studies. The results suggest that all examined variables influence the speed of
processing. Respondents (Czech and Chinese, N = 69) identified target stimuli faster without a map background,
with larger stimuli, and with triangular and circular shapes. Czech respondents were universally faster than
Chinese respondents. The implications of our research were discussed, and further directions were outlined.
Paper 6
Stachoň, Z., Šašinka, Č., Čeněk, J., Štěrba, Z., Angsüesser, S., Fabrikant, S., I., Štampach, R., & Morong,
K. (2019). Cross-cultural differences in figure–ground perception of cartographic stimuli. Cartography
and Geographic Information Science, 46(1), 82-94. https://doi.org/10.1080/15230406.2018.1470575.
(WoS JIF QUARTILE: Q3)
Abstract
This article reports on an empirical study investigating cultural differences in the visuospatial perception and
cognition of qualitative point symbols shown on reference maps. We developed two informationally equivalent
symbol sets depicted on identical reference maps that were shown to Czech and Chinese map readers. The
symbols varied in visual contrast with respect to the base map. Our empirical results suggest the existence of
cultural influences on map reading, but not in the predicted direction based on the previous cross-cultural
studies. Our findings stress the importance of considering the cultural background of map readers, especially
when designing reference maps aimed for global online use.
Paper 7
Kubíček, P., Šašinka, Č., Stachoň, Z., Štěrba, Z., Apeltauer, J., & Urbánek, T. (2017). Cartographic
Design and Usability of Visual Variables for Linear Features. Cartographic Journal, 54(1), 91-102.
https://doi.org/10.1080/00087041.2016.1168141. (WoS JIF QUARTILE: Q4)
Abstract
This article addresses the measurement and assessment of response times and error rates in map-reading tasks
relative to various modes of linear feature visualization. In a between-subject design study, participants
completed a set of map-reading tasks generated by approaches to a traffic problem. These entailed quick and
correct decoding of graphically represented quantitative and qualitative spatial information. The tasks first
involved the decoding of one graphic variable, then of two variables simultaneously. While alternative
55
representations of qualitative information included colour hue and symbol shape, the quantitative information
was communicated either through symbol size or colour value. In bivariate tasks, quantitative and qualitative
graphical elements were combined in a single display. Individual differences were also examined. The concept of
cognitive style partially explains the variability in people’s perception and thinking, describing individual
preferences in object representation and problem-solving strategies. The data obtained in the experiment suggest
that alternative forms of visualization may have different impacts on performance in map-reading tasks: colour
hue and size proved more efficient in communicating information than shape and colour value. Apart from this,
it was shown that individual facets of cognitive style may affect task performance, depending on the type of
visualization employed.
Paper 8
Šašinka, Č., Stachoň, Z., Kubíček, P., Tamm, S., Matas, A., & Kukaňová, M. (2018). The Impact of
Global/Local Bias on Task-solving in Map-related Tasks Employing Extrinsic and Intrinsic Visualization
of Risk Uncertainty Maps. The Cartographic Journal, ISSN 0008-7041.
https://doi:10.1080/00087041.2017.1414018. (WoS JIF QUARTILE: Q4)
Abstract
The form of visual representation affects both the way in which the visual representation is processed and the
effectiveness of this processing. Different forms of visual representation may require the employment of different
cognitive strategies in order to solve a particular task; at the same time, the different representations vary as to
the extent to which they correspond with an individual’s preferred cognitive style. The present study employed a
Navon-type task to learn about the occurrence of global/local bias. The research was based on close
interdisciplinary cooperation between the domains of both psychology and cartography. Several different types
of tasks were made involving avalanche hazard maps with intrinsic/extrinsic visual representations, each of them
employing different types of graphic variables representing the level of avalanche hazard and avalanche hazard
uncertainty. The research sample consisted of two groups of participants, each of which was provided with a
different form of visual representation of identical geographical data, such that the representations could be
regarded as ‘informationally equivalent’. The first phase of the research consisted of two correlation studies, the
first involving subjects with a high degree of map literacy (students of cartography) (intrinsic method: N = 35;
extrinsic method: N = 37). The second study was performed after the results of the first study were analyzed. The
second group of participants consisted of subjects with a low expected degree of map literacy (students of
psychology; intrinsic method: N = 35; extrinsic method: N = 27).The first study revealed a statistically significant
moderate correlation between the students’ response times in extrinsic visualization tasks and their response
times in a global subtest (r = 0.384, p < 0.05); likewise, a statistically significant moderate correlation was found
between the students’ response times in intrinsic visualization tasks and their response times in the local subtest
(r = 0.387, p < 0.05). At the same time, no correlation was found between the students’ performance in the local
subtest and their performance in extrinsic visualization tasks, or between their scores in the global subtest and
their performance in intrinsic visualization tasks. The second correlation study did not confirm the results of the
first correlation study (intrinsic visualization/‘small figures test’: r = 0.221; extrinsic visualization/‘large figures
test’: r = 0.135). The first phase of the research, where the data was subjected to statistical analysis, was followed
by a comparative eye-tracking study, whose aim was to provide more detailed insight into the cognitive strategies
employed when solving map-related tasks. More specifically, the eye-tracking study was expected to be able to
detect possible differences between the cognitive patterns employed when solving extrinsic- as opposed to
intrinsic visualization tasks. The results of an exploratory eye-tracking data analysis support the hypothesis of
different strategies of visual information processing being used in reaction to different types of visualization.
56
Paper 9
Šašinka, Č., Stachoň, Z. (corresponding author), Čeněk, J., Šašinková, A., Popelka, S., Ugwitz, P., &
Lacko, D. (2021). A Comparison of the Performance on Extrinsic and Intrinsic Cartographic
Visualizations through Correctness, Response Time and Cognitive Processing. PLoS ONE, 16(4).
https://doi.org/10.1371/journal.pone.0250164. (WoS JIF QUARTILE: Q2)
Abstract
The aim of this study was to compare the performance of two bivariate visualizations by measuring response
correctness (error rate) and response time, and to identify the differences in cognitive processes involved in mapreading
tasks by using eye-tracking methods. The present study is based on our previous research and the
hypothesis that the use of different visualization methods may lead to significant cognitive-processing differences.
We applied extrinsic and intrinsic visualizations in the study. Participants in the experiment were presented maps
which depicted two variables (soil moisture and soil depth) and asked to identify the areas which displayed either
a single condition (e.g., “find an area with low soil depth”) or both conditions (e.g., “find an area with high soil
moisture and low soil depth”). The research sample was composed of 31 social sciences and humanities university
students. The experiment was performed under laboratory conditions, and Hypothesis software was used for data
collection. Eye-tracking data were collected for 23 of the participants. An SMI RED-m eye-tracker was used to
determine whether either of the two visualization methods was more efficient for solving the given map-reading
tasks. Our results showed that with the intrinsic visualization method, the participants spent significantly more
time with the map legend. This result suggests that extrinsic and intrinsic visualizations induce different cognitive
processes. The intrinsic method was observed to generally require more time and led to higher error rates. In
summary, the extrinsic method was found to be more efficient than the intrinsic method, although the difference
was less pronounced in the tasks which contained two variables, which proved to be better suited to intrinsic
visualization.
Paper 10
Lacko, D., Šašinka, Č., Čeněk, J., Stachoň, Z., & Lu W.-L. (2020). Cross-Cultural Differences in Cognitive
Style, Individualism/Collectivism and Map Reading between Central European and East Asian
University Students. Studia Psychologica, 62(1), 23-42. http://dx.doi.org/10.31577/sp.2020.01.789.
(WoS JIF QUARTILE: Q4)
Abstract
The article examines cross-cultural differences encountered in the cognitive processing of specific cartographic
stimuli. We conducted a comparative experimental study on 98 participants from two different cultures, the first
group comprising Czechs (N = 53) and the second group comprising Chinese (N = 22) and Taiwanese (N = 23). The
findings suggested that the Central European participants were less collectivistic, used similar cognitive style and
categorized multivariate point symbols on a map more analytically than the Asian participants. The findings
indicated that culture indeed influenced human perception and cognition of spatial information. The entire
research model was also verified at an individual level through structural equation modelling (SEM). Path analysis
suggested that individualism and collectivism was a weak predictor of the analytic/holistic cognitive style. Path
analysis also showed that cognitive style considerably predicted categorization in map point symbols.
Paper 11
Kubíček, P., Šašinka, Č., Stachoň, Z., Herman, L., Juřík, V., Urbánek, T., & Chmelík, J. (2019).
Identification of altitude profiles in 3D geovisualizations: the role of interaction and spatial abilities.
57
International Journal of Digital Earth, 12(2), 156-172.
https://doi.org/10.1080/17538947.2017.1382581. (WoS JIF QUARTILE: Q1)
Abstract
Three-dimensional geovisualizations are currently pushed both by technological development and by the
demands of experts in various applied areas. In the presented empirical study, we compared the features of real
3D (stereoscopic) versus pseudo 3D (monoscopic) geovisualizations in static and interactive digital elevation
models. We tested 39 high-school students in their ability to identify the correct terrain profile from digital
elevation models. Students’ performance was recorded and further analysed with respect to their spatial abilities,
which were measured by a psychological mental rotation test and think aloud protocol. The results of the study
indicated that the influence of the type of 3D visualization (monoscopic/stereoscopic) on the performance of the
users is not clear, the level of navigational interactivity has significant influence on the usability of a particular 3D
visualization, and finally no influences of the spatial abilities on the performance of the user within the 3D
environment were identified.
Paper 12
Zhu, L., Shen, J., Zhou, J., Stachoň, Z., Hong, S., & Wang, X., (2022). Personalized landmark adaptive
visualization method for pedestrian navigation maps: Considering user familiarity. Transactions in
GIS. Wiley, 26/2, pp. 669-690. ISSN 1361-1682. https://doi.org/10.1111/tgis.12877. (WoS JIF
QUARTILE: Q3)
Abstract
Landmark-based pedestrian navigation can assist pedestrians in navigating successfully. Previous studies have
investigated the factors affecting the cognitive efficiency of landmark visualization in terms of both the visual
salience of landmarks and the personal characteristics of users. However, empirical studies and applications that
consider the influence of spatial familiarity on landmark representation are limited. In this article, we propose a
personalized landmark adaptive visualization method for pedestrian navigation maps considering user
familiarity. We first explore the influence of spatial familiarity on landmark salience and symbols using cognitive
experiments. The results showed that unfamiliar people preferred strong visual salience landmarks and imagebased
symbols, while familiar people preferred strong semantic salience landmarks and text-based symbols.
Based on these results, a mathematical model of landmark salience for selecting personalized landmarks is
proposed, and association rules between landmark salience and symbols are mined. Finally, the framework of a
landmark visualization method is proposed based on the rules. To verify the effectiveness of the proposed method,
a prototype system is developed, and a comparative experiment is conducted with a Baidu map. Experimental
results showed that the proposed method has direct practical implications for the development of pedestrian
navigation systems, depending on different target users.
Research Article
EyeTribe Tracker Data Accuracy Evaluation and
Its Interconnection with Hypothesis Software for
Cartographic Purposes
Stanislav Popelka,1
Zdenjk Stacho^,2
Henjk Šašinka,2
and Jitka DoleDalová1
1
Palack´y University, Olomouc, 77146 Olomouc, Czech Republic
2
Masaryk University, 61137 Brno, Czech Republic
Correspondence should be addressed to Stanislav Popelka; standa.popelka@gmail.com
Received 25 November 2015; Revised 8 February 2016; Accepted 22 February 2016
Academic Editor: Ying Wei
Copyright © 2016 Stanislav Popelka et al. This is an open access article distributed under the Creative Commons Attribution
License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly
cited.
The mixed research design is a progressive methodological discourse that combines the advantages of quantitative and qualitative
methods. Its possibilities of application are, however, dependent on the efficiency with which the particular research techniques
are used and combined. The aim of the paper is to introduce the possible combination of Hypothesis with EyeTribe tracker. The
Hypothesis is intended for quantitative data acquisition and the EyeTribe is intended for qualitative (eye-tracking) data recording.
In the first part of the paper, Hypothesis software is described. The Hypothesis platform provides an environment for web-based
computerized experiment design and mass data collection. Then, evaluation of the accuracy of data recorded by EyeTribe tracker
was performed with the use of concurrent recording together with the SMI RED 250 eye-tracker. Both qualitative and quantitative
results showed that data accuracy is sufficient for cartographic research. In the third part of the paper, a system for connecting
EyeTribe tracker and Hypothesis software is presented. The interconnection was performed with the help of developed web
application HypOgama. The created system uses open-source software OGAMA for recording the eye-movements of participants
together with quantitative data from Hypothesis. The final part of the paper describes the integrated research system combining
Hypothesis and EyeTribe.
1. Introduction
The paper presents methodological-technical approach combining
quantitative and qualitative methods which are based
on specific technical tools. The aim of this paper is to
introduce the newly developed technical research system and
results of its validation: specifically, the creation and empirical
verification of an interconnection of a web-based platform
Hypothesis with an EyeTribe eye-tracking system connected
to open-source software OGAMA. The interconnection was
done by the creation of a new web application HypOgama.
The introduction of the paper discusses the methodology
and mixed-research design (combination of quantitative
and qualitative, resp., explorative methods) in the area of
cognitive visualization and cartography. The paper consists of
three parts which are ordered due the logic and procedure of
the research system creation and verification. The first part is
focused on the presentation of a tool for mass data collection:
web-based platform Hypothesis. The second part of the paper
presents the new low-cost eye-tracking system EyeTribe,
which allows efficient realization of qualitative, respectively,
explorative studies. In this part, close attention is paid to
empirical study verifying the truthfulness of the low-cost EyeTribe
tracker in comparison with SMI RED 250 system. The
final part of the paper describes the research system which
combines and integrates above-mentioned tools. Part of this
last section is also an illustration of possible empirical study,
where the interconnection of Hypothesis and EyeTribe for
cartographic and psychology research is presented. However
this case study is only an example of how the integrated
Hindawi Publishing Corporation
Computational Intelligence and Neuroscience
Volume 2016,Article ID 9172506, 14 pages
http://dx.doi.org/10.1155/2016/9172506
2 Computational Intelligence and Neuroscience
research system and HypOgama application works, and it
should only illustrate the procedure of conducting a mixedresearch
design.
A significant portion of experimental studies in the area
of cognitive visualization can be sorted into two main categories.
The studies in the first category monitor and record
the behaviour of individuals or, rather, their conscious actions
and general work methods when completing tasks with a use
of a map. The most common aspects of studies are completion
speed, accuracy, and correctness or frequency of a given
solution (see [1–5]). The mentioned studies use a quantitative
approach and subsequent statistical methods of data analysis.
A second significant category is the use of eye-tracking systems.
Eye-tracking studies are in many cases combined with
the recording of conscious behaviour, that is, user actions (see
the first category), but the crucial activities recorded are eyemovements,
which offer continuous data about (even unconscious)
behaviour of the participant while solving a task. In
other words, the focus of the user’s attention is foregrounded
[6]. Due to the high processing requirements, these studies
are often performed on a small sample of participants and
methods other than statistical data analysis are being used, for
example, explorative data analysis [7].
Eye-tracking was used for the evaluation of maps for the
first time already in the late 1950s [8], but it has been increasingly
used in the last ten to fifteen years. The main reasons are
the declining prices of the equipment and the development
of computer technology that allows faster and more efficient
analysis of measured data. For usability research, eye-tracking
data should be combined with additional qualitative data,
since eye-movements cannot always be clearly interpreted
without the participant providing context to the data [9].
An example of comprehensive research in the field of
cognitive visualization by using eye-tracking is the work
of Alac¸am and Dalcı [10], who compared four map portals
(Google Maps, Yahoo Maps, Live Search Maps, and
MapQuest). The basic assumption of the study was that
lower average fixation duration indicates more intuitive map
portal environment. The shortest average fixation duration
was found in the case of Google Maps. Fabrikant et al. [11]
used eye-tracking for the evaluation of map series expressing
the evolution of the phenomenon over time, or for evaluation
of user cognition of weather maps [12]. Ooms et al. [13] dealt
with the suitability of map label positions and differences
in map reading between experts and novices. Popelka and
Brychtova [14] investigated the role of 2D and 3D terrain
visualization in maps.
Olson [15] compared cognitive visualization and cognitive
psychology, arguing that cartographers can adapt ideas
and experiments in methodology from cognitive psychologists.
Equally, psychologists can use maps as stimuli in their
studies. Both disciplines can examine the cognitive processes
while reading and understanding maps. However, cognitive
psychologists are interested in different types of cognitive
processes such as attention, visual perception, memorizing,
or decision-making. A map is only a tool in this context. For
a cognitive cartographer, the map is far more important.
The approach mentioned above is based on close cooperation
between cartographers and psychologists and shows
Small-scale study
(i) Limited research sample
(ii) Combination of Hypothesis and EyeTribe systems
(iii) Logging user actions and gaze tracking
Large-scale study
(i) Large research sample
(ii) Extensive and mass data collection on the Hypothesis platform
(iii) Event logging: user actions (conscious behaviour)
Figure 1: The combination of large-scale and small-scale study.
the possibility of a connection between large-scale studies
and small-scale studies based on gathering and analysing
eye-tracking data. Differences between large-scale and smallscale
studies are described in Figure 1.
As it is discussed in ˇStˇerba et al. [16], using only a qualitative
(explorative) or quantitative type of evaluation method
is not sufficient. Therefore, it is necessary to combine those
methods, enabling their suitable completion, obtaining more
valid results, and achieving better interpretation. A combination
of quantitative and qualitative methods was established
as mixed-research design [17]. The key idea and innovation
of our method are the interconnection of two approaches
in the area of cognitive visualization and also finding
a technological solution.
The Hypothesis platform serves primarily for the creation
of experimental test batteries, online administration, and
extensive data gathering. After connecting with the eyetracking
system, more detailed data on the experimental task
processing methods are gathered, which allow deeper insight
into the postulated cognitive processes that underlie the
behavioural reactions.
ˇStˇerba et al. [18] propose two variants of mixed-research
design:
(i) Using the eye-tracking system for a pilot study examining
a quality of experiment design with results
from this pilot study being used for improvement of
experiment design before large-scale data collection.
(ii) Using Hypothesis for large-scale quantitative
approach and secondary using of eye-tracking
method for the subsequent specification of certain
results with adjusted or changed types of tasks.
Both approaches and technical specification of Hypothesis
platform are described in detail in [18] and are available
online in English.
2. A Tool for Mass Data Collection: Web-Based
Platform Hypothesis
For the purposes of large-scale experimental investigation,
the creation of psychological tests, and evaluation of cartographic
works, new research software concept was designed
within the project “Dynamic Geovisualization in Crisis Management”
[19]. Subsequently, this concept has been realized,
and original software MuTeP was developed [20, 21]. MuTeP
Computational Intelligence and Neuroscience 3
Figure 2: Example task on WMS interactive map. The user indicates
the requested objects, draws lines, and marks out target areas by
polygons. In the example shown, the user called up an orthophoto
map in a dialogue-window. All the actions including the drawn point
coordinates, lines, and polygons are saved in the database, and the
correctness of the solution is automatically evaluated under preset
conditions.
was primarily created for the purposes of objective experimental
exploration and evaluation of cartographic products
in the perspective of user personality.
Although MuTeP was practically proven [22], it was
clear that the conception used will soon reach its limits.
Another impulse for the search for a more flexible solution
was an effort to involve dynamic cartographic visualization as
stimuli, randomization, nonlinear test batteries, connection
with eye-tracking technology, and so forth, which were not
possible to implement into MuTeP software.
Based on experience with MuTeP and in the context of
current requirements, a new software concept was designed.
This new software should have the potential for longterm
growth and development [23]. Hypothesis has several
important advantages in comparison with MuTeP. Above
all, Hypothesis enables computer adaptive testing and offers
a modular solution with plugin support (such as video or
interactive animation plugins) and enables the work with
interactive maps (such as web map services; see Figure 2).
The technology used for designing Hypothesis consists
of the following: (1) the application core and user interface
are built on framework Vaadin 7; work with the database is
provided by ORM Hibernate; and (2) PostgreSQL in version
9.1 (and higher) is used as a primary database system [18].
The architecture of the system is three-layer: a client,
server, and database. The client part is designed for communication
and interaction with the user, and its operation
is provided by standard web browsers (thin client) or a
special browser distributed in the application package—
special Hypothesis Browser. Hypothesis Browser is based on
Standard Widget Toolkit (SWT) components and ensures
more strict conditions and control over running tests [18, 24].
Hypothesis works as an event-logger application, which
logs all user actions and events (coordinates and timestamps
of clicks, key presses, start and end time of each presented
slide, exposition time of every component such as a picture
or dialogue-window, zoom of maps, rotation of 3D objects,
Figure 3: Management module in the Hypothesis platform. The
user can launch the available tests in two modes: (a) legacy (launches
in a normal browser) and (2) featured (launches in a controlled
mode in SWT browser). The manager and the superuser have an
extended access and can unlock the tests, create users, export results,
and so forth.
etc.). Extensive logging of user actions and events is enabled
through the structure of the final slides used for the test
battery (package). The package comprises the hierarchical
structure of branches which contain one or more tasks, and
each task contains at least one slide. The slide consists of
a template and content. Such structure enables nonlinear
branching of the test slides or randomization of slides. All
parts of the package are stored in structured XML format.
After starting a test, a selected package is loaded from the
database to the server application and a new test is created.
Emphasis was placed on variability and range of software
usability. Figure 2 shows an example of the slide using WMS.
The slide consists of two layers. The underlaying image
is created with a layer: ImageLayer. Above it, there is a
transparent layer: FeatureLayer, which is designed to draw
demanded points, polylines, or areas by mouse and store the
events [18].
Hypothesis is also improved with two new key functionalities
that are vital for the interconnection between eyetracking
systems (or other peripherals such as EEG) and
enable the realization of experiments with high reliability.
These functionalities involve the use of SWT browser that
allows the client to monitor and control the testing process. In
other words, when using the controlled mode (see Figure 3),
the participant has no way to intentionally or unintentionally
exit the test by, for example, pressing alt + F4. Other common
functions of web browsers are also strictly disabled, such
as page refreshing or opening menus by right-clicking the
mouse. The second key functionality is the recording of two
time sets in the database. To avoid the problem of slow
internet connection, both server time and local PC time
are recorded, which means that events on the client side
can be accurately synchronized (e.g., synchronizing stimulus
exposition with data from the eye-tracker).
Researchers can effectively create new test batteries
thanks to a combination of a number of subfunctions and
tools. Emphasis is also placed on the efficiency of the
software. Researchers can effectively change the content of
already finished test slides and create derivatives from sample
4 Computational Intelligence and Neuroscience
Table 1: Summary of calibration results for all participants.
Participant SMI 𝑋 SMI 𝑌 EyeTribe
P01 0,4 0,2 Good
P02 0,3 0,1 Poor
P03 0,4 0,6 Moderate
P04 0,4 0,4 Perfect
P05 0,9 0,5 Good
P06 0,3 0,5 Redo
P07 0,2 0,4 Moderate
P08 0,6 0,3 Moderate
P09 0,4 0,1 Perfect
P10 0,3 0,4 Poor
P11 0,6 0,3 Poor
P12 0,5 0,5 Moderate
P13 0,3 0,3 Moderate
P14 0,4 0,6 Poor
templates through the modules for user access administration
and also export structured results.
Hypothesis software is freely available for collaboration
on a various research topic in the Czech Republic and
abroad. Access to the database and modules is provided after
registration.
3. In-Depth Analysis of Cognitive Processes
Using Eye-Tracking System
3.1. EyeTribe Tracker. Eye-tracking technology is becoming
increasingly cheaper, both on the hardware and on the
software front. Currently, the EyeTribe tracker is the most
inexpensive commercial eye-tracker in the world, at a price
of $99. More information about the device is available at
the web page of the manufacturer (https://theeyetribe.com/).
The low-cost makes it a potentially interesting resource for
research, but no objective testing of its quality has been performed
as of yet [25]. Dalmaijer in his study [25] with five participants
compared the EyeTribe tracker with high-frequency
EyeLink 1000. He states that concurrent tracking by both
devices of the same eye-movements proved to be impossible,
due to the mutually exclusive way in which both devices work.
One of the reasons was that EyeLink uses only one eye for the
recording. Delmaijer [25] also states that recording with both
devices at the same time results in deterioration of results
of both and often leads to a failure to calibrate at least one.
Ooms et al. [26] compared EyeTribe with SMI RED 250 but
also did not use the concurrent recording. In our study, we
compared the EyeTribe tracker with SMI RED 250. In our
case, we have not noticed any problems with calibration (see
Table 1).
3.2. Methods of EyeTribe Accuracy Evaluation. For the comparison
study, recording with SMI RED 250 and the EyeTribe
tracker at the same time was performed. Laboratory setup is
displayed in Figure 4. The EyeTribe tracker stands in front of
the SMI device.
Figure 4: Laboratory setting for EyeTribe and SMI accuracy
comparison.
EyeTribe tracker was connected with the OGAMA software
[27], where the experiment with six static image stimuli
was prepared. At the same time, screen recording experiment
was created in SMI experiment center (sampling frequency
was set up to 60 Hz, to be the same as EyeTribe). Both devices
were calibrated separately (but the eye-trackers were at their
positions and turned on).
After calibrations, recording with SMI started. After that,
experiment with static images in OGAMA was performed.
That means the SMI device recorded the experiment data
as well (as a screen recording video). The whole experiment
procedure was done with fourteen participants. The purpose
of the study was to verify how trustworthy data from EyeTribe
tracker are. Recorded fixations from both eye-trackers were
compared qualitatively and quantitatively. A diagram of the
whole recording procedure is displayed in Figure 5.
For the comparison of recorded data from both devices,
the OGAMA environment was used. Data from EyeTribe
were displayed in OGAMA directly; SMI data had to be converted.
For this conversion, the tool smi2ogama developed by
S. Popelka was used. The tool is available at http://eyetracking
.upol.cz/smi2ogama/.
The recorded screen data were cropped according to
the pertinence to individual stimuli. For that, recorded key
presses (for a slide change) were used.
3.3. Participants. Total of 14 respondents participated in this
part of the study (ten males and four females with an average
age of 29.5). They were employees and postgraduate students
of department of geoinformatics. 16-point calibration was
used for both devices. Results of calibration are summarized
in Table 1. With the EyeTribe, it was almost not possible to
achieve perfect calibration result. Figure 6 shows the details of
calibration results for participant P03. The results in OGAMA
show calibration result for each of the 16 calibration points
(with the use of colour); SMI shows only the average value in
degrees of visual angle for axes 𝑋 and 𝑌.
For all recordings, I-DT fixation detection in OGAMA
was used with the same settings. A value of 20 px was used as
“maximum distance”; “minimum number of samples” was set
up to 5. More information about fixation detection settings is
available in [28, 29].
Computational Intelligence and Neuroscience 5
Screen recording
SMI RED 250
EyeTribe
OGAMA
SMI
experiment
center
Experiment
with image
stimuli
Screen
recording
experiment
Image stimuli from
OGAMA experiment
I1–I5
Calibration
S--------------
----------------
Calibration
-----------------
E
Split into trials
based on key
presses
Import of SMI raw data sequences into OGAMA experiment
S---------------
-----------------
I1--------------
-----------------
I2--------------
-----------------
I3--------------
-----------------
I4--------------
-----------------
I5--------------
-----------------
E---------------
---------------E
S----------------
-----------------
I1--------------
-----------------
I2--------------
-----------------
I3--------------
-----------------
I4--------------
-----------------
I5--------------
-----------------
Figure 5: Diagram of concurrent eye-movements recording with SMI RED 250 and EyeTribe.
(a) (b)
Figure 6: Calibration results from EyeTribe (a) and SMI RED (b) for participant “P03.”
3.4. Stimuli. The experiment contained six static images. The
first one contained a grid with nine numbers; second one
(Slide 2, Figure 7) contained sixteen numbers. The task of
the participants was to read numbers in ascending order
(from top to the bottom). Next three stimuli contained
different types of maps, but the results of these stimuli are not
described in this paper. The last stimulus (Slide 6, Figures 8
and 9) contained a map of the world and respondents’ task
was to move the eyes around Africa.
3.5. Results and Discussion of EyeTribe Evaluation. Eyemovement
data recorded from participant P03 are displayed
in Figure 7. Red points represent fixations from SMI, and blue
points are fixations from EyeTribe. The task in this stimulus
was only to read the numbers.
From Figure 7, it can be seen that both devices recorded
around one or two fixations over each number. The accuracy
of the recording is comparable. Accuracy reflects the eyetracker’s
ability to measure the point of regard and is defined
6 Computational Intelligence and Neuroscience
1 2 3 4
5 6 7 8
9
10 11 12
13 14
15
16
Figure 7: Comparison of recorded eye-movement data from participant P03 in Slide 2 from EyeTribe (blue) and SMI RED (red).
20px
10px
Original Shifted
Figure 8: Comparison of recorded eye-movement data from participant P03 in Slide 6 from EyeTribe (blue) and SMI RED (red).
as the average difference between a test stimulus position and
the measured gaze position [30]. The largest deviations of
the EyeTribe tracker data were observed for two points in
the middle of the bottom line. This situation was observed in
almost all recorded data. The situation can be seen in Figure 7
in the case of points 14 and 15 (middle points in the lowest line
of numbers). Gaze position recorded by EyeTribe is shifted
upwards.
Another example is visible in Figure 8, which is the crop
of Slide 6 stimuli. In this stimulus, the task was to move
the eyes around the continent of Africa on the map. The
data recorded by EyeTribe tracker were moved to the left by
20 px, but this systematic error can be corrected by a manual
shift of fixations in OGAMA. This situation is depicted
in Figure 8. On the left side, original data are displayed.
On the right, data after horizontal shift (20 px to the right
for EyeTribe and 10 px to the left for SMI) are depicted.
Eye-movement data from EyeTribe for horizontally central
fixations are shifted upwards, especially in the bottom part of
the stimuli. See Figure 12 for more detailed analysis of fixation
locations. The same issue was reported in all stimuli for most
of the participants. Visualization of gaze trajectories of all
participants is in Figure 9. The solution for dealing with this
inaccuracy is to avoid placing important parts of the stimulus
to the bottom of the screen. It will be possible to compare
recorded raw data, but, in cartographic research, fixations
are used for analysis, so it was more meaningful to compare
fixations (identified with the same algorithm).
As an alternative for the comparison of raw data, comparison
of data loss was performed. In the case of SMI recordings,
average data loss (samples with coordinates 0, 0) was 0.57%
of all recorded data. With the EyeTribe, the average data loss
was 1.22%. Although the value is more than twice higher than
in the case of SMI, it is still acceptable.
The graph in Figure 10 shows the percentage of data loss
for Slide 2. It is evident that data loss is higher in the case of
EyeTribe recordings, but, in most cases, less than 2% of data
is missing. The highest values were observed for participants
Computational Intelligence and Neuroscience 7
Figure 9: Problems with data recorded by EyeTribe (blue) at the bottom of the stimuli in comparison with SMI data (red).
0
1
2
3
4
5
6
7
8
P01 P02 P03 P04 P05 P06 P07 P08 P09 P10 P11 P12 P13 P14
Gazesampleswithvalue(0,0)(%)
Data loss—Slide 2
Figure 10: Comparison of data losses of fourteen participants
during observation of Slide 2. Red bars represent SMI RED 250; blue
ones represent EyeTribe tracker.
P06 and P13. Participant P06 had the worst calibration from
all respondents. Participant P13 has worn glasses which can
possibly cause the high data loss.
In the next step of accuracy evaluation, values of eyetracking
metric fixation count recorded by SMI RED 250 and
the EyeTribe tracker were compared for all six stimuli in the
experiment. A summary of the results is shown in Figure 11.
The correlation between numbers of detected fixations was
between 0.949 and 0.989 with the exception of participant
P13 with the correlation of 0.808. The ratio between a number
of recorded fixations with SMI device and EyeTribe was
also investigated. On average, EyeTribe recorded 88.2% of
fixations that were recorded by SMI device. The correlation
and ratio values for each participant are presented as part of
Figure 11.
Beside the number of fixations, their location was compared.
For this evaluation, Slide 2 with a grid of 16 numbers
was chosen (Figure 7). For each participant, the deviations
between coordinates of the target (number) and closest
fixation were calculated. The graphs in Figure 12 show the
median size and direction of the deviation for each of the 16
targets in the stimuli. It is evident that the largest deviations
(heading upwards) for EyeTribe were observed for the points
in the bottom part of the image (numbers 14 and 15). Each
graph contains the value of the Euclidean distance of median
deviations from the origin. Average deviation was 26 px for
EyeTribe and 22 px for SMI.
The evaluation of truthfulness was performed on fourteen
participants. According to Nielsen [31], this number should
be sufficient. The evaluation of qualitative (Figures 7, 8, and
9) and quantitative (Figures 10, 11, and 12) data indicates that
accuracy of low-cost EyeTribe tracker is sufficient for the
use in cartographic research. Similar results were found by
Ooms et al. [26], who measured the accuracy by the distance
between recorded fixation locations and the actual location.
The limitation of the low-cost device is the sampling
frequency, which is only 60 Hz (compare with 250 Hz of
SMI RED eye-tracker). Another problem is shift of fixation
locations in the bottom part of the screen. Taking into
account described limits of the device, the EyeTribe may be
an appropriate tool for cartographic research.
4. Integrated Research System:
Interconnection of Hypothesis Software
and EyeTribe
As one of the practical applications of the mixed-research
experiment design, the Hypothesis software interconnected
with the EyeTribe tracker was chosen. For the recording of
eye-tracking data, the OGAMA software was used because
the EyeTribe tracker is intended for developers and contains
no software for data recording and analysis. OGAMA has
an inbuilt slide show viewer, but the range of functionality
of this viewer in comparison with SW Hypothesis is quite
8 Computational Intelligence and Neuroscience
0
20
40
60
80
100
120
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
P01Cor = 0.989
Ratio = 84,6%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
Cor = 0.956
Ratio = 68,5%
0
20
40
60
80
100
120 P02
Cor = 0.953
Ratio = 76,5%
0
20
40
60
80
100
120 P03
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
Cor = 0.949
Ratio = 75,3%
0
20
40
60
80
100
120 P04
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120
P08Cor = 0.960
Ratio = 87,7%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120 P07Cor = 0.966
Ratio = 107,9%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120 P06Cor = 0.956
Ratio = 86,9%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120
P05Cor = 0.988
Ratio = 84,6%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120
P09Cor = 0.977
Ratio = 95,8%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120
P10Cor = 0.970
Ratio = 98,6%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120
P11Cor = 0.971
Ratio = 86,3%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120
P12Cor = 0.975
Ratio = 91,6%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120
P14Cor = 0.960
Ratio = 87,3%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
0
20
40
60
80
100
120
P13Cor = 0.808
Ratio = 102,7%
Slide1
Slide2
Slide3
Slide4
Slide5
Slide6
Figure 11: Comparison of fixation count eye-tracking metric for fourteen participants. EyeTribe data are displayed as blue line; SMI data are
displayed as red line.
limited. Desktop application OGAMA principally does not
allow working with web-based interactive maps and mouse
clicks are recorded but not shown. Oppositely, Hypothesis
visualizes clicks and allows drawing of lines and polygons.
This functionality is crucial in the context of working with
maps. Because of this functionality, Hypothesis connected to
OGAMA via HypOgama was used.
4.1. Methods of Hypothesis and EyeTribe Interconnection. For
the study, a simple Hypothesis experiment containing five
stimuli (intro, three pairs of maps, and last slide) was used.
Participants’ task was to identify the differences between the
maps. Coordinates of the clicks representing differences were
also recorded.
OGAMA experiment was designed with only one screen
recording stimulus. OGAMA in version 5.0 can record
dynamic web stimuli, but it is not possible to use slides from
Hypothesis as separate stimuli.
Recorded data were split according to their belonging
to particular slides in the Hypothesis experiment. For the
split, timestamps from Hypothesis indicating the slide change
were used. The splitting and conversion of recorded data
Computational Intelligence and Neuroscience 9
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
22px
15px
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
18px
13px
17px
32px
15px
12px
18px
21px
−100
−80
−60
−40
−20
0
20
40
60
80
100
−100
−80
−60
−40
−20
0
20
40
60
80
100
5px
7px
10px
15px
15px
57px
7px
16px
51px
14px
32px
4px
10px
55px
16px
25px
100 px
31px
54px
5px
21px
23px
Figure 12: Comparison of fixation positions in Slide 2 for fourteen participants. Distance from the center of the image shows fixation deviation
in pixels. EyeTribe data are displayed as blue dots; SMI data are displayed as red dots.
manually were time-consuming and not user-friendly. Thus, a
web application called HypOgama was written in PHP for the
automation of the process. The functionality of HypOgama
application is illustrated in Figure 13.
The HypOgama application (Figure 14) is freely available
at http://eyetracking.upol.cz/hypogama/.
The application synchronizes the Hypothesis time with
the timestamp from the eye-tracking recording in OGAMA.
The synchronization is processed by the key press that was
used to start the Hypothesis experiment and which was
recorded in both systems—in Hypothesis and OGAMA.
In the next step, the application scans the Hypothesis file
and finds the timestamps of slide changes. These timestamps
are then used for splitting raw eye-tracking data into blocks
belonging to particular slides. The name of the relevant stimuli
is added to all records from each block. In the final step,
the data structure is modified for the direct import into a new
OGAMA project.
The application contains six input fields:
(1) Exported file from Hypothesis manager containing
data for one participant.
10 Computational Intelligence and Neuroscience
Screen recording
EyeTribe
Hypothesis
OGAMA +
EyeTribe
Hypothesis
experiment
Hypothesis experiment
Calibration
S----------------
------------------
------------------
------------------
------------------
------------------
------------------
------------------
------------------
------------------
------------------
------------------
------------------
------------------
----------------E
Screen
recording
experiment
------------------
key-------------E
HypOgama
Split raw data into trials based on timestamps
of key presses (start and end of experiment)
and timestamps of slide change from
Hypothesis database
OGAMA raw data Timestamps
New OGAMA project
containing five static images
overlaid with eye-movement data
Export
Images
S-----------------
key--------------
------------------
I1----------------
------------------
I2----------------
------------------
I3----------------
------------------
I5----------------
------------------
I4----------------
------------------
Figure 13: Process of splitting recorded data (screen recording) into trials with the use of HypOgama web application.
Figure 14: Environment of HypOgama web application.
(2) Exported raw data from the OGAMA application for
one participant.
(3) Name of the output file.
(4) Subject name (if blank, the ID from Hypothesis will
be used).
(5) Frequency of an eye-tracker (30 or 60 Hz).
(6) Synchronization variables: these values indicate
which key was used for the synchronization of
Hypothesis and OGAMA (default value is “Key:
Down” in OGAMA format and “Down” in the
format of Hypothesis application).
In the Hypothesis file (ad 1), HypOgama finds the row with
the key press (default Key: Down) and the corresponding
time, which corresponds to the beginning of the experiment.
In the next step, the column containing the slide names is
scanned and the time of the first occurrence of each slide
is also stored. According to this time, OGAMA recording is
split. The last information obtained from the Hypothesis file is
the name of the subject, overwriting the subject name in the
OGAMA file.
In OGAMA file, all records prior to the synchronization
key press are erased. Stimuli names are replaced by those from
Hypothesis file.
Outputs of the created script are raw eye-movement
data for each slide that could be directly imported into the
OGAMA project. The only one necessary thing is to put
image files (stimuli) into OGAMA project folder. If it is the
same filename as the one contained in the Hypothesis file,
images will be automatically assigned to proper data. After
the whole process, a user has OGAMA project containing
static image stimuli with all corresponding eye and mouse
movement data. The proposed concept was applied and
verified through a selected case study described below. The
purpose of this short study was to illustrate the functionality
of interconnection of EyeTribe and OGAMA.
For the verification of the designed process of Hypothesis
and EyeTribe combination, simple test battery was designed.
For chosen procedure, Hypothesis was used for large-scale
quantitative approach and eye-tracking method for the subsequent
specification of certain results.
The test battery was established in the Hypothesis software
and was focused on verification of Gestalt principles,
Computational Intelligence and Neuroscience 11
Figure 15: Example of stimuli—the first pair of topographic maps.
0
1
2
3
4
5
6
7
8
9
Average number of correct answers
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
Figure 16: An example of results from Hypothesis. An average number of correct answers for each of the participants.
respectively, figure-ground organization, and on the crosscultural
comparison in the context of visual perception of
cartographic stimuli [22, 32–35] on the example of specific
cartographic products. The cartographic tasks were part of
these more complex research batteries. The main purpose
of this short cartographic study was the verification of
HypOgama application and whole integrated research system
for further research studies.
4.2. Participants. Participants of this illustrative case study
were 64 students from the Masaryk University, Czech Republic,
and 64 students from Wuhan University, China. In the
first phase, participants were tested only on the web-based
platform Hypothesis. Only a half of the dataset (Czech population)
was further used in context of this particular study
where the topographic and thematic maps were compared.
In the second phase, the experiment was conducted with the
use of eye-tracking system and the research sample is still
continually extended.
4.3. Stimuli. The stimuli were represented by three pairs
of maps that differed in 10 variables, for example, different
colours of map signs, different position of the signs, and
missing map signs. First two pairs of stimuli contained
topographic maps. The third pair of the maps contained a
thematic map.
The test was structured in three main parts. In the first
part, participants filled out a personal questionnaire; in the
second part, a representative example of the stimuli was presented
to familiarize the participants with the environment
of Hypothesis. In the third part, three tasks containing pairs
of stimuli described above were presented. Participants were
asked to mark the differences between presented maps. The
time limit for each task was 45 seconds. An example of a
topographic map (Slide 1) is displayed in Figure 15. On Slide
2, similar topographic map in different scale was shown. The
last slide contained thematic map (see Figure 17).
4.4. Results and Discussion of Hypothesis and EyeTribe Interconnection.
The performed study verified stability of proposed
system on long distances and, at the same time, part
of the test battery was used as a pilot study to verify the functionality
of an integrated research system. Stimuli comparing
the effectiveness of visual search between topographic and
thematic maps were selected.
In the first phase, the test was performed in the Hypothesis
application only. A number of differences identified
between pairs of maps on Czech population were analysed
(see Figure 16).
In the case of two pairs of topographic maps, the average
number of correct answers was four. In the case of the stimuli
12 Computational Intelligence and Neuroscience
Figure 17: Example of eye-movement data recorded during the Hypothesis experiment. Circles represent fixations; blue line on the right is a
mouse trajectory.
with a thematic map, the average number of correct answers
was five.
To generalize the findings, an increase of the number of
maps per condition would be necessary. However, this difference
was the first clue to establish working hypotheses. Based
on the data from the first phase of testing, hypotheses were
established only at the level of stimulus-reaction. The way of
task processing by users and their solving strategies were still
a black-box; thus there was a need for more detailed procedural
data, especially for information about distinct search
strategies.
To explore differences in the visual search, eye-tracking
can be used due to the ability to provide more detailed
information (e.g., which kind of object was omitted, which
kind of object could be found at first glance, and which areas
attracts main attention).
Therefore, in the second phase, the already used experimental
battery created in Hypothesis was interconnected
with OGAMA through HypOgama application and the
experiment was launched with the EyeTribe system. Cartographic
stimuli and the eye-tracking data were linked
together and further analysed with OGAMA.
The example in Figure 17 shows outputs from OGAMAscan
path and mouse trajectory of one participant over the
stimulus with thematic maps. In this case, fixations are distributed
mainly over the text labels in the map. Participant did
not find the difference in the colour of the Odisha state (on the
east coast of India) under the relatively large graph. At the
same time, eye-tracking metrics (e.g., fixation count, dwell
time for each map, and a number of saccades between these
maps) can be statistically analysed. Based on findings from
both types of analyses, the hypotheses for subsequent study
can be established.
The functionality of the integrated research system has
been fully verified in the above-mentioned pilot study.
The experiment created on the Hypothesis platform was
connected with OGAMA and EyeTribe via HypOgama.
Data capture including eye-tracking recording continued and
exploratory analyses of these data were performed.
5. Conclusion
The aim of the paper was to prove the concept of the mixedresearch
design through the interconnection of Hypothesis
(software for experiment creation, experiment execution, and
data collection) and the EyeTribe tracker (the most inexpensive
commercial eye-tracker). This system could prove to be
a valuable tool for cognitive cartography experiments and
evaluation of user behaviour during map reading process.
The first necessary step was to evaluate the accuracy of
the EyeTribe tracker with the use of concurrent recording
together with the SMI RED 250 eye-tracker. The results of the
comparison show that the EyeTribe tracker can be a valuable
resource for cartographical research.
The next part of the study was focused on the interconnection
of the EyeTribe with the Hypothesis platform, developed
at Masaryk University in Brno. The connection was
made through a newly created web application that modifies
eye-movement data recorded during screen recording experiment
in the OGAMA open-source application. The application
is publicly available for the community of cartographers
and psychologists at web page http://eyetracking.upol.cz/
hypogama.
The interconnection advantages were illustrated on an
example of simple case study containing three pairs of maps.
The performed case study demonstrated the ability of the
combined system of the Hypothesis platform and the EyeTribe
tracker to support each other and to serve as an effective
tool for cognitive studies in cartography.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
This paper was supported by projects of Operational Program
Education for Competitiveness (European Social Fund)
(Projects CZ.1.07/2.3.00/20.0170 and CZ.1.07/2.3.00/30.0037)
Computational Intelligence and Neuroscience 13
of the Ministry of Education, Youth and Sports of the Czech
Republic and the Student Project IGA PrF 2015 012 of the
Palacky University.
References
[1] G. L. Allen, C. R. Miller Cowan, and H. Power, “Acquiring
information from simple weather maps: influences of domainspecific
knowledge and general visual-spatial abilities,” Learning
and Individual Differences, vol. 16, no. 4, pp. 337–349, 2006.
[2] D. Edler, A.-K. Bestgen, L. Kuchinke, and F. Dickmann, “Grids
in topographic maps reduce distortions in the recall of learned
object locations,” PLoS ONE, vol. 9, no. 5, Article ID e98148,
2014.
[3] P. Kub´ıˇcek, ˇC. ˇSaˇsinka, and Z. Stachoˇn, “Vybran´e kognitivn´ı
aspekty vizualizace polohov´e nejistoty v geografick´ych datech,”
Geografie, vol. 119, no. 1, pp. 67–90, 2014.
[4] E. S. Nelson, “Using selective attention theory to design bivariate
point symbols,” Cartographic Perspectives, vol. 32, pp. 6–28,
1999.
[5] E. S. Nelson, “Designing effective bivariate symbols: the influence
of perceptual grouping processes,” Cartography and Geographic
Information Science, vol. 27, no. 4, pp. 261–278, 2000.
[6] A. H. Duc, P. Bays, and M. Husain, “Eye movements as a probe
of attention,” in Progress in Brain Research, C. Kennard and R. J.
Leigh, Eds., vol. 171, chapter 5.5, pp. 403–411, Elsevier, 2008.
[7] N. Andrienko and G. Andrienko, Exploratory Analysis of Spatial
and Temporal Data. A Systematic Approach, Springer, New York,
NY, USA, 2005.
[8] J. M. Enoch, “Effect of the size of a complex display upon visual
search,” Journal of the Optical Society of America, vol. 49, no. 3,
pp. 280–286, 1959.
[9] A. Hyrskykari, S. Ovaska, P. Majaranta, K.-J. R¨aih¨a, and M.
Lehtinen, “Gaze path stimulation in retrospective think-aloud,”
Journal of Eye Movement Research, vol. 2, no. 4, pp. 1–18, 2008.
[10] ¨O. Alac¸am and M. Dalcı, “A usability study of WebMaps with
eye tracking tool: the effects of iconic representation of information,”
in Human-Computer Interaction. New Trends, vol. 5610
of Lecture Notes in Computer Science, pp. 12–21, Springer, Berlin,
Germany, 2009.
[11] S. I. Fabrikant, S. Rebich-Hespanha, N. Andrienko, G.
Andrienko, and D. R. Montello, “Novel method to measure
inference affordance in static small-multiple map displays
representing dynamic processes,” Cartographic Journal, vol. 45,
no. 3, pp. 201–215, 2008.
[12] S. I. Fabrikant, S. R. Hespanha, and M. Hegarty, “Cognitively
inspired and perceptually salient graphic displays for efficient
spatial inference making,” Annals of the Association of American
Geographers, vol. 100, no. 1, pp. 13–29, 2010.
[13] K. Ooms, P. De Maeyer, and V. Fack, “Study of the attentive
behavior of novice and expert map users using eye tracking,”
Cartography and Geographic Information Science, vol. 41, no. 1,
pp. 37–54, 2014.
[14] S. Popelka and A. Brychtova, “Eye-tracking study on different
perception of 2D and 3D terrain visualisation,” Cartographic
Journal, vol. 50, no. 3, pp. 240–246, 2013.
[15] J. M. Olson, “Cognitive cartographic experimentation,” Cartographica,
vol. 16, no. 1, pp. 34–44, 1979.
[16] Z. ˇStˇerba, ˇC. ˇSaˇsinka, Z. Stachoˇn, P. Kub´ıˇcek, and S. Tamm,
“Mixed research design in cartography: a combination of
qualitative and quantitative approaches,” Kartographische
Nachrichten, vol. 64, no. 5, pp. 262–269, 2014.
[17] J. W. Creswell, Research Design: Qualitative, Quantitative, and
Mixed Methods Approaches, Sage, Thousand Oaks, Calif, USA,
2nd edition, 2003.
[18] Z. ˇStˇerba, ˇC. ˇSaˇsinka, Z. Stachoˇn, K. Morong, and R. ˇStampach,
Selected Issues of Experimental Testing in Cartography, Masaryk
University, 2015, http://munispace.muni.cz/index.php.
[19] M. Koneˇcn´y, ˇS. Bˇrezinov´a, M. V. Dr´apela et al., Dynamick´a
Geovizualizace v Krizov´em Managementu, Masarykova Univerzita,
Brno, Czech Republic, 2011 (Czech).
[20] P. Kub´ıˇcek, ˇC. ˇSaˇsinka, and Z. Stachoˇn, “Uncertainty visualisation
testing,” in Proceedings of the 4th Conference on Cartography
and GIS, Sofia, Bulgaria, June 2012.
[21] Z. Stachoˇn, ˇC. ˇSaˇsinka, P. Kub´ıˇcek, and Z. ˇStˇerba, “MuTeP—
alternativn´ı n´astroj pro testov´an´ı kartografick´ych vizualizac´ı a
sbˇer dat,” in Proceedings of the 23rd Sjezd ˇCesk´e Geografick´e
Spoleˇcnosti, Prague, Czech Republic, 2014 (Czech).
[22] Z. Stachoˇn, ˇC. ˇSaˇsinka, Z. ˇStˇerba, J. Zboˇril, ˇS. Bˇrezinov´a, and J.
ˇSvancara, “Influence of graphic design of cartographic symbols
on perception structure,” Kartographische Nachrichten, vol. 63,
no. 4, pp. 216–220, 2013.
[23] K. Morong and ˇC. ˇSaˇsinka, “Hypothesis—online software platform
for objective experimental testing,” in Proceedings of the
Applying Principles of Cognitive Psychology in Practice, Brno,
Czech Republic, May 2014.
[24] S. W. T. Widgets, https://www.eclipse.org/swt/.
[25] E. Dalmaijer, “Is the low-cost EyeTribe eye tracker any good for
research?” PeerJ PrePrints, vol. 2, Article ID e585v1, 2014.
[26] K. Ooms, L. Dupont, L. Lapon, and S. Popelka, “Accuracy and
precision of fixation locations recorded with the low-cost Eye
Tribe tracker in different experimental set-ups,” Journal of Eye
Movement Research, vol. 8, no. 1, pp. 1–24, 2015.
[27] A. Voßk¨uhler, V. Nordmeier, L. Kuchinke, and A. M. Jacobs,
“OGAMA (Open Gaze and Mouse Analyzer): open-source
software designed to analyze eye and mouse movements in
slideshow study designs,” Behavior Research Methods, vol. 40,
no. 4, pp. 1150–1162, 2008.
[28] S. Popelka, “Optimal eye fixation detection settings for cartographic
purposes,” in Proceedings of the 14th SGEM GeoConference
on Informatics, Geoinformatics and Remote Sensing (SGEM
’14), vol. 1, pp. 705–712, June 2014.
[29] S. Popelka and J. Doleˇzalov´a, “Non-photorealistic 3D visualization
in city maps: an eye-tracking study,” in Modern Trends in
Cartography, pp. 357–367, Springer, 2015.
[30] K. Holmqvist, M. Nystr¨om, R. Andersson, R. Dewhurst, H.
Jarodzka, and J. Van de Weijer, Eye Tracking: A Comprehensive
Guide to Methods and Measures, Oxford University Press,
Oxford, UK, 2011.
[31] J. Nielsen, Why You Only Need to Test with 5 Users, Nielsen
Norman Group, 2000, http://www.nngroup.com/articles/why-
you-only-need-to-test-with-5-users/.
[32] J. ˇCenek, “Individualism and collectivism and their cognitive
correlates in cross-cultural research,” The Journal of Education,
Culture and Society, vol. 2, pp. 210–225, 2015.
14 Computational Intelligence and Neuroscience
[33] R. E. Nisbett, I. Choi, K. Peng, and A. Norenzayan, “Culture
and systems of thought: holistic versus analytic cognition,”
Psychological Review, vol. 108, no. 2, pp. 291–310, 2001.
[34] E. Rubin, “Figure and ground,” in Visual Perception, S. Yantis,
Ed., pp. 225–229, Psychology Press, Philadelphia, Pa, USA, 2001.
[35] J. Wagemans, J. H. Elder, M. Kubovy et al., “A century of Gestalt
psychology in visual perception: I. Perceptual grouping and
figure-ground organization,” Psychological Bulletin, vol. 138, no.
6, pp. 1172–1217, 2012.
International Journal of
Geo-Information
Article
The Hypothesis Platform: An Online Tool for
Experimental Research into Work with Maps and
Behavior in Electronic Environments
ˇCenˇek Šašinka 1,* ID
, Kamil Morong 1 and Zdenˇek Stachoˇn 2,*
1 Department of Psychology, Centre for Experimental Psychology and Cognitive Sciences, Faculty of Arts,
Masaryk University, 60200 Brno, Czech Republic; kamil.morong@gmail.com
2 Department of Geography, Centre for Experimental Psychology and Cognitive Sciences, Faculty of Science,
Masaryk University, 61137 Brno, Czech Republic
* Correspondence: ceneksasinka@gmail.com ( ˇC.Š.); zstachon@geogr.muni.cz (Z.S.);
Tel.: +420-541-145-958 ( ˇC.Š.); +420-549-494-925 (Z.S.)
Received: 4 November 2017; Accepted: 11 December 2017; Published: 20 December 2017
Abstract: The article presents a testing platform named Hypothesis. The software was developed
primarily for the purposes of experimental research in cartography and psychological diagnostics.
Hypothesis is an event-logger application which can be used for the recording of events and their
real-time processing, if needed. The platform allows for the application of Computerized Adaptive
Testing. The modularity of the platform makes it possible to integrate various Processing.js-based
applications for creation and presentation of rich graphic material, interactive animations, and tasks
involving manipulation with 3D objects. The Manager Module allows not only the administration
of user accounts and tests but also serves as a data export tool. Raw data is exported from the
central database in text format and then converted in the selection module into a format suitable for
statistical analysis. The platform has many functions e.g., the creation and administration of tasks
with real-time interaction between several participants (“multi-player function”) and those where
a single user completes several tests simultaneously (“multi-task function”). The platform may be
useful e.g., for research in experimental economics or for studies involving collaborative tasks. In
addition, connection of the platform to an eye-tracking system is also possible.
Keywords: experimental testing; cognitive cartography; web-based software; behavior research
method; psychological diagnostic; eye tracking
1. Introduction
Research on user aspects of cartographic visualization can be traced back to the middle of the
twentieth century. Contemporary technological developments enabled the extension of existing
methods of spatial data presentation, such as contextual visualization [1], interactive (stereoscopic)
3D visualization [2–4], audiovisual communication [5], augmented reality [6] etc. Thus, there arises
a need for the establishment of new research tools. The need for a distributed web-based approach
is mentioned e.g., by Robinson [7]. There are several strategies on how to handle the issue. Most of
the studies use an environment designed for a particular study. Robinson [7] demonstrated the usage
of the e-Delphi platform and the e-Symbology Portal. This approach brings increased demands on
software development for each performed study. Other studies usually use general software tools such
as WebEx, Microsoft PowerPoint, Microsoft NetMeeting, etc. The advantage of the aforementioned
general software is the general availability without the need for further application development.
The main disadvantage consists of the limited possibility to customize the tools for a particular study.
As there was no tool specifically designed for the large variety of usability testing issues in cartography,
ISPRS Int. J. Geo-Inf. 2017, 6, 407; doi:10.3390/ijgi6120407 www.mdpi.com/journal/ijgi
ISPRS Int. J. Geo-Inf. 2017, 6, 407 2 of 22
we decided to start development in this area. As many authors proved validity and profitability of
remote usability testing [8], we began development in this direction.
The character of the newly designed platform was determined by cooperation between
psychologists and cartographers. In the course of a research project entitled “Dynamic Geovisualization
in Crisis Management” [9], a need arose to develop a research tool which would record the behavior of
research subjects when working with cartographic materials (including interactive ones) while making
it possible to create and administer psychological tests. Interactive electronic maps usually involve
web browsing [10,11], so the new platform was to be designed as a web-based application. Based
on a set of specifications, a client-server Multivariate Testing Programme (MuTeP) was developed
which made it possible to perform varying map reading-related operations, including clicking on
objects, route plotting, and others [12–17]. The same functionality was used to create psychological
performance tests and tasks (e.g., adaptation of the framed-line test [18] and Embedded Figures
Test [19]. However, the MuTeP application showed principal limitations in several aspects, one being
the absence of adaptive testing principles. Based on the experience with the MuTeP application,
a completely new platform, Hypothesis, was developed to overcome the limitations of the previous
software [20]. In defining the architecture of the new platform, emphasis was placed on the range and
variability of its functionality, and its flexibility when creating test tasks and batteries.
1.1. Application in the Field of Cognitive Cartography
Cartographic visualizations can be viewed as complex visual stimuli [21,22], where content
and form interact. When creating test batteries, every task of the same tape is related to a different
territory, and so different “correct answers” need to be defined. In Hypothesis, the necessary level of
flexibility was achieved by the unique connection of slide templates with related slide contents
(see Section 2.1), which makes it possible to perform all the necessary modifications of stimuli
and correct answers. The varying activities and operations [23,24] performed by research subjects
may range from simple visual searches, clicking on target objects [25], and sorting of objects of a
single category [26] to more elaborate operations which include optimal route planning, terrain
passability investigation [27,28], and even highly complex operations, such as those involving crisis
management [29–31], agriculture-related decision-making [32], and crime analysis [33]. The unique
design of Hypothesis makes it possible to conduct not only “molecular-level” experiments, but also
studies focusing on “molar behavior” [34]. The aim of the former is to study low-level cognitive
processes; they include visual search or memory tasks, where the speed, accuracy, and precision of the
participant’s solution are analyzed. Molar-behavior studies, on the other hand, focus on high-level
cognitive processes, investigating the strategies employed in the process of task solving. The analyzed
behavior then includes a sequence of map-related operations (e.g., frequency of legend consulting,
zooming, map shifting, switching between the varying map layers, selection of target areas, and their
interconnection with optimal routes.
1.2. Application in the Field of Psychodiagnostics
Its wide functionality, central database, and well worked-out management module, along with
the possibility to present rich and interactive visual stimulus, makes the Hypothesis platform suitable
for psychological diagnostics as well. The platform allows for the creation (or adaptation) of a range of
psychological performance tests which used to be available only in the paper-and-pencil version [35,36].
The platform contains expression evaluators so that the operations performed by participants can
be evaluated in real time (while the participants are completing the tests). The table of results then
contains not only raw data, but also the calculated scores and indices. The Hypothesis platform has
already been used for the creation or adaptation of several psychological tests, including the D2 Test of
Attention [37], Trail-making Test [38], and Grammatical Transformation Test [39], Perspective Taking
Test or Mental Rotation Test [40,41].
ISPRS Int. J. Geo-Inf. 2017, 6, 407 3 of 22
2. Hypothesis: Characteristics and Architecture
The Hypothesis platform consists of a Module Manager and a Database. The Database, which
contains whole Packs (Test Batteries) as well as the participants’ results, is located on the central server.
The participants and database administrators access the Management Module (Figure 1) through
a web browser. After log-in, the Management Module interface offers several functional modes.
The participants can access only the Pack Menu (list of test batteries). Depending on the setting,
the tests can be run in either the “Basic” mode (full screen window of a standard web browser) or in
the “Controlled” mode (SWT browser). Administrators (managers or super-users) can, in addition to
the list of tests, access the Administrator Account which makes it possible to control and manage all
the user accounts. Also, the Module Manager contains an Export Module for the export of raw data
(from the table of results) in .xlsx format. For the purpose of raw-data post-processing, a Selection
Application was developed. When creating new tests, administrators also make use of the Slide Editor,
where the individual slides can be displayed and edited. The slides can be edited using a web browser,
while tests as such, are created in the Database.
Figure 1. Basic functional components of the Hypothesis platform (modified and extended from
Štˇerba et al. [42]): The first part, Database (see Section 4); the second part, Module Manager (see
Section 2.4) and Export Module (see Section 2.4.4), Administration Account (see Section 2.4.1) with
the Pack Menu available (see Section 2.4.3) and a Slide Editor (see Section 2.4.2); the third part, user
interface of the web browser, allowing for data collection using a special SWT browser (see Section 2.4.3);
and the fourth part, Selection Application (see Section 2.4.5).
The use and development of the Hypothesis platform are expected to be based on two principles:
sharing and accumulation. Effective accumulation requires that the database is located in a centrally
accessible server. “Sharing” involves not only utilization of already created Packs or their parts, but
also the possibility to take part in their creation. The web-based design of the platform allows the
publication of research studies along with the original stimulus (i.e., the tests used in the experiments)
and experiment design. In addition, raw data can be made available for the research community.
The process of “accumulation” can be done in two ways. First, varying tests and/or individual tasks
can be created and then made accessible for future use (for instance, psychodiagnostic tests), whether
in their original or partially modified form. The second method of “accumulation” consists of gradual
implementation of new functionalities necessary for the creation of new tests. The newly created
ISPRS Int. J. Geo-Inf. 2017, 6, 407 4 of 22
functionalities are then made available in the form of Slide Templates. Gradual accumulation of slide
templates broadens the usability potential of the platform.
2.1. Hierarchical Structure of Packs
The highest-level unit of the hierarchy is a Pack (basically, a test) which is further divided into
lower-level units. Each Pack consists of a set of slides, usually linearly arranged, meaning that they are
presented to the participant in a fixed order. The option of random, tree or cyclic ordering of slides
is also available (see Section 2.2). The slides are arbitrarily clustered into Tasks (usually based on
the thematic link). Typically, the administrator includes several Tasks in a Pack, for instance: Task 1.
Introduction and personal information; Task 2. Instructions; Task 3. Training task; Task 4. Test tasks;
Task 5. Feedback and Conclusion. The clustering of slides into sections allows for simple and effective
utilization of the thematic clusters in later tests. The individual levels of the Pack hierarchy are defined
in the Database (Figure 2).
Figure 2. The content of Slide, Task and Branch Tables is associated with a higher level in the hierarchy
through the following intermediary tables: Task_Slide, Branch_Task and Pack_Branch.
The basic unit of each Pack is the Slide. There are two components in each Slide: Slide Template
and Slide Content (Figure 3). Each Slide Template should contain all the details concerning the structure
of the relevant Slide; most importantly, it should include information on the visual arrangement of
components, dialog windows, and functional logic of the Slide. Slide Templates (and Slides) are stored
in the Database and are available to be used (whether in their original or customized form) by other
research groups and in different experiments. The “functional logic” of a Slide concerns the response
by the system to the operations performed by participants. The response is ensured by various control
tools (for example, zoom, map shifting, point (polygon, broken line etc.) plotting, dialog window,
timer), means of control (e.g., clicking, pressing of a button), definitions of actions related to the above
events, definitions of variables used in each slide, definitions of any counters and, finally, definitions
of slide-related output values saved into the Database. Each Slide Content is linked to the relevant
Slide Template and determines the content that is to be presented in each Slide. For instance, if a
Slide Template contains information about buttons (including their format), a picture and a text field,
the Slide Content defines what buttons and picture are to be shown, what text is to be presented in the
text field, and so on.
ISPRS Int. J. Geo-Inf. 2017, 6, 407 5 of 22
Figure 3. Database tables with hierarchical relations between Slide Templates, Slide Contents and Task
slides. The slide template table contains Slide Templates defining the character of Slides. Linking of
Slide Templates to Slide Contents is ensured by GUIDs. Typically, several Slide Contents are linked to a
single Slide Template.
Tasks are linked to Packs indirectly, through Branches; they are linearly arranged within these
Branches and presented to the participants in a fixed order. The purpose of a Branch is to allow for
the effective creation of Packs for Computerized Adaptive Testing. In its simplest, linear form, a Test
comprises a single Branch with a given number of Tasks (Figure 4a). More complex Packs have a tree
structure, with participants going through the branches based on their current performance. Each
Branch can be cyclically repeated until the participant achieves the required level of performance (e.g.,
percentage of correct answers in the training task; Figure 4b).
Figure 4. (a) Structure of Pack No. 5 with only one branch (5). The branch contains two Sections (Tasks)
(3 and 4) with several linearly arranged Slides; (b) Pack No. 6 with multiple branches.
2.2. Computerized Adaptive Testing
The Hypothesis platform makes it possible to conduct Computerized Adaptive Testing (CAT) [43,44]
at several levels of the hierarchy: slide level, task level, and branch level. At the slide level, events (e.g.,
ISPRS Int. J. Geo-Inf. 2017, 6, 407 6 of 22
mouse clicking) are paired with particular actions, with variables being assigned specific values which
are subject to arithmetic and logic operations. Available algorithms include branching algorithms
(IF-THEN-ELSE, SWITCH) and loops (WHILE), which are used to control slide-related actions. For the
purposes of slide-level CAT optimization, a specialized slide was created (Image-sequence Layer; see
Section 3.8). At the task level, each slide in a sequence of slides can be chosen based on the output
related to the previous slide as well as on a set of pre-defined rules. The rules determining the course of
each Task are a part of the Task table in the Database. A special way of ordering slides is randomization.
In other words, there are three possible ways of slide ordering: linear, randomized, and adaptive.
Adaptive ordering is available for the branch level as well, where each branch is chosen at the end of
the previous branch.
2.3. Multi-Player/Task Tests
A significant aspect of the platform is the multi-player/task mode. It allows for real-time
participation of two or more users in solving a single task (asynchronous testing); alternatively,
one user may work on two different but interconnected tasks (synchronous testing).
2.3.1. Multi-Task Mode (Synchronous Testing)
Synchronous testing involves master and slave Packs. The Multi-task mode was developed for
experiments requiring the participant to complete several tasks simultaneously, or to attend to several
screens at the same time. The Multi-task mode was tested using an adaptation of the Peripheral
Perception Test (PP-R Periphere Wahrnehmung-R) developed by Schuhfried [45]. The test is a part of
the Vienna Test System (Germ. das Wiener Testsystem) and requires specialized hardware [46]. In the
pilot experiment, whose purpose was to verify the synchronous testing functionality, the participants
were asked to simultaneously complete a primary task (by clicking on target objects on the screen) and
secondary tasks on peripheral screens using keyboard keys (Figure 5).
Figure 5. Multi-task mode scheme: while completing a primary task displayed on the central,
mouse-operated screen (Master Pack), the participant uses keyboard to react to target stimuli presented
on peripheral screens (Slave Packs).
ISPRS Int. J. Geo-Inf. 2017, 6, 407 7 of 22
2.3.2. Multi-Player Mode (Asynchronous Testing)
The Multi-player Mode was originally intended for research in the field of geography, online
collaboration respectively [47,48]. The mode makes it possible for several users to participate in solving
a particular task. All the tests involved are controlled independently by the respective users; however,
pauses in synchronicity may occur where an event requires a particular operation to be performed.
While all participants are working on the same task, it is the administrator’s choice to define the
conditions determining the course of testing, including, for instance, which participant has the first
turn, in what order a picture is shown to the participants and so on. The Multi-player Mode has
already been utilized in a series of adaptations of experiments related to social psychology (Figure 6)
and behavioral economics [49,50].
Figure 6. Use of the Multi-player Mode in a computerized adaptation of Asch conformity experiments.
First, all three participants were simultaneously shown an identical object; then, they were asked to
estimate its size by drawing a corresponding line. The line was then displayed to all the participants.
2.4. Module Manager
The Manager Module represents an interface through which the software is accessed. Three user
roles are defined: User, Manager and Superuser. “User” is at the lowest access level and can only
select and start the tests made available by higher-level users. Superuser and Manager can access the
Administrator Account, Export Module, and Slide Editor. Login is done via entering a user name and
password. Unlogged users can enter the Module Manager as Guests, in which case only the free Packs
are available.
2.4.1. Administrator Account
In the Administrator Account, each user can be provided with a name, password and access
level (user role). The Superuser (highest-level user role) has unlimited administration rights with
respect to all the other users, tests, and results. A Manager has administration rights over his/her
group, within which s/he can create user accounts, allot Packs, and access all the results obtained
by members of his/her group. S/he can also use the Slide Editor. The User role is to be assigned to
research participants, whose rights are limited to starting a test Pack. Expiration date and manner of
starting each Pack are specified by the Manager or Superuser (Figure 7). The Administrator Account
allows for mass creation and editing of multiple user accounts; also, it makes it possible to export
ISPRS Int. J. Geo-Inf. 2017, 6, 407 8 of 22
lists of user accounts (along with all the necessary details, including e.g., automatically generated
passwords) in .xlsx format.
Figure 7. Sample Administrator Account. Editing of a new user account.
2.4.2. Slide Editor
Slide Editor is a part of the Module Manager functionality at the Superuser and Manager levels.
In order to edit a slide using the editor, the user needs to copy its XML code (related to Content and
Template) into corresponding windows of the Module Manager. The given slide then can be opened
and edited. Verified XML codes are recorded into the Database.
2.4.3. Testing Modes and Pack Menu
In Hypothesis, there are several options for starting and running a test; of these, the most
suitable one is chosen by the administrator. For the purpose of data collection where no control over
experimental conditions is required (e.g., collection of data through online questionnaires), the test can
be run using a standard web browser. The test opens in a pop-up, full-screen window; alternatively,
a new browser window can be opened by clicking on a HASH link. The administrator can choose
whether a user will be allowed to open a particular test in the controlled (featured) mode, in the
standard (legacy) mode, or whether they will be allowed to choose (Figure 8). The controlled mode
is realized through a browser based on SWT components. When opened, the SWT browser [51]
requires installation of Java Runtime Environment. In the controlled mode, nearly all standard
browser functions are disabled that could disrupt the highly controlled environment of the experiment.
The SWT browser will open the test in a full-screen mode and will prevent the user from closing the
application using standard means (Alt + F4 or Esc key); page refreshing and context menu are disabled
ISPRS Int. J. Geo-Inf. 2017, 6, 407 9 of 22
as well. As Java gradually ceases to be supported in web browsers, an alternative way of running a
controlled mode is currently being tested (see Section 6).
Figure 8. Opening of a Pack in a controlled (featured) or standard (legacy) mode.
2.4.4. Export Module
In the Hypothesis platform, operations and events related to the course of testing are saved in
the Database (table of Results). Key target variables are defined by the author (maker) of the test,
who can also define the rules for their clustering and evaluation. The pre-defined values are then
saved as special variables called Slide Outputs. The Export Module (Figure 9) is a part of the Module
Manager; here, test Packs to be exported are selected (based on, for instance, the time of administration).
The exported file is in .xslx format and contains raw data for further processing.
Figure 9. Export Module with finished tests (marked).
ISPRS Int. J. Geo-Inf. 2017, 6, 407 10 of 22
2.4.5. Selection Application
Effective selection (filtration) of variables from the raw data file is done in the selection application
created in Visual Basic. The application searches through the data file, selecting values or chain parts
in accordance with pre-defined rules. It always contains the following triplet:
1. List of raw data;
2. List of pre-defined variables, whose values are to be selected;
3. List of target values.
3. Slide Functionality
An important characteristic of the Hypothesis platform is modularity, which allows for continual
development and broadening of the functionality of the software. The platform can be viewed as
a high-performance core consisting of a content manager, processor for slide creation, and event
logger with data store. An Open API makes it possible to create external modules and/or functions.
The default configuration contains a set of basic components, including screen segmentation (vertical
and horizontal), form features (e.g., TextField, ComboBox, SelectPanel), and also Button, Image,
Panel, Label, and components for Audio and Video. Specific functionality is represented by a Dialog
Window and Timer. Other components are a part of two separately supplied modules: Maps (for
interactive maps) and Processing (for interactive animations created in the Processing language). See
Supplementary Materials for examples of functionality.
3.1. Questionnaires
Questionnaires are commonly included in psychological research experiments. They may
range from simple, personal information forms to Likert-type questionnaires. For the purpose of
questionnaire creation, the software offers text fields, dropdown lists, scales, and others, with the
possibility of intermediate validation of input values, which can be used to check whether obligatory
information has been entered.
3.2. Visual Stimuli
Typically, visual stimuli are in the form of images. All standard raster graphics formats are
supported (PNG, JPEG, GIF and similar ones). It is recommended to choose the format based on the
type of content and with a view to minimum size of the file. For instance, GIF and PNG are suitable
for schematic images with few colors, while JPEG should be used for photographs. The BMP file
format is not recommended because it uses no data compression which results in enormously large
files. When creating a slide with an image, longer loading time is to be expected, depending primarily
on the file size and internet connection speed. Web browsers usually load images from cache, which
makes loading of previously displayed images (identical image url) much faster. Different loading
times can be compensated for by temporary masking of the images while they are loading. The Image
component ensures recording of mouse clicks, which are provided with pixel coordinates.
3.3. Slide Control
Interaction with slide content can be realized through buttons or a button panel, e.g., for choosing
between several options; alternatively, keyboard keys can be used (alphanumeric, arrow, or functional).
The course of the test can also be controlled externally by Timer, with an optional display of time
left. The timer can be stopped at any time and/or repeatedly activated. The most frequently used
timer-related event is a timer end; however, timer start can be utilized as well. Timer can also be used
to indicate that a pre-defined time interval has passed (event update). Button clicking, key pressing,
and timer events are usually linked to slide-level actions.
ISPRS Int. J. Geo-Inf. 2017, 6, 407 11 of 22
3.4. Control of User Actions
At the slide level, various user actions can be defined which represent independent subroutines
with specific names. Actions consist of evaluable expressions; these may be logical, mathematical,
object-variable related, or branching expression-dependent (IF-THEN-ELSE, SWITCH-CASE). They
are used to manage events, including, for instance, timer stop, key pressing, and clicking on a
button/picture. Importantly, actions can call other actions, which is especially useful when branching
is used, or when a more complex action is compiled and called by the individual parts. The advantage
of actions is that they prevent the need to use an identical code several times; instead, a single action is
called. Also, each action triggering is saved in the table of events. It follows from the above that using
actions leads to higher transparency and comprehensibility of the expression of a functional algorithm
related to a particular slide.
3.5. Dialog Window
The Dialog Window function may be used in order to repeatedly display Help while solving a task.
The window appears after the calling of an appropriate function, which can be done e.g., by button
clicking or any other pre-defined action. Closing is done by clicking on the cross button. Both the start
and end of an action are saved into the Database. Dialog Window can either be non-modal (while
it is active you can still work with the slide) or modal, meaning that the main window is disabled.
Modal windows can be used to distinguish between currently running activities of the participant.
The participant can only continue with his/her work after closing the modal window.
3.6. Maps
The map-related component is a part of the external plug-in module. The main purpose of the
component consists of the implementation of maps into experiments. We distinguish between two
main map layers: Base Layer and Feature Layer (with vector graphics). Examples of base layers include
interactive Web Map Service (WMS) maps provided by servers offering WMS. The WMS layer requires
specification of a coordinate reference system (CRS) and a bounding box. Depending on the data
provided by the server, the WMS layer can contain either a single type of information or a complete set
of cartographic details. Interactive features include zooming and shifting of the bounding box. Each
change sends a request for data update to the server.
The Feature Layer contains vector objects; these can either be part of the default setting or they
can be created by the participant using one of the special vector graphics tools. Various parameters of
vector objects can be set including: stroke line color/style and fill style and color. Transparent vector
objects are suitable to be used as Areas of Interest and may be utilized to assess a participant’s actions.
Another important map layer is one with a static image, called Image Layer. This type of layer can
be used instead of the “Image” basic component because it contains both the options offered by the
Vector Layer and tools for vector object creation. The map component makes it possible to gather
data about clicking on an object and/or using tools for the creation of lines, sections, broken lines,
polygons and other geometric shapes/objects. If a coordinate reference system is specified (especially
in connection with the WMS layer), the collected data may contain not only pixel coordinates, but also
real geographical coordinates.
3.7. Audio and Video
The Hypothesis platform has two basic multimedia components: Audio (typically for MP3 audio
format) and Video, which is usually used for MP4 files. An advantage of the Video component involves
the possibility to record mouse clicking in a way similar to a static image. In addition to coordinates,
the current time of audio-visual elements is recorded as well.
ISPRS Int. J. Geo-Inf. 2017, 6, 407 12 of 22
3.8. Image-Sequence Layer
The Image-Sequence Layer component significantly broadens the applicability of the Hypothesis
platform. A test typically consists of a sequence of slides, which requires some loading time to be
allowed. Therefore, no fast sequences of visual stimuli (at the order of tens of millisecond) would be
possible. The speed of presentation of visual stimuli is normally limited by the speed and reliability
of client-server communication. In order to overcome the above problem, a special component was
designed which allows pre-loading of the stimulus material into the cache of the browser so that
tachistoscopic tests can be performed [52]. Here, a presented slide is inactive (masked) until all
the content has been loaded. Another key characteristic of the component is slide programmability.
An algorithm defining the course of a slide can be written which also allows for slide-level adaptive
testing (see above).
3.9. Processing.js Component
Another external module which broadens the applicability of the software is called Processing.
Its purpose consists of the utilization of applications created in the Processing language (i.e.,
its web implementation, Processing.js), a visual programming language featuring a user-friendly
developmental interface suitable for those with only basic knowledge of programming. The language is
intended for data visualization, interactive animation, videogames, and other graphic material [53,54].
Processing.js is the sister project of the Processing programming language; it has been designed
for online environments. The Processing code is based on JavaScript and can be run by any
HTML5-compatible browser [55]. The interface between the Hypothesis platform and Processing.js
makes it possible to create and administer a range of highly interactive tasks with visually rich and
complex environments. Due to two-way communication between the Processing application and the
core of the Hypothesis platform, the course of a task running in Processing.js can be controlled; at the
same time, key events are being saved (in a pre-defined form) into the Database of the Hypothesis
platform. Using the so called “call-back method”, each event/action occurring in Processing.js can call
an action defined within a slide, which will affect the course of the slide. Similarly, a slide can call the
processing code and thus influence the running of the Processing application.
4. Technical Design
The platform has been developed using the modern technology of the dynamic web page. The core
and user interface are built on the Vaadin 7 framework [56]; database operations are ensured by
ORM Hibernate, and PostgreSQL version 9.1 (or higher) is used as the primary database system.
The application architecture is three-layer; a client, server, and database (Figure 10). The client part is
responsible for user interaction and its operation is provided by a standard web browser (thin client)
or a special browser distributed in the application package—Hypothesis Browser. This browser is
built on Standard Widget Toolkit components and ensures more strict conditions for running tests.
The client layer communicates in the background with the server through the technology Ajax RPC
(remote procedure call). The server layer is implemented as a servlet of the application server (e.g.,
Apache Tomcat) which is responsible for the client’s request handling and updating the user interface.
The servlet then communicates with the database layer by methods of object mapping of entities
through the Hibernate library. This library provides a unified interface for the connection to all
commonly used database systems (PostgreSQL, MySQL, MS SQL, Oracle, etc.). Individual Packs (test
batteries) are structurally stored in the database. After starting a test, a selected package is loaded from
the database to the server application and a new test entity is created. For more details see [42].
ISPRS Int. J. Geo-Inf. 2017, 6, 407 13 of 22
Figure 10. Scheme of the technical structure (adapted from Šašinka et al. [20]).
Time Measurement Accuracy
In the development of the software, great attention was paid to time measurement precision.
The disadvantage of a client-server solution (as opposed to desktop applications), especially if the
system responses are controlled by the server (which is the case of Hypothesis), which can lead to
time delays during client-server communication. Delays in the server-to-client direction may occur
e.g., as a result of the loading of images from a distant store. Client-to-server communication leads
to a delayed response of the system to user actions. For instance, when a mouse click is sent from
client to server, a delay occurs both in saving the event of clicking and sending back the system
response. Negligible as they may seem (they are typically in the order of milliseconds), the delays
are not invariable. The variability in client-server communication thus represents “noise” resulting
in measurement inaccuracies. In order to compensate for this noise, several solutions have been
implemented. First, the time of event occurrence at the client is recorded in addition to the time the
event was accepted by the server, with the difference between these two representing the immediate
time delay. When the system is located in a local network the delay tends to be negligible; with an
online system it is dependent on external factors. Time delays can also be eliminated by the use of
specialized “Image-Sequence Layer” slides (see Section 3.8) which prevent the delays related to the
loading of content of the slide. Slides using the Processing.js plug-in relegate most of the event control
to the client; therefore, the whole task is controlled locally, with only key events being sent to the server
(for more details see the Supplement). Implementation of Processing.js fully eliminates the limits of
the client-server solution and makes it possible to create tasks with high time accuracy (e.g., when an
eye-tracking system is used).
5. Combination with Eye-Tracking
Although primarily intended for extensive research, the Hypothesis platform has, from the
very beginning, been designed to support a relatively new intensive research method: eye-tracking.
The use of Ogama, an already existing open-source application [57–59], and subsequent development
of an online version called HypOgama made it possible to perform experiments involving a remote
eye-tracking system SMI RED250 mobile or The Eyetribe. The existing solution is based on ex-post
synchronization of an eye-tracker with the Hypothesis [60]. The above solution is a part of a wider
concept which includes a ScanGraph tool for explorative data analysis [61]. At present, another
version is being developed which will allow unified connection and real-time interaction between the
Hypothesis platform and an eye-tracker. The solution is based on WebSocket communication elements
of the Vaadin framework and its functionality has been verified in practice.
6. Comparison to Similar Tools
Nowadays, a range of tools are available which can be used to conduct computer-based
experimental research. The development of the Hypothesis platform was motivated by the fact that
ISPRS Int. J. Geo-Inf. 2017, 6, 407 14 of 22
there are currently no tools that would make it possible to collect a real-time record of an individual’s
work with interactive maps and of collaboration between multiple users. The software’s architecture
is highly universal and offers a unique combination of functions, one which is not provided by any
other tool currently available. The software obviously has its limitations, too, and some types of
experiments may want more specialized tools. Some applications, such as Presentation [62] and
Psychtoolbox [63,64] were designed for experiments where high levels of precision are required;
they are used for instance in neurocognitive research. Another example is E-prime [65], a software
application with a large user base, which makes it possible to gather data using Tobii and SMI
eye-tracking systems. There are numerous other applications available which offer eye-tracking-based
data collection, including, for instance, the Paradigm software application [66], which allows for
experiments to be conducted using both desktop and mobile devices. Free software applications
from the above category include OpenSesame [67] and PsychoPy [68]; the latter is available online.
In implementing eye-tracking into research experiments, some applications go a step further than the
above-mentioned ones, combining inventive data collection using an eye-tracker with effective data
analysis. Examples include the “Experiment Builder” and “Data Viewer” [69], Experiment Center
and BeGaze [70] and Ogama [57]; all of these are applications provided by eye-tracker manufacturers.
The limitations of these applications are related to basic functions in test creation. For instance,
the Experiment Center and Ogama do not allow for several Likert scales to be used within a single
slide; also, they do not make it possible to collect data online. Examples of free psychology software
applications for online data collection include PEBL [71], Tatool [72], Social Lab [73] and jsPsych [74].
The latter two web-based tools in particular are conceptually similar to the Hypothesis platform, which,
however, offers significantly broader functionality, with an emphasis on the controlled nature of the
experiment. Among the areas which can benefit from the advantages of the Hypothesis platform
is the field of cartography, which has a long tradition of employing usability tests; the importance
of usability studies for cartography was emphasized in the International Cartographic Association
Research Agenda published in 2009 [75]. Yet, the field has long been lacking a suitable tool for
conducting usability tests. The lack of tools for usability studies in cartography is mentioned in
Nivala et al. [76]. Even recently published papers still use domain-unspecific software and tools [77]
and self-developed tools created ad hoc [78]. To our knowledge, cartography at present has no
universal experimental tool for usability tests; thus, researchers have to use the software and tools
mentioned above. Unlike these, the Hypothesis platform makes it possible to collect data online and in
a controlled environment, offering high temporal precision of measurements. It is also suitable for
multitask/multiplayer experiments and for investigating a user’s behavior when they are working
with interactive maps.
7. Test of Resources
During its development, the Hypothesis application was tested several times; also, feedback was
requested from the researchers who used the platform when carrying out their experiments. In 2015,
Z. Štˇerba and J. ˇCenˇek (both Masaryk University) performed an intercultural study consisting of a
series of experiments in which 106 university students participated. The experiments were conducted
at universities in the Czech Republic, China and Switzerland. The study used a set of tests involving
different functionalities (including clicking, line drawing and Likert scales). When administering
the tests, two relatively marginal problems were encountered. One of them consisted in different
load times of the stimulus material, which was caused by varying quality of internet connection in
different places; the application runs on a server located at Masaryk University in Brno, Czech Republic.
A more substantial problem was related to the instability of the system during mass testing, when
administering 10 to 15 (and more) tests simultaneously. There were several cases of the application
“freezing” in mid-test during the first load test. A similar situation was encountered by Svatoˇnová and
Kolejka [79] or Knedlová [80] and Helísková [81] while they were conducting research towards their
theses. When a high number of tests were run simultaneously, the test shut down prematurely, leading
ISPRS Int. J. Geo-Inf. 2017, 6, 407 15 of 22
to a loss of data. The tests were administered to 52 participants at the premises of Masaryk University.
The functionalities used in the tests included multi-clicking (with feedback), line and area drawing with
automatic evaluation, text fields, scroll-down windows, an image-sequence layer slide with a button
bar and feedback, and keyboard control. Apart from the limitation in the number of tests that could be
run simultaneously, no other significant problem was encountered during the testing. The cause of the
problem was revealed to be related to “sessions”, which are automatically created upon starting a test
but did not always close as expected. After the error was resolved, the application was used by other
research groups, for instance by Kubíˇcek et al. [82], who conducted a study concerning mental rotations
where a maximum of 5 participants were tested simultaneously; Opach et al. [83] (individual testing of
25 participants). No problem related to the Hypothesis platform was reported by any of the above
research groups. After the implementation of the newest version of the platform, another load test was
carried out. The test consisted of logging into the Manager Module of the platform 16 times (8× in
Firefox and 8× in Chrome) and launching 82 identical tests simultaneously. Each test contained 2 slides
of the image-sequence-layer type. As was expected, no complications occurred during the second load
test, with all the data collected being saved into the database (see Supplementary Materials).
8. An Experiment Illustrating the Usability of the Hypothesis Platform
8.1. Introduction
We are presenting a conducted experiment in order to document the wide range of functionality,
usability and stability of the Hypothesis platform. Our colleague, Markéta Kukaˇnová, conducted
a complex study which explored the effectiveness and efficiency of contextual cartographic
visualizations [84]. She compared work with usual topographic map and work with a combination of
topographic and transport maps. She conducted a simulation using contextual visualization techniques
in the transport planning and also measured the cognitive abilities of users which may have positively
influenced performance during the map reading process. Because of the above-mentioned reasons,
she used the Hypothesis platform, which made the creation and administration of all the necessary
tasks and tests possible.
8.2. Methods
The main goal of the empirical study was to compare two variants of spatial data representation.
Throughout the entire experiment, the control group only worked with standard military topographic
maps. Alternately, the experimental group worked with standard military topographic maps and
with adapted visualization focused on the tasks connected with transportation. Both visualizations
were informationally equivalent. The whole battery consisted of two cartographic tests and four
psychological tests (identical for both groups; Figure 11).
Figure 11. Scheme of the whole experimental design (adapted from Kukaˇnová [84]).
ISPRS Int. J. Geo-Inf. 2017, 6, 407 16 of 22
The main principle of the contextual maps is that the visualization adapts to the specific activity
of the user. During free map exploration or a visual search for target objects, the basic topographic
map is used. When the type of activity is changes e.g., to transportation planning, visualization is
switched to a specific transport map, where the transport infrastructure is highlighted and unnecessary
information is suppressed (Figure 12).
Figure 12. Topographic map (a) and contextual transport map (b).
Cartographic test 1 consisted of two subtests. The participants should search for target objects in
the first subtest. The second subtest was similar to the first one but the participants were listening to an
audio recording simultaneously, and were expected to react with a key press according to word cues.
The correctness, visual search time, and reaction time to target words were measured. Alternately,
the experimental group worked with topographic and transport maps. The level of cognitive load was
explored in these types of tasks.
Cartographic test 2 consisted of four subsequent stages. Participants explored a map with
multi-click marked target objects in the first stage. The reaction time and correctness was measured. In
the second stage, participants localized a fire based on the instructions; afterwards they located the
position of the fire truck. In the third stage, participants drew the optimal route between the fire and
the fire truck. The control group always worked with a topographic map while the experimental group
worked with a transport map during the third stage. The routes drawn were recorded to a database
as were their reaction times. In the final and fourth stage, a blank map was presented. Participants
were required to estimate the position of the objects which were previously presented in the first three
stages. Accuracy of the marked objects was measured in pixels.
Every single test from psychological portion used different functionality. The Flagtest
(measurement of selective attention) involved, among others, following functionality: multi-click,
timer or feature layer. The Compound Figure Test (measurement of global/local precedence) used
ISPRS Int. J. Geo-Inf. 2017, 6, 407 17 of 22
image sequence layer, real-time feedback, automatic evaluation of the assessment, button bar. In the
Absolute Relative Test (measurement of holistic/analytic cognitive style) users estimated the size
of previously seen objects with the help of drawn horizontal lines. Hypothesis calculated deviation
(accuracy) in pixels. And finally, the Stroop Spatial Test (measurement of cognitive control) was based
on image sequence layer, participants used a keyboard to react to the stimuli.
Totally 43 individuals (age was between 18 and 44; m = 26.6; men = 24, women = 19) participated
in the study. There were 20 participant in control group and 23 in experimental group. A part of
the research sample (n = 27) was tracked simultaneously with the eye-tracking system (The Eyetribe
eyetracker and software OGAMA).
8.3. Results
After the data collection was finished, all data were exported to .xlsx files from the database of
Hypothesis with the help of the export module. Afterwards, the selection application was used and
the required variables were filtered and stored in a format enabling subsequent analysis. Data were
analyzed in the IBM SPSS program and some analysis were also provided and visualized with the help
of ArcGIS 10.2.2. No problems occurred during data collection that could be linked to the Hypothesis
platform. However, some participants did not finish all the tests included in the test battery because
of problems with the administrator’s PC and internet connectivity issues. Also, there was some data
loss with those participants who were measured with the eye-tracking system. The OGAMA software
showed instability during longer tests. Figure 13 shows an analysis of the routes drawn in cartographic
experiment 1. Green digits refer to the experimental group (context transport visualization), blue refer
to the control group (topographic map). Routes drawn by 36 participants were analyzed.
Figure 13. Yellow lines (thickness) express frequency of the routes chosen by the experimental group
(green digits) and control group (blue digits) together. Data from both groups are presented at once in
the transport map.
ISPRS Int. J. Geo-Inf. 2017, 6, 407 18 of 22
8.4. Discussion
The experiment conducted by Markéta Kukaˇnová clearly showed that the Hypothesis platform
provides a range scale of functionality which enables the creation and administration of very different
types of tasks. We are not aware of another research tool which would be able to provide all the
necessary functionality for the realization of the above-described study. Without Hypothesis, a research
study of this nature would have to use two or more different tools, and this of course would increase
the costs of the research and make it more complicated. The Hypothesis platform also enabled the
exportation and filtering of all the necessary variables in order to conduct the intended analysis.
Figure 13 showed that even a research variable which has many degrees of freedom could be properly
analyzed. These features offer a strong advantage over other research tools which are mostly able to
present only very simple stimuli and collect simple responses such as a mouse click or a keypress.
The most important thing should always be the robustness of the research tools. Hypothesis worked
without a single problem, however, the general limitations of the web-based tools were detected.
The stability of the internet connections played a crucial role in this case, too. An internet connection lost
in even one single test may devalue the data from the affected participant for the entire study. For these
reasons, it should be recommended that data be collected only under the highest quality conditions.
9. Conclusions
An original research tool, the Hypothesis platform, was described and its usability was presented
in detail in the context of cognitive cartography and psychological testing. Its comparison to other
research tools was presented, as was the testing and evaluation of the Hypothesis platform. One
conducted experiment was chosen and described in more detail in order to make visible the range of
functionality which was involved in one particular study. Other experiments were also referred to,
some which were conducted by students for their diploma thesis, some by researchers who published
their work in well-established journals. The usability of the current version of the Hypothesis platform
was proven for research purposes and for educational purposes. However, the software is still being
developed and new functionality is constantly incorporated in order to react to the development in the
area of cognitive cartography. Current developmental efforts focus on 3D technology implementation
(pilot experimental testing of 3D visualization using web technologies has already been made [85,86]),
optimization for tablets or other devices with touch screens [87], interconnection with an eye-tracking
system, and further development of a Firefox-based browser. The new application using Firefox was
able to replace the original SWT-browser based solution in controlled data collection. Also, an entire
graphical user interface is being developed which can be utilized in test battery creation, and which
will allow access (and editing) through a standard web browser. The Hypothesis platform is now
available for academic and other non-commercial purposes (under Masaryk University). However,
special agreements may be negotiated with respect to commercial use or other specific purposes.
Supplementary Materials: The test data are available at: https://doi.org/10.6084/m9.figshare.4697674.v1;
Installation of Hypothesis with examples of functionality for demonstrative purposes is available at (user name:
isprs password: isprs): http://demo-hypothesis.phil.muni.cz; Tutorial “how to modify slides and work with the
Slide Management”.
Acknowledgments: This research was funded by projects “Influence of cartographic visualization methods on the
success of solving practical and educational spatial tasks” [grant number MUNI/M/0846/2015] and “Integrated
research on environmental changes in the landscape sphere of Earth II” [grant number MUNI/A/1419/2016],
both awarded by Masaryk University, Czech Republic. This research was supported by the research infrastructure
HUME lab Experimental Humanities Lab, Faculty of Arts, Masaryk University too.
Author Contributions: Kamil Morong and ˇCenˇek Šašinka designed the architecture of the software. Kamil Morong
wrote the code of the software. ˇCenˇek Šašinka and Zdenˇek Stachoˇn designed and performed all empirical testing
and analyzed the data. ˇCenˇek Šašinka, Zdenˇek Stachoˇn and Kamil Morong wrote the paper.
Conflicts of Interest: The authors declare no conflict of interest.
ISPRS Int. J. Geo-Inf. 2017, 6, 407 19 of 22
References
1. Mulíˇcková, E.; Šafr, G.; Stanˇek, K. Context map—A Tool for Cartography Support in Crisis Management.
In Proceedings of the 3rd International Conference on Cartography and GIS, Nessebar, Bulgaria,
15–20 June 2010.
2. Herman, L.; Popelka, S.; Hejlová, V. Eye-tracking Analysis of Interactive 3D Geovisualizations. J. Eye
Mov. Res. 2017, 10, 1–15. [CrossRef]
3. Špriˇnarová, K.; Juˇrík, V.; Šašinka, ˇC.; Herman, L.; Štˇerba, Z.; Stachoˇn, Z.; Chmelík, J.; Kozlíková, B.
Human-computer Interaction in Real 3D and Pseudo-3D Cartographic Visualization: A Comparative
Study. In Cartography—Maps Connecting the World; Sluter, C.R., Cruz, C.B.M., de Menezes, P.M.L., Eds.;
Springer International Publishing: Basel, Switzerland, 2015; pp. 59–73.
4. Herbert, G.; Chen, X. A Comparison of Usefulness of 2D and 3D Representations of Urban Planning.
Cartogr. Geogr. Inf. Sci. 2014, 42, 22–32. [CrossRef]
5. Lammert-Siepmann, N.; Bestgen, A.-K.; Edler, D.; Kuchinke, L.; Dickmann, F. Audiovisual communication
of object-names improves the spatial accuracy of recalled object-locations in topographic maps. PLoS ONE
2017, 12, e0186065. [CrossRef] [PubMed]
6. Chmelaˇrová, K.; Šašinka, ˇC.; Stachoˇn, Z. Visualization of Environment-related Information in Augmented
Reality: Analysis of User Needs. In Advances in Cartography and GIScience; Peterson, M., Ed.; ICACI 2017,
Lecture Notes in Geoinformation and Cartography; Springer: Washington, DC, USA, 2017; pp. 283–292.
7. Robinson, A. Challenges and opportunities for web-based evaluation of the use of spatial technologies.
In Proceedings of the ICA Conference, The Pennsylvania State University, University Park, PA, USA,
3–8 July 2011.
8. Tullis, T.; Fleischman, S.; Mcnulty, M.; Cianchette, C.; Bergel, M. An empirical comparison of lab and remote
usability testing of Web sites. In Proceedings of the Usability Professionnal Association Conference, Boston,
MA, USA, 8–12 July 2002.
9. Koneˇcný, M.; Bˇrezinová, Š.; Drápela, M.V.; Friedmannová, L.; Herman, L.; Hübnerová, Z.; Koláˇr, M.;
Kolejka, J.; Kozel, J.; Kubíˇcek, P.; et al. Dynamická Geovizualizace V Krizovém Managementu, 1st ed.; Masarykova
univerzita: Brno, Czech Republic, 2011.
10. Evans, B.; Sabel, C.E. Open-Source web-based geographical information system for health exposure
assessment. Int. J. Health Geogr. 2012, 11, 1–12. [CrossRef] [PubMed]
11. ˇRezník, T.; Horáková, B.; Szturc, R. Geographic Information for Command and Control Systems
Demonstration of Emergency Support System. In Intelligent Systems for Crisis Management: Geo-Information
for Disaster Management (GI4DM) Lecture Notes in Geoinformation and Cartography; Zlatanova, S., Peters, R.,
Dilo, A., Scholten, H., Eds.; Springer: Berlin/Heidelberg, Germany, 2013; pp. 263–275. [CrossRef]
12. Kubíˇcek, P.; Šašinka, ˇC.; Stachoˇn, Z.; Štˇerba, Z.; Apeltauer, J.; Urbánek, T. Cartographic Design and Usability
of Visual Variables for Linear Features. Cartogr. J. 2016, 2016, 1–11. [CrossRef]
13. Kubíˇcek, P.; Šašinka, ˇC. Thematic Uncertainty Visualization Usability–Basic Methods Comparison. Ann. GIS
2011, 17, 253–263. [CrossRef]
14. Štˇerba, Z.; Šašinka, ˇC.; Stachoˇn, Z.; Kubíˇcek, P.; Tamm, S. Mixed Research Design in Cartography:
A Combination of Qualitative and Quantitative Approaches. Kartogr. Nachr. 2014, 64, 262–269.
15. Kubíˇcek, P.; Šašinka, ˇC.; Stachoˇn, Z. Vybrané kognitivní aspekty vizualizace polohové nejistoty v
geografických datech. Geogra-Sb. ˇCeské Geogra Spol. 2014, 119, 67–90.
16. Koneˇcný, M.; Kubíˇcek, P.; Stachoˇn, Z.; Šašinka, ˇC. The usability of selected base maps for crises management:
Users’ perspectives. Appl. Geomat. 2011, 3, 189–198. [CrossRef]
17. Stachoˇn, Z.; Šašinka, ˇC.; Štˇerba, Z.; Zboˇril, J.; Bˇrezinová, Š.; Švancara, J. Influence of Graphic Design of
Cartographic Symbols on Perception Structure. Kartogr. Nachr. 2013, 4, 216–220.
18. Kitayama, S.; Duffy, S.; Kawamura, T.; Larsen, J.T. Perceiving an object and its context in different cultures:
A cultural look at new look. Psychol. Sci. 2003, 14, 201–206. [CrossRef] [PubMed]
19. Witkin, H.A.; Oltman, P.K.; Raskin, E.; Karp, S. A Manual for the Embedded Figures Test; Consulting
Psychologists Press: Palo Alto, CA, USA, 1971.
20. Morong, K.; Šašinka, ˇC. Hypothesis—online software platform for objective experimental testing. In Applying
Principles of Cognitive Psychology in Practice: Conference Proceedings; Bartošová, K., ˇCerˇnák, M., Kukaˇnová, M.,
Šašinka, ˇC., Eds.; Masaryk University: Brno, Czech Republic, 2015; Volume 5, pp. 15–16.
ISPRS Int. J. Geo-Inf. 2017, 6, 407 20 of 22
21. Thorndyke, P.W.; Stasz, C. Individual differences in procedures for knowledge acquisition from maps.
Cogn. Psychol. 1980, 12, 137–175. [CrossRef]
22. Edler, D.; Bestgen, A.-K.; Kuchinke, L.; Dickmann, F. Grids in Topographic Maps Reduce Distortions in the
Recall of Learned Object Locations. PLoS ONE 2014. [CrossRef] [PubMed]
23. Crampton, J.W. Interactivity types in geographic visualization. Cartogr. Geogr. Inf. Sci. 2002, 29, 85–98.
[CrossRef]
24. Roth, R.E. An empirically-derived taxonomy of interaction primitives for Interactive Cartography and
Geovisualization. Trans. Vis. Comput. Graph. 2013, 19, 2356–2365. [CrossRef] [PubMed]
25. Popelka, S.; Brychtová, A. Eye-tracking Study on Different Perception of 2D and 3D Terrain Visualization.
Cartogr. J. 2013, 50, 240–246. [CrossRef]
26. Brychtová, A.; Coltekin, A. An Empirical User Study for Measuring the Influence of Colour Distance and
Font Size in Map Reading Using Eye Tracking. Cartogr. J. 2016, 53, 202–212. [CrossRef]
27. Hofmann, A.; Hošková-Mayerová, Š.; Talhofer, V.; Kovaˇrík, V. Creation of models for calculation of
coefficients of terrain passability. Qual. Quant. 2015, 49, 1679–1691. [CrossRef]
28. Devlin, A.S.; Bernstein, J. Interactive way-finding: map style and effectiveness. J. Environ. Psychol. 1997, 17,
99–110. [CrossRef]
29. Vakalis, D.; Sarimveis, H.; Kiranoudis, C.T.; Alexandridis, A.; Bafas, G. A GIS based operational system
for wildland fire crisis management II. System architecture and case studies. Appl. Math. Model. 2004, 28,
411–425. [CrossRef]
30. Ilmavirta, A. The use of GIS-system in catastrophe and emergency management in Finnish municipalities.
Comput. Environ. Urban Syst. 1995, 19, 171–178. [CrossRef]
31. Koneˇcný, M.; Friedmannová, L.; Stanek, K. An adaptive cartographic visualization for support of the crisis
management. In CaGIS Publications—Autocarto 2006; CaGIS: Vancouver, WA, USA, 2006; pp. 100–105.
32. Kubíˇcek, P.; Kozel, J.; Štampach, R.; Lukas, V. Prototyping the visualization of geographic and sensor data
for agriculture. Comput. Electron. Agric. 2013, 97, 83–91. [CrossRef]
33. Roth, R.E.; Ross, K.S.; MacEachren, A.M. User-centered design for interactive maps: A case study in crime
analysis. Int. J. Geo-Inf. 2015, 4, 262–301. [CrossRef]
34. Baum, W.M. From molecular to molar: A paradigm shift in behavior analysis. J. Exp. Anal. Behav. 2002, 78,
95–116. [CrossRef] [PubMed]
35. Mead, A.D.; Drasgow, F. Equivalence of computerized and paper-and-pencil cognitive ability tests:
A meta-analysis. Psychol. Bull. 1993, 114, 449–458. [CrossRef]
36. Kvˇeton, P.; Jelínek, M.; Voboˇril, D.; Klimusová, H. Computer-based tests: The impact of test design and
problem of equivalency. Comput. Hum. Behav. 2007, 23, 32–51. [CrossRef]
37. Ross, R.M. The D2 Test of Attention: An Examination of Age, Gender, and Cross-Cultural Indices; Argosy
University: Chicago, IL, USA, 2005.
38. Arnett, J.A.; Labovitz, S.S. Effect of physical layout in performance of the Trail Making Test. Psychol. Assess.
1995, 7, 220–221. [CrossRef]
39. Baddeley, A.D. A 3 min reasoning test based on grammatical transformation. Psychon. Sci. 1968, 10, 341–342.
[CrossRef]
40. Kozhevnikov, M.; Hegarty, M. A dissociation between object-manipulation and perspective-taking spatial
abilities. Mem. Cognit. 2001, 29, 745–756. [CrossRef] [PubMed]
41. Hegarty, M.; Waller, D. A dissociation between mental rotation and perspective-taking spatial abilities.
Intelligence 2004, 32, 175–191. [CrossRef]
42. Štˇerba, Z.; Šašinka, ˇC.; Stachoˇn, Z.; Štampach, R.; Morong, K. Selected Issues of Experimental Testing in
Cartography; Masaryk University: Brno, Czech Republic, 2015.
43. Žitný, P.; Halama, P.; Jelínek, M.; Kvˇeton, P. Validity of cognitive ability tests–comparison of computerized
adaptive testing with paper and pencil and computer-based forms of administrations. Stud. Psychol. 2012,
54, 181–194.
44. Gershon, R.C. Computer Adaptive Testing. J. Appl. Meas. 2005, 6, 109–127. [PubMed]
45. Karner, T.; Neuwirth, W. Die Bedeutung der peripheren Wahrnehmung in der verkehrspsychologischen
Untersuchung. Psychol. Österreich 2001, 21, 183–186.
46. Schuhfried, G. PP-R Periphere Wahrnehmung—R. Available online: https://www.schuhfried.at/test/PP-R
(accessed on 20 October 2016).
ISPRS Int. J. Geo-Inf. 2017, 6, 407 21 of 22
47. Krek, A.; Bortenschlager, M. Geo-collaboration and P2P Geographic Information Systems. Current
Developments and Research Challenges. Collaborative Peer to Peer Information Systems (COPS06)
Workshop-WETICE. Manchester, UK, 2006. Available online: http://www.sel.uniroma2.it/cops06/papers/
COPS06-Krek.pdf (accessed on 20 October 2016).
48. MacEachren, A.M.; Cai, G.; Sharma, R.; Rauschert, I.; Brewer, I.; Bolelli, L.; Shaparenko, B.; Fuhrmann, S.;
Wang, H. Enabling collaborative geoinformation access and decision-making through a natural, multimodal
interface. Int. J. Geogr. Inf. Sci. 2005, 19, 293–317. [CrossRef]
49. Asch, S.E. Effects of Group Pressure on the Modification and Distortion of Judgments. In Groups, Leadership
and Men; Guetzkow, H., Ed.; Carnegie Press: Pittsburgh, PA, USA, 1951; pp. 177–190.
50. Ciampaglia, G.L.; Lozano, S.; Helbing, D. Power and Fairness in a Generalized Ultimatum Game. PLoS ONE
2014. [CrossRef] [PubMed]
51. Cornu, C.H. SWT Browser: Viewing HTML Pages with SWT Browser Widget. 26 August 2004.
Available online: https://eclipse.org/articles/Article-SWT-browser-widget/browser.html (accessed on
25 October 2016).
52. Godnig, E.C. The Tachistoscope: Its History and Uses. J. Behav. Opt. 2003, 14, 39–42.
53. Shiffman, D. Learning Processing: A Beginner’s Guide to Programming Images, Animation, and Interaction, 1st ed.;
Morgan Kaufmann: Burlington, MA, USA, 2008.
54. Reas, C.; Fry, B. Getting Started with Processing, 1st ed.; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2010.
55. A Port of the Processing Visualization Language. Available online: http://processingjs.org/ (accessed on
25 October 2016).
56. Vaadin Ltd. 2016. Available online: https://vaadin.com/home (accessed on 25 October 2016).
57. Voßkühler, A.; Nordmeier, V.; Kuchinke, L.; Jacobs, A.M. OGAMA–OpenGazeAndMouseAnalyzer:
Open source software designed to analyze eye and mouse movements in slideshow study designs.
Behav. Res. Method. 2008, 40, 1150–1162. [CrossRef] [PubMed]
58. Voßkühler, A. OGAMA Description (for Version 2.5). A Software to Record, Analyze and Visualize Gaze and
Mouse Movements in Screen Based environments. 2009. Available online: http://www.ogama.net/sites/
default/files/pdf/OGAMA-DescriptionV25.pdf (accessed on 30 October 2016).
59. Voßkühler, A. OGAMA (OpenGazeAndMouseAnalyzer): An Open Source Software Designed to Analyze
Eye and Mouse Movements in Slideshow Study Designs. 16 May 2015. Available online: www.ogama.net
(accessed on 30 October 2016).
60. Popelka, S.; Stachoˇn, Z.; Šašinka, ˇC.; Doležalová, J. Eyetribe Tracker Data Accuracy Evaluation and Its
Interconnection with Hypothesis Software for Cartographic Purposes. Comput. Intell. Neurosci. 2016, 1–14.
[CrossRef] [PubMed]
61. Doležalová, J.; Popelka, S. ScanGraph: A novel scanpath comparison method using graph cliques
visualization. J. Eye Mov. Res. 2016, 9, 1–13. [CrossRef]
62. Neuro behavioral systems, Inc. Available online: www.neurobs.com (accessed on 24 February 2017).
63. Cornelissen, F.W.; Peters, E.M.; Palmer, J. The Eyelink Toolbox: Eye tracking with MATLAB and the
Psychophysics Toolbox. Behav. Res. Method. Instrum. Comput. 2002, 34, 613–671. [CrossRef]
64. Psychtoolbox-3. Available online: http://psychtoolbox.org (accessed on 24 February 2017).
65. Psychology Software Tools, Inc. Available online: www.pstnet.com (accessed on 24 February 2017).
66. Paradigm. Available online: http://www.paradigmexperiments.com (accessed on 24 February 2017).
67. Mathôt, S.; Schreij, D.; Theeuwes, J. OpenSesame: An open-source, graphical experiment builder for the
social sciences. Behav. Res. Method. 2012, 44, 314–324. [CrossRef] [PubMed]
68. Pliatsikas, C.; Johnstone, T.; Marinis, T. FMRI evidence for the involvement of the procedural memory system
in morphological processing of a second language. PLoS ONE 2014, 9, e97298. [CrossRef] [PubMed]
69. SR Research: Fast, Accurate, Reliable Eye Tracking. Available online: http://www.sr-research.com (accessed
on 24 February 2017).
70. SMI SensoMotoric Instruments. Available online: www.smivision.com (accessed on 24 February 2017).
71. PEBL. The Psychology Experiment Building Language. Available online: http://pebl.sourceforge.net
(accessed on 24 February 2017).
72. Von Bastian, C.C.; Locher, A.; Ruflin, M. Tatool: A Java-Based Open-Source Programming Framework for
Psychological Studies. Behav. Res. Methods 2013, 45, 108–115. [CrossRef] [PubMed]
ISPRS Int. J. Geo-Inf. 2017, 6, 407 22 of 22
73. Garaizar, P.; Reips, U.D. Build your own social network laboratory with Social Lab: A tool for research in
social media. Behav. Res. Methods 2014, 46, 430–438. [CrossRef] [PubMed]
74. De Leeuw, J.R. Jspsych: A JavaScript library for creating behavioral experiments in a web browser.
Behav. Res. Methods 2015, 47, 1–12. [CrossRef] [PubMed]
75. Virrantaus, K.; Fairbairn, D.; Kraak, M. ICA research agenda on cartography and GIScience. Cartogr. Geogr.
Inf. Sci. 2009, 36, 209–222. [CrossRef]
76. Nivala, A.-M.; Sarjakoski, L.T.; Sarjakoski, T. Usability methods’ familiarity among map application
developers. Int. J. Hum.-Comput. Stud. 2007, 65, 784–795. [CrossRef]
77. Ooms, K.; Coltekin, A.; De Maeyer, P.; Dupont, L.; Fabrikant, S.; Incoul, A.; Kuhn, M.; Slabbick, H.;
Vansteenkiste, P.; Van der Haegen, L. Combining user logging with eye tracking for interactive and dynamic
applications. Behav. Res. Methods 2015, 47, 977–993. [CrossRef] [PubMed]
78. Demšar, U. Investigating visual exploration of geospatial data: An exploratory usability experiment for
visual data mining. Comput. Environ. Urban Syst. 2007, 31, 551–571. [CrossRef]
79. Svatoˇnová, H.; Kolejka, J. Comparative Research of Visual Interpretation of Aerial Images and Topographic
Maps for Unskilled Users: Searching for Objects Important for Decision-Making in Crisis Situations.
ISPRS Int. J. Geo-Inf. 2017, 6, 231. [CrossRef]
80. Helisková, M. Možnosti A Limity Online Poˇcítaˇcové Diagnostiky. The Capabilities and the liMits of Online
PC Diagnostics. Master’s Thesis, Masaryk University, Brno, Czech Republic, December 2016.
81. Knedlová, P. Kognitivní styl a jeho vliv na ˇctení mapy. The Influence of Cognitive Style on Map Reading.
Master’s Thesis, Masaryk University, Brno, Czech Republic, 2016.
82. Kubíˇcek, P.; Šašinka, ˇC.; Stachoˇn, Z.; Herman, L.; Juˇrík, V.; Urbánek, T.; Chmelík, J. Identification of altitude
profiles in 3D geovisualizations: The role of interaction and spatial abilities. Int. J. Digit. Earth 2017, 1–17.
[CrossRef]
83. Opach, T.; Popelka, S.; Doležalová, J.; Rod, J.K. Star and Polyline Glyphs in a Grid Plot and on a Map Display:
Which Perform Better? Cartogr. Geogr. Inf. Sci. 2017. [CrossRef]
84. Kukaˇnová, M. Porovnání dvou typ ˚u vizualizací z hlediska percepˇcní a kognitivní zátˇeže a kognitivních
schopností jedince. Comparison of the two Types of Visualization in Terms of Perceptual and Cognitive Load,
and Personal Cognitive Abilities. Ph.D. Thesis, Masaryk University, Brno, Czech Republic, September 2017.
85. Herman, L.; ˇRezník, T. 3D Web Visualization of Environmental Information–Integration of Heterogeneous
Data Sources when Providing Navigation and Interaction. In ISPRS Archives of the Photogrammetry,
Remote Sensing and Spatial Information Sciences; Mallet, C., Paparoditis, N., Dowman, I., Elberink, S.O.,
Raimond, A.-M., Sithole, G., Rabatel, G., Rottensteiner, F., Briottet, X., Eds.; Copernicus GmbH: Gottingen,
Germany, 2015; Volume XL-3/W3, pp. 479–485. [CrossRef]
86. Herman, L.; Stachoˇn, Z. Comparison of User Performance with Interactive and Static 3D Visualization–Pilot
Study. In ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences; Halounová, L.,
Li, S., Šafáˇr, V., Tomková, M., Rapant, P., Brázdil, K., Shi, W., Anton, F., Liu, Y., Stein, A., et al., Eds.;
Copernicus GmbH: Gottingen, Germany, 2016; Volume XLI-B2, pp. 655–661. [CrossRef]
87. Herman, L.; Stachoˇn, Z.; Stuchlík, R.; Hladík, J.; Kubíˇcek, P. Touch Interaction with 3D Geographical
Visualization on Web: Selected Technological and User Issues. In ISPRS Archives of the Photogrammetry,
Remote Sensing and Spatial Information Sciences; Dimopoulou, E., van Oosterom, P., Eds.; Copernicus GmbH:
Gottingen, Germany, 2016; Volume XLII-2/W2, pp. 33–40. [CrossRef]
© 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).
International Journal of
Geo-Information
Article
Collaborative Immersive Virtual Environments for
Education in Geography
ˇCenˇek Šašinka 1 , Zdenˇek Stachoˇn 2,* , Michal Sedlák 3,* , Jiˇrí Chmelík 4,* , Lukáš Herman 2,
Petr Kubíˇcek 2 , Alžbˇeta Šašinková 1,5, Milan Doležal 4, Hynek Tejkl 3, Tomáš Urbánek 3,
Hana Svatoˇnová 6, Pavel Ugwitz 2,7 and Vojtˇech Juˇrík 3,7
1 Division of Information and Library Studies, Faculty of Arts, Masaryk University, 60200 Brno, Czech
Republic; cenek.sasinka@mail.muni.cz ( ˇC.Š.); st.betty@mail.muni.cz (A.Š.)
2 Department of Geography, Faculty of Science, Masaryk University, 61137 Brno, Czech Republic;
herman.lu@mail.muni.cz (L.H.); kubicek@geogr.muni.cz (P.K.); 172577@mail.muni.cz (P.U.)
3 Department of Psychology, Faculty of Arts, Masaryk University, 60200 Brno, Czech Republic;
449327@mail.muni.cz (H.T.); tour@mail.muni.cz (T.U.); jurik.vojtech@mail.muni.cz (V.J.)
4 Department of Visual Computing, Faculty of Informatics, Masaryk University, 60200 Brno, Czech Republic;
legacycz@mail.muni.cz
5 Department of Psychology, Faculty of Social Studies, Masaryk University, 60200 Brno, Czech Republic
6 Department of Geography, Faculty of Education, Masaryk University, 60300 Brno, Czech Republic;
svatonova@mail.muni.cz
7 HUME Lab, Faculty of Arts, Masaryk University, 60200 Brno, Czech Republic
* Correspondence: zstachon@geogr.muni.cz (Z.S.); m.sedlak@mail.muni.cz (M.S.);
jchmelik@mail.muni.cz (J.C.); Tel.: +420-549-494-925 (Z.S.); +420-549-494-382 (J.C.)
Received: 5 November 2018; Accepted: 18 December 2018; Published: 23 December 2018
Abstract: Immersive virtual reality (iVR) devices are rapidly becoming an important part of our
lives and forming a new way for people to interact with computers and each other. The impact and
consequences of this innovative technology have not yet been satisfactory explored. This empirical
study investigated the cognitive and social aspects of collaboration in a shared, immersive virtual
reality. A unique application for implementing a collaborative immersive virtual environment (CIVE)
was developed by our interdisciplinary team as a software solution for educational purposes, with
two scenarios for learning about hypsography, i.e., explanations of contour line principles. Both
scenarios allow switching between a usual 2D contour map and a 3D model of the corresponding
terrain to increase the intelligibility and clarity of the educational content. Gamification principles
were also applied to both scenarios to augment user engagement during the completion of tasks.
A qualitative research approach was adopted to obtain a deep insight into the lived experience of
users in a CIVE. It was thus possible to form a deep understanding of very new subject matter.
Twelve pairs of participants were observed during their CIVE experience and then interviewed either
in a semistructured interview or a focus group. Data from these three research techniques were
analyzed using interpretative phenomenological analysis, which is research method for studying
individual experience. Four superordinate themes—with detailed descriptions of experiences shared
by numerous participants—emerged as results from the analysis; we called these (1) Appreciation
for having a collaborator, (2) The Surprising “Fun with Maps”, (3) Communication as a challenge,
and (4) Cognition in two realities. The findings of the study indicate the importance of the social
dimension during education in a virtual environment and the effectiveness of dynamic and interactive
3D visualization.
Keywords: immersive virtual reality; collaborative immersive virtual environment; immersion; sense
of presence; telepresence; Head-mounted display; cyberpsychology; human–computer interaction;
collaborative learning; hypsography; contour lines; map literacy
ISPRS Int. J. Geo-Inf. 2019, 8, 3; doi:10.3390/ijgi8010003 www.mdpi.com/journal/ijgi
ISPRS Int. J. Geo-Inf. 2019, 8, 3 2 of 25
1. Introduction
Recent rapid and continuous development of immersive VR technology has opened the possibility
for a wide range of applications. Decreasing prices and easy accessibility are factors helping to
distribute these devices to different institutions as well as regular households. Immersive VR finds
a purpose in many fields, for example, in psychotherapy and diagnostics [1–4], cognitive training [5,6],
relaxation [7,8], rehabilitation [9,10], medicine [11,12], training in the industry [13–15], tourism and
cultural heritage [16–18], journalism [19], and sport [20,21]. The rich potential of immersive virtual
reality is also utilized in areas that use geographical data, for example, evacuation planning [22,23],
geospatial data exploration and analysis [24–27], navigation in urban areas [28,29], visualization of
spatial data quality [30], and urban planning [31].
Immersive virtual reality is also significantly employed as an educational tool in many areas. We
can find its educational application in domains such as engineering [32,33], biology [34,35], foreign
languages [36], geometry [37], emergency management [38], physics [39], design [40], geography
and earth sciences in general [41–45], and in other more singular domains such as martial arts [46]
and communication skills training for individuals with autism [47]. Virtual environments including
VR have a long tradition in geographical research and education [48–51], but until recently, user
experiences have only been rarely reported. Several recent studies analyzed the potential benefits of
immersive technologies for education in geography and task solving. Philips et al. [52] examined the
usage of immersive 3D geovisualization and its usefulness in a research-based learning module (flood
risk assessment). The findings of a qualitative student survey showed a range of benefits (improved
orientation in the study area, higher interactivity with the data, and enhanced motivation through
immersive 3D geovisualization) and suggested that an immersive 3D visualization can increase
learning effectiveness in higher education. Focusing specifically on hypsography education using
modern technology, Carrera et al. [53] studied the possibilities of Augmented Reality technology
(AR). They experimented with 63 students and tested the usability of AR to interpret relief (maximum
slope, visibility between points, contour interval, and altitude interpretation). Usability was further
assessed in terms of efficiency (time to accomplish the task), effectiveness (number of mistakes) and
motivation (subjective satisfaction). The results of the study confirmed the enhanced usability of an AR
environment for specific tasks dealing with questions of interpreting relief. None of the aforementioned
studies combined both VR and a collaborative environment.
Merchant et al. [54] distinguish three types of instruction based on virtual reality technology:
simulation, games, and virtual worlds. They conducted a meta-analysis of available empirical studies
using desktop-based virtual reality of all three mentioned types of educational approaches. They
found that games provided the highest learning outcome gain. They defined the important attributes
of educational games, also called serious games [55]. Such games should provide players with sense of
autonomy, identity, and interactivity [56] and enable them to test hypotheses, strategize their moves,
and solve problems [57].
Collaborative learning is a trend in modern pedagogy for improving the quality of educational
outcomes and processes [58–60]. It allows two or more users to interact and solve tasks together—with
a critical approach towards the overly ambiguous definitions often used— and may be defined as
a situation which Dillenbourg [61] (p. 7) described as “particular forms of interaction among people
are expected to occur, which would trigger learning mechanisms.” Dillenbourg himself noted that
the main concern of learning process designers was to find ways of raising the likelihood that certain
types of interaction would occur. What we expected when designing our collaborative immersive
virtual environment (CIVE) application was that students would use conversation to continually build,
monitor, and repair a joint problem solution, as depicted by Dillenbourg [61]. Collaborative learning
principles in college education of technical disciplines were introduced for example by Gokhale [62].
He evaluated the advantages of collaboration in a team of college students and confirmed a positive
feedback of collaboration for analysis and synthesis compering to the traditional individual training.
Another interesting aspect of collaboration within the VR is a distant cooperation of specialists from
ISPRS Int. J. Geo-Inf. 2019, 8, 3 3 of 25
different disciplines solving complex problems like geohazards (tsunamis, landslides, and floods) [63,
64]. Collaborative learning principles applied in college education for technical disciplines were
introduced, for example, by Gokhale [62]. He evaluated the advantages of collaboration in a team
of college students and confirmed the positive feedback of collaboration for analysis and synthesis
compared to traditional individual training. Another interesting aspect of collaboration in VR is the
remote cooperation of specialists from other disciplines engaged in solving complex problems such as
geohazards (tsunamis, landslides, and floods) [63,64].
Computer-supported collaborative learning was introduced in the early 1980s as an overarching
framework for various attempts to design a “technologically sophisticated collaborative learning
environment designed according to cognitive principles” that “could provide advanced support for
a distributed process of inquiry, facilitate advancement of a learning community’s knowledge as well
as transform participants’ epistemic states through a socially distributed process of inquiry” [65] (p.
4). Jackson and Fagan [66] conducted a qualitative study where learning processes were explored
by comparing individual users, two peer users, and student-expert modes. They used an immersive
virtual environment called Global Change World, which is used to educate about concepts concerning
global climate change. Other instances of collaborative learning using immersive virtual reality can be
found in, for example, the domain of martial arts [67], geometry education [68], and training power
system operators [69]. Innovative technologies for collaborative immersive virtual reality may be able
to create a shift in the educational paradigm. Siemens [70] has challenged the traditional learning
theories through his “connectivism” conception and emphasized that people in the digital age are no
longer isolated individuals but located in a network where they continuously interact with human
and nonhuman systems. Learning should be considered a lifelong net-building activity. Horvath [71]
presents a technological solution in the form of a learning environment enabling collaboration in 3D
virtual reality to teach the concept of the memristor.
The main advantage of using immersive virtual reality for educational purposes is overcoming
the boundaries of a specific place and time and having a virtual experimental space [72]. This offers
possibilities which are barely achievable or not possible to build in a classic classroom. Our geography
learning CIVE application offers a high level of interactivity for the user, which was achieved through
iterative testing and development. We also intentionally used gamification principles when creating
instructional tasks in order to facilitate the learning process. Our solution incorporates immersive
virtual reality, real-time social collaboration, and gamification principles. We chose hypsography as
an educational topic, as it is one of the most insufficiently understood areas by our university students
(according to the results of the Faculty of Science entrance exams: error rate was 86% in 2016 and 73%
in 2017). The objective of this study was to describe the cognitive and social tendencies of participants
during collaboration on geography learning tasks by applying the interpretative phenomenological
analysis methodology.
2. Methods
2.1. Materials and Technology
This study utilized a geography education CIVE application developed by our interdisciplinary
team. It makes use of the Unity cross-platform game engine version 2017.3, which facilitates data
loading, real-time rendering, and communication with VR equipment. The CIVE application was
built in a virtual environment described by Doležal, Chmelík & Liarokapis [73]. It is used in
combination with SteamVR for the proper functionality of VR equipment. Authentic geospatial
data were implemented as stimuli in the application. Digital terrain models (DTMs) were used as
the main input data. A fifth-generation digital terrain model (DTM 5G) created by airborne laser
scanning was acquired from the Czech Office for Surveying, Mapping and Cadastre. DTMs in the
application represent various parts of the Czech Republic with a similar relief. Data were transformed
by doubling the vertical values to accentuate the relatively small variation in landscape altitude. DTMs
ISPRS Int. J. Geo-Inf. 2019, 8, 3 4 of 25
were supplemented by contour lines also generated from the DTM data as well as orthophoto images
provided from a WMS (Web Map Service).
The application creates a shared virtual room for multiple users. Even though users are physically
located in separate objective reality rooms, the VR headset lets them share a virtual room to collaborate
on a given task. Physical movements in the objective reality room are tracked and transferred to the
virtual room, which means users can walk around the room and examine geospatial material from all
sides, angles and distances. In the virtual room, each participant is displayed as an avatar with virtual
representations of controllers he or she is holding in objective reality (Figure 1). Controllers are used to
manipulate the virtual environment and provide a laser pointer for communication. Users can also
talk to each other via standard audio recording and reproduction devices. Objects added to the scene,
such as houses and dams, are visualized abstractly and simply. It is considered a suitable method for
highlighting task relevant objects [74].
Figure 1. Objective reality (left corner) and virtual room.
The application includes two geospatial tasks. For each task, a different workplace in the room is
offered. The room has a table with a map for the first task and a large map on the floor for the second
task. Both geospatial tasks in the application require the user to examine contour lines on a 2D map to
determine the shape of the terrain in order to find the correct solution.
The default visualization in both tasks is a 2D map. If the user cannot solve the task correctly
on a 2D map, they can use various educational tools to help examine and manipulate the map.
The application provides a virtual control panel (Figure 2) next to the map in the CIVE. One of the
main advantages is the possibility to switch the map from 2D to 3D at any time. The map can also be
switched between a white contour map and an orthophoto contour map. Contour line equidistance
can be customized using a slider. Finally, when the user wants to verify their solution, they can use the
Evaluate button.
ISPRS Int. J. Geo-Inf. 2019, 8, 3 5 of 25
Figure 2. Virtual control panel for model manipulation.
2.2. Instructional Tasks in a CIVE Environment
For purposes of this research, two tasks were designed: Task 1—Mirror Signals and Task
2—Flooded Valley. In case of the Mirror Signals task, a map was presented to participants, with
two fixed flags marking the start point (flag A) and the end point (flag B). Next to the map were five
more available flags numbered 1, 2, 3, 4, and 5, which could be picked up and placed onto the map
(see Figure 3). The task was to connect start point A with end point B using these additional flags in
a way that mirror signals (or fire signals) could be transmitted between neighboring flags only with
direct visibility. This means that the view to flag 1 from flag A, flag 2 from flag 1, and so on had to be
unobstructed until an unobstructed view to flag B was obtained. The goal was to use the least number
of flags possible to link the start point with the end point (see Supplement for Video S1). In the first
task, the 3D model of the terrain can be dissected into individual layers and a cross-section of the
terrain can be viewed.
Figure 3. Incorrect (upper) and correct (lower) answers for the mirror signals task in the collaborative
immersive virtual environment (CIVE).
ISPRS Int. J. Geo-Inf. 2019, 8, 3 6 of 25
As in Task 2—Flooded Valley, a 2D map was presented to the participants that included houses
(orange rectangles) in a recognizable valley surrounded by mountain ranges and a dam (red line)
(Figure 4). Just as in the previous task, five flags numbered 1, 2, 3, 4, and 5 were next to the map and
could be picked up and placed onto the map. The scenario and task were as follows. A new dam has
been built to transform a valley with houses into a water reservoir. The water in the valley will gradually rise
and flood the houses one by one. Use flags with numbers to mark the order in which the houses will be flooded.
After submitting the solution, the participants could watch the rising water gradually flood the houses
(see Supplement for Video S2). The water level can be manipulated by user too, which lets the user
gradually flood the terrain to see water flooding one contour line after another.
Figure 4. Flooded valley educational task in the CIVE. Left—flags with numbers are placed onto the 2D
contour line map, middle—water level starts rising in the 3D visualization, and right—3D visualized
dam is completely flooded.
2.3. Research Approach
To examine the user experience in our geography learning CIVE application, an experiential
qualitative approach of Interpretative Phenomenological Analysis (IPA) was applied. This approach
explores the lived experience of a person and the meaning he or she attributes to it while exposed to
a specific phenomenon, for example, a short-term event or a long-term process. Its aim is to create
an in-depth description of a person’s lived experience during exposure to a particular phenomenon.
IPA is a frequently used strategy for research topics in weakly examined areas where the
background theory has not yet been sufficiently developed. It is flexible in dealing with unexpected
data that occur during research. It is therefore an ideal tool for gaining insight into and understanding
the innovative use of a CIVE for geography learning or learning in general [75]. A research question
in IPA is open, and although IPA is not a theory-driven approach, literature usually contributes to
formulating a research question [76], as was also the case in our study. IPA does not test hypotheses
and attempts to avoid creating preconditions before research. It is an inductive approach which is
rather “bottom-up” than “top-down” [77].
The number of participants in IPA research depends on the richness and saturation of individual
cases. Participants are experts on their own experiences and can offer the researcher an understanding
of their ideas, associations, and feelings. The recommended upper limit of participants is ten [78].
Creating a research sample is based on purposive sampling and participants are selected according to
relevance criteria for the research question.
Data collection in our IPA study implemented triangulation [79,80] from three research techniques
(Figure 5). Using three different and complementary research techniques for data collection makes
ISPRS Int. J. Geo-Inf. 2019, 8, 3 7 of 25
it possible to harvest the strongest aspects of all the techniques and mutually compensate their
weak spots.
Figure 5. Triangulation of research techniques for data collection.
Half of the participants involved in the study were interviewed in pairs in a semistructured
interview. The researcher sets up the key topics before interviewing, such as learning experience,
gained understanding of the learning topic, and means and effectiveness of communication with
a collaborator. The advantage of an individual or dyadic interview is a controlled, detailed, and deep
exploration of an individual’s unique experience. The other half of participants was interviewed in
a focus group. As a research technique, the focus group minimizes the influence of the researcher and
any preconceptions which could direct or distort the participants’ statements. The researcher moderates
a discussion and gives participants free space to share their individual experiences. However, some
important topics can be omitted by participants, which is the most significant disadvantage of the
focus group technique and the reason for our choice to use semistructured interviews to compensate
for this potential weakness. Nevertheless, the key advantage of both techniques mentioned is that they
bring new topics to light.
Subjectivity of the acquired data also poses a challenge. To overcome the potential risk of low
validity, we conducted observations. All participants were observed and video recorded during
their experience of the CIVE. We monitored voice communication, movement in objective reality
and the avatars in virtual reality as tasks were completed. This data provides researchers not only
with objective complementary information to the subjective reports, but also captures reactions and
behavior performed unconsciously by the participants.
2.4. Research Environment and Equipment
The study took place at Masaryk University in Brno, Czech Republic, in two separate rooms.
Each room was equipped with a computer (Intel® Core™ i5-6500 processor, Nvidia GeForce GTX 1080
graphics card, 16 GB RAM) connected to an HTC Vive headset (1080 × 1200 px resolution for each eye,
90 Hz refresh rate), sensors, and a controller. A participant and a researcher were present in each room.
The rooms offered enough space for participants to move around and were sound insulated from the
outside environment.
ISPRS Int. J. Geo-Inf. 2019, 8, 3 8 of 25
2.5. Participants
To design and structure the interview questions, one pair of participants was interviewed in the
preparation phase. It was an in-depth phenomenological interview with a pair of “experienced” VR
users conducted after collaboration in the CIVE application. Researchers themselves were involved
as preparation phase participants to gain personal experience with the CIVE and educational tasks.
The initial analysis resulted in a few changes to the research procedure, task setup, and virtual control
panel being made. The interview with preparation phase participants also focused on their overall
experience in the CIVE. Based on the information acquired, a semistructured interview schedule was
created for interviewing research participants.
Research participants were recruited from the pool of volunteer students and academic teachers
from the Faculty of Arts. Two exclusion criteria were applied. The first exclusion criterion was previous
formal training in cartography. The second exclusion criterion was the occurrence of cybersickness in
previous experiences with virtual reality or during this study. Participants were asked to report any
cybersickness and were briefed on options to end participation at any moment if required.
The final research sample consisted of 12 participants who collaborated on geospatial tasks in the
CIVE application in pairs. The pairs were established randomly. Seven participants were women and
five were men. The mean age of the participants was 27.58 years, the minimum age was 22, and the
maximum was 43. None of the participants had undergone specific GIS user training and none were
significantly experienced VR users (including, for example, VR gaming).
2.6. Procedure
Participants who volunteered to this study underwent a procedure consisting of five steps: 1.
Informed consent and collection of demographic data; 2. VR manipulation training; 3. Research
procedure instruction and contour lines principle explanation; 4. Collaboration in the CIVE (Figure 6);
and 5. Inquiry. With a pair of participants, the procedure varied from one to two hours.
Figure 6. Step 4: collaboration in the CIVE—research tasks administration in detail.
2.7. Analysis
We employed specific idiographic case study data analysis in the IPA and the variation for
multiple cases (respectively, multiple participants) as described in Smith et al. [78,81,82]. This analysis
focused mainly on the shared experience (common characteristics of experience) of participants, but
also mentioned significant and distinct experiences [77]. An analysis is slowly built-up by reading
individual cases and creating statements about the whole group of participants. The analytic process is
cyclic (iterative). The themes are reconsidered and rebuilt many times [78]. The results are transparent
because they are evidenced by data examples (quotations). The results are structured according to
theme. As shown in Figure 7, the analytic process cycle is as follows.
ISPRS Int. J. Geo-Inf. 2019, 8, 3 9 of 25
Figure 7. Scheme of the interpretative phenomenological analysis.
After transcribing the first interview, its content was read by a researcher repeatedly and significant
quotations were marked and annotated (comments included preliminary interpretations and ideas
from the researchers). This phase was repeated several times and the comments were then coded into
keywords from which important themes were identified. These themes were then structured into
a list of superordinate themes and subordinate themes belonging to each superordinate theme. Next,
the themes were rechecked for evidence in verbatim excerpts (participant quotations), after which
the analysis could proceed to another participant using the preliminary concepts gained from the
previous interview as a framework for analysis of the next interview (Figure 7). Each participant’s
interview analysis was thus thoroughly considered when the next interview was analyzed, and the
final list of themes applying to all participants (as described below) was based on in-depth analysis of
all interviews.
3. Results
Four superordinate themes emerged from the analysis. Under these superordinate themes, several
subthemes were identified, all of which are introduced in detail in the text below. The themes are well
illustrated by verbatim excerpts from the data corpus, which are included in the tables. The main
structure follows the most relevant topics: collaboration, learning, map literacy, communication,
and cognition.
3.1. Appreciation for Having a Collaborator
The first superordinate theme relates to the thoughts and feelings of the participants towards their
collaborative partners and is characterized by the appreciation of having a collaborator to solve the
tasks. The collaborator motivated them and provided the opportunity to consult on the solution.
This superordinate theme includes two subthemes which we called ‘Lost without a collaborator’
and ‘Verification and consensus with a collaborator’ and are also described below.
3.1.1. Lost without a Collaborator
A key aspect prevalent throughout the accounts of collaboration was that the participants would
have felt lost without a collaborator. They expressed doubt as to whether they would be able to solve
the task individually. Collaboration helped them solve the tasks. They were very happy they could
talk to their collaborator. Participants talked a lot, which made it easier for them to understand the
task. They believed that they would have been staring at the task for a long time if they had not been
ISPRS Int. J. Geo-Inf. 2019, 8, 3 10 of 25
working with a collaborator and would have felt uncertain and stagnated. Participants estimated that
a collaborative solution was more effective than solving the task individually and did not believe that
independent work on this task would have had any benefit (Table 1).
Table 1. Verbatim excerpts of statements by participants: Lost without a collaborator.
P13
“I wouldn’t have managed it myself, so, like, I don’t even have experience with it, so I was
really glad that there was someone with me who would say to me: ‘Yeah, this . . . ’”
P14
“I was quite pleased to hear the other person’s opinion. If I had done it like alone, it might
have been a bit sad ( . . . ) would’ve had to think about whether it was correct, whether I’d
made the right decision. But this way, when I had you and you helped me with it . . . ”
P09
“We were looking at it together and solved it together, because we talked a lot the whole
time, so . . . it seemed more, like, understandable to me.”
“It helped me a lot to have a collaborator, because I would have stared at it for a very long
time and would have been very uncertain, because I was wandering a little in it, so it
helped me a lot when the collaborator said: ‘Let’s just solve it in one minute and go to the
next one’ . . . and that was just great, because we just needed to try it and we tried it.”
P12
“This way, it was even more fun and faster.” “This interaction is always as if time always
flies faster. If I had been there alone, I would have looked around more.”
“If I had been there alone, I would’ve looked half an hour somewhere else where
something else was, but because there were two of us, I focused more on the task, because
you always pulled me back to it.”
3.1.2. Verification and Consensus with the Collaborator
Participants described that as they made decisions about solutions to the task, they usually
consulted their collaborator to verify the answer before submitting it. They sought consensus on the
right solution together. Participants discussed their viewpoints and the specifics of a particular task
which could influence the answer. They talked to the collaborator about which strategy or key they
should use to solve the task. The usual modus operandi among the pairs of participants was to talk
about a strategy for the solution, reach an agreement on the correct answer and then submit it. They
were therefore much more confident when submitting the answer and felt better about it. It also helped
them to inspire each other. When anyone in the dyad discovered a useful strategy, they shared it and
both then used it. Solving the task with a collaborator was reported as more effective. Participants
usually discussed and decided on a solution together (Table 2).
Table 2. Verbatim excerpts of statements by participants: Verification and consensus with
the collaborator.
P04
“I consider collaboration as good, because it verifies that I thought about something. I asked the
collaborator whether he saw it the same way, and he said that he did, and in that case we
submitted it.”
P03
“We just agreed on how to do it.” “It seemed to me as less ambiguous and so we found some . . .
consensus about the right solution, and whether we saw something special about that case.”
P09
“We discussed the task, searched the key which we would establish as a solving strategy, and
using laser pointers, showed each other the things we were currently talking about.”
P11
“We always talked nicely about it, to both agree on it before we completely submitted it.”
“If the collaborator was thinking the same thing I was, then it was easier to submit the answer,
but individually I would maybe have thought about it more or . . . but I think it also helped me
to understand it, because both of us found something relevant and then we both used it further,
so we were enriching each other.”
P12
“I think that thanks to the fact that we evaluated it little by little, the process was more efficient
than it would have been if I had been there alone, then I would either think about it more or
have made more mistakes.”
ISPRS Int. J. Geo-Inf. 2019, 8, 3 11 of 25
3.2. The Surprising “Fun with Maps”
The second superordinate theme relates to the reported level of excitement the participants felt
while working on geospatial tasks in the CIVE application, although most participants verbalized that
they usually did not enjoy working with maps and considered them boring. Moreover, working in
CIVE also enhanced the educational effect.
This superordinate theme includes the two subthemes. Finally, seeing what contour lines represent
in reality and learning a skill to work with maps.
3.2.1. Finally Seeing What Contour Lines Represent in Reality
Many participants explained that thanks to being able to switch the 2D map to the 3D model of
the terrain, which they could examine and walk around freely, an association developed in their minds
between what a contour line looked like on paper and what it represented. Virtual reality helped them
solve the task and see the correct answer more clearly. They found it helpful to use educational tools
for visualization in 3D, for example, raising the terrain’s water level to see how contour lines were
flooded one by one. All of this helped them learn about contour lines and create associations between
2D maps and real 3D terrain (Table 3).
Table 3. Verbatim excerpts of statements by participants: Finally seeing what contour lines represent
in reality.
P03
“On one hand it was great, because now you know where the contours are and what they
look like and stuff, and then the virtual reality is turned on and then it (the 3D model)
emerged, which I think is great.”
P09
“It really was helpful and I could see it in that more clearly . . . It was like it allowed me to
concentrate really well on the task.”
P12
“And I think it really helped us to gain insight, so in the next task our imagination was
better, that we could imagine it better, and that what we created is some model of how
those contours look like in reality.”
P11
“Yes, I think it helped even in understanding what contours actually are, when the person
then really sees it as the real differences in height, that it’s not just lines.”
P14
“In fact, normally a person can’t see it like this, as in that virtual reality.”
“With that flooding, certainly, like flooding, imagining that water, as it rises. When we
evaluated it, it was then easier to shift it and see how the water would rise than if I had to
imagine it on the table.”
“In fact, this can be utilized in many ways, even, like, only in education when a person
imagines how it looks in reality. So this was great.”
3.2.2. Learning a Skill to Work with Maps
An additional and clearly identifiable subtheme which emerged from the analysis relates to the
new skill participants learned for working with maps. Participants explained that they had acquired
a better understanding of contour lines and that if they came across similar tasks in the future, they
would be able to solve it faster. Our educational application was seen as a good learning and training
tool for improving map orientation skill and decreasing the time it occupied. For some participants,
maps were an alien territory, but with our CIVE application, their map orientation skills improved.
Their map reading speed increased and it was now easier for them to imagine terrain (Table 4).
ISPRS Int. J. Geo-Inf. 2019, 8, 3 12 of 25
Table 4. Verbatim excerpts of statements by participants: Learning a skill to work with maps.
P04
(if they had to solve a similar task in the future) “Well, certainly it would be better. On that
level, we wouldn’t be looking at it for five minutes and . . . (participant laughs) . . . and
wouldn’t be trying to understand contours.”
P11
(if they had to solve a similar task in the future) “Like, I would probably be pretty much
orientated in it . . . such good training.”
P03
“We learned to read contours much better, perceive them and search for them even in that
not completely biblical environment, because neither of us is a geographer and we weren’t
used to using contours every day, and that ability to orientate gradually improved.”
P04
“We are certainly a little, or at least I am, someone who doesn’t work very often with
contours. I have oriented myself in that environment, so now I’m on . . . now it would take
me less time to recognize, where the hills and valleys are.”
“I actually didn’t see that terrain at all, so I think that actually for a long time both of us only
looked at it.” “Later we were faster reading the map.”
P13 “Much better, I can imagine it better as, like . . . now I can imagine it better.”
3.3. Communication as a Challenge
The third superordinate theme relates to the effort participants made to communicate their
thoughts and feelings to the collaborator. Participants described that they had to concentrate more on
facilitating communication by the means they had available in the virtual environment.
It includes three subthemes: Absence of avatar faces and invisibility of emotions, limited gestures
via controllers, and having an intangible body.
3.3.1. Absence of Avatar Faces and Invisibility of Emotions
Many participants felt they had limited options when it came to communicating emotions to
their collaborator. One of the first things they noticed was that avatars had no faces. Some of the
participants looked at their collaborator’s avatar when they talked to them, and some did not. Most of
the participants considered faces as important and missed them in the virtual reality environment for
conveying emotion. Besides, participants considered it important to see where their collaborator was
looking. This was possible in our CIVE application, but they were not surprised at being unable to
look their collaborator in the eyes, as they had expected it this way. Some of the participants wondered
whether it would be strange or disturbing if their collaborator’s avatar had some representation of
a face, as there could be discrepancies between what the person was trying to communicate and the
emotions the artificial face managed to convey (Table 5).
Table 5. Verbatim excerpts of statements by participants: Absence of avatar faces and invisibility
of emotions.
P10
(collaborator having no face) “It seemed terribly comic to me, like, when we talked to each
other—that alone made me laugh for a long time.”
P12
“Yes, I had to adjust my ways a little in communicating, for example, gestures and . . .
Basically, we didn’t even see each other’s facial expressions, and facial expressions are also
quite important, so I didn’t see what facial expressions she had . . . when I heard her
laughing, I heard that, but when you can’t see the other person’s eyes, it’s that different.”
P04
“Sometimes I missed those, like, emotions there, that I was, like, smiling (participant laughs)
. . . and there was no way of passing it over and then I always realized that I’m smiling in an
empty room.”
P03
“I considered it important to be able to see where that person was looking or what he was
calculating or something.”
P06
(collaborator having no face) “I probably expected it. I’ve seen it before.” “Natural, it
probably wasn’t . . . (participant laughs) . . . but it was probably not surprising.”
ISPRS Int. J. Geo-Inf. 2019, 8, 3 13 of 25
Table 5. Cont.
P10
“I wonder if it would have been stranger if it had simulated a person even more and then
there was some discrepancy, even like, that it would have had a more realistic face, then it
might have been even stranger.”
P08
“There should rather be a square than a simulation of the shape of a head, or that, as you
said, being deformed, that is probably not what a person really wants, for it to resemble
a person.”
P10
“Maybe it also then even evokes some emotions, and that is probably not what is intended,
for people to act according to it.”
3.3.2. Limited Gestures via Controllers
When participants were asked to describe their experience of communicating with their
collaborator, they often described the need to modify their communications and actions because
they had avatars instead of real bodies. Participants mentioned that the collaborator’s representation
was fine, but if the avatar had been more detailed, it would have been even better, as they wanted to see
their collaborator’s gestures. They would then look at their collaborator during communication more
often. However participants say it was possible to read information from the posture and proximity of
the collaborator’s avatar. They had to think about how to depict something during communication
when the collaborator could not see them fully. It was apparently demanding, but also fun. It required
an unusual style of thinking which required the participants to consider the selection of gestures. They
managed, however, to adapt to the visible parts of their avatar and used only those to communicate
(Table 6).
Table 6. Verbatim excerpts of statements by participants: Limited gestures via controllers.
P04
“To me it seemed okay, but certainly if that avatar functioned better and I could even look at
gestures, then it would be better. That probably . . . like I would use more, I would try to
communicate more and I would even look at him more.”
P12
“That is true, I actually also gestured with my hands and . . . it wasn’t actually being seen . . .
so I had to, like, with that controller.”
P04
“Sometimes I looked where you were standing, but I couldn’t make out a lot from that, and
yeah, I saw for example, that he was currently leaned over the numbers, yeah, and . . .
actually yes, when I think about it, you looked, for example, I saw, that he was currently
leaned over the numbers, so I assumed he was currently solving something.”
P12
“One tries out a little different way of communicating. Maybe he concentrates a little more
on what he’s doing with the hands, legs, what and how he moves and how to communicate
something to the other person, so you have to take into consideration what he sees.”
P11
“Well yeah, well, since it was a little bit more limited with those avatars, as with that, that
there weren’t all the details, then it was a little bit more interesting, so . . . that sometimes
there’s too much detail.” “I actually had to think about it, how to show something
considering that I will not be seen as a whole, and, like, when I show something with my
hands, that the hands actually were not seen.”
“Yeah, I’ve been able to adjust to what is actually visible and somehow just move only with
those things that are visible.”
P12 “So I think a person gets used to it quickly . . . he learns how to work with what he sees.”
3.3.3. Having an Intangible Body
All the participants dealt with the fact that their body in VR was not composed of any physical
material. The situation when the avatars of both participants stood in the same virtual space or when
an avatar stood “inside” a virtual object occurred. The physical area around a participant was always
free, and the decision not to walk through virtual objects was always up to the participant. Participants
ISPRS Int. J. Geo-Inf. 2019, 8, 3 14 of 25
tried to keep a usual personal distance between themselves and the collaborator, even though it was
only an avatar.
Many participants mentioned the problem of obstructing or shadowing each other’s view. When
a collaborator stood in the map, it was quite a big problem and hard for the other person to read
contour lines, but they did not realize they were doing it. Participants usually did not tell each other.
They recognized the problem, but it usually only lasted a few seconds before the collaborator changed
position and they could see the map again.
However, participants mostly did not mind that their avatar was not physical. It only bothered
them at the beginning on account of habit (Table 7).
Table 7. Verbatim excerpts of statements by participants: Having an intangible body.
P04
“When my collaborator appeared in that first room during the first task, and when you, like,
moved to the flags and I stepped out of your way, right, to like make space for you, so in that
moment I was fully aware of the fact that I actually didn’t have to move and that we could
be both there, one through another, but, but I made a step back (participant laughs), because
it seemed a little bit awkward.”
P03
“I guess I would make him space, if I could. I would probably respect the personal zone
even in cyberspace.”
P11
“When I was looking at something and I was, like, leaned over it, like that I was thinking
about something, and since she didn’t see me as a whole, so I thought I wouldn’t be
obstructing and then I found out that I’m obstructing there with my whole body, and that
she didn’t see the map at all.”
P12
“Well, there was such, such a funny thing, when you were standing there somehow through
the table and it wasn’t possible to see through it at all and it was so weird.”
P04
“it was quite a big problem, but it didn’t occur to me at all that I could be shadowing you,
but because we both got in there, that we both walked inside that map since it was difficult
to read, but then I didn’t see those contour numbers or the contour heights actually in that
moment, when the person was standing exactly through the map, that’s true.”
“And we never said to each other: ‘hey please make a step back’.”
“I was dealing with it as with a problem, certainly, but probably in the same moment you
walked further on, so it didn’t bother me anymore.”
“I think it was really a matter of a few seconds.” “The fact that the body actually took up
quite a lot of space was not okay.”
“I found it good that we could place the flags through each other in there, that you could
place them here and I could place it there and you here and that we could cross over each
other like this.”
3.4. Cognition in Two Realities
The fourth and final superordinate theme relates to the cognitive aspects of simultaneously
existing in two realities: objective and virtual reality. Participants were present in objective reality but
also felt the sense of presence in virtual reality.
This superordinate theme includes three subthemes: Where are my legs? Immersion and
involvement in the artificial world and confusion during the return to objective reality.
3.4.1. Where Are My Legs?
This question was asked by one of the participants, while other participants also wondered why
their avatar looked so rudimentary. Many participants could not adapt to not seeing their own legs.
They were strongly conscious of their absence, some even intrigued by it, as they were accustomed to
seeing their legs as they looked down. Most participants would have been happier to have virtual legs
in the virtual environment. By contrast, one participant did not mind that she had no legs, but did not
like that the collaborator was missing legs (Table 8).
ISPRS Int. J. Geo-Inf. 2019, 8, 3 15 of 25
Table 8. Verbatim excerpts of statements by participants: Where are my legs?
P03
“When I put the headset on, I couldn’t get used to, like, that I don’t have any legs, I don’t
have a wristwatch. I was aware of these two things very strongly from self-perception.”
“I found it interesting not to actually see the legs. Because always when I put my glasses on
and look, when a person looks down and he is walking somewhere, then he can see his legs
and now I didn’t see them, so that was interesting to me.”
P08
“It wasn’t very pleasant when I looked down, then I felt that I just have them (legs), but
they’re simply not there.”
P11
“It was quite odd that I was actually in the table, or, like (participant laughs) . . . moving, not
actually seeing my own legs, knowing that the legs are probably right where the table was
(participant laughs) . . . kind of a strange feeling.”
P09
“Maybe just because I didn’t see the legs in there, then I didn’t mind going through the
table.”
P11
“I was quite glad that if the legs had been displayed in there and I saw them go through the
table, then it might have been even stranger.”
3.4.2. Immersion and Involvement in the Artificial World
The experience of being in immersive VR was characterized by the loss of tracking objective
reality and having a stronger sense of presence in the virtual environment. Immersed in VR and
wholly engaged in the task, participants felt a stronger sense of presence in virtual reality. They did
not perceive or think about what may have been happening around them in objective reality. While in
VR, they had no need to be in touch with the outside world. Only when they bumped into something
or heard the experimenter speak did they think about where someone or something was and feel
disoriented (Table 9).
Table 9. Verbatim excerpts of statements by participants: Immersion and involvement in the
artificial world.
P09
“On one hand, I was really, like, immersed in that task and in that activity, and on the other
hand . . . so it was really, like, absorbing for me.”
P13
“Well, I just bumped into something there, but otherwise I had no clue who was doing what in
here.” (in the objective reality room)
P11
“I was actually more in that virtual reality than in the real reality, actually. Sometimes I really
didn’t perceive the real reality, I put myself into it a lot.”
P14 “I actually didn’t have a clue what was actually happening around me.”
P04
“I have to say that I didn’t quite perceive that much.” “Only when I took my headset off did I
find out that I was terribly sweaty under it.” “Then I, like, realized more that it was actually
warm in there.”
P12
“At first I was thinking about what it looked like from the outside, the things we were doing
there, that it must have looked really funny.”
“At the back of my mind I knew that I was, like, in the real reality and that I’m doing those
things others can’t see, but I was also, like, quite able to put myself into it, that I’m simply in
some room without a ceiling and where there’s just some map on the floor.”
P03
“When I was solving it, the task, I perceived more the virtual reality, but when there was
nothing going on at time, then I perceived more the physical world again.”
3.4.3. Confusion during the Return to Objective Reality
All of the participants liked the virtual environment and became accustomed to it, and most did
not want to leave it. Although most of the participants described that they did not have any problems
after taking their headsets off, some described specific feelings and perceptions which they experienced
for a short time after they had returned from VR to objective reality.
ISPRS Int. J. Geo-Inf. 2019, 8, 3 16 of 25
For instance, one participant described how shocked he felt seeing his real hands again after
leaving VR. A moment after leaving VR he felt lightweight and thought he would faint and felt strange
even after some time. The time after exiting VR was more disorienting to him than the time spent
in VR.
As mentioned above, though, most participants described no awkward feelings after taking their
headsets off. They did not need to adapt to objective reality; it was completely normal for them to
return to the objective reality room (Table 10).
Table 10. Verbatim excerpts of statements by participants: Confusion during the return to
objective reality.
P08
“For me it was a shock to see my hands again after I took the headset off, and I had a clear
feeling that when I was standing in that other room, I could simply walk through the person in
there, that my hand could just pass through that person.”
“It’s still strange. For a moment after sitting down, I felt as if I’d pass out.”
“So that one feels lightweight and just feels as if the wall isn’t there, that I could walk through
it. For me, it was probably a much more shocking experience after than with the headset on.
And I wondered how I would feel if I layed down now, because I actually didn’t even see my
hands, I perceived it as those hands when I looked, like: “Wow” and I would probably, I
would certainly not want to willingly go out of the room and go, for example, out onto the
street, because I would be afraid that I, like, can’t control it and that something could happen.”
P11
“Yeah, I liked it there, that I didn’t want to come back, but then when I took the headset off,
then it was actually . . . quite strange, that it was, like, drawn, the real things. So I had to
acclimatize a little bit.”
“It seemed to me that things were a little bit smaller, or as the details displayed there, like in
reality, those details are displayed normally, so it was, like, more detailed.”
“Then I had problems with reality, that it’s, like, too detailed and that I, like, can’t perceive it.
Because in the virtual reality I could perceive everything, because it was simpler, so there were,
like, simple stimuli, but in the genuine reality a person has to distinguish what he actually
perceives, because there’s a lot of it, so that he can no longer see it as an overall picture with all
the things that are actually there. It’s, like, more understandable in there.”
“Yeah, yeah, a little bit yes, just only on those details, that I had to perceive reality again, as to
distinguish what I would actually look at, as I already said. Well, but it was just for a moment
. . . in about three minutes I was okay again. But it wasn’t even unpleasant, so it’s stupid to say
‘okay’, but simply, that I wasn’t even perceiving it anymore after I adapted.”
P12
“I didn’t want to go back, I liked it there very much. It was, like, when I took the headset off, it
was like, like at first unpleasant, until the eyes got used to it, that I was used to that virtual
reality and to that light and to how it looked there, and then I took the headset off and it was,
like, “Ouuu”, a little bit unpleasant.”
4. Discussion
In this section, we discuss the results of our study in the context of referenced literature and
challenge it with our preliminary expectations and recommend further research and applicational
options. The results are already interpretative and deeply descriptive, therefore the discussion to each
subtheme will be concise.
One of the main findings of this study was that participants would have felt lost without
a collaborator and that working in a dyad brought more entertainment and better results. From a social
psychological perspective on collaborative learning [83], collaborators can be explained as providing
social and emotional support to each other, enjoying mutual interaction, and having a positive effect
on satisfaction and results. Participants in our study felt motivated by their collaborator. A social
psychological perspective considers motivation as a precursor to effective cognitive processes during
collaborative learning. Motivation in collaborative learning can be viewed from two points of view.
From a socio-motivational point of view, collaborators are motivated to work together because they
ISPRS Int. J. Geo-Inf. 2019, 8, 3 17 of 25
share the rewards for completing the task. From a social cohesion point of view, cohesiveness arises
between collaborators and draws them into looking after each other and cooperating and working
together. Slavin [84] and Johnson and Johnson [85] discovered that students are more motivated during
collaborative learning than individual learning.
Another important finding is that collaborators debated a lot during the problem-solving process
and sought verification and consensus with their collaborator. From a cognitive perspective on
collaborative learning [86], collaboration with a peer can be explained as achieving better quality
in basic information processing components such as coding, rehearsal and retrieval of information,
activation of strategies and metacognition. Participants in our study claimed it would have taken
them longer to solve the tasks individually. O’Donnell and Dansereau [87] explain that the presence
of collaborator helps the student stay focused on a task and gives them an opportunity to verify
understanding of the subject matter.
According to Webb and Farivar [88], if a student explains the task to the collaborator, it allows the
student to identify flaws in their own reasoning. Collaborative learning and negotiation of meaning
between people can support greater coherence in understanding subject matter [89].
Participants also explained that finally seeing what contour lines represent in reality helped
them gain insight. From the perspective of Piaget’s theory of cognitive development, specifically
of the concept of mental schemas [90], the educational tools implemented in our CIVE application,
which enabled participants to switch between a 2D map and 3D model or to raise water level in the
terrain, served as a means to confront the participant’s understanding of the subject matter. In Piaget’s
terms, participants underwent the process of accommodation, during which their preexisting schemas
were adjusted according to the new experience. The importance of experience was emphasized
both by Piaget’s predecessors as Dewey [91], and his followers, who further elaborated his work:
Kolb [92] understands learning as a circular process of creating knowledge via transformation of
experience. Participants in our study first tried to complete the task on a 2D map. Their assumptions
and understanding were then challenged by the 3D model which visualized their solution. In the case
of an incorrect solution, cognitive conflict or disequilibrium occurs as a result, which drives the student
to reduce this state and to renew equilibrium. The collaborator serves as another potential source of
cognitive conflict. This is in accordance with the general educational approach proposed by Neale,
Smith, and Johnson [93], to first give students an opportunity to create assumptions about the subject
matter and then let them test it against evidence to discover contradictions. This strategy aims to make
students aware of their predictions and present contradictory evidence to create cognitive conflict.
The participants expressed that they had a better understanding of contour lines after the
experiment. Several aspects could contribute to learning a skill to work with maps. One of them is
from the perspective of Vygotsky’s theory of cognitive development [94]. According to his concept of
the zone of proximal development, if a student receives appropriate support during interaction with
another person during the task solving process, they can internalize the process, reorganize cognitive
structures, and develop new competence. This concept resembles the concept of scaffolding, which,
according to Hogan and Pressley [95], is a support enabling a student to solve new tasks, teaches
competence and fades over time. Modern usage of this term often incorporates not only interpersonal
support but also software based educational tools. Our CIVE application provided scaffolding for
learning through problem solving, which according to Guzdial et al. [96] helps students acquire deep
understanding of subject matter and new competence. Our application was also a case of scientific
discovery learning, which Chen and Zhang [97] consider as a learning process during which students
generate and test their hypothesis. In their study, they found a prominent effect of collaborative
scientific discovery learning in VR on intuitive understanding and discovery outcomes. The results of
the study by Okada & Simon [98] show that collaborative discovery learning in pairs is more effective
compared to individual discovery learning.
Participants had problems with the absence of avatar faces and invisibility of emotions.
According to Ekman and Friesen [99], people gather information about another person from four main
ISPRS Int. J. Geo-Inf. 2019, 8, 3 18 of 25
sources in the visual informational channel: the face, tilts of the head, body posture, and skeletal
muscle movements. They described that during conversation people do not continuously look at
a listener but look to determine the listener’s emotions or find out whether they are paying attention,
agreeing, or attempting to respond with their own speaking. The participants of our study did not
have a virtual face and could not make these distinctions. Some of them therefore did not even look at
their collaborator’s face while they were speaking. Most participants, however, missed having a face
as a channel of information and did not know how to substitute its role.
Participants described their experiences with limited gestures via controllers and how they had
to learn to work with it. Tu [100] explained that virtual communication differs from communication in
objective reality. According to him, because of the limited communication channels, participants miss
the clues for social context, and communication may be impersonal or cold. Virtual communication
therefore requires different communication styles and strategies to maintain personal and social
communication. In our study, we observed that participants sought personal contact with their
collaborator, and even though the communication channels were limited, found innovative ways of
using controllers and avatars for communication.
Participants described that having an intangible body created situations of obstructing
each other’s view but did not influence the proximity and personal space rules they followed.
Bailenson et al. [101,102] discovered in several studies that participants seek to maintain the same
interpersonal distance in immersive virtual reality as in objective reality. This is in accordance with
our observations and what participants expressed in the focus group and interviews. They used
their avatars as nonverbal communication tools and kept the same proximity to the collaborator as
they would in objective reality. However, because they did not have full control over the avatar’s
movements and position, obstruction of each other’s view sometimes occurred.
Some participants described strange sensations related to the cognitive discrepancy between their
tactile sensations of objective reality and their visual perception of virtual reality. Some of them asked
themselves Where are my legs? It is important, though, that this was not a case of cybersickness, which,
according to LaViola [103] and Davis, Nesbitt, and Nalivaiko [104], is a type of motion sickness caused
by cognitive discrepancy between the tactile sensations of a static position and the visual perception of
movement. It seems, however, to be based on the same principle of cognitive discrepancy.
A common experience shared by participants was immersion and involvement in the artificial
world. Witmer and Singer [105] describe immersion and involvement as preconditons for a sense
of presence. Immersion as a psychological state can be characterized as perceiving the particular
environment which surrounds us and perceiving self as a part of that environment. In the context
of virtual reality, it means ignoring the medium and being absorbed by the simulation [106].
The participants of our study described losing track of objective reality and not knowing what was
happening around them in objective reality. Involvement occurs as a result of being engaged in
a meaningful task and focusing attention on specific content. Csikszentmihalyi’s [107] well-known
psychological concept of flow describes a similar state characterized by being fully involved and
absorbed in a task, feeling energized focus and enjoyment, and losing a sense of time and space.
The participants of our study described the task as capturing their whole attention and eliminating the
perception of external stimuli. According to Witmer and Singer [105], participants feel a stronger sense
of presence in virtual reality than objective reality as result of both immersion and involvement, which
is precisely what out participants described.
Several participants in our study described their confusion during the return to objective
reality. Two of the participants described states of derealization, which is defined by DSM-5 [83]
as the detachment from a person’s surroundings (world, people, or objects) and experience of the
surroundings as unreal, dreamlike, or visually distorted. Research conducted by Aardema et al. [108]
demonstrated that exposure to immersive VR induces a dissociative experience and temporarily
increases the symptoms of depersonalization and derealization from objective reality.
ISPRS Int. J. Geo-Inf. 2019, 8, 3 19 of 25
5. Conclusions
In this study, we explored the experience of geography learning in a CIVE. The experiment
centered on collaborative learning, development of geography competences and cognitive and
social aspects. The objective was to broaden knowledge and understanding of these areas in
the specific context of a CIVE. Using a uniquely-developed geography learning CIVE application,
twelve participants experienced an educational intervention during which they collaborated in pairs
on geospatial tasks. By means of observation, semistructured interviews, a focus group, and an
interpretative phenomenological analysis, we gained deep insight into the participants’ experiences.
From these data, four superordinate themes emerged, each including the above depicted subthemes.
From these superordinate themes, we concluded the following:
1. Appreciation for having a collaborator. Collaborative educational interventions have previously
been shown as more efficient than individual task solving [85–87] (among others). Whether this
applies to a VR environment is yet to be empirically tested at a quantitative level, but based on
our study’s results, we may conclude that collaborative VR education has great potential both in
terms of improving learning outcomes and decreasing task related anxiety.
2. The Surprising “Fun with Maps”. Motivational potential is believed to be one of the greatest
expected advantages of VR educational interventions. As far as we can estimate from the
qualitative analyses, when the topic of the educational session is well chosen, VR offers ways of
exciting learners and making them interested in a topic they would find (or expect to be) boring.
However, such a qualified choice needs to be based on the necessary knowledge of the lesson’s
subject (geography in our case), educational principles and VR technology specifics.
3. Communication as a challenge. As some participants reported that communication with their
partner was challenging when no facial expressions were transmittable and because gestures
were not precisely transferred into VR, the means of communication in a CIVE appear to be one
of the key topics for future research. However, since we observed that many of the participants
managed to innovate ways of communicating within a relatively short time (approx. 60–120 min),
we believe that in a long-term educational intervention (for example, a regular semester course)
learners would likely adapt and communication would no longer feel challenging. This is also
yet to be confirmed experimentally.
4. Cognition in two realities. Since some of the participants reported negative or confused feelings
after the VR session during their return to objective reality, some future research challenges have
emerged. The first will be to eliminate the negative impact of VR immersion in some participants.
Predictors of the depersonalization and derealization states need to be identified in order to
provide special care to those learners at risk (or to exclude them from the intervention before they
are allowed to begin). The second and a worthwhile consideration will be to search for ways of
adapting the VR environment or sessions to decrease the risk of such states. However, most of
the participants showed no indications of negative feelings, and hence, the overall results of our
study are more than motivating for further elaboration of the CIVE intervention design.
This IPA-based study identified key areas that may play a key role in using collaborative iVR
technology and suggested its potential benefits and limits in the field of education. In future studies,
quantitative confirmation of the findings will be extensive and include effectiveness comparisons with
traditional tools such as GIS (Parong & Mayer) [109]. Broadening the list of learning tasks is also
yet to be done. Challenges for further research will include the impact of intervening variables such
as the level of user experience with iVR, educational intervention length (repeated measurements),
and interindividual differences (e.g., cognitive style or map literacy).
Supplementary Materials: The following are available online at http://hci.fi.muni.cz/CIVE-papers/Task_
1_Mirror_signals.mp4 (Video S1—task 1) http://hci.fi.muni.cz/CIVE-papers/Task_2_Flooded_valley.mp4
(Video S2—task 2).
ISPRS Int. J. Geo-Inf. 2019, 8, 3 20 of 25
Author Contributions: Only the substantial activities are listed for each author. Conceptualization, ˇC.Š., Z.S.,
J.C., V.J. and P.K.; Data Curation, H.T.; Formal Analysis, M.S.; Funding Acquisition, H.S.; Investigation, H.T.;
Methodology, ˇC.Š., Z.S., M.S. and T.U.; Project Administration, P.K.; Resources, L.H.; Software, J.C. and M.D.;
Supervision, ˇC.Š., Z.S. and J.C.; Visualization, P.U.; Writing—Original Draft, M.S.; Writing—Review & Editing,
A.S.
Funding: This research was funded by Masaryk University, Czech Republic, Grant No. MUNI/M/0846/2015,
“Influence of cartographic visualization methods on the success of solving practical and educational spatial tasks”.
Acknowledgments: The authors would like to thank all students and colleagues who participated in this study.
This work was supported by the research infrastructure of the HUME Lab Experimental Humanities Laboratory,
Faculty of Arts, Masaryk University.
Conflicts of Interest: The authors declare no conflict of interest.
References
1. Romano, D.M. Virtual Reality Therapy. Dev. Med. Child Neurol. 2005, 47, 580. [CrossRef] [PubMed]
2. Botella, C.; García-Palacios, A.; Villa, H.; Baños, R.; Quero, S.; Alcañiz, M.; Riva, G. Virtual Reality Exposure
in the Treatment of Panic Disorder and Agoraphobia: A controlled Study. Clin. Psychol. Psychother. 2007, 14,
164–175. [CrossRef]
3. Garcia-Palacios, A.; Hoffman, H.; Kwong See, S.; Tsai, A.; Botella, C. Redefining Therapeutic Success with
Virtual Reality Exposure Therapy. CyberPsychol. Behav. 2001, 4, 341–348. [CrossRef] [PubMed]
4. Bertella, L.; Marchi, S.; Riva, G. Virtual Environment for Topographical Orientation (VETO): Clinical Rationale
and Technical Characteristics. Presence Teleoperators Virtual Environ. 2001, 10, 440–449. [CrossRef]
5. Cho, B.H.; Ku, J.; Jang, D.P.; Kim, S.; Lee, Y.H.; Kim, I.Y.; Lee, J.H.; Kim, S.I. The Effect of Virtual Reality
Cognitive Training for Attention Enhancement. Cyberpsychol. Behav. 2002, 5, 129–137. [CrossRef] [PubMed]
6. Anderson-Hanley, C.; Arciero, P.; Brickman, A.; Nimon, J.; Okuma, N.; Westen, S.; Merz, M.; Pence, B.;
Woods, J.; Kramer, A.; et al. Exergaming and Older Adult Cognition. Am. J. Prevent. Med. 2012, 42, 109–119.
[CrossRef]
7. Moller, H.J.; Bal, H.; Sudan, K.; Potwarka, L.R. Recreating Leisure: How Immersive Environments can
Promote Wellbeing. In Interacting with Presence: HCI and the Sense of Presence in Computer-Mediated
Environments; Riva, G., Waterworth, J., Murray, D., Eds.; De Gruyter Open Ltd.: Warsaw/Berlin, Germany,
2014; pp. 102–122.
8. Waterworth, J.; Waterworth, E.L. Relaxation Island: A Virtual Tropical Paradise. In Proceedings of the British
Computer Society HCI Conference 2004: Designing for Life, Leeds, UK, 6–10 September 2004.
9. Keefe, F.; Huling, D.; Coggins, M.; Keefe, D.; Rosenthal, Z.; Herr, N.; Hoffman, H. Virtual Reality for
Persistent Pain: A New Direction for Behavioral Pain Management. Pain 2012, 153, 2163–2166. [CrossRef]
10. Mirelman, A.; Maidan, I.; Herman, T.; Deutsch, J.; Giladi, N.; Hausdorff, J. Virtual Reality for Gait Training:
Can It Induce Motor Learning to Enhance Complex Walking and Reduce Fall Risk in Patients With
Parkinson’s Disease? J. Gerontol. Ser. A Biol. Sci. Med. Sci. 2010, 66, 234–240. [CrossRef]
11. Huber, T.; Paschold, M.; Hansen, C.; Wunderling, T.; Lang, H.; Kneist, W. New Dimensions in Surgical
Training: Immersive Virtual Reality Laparoscopic Simulation Exhilarates Surgical Staff. Surg. Endosc. 2017,
31, 4472–4477. [CrossRef]
12. Kilmon, C.A.; Brown, L.; Ghosh, S.; Mikitiuk, A. Immersive Virtual Reality Simulations in Nursing Education.
Nurs. Educ. Perspect. 2010, 31, 314–317.
13. Ordaz, N.; Romero, D.; Gorecky, D.; Siller, H. Serious Games and Virtual Simulator for Automotive
Manufacturing Education & Training. Procedia Comput. Sci. 2015, 75, 267–274. [CrossRef]
14. Matsas, E.; Vosniakos, G. Design of a Virtual Reality Training System for Human–robot Collaboration in
Manufacturing Tasks. Int. J. Interact. Design Manuf. 2015, 11, 139–153. [CrossRef]
15. El-Mounayri, H.; Rogers, C.; Fernandez, E.; Satterwhite, J. Assessment of STEM e-Learning in an Immersive
Virtual Reality (VR) Environment. In Proceedings of the 2016 ASEE Annual Conference & Exposition
Proceedings, ASEE Conferences, New Orleans, LA, USA, 26–29 June 2016. [CrossRef]
16. Guttentag, D. Virtual Reality: Applications and implications for tourism. Tour. Manag. 2010, 31, 637–651.
[CrossRef]
ISPRS Int. J. Geo-Inf. 2019, 8, 3 21 of 25
17. Callieri, M.; Chica, A.; Dellepiane, M.; Besora, I.; Corsini, M.; Moyés, J.; Ranzuglia, G.; Scopigno, R.; Brunet, P.
Multiscale Acquisition and Presentation of very Large Artifacts. J. Comput. Cultural Heritage 2011, 3, 1–20.
[CrossRef]
18. Jung, T.H.; tom Dieck, M.C. Augmented Reality, Virtual Reality and 3D Printing for the Co-Creation of Value
for the Visitor Experience at Cultural Heritage Places. J. Place Manag. Dev. 2017, 10, 140–151. [CrossRef]
19. De la Peña, N.; Weil, P.; Llobera, J.; Spanlang, B.; Friedman, D.; Sanchez-Vives, M.; Slater, M. Immersive
Journalism: Immersive Virtual Reality for the First-Person Experience of News. Presence Teleoperators
Virtual Environ. 2010, 19, 291–301. [CrossRef]
20. Bideau, B.; Kulpa, R.; Vignais, N.; Brault, S.; Multon, F.; Craig, C. Virtual reality, a Serious Game for
Understanding Performance and Training Players in Sport. IEEE Comput. Gr. Appl. 2009. [CrossRef]
21. Staurset, E.; Prasolova-Førland, E. Creating a Smart Virtual Reality Simulator for Sports Training and
Education. Smart Innov. Syst. Technol. 2016, 59, 423–433.
22. Mól, A.C.; Jorge, C.A.; Couto, P.M. Using a Game Engine for VR Simulations in Evacuation Planning.
IEEE Comput. Gr. Appl. 2008, 28, 6–12. [CrossRef]
23. Shen, S.; Gong, J.; Liang, J.; Li, W.; Zhang, D.; Huang, L.; Zhang, G. A Heterogeneous Distributed Virtual
Geographic Environment—Potential Application in Spatiotemporal Behavior Experiments. ISPRS Int.
J. Geo-Inf. 2018, 7, 54. [CrossRef]
24. Cook, D.; Cruz-Neira, C.; Kohlmeyer, B.D.; Lechner, U.; Lewin, N.; Nelson, L.; Olsen, A.; Pierson, S.;
Symanzik, J. Exploring Environmental Data in a Highly Immersive Virtual Reality Environment.
Environ. Monit. Assess. 1998, 51, 441–450. [CrossRef]
25. Kreylos, O.; Bawden, G.; Bernardin, T.; Billen, M.I.; Cowgill, E.S.; Gold, R.D.; Hamann, B.; Jadamec, M.;
Kellogg, L.H.; Staadt, O.G.; et al. Enabling scientific workflows in virtual reality. In Proceedings of the 2006
ACM international conference on Virtual reality continuum and its applications, VRCIA ’06, Hong Kong,
China, 14–17 June 2006; ACM: New York, NY, USA, 2006; pp. 155–162. [CrossRef]
26. Lin, H.; Gong, J. Exploring Virtual Geographic Environments. Geogr. Inf. Sci. 2001, 7, 1–7. [CrossRef]
27. Kubíˇcek, P.; Šašinka, ˇC.; Stachoˇn, Z.; Herman, L.; Juˇrík, V.; Urbánek, T.; Chmelík, J. Identification of altitude
profiles in 3D geovisualizations: The role of interaction and spatial abilities. Int. J. Dig. Earth 2017, 1–17.
[CrossRef]
28. Carbonell-Carrera, C.; Saorín, J. Geospatial Google Street View with Virtual Reality: A Motivational Approach
for Spatial Training Education. ISPRS Int. J. Geo-Inf. 2017, 6, 261. [CrossRef]
29. Virtanen, J.; Hyyppä, H.; Kämäräinen, A.; Hollström, T.; Vastaranta, M.; Hyyppä, J. Intelligent Open Data 3D
Maps in a Collaborative Virtual World. ISPRS Int. J. Geo-Inf. 2015, 4, 837–857. [CrossRef]
30. Zanola, S.; Fabrikant, S.I.; Çöltekin, A. The Effect of Realism on the Confidence in Spatial Data Quality
in Stereoscopic 3D Displays. In Proceedings of the 24th International Cartography Conference, ICC 2009,
Santiago, Chille, 15–21 November 2009; pp. 15–21.
31. Zhang, S.; Moore, A. The Usability of Online Geographic Virtual Reality for Urban Planning.
ISPRS—International Archives of the Photogrammetry. Remote Sens. Spat. Inf. Sci. 2013, XL-2/W2, 145–150.
[CrossRef]
32. Abulrub, A.; Attridge, A.; Williams, M. Virtual Reality in Engineering Education: The Future of Creative
Learning. Int. J. Emerg. Technol. Learn. 2011, 6. [CrossRef]
33. Dinis, F.M.; Guimaraes, A.S.; Carvalho, B.R.; Martins, J.P.P. Development of Virtual Reality Game-based
Interfaces for Civil Engineering Education. In Proceedings of the 2017 IEEE Global Engineering Education
Conference, EDUCON, Athens, Greece, 26–28 April 2017; pp. 1195–1202. [CrossRef]
34. Lalley, J.P.; Piotrowski, P.S.; Battaglia, B.; Brophy, K.; Chugh, K. A Comparison of V-Frog[C] to Physical Frog
Dissection. Int. J. Environ. Sci. Educ. 2010, 5, 189–200.
35. Ai-Lim Lee, E.; Wong, K.; Fung, C. How Does Fesktop Virtual Reality Enhance Learning Outcomes?
A Structural Equation Modeling Approach. Comput. Educ. 2010, 55, 1424–1442. [CrossRef]
36. Lee, B.W.; Shih, H.Y.; Chou, Y.T.; Chen, Y.S. Educational Virtual Reality Implementation on English for
Tourism Purpose Using Knowledge-based Engineering. In Proceedings of the 2017 International Conference
on Applied System Innovation, ICASI, Sapporo, Japan, 13–17 May 2017; pp. 792–795. [CrossRef]
37. Lai, C.; McMahan, R.P.; Kitagawa, M.; Connolly, I. Geometry Explorer: Facilitating Geometry Education
with Virtual Reality. In Virtual, Augmented and Mixed Reality; Springer International Publishing: Cham,
Switzerland, 2016; pp. 702–713. [CrossRef]
ISPRS Int. J. Geo-Inf. 2019, 8, 3 22 of 25
38. Molka-Danielsen, J.; Prasolova-Førland, E.; Fominykh, M.; Lamb, K. Reflections on Design of Active
Learning Module for Training Emergency Management Professionals in Virtual Reality. In NOKOBIT—Norsk
Konferanse for Organisasjoners Bruk av Informasjonsteknologi; NOKOBIT, Bibsys Open Journal Systems: Oslo,
Norway, 2017; Volume 25, p. 1.
39. Van der Linden, A.; van Joolingen, W. A Serious Game for Interactive Teaching of Newton’s Laws. In
Proceedings of the 3rd Asia-Europe Symposium on Simulation & Serious Gaming—VRCAI ’16, New York,
NY, USA, 3–4 December 2016; pp. 165–167. [CrossRef]
40. ¸Stefan, L. Immersive Collaborative Environments for Teaching and Learning Traditional Design. Procedia Soc.
Behav. Sci. 2012, 51, 1056–1060. [CrossRef]
41. Hirmas, D.; Slocum, T.; Halfen, A.; White, T.; Zautner, E.; Atchley, P.; Liu, H.; Johnson, W.; Egbert, S.;
McDermott, D. Effects of Seating Location and Stereoscopic Display on Learning Outcomes in an Introductory
Physical Geography Class. J. Geosci. Educ. 2014, 62, 126–137. [CrossRef]
42. Stojši´c, I.; Ivkov Džigurski, A.; Mariˇci´c, O.; Ivanovi´c Bibi´c, L.; Đukiˇcin Vuˇckovi´c, S. Possible Application of
Virtual Reality in Geography Teaching. J. Subj. Didact. 2017, 1, 83–96. [CrossRef]
43. Koneˇcný, M. Cartography: Challenges and Potential in the Virtual Geographic Environments Era. Ann. GIS
2011, 17, 135–146. [CrossRef]
44. Herman, L.; Kvarda, O.; Stachoˇn, Z. Cheap and Immersive Virtual Reality: Application in Cartography.
In ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences; Zlatanova, S.,
Dragicevic, S., Sithole, G., Eds.; Copernicus GmbH: Gottingen, Germany, 2018; Volume XLII-4, pp. 261–266.
[CrossRef]
45. Masrur, A.; Zhao, J.; Wallgrün, J.O.; LaFemina, P.; Klippel, A. Immersive Applications for Informal and
Interactive Learning for Earth Science. In Proceedings of the Workshop on Immersive Analytics, Exploring
Future Interaction and Visualization Technologies for Data Analytics, VIS2017, Phoenix, AZ, USA, 1–6
October 2017; pp. 1–5.
46. Hsu, W.C.; Lin, H.C.K.; Lin, Y.H. The research of applying Mobile Virtual Reality to Martial Arts learning
system with flipped classroom. In Proceedings of the 2017 International Conference on Applied System
Innovation, ICASI, Sapporo, Japan, 13–17 May 2017; pp. 1568–1571. [CrossRef]
47. Halabi, O.; Abou El-Seoud, S.; Alja’am, J.; Alpona, H.; Al-Hemadi, M.; Al-Hassan, D. Design of Immersive
Virtual Reality System to Improve Communication Skills in Individuals with Autism. Int. J. Emerg.
Technol. Learn. 2017, 12, 50. [CrossRef]
48. Fisher, P.; Unwin, D. Virtual Reality in Geography; Taylor and Francis Ltd.: London, UK, 2001.
49. Batty, M. Virtual Reality in Geographic Information Systems. In The Handbook of Geographic Information
Science; Wilson, J.P., Fotheringham, A.S., Eds.; Blackwell Publishing: Oxford, UK, 2008; pp. 317–334.
50. Lin, H.; Chen, M.; Lu, G.; Zhu, Q.; Gong, J.; You, X.; Wen, Y.; Xu, B.; Hu, M. Virtual Geographic Environments
(VGEs): A New Generation of Geographic Analysis Tools. Earth-Sci. Rev. 2013, 126, 74–84. [CrossRef]
51. Lin, H.; Batty, M.; Jorgensen, S.E.; Fu, B.; Koneˇcný, M.; Voinov, A.A.; Torrens, P.; Lu, G.; Zhu, A.X.;
Wilson, J.P.; et al. Virtual Environments Begin to Embrace Process-based Geographic Analysis. Trans.
GIS 2015, 19, 493–498. [CrossRef]
52. Philips, A.; Walz, A.; Bergner, A.; Graeff, T.; Heistermann, M.; Kienzler, S.; Korup, O.; Lipp, T.;
Schwanghart, W.; Zeilinger, G. Immersive 3D Geovisualization in Higher Education. J. Geogr. Higher Educ.
2015, 39, 437–449. [CrossRef]
53. Carbonell Carrera, C.; Saorin, J.L.P.; de la Torre Cantero, J. Teaching with AR as a Tool for Relief Visualization:
Usability and Motivation Study. Int. Res. Geogr. Environ. Educ. 2017, 27, 69–84. [CrossRef]
54. Merchant, Z.; Goetz, E.; Cifuentes, L.; Keeney-Kennicutt, W.; Davis, T. Effectiveness of Virtual Reality-based
Instruction on Students’ Learning Outcomes in K-12 and Higher Education: A Meta-analysis. Comput. Educ.
2014, 70, 29–40. [CrossRef]
55. Zyda, M. From Visual Simulation to Virtual Reality to Games. Computer 2005, 38, 25–32. [CrossRef]
56. Gee, J. What Video Games Have to Teach us About Learning and Literacy; Palgrave Macmillan: Basingstoke,
UK, 2008.
57. Ang, C.S.; Rao, G.S.V.R.K. Computer Game Theories for Designing Motivating Educational Software:
A Survey Study. Int. J. E-Learn. 2008, 7, 181–199.
ISPRS Int. J. Geo-Inf. 2019, 8, 3 23 of 25
58. Kyndt, E.; Raes, E.; Lismont, B.; Timmers, F.; Cascallar, E.; Dochy, F. A Meta-analysis of the Effects of
Face-to-face Cooperative Learning. Do Recent Studies Falsify or Verify Earlier Findings? Educ. Res. Rev.
2013, 10, 133–149. [CrossRef]
59. Harding-Smith, T. Learning Together: An Introduction to Collaborative Learning; HarperCollins College
Publishers: New York, NY, USA, 1993.
60. Vygotsky, L. Interaction between Learning and Development; W.H. Freeman and Company: New York, NY,
USA, 1997.
61. Dillenbourg, P. Collaborative Learning: Cognitive and Computational Approaches; Advances in Learning and
Instruction Series; Elsevier Science, Inc.: New York, NY, USA, 1999.
62. Gokhale, A.A. Collaborative Learning Enhances Critical Thinking. J. Technol. Educ. 1995, 7, 22–30. [CrossRef]
63. Havenith, H.-B.; Cerfontaine, P.; Mreyen, A.-S. How Virtual Reality Can Help Visualise and Assess
Geohazards. Int. J. Digit. Earth 2017. [CrossRef]
64. Lin, H.; Chen, M.; Lu, G. Virtual Geographic Environment: A Workspace for Computer-Aided Geographic
Experiments. Ann. Assoc. Am. Geogr. 2013, 103, 465–482. [CrossRef]
65. Lehtinen, E.; Hakkarainen, K.; Lipponen, L.; Veermans, M.; Muukkonen, H. Computer Supported
Collaborative Learning: A Review. JHGI Giesbers Rep. Educ. 1999, 10, 1999.
66. Jackson, R.L.; Fagan, E. Collaboration and learning within immersive virtual reality. In Proceedings of the 3rd
International Conference on Collaborative Virtual Environments; Churchill, E., Reddy, M., Eds.; ACM: New York,
NY, USA, 2000; pp. 83–92. [CrossRef]
67. He, T.; Chen, X.; Chen, Z.; Li, Y.; Liu, S.; Hou, J.; He, Y. Immersive and Collaborative Taichi Motion Learning
in Various VR Environments. In Proceedings of the 2017 IEEE Virtual Reality (VR), Los Angeles, CA, USA,
18–22 March 2017; IEEE: Los Angeles, CA, USA, 2017; pp. 307–308. [CrossRef]
68. Kaufmann, H.; Schmalstieg, D. Designing Immersive Virtual Reality for Geometry Education. In Proceedings
of the IEEE Virtual Reality Conference (VR 2006), Alexandria, VA, USA, 25–29 March 2006; pp. 51–58.
[CrossRef]
69. Dos Reis, P.R.; Matos, C.E.; Diniz, P.S.; Silva, D.M.; Dantas, W.; Braz, G.; de Paiva, A.C.; Araújo, A.S.
An Immersive Virtual Reality Application for Collaborative Training of Power Systems Operators.
In Proceedings of the 2015 XVII Symposium on Virtual and Augmented Reality, Sao Paulo, Brazil, 25–28
May 2015; pp. 121–126. [CrossRef]
70. Siemens, G. Connectivism: A Learning Theory for the Digital Age. Int. J. Instruct. Technol. Distance Learn.
2014, 2, 1–8.
71. Horvath, I. Innovative Engineering Education in the Cooperative VR Environment. In Proceedings of the
2016 7th IEEE International Conference on Cognitive Infocommunications, CogInfoCom, Wroclaw, Poland,
16–18 October 2016; pp. 359–364. [CrossRef]
72. Hew, K.; Cheung, W. Use of Three-dimensional (3-D) Immersive Virtual Worlds in K-12 and Higher Education
Settings: A Review of the Research. Br. J. Educ. Technol. 2010, 41, 33–55. [CrossRef]
73. Doležal, M.; Chmelík, J.; Liarokapis, F. An Immersive Virtual Environment for Collaborative Geovisualization.
In Proceedings of the 9th International Conference on Virtual Worlds and Games for Serious Applications,
VS-Games 2017, Athens, Greece, 6–8 September 2017; pp. 272–275. [CrossRef]
74. Juˇrík, V.; Herman, L.; Kuníˇcek, P.; Stachoˇn, Z.; Šašinka, ˇC. Cognitive Aspects of Collaboration in 3D Virtual
Environments. In ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences;
Halounova, L., Ed.; Copernicus GmbH: Gottingen, Germany, 2016; Volume XLI-B2, pp. 663–670. [CrossRef]
75. Symeonides, R.; Childs, C. The Personal Experience of Online Learning: An Interpretative Phenomenological
Analysis. Comput. Hum. Behav. 2015, 51, 539–545. [CrossRef]
76. Smith, J.A.; Flowers, P.; Larkin, M. Interpretative Phenomenological Analysis: Theory, Method and Research; Sage
Publications Ltd.: London, UK, 2009.
77. Reid, K.; Flowers, P.; Larkin, M. Exploring Lived Experience. Psychologist 2005, 18, 20–23.
78. Smith, J.A.; Jarman, M.; Osborn, M. Doing Interpretative Phenomenological Analysis. In Qualitative Health
Psychology; Murray, M., Chamberlain, K., Eds.; Sage Publications Ltd.: London, UK, 1999; pp. 218–240.
79. Gorard, S.; Taylor, C. Combining Methods in Educational and Social Research; Open University Press: Berkshire,
UK, 2004.
80. Brannen, J. Combining Qualitative and Quantitative Approaches: An Overview. In Mixing Methods:
Qualitative and Quantitative Research; Brannen, J., Ed.; Avebury: Aldershot, UK, 1992.
ISPRS Int. J. Geo-Inf. 2019, 8, 3 24 of 25
81. Smith, J.A.; Osborn, M. Interpretative Phenomenological Analysis. In Qualitative Psychology: A Practical Guide
to Research Methods; Smith, J.A., Ed.; Sage Publications Ltd.: London, UK, 2003; pp. 53–80.
82. Smith, J.A. Reflecting on the Development of Interpretative Phenomenological Analysis and its Contribution
to Qualitative Research in Psychology. Qual. Res. Psychol. 2004, 1, 39–54.
83. O’Donnell, A.M.; Hmelo-Silver, C.E. Introduction What is Collaborative Learning?: An Overview. In The
International Handbook of Collaborative Learning; Routledge: New York, NY, USA, 2013; pp. 13–28.
84. Slavin, R. When Does Cooperative Learning Increase Student Achievement? Psychol. Bull. 1983, 94, 429–445.
[CrossRef]
85. Johnson, D.; Johnson, R. Making Cooperative Learning Work. Theory Pract. 1999, 38, 67–73. [CrossRef]
86. O’Donnell, A.M.; Dansereau, D.F.; Hythecker, V.I.; Larson, C.O.; Rocklin, T.R.; Lambiotte, J.G.; Young, M.D.
The Effects of Monitoring on Cooperative Learning. J. Exp. Educ. 1986, 54, 169–173. [CrossRef]
87. O’Donnell, A.M.; Dansereau, D.F. Scripted Cooperation in Student Dyads: A Method for Analyzing and
Enhancing Academic Learning and Performance. In Interaction in Cooperative Groups: The Theoretical Anatomy
of Group Learning; Hertz-Lazarowitz, R., Miller, N., Eds.; Cambridge University Press: Cambridge, UK, 1992;
pp. 120–141.
88. Webb, N.; Farivar, S. Promoting Helping Behavior in Cooperative Small Groups in Middle School
Mathematics. Am. Educ. Res. J. 1994, 31, 369. [CrossRef]
89. Duschl, R.; Osborne, J. Supporting and Promoting Argumentation Discourse in Science Education.
Stud. Sci. Educ. 2002, 38, 39–72. [CrossRef]
90. Piaget, J. The Psychology of Intelligence; Taylor & Francis: London, UK, 2005; ISBN 0-203-98152-9.
91. Dewey, J. Experience and Education; Macmillan: New York, NY, USA, 1938.
92. Kolb, D. Experiential Learning, 2nd ed.; Pearson Education: Upper Saddle River, NJ, USA, 2015.
93. Neale, D.; Smith, D.; Johnson, V. Implementing Conceptual Change Teaching in Primary Science. Elem. Sch. J.
1990, 91, 109–131. [CrossRef]
94. Vygotsky, L.S. Mind in Society: The Development of Higher Psychological Processes; Harvard University Press:
Cambridge, MA, USA, 1979.
95. Hogan, K.; Pressley, M. Scaffolding Scientific Competencies within Classroom Communities of Inquiry. In
Advances in Learning & Teaching. Scaffolding Student Learning: Instructional Approaches and Issues; Hogan, K.,
Pressley, M., Eds.; Brookline Books: Cambridge, MA, USA, 2017; pp. 74–107.
96. Guzdial, M.; Kolodner, J.; Hmelo, C.; Narayanan, H.; Carlson, D.; Rappin, N.; Hübscher, R.; Turns, J.;
Newstetter, W. Computer support for learning through complex problem solving. Commun. ACM 1996, 39,
43–45. [CrossRef]
97. Chen, Q.; Zhang, J. 2 Collaborative Discovery Learning Based on Computer Simulation. In Collaborative
Learning, Reasoning and Technology, 2nd ed.; O’Donnell, A., Hmelo-Silver, C.E., Erkens, G., Eds.; Routledge:
New York, NY, USA, 2012; pp. 154–196.
98. Okada, T.; Simon, H. Collaborative Discovery in a Scientific Domain. Cognit. Sci. 1997, 21, 109–146.
[CrossRef]
99. Ekman, P.; Friesen, W. Unmasking the Face; Malor Books: Cambridge, MA, USA, 2003.
100. Tu, C.H. Online Collaborative Learning Communities: Twenty-one Designs to Building an Online Collaborative
Learning Community; Libraries Unlimited: Westport, Ireland, 2004.
101. Bailenson, J.; Blascovich, J.; Beall, A.; Loomis, J. Equilibrium Theory Revisited: Mutual Gaze and Personal
Space in Virtual Environments. Presence Teleoperators Virtual Environ. 2001, 10, 583–598. [CrossRef]
102. Bailenson, J.; Blascovich, J.; Beall, A.; Loomis, J. Interpersonal Distance in Immersive Virtual Environments.
Personal. Soc. Psychol. Bull. 2003, 29, 819–833. [CrossRef] [PubMed]
103. LaViola, J. A Discussion of Cybersickness in Virtual Environments. ACM Sigchi Bull. 2000, 32, 47–56.
[CrossRef]
104. Davis, S.; Nesbitt, K.; Nalivaiko, E. A Systematic Review of Cybersickness. In Proceedings of the 2014
Conference on Interactive Entertainment, IE2014, Newcastle, Australia, 2–3 December 2014; Blackmore, K.,
Nesbitt, K., Smith, S.P., Eds.; ACM: New York, NY, USA, 2014. [CrossRef]
105. Witmer, B.; Singer, M. Measuring Presence in Virtual Environments: A Presence Questionnaire.
Presence Teleoperators Virtual Environ. 1998, 7, 225–240. [CrossRef]
106. Cruz-Neira, C.; Sandin, D.; DeFanti, T.; Kenyon, R.; Hart, J. The CAVE: Audio Visual Experience Automatic
Virtual Environment. Commun. ACM 1992, 35, 64–72. [CrossRef]
ISPRS Int. J. Geo-Inf. 2019, 8, 3 25 of 25
107. Csikszentmihaiyi, M. Flow: The Psychology of Optimal Experience; Harper & Row Publishers: New York, NY,
USA, 1990.
108. Aardema, F.; O’Connor, K.; Côté, S.; Taillon, A. Virtual Reality Induces Dissociation and Lowers Sense of
Presence in Objective Reality. Cyberpsychol. Behav. Soc. Netw. 2010. [CrossRef] [PubMed]
109. Parong, J.; Mayer, R.E. Learning Science in Immersive Virtual Reality. J. Educ. Psychol. 2018, 110, 785–797.
[CrossRef]
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).
International Journal of
Geo-Information
Article
Effect of Size, Shape and Map Background in
Cartographic Visualization: Experimental Study on
Czech and Chinese Populations
Zdenˇek Stachoˇn 1 , ˇCenˇek Šašinka 2 , Jiˇrí ˇCenˇek 3 , Stephan Angsüsser 4,* , Petr Kubíˇcek 1 ,
Zbynˇek Štˇerba 1 and Martina Bilíková 1
1 Department of Geography, Faculty of Science, Masaryk University, 61137 Brno, Czech Republic;
zstachon@geogr.muni.cz (Z.S.); kubicek@geogr.muni.cz (P.K.); zbynek.ste@gmail.com (Z.Š.);
bilikova.martina@email.cz (M.B.)
2 Division of Information and Library Studies and HUME Lab, Faculty of Arts, Masaryk University,
60200 Brno, Czech Republic; ceneksasinka@gmail.com
3 Department of Social Studies, Faculty of Regional Development and International Studies,
Mendel University in Brno, 61300 Brno, Czech Republic; jiricendacentrum@gmail.com
4 Department of Geographic Information Science and Cartography, School of Resource and Environmental
Science, Wuhan University, Wuhan 430079, China
* Correspondence: stephan.angsuesser@gmx.net; Tel: +86-27-68778319
Received: 19 September 2018; Accepted: 27 October 2018; Published: 1 November 2018
Abstract: This paper deals with the issue of the perceptual aspects of selected graphic variables
(specifically shape and size) and map background in cartographic visualization. The continued
experimental study is based on previous findings and the presupposed cross-cultural universality of
shape and size as a graphic variable. The results bring a new perspective on the usage of shape, size and
presence/absence of background as graphic variables, as well as a comparison to previous studies.
The results suggest that all examined variables influence the speed of processing. Respondents (Czech
and Chinese, N = 69) identified target stimuli faster without a map background, with larger stimuli,
and with triangular and circular shapes. Czech respondents were universally faster than Chinese
respondents. The implications of our research were discussed, and further directions were outlined.
Keywords: cartography; cross-cultural research; geometric map symbols; graphic variables; visual
perception; visualization
1. Introduction
Research on methods of cartographic visualization has a long tradition and implications for
many fields of human activity, including crisis and disaster visualizations [1,2]. The issue of graphic
variables defined by Jacques Bertin [3] is mentioned in many scientific cartographic publications.
Most of the works on graphic variables focus on Bertin’s work and try to expand on his conclusions,
which are considered generally valid in cartography. Many scientific works focus on this theory [4–13],
and some papers try to extend the usage of the principles defined [14–17], but only a small part of
published research focuses on verifying his theories (see further). For example, in his presentation at
the International Cartographic Conference in 2017, Alan MacEachren mentioned that Bertin’s work
is “ . . . often uncritically accepted” and highlighted the few uses of multivariate data/graphics [18].
Another example is the work by Christophe and Hoarau [19], where the influence of aesthetics is
emphasized, whereas aesthetics in cartographic visualization is missing from Bertin’s concept.
Bertin [3] postulated that map elements are a specific graphics system: a set of six basic variables
mapping each character, of which variable characteristics are dedicated more to the graphic system,
ISPRS Int. J. Geo-Inf. 2018, 7, 427; doi:10.3390/ijgi7110427 www.mdpi.com/journal/ijgi
ISPRS Int. J. Geo-Inf. 2018, 7, 427 2 of 15
describing their features, characters and relationships. Bertin defined the basic variables as follows
(see also Figure 1):
• Size (taille): variation in the area size covered by a sign at a constant shape
• Value (valeur): variation in the ratio of the total amount of black and white in the perceived color
of a given area
• Texture (grain): variation in the amount of discernable uniform marks per unit area at a constant
value (there should be no variation in value, which is often ignored in the literature about
Bertin’s variables)
• Hue (couleur): wavelength variation within the visible part of the electromagnetic spectrum
between two areas at a constant value
• Orientation (orientation): angular difference between several arrays of parallel signs
• Shape (forme): variation in the outline character (form) of a sign at a constant size
Figure 1. Bertin’s six basic variables ([3]; depiction adapted from [20]).
These variables can be used to express quantitative and qualitative characteristics while also
representing an aesthetic function. Bertin assigned five basic characteristics to the six basic variables:
association, qualitative difference, selection, arrangement, and proportionality. Features and variables
make up a total of 63 combinations available for map symbol creation. For a detailed description of
graphic variables, see [10,21,22]. MacEachren [18] also mentioned that the main challenge currently is
to focus on pre-cognitive perception rather than the further stages of cognitive processing.
Based on the above-mentioned theoretical issues, we researched using methodological approaches
inspired by psychology and focused on verifying the perceptual aspects of selected graphic
variables (specifically shape and size) in cartography and the cross-cultural universality of selected
graphic variables.
2. Perception
A psychophysics movement in psychology was inspired by methods of natural sciences and
concerned with the relationship between physical stimuli and the perception they produce was
established. It introduced, among others, the key concept of sensory threshold as respectively
differential and absolute sensitivity [23]. Later, at the start of twentieth century, the perceptual
organization was first studied by a Gestalt psychologist, who investigated the principles of how
elementary sensation is grouped into more complex structures and how figure-ground segregation
proceeds [24].
Gestalt laws are also applied in cartographic visualization [8,25]. The basic assumptions and
principles of the Gestalt school are still accepted in the scientific community [26], although the influence
of top-down processes was later incorporated into this theoretical approach, which formally stressed
ISPRS Int. J. Geo-Inf. 2018, 7, 427 3 of 15
only a bottom-up direction [27]. The work of Ulric Neisser [28] led to a cognitive turn in psychology.
He highlighted the role of attention and anticipation in perception and stressed the importance of
past experience. Based on this, it can be assumed that different individual experiences including
socio-cultural context may cause differences in visual perception. Visual perception is determined by
neurophysiological aspects (e.g., “the oblique effect” describes the decrease in performance when users
deal with oblique shapes compared to cardinally (vertical and horizontal) oriented figures [29,30]),
as well as by environmental (cultural) factors shaping the human experience.
It may be possible to use the findings from psychological research as guidelines for cartographers
during map creation. However, in our opinion, several limitations do not allow for the direct
application of findings from psychological studies. Psychological experiments do not fit the criteria of
representative research design and an ecological approach [31,32]. Stimulus material in psychological
experiments is usually very simplified to ensure the high internal validity of conducted studies and
is also abstract without meaning. By contrast, maps are complex visual representations that always
communicate meanings. Both factors lead to a lack of external validity in the results and limit their use
in the field of cognitive cartography.
3. Research on Graphic Variables in Cartography
As previously mentioned, much research on graphic variables has been done in the field of
cognitive cartography. A detailed description is provided, for example, by Montello [33].
Some of the foremost research focusing on the practical verification of graphic variables was done
by Czech cartographer, Antonín Koláˇcný, in the 1960s on school atlases [34]. A crucial part of the
extensive, empirical research dealt with differential thresholds, which are the smallest variation in the
height of various shapes of map features sufficient to be noticeable. Table 1 shows the final summary
of recommended differential thresholds for each specific shape [34]. Figure 2 shows the minimum size
of certain shapes at a reading distance of 40 cm. Figure 3 displays the influence of the quantity of signs
on the map on the user speed.
Table 1. Difference threshold for each shape [34].
Shape Threshold (%)
SquareSquare 88
CircleCircle 1212
RectangleRectangle 1212
Equilateral triangleEquilateral triangle 88
Isosceles triangleIsosceles triangle 1515
Figure 2. Minimum size (mm) at a reading distance of 40 cm (adapted from [34]).
ISPRS Int. J. Geo-Inf. 2018, 7, 427 4 of 15
Figure 3. Map reading speed (adapted from [34]).
In their 2011 article titled “The influence of Jacques Bertin”, Deeb et al. [35] examined the
applicability of Bertin’s variables for the purposes of labelling. In this particular study, testing was
performed on two types of map users (laymen and professionals), who expressed certain preferences.
In the conclusion, additional research is recommended on user groups matched by other criteria,
such as age, culture, education or gender.
Although Bertin’s theory is well known and widely used in the field of cartography and
geoinformatics, a certain amount of research tacitly rejects his entire theory of visual variables.
Garlandini and Fabrikant [36] mentioned that Bertin unfortunately did not use any citation of
published researches from the field of psychology that could possibly support the designed system.
Reimer claims, in his 2011 article titled “Squaring the Circle? Bivariate Color Maps and Jacques
Bertin’s Concept of ‘Disassociation’” [37], that Bertin’s concept of associativity and disassociativity
of relevant visual variables is mostly ignored or misinterpreted, especially in the English-speaking
world. Anglo-American tradition follows an evolutionary path that is heavily influenced by computer
technology and the representation of color in a color space, while continental European tradition is
influenced more by heuristics, which uses cartographic best practices. The differences mentioned
brought us to the additional focus on the influence of different cultural backgrounds on the perception
of certain graphic variables.
4. Cross-Cultural Research and Cartography
Cross-cultural research is still a neglected area in cartography. As Rundstrom [38] described,
it started in the 1970s and 1980s [17,39–41] and since then, has mainly dealt with historical and
indigenous cartographies. Cultural aspects are only occasionally the focus of studies concerning
mainstream contemporary mapping procedures and their products. Cross-cultural aspects are taken
into consideration even more rarely. As a consequence, we often need to rely on the results obtained
from related fields, such as cross-cultural user interface design or more general subjects such as
cross-cultural psychology, to get a better understanding of how cultures might influence maps
and mapping.
Not surprisingly, it is difficult to find any cross-cultural research on the application of graphic
variables in maps. In a comparative study on Swiss and Chinese regional planning maps [42], the use
of extended variables, such as transparency, color gradient or shading (in both countries), is mentioned
but not further discussed. In a recent study by Stachoˇn et al. [43], Czech and Chinese students had
to identify differently shaped figural symbols in a map reading task. Main differences between the
two used symbol sets were the degree of schematization and the presence or absence of a figure
background with outline. Results showed that the Czech participants identified symbols from both sets
faster than the Chinese participants, but this difference was only significant for the less schematized
(more iconic) symbols without backgrounds/outlines. As such, an identification task is more analytic
in nature, and this result could be interpreted as a consequence of the more analytic cognitive style
of Czech participants. However, the also conducted framed-line test resulted in a different picture.
There, the Czech students performed significantly worse than the Chinese students in the absolute
ISPRS Int. J. Geo-Inf. 2018, 7, 427 5 of 15
task, which indicates a more holistic cognitive style of Czech participants. Such contradictory results
show the need for further research.
One starting point for our study was the comparison of point symbol sizes used in printed
German and Chinese city maps (published from 2000 to 2009) with recommended minimum size
dimensions [44]. In this study, striking average size differences were documented. Based on the longest
side measurement, more than 90% of the German point symbols were larger than the recommended
minimum size dimensions, while this held true only for a third of the Chinese point symbols.
This raised the question whether minimum size recommendations are universally valid and what
the reason for this result could be. Among the potential answers discussed in that paper, the most
interesting answers are briefly summarized below.
One factor considered was the influence of Chinese script. It consists of several thousand symbols
with varying complexities. Consequently, Chinese students must invest a lot of time in learning the
written form of their language. However, this effort also brings some additional benefits. Referring to
several studies [45–50], Choong and Salvendy concluded that “Chinese tend to have superior visual
form discrimination abilities.” [51] (p. 420). However, this does not automatically lead to an increased
ability to discriminate very small signs or shape details. In other words, it is not clear whether Chinese
possess a culturally induced increased visual acuity as a consequence of their script learning efforts
(a kind of “trained eyesight”).
Another study discussed here [52] determined the minimum legible sizes of Chinese characters
with different complexities and the readability of strings four characters long. Its results document
a higher influence from character string familiarity than from legibility according to the minimum
legible sizes found. This is supported by a recent study [53] that discovered evidence for a “holistic
neural representation of Chinese characters” (p. 32), which is also in line with the characterization
of cognitive styles, analytic and holistic thinking [54] (p. 293). Even if parts of a symbol are illegible,
the character can still be read based on its overall impression. As a consequence, it seems less likely
that increased visual acuity evolves from learning the Chinese script, and yet a certain training effect
on eyesight cannot be ruled out.
This view is supported by two other cultural dimensions (sometimes called cultural variables):
communication orientation and uncertainty avoidance. The first, described by Nisbett [55] (pp. 60–61),
differentiates between a Western transmitter and an Asian receiver orientation. In transmitter-oriented
cultures, it is the transmitter who is mainly responsible for a successful communication, i.e., a message
is understood as intended, while in receiver-oriented cultures, the responsible person is the receiver.
A transmitter orientation is more likely adopted by cultures that usually try to avoid uncertainty and
therefore will not accept symbols that are very small, and hard to identify. Cultures scoring high
on Hofstede’s “uncertainty avoidance” [56] (p. 336) dimension usually tend to “shun ambiguous
situations” [57] (p. 197). As expected, countries like Germany (score: 65; all scores are from [57]
(pp. 192–194, Table 6.1)) and the Czech Republic (score: 74) score considerably higher than China
(score: 30). We can therefore argue that while it is important for Germans or Czechs to discriminate all
parts of a symbol without undue difficulty, it is enough for Chinese to identify a symbol as a whole,
i.e., being able to match it with its counterpart in the key. Acceptable minimum symbol dimensions
would then be considerably lower for Chinese than for Germans or Czechs.
Besides small size and readability issues, another difference might exist between Chinese and
European point symbols regarding preferences for basic geometric shapes. Angsüsser [20] found more
frequent use of circles and squares and near absence of triangles as point symbols in Chinese city
maps compared to German city maps. On the other hand, diamonds were sometimes found in the
Chinese sample while being absent in the German sample. As this shape has an important significance
in China’s everyday life (e.g., as an ornament or background for Chinese characters), we can assume
that the three main geometric shapes in China are circles, squares, and diamonds, whereas in Germany
and the so-called Western world, the circle, square, and triangle dominate.
ISPRS Int. J. Geo-Inf. 2018, 7, 427 6 of 15
5. Materials and Methods
5.1. Experiment
The general objective of the study was to observe the differences in how map symbols of
various shapes and sizes were perceived and the influence of different map backgrounds and cultural
backgrounds. The experiments were based on the theories [3,8] and experiments [34] described in the
first section. From the previous research, we hypothesized:
At the general level:
• Greater complexity of the map background significantly decreases the speed of processing map
stimuli during the search task.
• Increasing size significantly increases the speed of processing map stimuli during the search task.
• The shape of the point symbol does not significantly influence the speed of processing map stimuli
during the search task.
At the cultural level:
• Chinese are faster than Czechs in search tasks performed with small map stimuli.
5.2. Stimuli
The test consisted of two parts, each using a different type of visual stimuli. Both types of stimuli
differed in the presence or absence of a background map. While the stimuli in the first part of the test
were free of any background map and contained only point map symbols, the visual stimuli in the
second part also contained a background map (Figure 4 as an example).
Figure 4. Differences in the stimuli used with and without a map background (task instruction: find the
shape shown at the left side of the screen).
ISPRS Int. J. Geo-Inf. 2018, 7, 427 7 of 15
The point symbols used, which were identical for each of the presented tasks, were simple
geometric symbols in black color: a circle, a triangle, and a diamond (square rotated by 45 degrees).
The use of three different shapes should have provided enough elementary shape diversity, generally
typical for a cartographic visualization, which may have also helped increase the ecological validity of
the experiment. Each task contained 25 point symbols that were randomly placed in the map field,
one symbol being the attractor and the other 24 symbols being distractors, i.e., symbols with a shape
different from the attractor (12+12 symbols of both remaining shapes). Eight sizes were used for
each symbol, each differing in size by 1 pixel (from 6 pixels to 13 pixels). The absolute dimensions
calculated from the pixel size of used 22 inch monitors varied from 1.68 mm to 3.64 mm. The first
three categories (6, 7, and 8 pixels) close to the limits defined by Koláˇcný (Figure 2) were considered
as small, two middle categories (9, and 10 pixels) as medium size and the last three categories (11,
12, and 13 pixels) as large. The sizes of the symbols were designed to accommodate 24 tasks and
combined both parts of the test. Tasks in the first part did not include any background. Tasks in the
second part of the test included a background map covering a randomly chosen area with different
proportions of built-up and un-built areas. For the 24 tasks with map backgrounds, 24 different map
sections were used, identical for both cultural groups. For these purposes, we used OpenStreetMap
data (of a Greek city) that were visualized to be distinct from the most commonly used web map
portals of either cultural group (i.e., Google maps and Mapy.cz for the Czech group; Baidu maps for
the Chinese group).
5.3. Participants
To analyze the speed of detection of selected graphic variables in a cartographic visualization,
we gathered data from two groups (Czech and Chinese) of participants (69 participants in total) in
our study. The first group, located in Brno in the Czech Republic, consisted of 33 participants (57.5%
males). All the participants were Czech citizens and cartography and GIScience students at Masaryk
University (2nd and 3rd year of bachelor’s degree) in the Czech Republic. The second group consisted
of 36 participants (41.6% males). Participants of the second group were Chinese students of GIS
(3rd year of bachelor’s degree) from the University of Wuhan in China. The mean age of all participants
was 21.3 years (SD = 1.2 years). The mean age of the Czech group was 21.6 years (SD = 1.4 years) and
the mean age of the Chinese participants was 21 years (SD = 0.80 year). Based on the information
given by the participants, they all had normal or corrected-to-normal vision. All participants took part
in the experiment voluntarily and were told that the better half of each group would be rewarded with
a small gift (which was identical for both groups).
The minimum sample size was calculated using G*Power 3 [58] for repeated measure ANOVA
with within–between interactions (medium effect size f = 0.25, p = 0.80, α = 0.05, number of groups = 2,
number of measurements = 3, and estimated correlation between measurements = 0.3). The required
total sample size was estimated to the minimum of 38 participants. From this perspective, the number
of tested participants was sufficient to provide the adequate test power.
5.4. Procedure
The test started with a short questionnaire requesting basic personal information (age, sex,
nationality, education, etc.). The two parts of the map reading test were then presented to respondents.
Both parts consisted of 24 stimuli. Altogether, the test presented 48 stimuli (Figure 5).
ISPRS Int. J. Geo-Inf. 2018, 7, 427 8 of 15
Figure 5. Schema of the procedure used for testing.
Each respondent conducted the entire test via the interactive web-based application Hypothesis,
which was designed to aid in creating and administering effective visual perception and cognitive tests.
For more information about the application, see Šašinka et al. [59] and Popelka et al. [60]. Both Czech
and Chinese participants conducted the test using this application. We attempted to create the same
ambient conditions for testing, such as similar lighting conditions and the same number of participants
tested at one time (usually 15 participants). Nevertheless, we could not absolutely exclude other
factors that may have influenced the results. The most challenging task was to provide the same
stability for the internet connection at both locations. This was the reason for excluding several Chinese
participants from further analysis. Additionally, several participants in China did not complete the test
and were therefore removed from further analysis.
6. Results
Data analysis was performed using IBM SPSS Statistics 22 statistical software. Prior to data
analysis, an outlier analysis was performed on the reaction times for each of the 48 items. Based on
outlying reaction times (values below Q1-1.5 IQR or above Q3+1.5 IQR), 169 data points were deleted
from the data matrix. The number of deleted values in each item varied between 0 and 10; the mean
number of deleted values per item was 3.5.
As mentioned above, the size of the point symbols in the test items varied (eight size levels).
We calculated the average time required to identify the items that contained the three smallest and
the three largest point symbols. The items that contained the two medium-sized point symbols were
eliminated from further analysis. We analyzed the mean reaction times for each type of stimuli
(background—no background, and map background; size of point symbol—large, and small; shape of
point symbol—circle, triangle, and diamond). We also calculated the overall mean reaction time for
each participant and performed a stepwise linear regression with backward elimination, including
gender and nationality as explanatory variables and overall mean reaction time as a dependent variable.
We did not include age in the model because of its low variance. Gender was not a significant predictor
of overall reaction time. A significant regression equation (F(1, 67) = 18.59, p < 0.001, R2 = 0.35) was
found for nationality as a predictor of point symbol detection speed. Descriptions for stimuli with and
without map backgrounds, combined point symbol size and shape, and shape are in Figures 6 and 7
and Table 2 respectively.
ISPRS Int. J. Geo-Inf. 2018, 7, 427 9 of 15
Figure 6. Comparison of mean detection times with background and without background (in seconds,
N = 69).
Figure 7. Mean detection times regarding point symbol shape (in seconds, N = 69).
Table 2. Mean detection times regarding point symbol size (in seconds, N = 69).
Symbol
Size
Group
All Shapes:
Mean Time
(SD)
Circle: Mean
Time (SD)
Diamond:
Mean Time
(SD)
Triangle:
Mean Time
(SD)
Small Chinese 5.24 (0.93) 4.88 (0.99) 7.06 (1.73) 3.75 (0.52)
Czech 4.19 (0.72) 3.75 (0.77) 5.85 (1.57) 3.12 (0.62)
Total 4.74 (0.98) 4.34 (1.05) 6.48 (1.75) 3.45 (0.65)
Large Chinese 4.25 (0.60) 3.81 (0.58) 5.51 (0.96) 3.75 (0.66)
Czech 3.37 (0.59) 3.01 (0.49) 4.15 (1.16) 3.03 (0.65)
Total 3.83 (0.74) 3.43 (0.67) 4.67 (1.17) 3.40 (0.75)
ISPRS Int. J. Geo-Inf. 2018, 7, 427 10 of 15
The results show that reaction times were generally higher for items with a topographic
background (M = 4.24 s, SD = 0.79 s) compared to items with no background (M = 3.95 s, SD = 0.79 s).
The results were consistent across the subsamples. Czech participants were faster in both conditions
(Czechs: no background, M = 3.44 s, SD = 0.60 s; with a topographic background M = 3.84 s, SD = 0.59 s;
Chinese: no background, M = 4.41 s, SD = 0.66 s; with a topographic background, M = 4.61 s, SD = 0.79 s;
see Figure 6).
Reaction times were generally higher for items with small point symbols (M = 4.74 s) compared
to items with large point symbols (M = 3.83 s). The results were consistent across subsamples.
Czech participants were faster in both conditions.
Diamonds had the highest mean detection times (M = 5.35 s) in the entire sample. The mean
detection time for circles (M = 3.64 s) was slightly higher than for triangles (M = 3.30 s). The patterns
of detection speed for diamonds, circles, and triangles were consistent across nationalities.
To test the above-mentioned hypotheses, we subjected the mean reaction times to a series of
repeated measurements ANOVA, with nationality as a between-subjects factor (nationality—Czech vs.
Chinese) and one within-subjects factor (background—no background vs. topographic background;
symbol size—small vs. large; symbol shape—circle vs. diamonds or vs. triangles, etc.). Because of the
number of groups (2), post hoc tests were not performed.
As predicted, we found that respondents detected symbols on a blank background (M = 3.95 s)
significantly faster than symbols on a topographic background (M = 4.24 s), F(1,67) = 20.86, p < 0.001,
ηp2 = 0.24. We also found a significant between-subject effect in the Czech respondents, who were
faster than the Chinese, F(1,67) = 34.93, p < 0.001, ηp2 = 0.34. The effect of interaction between the
background type and nationality was not significant.
Furthermore, we found an expected significant relationship between the speed of detection of
small and large point symbols. Large point symbols (M = 3.83 s) were detected universally faster than
small point symbols (M = 4.74 s), F(1,67) = 130.24, p < 0.001, ηp2 = 0.66. Again, Czech respondents
detected the symbols faster than the Chinese, F(1,67) = 38.10, p < 0.001, ηp2 = 0.36. The effect of
interaction between symbol size and nationality was not significant.
Finally, we tested for the potential effects of symbol shape on the detection speed. The detection
of triangles was significantly faster in both background conditions (M = 3.30 s), followed by
circles (M = 3.64 s), and then diamonds (M = 5.35 s), F(1.5,100.8) = 300.1, p < 0.001, ηp2 = 0.82
(Greenhouse-Geisser correction was applied due to the violations of sphericity). Czech respondents
were significantly faster in detecting all point symbol shapes, F(1,67) = 34.93, p < 0.001, ηp2 = 0.34.
The effect of interaction between symbol shape and nationality was not significant.
7. Discussion and Conclusions
The results of the study show significant differences in the speed of perception of the investigated
graphic variables (size and shape) of basic geometric figures presented with and without map
backgrounds. Consistent with the first hypothesis (see Section 5.1), we found that the effect of a
topographic background on user performance significantly increased the time spent on the task.
This finding corresponds to the findings of Koláˇcný [34] in that reading speed significantly decreases
with increasing graphic density of the map (Figure 3).
In terms of point symbol size (second hypothesis), we concluded that increasing symbol size
brings a positive effect on lessening the time required to complete a task. Significant differences were
observed between the small (6–8 pixels) and large point symbols (11–13 pixels). As the small symbols
were close in size to the smallest possible symbol sizes identified by Koláˇcný [34], we concluded that
smaller sizes have only limited usability for application in cartographic visualization.
Furthermore, based on the results, we claimed that certain shapes (third hypothesis) do not
provide the same degree of comprehension, and therefore the shape itself affects the way how it
is perceived. This conclusion sheds new light on the results published in Koláˇcný [34], Garlandini
and Fabrikant [36] and others. This effect is demonstrated by statistically significant differences
ISPRS Int. J. Geo-Inf. 2018, 7, 427 11 of 15
between circles/triangles and diamonds. The significantly poorer results with diamonds may have
possibly been caused by the oblique effect mentioned by [29,30]. This finding is applicable in several
ways. For example, when maps are used in time-critical situations like human and natural disasters,
certain symbol shapes should be omitted or used only for the less important parts of the map content.
A brief, worldwide analysis of map legends used for emergency management was conducted earlier by
Dymon [61] and recently by Leitner [62]. Seven out of the nine analyzed map legends for emergency
management purposes used diamonds for important map content (e.g., incidents, hazards, and units).
According to our results, this might slow down the process of decision-making.
Another issue requiring discussion is the consistent differences in the speed of detecting target
point symbols between the Czech and Chinese participants. The fact that Czech participants were
universally faster in all conditions (our hypothesis was disproved) could be explained in several ways,
such as the presence of a method bias [63], differences in motivation, different instructions (with regard
to the task-solving speed), differential familiarity with this type of task, sample bias, or the presence of
an interviewer effect.
Alternatively, it could be explained by genuine differences in the performance or perceptual
style (e.g., attention to the field or field dependence; [54]). In this case, we would expect at least the
presence of the main effect in no background vs. with background conditions according to the holistic
vs. analytic attention paradigms. To be able to unequivocally interpret the differences in performance
of both groups as the existence of genuine differences in the cognitive style during point symbol
detection, more than two cultural groups would have to be compared in future research, with more
variable research samples used, and more individual (e.g., perceived social class) and group level (such
as affluence or religion) variables gathered and included in the statistical model.
The weaker performance of the Chinese participants sheds new light on the discussion about
the very small point symbols often used in Chinese city maps [44] (see Section 4). Our results do
not support the “trained eyesight” thesis assuming an increased visual acuity of Chinese people.
More likely, cultural factors like the discussed communication orientation and uncertainty avoidance,
among others, might play an important role. However, this research is still in an early stage and many
potential influencing factors have not yet been tested.
We are aware that there are several limitations of this study. At first, even if the number of the
participants is sufficient (see Section 5.3), there is still the issue of the specific user groups used for
the study. As most of the participants were relatively young university students, it is debatable if
the results are valid universally or only for a particular user group. Secondly, the majority of the
participants have a background in geography or related fields which might also have influenced
the results.
The presented study follows the research line focused on the cross-cultural differences in map
perception. This paper ties up with the recently published study by Stachoˇn et al. [43] in some respects.
We have also conducted an identification task on Czech/Chinese populations, but focused on different
characteristics of point symbols, i.e., three geometric shapes varied by size instead of several pictorial
symbols varied by degree of schematization. Despite these differences, the Czech participants were
faster than the Chinese participants in both studies. A possible reason for these results might be
differences in cognitive styles of the two tested groups. However, the framed-line test conducted by
Stachoˇn et al. [43] did not support this hypothesis. In the research presented here, no test assessing
cognitive style was included. Although the participating student groups were somewhat similar (age,
education, etc.), we do not have valid information about our groups’ cognitive styles. If we assume the
same cognitive style prevalence found in [43], i.e., Czechs are more holistic than Chinese, this again
would contradict our assumption that such identification tasks are completed faster by participants
with a more analytic cognitive style. However, the presence of a topographic background would
slow down holistic thinkers more, and indeed the Czech participants showed a stronger, although not
significant, decrease in the detection speed than the Chinese participants (but Czechs were still faster
than Chinese; see Figure 6). These contradictory results again show the need for a deeper scientific
ISPRS Int. J. Geo-Inf. 2018, 7, 427 12 of 15
investigation, not only focusing on the cross-cultural differences but also on the role of cognitive styles
in such tasks.
The obtained results support and extend the previous findings and also bring new directions for
future research. Interesting topics are, for example, the comparison of more complex shapes routinely
used in cartographic symbolization or the perception of cartographic information by people with
different ages, education or profession levels. We are also planning an extension of the experimental
sample to include worldwide participants for a better understanding of the identified differences.
Author Contributions: Conceptualization: S.A. and Z.Š.; methodology: S.A., Z.S. and ˇC.Š.; software: Z.S. and
ˇC.Š; validation: Z.S.; formal analysis: J. ˇC. and Z.S.; investigation: S.A., Z.S. and ˇC.Š.; resources: S.A. and Z.S.;
data curation: J. ˇC., Z.S. and ˇC.Š.; writing of the original draft preparation: S.A., J. ˇC., Z.S. and ˇC.Š.; writing of
review and editing: S.A., J. ˇC., P.K., Z.S. and ˇC.Š.; visualization: Z.Š. and M.B.; supervision: S.A. and Z.S.; project
administration: S.A. and Z.S.; funding acquisition: S.A., Z.S., ˇC.Š. and P.K.
Funding: This research was funded by Masaryk University, Czech Republic, Grant No. MUNI/M/0846/2015,
“Influence of cartographic visualization methods on the success of solving practical and educational spatial tasks”,
by the grant of the Ministry of Education, Youth and Sports of the Czech Republic, Grant No. LTACH-17002,
“Dynamic mapping methods oriented to risk and disaster management in the era of big data” and by the State
Key Laboratory for Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University,
China, Project No. 904090405, “Special Research Fund”. The APC was funded by Masaryk University, Brno,
Czech Republic and Wuhan University, Wuhan, PR China.
Acknowledgments: The authors would like to thank the following collaborators: Teng Fei (technical
coordinator, technical implementation), Xu Wang-Angsüsser (translations, test evaluation), Yi Lu (test evaluation,
test preparation and supervision), Guang Hu (technical implementation), and at the Laboratory Center (responsible
for the computer lab): Qinming Zhan (head), Xiuqin Lyu, Yongqiong Liu. The authors would also like to thank all
students who participated in this study.
Conflicts of Interest: The authors declare no conflicts of interest.
References
1. Bandrova, T.; Koneˇcný, M.; Zlatanova, S. Disaster management in practice—Concerning 5th ICC&GIS flood
and evacuation of 92 participants. In Proceedings of the 6th International Conference on Cartography & GIS,
Albena, Bulgaria, 13–17 June 2016; Bandrova, T., Koneˇcný, M., Eds.; Bulgarian Cartographic Association:
Sofia, Bulgaria, 2016; Volume 2, pp. 770–780. Available online: http://iccgis2016.cartography-gis.com/
iccgis2016/wp-content/uploads/2016/08/ICCGIS2016_PROCEEDINGS-HQ.pdf (accessed on 7 September
2018).
2. Black, J.; Arrowsmith, C.; Black, M.; Cartwright, W. Comparison of Techniques for Visualising Fire Behaviour.
Trans. GIS 2007, 11, 621–635. [CrossRef]
3. Bertin, J. Graphische Semiologie: Diagramme, Netze, Karten; Translated from the 2nd French Edition (1973);
Walter de Gruyter: Berlin, Germany, 1974; ISBN 3-11-003660-6.
4. Caivano, J.L. Visual texture as a semiotic system. Semiotica 1990, 80, 239–252. [CrossRef]
5. Carpendale, M.S.T. Considering Visual Variables as a Basis for Information Visualization, Research Report
2001-693-16; University of Calgary: Calgary, AB, Canada, 2003; Available online: http://innovis.cpsc.ucalgary.
ca/innovis/uploads/Publications/Publications/Carpendale-VisualVariablesInformationVisualization.
2003.pdf (accessed on 9 September 2018).
6. Halík, Ł. The analysis of visual variables for use in the cartographic design of point symbols for mobile
Augmented Reality applications. Geod. Cartogr. 2012, 61, 19–30. [CrossRef]
7. Keates, J.S. Cartographic Design and Production, 1st ed.; Longman Group: London, UK, 1973;
ISBN 0-582-48283-6.
8. MacEachren, A.M. How Maps Work: Representation, Visualization, and Design, 1st ed.; Guilford Press: New York,
NY, USA, 1995; ISBN 0-89862-589-0.
9. Morrison, J.L. A theoretical framework for cartographic generalization with the emphasis on the process of
symbolization. Int. Yearb. Cartogr. 1974, 14, 115–127.
10. Roth, R. Visual variables. In International Encyclopedia of Geography: People, the Earth, Environment and
Technology; Wiley: New York, NY, USA, 2017.
ISPRS Int. J. Geo-Inf. 2018, 7, 427 13 of 15
11. Spiess, E. Eigenschaften von Kombinationen graphischer Variablen. In Grundsatzfragen der Kartographie,
1st ed.; Arnberger, E., Ed.; Österreichische Geographische Gesellschaft: Vienna, Austria, 1970; pp. 280–293.
12. Swienty, O.; Zhang, M.; Reichenbacher, T.; Meng, L. Establishing a neurocognition-based taxonomy of
graphical variables for attention-guiding geovisualisation. In Proceedings of the SPIE-Geoinformatics 2007:
Cartographic Theory and Models, Nanjing, China, 3 August 2007; Li, M., Wang, J., Eds.; Volume 6751,
pp. 1–13.
13. Tyner, J.A. Principles of Map Design, 1st ed.; Guilford Press: New York, NY, USA, 2010; ISBN 978-1-60623-544-7.
14. Burian, J.; Brychtová, A.; Vávra, A.; Hladišová, B. Analytical Material for Planning in Olomouc,
Czech Republic. J. Maps 2016, 12, 649–654. [CrossRef]
15. Kessler, F.C.; Slocum, T.A. Analysis of thematic maps published in two geographical journals in the twentieth
century. Ann. Assoc. Am. Geogr. 2011, 101, 292–317. [CrossRef]
16. White, T.M.; Slocum, T.A.; McDermott, D. Trends and Issues in the Use of Quantitative Color Schemes in
Refereed Journals. Ann. Am. Assoc. Geogr. 2017, 107, 829–848. [CrossRef]
17. Wood, D. Now and then: Comparisons of ordinary Americans’ symbol conventions with those of past
cartographers. Prologue 1977, 9, 151–161. Available online: https://babel.hathitrust.org/cgi/pt?id=mdp.
39015014516770;view=1up;seq=183 (accessed on 9 September 2018).
18. MacEachren, A.M. (Re)Considering Bertin in the Age of Big Data and Visual Analytics. Presented at the
28th International Cartographic Conference, Washington DC, USA, 2–7 July 2017.
19. Christophe, S.; Hoarau, C. Expressive Map Design Based on Pop Art: Revisit of Semiology of Graphics?
Cartogr. Perspect. 2012, 73, 61–74. Available online: https://www.researchgate.net/publication/257935475_
Expressive_Map_Design_Based_on_Pop_Art_Revisit_of_Semiology_of_Graphics (accessed on 9 September
2018). [CrossRef]
20. Angsüsser, S. Kartenicons im Interkulturellen Vergleich. Kartosemiotische Untersuchung Anhand
Gedruckter Deutscher Und Chinesischer Stadtpläne. Ph.D. Thesis, Technische Universität München,
München, Germany, 2011. Available online: http://mediatum.ub.tum.de/doc/1084383/1084383.pdf
(accessed on 9 September 2018).
21. Robinson, A.H.; Morrison, J.L.; Muehrcke, P.C.; Kimerling, A.J.; Guptill, S.C. Elements of Cartography, 6th ed.;
John Wiley & Sons: New York, NY, USA, 1995; ISBN 0-471-55579-7.
22. Slocum, T.A.; McMaster, R.B.; Kessler, F.C.; Howard, H.H. Thematic Cartography and Geovisualization, 3rd ed.;
Pearson/Prentice Hall: Upper Saddle River, NJ, USA, 2009; ISBN 978-0-13-229834-6.
23. Gescheider, G. Psychophysics: The Fundamentals, 3rd ed.; Lawrence Erlbaum Associates: Mahwah, NJ, USA,
1997; ISBN 0-8058-2281-X.
24. Pinna, B. New Gestalt Principles of Perceptual Organization: An Extension from Grouping to Shape
and Meaning. Gestalt Theory 2010, 32, 11–78. Available online: https://pdfs.semanticscholar.org/f54b/
87e262bae5daeeab0cc5a17a36318af4537c.pdf (accessed on 9 September 2018).
25. Carvalho, G.A.; Moura, A.C.M. Applying Gestalt Theories and Graphical Semiology as Visual Reading
Systems Supporting Thematic Cartography. In Proceedings of the 24th International Cartographic
Conference, Santiago, Chile, 15–21 November 2009; Available online: http://icaci.org/files/documents/
ICC_proceedings/ICC2009/html/refer/20_4.pdf (accessed on 9 September 2018).
26. Wagemans, J.; Elder, J.H.; Kubovy, M.; Palmer, S.E.; Peterson, M.A.; Singh, M.; von der Heydt, R. A century of
Gestalt psychology in visual perception: I. Perceptual grouping and figure-ground organization. Psychol. Bull.
2012, 138, 1172–1217. [CrossRef] [PubMed]
27. Beck, D.M.; Palmer, S.E. Top-down influences on perceptual grouping. J. Exp. Psychol. Hum. 2002, 28,
1071–1084. [CrossRef]
28. Neisser, U. Cognitive Psychology, 1st ed.; Appleton-Century-Crofts: New York, NY, USA, 1967;
ISBN 978-0131396678.
29. Cavanagh, P.; Arguin, M.; Treisman, A. Effect of surface medium on visual search for orientation and size
features. J. Exp. Psychol. Hum. 1990, 16, 479–491. [CrossRef]
30. Treisman, A.; Gormican, S. Feature analysis in early vision: evidence from search asymmetries. Psychol. Rev.
1988, 95, 15–48. [CrossRef] [PubMed]
31. Brunswik, E. Representative design and probabilistic theory in a functional psychology. Psychol. Rev. 1955,
62, 193–217. [CrossRef] [PubMed]
ISPRS Int. J. Geo-Inf. 2018, 7, 427 14 of 15
32. Dhami, M.K.; Hertwig, R.; Hoffrage, U. The Role of Representative Design in an Ecological Approach to
Cognition. Psychol. Bull. 2004, 130, 959–988. [CrossRef] [PubMed]
33. Montello, D.R. Cognitive Map-Design Research in the Twentieth Century: Theoretical and Empirical
Approaches. Cartogr. Geogr. Inf. Sci. 2002, 29, 283–304. [CrossRef]
34. Koláˇcný, A. Utilitární kartografie, cesta k optimální úˇcinnosti kartografické informace. Geod. Kartogr. Obz.
1969, 15, 301–308.
35. Deeb, R.; De Maeyer, P.; Ooms, K. The influence of Jacques Bertin. In Proceedings
of the 25th International Cartographic Conference, Paris, France, 3–8 July 2011; Available
online: http://icaci.org/files/documents/ICC_proceedings/ICC2011/Oral%20Presentations%20PDF/B1Graphical%20Semiology,%20visual%20variables/CO-081.pdf
(accessed on 9 September 2018).
36. Garlandini, S.; Fabrikant, S.I. Evaluating the Effectiveness and Efficiency of Visual Variables for Geographic
Information Visualization. In Proceedings of the International Conference on Spatial Information Theory
2009, Aber Wrac’h, France, 21–25 September 2009; Volume 5756, pp. 195–211.
37. Reimer, A. Squaring the Circle? Bivariate Colour maps and Jacques Bertin’s concept of ‘Disassociation’.
In Proceedings of the 25th International Cartographic Conference, Paris, France, 3–8 July 2011; Available
online: http://icaci.org/files/documents/ICC_proceedings/ICC2011/Oral%20Presentations%20PDF/A4Jacques%20Bertin%20and%20graphic%20semiology%202/CO-054.pdf
(accessed on 9 September 2018).
38. Rundstrom, R.A. Introduction. Cartographica 1993, 30, 32–37.
39. Harley, J.B.; Blakemore, M.J. Cultural Meaning: The Iconography of Maps. Cartographica 1980, 17, 76–86.
40. Wood, D. Cultured symbols: Thoughts on the cultural context of cartographic symbols. Cartographica 1984,
21, 9–37. [CrossRef]
41. Wood, D.; Fels, J. Designs on Signs/Myth and Meaning in Maps. Cartographica 1986, 23, 54–103. [CrossRef]
42. Tang, X.; Hurni, L. Regional Spatial Planning Maps: A Sino-Swiss Comparison of Cartographic Visualization
Methodologies. In Proceedings of the 24th International Cartographic Conference, Santiago, Chile,
15–21 November 2009; Available online: https://icaci.org/files/documents/ICC_proceedings/ICC2009/
html/nonref/22_2.pdf (accessed on 9 September 2018).
43. Stachoˇn, Z.; Šašinka, ˇC.; ˇCenˇek, J.; Štˇerba, Z.; Angsuesser, S.; Fabrikant, S.I.; Štampach, R.; Morong, K.
Cross-cultural differences in figure–ground perception of cartographic stimuli. Cartogr. Geogr. Inf. Sci. 2018.
[CrossRef]
44. Angsüsser, S. Possible Reasons for Size Differences Between Point Symbols in German and Chinese City Maps.
In Proceedings of the 5th International Conference on Cartography & GIS, Riviera, Bulgaria, 15–20 June 2014;
Bandrova, T., Konecny, M., Eds.; Bulgarian Cartographic Association: Sofia, Bulgaria, 2014; Volume 1,
pp. 268–277. Available online: https://cartography-gis.com/docsbca/5ICCandGIS_Proceedings.pdf
(accessed on 9 September 2018).
45. Carlson, E.R. Generality of order of concept attainment. Psychol. Rep. 1962, 10, 375–380. [CrossRef]
46. Leifer, A. Ethnic Patterns in Cognitive Tasks. Ph.D. Thesis, Yeshiva University, New York, NY, USA, 1972.
47. Lesser, G.S.; Fifer, G.; Clark, D.H. Mental abilities of children from different social class and cultural groups.
Monogr Soc. Res. Child Dev. 1965, 30, 1–115. [CrossRef] [PubMed]
48. Lynn, R.; Pagliari, C.; Chan, J. Intelligence in Hong Kong measured for Spearman’s g and the visuospatial
and verbal primaries. Intelligence 1988, 12, 423–433. [CrossRef]
49. Lynn, R. Intelligence in China. Soc. Behav. Personal. 1991, 19, 1–4. [CrossRef]
50. Jensen, A.R.; Whang, P.A. Speed of accessing arithmetic facts in long-term memory: a comparison of
Chinese-American and Anglo-American children. Contemp. Educ. Psychol. 1994, 19, 1–12. [CrossRef]
51. Choong, Y.; Salvendy, G. Design of Icons for Use by Chinese in Mainland China. Interact. Comput. 1998, 9,
417–430. [CrossRef]
52. Hsu, S.-H.; Huang, K.-C. Effects of minimal legible size characters on Chinese word recognition. Visible
Lang. 2001, 35, 178–191. Available online: https://s3-us-west-2.amazonaws.com/visiblelanguage/pdf/35.2/
effects-of-minimal-legible-size-characters-on-chinese-word-recognition.pdf (accessed on 9 September 2018).
53. Mo, C.; Yu, M.; Seger, C.; Mo, L. Holistic neural coding of Chinese character forms in bilateral ventral visual
system. Brain Lang. 2015, 141, 28–34. [CrossRef] [PubMed]
54. Nisbett, R.E.; Peng, K.; Choi, I.; Norenzayan, A. Culture and systems of Thought: Holistic Versus Analytic
Cognition. Psychol. Rev. 2001, 108, 291–310. [CrossRef] [PubMed]
ISPRS Int. J. Geo-Inf. 2018, 7, 427 15 of 15
55. Nisbett, R.E. The Geography of Thought. How Asians and Westerners Think Differently . . . and Why, 1st ed.;
The Free Press: New York, NY, USA, 2003; ISBN 0-7432-1646-6.
56. Hofstede, G. Dimensions of national cultures in fifty countries and three regions. In Expiscations in
Cross-Cultural Psychology, 1st ed.; Deregowski, J.B., Dziurawiec, S., Annis, R.C., Eds.; Swets and Zeitlinger:
Lisse, The Netherlands, 1983; pp. 335–355.
57. Hofstede, G.; Hofstede, G.J.; Minkov, M. Cultures and Organizations—Software of the Mind. Intercultural
Cooperation and Its Importance for Survival, 3rd ed.; McGraw-Hill: New York, NY, USA, 2010;
ISBN 978-0071664189.
58. Faul, F.; Erdfelder, E.; Lang, A.-G.; Buchner, A. G*Power 3: A flexible statistical power analysis program for
the social, behavioral, and biomedical sciences. Behav. Res. Methods 2007, 39, 175–191. [CrossRef] [PubMed]
59. Šašinka, ˇC.; Morong, K.; Stachoˇn, Z. The Hypothesis Platform: An Online Tool for Experimental Research
into Work with Maps and Behavior in Electronic Environments. ISPRS Int. Geo-Inf. 2017, 6, 407. [CrossRef]
60. Popelka, S.; Stachoˇn, Z.; Šašinka, ˇC.; Doležalová, J. Eyetribe Tracker Data Accuracy Evaluation and Its
Interconnection with Hypothesis Software for Cartographic Purposes. Comput. Intell. Neurosci. 2016, 2016,
9172506. [CrossRef] [PubMed]
61. Dymon, U. An analysis of emergency map symbology. Int. J. Emerg. Manag. 2003, 1, 227–237. [CrossRef]
62. Leitner, F. Analýza Využitelnosti Geoinformací ze Zdroj ˚u Typu Big data v Procesech Krizového ˇRízení.
Ph.D. Thesis, Masaryk University, Brno, Czech Republic, 2018. Available online: https://is.muni.cz/th/
pxb9l/BP_Leitner_qllmmeem.pdf (accessed on 9 September 2018).
63. Van de Vijver, F.; Tanzer, N.K. Bias and equivalence in cross-cultural assessment: an overview. Eur. Rev.
Appl. Psychol. 2004, 54, 119–135. [CrossRef]
© 2018 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access
article distributed under the terms and conditions of the Creative Commons Attribution
(CC BY) license (http://creativecommons.org/licenses/by/4.0/).
RESEARCH ARTICLE
A comparison of the performance on extrinsic
and intrinsic cartographic visualizations
through correctness, response time and
cognitive processing
en k Sˇ asˇinka1
, Zden k Stacho ID
1,2
, Ji ı´ en k1
, Alzˇb ta Sˇ asˇinkova´1
,
Stanislav PopelkaID
3
, Pavel Ugwitz1
, David LackoID
1
1 Department of Information and Library Studies, Faculty of Arts, Masaryk University, Brno, Czech Republic,
2 Department of Geography, Faculty of Science, Masaryk University, Brno, Czech Republic, 3 Department of
Geoinformatics, Faculty of Science, Palacky´ University Olomouc, Olomouc, Czech Republic
* zstachon@geogr.muni.cz
Abstract
The aim of this study was to compare the performance of two bivariate visualizations by
measuring response correctness (error rate) and response time, and to identify the differences
in cognitive processes involved in map-reading tasks by using eye-tracking methods.
The present study is based on our previous research and the hypothesis that the use of different
visualization methods may lead to significant cognitive-processing differences. We
applied extrinsic and intrinsic visualizations in the study. Participants in the experiment were
presented maps which depicted two variables (soil moisture and soil depth) and asked to
identify the areas which displayed either a single condition (e.g., “find an area with low soil
depth”) or both conditions (e.g., “find an area with high soil moisture and low soil depth”).
The research sample was composed of 31 social sciences and humanities university students.
The experiment was performed under laboratory conditions, and Hypothesis software
was used for data collection. Eye-tracking data were collected for 23 of the
participants. An SMI RED-m eye-tracker was used to determine whether either of the two
visualization methods was more efficient for solving the given map-reading tasks. Our
results showed that with the intrinsic visualization method, the participants spent significantly
more time with the map legend. This result suggests that extrinsic and intrinsic visualizations
induce different cognitive processes. The intrinsic method was observed to
generally require more time and led to higher error rates. In summary, the extrinsic method
was found to be more efficient than the intrinsic method, although the difference was less
pronounced in the tasks which contained two variables, which proved to be better suited to
intrinsic visualization.
PLOS ONE
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 1 / 23
a1111111111
a1111111111
a1111111111
a1111111111
a1111111111
Citation: Sˇasˇinka , Stacho Z, en k J, Sˇasˇinkova´
A, Popelka S, Ugwitz P, et al. (2021) A comparison
of the performance on extrinsic and intrinsic
cartographic visualizations through correctness,
response time and cognitive processing. PLoS
ONE 16(4): e0250164. https://doi.org/10.1371/
journal.pone.0250164
Editor: Song Gao, University of Wisconsin
Madison, UNITED STATES
Received: July 19, 2020
Accepted: April 1, 2021
Published: April 21, 2021
Copyright: 2021 Sˇasˇinka et al. This is an open
access article distributed under the terms of the
Creative Commons Attribution License, which
permits unrestricted use, distribution, and
reproduction in any medium, provided the original
author and source are credited.
Data Availability Statement: All relevant data are
within the manuscript, its Supporting Information
files, and in the Open Science Framework database
(osf.io/2t4ms/).
Funding: This publication was supported by the
Czech Science Foundation (GC19-09265J: The
influence of socio-cultural factors and writing
system on perception and cognition of complex
visual stimuli). This work was supported by the
research infrastructure HUME Lab Experimental
Introduction
The awareness that maps serve as tools for the creation of mental representations of the world
and cannot therefore be considered transparent or direct depictions has long been discussed
in cartography [1]. As a research topic, the cognition of maps is rooted in the early twentieth
century [2]. A key question is how a particular form of cartographic visualization affects the
effectiveness of cartographic communication [3–5]. The same data can be represented by different
cartographic visualization methods. An unsuitable method not only reduces performance
but also places various requirements which correspond to the type of cartographic
visualization on different types of users and tasks [6, 7]. User characteristics (such as cartographic
skills, [8, 9]) and the type of task [10, 11] must therefore be considered when we conduct
empirical studies on the performance of alternative visualizations.
The primary aim of the present study was an empirical and objective comparison of two
alternative bivariate visualizations (Fig 1) to assess the performance of a selected population
which possessed a basic level of cartographic skill [12–16] through two different types of task.
Another aim was to understand the cognitive processes which underlie the potential differences
in objective performance [17–20].
Olson [21] stressed that maps are considered highly valuable visual stimuli in experimental
psychology since the variables they represent can be accurately controlled. The manner of presenting
geographic information can have a significant effect on user cognitive processing
(internal mental processes) during map-related tasks. Larkin and Simon [22] presented the
concept of informational and computational equivalence and argued that different visualizations
can be informationally equivalent if all the information available in one of them is available
in the other, and vice versa. The establishment of informational equivalence between
bivariate cartographic visualizations permits us to investigate the extent of computational
equivalence between the two.
Cartographic visualization offers numerous methods of presenting geographical data.
These methods differ in their ability to visualize certain data types, the level of detail they provide,
and the number of variables they simultaneously portray [23]. The graphic display of
multiple geographic phenomena is known as multivariate mapping [24], and its purpose is to
investigate the relationships between the given phenomena. Bivariate maps encode two separate
variables simultaneously [25]. Bivariate mapping can be further divided into extrinsic (the
variables carrying the information are visually separable) and intrinsic (the variables are visually
inseparable [26]).
The present study applies both extrinsic and intrinsic bivariate encoding of geographic variables
(Fig 1) to investigate the cognitive processes of map users.
The extrinsic bivariate method employs two visually distinct variables (the differences may
represent, for example, size, shape or color lightness) to display two different geographical phenomena,
such as soil depth and moisture. In the present study, each of the two phenomena
had three levels of intensity (low, medium and high), which provided a total of six options in
the map legend (Fig 1, EN1 left). From the map legend, the map users were required to create
a mental representation of nine possible combinations (Fig 1, EN2 center). Intrinsic bivariate
visualizations apply visual variables which are visually inseparable (typically, the visual variables
include hue, color lightness and opacity), resulting in a map legend comprising nine
combinations (Fig 1, IN right). In this latter case, the map legend was identical to the mental
representation of all the possible combinations. Although, color lightness, hue and opacity are
considered to interact with each other in the psychology of perception [27–29], cartography
regards them as mutually independent entities [30]. Therefore, in cartography, these parameters
are used as independent visual variables.
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 2 / 23
Humanities Laboratory, Faculty of Arts, Masaryk
University.
Competing interests: The authors have declared
that no competing interests exist.
Each visualization type can be expected to induce a different cognitive and perceptual load
on the user [31–33]. The differences in cognitive processing relate to selective attention theory,
which specifies that only a limited number of elements can be processed at one time [34]. The
perception aspect can be explained according to pre-attentive visual processing theory [35,
36]. Some visual elements, designated pre-attentive, can be detected in a single glance and
thereby serve as the central components of a visualization. In map reading, pre-attentive elements
can aid in identifying boundaries and detecting the presence or absence of other elements;
for example, size (an extrinsic variable) is pre-attentive, while lightness (an intrinsic
variable) cannot be considered a pre-attentive feature. Since the processing of extrinsic and
intrinsic visual elements is not only based on perception but involves a broader cognitive context,
it appears reasonable to assume that the situation will be more complex when both extrinsic
and intrinsic visual variables are employed.
Bivariate mapping and the use of various visual variable combinations have been the subject
of numerous research studies [e.g., 21, 37–42]. Elmer conducted an extensive comparison of
Fig 1. Examples of extrinsic and intrinsic bivariate encoding of geographic variables. EN1: extrinsic (separable) encoding of variables according to size and
color lightness; EN2: mental representation of extrinsic visualization (all the possibilities); IN: intrinsic (inseparable) encoding of variables according to hue and
lightness.
https://doi.org/10.1371/journal.pone.0250164.g001
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 3 / 23
visual variable combinations [43]. Kunz studied the use of bivariate visualization methods
(extrinsic and intrinsic) to produce visualizations of natural hazards (avalanches) and the levels
of uncertainty in the presented data (avalanche hazard prediction) [44]. Sˇasˇinka et al. investigated
the differences in processing intrinsic and extrinsic visualizations, focusing mainly on
cognitive style and map reading skills [45]. The results of the study (and of related eye-tracking
studies) revealed significant differences between extrinsic and intrinsic visualizations associated
with both map-reading performance and task processing. Although the aim of the study
was not to compare methods of visualization, the results showed that a group of laypersons
(psychology students) worked more effectively with the extrinsic method, while participants
with better map reading skills (cartography students) demonstrated better performance using
intrinsic visualization.
However, the authors noted an important limitation in their study, consisting in a relatively
sophisticated topic (avalanche risk and its uncertainty) which the participants (especially psychology
students) may have found difficult to understand.
The present study was designed based on the results of the above studies. The stimulus
material included two bivariate maps with the same content; an example is shown in Fig 2. We
selected soil depth and soil moisture as suitable phenomena for depiction since they are considered
generally comprehensible and quantifiable. Information gathered from volunteers
informed our selection of the topic during the experiment design process. It was critical that
participants intuitively understood the relationship between the visual variables and the
depicted phenomena in the legend’s design. We followed the principle of cultural metaphors
and applied a visual representation of the data to match the metaphors which aid conceptual
thinking [6, 46]. We used a combination of color and size for extrinsic visualizations and different
colors for intrinsic visualizations. Fig 2 illustrates examples.
Methods
To compare the performance of working with maps which use different cartographic visualizations,
we conducted an experiment with two tests (one for each of the two selected methods of
visualization); for more details of the research design, see Fig 5. We applied a combination of
confirmatory and exploratory data analysis methods [45, 47, 48].
The confirmatory analysis tested our hypotheses on the differences between extrinsic and
intrinsic visualizations in map reading performance. The data collected were response time
and correctness (as investigated by Elmer [43]). Several evaluation methods and concepts
allow the measurement of user performance with an information system [49]. The most common
parameters are effectiveness and efficiency. According to ISO 9241–11 [50], effectiveness
is defined as the “accuracy and completeness with which users achieve specified goals”, and
efficiency corresponds to the necessary resources (e.g., time) to achieve a desired result. We
Fig 2. Examples of extrinsic visualization (left) and intrinsic visualization (right) used in the study. Legends for
both extrinsic and intrinsic visualization are also given. Areas with identical values are depicted with two different
encoding systems to enable a visual comparison of the differences between each visualization.
https://doi.org/10.1371/journal.pone.0250164.g002
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 4 / 23
calculated effectiveness as the rate of correctness and efficiency as the task completion time
[51]. The aim of the exploratory analysis of the eye-tracking data [52] was to gain deeper
insight into the differences between the visualizations at the level of individual elements and to
employ eye-tracking as a means of collecting objective data [53–55]. Eye-tracking is a valuable
tool for studying eye behavior which occurs during map reading since it provides objective
measurement of the visual strategies employed by map readers. The review article from Krassanakis
and Cybulski [56] provides an overview of existing eye-tracking studies which have
appeared in cartographic research over the last decade. The review showed that cartographers
used eye tracking mainly in the evaluation of cartographic symbolization and design
principles.
Map and items design
The task layout was identical in both tests (Figs 3 and 4): instructions for the tasks were displayed
in the upper area of the screen, the map legend was at the right, and the visual field of
the map was in the center. The lower area of the screen displayed a button bar with four possible
selections for the correct answer. The participants selected an area which satisfied the given
condition (e.g., “Find the area with low soil moisture.”). In subtest A (Fig 3), the marked areas
covered four square units; in subtest B, the marked area only covered one square unit (Fig 4).
To answer the questions, participants were required to click on the correct button. Only one
correct answer was possible.
We generated the visualization using ArcMap (version 10.7) using the color schemas from
ColorBrewer 2.0 [57]. The extrinsic visualization used three circle sizes (6, 10 and 14 pts;
#deebf7) to indicate soil moisture, and three color classes (#fee8c8, #fdbb84 and #e34a33) to
indicate soil depth. The colors were selected to suit a realistic representation of the phenomena
as they occurred in reality, such as blue for moisture and brown for soil depth. Three colors
were used to create the intrinsic visualization (A: #e0f3db, #a8ddb5, #43a2ca; B: #e0ecf4,
#9ebcda, #8856a7; C: #fee8c8, #fdbb84, #e34a33). A brown color scheme was used to indicate
dry areas, and green-blue was used to indicate wet areas. Soil depth was indicated using a geographical
principle, darker shades representing greater depth. All colors had a transparency of
40% to allow the base map to be visible. To create the base map, OpenStreetMap data was used
[58].
Procedure
The study was designed to illustrate the effect of various types of task. As mentioned in the
introduction, we evaluated the maps / visualizations according to their purpose. We therefore
designed the study to depict two types of phenomena. In the first scenario, the aim was to
answer a question which related to only one variable (either soil moisture or soil depth). In the
second scenario, participants were required to think about both phenomena in parallel. We
assumed that the extrinsic method would be more suitable for an isolated assessment of phenomena
because of its properties (both variables are presented separately through different
visual qualities). The intrinsic method, however, is relatively more suitable for tasks which
involve a unified search. Another reason for diversifying the task types (division into subtests 1
and 2) was to produce greater informative value and reliability in the achieved results. If performance
of the extrinsic method possessed greater stability for each of the types in all tasks,
the assumption that this method produces better results from the examined lay population
would be more strongly supported.
The test involved a total of 30 items. The first parts of each subtest (A.1 and B.1) contained
six items which focused on a single phenomenon (Fig 3). The items covered six possible
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 5 / 23
options: low, medium and high soil moisture, and low, medium and high soil depth. In the second
parts of each subtest (A.2 and B.2), participants were asked questions about the two phenomena
in each item item (Fig 4), with both A.2 and B.2 covering nine options (low moisture
and low soil depth, low moisture and medium soil depth, etc.). We employed a between-subject
design (Fig 5) to eliminate the effect of interference caused by experience with the given
type of task.
Each participant completed an informed consent form, received a financial reward and was
randomly allocated to one of the two research groups. Before the experiment, they were
informed about the expected duration of the tests and given the opportunity to ask the experimenter
questions. The instructions required the participants to work without interruption
during the assessed part of the experiment. No participant required any additional explanation,
and during a brief follow-up inquiry, no participant reported any problem in comprehending
the content of the tasks. The participants received feedback on the correctness of their
Fig 3. Example of an intrinsic visualization item (subtest A–part A.1.). The task was to select the area which contained “medium soil depth”; the correct answer was
area No. 1.
https://doi.org/10.1371/journal.pone.0250164.g003
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 6 / 23
responses for the two sample items (one sample item was presented at the beginning of each
subtest, A and B). No feedback was given during the assessed component of the tests. Each test
item was preceded with a fixation cross displayed for 500 ms in the same position each time in
the upper area of the screen.
Apparatus
The test was administered using a DELL Precision M4800 notebook with a 2200
, 60 Hz AOC
E2260P external monitor. The resolution was set at 4:3 (1024 x 768) to correspond exactly to
the stimuli (Figs 3 and 4). The participants used a mouse to select their answers. The experimenter
was present throughout the experiment to monitor its course. Mounted to the monitor
was a remote SMI RED-m eye-tracker with a sampling rate of 60 Hz to collect eye-tracking
data. Eye-tracking data collection, calibration and validation was done using the SMI Experiment
Center 3.7 software. The calibration procedure was only considered satisfactory when
Fig 4. Example of an extrinsic visualization item (subtest B—part B.2). The task was to select the area which best satisfied the conditions of “low soil depth” and
“medium soil moisture”; the correct answer was area No. 3.
https://doi.org/10.1371/journal.pone.0250164.g004
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 7 / 23
the values returned by the eye-tracker were within 0.5˚. The experiment was administered
using the Hypothesis software tool [45, 53] (a web-based tool used in research and psychological
diagnostics [59]). The behavioral raw data were exported from Hypothesis in “.xlsx” format
and then processed using R (version 4.0.0) with the “rstatix” [60], “rcompanion” [61] and
“multicon” [62] packages. Because of the relatively small sample size, we incorporated several
specific procedures in our analyses. First, we used non-parametric statistical tests (i.e., Wilcoxon’s
rank-sum test for independent samples and Wilcoxon’s signed-rank test for paired samples),
which do not require Gaussian data distribution and can process potential outliers.
Second, we reported not only the related effect sizes (i.e., rank-biserial correlation; r) but also
their 95% confidence intervals (CIs), which were computed on the basis of 10,000 bootstraps.
Third, we computed 95% CIs for the descriptive statistics of means and medians. This step
gave us deeper insight into the obtained results, especially with respect to the small sample size
since CIs tend to be very wide in small samples, and therefore for reliability, any potential significant
differences should not be permitted to overlap. Eye-tracking data were imported into
the OGAMA 5.0 software and paired with the behavioral data via HypOgama [53]. The fixations
were calculated using the I-DT model with the parameters set to the following values (as
recommended by Popelka et al. [53]): maximum distance = 20 px, minimum number of samples
= 5; “do not merge consecutive fixations”.
Participants
The Research Ethics Committee of Masaryk University approved this project (No.: 0257/
2018). Participants were recruited via social networks and each signed an informed consent
form. They received a financial reward (approx. 8 euros) for participation in the study.
The research sample was composed of 31 students (8 males and 23 females), aged between
19 and 28 (m = 21.8, med = 21). The sample was randomly divided into an “intrinsic” group
and an “extrinsic” group (block randomization was used). The former (intrinsic) group consisted
of 15 students (2 males and 13 females; m = 21.4). The extrinsic group consisted of 16
students, 6 males and 10 females (m = 22.3). All the participants were students of social sciences
and humanities (Faculty of Arts or Faculty of Social Studies) at Masaryk University. Students
of geography and related fields were excluded from the study.
Fig 5. Between-subject experimental design (subtests A and B; parts A.1, A.2, B.1 and B.2). Both independent experimental groups, Intrinsic and Extrinsic, performed
the test in exactly the same manner. The order of all items was constant for both groups, and all participants.
https://doi.org/10.1371/journal.pone.0250164.g005
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 8 / 23
The eye-tracking part of the study yielded 23 datasets; for the remainder of the participants
(8), no data were recorded during the session for technical reasons. The data were from 4
males and 19 females, aged between 19 and 28 (m = 22.22, med = 22). The “extrinsic” group
was composed of 12 students; the “intrinsic” group consisted of 11 students.
After completing the experiment, we performed a quality check of the eye-tracking data.
The total data loss was 2.65% for the extrinsic group and 4.1% for the intrinsic group. All items
with a dropout rate of above 10% were excluded from the analysis: this was 22 data points (out
of a total 330 data points) in the case of the intrinsic method and 6 data points (out of a total
360 data points) in the case of the extrinsic method. No participant was excluded completely
(because of a high dropout rate throughout the test).
Results
We used several metrics which employ extrinsic and intrinsic methods of visualization to evaluate
the differences between the groups in participant performance. We examined both behavioral
(correctness, response time) and eye-tracking (dwell time, direct saccades) metrics. Details of the
metrics calculations are specified in the respective section of the Results chapter. Non-parametric
statistics were used to calculate the differences between and within the groups. A Wilcoxon ranksum
test was applied to compare independent groups (i.e., extrinsic vs. intrinsic); a Wilcoxon
signed-rank test for dependent samples was used to compare performance between subtests.
Effect size (r) was calculated for all results to determine the size of the differences [63].
A post-hoc sensitivity analysis of the differences between two independent means according
to GÃ
Power [64] (1- = 0.80, = 0.05, n1 = 16, n2 = 15, two-tailed) showed that with the given
sample, we would only be able to detect medium to large effect sizes with differences between the
two groups greater than a standard deviation of 1 (non-centrality parameter = 2.899, critical
t = 2.8987, df = 29, d = 1.042). We therefore did not interpret any results with small effect sizes.
Split-half reliability coefficients performed on two random halves and adjusted with the
Spearman-Brown prophecy formula were also calculated for each subtest. The results indicated
that all the task subtests were reliable (mean of the split-half correlations for A1 = 0.847,
A2 = 0.835, B1 = 0.889, and B2 = 0.736).
Correctness
Response correctness was one of the key parameters observed in the map-related tasks. Using
the Wilcoxon rank-sum test, we compared the overall correctness of the responses related to
the extrinsic and intrinsic groups. The extrinsic visualization showed a significantly higher
overall correctness (N = 16, 96.3%) than the intrinsic visualization (N = 15, 90.0%), with a
moderate effect size (Z = 173, p = 0.031, r = 0.390 [95% CI: .059, .656]). The results are charted
in Fig 6.
We also investigated incorrect responses to explore the error rate at the level of individual
items (i.e., the distractors selected). Particular attention was given to items with a significant
difference between the two visualizations, namely items No. 1, 2, 9, 21 and 28 (Fig 7). In the
case of all items with the exception of No. 2, the intrinsic method was associated with higher
error rates (item No. 2 showed a reverse scenario). A plausible explanation of the above phenomenon
was identified only with respect to item No. 21 (Fig 7). The item required the participants
to select the area with the lowest soil depth. The correct answer was unit No. 1 (lowest
soil depth/highest moisture). In the intrinsic visualization, participants tended to select unit
No. 3 (medium soil depth/medium soil moisture), which can likely be explained by unit No. 3
being surrounded by a darker color and thus appearing lighter (see [65–67]) and could
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 9 / 23
therefore have been misinterpreted as the neighboring value (lowest soil depth/medium soil
moisture). We identified no other trends.
Response time
For a comparison of processing speeds (response times; RTs), we applied the Wilcoxon ranksum
test. For each subtest, we performed a separate univariate outlier analysis. The analysis
revealed three cases of extremely long and irregular response times (over 20,000 ms, different
participants) and were excluded from further analysis. However, reaction times are usually distributed
ex-Gaussian and demonstrate a rapid rise on the left and have a long positive tail on
the right [68, 69]; the traditional outlier detections (e.g., ± 2 SD or 1.5 IQR) are therefore not
recommended [70] since these extreme values should not be understood as outliers. Hence, we
decided to keep the remainder of the outliers and applied non-parametric statistical analyses
Fig 6. Response correctness for the entire test. Correctness was calculated as a ratio of the number of correct answers to the number of all answers.
https://doi.org/10.1371/journal.pone.0250164.g006
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 10 / 23
instead. The response time analysis therefore covered both correct and incorrect answers. The
total response time was significantly less for the extrinsic method (N = 16, median = 6.494 ms
[95% CI: 5536, 8726]) than for the intrinsic method (N = 15, median = 10.217 ms [95% CI:
9588, 11597]), with a large effect size (Z = 12, p < 0.001, r = -0.767 [95% CI: -0.844, -0.607]; see
Fig 8 and Table 1).
The same pattern was observed in a comparison of the RTs of individual subtests. The
extrinsic stimuli consistently indicated lower RTs than the intrinsic stimuli. We identified the
largest differences between visualizations in parts A1 and B1; the differences between visualizations
in parts A2 and B2 were moderate. All the differences, with the exception of those related
to A2, were significant at a significance level of 5% (Table 1). All the differences, with the
exception of those related to A2 and B2, yielded large effect sizes; we also observed large gaps
in the upper bounds in the confidence intervals of the extrinsic group and the lower bounds of
the confidence intervals in the intrinsic group, suggesting that the obtained statistically significant
results were reliable.
At the individual subtest levels (A1, A2, B1, B2), we examined the differences between the
test items with one and two variables using the Wilcoxon signed-rank test. An exploration of
response times at the subtest level revealed an interesting pattern (Fig 9). In the extrinsic “A”
levels, A2 (two variables) resulted in significantly longer response times than A1 (one variable),
with a large effect size (Z = 7, p < 0.001, r = -0.789 [95% CI: -0.880, -0.558]). We observed a
similar effect in relation to the extrinsic “B” levels, where B2 showed significantly longer
response times than B1 (Z = 0, p < 0.001, r = -0.880 [95% CI: -0.882, -0.879]). However, we
noted an inverse pattern in relation to the intrinsic visualizations, where A1 (one variable)
received significantly longer response times than A2 (large effect size; Z = 68, p = 0.021,
r = 0.655 [95% CI: 0.227, 0.886]), and similarly, B1 resulted in significantly longer response
Fig 7. Error rate per item. INT–red/yellow, EXT–blue. Particular attention was given to items with a significant difference between the two visualizations (1, 2, 9, 21, 28).
The error rate was calculated as a percentage of incorrect answers of all answers.
https://doi.org/10.1371/journal.pone.0250164.g007
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 11 / 23
times than B2 (Z = 115, p < 0.001, r = 0.806 [95% CI: 0.589, 0.883]). In a comparison of the
effect sizes for both the extrinsic and intrinsic visualizations, we can see that the effect sizes of
the differences between one and two variables were greater in the extrinsic group. It can therefore
be assumed that extrinsic visualization is more efficient when a single variable is applied,
while intrinsic visualization is more suitable for two variables.
In addition to the above, we performed a response time comparison at the item level. For
most items, extrinsic visualization resulted in shorter response times than intrinsic visualization.
The opposite was true for only four items, intrinsic visualization only inducing slightly
Fig 8. Mean response time per extrinsic/intrinsic visualizations (calculated from the response times to all
extrinsic/intrinsic items for all participants).
https://doi.org/10.1371/journal.pone.0250164.g008
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 12 / 23
Table 1. Response times for the individual subtest parts (ms).
Extrinsic Intrinsic Wilcoxon rank-sum test
part mean [95% CI] sd median [95% CI] iqr mean [95% CI] sd median [95% CI] iqr Z p-value Effect size r [95% CI]
A1 6622 [5571,
7673]
1972 6406 [5094, 7286] 2007 13634 [12009,
15260]
2934 14917 [10800,
16056]
3436 5 p < 0.001 -0.779 [-0.839, -0.650] large
A2 8609 [7272,
9946]
2509 7827 [6749,
10230]
3066 10102 [8944, 11259] 2090 10184 [7912, 11915] 3237 71 0.093 -0.305 [-0.607, 0.043]
moderate
B1 4945 [4266,
5623]
1273 4762 [3992, 6041] 1781 10798 [8928, 12668] 3377 9825 [8275, 14162] 4416 3 p < 0.001 0.831 [-0.853, -0.735] large
B2 6666 [5861,
7470]
1509 6638 [5132, 7708] 2159 8192 [6996, 9388] 2159 7738 [6542, 9604] 2359 68 0.041 0.369 [-0.643, -0.036]
moderate
whole
test
6896 [6035,
7756]
1615 6494 [2172, 5536] 2172 10929 [9576, 12281] 2442 10217 [9588, 11597] 1705 12 p < 0.001 -0.767 [-0.844, -0.607] large
https://doi.org/10.1371/journal.pone.0250164.t001
Fig 9. Mean response time (ms) per item (calculated for the individual subtest levels).
https://doi.org/10.1371/journal.pone.0250164.g009
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 13 / 23
shorter response times (Fig 10). The differences were significant for most items. An analysis at
the item-level also revealed two other interesting phenomena: the first consisted in significant
variability across the items observed, even within the individual subtests. This variability
reflected the complex nature of maps as research stimuli. The difficulty of a test item depended
on an array of interacting factors, including the type of correct answer, the distractors selected
and the visualized territory. The second observed phenomenon was that the obtained performance
curves associated with both visualization types did not overlap, meaning that the difficulty
of the test items varied depending on the type of visualization. In other words, the items
that were relatively simple to solve in combination with intrinsic visualization were more difficult
with extrinsic visualization, and vice versa.
Eye-tracking analysis
For the purposes of the eye-tracking analysis, the stimuli were divided into three key Areas of
Interest (AOI): instructions (the textual component), map legend and map. The analysis consisted
in a comparison of the dwell times related to the AOI of the individual items (Fig 10 and
Table 2). We were also curious about a comparison of the total dwell times for the extrinsic
Fig 10. Mean response times (ms) per individual items for extrinsic and intrinsic visualization (all items).
https://doi.org/10.1371/journal.pone.0250164.g010
Table 2. Summary of AOI dwell times (ms).
Extrinsic Intrinsic Wilcoxon rank-sum test
AOI mean [95% CI] sd median [95% CI] iqr mean [95% CI] sd median [95% CI] iqr Z p-value Effect size r [95% CI]
Instructions 2514 [2172, 2856] 539 2570 [2193, 2703] 470 2566 [1807, 3325] 1130 2350 [1579, 4252] 1366 74 0.6505 -0.103 [-0.338, 0.53]
Map Legend 216 [84, 348] 208 180 [62, 246] 127 3740 [2921, 4559] 1219 3675 [2411, 4864] 1841 0 p < 0.001 -0.847 [-0.851, -0.749]
Map 3773 [3048, 4497 1141 3592 [2683, 5117] 1785 3769 [2895, 4642] 1300 3188 [2771, 5847] 1354 70 0.833 - 0.051 [-0.363, 0.49]
All AOI 6503 [5549, 7457] 1502 5970 [5052, 8326] 2712 10075 [8350, 11800] 2567 9142 [8014, 13853] 2192 10 p < 0.001 -0.719 [-0.842, -0.475]
https://doi.org/10.1371/journal.pone.0250164.t002
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 14 / 23
(N = 12) and intrinsic (N = 11) groups (see S1 File). The results showed significant differences
in total dwell times, the extrinsic visualization indicating shorter dwell times with a large effect
size. A closer examination revealed that the differences were caused by map legend dwell
times. The “extrinsic” group displayed significantly shorter dwell times on the map legend
than the intrinsic group, with a large effect size and also with a very large gap between the
upper bounds of the confidence intervals of the extrinsic group and the lower bounds of the
confidence intervals of the intrinsic group. No significant differences were observed in the
dwell times during the instructions.
We also visually inspected the oculomotor data in this study at both the item and subtest
levels. The times spent on AOI were converted into percentages. The graphs in Figs 11 and 12
show the ratio of time spent on the instructions, map and map legend. We can observe that at
the beginning of the experiment, the participants in the extrinsic group needed approximately
10% of the total time-on-task to decode the map’s legend; as their experience increased, the
time needed to decode the map legend decreased to as little as zero for some items. The
“intrinsic” group, by contrast, initially spent about 40% of the time exploring the map legend,
with the percentage decreasing with experience, although it remained relatively high (30%).
A comparison of direct saccades (transitions) between the map legend and the visual field
of the map reveal a pattern similar to that described for dwell times. Four AOI were defined
(instructions, map, map legend, button bar), and a matrix of transitions between the AOI for
each item was generated. Fig 13 displays the ratio of the direct map-to-legend/legend-to-map
transitions to the total number of transitions between the AOI. It is clear from the graph that
the “extrinsic” group only made use of the map legend at the beginning of the experiment;
later, direct saccades occurred less. The “intrinsic” group, by contrast, made use of the legend
throughout the tasks, with the number of repeated map-to-legend transitions being higher for
the tasks with a single variable (A1 and B1).
Discussion
The results of the present study showed that the intrinsic visualization employed was significantly
less effective and efficient than extrinsic visualization. In the case of intrinsic visualization,
the participants needed significantly more time to solve the tasks and simultaneously
produced more errors.
Nevertheless, the response time differences between the two visualization methods were
less pronounced when two variables were considered (soil moisture and soil depth). This levelling
was caused by the increase in the time needed to solve the tasks with two variables in both
extrinsic subtests (A and B). The effect was not observed with the intrinsic visualization. The
above finding is in accordance with the studies performed by Nelson [39] and Elmer [43]. We
emphasize that the findings and differences between the visualizations can be generalized only
with regard to the population on which the research was conducted. It is a lay population with
a basic level of map skills and who may also achieve higher education in humanities and social
sciences. Conversely, as the results of the study [6] suggest, a population with a higher level of
map literacy may prefer the intrinsic method in certain tasks. Another potentially significant
change which affects how we work with maps is the type of formal education or the cultural
background of users [71–73].
An exploratory analysis of eye-tracking data provided a deeper insight into the above
results. Dwell time analysis showed that both groups spent comparable time on the instructions
and the map; the reason for longer response times of the “intrinsic” group consisted in
the time needed to decode the map legend. While the “extrinsic” group took only a fraction of
the total dwell time to interpret the map legend, in the case of the “intrinsic” group, it was over
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 15 / 23
a third of the total time-on-task. An analysis at the item level revealed yet another tendency: at
the beginning of the experiment, the participants in the extrinsic group needed approximately
10% of the total time-on-task to interpret the map legend; as their experience increased, this
time decreased to as little as zero for some items. The “intrinsic” group initially spent about
40% of the time decoding the map legend, and although this percentage decreased with experience,
it remained as high as 30%. The above results appear to indicate that the map legend of
an intrinsic visualization is so complex and essential that it needs to be referred to throughout
Fig 11. Mean AOI dwell time per extrinsic/intrinsic group (ms).
https://doi.org/10.1371/journal.pone.0250164.g011
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 16 / 23
the task. The same conclusion could be drawn from an analysis of direct saccades between the
defined AOI.
A comparison of the performance of the “extrinsic” and “intrinsic” verified the greater
effectiveness and efficiency of extrinsic visualization. The results also showed that the type of
task (i.e., whether it concerned a single variable or two variables) had a definitive effect on performance,
which is in accordance with the statement [e.g., 7, 6] that the performance of map
work partly depends on whether the given type of visualization is suitable for the task at hand.
Fig 12. Dwell time on AOI (%) for the extrinsic visualization (top) and intrinsic visualization (bottom). Extrinsic visualization–proportion of dwell time at AOI in
single items; (top); Intrinsic visualization–proportion of dwell time at AOI in single items (bottom).
https://doi.org/10.1371/journal.pone.0250164.g012
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 17 / 23
If we want to understand the effects of different forms of visualizations during the process of
cartographic communication, we must first understand the underlying cognitive processes
[17–20]. A particular type of task may require the activation of specific cognitive processes
which are appropriate to a particular visualization type. Anderson [74] emphasized that visual
representations differ not only in the coding system, but, importantly, in the cognitive processes
they evoke.
The results of the present study indicate that in the case of extrinsic visualization, the map
user first perceives and processes both visually distinct variables consecutively, subsequently
“putting them together” in their working memory when solving the task. Intrinsic visualization,
by contrast, requires only one variable to be kept in working memory at any moment
when the task concerns two variables (soil moisture and depth). When the task involves a single
variable, however, the user must first decode the map legend and keep all three levels of the
variable in their working memory. In the above, the results confirm our assumption that the
cartographic visualization must be selected according to the type of task or operation to be performed
with the particular map.
Our study is not without limitations. One of the limitations was the small sample size and
resulting low power in the statistical tests. Low power may lead to an increase of the risk that
the existing differences in performance will be falsely not detected as statistically significant.
However, we took several (mostly statistical) precautions to prevent the misinterpretation of
our data. We conducted a post-hoc sensitivity analysis which suggested that with a given sample
size, medium to large effect sizes could be acceptably interpreted (see the first section of the
Results chapter), whereas results with small effect sizes would be inconclusive. Rigorous statistical
procedures which allow the interpretation of results on smaller sample sizes were also
employed in the study (including bootstrapped confidence intervals for means, medians and
effect sizes). Regarding the research sample’s composition, we attempted to form a sample
which was as homogenous as possible (age, level of education, field of study, experience with
Fig 13. Ratio of the direct map-to-legend/legend-to-map saccades to the total number of direct saccades (%) between the defined AOI (instructions, legend, map,
button bar).
https://doi.org/10.1371/journal.pone.0250164.g013
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 18 / 23
maps, etc.) and randomly added participants to the extrinsic/intrinsic groups to obtain an
equally balanced sample size for each experimental condition (block randomization) and to
reduce potentially confounding effects.
Furthermore, the sample size in our study does not deviate from the standard practice of
the field of research in question. King [52] pointed out that many studies work with the relatively
small samples given by the high requirements for laboratory equipment. Cognitive cartography
surely is one of the fields in which certain studies have contributed significantly to
increasing knowledge, regardless of their sample sizes [75–78].
However, the size of the research sample and its composition (European university students
of humanities and social sciences with common map literacy skills) permitted us to generalize
the conclusions for similar populations. Further research with this method on different samples
is required to expand the results of the present study and explore how different population
characteristics (map literacy, level and type of formal education) affect the preference for specific
types of visualization.
Conclusion
The performed confirmatory analysis verified the superiority of extrinsic visualization in the
case of a population of individuals with higher formal education in humanities and social sciences,
both in terms of effectiveness and efficiency. Complementary exploratory analysis of
eye tracking data suggests that the reason is the character of the intrinsic map legend, which
demands greater cognitive resources from map readers during processing. The present study’s
findings are significant not only for basic research in visualization and cognitive processes but
also in their implications for cartographic practices. Even despite a relatively small sample size,
the results were statistically significant, but also, importantly, very large effects were discovered.
The extrinsic method can be considered convincingly proved as a more suitable visualization
type for the given types of task and the lay population.
The results of the present study also raise the question of whether a higher level of efficiency
and effectiveness would be maintained with the extrinsic method even if the target population
was composed of individuals with high levels of map literacy and different formal education,
and whether cultural background plays a role. The specific character of a cartographic visualization
sets a significant limit on empirical testing. Even a relatively minor change in partial
parameters (e.g., the absolute size of the circles in the case of the extrinsic method, or using a
different color scheme in case of the intrinsic method) may affect, for example, the processing
speed of visual search or the memorability of the legend, and consequently result in a difference
in overall performance. Therefore, to maintain research rationality, it seems reasonable
to conduct more partial studies with relatively smaller samples while comparing a wider variability
in the applied visualizations and their partial modifications. That is also our objective
for future research: if the trends we uncovered are confirmed in studies which examine modified
legends, the findings may then be generalized to a wider population, and the principle of
the varying methods which were applied can be verified as the cause of the differences in processing.
Changes in visualization parameters may also explain the revealed differences at the
level of individual items.
Supporting information
S1 File.
(DOCX)
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 19 / 23
S1 Fig.
(TIF)
Author Contributions
Conceptualization: Čeneˇk Sˇasˇinka, Zdeneˇk Stachoň.
Data curation: Zdeneˇk Stachoň, Jiřı´ Čeneˇk, Alzˇbeˇta Sˇasˇinkova´.
Formal analysis: Čeneˇk Sˇasˇinka, Zdeneˇk Stachoň, Jiřı´ Čeneˇk, David Lacko.
Funding acquisition: Čeneˇk Sˇasˇinka.
Methodology: Čeneˇk Sˇasˇinka.
Resources: Čeneˇk Sˇasˇinka, Jiřı´ Čeneˇk, Alzˇbeˇta Sˇasˇinkova´.
Software: Pavel Ugwitz.
Supervision: Čeneˇk Sˇasˇinka, Zdeneˇk Stachoň.
Validation: Jiřı´ Čeneˇk, Alzˇbeˇta Sˇasˇinkova´, Stanislav Popelka, David Lacko.
Visualization: Zdeneˇk Stachoň.
Writing – original draft: Čeneˇk Sˇasˇinka, Zdeneˇk Stachoň, Alzˇbeˇta Sˇasˇinkova´, David Lacko.
Writing – review & editing: Čeneˇk Sˇasˇinka, Zdeneˇk Stachoň, Jiřı´ Čeneˇk, Stanislav Popelka,
Pavel Ugwitz, David Lacko.
References
1. Montello DR. Cognitive map-design research in the twentieth century: theoretical and empirical
approaches. Cartogr Geogr Inf Sci. 2002; 29: 283–304. https://doi.org/10.1559/152304002782008503
2. Montello DR., Freundschuh S. Cognition of geographic information. In Usery LY, McMaster RB, editors.
A research agenda for geographic information science, Florida: CRC Press; 2004. pp. 61–91. PMID:
15531985
3. Kolacny A. Cartographic information: A fundamental concept and term in modern cartography. Cartogr
J. 1969; 6: 47–49. https://doi.org/10.1179/caj.1969.6.1.47
4. Stan k K, Friedmannova´ L, Kubı´ ek P, Kone ny´ M. Selected issues of cartographic communication
optimization for emergency centers. Int J Digit Earth. 2010; 3: 316–339. https://doi.org/10.1080/
17538947.2010.484511
5. Kone ny´ M, Kubı´ ek P., Stacho Z., Sˇ asˇinka . The usability of selected base maps for crises management:
users’ perspectives. Appl. Geomat. 2011; 3: 189–198. https://doi.org/10.1007/s12518-011-
0053-1
6. Sˇ asˇinka , Stacho Z, Kubı´ ek P, Tamm S, Matas A, Kuka ova´ M. The Impact of Global/Local Bias on
Task-Solving in Map-Related Tasks Employing Extrinsic and Intrinsic Visualization of Risk Uncertainty
Maps. Cartogr J. 2019; 56: 175–191. https://doi.org/10.1080/00087041.2017.1414018
7. Lokka I, C¸ o¨ltekin A. Simulating Navigation with Virtual 3D Geovisualizations—A focus on memory
related factors. Int Arch Photogramm Remote Sens Spatial Inf Sci. 2016; XLI-B2: 671–673. https://doi.
org/10.5194/isprs-archives-XLI-B2-671-2016
8. Ooms K, De Maeyer P, Fack V, Van Assche E., Witlox F. Interpreting maps through the eyes of expert
and novice users. Int. J. Geogr. Inf. Sci. 2012; 26: 1773–1788. https://doi.org/10.1080/13658816.2011.
642801
9. Ooms K, De Maeyer P, Fack V. Study of the attentive behavior of novice and expert map users using
eye tracking. Cartogr. Geogr. Inf. Sci. 2014; 41: 37–54. https://doi.org/10.1080/15230406.2013.
860255
10. Roth RE. Cartographic Interaction Primitives: Framework and Synthesis. Cartogr J. 2012; 49: 376–
395. https://doi.org/10.1179/1743277412Y.0000000019
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 20 / 23
11. Rautenbach V, Coetzee S, C¸ o¨ltekin C. Development and evaluation of a specialized task taxonomy for
spatial planning–A map literacy experiment with topographic maps. ISPRS J Photogramm Remote
Sens. 2017; 127: 16–26. https://doi.org/10.1016/j.isprsjprs.2016.06.013
12. Innes L. Maths for map users. Proceedings of the 21st International Cartographic Conference.
2003; pp. 727–738. Available: https://icaci.org/files/documents/ICC_proceedings/ICC2003/Papers/
096.pdf.
13. Rinner C, Ferber S. The effects of map reading expertise and map type on eye movements in map comparison
tasks. Abstract and poster presentation at the Conference on Spatial Information Theory.
2005; pp. 14–18. Available: https://digital.library.ryerson.ca/islandora/object/RULA%3A79.
14. Ooms K, De Maeyer P, Dupont L, Van Der Veken N, Van de Weghe N, Verplaetse S. Education in cartography:
what is the status of young people’s map-reading skills? Cartogr Geogr Inf Sci. 2016; 43:
134–15. https://doi.org/10.1080/15230406.2015.1021713
15. Koc¸ H, Demir S. Developing valid and reliable map literacy scale. Rev Int Geograph Educ Online. 2014;
4: 120–137.
16. Clarke D. Are you functionally map literate? Proceedings of the 21st International Cartographic Conference.
2003; pp. 10–16. Available: http://lazarus.elte.hu/cet/publications/088.pdf.
17. Lobben AK. Tasks, Strategies, and Cognitive Processes Associated with Navigational Map Reading: A
Review Perspective. Prof Geogr. 2014; 56: 270–281. https://doi.org/10.1111/j.0033-0124.2004.
05602010.x
18. Fabrikant SI, Lobben AK. Introduction: Cognitive Issues in Geographic Information Visualization. Cartographica
The International Journal for Geographic Information and Geovisualization. 2009; 44:139–
143. https://doi.org/10.3138/carto.44.3.139
19. Slocum TA, Blok C, Jiang B, Koussoulakou A, Montello DR, Fuhrmann S, Hedley NR. Cognitive and
usability issues in geovisualization. Cartogr Geogr Inf Sci. 2001; 28, 61–75. https://doi.org/10.1559/
152304001782173998
20. MacEachren AM, Kraak M-J. Research Challenges in Geovisualization. Cartogr Geogr Inf Sci. 2001;
28: 3–12. https://doi.org/10.1559/152304001782173970
21. Olson JM. Spectrally Encoded Two-Variable Maps. Ann Am Assoc Geogr. 1981; 71: 259–276. https://
doi.org/10.1111/j.1467-8306.1981.tb01352.x
22. Larkin JH, Simon HA. Why a diagram is (sometimes) worth ten thousand words. Cogn Sci. 1987; 11:
65–99. https://doi.org/10.1016/S0364-0213(87)80026-5
23. MacEachren AM. How Maps Work: Representation, Visualization, and Design. 1st ed. New York: Guilford
Press; 1995.
24. Robinson AH, Morrison JL, Muehrcke PC, Kimerling AJ, Guptill SC. Elements of Cartography. 6th ed.
New York: John Wiley & Sons; 2009.
25. Slocum TA, McMaster RB, Kessler FC, Howard HH. Thematic cartography and geovisualization. 3rd
ed. New jersey: Pearson Prentice Hall; 2008.
26. Gershon N. Visualization of an imperfect world. IEEE Comput Graph. 1998; 18: 43–45. https://doi.org/
10.1109/38.689662
27. D’Zmura M. Color in visual search. Vision Res. 1991; 31: 951–966. https://doi.org/10.1016/0042-6989
(91)90203-h PMID: 1858326
28. Lindsey DT, Brown AM, Reijnen E, Rich AN, Kuzmova YI, Wolfe JM. Color channels, not color appearance
or color categories, guide visual search for desaturated color targets. Psychol Sci. 2010; 21:
1208–1214. https://doi.org/10.1177/0956797610379861 PMID: 20713637
29. Itti L, Kof CH. A saliency-based search mechanism for overt and covert shifts of visual attention. Vision
Res. 2000; 40: 1489–1506. https://doi.org/10.1016/s0042-6989(99)00163-7 PMID: 10788654
30. Bertin J. Se´miologie graphique. 2nd ed. Paris: Gauthier-Villars; 1973.
31. Lavie N., Tsal Y. Perceptual load as a major determinant of the locus of selection in visual attention. Percept
Psychophys. 1994; 56: 183–197. https://doi.org/10.3758/bf03213897 PMID: 7971119
32. Granholm E, Asarnow RF, Sarkin AJ, Dykes KL. Pupillary responses index cognitive resource limitations.
Psychophysiology. 1996; 33: 457–461. https://doi.org/10.1111/j.1469-8986.1996.tb01071.x
PMID: 8753946
33. Bru¨nken R, Steinbacher S, Plass JL, Leutner D. Assessment of cognitive load in multimedia learning
using dual-task methodology. Exp Psychol. 2002; 49: 109–119. https://doi.org/10.1027//1618-3169.49.
2.109 PMID: 12053529
34. Nelson ES. Using Selective Attention Theory to Design Bivariate Point Symbols. Cartogr Perspect.
1999; 32: 6–28. https://doi.org/10.14714/CP32.625
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 21 / 23
35. Treisman AM, Gelade G. A feature-integration theory of attention. Cogn Psychol. 1980; 12: 97–136.
https://doi.org/10.1016/0010-0285(80)90005-5 PMID: 7351125
36. Wolfe JM, Cave KR, Franzel SL. Guided search: an alternative to the feature integration model for
visual search. J Exp Psychol Hum Percept Perform. 1989; 15: 419–433. https://doi.org/10.1037//0096-
1523.15.3.419 PMID: 2527952
37. MacEachren AM. Visualizing Uncertain Information. Cartogr Perspect. 1992; 13: 10–19. https://doi.org/
10.14714/CP13.1000
38. Nelson ES. Designing Effective Bivariate Symbols: The Influence of Perceptual Grouping Processes.
Cartogr Geogr Inf Sci. 2000; 27: 261–278. https://doi.org/10.1559/152304000783547786
39. Nelson ES. The Impact of Bivariate Symbol Design on Task Performance in a Map Setting. Cartographica.
2002; 37: 61–78. https://doi.org/10.3138/V743-K505-5510-66Q5
40. Kubı´ ek P, Sˇ asˇinka , Stacho Z, Sˇ t rba Z, Apeltaur J, Urba´nek T. Cartographic Design and Usability
of Visual Variables for Linear Features. Cartogr J. 2017; 54: 91–102. https://doi.org/10.1080/
00087041.2016.1168141
41. Kubı´ ek P, Sˇ asˇinka , Stacho Z. Uncertainty Visualization Testing. Proceedings of the 4th conference
on Cartography and GIS. 2012; pp. 247–256. Available: https://www.researchgate.net/publication/
292155643_UNCERTAINTY_VISUALIZATION_TESTING.
42. Brus J, Ku era M, Popelka S. Intuitiveness of geospatial uncertainty visualizations: a user study on
point symbols. Geografie. 2019; 124:163–85. https://doi.org/10.37040/geografie2019124020163
43. Elmer M. Symbol Considerations for Bivariate Thematic Maps. M.Sc. Thesis, University of Wisconsin–
Madison. 2012. Available from: http://resources.maphugger.com/melmer_webedition.pdf.
44. Kunz M, Gr t-Regamey A, Jurni L. Visualizing natural hazard data and uncertainties—Customization
through a web-based cartographic information system. Int arch photogramm remote sens spat inf sci.
2010; 38: 1–7. Available from: https://www.isprs.org/proceedings/XXXVIII/part4/files/Kunz.pdf.
45. Lakoff G., Johnson M. The metaphorical structure of the human conceptual system. Cognitive Sci.
2018; 4: 195–208.
46. Harold J., Lorenzoni I., Shipley T. F., Coventry K. R. Cognitive and psychological science insights to
improve climate change data visualization. In Nature Climate Change. 2016; 6(12): 1080–1089. https://
doi.org/10.1038/nclimate3162
47. Tukey JW. Exploratory data analysis. Reading: Addison-Wesley; 1977.
48. Behrens JT. Principles and procedures of exploratory data analysis. Psychol Methods. 1997; 2: 131–
160. https://doi.org/10.1037/1082-989X.2.2.131
49. Madan A, Kumar S. (2012). Usability evaluation methods: a literature review. Int. J. Eng. Sci. Technol.
2012; 4: 590–599.
50. ISO 9241–11:2018 Ergonomics of human-system interaction—Part 11: Usability: Definitions and concepts.
2018 [cited 19 November 2020]. In: ISO Online Browsing Platform [Internet]. Available: https://
www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en.
51. Quesenbery W. The Five Dimensions of Usability. In Albers MJ, Mazur MB, editors. Content and Complexity:
Information Design in Technical Communication. New York: Routlege; 2013. pp. 81–102.
52. King AJ, Bol N, Cummins RG, John KK. Improving Visual Behavior Research in Communication Science:
An Overview, Review, and Reporting Recommendations for Using Eye-Tracking Methods. Commun
Methods Meas. 2019; 13: 149–177. https://doi.org/10.1080/19312458.2018.1558194
53. Popelka S, Stacho Z, Sˇ asˇinka , Dolezˇalova´ J. EyeTribe Tracker Data Accuracy Evaluation and Its
Interconnection with Hypothesis Software for Cartographic Purposes. Comput Intell Neurosci. 2016;
9172506. https://doi.org/10.1155/2016/9172506 PMID: 27087805
54. Brychtova A, Coltekin A. An Empirical User Study for Measuring the Influence of Colour Distance and
Font Size in Map Reading Using Eye Tracking. Cartogr J. 2016: 202–212. https://doi.org/10.1179/
1743277414Y.0000000103
55. Keil J, Edler D, Kuchinke L, Dickmann F. Effects of visual map complexity on the attentional processing
of landmarks. PLoS One. 2020; 15: e0229575. https://doi.org/10.1371/journal.pone.0229575 PMID:
32119712
56. Krassanakis V, Cybulski P. A review on eye movement analysis in map reading process: the status of
the last decade. Geodesy Cartogr. 2019; 68: 191–209. https://doi.org/10.24425/gac.2019.126088
57. Brewer CA, Hatchard GW, Harrower MA. ColorBrewer in Print: A Catalog of Color Schemes for Maps.
Cartogr Geogr Inf Sci. 2003; 30: 5–32. https://doi.org/10.1559/152304003100010929
58. OpenStreetMap contributors. Planet dump [Data file from 20180404]. 2018. Available: https://planet.
openstreetmap.org.
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 22 / 23
59. Sˇ asˇinka , Morong K, Stacho Z. The Hypothesis Platform: An Online Tool for Experimental Research
into Work with Maps and Behavior in Electronic Environments. ISPRS International Journal of GeoInformation.
2017; 6(12):407. https://doi.org/10.3390/ijgi6120407
60. Tomczak M, Tomczak E. The need to report effect size estimates revisited. An overview of some recommended
measures of effect size. Trends Sport Sci. 2014; 1: 19–25.
61. Mangiafico S. rcompanion: Functions to Support Extension Education Program Evaluation. R package
version 2.3.25. 2020. Available: https://cran.r-project.org/package=rcompanion.
62. Sherman R. multicon: Multivariate Constructs. R package version 1.6. 2015. Available: https://cran.r-
project.org/package=multicon.
63. Rosenthal R. Parametric measures of effect size. In Cooper H, Hedges LV, editors. The handbook of
research synthesis. New York: Russell Sage Foundation; 1994. pp. 231–244.
64. Faul F, Erdfelder E, Buchner A, Lang A.-G. Statistical power analyses using G*Power 3.1: Tests for correlation
and regression analyses. Behav Res Methods. 2009; 41: 1149–1160. https://doi.org/10.3758/
BRM.41.4.1149 PMID: 19897823
65. Brewer CA. Review of Colour Terms and Simultaneous Contrast Research for Cartography. Cartographica.
1992; 29: 20–30. https://doi.org/10.3138/80ML-3K54-0204-6172
66. Nothdurf HC. Salience from feature contrast: additivity across dimensions. Vision Res. 2000; 40:
1183–1201. https://doi.org/10.1016/s0042-6989(00)00031-6 PMID: 10788635
67. Choudhury AMR. 5—Unusual visual phenomena and colour blindness. In: Choudhury AMR. Principles
of Colour and Appearance Measurement. Object Appearance, Colour Perception and Instrumental
Measurement. 1st ed. Cambridge: Woodhead Publishing; 2014. pp. 185–220. https://doi.org/10.1533/
9780857099242.185
68. Van Zandt T. Analysis of Response Time Distributions. In Pashler H, Wixted J, editors. Stevens’ Handbook
of Experimental Psychology. New Jersey: John Wiley & Sons Inc.; 2002. pp. 461–516.
69. Whelan R. Effective analysis of reaction time data. Psychol Rec. 2008; 58: 475–482. https://doi.org/10.
1007/BF03395630
70. Ratcliff R. Methods for dealing with reaction time outliers. Psychol Bull. 1993; 114: 510–532. https://doi.
org/10.1037/0033-2909.114.3.510 PMID: 8272468
71. Chang K-T, Antes JR. Sex and Cultural Differences in Map Reading. American Cartographer. 1987;
14: 29–42, https://doi.org/10.1559/152304087783875345
72. Lacko D, Sˇ asˇinka , en k J, Stacho Z, Lu W-L. Cross-Cultural Differences in Cognitive Style, Individualism/Collectivism
and Map Reading between Central European and East Asian University Students.
Stud Psychol (Bratisl). 2020; 62: 23–43. https://doi.org/10.31577/sp.2020.01.789
73. Stacho Z, Sˇ asˇinka , en k J, Sˇ t rba Z, Angsuesser S, Fabrikant SI, et al. Cross-cultural differences
in figure–ground perception of cartographic stimuli. Cartogr Geogr Inf Sci. 2019; 46: 82–94. https://doi.
org/10.1080/15230406.2018.1470575
74. Anderson JR. Representational Types: A Tricode Proposal. Technical Report 82–1. Washington, D.C.:
Office of Naval Research, 1982. Available from: https://apps.dtic.mil/dtic/tr/fulltext/u2/a116887.pdf
75. Alac¸am O¨ , Dalcı M. A Usability Study of WebMaps with Eye Tracking Tool: The Effects of Iconic Representation
of Information. Human-Computer Interaction New Trends Lecture Notes in Computer Science.
2009; 12–21. https://doi.org/10.1007/978-3-642-02574-7_2
76. Fuchs S, Spachinger K, Dorner W, Rochman J, Serrhini K. Evaluating cartographic design in flood risk
mapping. Environmental Hazards. 2009; 8: 52–70. https://doi.org/10.3763/ehaz.2009.0007
77. C¸ o¨ltekin A, Brychtova´ A, Griffin AL, Robinson AC, Imhof M, Pettit C. Perceptual complexity of soil-landscape
maps: a user evaluation of color organization in legend designs using eye tracking. International
Journal of Digital Earth. 2016; 10: 560–581. https://doi.org/10.1080/17538947.2016.1234007
78. C¸ o¨ltekin A, Heil B, Garlandini S, Fabrikant SI. Evaluating the Effectiveness of Interactive Map Interface
Designs: A Case Study Integrating Usability Metrics with Eye-Movement Analysis. Cartography and
Geographic Information Science. 2009; 36: 5–17. https://doi.org/10.1559/152304009787340197
PLOS ONE A comparison of the performance on extrinsic and intrinsic cartographic visualizations
PLOS ONE | https://doi.org/10.1371/journal.pone.0250164 April 21, 2021 23 / 23
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
https://doi.org/10.31577/sp.2020.01.789
Cross-Cultural Differences in Cognitive Style,
Individualism/Collectivism and Map Reading between
Central European and East Asian University Students
David Lacko1, 2
, Čeněk Šašinka2
, Jiří Čeněk2, 3
, Zdeněk Stachoň2
, Wei-lun Lu2
1
Department of Psychology, Faculty of Arts, Masaryk University, Arna Nováka 1/1, 602 00 Brno, Czech Republic
2
The Division of Information and Library Studies, Faculty of Arts, Masaryk University, Arna Nováka 1/1, 602 00
Brno, Czech Republic
3
Department of Social Development, Faculty of Regional Development and International Studies, Mendel University,
třída Generála Píky 2005/7, 613 00 Brno, Czech Republic
The article examines cross-cultural differences encountered in the cognitive processing of specific
cartographic stimuli. We conducted a comparative experimental study on 98 participants from two different
cultures, the first group comprising Czechs (N = 53) and the second group comprising Chinese (N =
22) and Taiwanese (N = 23). The findings suggested that the Central European participants were less
collectivistic, used similar cognitive style and categorized multivariate point symbols on a map more
analytically than the Asian participants. The findings indicated that culture indeed influenced human
perception and cognition of spatial information. The entire research model was also verified at an individual
level through structural equation modelling (SEM). Path analysis suggested that individualism and
collectivism was a weak predictor of the analytic/holistic cognitive style. Path analysis also showed that
cognitive style considerably predicted categorization in map point symbols.
Key words: cognitive style, cross-cultural differences, categorization, individualism/collectivism, analytic/
holistic
Correspondence concerning this article should be addressed to David Lacko, Department of Psychology,
Faculty of Arts, Masaryk University, Arna Nováka 1/1, 602 00 Brno, Czech Republic. E-mail:
david.lacko@mail.muni.cz
ORCID https://orcid.org/0000-0002-2904-8118
Received March 21, 2019
Introduction
The objectives of the study were 1) to explore
the cross-cultural differences between Central
European and East Asian populations at
three distinct levels and 2) to examine how
these levels were connected. The presented
research examined whether the selected
populations differed in the degree of individualism/collectivism
and the cognitive style
measured by the Compound Figure Test
(CFT), and whether cultural differences manifested
during cartographic task solving, specifically
in the categorization of multivariate
point symbols.
The theory of analytic and holistic (A/H) cognition
postulates the existence of distinct cognitive
and perceptual styles – relatively stable
ways of cognitive processing (for review, see
Masuda, 2017; Nisbett & Masuda, 2003;
Nisbett & Miyamoto, 2005; Nisbett, Peng, Choi,
& Norenzayan, 2001). The majority of research
in this field focuses on comparing the charac-
24 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
teristics of cognitive processes in two world
regions: the “West” (e.g., North America, Western
Europe) and the “East” (mainly the countries
of East and Southeast Asia such as China,
Japan, South Korea, etc.; Nisbett, 2003). The
theory of A/H cognitive style assumes that
Westerners adopt relatively more analytic cognitive
style and East Asians the holistic one.
A/H cognitive style is defined as “the tendency
for individuals to process information either as
an integrated whole or in discrete parts of that
whole” (Graff, 2003, p. 21). Although the primary
focus of the theory is the comparison of
cognitive processes among cultures, it does
not rule out the existence of within-culture individual
differences in these processes. In other
words, if we compare two people from a certain
cultural background, one can perceive relatively
more analytically, while the other perceives
more holistically.
The A/H model is based on the classic
Witkin’s model of field dependent/independent
cognition (Witkin, Moore, Goodenough, & Cox,
1977) and the Gestalt principles of perceptual
grouping and figure-ground organization
(Wagemans et al., 2012). Recent findings suggest
that many differences exist among people
in higher cognitive processes, such as categorization,
classification, decision-making,
reasoning and causal attribution, and the lower
perceptual processes related to attention, such
as detection of change and field dependence
(for review, see Nisbett et al., 2001; Nisbett &
Masuda, 2003; Nisbett & Miyamoto, 2005).
More precisely, people perceiving relatively
more analytically tend to focus more on perceptually
salient (focal) objects and less on
background and contextual information, and
on the relationships between objects in the
perceptual field (Chua, Boland, & Nisbett, 2005;
Masuda & Nisbett, 2001; Nisbet & Masuda,
2003). Furthermore, people perceiving relatively
more analytically are also less dependent
on external reference frameworks than
their holistic counterparts (Ji, Peng, & Nisbett,
2000; Kitayama, Duffy, Kawamura, & Larsen,
2003), and are less sensitive to contextual
changes while being more sensitive to
changes in focal objects (Masuda & Nisbett,
2006). Researchers believe that cognitive style
also affects the processes of categorization
and classification. Whereas analytic individuals
categorize objects by applying formal rules
of reasoning, holistic individuals categorize
objects by their overall (or holistic) qualities,
similarity and mutual relationships (Chiu,
1972; Ji, Zhang, & Nisbett, 2004; Norenzayan,
Smith, Kim, & Nisbett, 2002).
The value dimension of individualism and
collectivism (I/C) in cross-cultural research is
commonly related to A/H cognitive style and
often used as a predictor of cognitive style and
other psychological phenomena (for review,
see Oyserman, Coon, & Kemmelmeier, 2002).
Some research suggested that collectivistic
individuals are field dependent and holistic,
whereas people from predominantly individualistic
societies are field independent and analytic
(Ji et al., 2000; Nisbett, 2003; Nisbett et
al., 2001; Triandis & Gelfand, 1998). However,
the relationship between I/C and A/H cognitive
styles is rarely measured at the individual level,
and many authors have only assumed the
aforementioned relationships. Other research
has failed to find any empirical evidence at all
of relationships at the individual level between
I/C and A/H cognitive styles (e.g., Davidoff,
Fonteneau, & Fagot, 2008; McKone et al.,
2010).
In the current literature though, theoretical
considerations (e.g., Hermans & Kempen,
1998; Matsumoto, 1999) and empirical evidence
(e.g., Levine et al., 2003; Oyserman et
al., 2002; Takano & Osaka, 1999; Takano &
Osaka, 2018) can be found, criticizing this dichotomous
approach as overly simplifying and
reductionist. Post-communist European countries
are significantly more holistic and collectivistic
than Western Europe (Varnum,
Grossmann, Katunar, Nisbett, & Kitayama,
2008). Other findings suggest the existence of
significant cultural differences not only across
national borders (e.g., Federici, Stella, Dennis,
& Hündsfelt, 2011; Kitayama, Park, Sevincer,
Karasawa, & Uskul, 2009; Varnum et al., 2008)
but also between people from different regions
in a single country (e.g., Kitayama, Ishii, Imada,
Takemura, & Ramaswamy, 2006; Knight &
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 25
Nisbett, 2007; Uskul, Kitayama, & Nisbett,
2008).
These critical findings suggest that the dichotomous
model of cognitive styles might be
overly reductionist. An alternative model was
proposed by Kozhevnikov, Evans, and Kosslyn
(2014). Their model is based on an older
model by Nosal (1990). It emphasizes the ecological
nature of cognitive style that is viewed
as a pattern of cognitive adaptation to the environment.
Cognitive style is in this model environmentally
dependent, flexible and task specific.
This model is hierarchical in the form of a
cognitive-style matrix organizing cognitive
styles on two axes: a) levels of information processing
(perception, concept formation,
higher-order processing, metacognitive processing),
and b) cognitive style families (context
dependence and independence, rulebased
and intuitive processing, internal and
external locus, integration and compartmentalization).
According to this model, various
components of cognitive style would not have
to be inevitably (cor)related – a specific environment
could, for example, elicit development
of local processing (analytic characteristic) and
focus on holistic regions of the map (holistic
characteristic). This theoretical model might
explain the absence of correlations between
various facets of cognitive style reported in
some studies (e.g., Hakim, Simons, Zhao, &
Wan, 2017; Kster, Castel, Gruber, & Kärtner,
2017).
It should be noted that the number of empirical
studies that extend beyond the East-West
dichotomy and explore the nature of cognitive
style and related factors in other cultural regions,
such as Central Europe, is rather limited
(with the exception of, for example,
Cieślikowska, 2006; Čeněk, 2015; Kolman,
Noorderhaven, Hofstede, & Dienes, 2003;
Stachoň et al., 2018; Varnum et al., 2008). The
current research suggests that the people of
Central Europe are rather moderately analytical
in cognitive style and relatively, although
not extremely, individualistic.
As mentioned above, the study employed
cartographic tasks and stimuli in order to explore
the manifestation of cognitive style. This
follows research that has evaluated cartographic
visualization methods that began with
the publication The Look of Maps (Robinson,
1952). These methods gradually developed
into the complex field of cognitive cartography.
Subsequent to cognitive cartography, mappsychology
research was later introduced by
Montello (2002). This approach uses maps
as stimuli in order to understand human perception
and cognition. Examples of map-psychology
research include studies on the influence
of alignment and rotation on memory
(Tversky, 1981) and the influence of cognitive
style while working with bivariate risk maps
(Šašinka et al., 2018). Categorization in cartographic
stimuli was part of the work of
Lewandowski et al. (1993), and research conducted
around the same time anticipated
cross-cultural differences in map reading (e.g.,
MacEachren, 1995; Wood, 1984) that was ultimately
observed (e.g., Angsüsser, 2014;
Stachoň et al., 2018; Stachoň et al., 2019). From
the cross-cultural perspective, especially in
A/H theory, a most interesting study was conducted
by McCleary (1975), who examined the
categorization of map point symbols. The author
found differences in the clustering of dot
symbols and identified two user groups from
these findings: atomists and generalists, who
analogously correspond to the concept of A/H
cognitive style. Nevertheless, another study
(Sadahiro, 1997) did not confirm this grouping,
even though the author also discovered
individual differences in the clustering of dot
symbols in maps (cf. Sadahiro, 1997).
Consequently, the objective of this research
was to further investigate the nature and manifestation
of cognitive style in relation to variables
such as individualism/collectivism in the
culture of Central Europe (Czechia), compared
to typical Eastern Asian cultures (China and
Taiwan) – specifically, 1) to analyze cross-cultural
differences between these two samples
in I/C, visual perception (global versus local
distribution of attention) and categorization
(clustering) in map stimuli, and 2) to verify the
entire theoretical model of relationships between
I/C and A/H cognitive styles at an individual
level and estimate the relationship be-
26 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
tween I/C and selected manifestations of A/H
cognitive style (global/local attention) and map
reading (categorization; see Figure 1).
Methods and Procedures
To achieve the above-mentioned objectives,
we applied several methods (described in
detail below) using Hypothesis online testing
platform (see Procedure section). We also
collected sociodemographic information such
as age, gender, socioeconomic status (SES),
cartography skills, eye defects, number of siblings,
etc.
Independent and Interdependent Self Scale
To measure the individual-level I/C, we administered
the IISS – Independent and Interdependent
Self Scale (Lu & Gilmour, 2007). The
IISS is derived from the CSC – Self-Construal
Scale (Singelis, 1994), the Individualism-Collectivism
Scale (Triandis & Gelfand, 1998) and
the concept of independent/interdependent
self-construal (Markus & Kitayama, 1991). The
IISS comprises 42 (21 for the Independent and
21 for Interdependent Self-Construal Scale)
seven-point Likert-type numerical items (1 =
strongly disagree, 7 = strongly agree). The
original version of the questionnaire was administered
in simplified Chinese (Lu &
Gilmour, 2007). It contains items such as
“I believe that people should try hard to satisfy
their interests.” (independent subscale) or
“I believe that family is the source of our self.”
(interdependent subscale). The Czech version
of the questionnaire was translated from English
in parallel by three independent translators.
Europeans should have higher independent
self-construal (individualistic), and East
Asians should be more interdependent (collectivistic;
Markus & Kitayama, 1991).
Compound Figure Test
The perceptual factors of cognitive style, more
specifically the global and local distribution of
attention, were measured using the CFT –
Compound Figure Test, which is a modified
version of the Navon method (Navon, 1977)
and has been previously used in several studies
(e.g., Kukaňová, 2017; Opach et al., 2018;
Šašinka et al., 2018). The CFT comprises six
practice trials and 32 test trials (blocked design,
same 16 trials for both local and global
processing). Number of trials was considered
satisfactory based on previous research
(Davidoff et al., 2008; von Mühlenen, Bellaera,
Figure 1 Research model
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 27
Singh, & Srinivasa, 2018). Each trial involves
presenting one “Navon figure” – a large number
composed of copies of a smaller different
number (Figure 2). In the local trial, participants
were asked to identify the small numbers as
quickly as possible. In the global trial, they were
required to identify the large number. Participants
used computer mouse to respond. Reaction
time and correct identification were
measured in each trial. The average reaction
time and average success rate was calculated
separately for the local (local reaction time,
indicating analytic processing) and global (global
reaction time, indicating holistic processing)
trials.
The main output of the CFT is the global precedence
score, which is computed as the difference
between the absolute global and local
reaction times (e.g., Gerlach & Poirel, 2018;
McKone et al., 2010). High values of the global
precedence score indicate a holistic cognitive
style (global precedence), low or even negative
values indicate an analytic cognitive style
(local precedence). According to previous research,
people should generally perceive global
features more quickly than local features
(Navon, 1977). Furthermore, analytic perceivers
should be relatively quicker in local and
relatively slower in global tasks than holistic
perceivers (Peterson & Deary, 2006).
Categorization of Multivariate Map Symbols
Map reading and understanding is considered
as a part of visual literacy (Peña, 2017). In addition,
the maps represent the complex stimuli,
which enable the user not only to understand
the presented information but also to derive
the additional information (Morita, 2004), therefore
we used the cartographic stimuli. The cartographic
visualization of multiple phenomena
is known as multivariate mapping. Multivariate
point symbols are one possible multivariate
mapping method (Slocum, McMaster,
Kessler, & Howard, 2005). We created specific
cartographic tasks for purposes of our
experiment. Categorization was measured with
CMMS – Categorization of Multivariate Map
Symbols, which is based on previous research
in categorization (Chiu, 1972; Ji et al.,
2004; Norenzayan et al., 2002) and on the relationship
between cognitive style and map
reading (e.g., Herman et al., 2019; Kubíček et
al., 2016; Opach, Popelka, Doležalová, & Rod,
2018; Stachoň et al., 2018; Šašinka et al.,
2018). The CMMS measures a specific component
of categorization, namely clustering (cf.
McCleary, 1975; Sadahiro, 1997).
The method comprised three practice trials
and twenty test trials. The administration took
between 15 and 30 minutes. In each trial, a
fictional map comprising multiple territorial
units was presented. Each territorial unit contained
one map symbol (Figure 3).
The map symbols contained information
about the four attributes of a particular spatial
unit, namely food costs (originally blue color,
top left position), housing costs (originally red
color, top right position), transport costs (originally
yellow color, bottom left position) and
costs of leisure activities (originally green color,
bottom right position), which were indicated
by the color and size of the map symbol components
(Figure 4). The position and color of
the abovementioned attributes were kept constant,
only their size was manipulated.
Figure 2 Example of the Navon figure used
in the CFT
28 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
Figure 3 Territorial unit and map symbol in CMMS
Figure 4 Multivariate map symbol (descriptions were in Czech and traditional/simplified Chinese
languages)
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 29
Each map was intentionally created to contain
one “holistic” and one “analytic” region
comprising several territorial units defined by
a specific combination of map symbol characteristics
(Figure 6). In the analytic region, one
of the map symbol components was kept constant
and the rest were random (one-dimensional
rule); in the holistic region, all map symbols
had globally similar components, but none
of them were constant (overall-similarity rule,
see Figure 5). The remaining map symbol
components were completely random to avoid
any categorization rule. The analytic and holistic
areas were balanced with respect to reading
direction.
In group A) the maximum value of the blue
parameter (food costs, upper left) was a common
attribute in all symbols. In group B), no
Figure 5 Example of the used analytic A) – left, and holistic B) – right, categorization rules
Figure 6 Example of constructed analytic (left solid line) and holistic (right dashed line) map
regions
30 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
specific attribute was common to any symbol;
they shared overall similarity and equal distribution
of values in different parameters (2x
maximum, 1x medium, and 1x minimum). Alternative
map symbols were created according
to principles published by Norenzayan et
al. (2002).
Participants were asked to identify and mark
a continuous map region comprising at least
four territorial units that, according to their
judgment, belonged together. The CMMS reported
each trial result as a value between -
1 and 1, where a negative value is defined
as a holistic categorization and a positive
value is defined as an analytic categorization.
This value represented the agreement between
the predetermined holistic and analytic
regions and the real marked areas.
A value of ±1 represented total agreement,
while 0 did not represent any agreement.
A control value, calculated as the ratio of
marked territorial units within the predetermined
areas to the sum of all marked territorial
units, was also reported to exclude participants
who had incorrectly marked only a
negligible number of predetermined areas.
A value of .60 and higher was considered a
valid response, and therefore 40% or less
marked territorial units beyond predetermined
areas. For example, if a trial consists of 10
analytic, 10 holistic and 30 random areas and
a participant marks out 11 areas (7 analytic
and 4 random), his/her control value is valid
(analytic marked areas/all marked areas =
7/11 = .636) and his/her score is .70 (analytic
marked areas/all analytic areas = 7/10 = .70).
From the research mentioned above, we
hypothesized that people with a holistic cognitive
style will show a tendency to mark out holistic
regions and people with an analytic cognitive
style will mark out analytic regions. Analogously,
we also assumed that East Asians will
mark out the holistic area more often (and the
analytic area less often) than Czechs.
Research Sample
Before data were collected, a power analysis
in G*Power (v. 3.1.9.2) was conducted. Setting
power at .80 and effect size f at .280 was sufficient
to test at least 104 participants (52 from
each culture).1
We gathered data from 103 participants. Five
participants were excluded from further data
analysis because of missing data. Out of the
remaining 98 participants, 53 participants
were Central Europeans (Czech), and 45 participants
were East Asians (22 Chinese, 23
Taiwanese). All participants were university
students, the majority (57.1%) were women
and most of them studied psychology (69.4%).
The age range was 16–55 years (M = 25.4,
SD = 5.52). From previous studies it seems
that several demographic variables are relevant
to cognitive style, therefore, we gathered information
about cartographic skills and experience
(Ooms et al., 2016), SES (Grossmann &
Varnum, 2011), marital status (Bartoš, 2010),
size of residence (Jha & Singh, 2011), number
of siblings (based on the number of family
members in residence, see Grossmann &
Varnum, 2011) or field of study (Choi, Koo &
Choi, 2007). The detailed descriptive characteristics
of the sample are shown in Table 1.
Our research sample was consequently
adequate for testing the hypotheses in the first
section of results (Cross-Cultural Differences).
In the second section (Relationship between
Sociocultural, Perception and Cognitive Factors),
however, with respect to the sample size,
more demanding methods of statistical analysis
were used, such as SEM, specifically path
analysis. The sample size was relatively inadequate
in this case (according to Ding, Velicer,
& Harlow, 1995, the minimum sample size for
conducting SEM is about 100–150). The results
of SEM therefore needed to be interpreted
cautiously. Normality tests were performed for
all subscales of the methods used. Non-parametric
statistics were used, where the data
were not normally distributed.
1
The value of f was selected from previous crosscultural
research using the Navon method, in which
the effect sizes were .229–.886 (M = .410, SD =
.216; e.g., Fu, Dienes, Shang, & Fu, 2013; McKone
et al., 2010; Tan, 2016). We selected the middle
effect size value f = .280.
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 31
Procedure
Participants were volunteers contacted through
university websites and social networks
Facebook (Czech and Taiwanese) and WeChat
(Chinese). The aforementioned methods were
administered in either simplified/traditional
Chinese or Czech on PCs using the Hypothesis
online testing platform (Popelka, Stachoň,
Šašinka, & Doležalová, 2016; Šašinka, Morong,
& Stachoň, 2017) in the presence of an instructor.
For their participation the participants got a
small reward (USB flash disc) or course credits.
The sequence of the tests was 1) CFT,
2) CMMS, 3) IISS, 4) sociodemographic questionnaire.
The length of the entire procedure
was approx. 35–55 minutes.
Table 1 Demographic characteristics of the participants
Western
Culture
Eastern Culture
Czechia China Taiwan
East Asia
Total
Gender
Male 25 (47.2%) 7 (31.8%) 10 (43.5%) 17 (37.8%)
Female 28 (52.8%) 15 (68.2%) 13(56.5%) 28 (62.2%)
Marital status
Single 31 (58.5%) 16 (72.7%) 13 (56.5%) 29 (64.4%)
Married - 2 (9.1%) 2 (8.7%) 4 (8.9%)
In a relationship 22 (41.5%) 4 (18.2%) 8 (34.8%) 12 (26.7%)
Socioeconomic
status
Poor 1 (1.9%) - 1 (4.3%) 1 (2.2%)
Low income 6 (11.3%) 4 (18.2%) 1 (4.3%) 5 (11.1%)
Middle income 24 (45.3%) 6 (27.3%) 13 (56.5%) 19 (42.2%)
Upper-middle
income
19 (35.8%) 7 (31.8%) 6 (26.1%) 13 (28.9%)
High income 3 (5.7%) 4 (18.2%) 2 (8.7%) 6 (13.3%)
Residence
(population)
< 1K 6 (11.3%) 2 (9.1%) - 2 (4.4%)
1–10K 11 (20.8%) 1 (4.5%) 4 (17.4%) 5 (11.1%)
10–50K 8 (15.1%) 1 (4.5%) 6 (26.1%) 7 (15.6%)
50–100K 14 (26.4%) 2 (9.1%) 1 (4.3%) 3 (6.7%)
100–500K 12 (22.6%) 4 (18.2%) 5 (21.7%) 9 (20%)
500K–1.5M 2 (3.8%) 4 (18.2%) 1 (4.3%) 5 (11.1%)
1.5M–3M - 3 (13.6%) 4 (17.4%) 7 (15.6%)
3M > - 4 (28.2%) 2 (8.7%) 6 (13.3%)
Field of study
Psychology 39 (73.6%) 12 (54.5%) 17 (73.9%) 29 (64.4%)
Other 14 (26.4%) 10 (45.5%) 6 (16.1%) 16 (33.6%)
Number of
siblings
0 6 (11.3%) 3 (13.6%) - 3 (6.7%)
1 31 (58.5%) 14 (63.6%) 12 (52.2%) 26 (57.8%)
2 11 (20.8%) 2 (9.1%) 10 (43.5%) 12 (26.7%)
3 4 (7.5%) 1 (4.5%) - 1 (2.2%)
4 or more 1 (1.9%) 1 (4.5%) 1 (4.3%) 2 (4.4%)
Age range
(mean, SD)
20–33
(M 23.6,
SD 2.32)
18–46
(M 27.5,
SD 7.43)
16–55
(M 27.5,
SD 7.24)
16–55
(M 27.5,
SD 7.25)
32 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
Results
The data were processed with IBM SPSS Statistics
(v. 25), IBM SPSS Amos (v. 25) and R
(v. 3.4.4, Lavaan and SemTools packages). The
results are presented in two sections: CrossCultural
Differences and Relationship between
Sociocultural, Perceptual and Cognitive Factors.
Analysis of the differences between Taiwanese
and Chinese participants and also
the individual differences between relevant
sociocultural variables (e.g., SES, gender,
number of siblings, age) were also performed,
with no significant differences found in any of
the variables. Because of these results, we
combined Taiwanese and Chinese participants
into a single “Chinese/Taiwanese” group
for any subsequent statistical analysis.
Cross-Cultural Differences
The IISS Questionnaire had a satisfactory reliability
in both the independent α = .895 (Czech
version α = .815, Chinese version α = .929)
and interdependent α = .872 (Czech version
α = .795, Chinese version α = .906) subscales.
Furthermore, the subscales did not correlate
with each other (Spearman partial rs
= .155,
p = .177, culture was a control variable).
The Chinese/Taiwanese were relatively
more collectivistic (interdependent subscale)
and less individualistic (independent subscale)
than the Czechs. The Chinese/Taiwanese
scored an average of 5.17 (SD = .761) in
the collectivistic subscale and 5.18 (SD = .911)
in the individualistic subscale, whereas the
mean scores of the Czechs were 4.66 (SD =
.564) in the collectivistic subscale and 5.35
(SD = .502) in the individualistic subscale (Figure
7). The statistical significance of these differences
was tested with one-way ANOVA. The
differences were significant only in the case of
collectivism: F(1, 96) = 14.456, p < .001, with
medium effect size (η2
= .131). No significant
differences were found between the groups in
the individualism subscale (Mann-Whitney
U = 1105.5, p = .535, r = .063). The data were
also analyzed with respect to sociodemographic
variables. No other significant relationships
were observed (for the complete list of
collected variables, see Table 1).
A medium correlation was found between
both local and global CFT tasks (Spearman
partial rs
= .564, p < .001, culture was a control
variable). Two participants were removed from
Figure 7 IISS – mean scores
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 33
further analysis because of their high error
rates (more than four errors in each task).
The results suggest that all participants had
significantly quicker reaction times in the global
task than in the local task (Wilcoxon
signed-rank test Z = -6.634, p < .001, r = -.677).
The findings also show that Czechs were
quicker than Chinese/Taiwanese in both local
and global tasks. The average reaction time of
the Czech participants in the global task was
0.99 sec. (SD = .209) compared to the Chinese/Taiwanese
participants with an average
reaction time of 1.66 sec. (SD = .466). A similar
pattern was observed in the local task, where
the average reaction time of the trial solution
was 1.13 sec. (SD = .144) for the Czechs and
1.77 s (SD = .387) for the Chinese/Taiwanese
participants (Figure 8). Czechs were significantly
quicker in both the global (U = 204, p <
.001, r = -.711) and local (F(1, 95) = 121.960, p
< .001, η2
= .562) tasks, with large effect sizes.
These differences in reaction times, however,
cannot be interpreted in the A/H paradigm
as any difference in cognitive style but rather
as differences in the emphasis that both
groups placed on the speed of the CFT solution
(Kukaňová, 2017; Yates et al., 2010). We
also calculated the global precedence score
using the aforementioned procedure of difference,
specifically by subtracting the local reaction
times from global reaction times. Although
the Czech participants had a relatively
higher global precedence score (M = .139,
SD = .210) than the Chinese/Taiwanese participants
(M = .108, SD = .574), this difference
was not significant (U = 949, p = .083, r =
-.175) (Figure 9).
The final method applied was CMMS. Four
participants were removed from further analysis
because of their high error rate (participants
that marked less than three territorial
units into one continuous map region). The
results on a scale between -1 (holistic) to 1
(analytic) show that Czechs categorized in
maps more analytically (M = .044, SD = .360)
and East Asians categorized in maps more
holistically (M = -.063, SD = .172) (Figure 10).
This cultural difference was statistically significant
(U = 795, p = .021), with a small effect
size (r = -.235). However, the results show that
both groups used a similar cognitive style to
categorize map symbols and only small differences
in cognitive strategies were found. Moreover,
both groups scored relatively close to zero,
which is probably caused by using various categorization
strategies across different trials,
Figure 8 CFT – mean reaction times
34 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
because absolute scores were higher for both
Czechs (M = .461, SD = .183) and Asians (M =
.247, SD = .148).
Relationship between Sociocultural, Perceptual
and Cognitive Factors
We performed a Spearman partial correlation
and a path analysis (type of SEM) to verify the
research model at individual level in order to
obtain an improved and deeper understanding
of the phenomena under scrutiny and their
mutual relationships.
Using a non-parametric Spearman partial
correlation with culture as control variable, only
weak correlations were found between the
CMMS scores and the CFT global reaction
times (rs
= .222, p = .035) and between the
Figure 10 CMMS – Cross-cultural differences in map categorization (High value = analytic,
low value = holistic).
Figure 9 CFT – Mean global precedence scores (higher values mean higher global prece-
dence)
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 35
CMMS scores and the CFT global precedence
scores (rs
= -.216, p = .040). The whole correlation
matrix is shown in Table 2:
A path analysis was also performed using
the expectation–maximization (EM) method to
estimate missing values and an asymptotically
distribution-free (ADF) method, which is
adequate for non-parametric data. Since both
cultures were analyzed together, culture was
used as a “control variable”. Two models were
analyzed: Model 1 comprised CFT reaction
times, and Model 2 was computed with the
calculated CFT global precedence score (Figure
11). Both models showed a very good fit
(Table 3).
Path analysis for Model 1 revealed a weak
direct effect of individualism (IISS independent
self-construal scale) on CFT local reaction
times (β = -.250, B = -.167, p = .003) and a
weak direct effect of collectivism (IISS interdependent
self-construal scale) on CFT global
reaction times (β = -.196, B = -.135, p = .047).
The higher score in individualism therefore
indicated a quicker reaction time in the local
Figure 11 Path analysis models – Model 1 (left), Model 2 (right)
Table 2 Spearman partial correlation matrix
1. 2. 3. 4. 5. 6.
1. Individualism –
2. Collectivism .155 –
3. CFT local RT .002 .073 –
4. CFT global RT -.026 -.140 .564** –
5. Global precedence score .125 .183 .176 -.546** –
6. CMMS .066 .147 -.063 .222* -.216* –
Note. * p < .05, ** p < .001
36 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
task, and the higher score in collectivism indicated
a quicker reaction time in the global task,
i.e., I/C scores weakly predicted the performance
in CFT tasks. Moderate direct effects of
the CFT global reaction times (β = .713, B =
.450, p < .001) and the CFT local reaction times
(β = -.776, B = -.521, p < .001) on the CMMS
scores were also found. These results suggest
that the analytic perceivers (persons with
a quicker CFT local reaction time) tended to
use an analytic manner of categorizing point
multivariate map symbols, and that the holistic
perceivers (persons with quicker CFT global
reaction time) used a rather holistic manner
of categorizing point multivariate map symbols.
In other words, the CFT reaction times
satisfactorily predicted the map categorization
style. In order to estimate the indirect effects of
I/C on point multivariate map symbol categorization,
bootstrapping (N = 2000, CI = 95%) was
performed, and a very weak indirect (mediation)
effect of collectivism (IISS interdependent
self-construal scale) on the CMMS score (β =
-.175, B = -.077, p = .028) was detected.
Path analysis performed on Model 2 showed
a weak direct effect of collectivism (IISS interdependent
self-construal scale) on the CFT
global precedence score (β = .357, B = .156,
p = .017). This finding suggests that collectivistic
people tended to use a more global distribution
of attention that is characteristic of the
holistic cognitive style. A moderate direct effect
of the CFT global precedence score on map
categorization (β = -.502, B = -.502, p < .001)
was also observed, i.e., participants who
showed a relatively more global distribution of
attention, categorized symbols in maps according
to relatively more holistic rules, and vice
versa, participants who showed a relatively
more local distribution of attention, were prone
to use relatively more analytic rules of categorization.
A very weak significant indirect (mediation)
effect of collectivism (ISS interdependent
self-construal scale) on map categorization
(β = -.179, B = -.078, p = .026) was also
found after bootstrapping (N = 2000, CI = 95%).
It should be noted that we reported only significant
relationships. However, as shown in
Figure 11, we included all plotted relations in
the models (i.e., IIISS independent on CFT global
RT and IISS interdependent on CFT local
RT in Model 1 and IISS independent on CFT
global precedence score in Model 2). Moreover,
we also performed indirect (meditation)
effect of individualism (IISS independent selfconstrual
scale) on map categorization with
no significant results in both models. The exogenous
control variable “culture” had statistically
significant and large regression coefficients
on all endogenous variables in the
models. Nevertheless, we did not report these
results because we added it to our models
only in order to weaken the influence of other
variables. Moreover, the apparent dissension
between insignificant correlation coefficients
and significant regression coefficients of path
analysis could be explained by suppression
effect and Simpson’s paradox (see MacKinnon,
Krull, & Lockwood, 2000; Tu, Gunnell, &
Gilthorpe, 2008), which postulates that a more
complex statistical model can reduce, reverse
or even enhance the relationships between
variables.
Discussion
The aims of the presented study were: 1) to
compare I/C and A/H cognitive styles and map
categorization in Czech and East Asian (Chinese/Taiwanese)
university students, and 2)
Table 3 Models fits
Model Chi-square p-value CFI RMSEA AIC BIC ECVI
Model 1 χ
2
(3) = 3.897 .273 .995 .057 39.897 85.289 .438
Model 2 χ
2
(3) = 4.435 .218 .960 .073 28.435 58.697 .312
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 37
to investigate and verify the theoretical model
of relationships between I/C and A/H cognitive
styles and between A/H cognitive styles and
their behavioral manifestation in the process
of map categorization.
The results suggest that the Czech participants
showed a significantly lower level of collectivism
(interdependent self-construal scale)
than did the Chinese/Taiwanese participants
and a similar level of individualism (independent
self-construal scale). Our results partly
support the current theory that describes the
West as relatively less collectivistic than the
East (Hofstede, 1983; Markus & Kitayama,
1991; Nisbett et al., 2001; Triandis & Gelfand,
1998). Furthermore, a similar level of individualism
corresponds to the empirical research
in I/C in post-communist countries (Kolman et
al., 2003; Varnum et al., 2008) and even with
previous research in I/C in Czechia (Bartoš,
2010; Čeněk 2015). This finding also supports
the claims of rapid individualization in the young
East Asian populations (e.g., Moore, 2005;
Steele & Lynch, 2013).
The results of the CFT show that all of the
participants performed the global tasks more
quickly than the local tasks, which is consistent
with previous findings (Navon, 1977). However,
our participants were generally slower
compared to the original study (Navon, 1977).
This fact was most probably caused by the
way of responding (mouse click instead of
keyboard buttons) because mouse response
process has in contrast with keyboard response
process one extra step (i.e., moving
the mouse cursor above the response option).
Our results also indicate cross-cultural differences
in the general reaction times of CFT
stimuli processing. The Czech participants
were significantly quicker in both the global
and local tasks. However, as mentioned
above, these differences in reaction times
demonstrated rather differences in the emphasis
that both groups placed on the speed than
differences in cognitive style (Kukaňová, 2017;
Yates et al., 2010). The comparison of the global
precedence scores (calculated from CFT
global and local reaction times) showed no
differences in global/local processing between
the Czechs and Chinese/Taiwanese, which
was contrary to our expectations. The results
of the CFT could be seen as a contradiction to
the notion of the “analytic West” and “holistic
East” (Nisbett et al., 2001; Nisbett & Masuda,
2003; Nisbett & Miyamoto, 2005). However, it
is still not clear whether Central Europeans
count as the “analytic West”. For example,
Varnum et al. (2008) showed that Central European
post-communist countries are relatively
more holistic in their patterns of attention
than Western Europe. Other empirical research,
comparing the sensitivity to the context
of Czech vs. Czech Vietnamese (Čeněk,
2015), and Czech vs. Chinese (Stachoň et al.,
2018, Stachoň et al., 2019), reported mixed or
contradictory results in terms of the expected
differences in cognitive style.
The results of the CMMS show that the Czech
participants categorized more analytically in
maps, whereas the Chinese/Taiwanese categorized
more holistically. This result agrees
with the theory that Westerners use slightly
more analytic categorization patterns and Easterners
use more holistic categorization (Chiu,
1972; Ji et al., 2004; Norenzayan et al., 2002).
However, the effect size of this significant difference
was relatively small.
Path analysis was used to test the validity of
two structural models of relationships between
the variables of interest. Both evaluated models
(CFT local and global reaction times and
the global precedence score) showed a satisfactory
good fit. The results of the path analysis
show that I/C is a weak predictor of the
level of global/local distribution of attention, i.e.,
collectivist persons tended to use a holistic
cognitive style, and individualistic persons
tended to use a rather analytic cognitive style.
These findings partly support the theory of holistic
and analytic cognitive styles (Nisbett,
2003; Nisbett et al., 2001; Triandis & Gelfand,
1998), although the values of the path coefficients
were relatively small. The path analysis
also did not find all of the expected direct and
indirect effects of I/C on the scores of the CFT
and the CMMS. The aforementioned findings
were, therefore not a conclusive argument to
support the A/H cognitive style theory in cross-
38 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
cultural context (cf. Nisbet et al., 2001). As with
several other studies that did not find any relationship
between I/C and A/H cognitive style
(e.g., Davidoff et al., 2008; McKone et al., 2010;
Takano & Osaka, 1999), it may be possible
that this relationship could be different from
what researchers expect, or perhaps even nonexistent.
Our findings of unconvincing yet significant
relationships could also be explained
in theoretical arguments, which maintain that
the I/C and A/H cognitive styles only manifest
at a cultural (i.e., cross-cultural comparison)
not individual level (i.e., SEM and path analysis;
cf. Na et al., 2010). Nevertheless, we would
like to emphasize that the sample size was, in
this case, relatively inadequate for SEM, therefore
its results should be understood as only
exploratory.
The concept of I/C and its measurement with
self-report scales have recently been subject
to disagreement from many scholars. This criticism
mainly cites the lack of concurrent, discriminant
and construct validity, insufficient
conceptualization, a reductionist and dichotomous
approach and insufficient psychometric
characteristics in questionnaires (for review,
see Levine et al., 2003; Matsumoto, 1999;
Oyserman et al., 2002; Vignoles et al., 2016).
For example, if the individual level of I/C can be
significantly influenced by priming, then it
means that I/C is not as stable in time as it is
generally assumed (Gardner, Gabriel, & Lee,
1999; Oyserman & Lee, 2008). Moreover, according
to the results of meta-analytical studies
and systematic reviews, the West should
not be described as strictly individualistic nor
the East as purely collectivistic (Levine et al.,
2003; Oyserman et al., 2002; Takano & Osaka,
1999; Takano & Osaka, 2018). Most recently,
for example, Hakim et al. (2017) compared the
levels of individualism and collectivism of
American and Asian international students and
found, contrary to expectations, that Americans
were significantly more collectivistic, whereas
the Asian students were significantly more in-
dividualistic.
Path analysis also found that global/local
distribution of attention had a moderate predictive
power on categorization in both of the
tested models, i.e., analytic perceivers (defined
by the CFT global precedence score) used
analytic categorization in maps, whereas holistic
perceivers used holistic categorization.
This finding is consistent with the research
theory (Chiu, 1972; Ji et al., 2004; Norenzayan
et al., 2002) and the empirical research
(Kubíček et al., 2016; Šašinka et al., 2018;
Stachoň et al., 2019). Consequently, the cognitive
style that is characterized as a perceptual
process is presumably manifested in
higher cognitive processes, such as map reading
and categorization.
Conclusions
The article describes cross-cultural differences
in western and eastern cultures, between
Czech and Chinese/Taiwanese university
students, respectively. The theoretical background
of the research was based on the
theory of analytic and holistic cognitive styles
and the dimensions of individualism and collectivism.
Two main objectives were defined:
first, to identify the possible cross-cultural differences
and similarities between Czechs and
Chinese/Taiwanese, and second, to verify the
entire research model and the relationships
between A/H cognitive style and I/C at individual
levels. For this purpose, we also developed a
new method (CMMS) in order to study how A/H
cognitive style was manifested during categorization
in map reading. The results suggest
that cross-cultural differences exist between
both cultures, especially at the level of collectivism
(Czechs were less collectivist than the
Chinese/Taiwanese) and categorization in
map reading (Czechs used more analytic and
less holistic categorization). Neither individual
differences (e.g., SES, gender, age) nor differences
in cognitive style measured by the CFT
between Czech and East Asians were found.
The findings also indicate that I/C is a weak
predictor of A/H cognitive style and that A/H cognitive
style moderately predicts categorization
in map reading.
These results contradict the East-West dichotomy
and suggest that the culture of Central
Europe (specifically Czechia) is much more
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 39
similar to the East than expected from the literature.
However, more cross-cultural research
of typically Western, typically Eastern and Central
European cultures is needed for an improved
understanding of the real influence of
culture on human perception and cognition in
regions outside the East-West dichotomy.
Based on the presented results, future research
should focus on verification of Nisbett’s
(2001) vs. Kozhevnikov’s (2014) models of
cognitive styles. Above all, specify the number
of cognitive style families, investigate the stability/flexibility
of cognitive styles, and inspect
the developmental aspects (e.g., children of
different age) of cognitive style and its adaptive
nature (e.g., research on expatriates during
the process of cultural adaptation) is also
suggested.
Acknowledgement
This publication was supported by the Czech
Science Foundation (GC19-09265J: The influence
of socio-cultural factors and writing system
on perception and cognition of complex
visual stimuli).
References
Angsüsser, S. (2014). Possible reasons for size differences
between point symbols in German and
Chinese city maps. In T. Bandrova & M. Konecny
(Eds.), Proceedings of the 5th International Conference
on Cartography & GIS (pp. 268–277),
Sofia: Bulgarian Cartographic Association. https:/
/cartography495gis.com/docsbca/5ICCandGIS_
Proceedings.pdf
Bartoš, F. (2010). Individualismus a kolektivismus v
české populaci a jejich souvislost s narcismem.
Sociólogia, 42(2), 134–160.
Čeněk, J. (2015). Cultural dimension of individualism
and collectivism and its perceptual and cognitive
correlates in cross-cultural research. The Journal
of Education, Culture and Society, 6(2), 210–
225. https://doi.org/10.15503/jecs20152.210.225
Chiu, L. H. (1972). A Cross-cultural comparison of
cognitive styles in Chinese and American children.
International Journal of Psychology, 7(4), 235–
242. https://doi.org/10.1080/00207597208246604
Choi, I., Koo, M., & Choi, J. (2007). Individual differences
in analytic versus holistic thinking. Personality
and Social Psychology Bulletin, 33(5), 691–
705. https://doi.org/10.1177/0146167206298568
Choi, I., & Nisbett, R. (1998). Situational salience and
cultural differences in the correspondence bias
and actor-observer bias. Personality and Social
Psychology Bulletin, 24(9), 949–960. https://
doi.org/10.1177/0146167298249003
Choi, I., Nisbett, R., & Norenzayan, A. (1999). Causal
attribution across cultures: Variation and universality.
Psychological Bulletin, 125(1), 47–63.
https://doi.org/10.1037/0033-2909.125.1.47
Chua, H., Boland, J., & Nisbett, R. (2005). Cultural
variation in eye movements during scene perception.
Proceedings of the National Academy of
Sciences of the United States of America, 102(35),
12629–12633. https://doi.org/10.1073/pnas.
0506162102
Cieślikowska, D. (2006). Object and context in perception
and memory: Polish-Chinese research on
analytic and holistic cognitive styles. Studia
Psychologiczne (Psychological Studies), 44(1),
80–99.
Cohen, N., & Ariely, T. (2011). Field research in conflict
environments: Methodological challenges and
snowball sampling. Journal of Peace Research,
48(4), 423–435. https://doi.org/10.1177/
0022343311405698
Davidoff, J., Fonteneau, E, & Fagot, J. (2008). Local
and global processing: Observations from a remote
culture. Cognition, 108(3), 702–709. https://
doi.org/10.1016/j.cognition.2008.06.004
Ding, L., Velicer, W., & Harlow, L. (1995). Effect of
estimation methods, number of indicators per factor
and improper solutions on structural equation
modeling fit indices. Structural Equation Modeling,
2(2), 119–143. https://doi.org/10.1080/
10705519509540000
Federici, S., Stella, A., Dennis, J., & Hündsfelt, T.
(2011). West vs. West like East vs. West? A comparison
between Italian and US American context
sensitivity and fear of isolation. Cognitive Processing,
12(2), 203–208. https://doi.org/10.1007/
s10339-010-0374-8
Fu, Q., Dienes, Z., Shang, J., & Fu, X. (2013). Who
learns more? Cultural differences in implicit sequence
learning. PLoS ONE, 8(8), e71625. https:/
/doi.org/10.1371/journal.pone.0071625
Gardner, W., Gabriel, S., & Lee, A. (1999). “I” value
freedom, but “we” value relationships: Selfconstrual
priming mirrors cultural differences in
judgment. Psychological Science, 10(4), 321–326.
https://doi.org/10.1111/1467-9280.00162
Gerlach, Ch., & Poirel, N. (2018). Navon’s classical
paradigm concerning local and global processing
relates systematically to visual object classifica-
40 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
tion performance. Scientific Reports, 8(1), 324.
https://doi.org/10.1038/s41598-017-18664-5
Graff, M. (2003). Learning from web-based instructional
systems and cognitive style. British Journal
of Educational Technology, 34(4), 407–418. https:/
/doi.org/10.1111/1467-8535.00338
Grossmann, I., & Varnum, M. (2011). Social class,
culture, and cognition. Social Psychological and
Personality Science, 2(1), 81–89. https://doi.org/
10.1177/1948550610377119
Hakim, N., Simons, D., Zhao, H., & Wan, X. (2017).
Do Easterners and Westerners differ in visual
cognition? A preregistered examination of three
visual cognition tasks. Social Psychological and
Personality Science, 8(2), 1–11. https://doi.org/
10.1177/1948550616667613
Herman, L., Juřík, V., Stachoň, Z., Vrbík, D., Russnák,
J., & Řezník, T. (2018). Evaluation of user performance
in interactive and static 3D maps. ISPRS
International Journal of Geo-Information, 7(11),
1–25. https://doi.org/10.3390/ijgi7110415
Hermans, H., & Kempen, H. (1998). Moving cultures:
The perilous problems of cultural dichotomies in a
globalizing world. American Psychologist, 53(10),
1111–1120. https://doi.org/10.1037/0003-066X.53.
10.1111
Hofstede, G. (1983). The cultural relativity of organizational
practices and theories. Journal of International
Business Studies, 14(2), 75–89. https://
doi.org/10.1057/palgrave.jibs.8490867
Jha, S. D., & Singh, K. (2011). An analysis of individualism-collectivism
across Northern India. Journal
of the Indian Academy of Applied Psychology,
37(1), 149–156.
Ji, L., Peng, K., & Nisbett, R. (2000). Culture, control,
and perception of relationships in the environment.
Journal of Personality and Social Psychology.
78(5), 943–955. https://doi.org/10.1037//0022-
3514.78.5.943
Ji, L., Zhang, Z., & Nisbett, R. (2004). Is it culture or
is it language? Examination of language effects in
cross-cultural research on categorization. Journal
of Personality and Social Psychology, 87(2),
57–65. https://doi.org/10.1037/0022-3514.87.1.57
Kitayama, S., Duffy, S., Kawamura, T., & Larsen, J.
T. (2003). Perceiving an object and its context in
different cultures: A cultural look at new look. Psychological
Science, 14(3), 201–206. https://
doi.org/10.1111/1467-9280.02432
Kitayama, S., Ishii, K., Imada, T., Takemura, K., &
Ramaswamy, J. (2006). Voluntary settlement and
the spirit of independence: Evidence from Japan’s
“Northern frontier.” Journal of Personality and
Social Psychology, 91(3), 369–384. https://doi.org/
10.1037/0022-3514.91.3.369
Kitayama, S., Park, H., Sevincer, A., Karasawa, M.,
& Uskul, A. (2009). A cultural task analysis of implicit
independence: Comparing North America,
Western Europe, and East Asia. Journal of Personality
and Social Psychology, 97(2), 236–255.
https://doi.org/10.1037/a0015999
Knight, K., & Nisbett, R. (2007). Culture, class, and
cognition: Evidence from Italy. Journal of Cognition
and Culture, 7(3), 283–291. https://doi.org/
10.1163/156853707X208512
Kolman, L., Noorderhaven, N., Hofstede, G., Dienes,
E. (2003). Crosscultural differences in Central
Europe. Journal of Managerial Psychology,
18(1), 76–88. https://doi.org/10.1108/
02683940310459600
Köster, M., Castel, J., Gruber, T., & Kärtner, J. (2017).
Visual cortical networks align with behavioral
measures of context-sensitivity in early childhood.
NeuroImage, 163(2017), 413–418. https://doi.org/
10.1016/j.neuroimage.2017.08.008
Kozhevnikov, M., Evans, C., & Kosslyn, S. M. (2014).
Cognitive style as environmentally sensitive individual
differences in cognition: A modern synthesis
and applications in education, business, and
management. Psychological Science in the Public
Interest, 15(1), 3–33. https://doi.org/10.1177/
1529100614525555
Kubíček, P., Šašinka, Č., Stachoň, Z., Štěrba, Z.,
Apeltauer, J., & Urbánek, T. (2016). Cartographic
design and usability of visual variables for linear
features. The Cartographic Journal, 54(1), 91–
102. https://doi.org/10.1080/00087041.2016.
1168141
Kukaňová, M. (2017). Porovnání dvou typů
vizualizací z hlediska percepční a kognitivní
zátěže a kognitivních schopností jedince. [Comparison
of the two types of visualization in terms
of perceptual and cognitive load, and personal
cognitive abilities]. [Doctoral Dissertation]. Brno:
Masaryk University. https://is.muni.cz/auth/th/
djk1o/
Levine, T., Bresnahan, M., Park, H., Lapinski, M.,
Wittenbaum, G., Shearman, S., … & Ohashi, R.
(2003). Self-construal scales lack validity. Human
Communication Research, 29(2), 210–252. https:/
/doi.org/10.1111/j.1468-2958.2003.tb00837.x
Lewandowsky, S., Herrmann, D. J., Behrens, J. T.,
Li, S., Pickle, L., & Jobe, J. B. (1993). Perception of
clusters in statistical maps. Applied Cognitive
Psychology, 7(6), 533–551. https://doi.org/10.1002/
acp.2350070606
Lu, L., & Gilmour, R. (2007). Developing a new measure
of independent and interdependent views of
the self. Journal of Research in Personality, 41(1),
249–257. https://doi.org/10.1016/j.jrp.2006.09.005
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 41
MacEachren, A. (1995). How maps work: Representation,
visualization, and design. New York:
Guilford Press.
MacKinnon, D. P., Krull, J. L., & Lockwood, C. M.
(2000). Equivalence of the mediation, confounding
and suppression effect. Prevention Science:
The Official Journal of the Society for Prevention
Research, 1(4), 173–181. https://doi.org/10.1023/
A:1026595011371
Markus, H., & Kitayama, S. (1991). Culture and the
self: Implications for cognition, emotion, and motivation.
Psychological Review, 98(2), 224–253.
https://doi.org/10.1037/0033-295X.98.2.224
Masuda, T. (2017). Culture and attention: Recent
empirical findings and new directions in cultural
psychology. Social and Personality Psychology
Compass, 11(12), e12363. https://doi.org/10.1111/
spc3.12363
Masuda, T., & Kitayama, S. (2004). Perceived-induced
constraint and attitude attribution in Japan
and in the US: A case for cultural dependence of
the correspondence bias. Journal of Experimental
Social Psychology, 40(3), 409–416. https://
doi.org/10.1016/j.jesp.2003.08.004
Masuda, T., & Nisbett, R. (2001). Attending holistically
versus analytically: Comparing the context
sensitivity of Japanese and Americans. Journal
of Personality and Social Psychology, 81(5), 922–
934. https://doi.org/10.1037//0022-3514.81.5.
922
Masuda, T., & Nisbett, R. (2006). Culture and change
blindness. Cognitive Science, 30(2), 381–399.
http://dx.doi.org/10.1207/s15516709cog0000_63
Matsumoto, D. (1999). Culture and self: An empirical
assessment of Markus and Kitayama’s theory of
independent and interdependent self-construals.
Asian Journal of Social Psychology, 2(3), 289–
310. https://doi.org/10.1111/1467-839X.00042
McCleary, G. (1975). In pursuit of the map user. In J.
Kavaliunas (Ed.), Proceedings of the International
Symposium on Computer-Assisted Cartography:
Auto Carto II (pp. 238–250), Washington: U.S.
Department of Commerce, Bureau of the Census.
http://cartogis.org/docs/proceedings/archive/auto-
carto-2/pdf/auto-carto-2.pdf
McKone, E., Davies, A., Fernando, F., Aalders, R.
Leung, H., Wickramariyaratne, T., & Platow, M.
(2010). Asia has the global advantage: Race and
visual attention. Vision Research, 50(16), 1540–
1549. https://doi.org/10.1016/j.visres.2010.05.010
Miyamoto, Y., & Kitayama, S. (2002). Cultural variation
in correspondence bias: The critical role of
attitude diagnosticity of socially constrained behavior.
Journal of Personality and Social Psychology,
83(5), 1239–1248. https://doi.org/10.1037/
0022-3514.83.5.1239
Montello, D. R., (2002). Cognitive map-design research
in the twentieth century: Theoretical and
empirical approaches. Cartography and Geographic
Information Science, 29(3), 283–304.
https://doi.org/10.1559/152304002782008503
Morita, T. (2004). Ubiquitous mapping in Tokyo. International
joint workshop on ubiquitous, pervasive
and Internet mapping (UPIMap2004), Tokyo,
Japan. https://www.geospatialweb.com/www.
ubimap.net/upimap2004/html/papers/UPIMap04-A-
01-Morita.pdf
Moore, R. (2005). Generation Ku: Individualism and
China’s millennial youth. Ethnology, 44(4), 357–
376. http://doi.org/10.2307/3774095
Morris, M., & Peng, K. (1994). Culture and cause:
American and Chinese attributions for social and
physical events. Journal of Personality and Social
Psychology, 67(6), 949–971. https://doi.org/
10.1037/0022-3514.67.6.949
Na, J., Grossmann, I., Varnum, M., Kitayama, S.,
Gonzalez, R., & Nisbett, R. (2010). Cultural differences
are not always reducible to individual differences.
Proceedings of the National Academy
of Sciences of the United States of America,
107(14), 6192–6197. https://doi.org/10.1073/
pnas.1001911107
Navon, D. (1977). Forest before trees: The precedence
of global features in visual perception. Cognitive
Psychology, 9(3), 353–383. https://doi.org/
10.1016/0010-0285(77)90012-3
Nelson, E., Dow, D., Lukinbeal, C., & Farley, R. (1997).
Visual search processes and the multivariate point
symbol. Cartographica, 34(4), 19–33. http://doi.org/
10.3138/15T3-3222-X25H-35JU
Nisbett, R. (2003). The Geography of thought: How
Asians and Westerners think differently… and
why. New York: The Free Press.
Nisbett, R., & Masuda, T. (2003). Culture and point of
view. Proceedings of the National Academy of
Sciences, 100(19), 11163–11170. https://doi.org/
10.1073/pnas.1934527100
Nisbett, R., & Miyamoto, Y. (2005). The Influence of
culture: Holistic versus analytic perception. Trends
in Cognitive Sciences, 9(10), 467–473. https://
doi.org/10.1016/j.tics.2005.08.004
Nisbett, R., Peng, K., Choi, I., & Norenzayan,A. (2001).
Culture and systems of thought: Holistic versus
analytic cognition. Psychological Review, 108(2),
291–310. https://doi.org/10.1037/0033-295x.108.
2.291
Norenzayan,A., Smith,E., Kim, B. & Nisbett, R. (2002).
Cultural preferences for formal versus intuitive
42 Studia Psychologica, Vol. 62, No. 1, 2020, 23-43
reasoning. Cognitive Science, 26(5), 653–684.
https://doi.org/10.1207/s15516709cog2605_4
Nosal, C. S. (1990). Psychologiczne modele umysłu.
[Psychological models of the mind]. Warsaw: PWN.
Ooms, K., De Maeyer, P., Dupont, L., Van der Veken,
N., Van de Weghe, N., & Verplaetse, S. (2016).
Education in cartography: What is the status of
young people’s map-reading skills? Cartography
and Geographic Information Science, 43(2), 134–
153. https://doi.org/10.1080/15230406.2015.
1021713
Opach, T., Popelka, S., Doležalová, J., & Rød, J. K.
(2018). Star and polyline glyphs in a grid plot and
on a map display: Which perform better? Cartography
and Geographic Information Science, 45(5),
400–419. https://doi.org/10.1080/15230406.
2017.1364169
Oyserman, D., & Lee, S. (2008). Does culture influence
what and how we think? Effects of priming
individualism and collectivism. Psychological Bulletin,
134(2), 311–342. https://doi.org/10.1037/
0033-2909.134.2.311
Oyserman, D., Coon, H., & Kemmelmeier, M. (2002).
Rethinking individualism and collectivism: Evaluation
of theoretical assumptions and meta-analyses.
Psychological Bulletin, 128(1), 3–72. https:/
/doi.org/10.1037/0033-2909.128.1.3
Peña, E. (2017). Mapping visual literacy [Doctoral
Dissertation]. Vancouver: The University of British
Columbia. http://doi.org/10.14288/1.0368982
Peterson, E., & Deary, I. (2006). Examining the
wholistic-analytic style using preferences in early
information processing. Personality and Individual
Differences, 41(1), 3–14. https://doi.org/10.1016/
j.paid.2005.12.010
Popelka, S., Stachoň, Z., Šašinka, Č., & Doležalová,
J. (2016). EyeTribe tracker data accuracy evaluation
and its interconnection with Hypothesis software
for cartographic purposes. Computational
Intelligence and Neuroscience, 2016, 1–14. https:/
/doi.org/10.1155/2016/9172506
Robinson, A. (1952). The look of maps: An examination
of cartographic design. Madison: University
of Wisconsin Press.
Sadahiro, Y. (1997). Cluster perception in the distribution
of point objects. Cartographica, 34(1), 49–
62. https://doi.org/10.3138/Y308-2422-8615-1233
Šašinka, Č., Morong, K., & Stachoň, Z. (2017). The
Hypothesis platform: An online tool for experimental
research into work with maps and behavior in
electronic environments. ISPRS International Journal
of Geo-Information, 6(12), 1–22. https://
doi.org/10.3390/ijgi6120407
Šašinka, Č., Stachoň, Z., Kubíček, P.,Tamm, S., Matas,
A., & Kukaňová, M. (2018). The Impact of global/
local bias on task-solving in map-related tasks
employing extrinsic and intrinsic visualization of
risk uncertainty maps. The Cartographic Journal,
55(4), 1–17. https://doi.org/10.1080/00087041.
2017.1414018
Singelis, T. (1994). The measurement of independent
and interdependent self-construal. Personality
and Social Psychology Bulletin, 20(5), 580–
591. https://doi.org/10.1177/0146167294205014
Slocum, T. A., McMaster, R. B., Kessler, F. C., &
Howard, H. H. (2005). Thematic cartography and
geographic visualization. New Jersey: Pearson
Prentice Hall.
Stachoň, Z., Šašinka, Č., Čeněk, J., Štěrba, Z.,
Angsuesser, S., Fabrikant, S., ... & Morong, K.
(2019). Cross-cultural differences in figure-ground
perception of cartographic stimuli. Cartography and
Geographic Information Science, 46(1), 82–94.
https://doi.org/10.1080/15230406.2018.1470575
Stachoň, Z., Šašinka, Č., Čeněk, J., Angsüsser, S.,
Kubíček, P., Štěrba, Z., & Bilíková. M., (2018). Effect
of size, shape and map background in cartographic
visualization: Experimental study on Czech
and Chinese populations. ISPRS International
Journal of Geo-Information, 7(427), 1–15. https:/
/doi.org/10.3390/ijgi7110427
Steele, L., & Lynch, S. (2013). The pursuit of happiness
in China: Individualism, collectivism, and subjective
well-being during China’s economic and
social transformation. Social Indicators Research,
114(2), 441–451. https://doi.org/10.1007/s11205-
012-0154-1
Takano, Y., & Osaka, E. (1999). An unsupported common
view: Comparing Japan and the US on individualism/collectivism.
Asian Journal of Social
Psychology, 2(3), 311–341. https://doi.org/10.1111/
1467-839X.00043
Takano, Y., & Osaka, E. (2018). Comparing Japan
and the United States on individualism/collectivism:
A follow-up review. Asian Journal of Social
Psychology, 21(4), 301–3016. https://doi.org/
10.1111/ajsp.12322
Tan, Y. (2016). East-West cultural differencies in
Visual Attention Tasks: Identifying multiple
mechanism and developing a predictive model
[Doctoral dissertation]. Michigan: Michigan Technological
University. https://digitalcommons.mtu.
edu/etdr/141
Triandis, H., & Gelfand, M. (1998). Converging measurement
of horizontal and vertical individualism
and collectivism. Journal of Personality and Social
Psychology, 74(1), 118–128. https://doi.org/
10.1037/0022-3514.74.1.118
Tu, Y. K., Gunnell, D., & Gilthorpe, M. S. (2008).
Simpson’s Paradox, Lord’s Paradox, and Suppres-
Studia Psychologica, Vol. 62, No. 1, 2020, 23-43 43
sion effects are the same phenomenon – the reversal
paradox. Emerging Themes in Epidemiology,
5(2), 1–9. https://doi.org/10.1186/1742-7622-
5-2
Tversky, B. (1981). Distortions in memory for maps.
Cognitive Psychology, 13(3), 407–433. https://
doi.org/10.1016/0010-0285(81)90016-5
Uskul, A., Kitayama, S., & Nisbett, R. (2008).
Ecocultural basis of cognition: Farmers and fishermen
are more holistic than herders. Proceedings
of the National Academy of Sciences of the
United States of America, 105(25), 8552–8556.
https://doi.org/10.1073/pnas.0803874105
Varnum, M., Grossmann, I., Katunar, D., Nisbett, R.,
& Kitayama, S. (2008). Holism in a European cultural
context: Differences in cognitive style between
Central and East Europeans and Westerners.
Journal of Cognition and Culture, 8(3), 321–
333. https://doi.org/10.1163/156853708X358209
Vignoles, V., Owe, E., Becker, M., Smith, P.,
Easterbrook, M., Brown, R., … & Bond, M. (2016).
Beyond the ‘east-west’ dichotomy: Global variation
in cultural models of selfhood. Journal of Experimental
Psychology: General, 145(8), 966–
1000. https://doi.org/10.1037/xge0000175
von Mühlenen, A., Bellaera, L., Singh, A., & Srinivasa,
N. (2018). The effect of sadness on global-local
processing. Attention, Perception, & Psychophysics,
80(5), 1072–1082. https://doi.org/10.3758/
s13414-018-1534-7
Wagemans, J., Elder, J., Kubovy, M., Pamer, M.,
Peterson, M., Sing, M., & von der Heydt (2012). A
Century of Gestalt psychology in visual perception:
I. Perceptual grouping and figure-ground organization.
Psychological Bulletin, 138(6), 1172–
1217. https://doi.org/10.1037/a0029333
Witkin, H., Moore, C., Goodenough, D., & Cox, P.
(1977). Field dependent and field independent
cognitive styles and their educational implications.
Review of Educational Research, 47(1), 1–64.
https://doi.org/10.3102/00346543047001001
Wood, D. (1984). Cultured symbols: Thoughts on the
cultural context of cartographic symbols.
Cartographica, 21(4), 9–37. https://doi.org/
10.3138/B88T-68X3-L0J1-0226
Yates, J., Ji, L-J., Oka, T., Lee, J-W., Shinotsuka, H.,
& Sieck, W. (2010). Indecisiveness and culture:
Incidence, values, and thoroughness. Journal of
Cross-cultural Psychology, 41(3), 428–444. https:/
/doi.org/10.1177/0022022109359692
Identification of altitude profiles in 3D geovisualizations:
the role of interaction and spatial abilities
Petr Kubíčeka
, Čeněk Šašinkab,c
, Zdeněk Stachoňa
, Lukáš Hermana
, Vojtěch Juříkb,c
,
Tomáš Urbánekc
and Jiří Chmelíkd
a
Department of Geography, Faculty of Science, Masaryk University, Brno, Czech Republic; b
HUME Lab, Faculty of Arts,
Masaryk University, Brno, Czech Republic; c
Department of Psychology, Faculty of Arts, Masaryk University, Brno,
Czech Republic; d
Department of Computer Graphics and Design, Faculty of Informatics, Masaryk University, Brno,
Czech Republic
ABSTRACT
Three-dimensional geovisualizations are currently pushed both by
technological development and by the demands of experts in various
applied areas. In the presented empirical study, we compared the
features of real 3D (stereoscopic) versus pseudo 3D (monoscopic)
geovisualizations in static and interactive digital elevation models. We
tested 39 high-school students in their ability to identify the correct
terrain profile from digital elevation models. Students’ performance was
recorded and further analysed with respect to their spatial abilities,
which were measured by a psychological mental rotation test and think
aloud protocol. The results of the study indicated that the influence of
the type of 3D visualization (monoscopic/stereoscopic) on the
performance of the users is not clear, the level of navigational
interactivity has significant influence on the usability of a particular 3D
visualization, and finally no influences of the spatial abilities on the
performance of the user within the 3D environment were identified.
ARTICLE HISTORY
Received 27 April 2017
Accepted 18 September 2017
KEYWORDS
3D geovisualizations;
stereoscopic and monoscopic
3D vision; spatial abilities;
level of navigational
interactivity
1. Introduction
The use of 3D technology in applied fields has increased rapidly in the last decades. Starting with the
vision of Digital Earth (DE; Gore 1998) and later with the advent of virtual globes (e.g. Google Earth),
3D geovisualizations became very popular in practical use under various labels and names. The original
idea of DE has developed, and in their vision for the next decade, Craglia et al. (2012, p. 18)
described DE as a ‘dynamic framework to share information globally’, targeting the improvement
of ‘the complex relationships between society and the environment we live in’. Such a shift of the
original paradigm emphasizes not only the technological aspects but also the user’’s ability to understand
and effectively harness the presented information.
However, the usability of 3D geovisualization as a tool in practical tasks still remains quite
unclear. 3D technologies are deployed in many geo-related areas such as teaching geography,
geology, urban planning, and emergency and crisis management (Hunter et al. 2016; Lin et al.
2015; Herman and Řezník 2015; Herbert and Chen 2015; Hirmas et al. 2014; Popelka and Dědková
2014; Řezník, Horáková, and Szturc 2013; Konečný 2011). Many applied areas usually emphasize the
precision of the human operator’’s situational judgement and decision-making, so the ways of
© 2017 Informa UK Limited, trading as Taylor & Francis Group
CONTACT Petr Kubíček kubicek@geogr.muni.cz Department of Geography, Faculty of Science, Masaryk University,
Kotlářská 2, 611 37 Brno, Czech Republic
Supplemental data for this article can be accessed at https://doi.org/10.1080/17538947.2017.1382581
INTERNATIONAL JOURNAL OF DIGITAL EARTH
2019, VOL. 12, NO. 2, 156–172
https://doi.org/10.1080/17538947.2017.1382581
representing information in the virtual environment (VE) play an important role in the cognitive
processing during tasks. Compared to two-dimensional cartographic visualizations, there are still
no clear standards or design principles for the creation of 3D cartographic visualizations (Pegg
2011; Haeberling, Bär, and Hurni 2008; MacEachren 1995). The cartographic principles for 3D cartographic
visualization are a current issue discussed in several studies (Lokka and Cöltekin 2017;
Hájek, Jedlička, and Čada 2016; Sieber et al. 2016; Semmo et al. 2015 ). Also, most of the research
on 3D maps was done only on participants who were highly skilled in map reading (Pegg 2011).
Moreover, the results of many studies indicate ambiguity in 3D cartographic visualization usability
and the need for extended research (e.g. Bleisch and Dykes 2015; Seipel 2013; Pegg 2011).
The particular factors influencing the effective and efficient design of virtual 3D environments
have been analysed from different viewpoints. Regarding this, Lokka and Cöltekin (2016) presented
an analytical study identifying three distinctive groups of key factors: (1) users (spatial abilities, age,
gender – both on the general and individual levels), (2) type of stimuli (including the visualization,
level of realism, type of interactivity, and other factors), and (3) tasks (particular context of use,
specific scenario). Earlier Šašinka (2012) introduced a similar triarchic model where they emphasized
the importance and interaction of all three of the above-mentioned categories when working
with a map. In their triarchic structural model, they suggested that, e.g. the cognitive strategy used
when dealing with particular task depends on the level of map literacy (user group factor), on visualization
type (type of stimuli group factor) and also on the nature of the problem which is solved
(task group factor).
The focus of this article is a synthesis of two selected stimuli aspects (level of interaction and visualization
type) influencing the usefulness of 3D visualization. Based on the previous findings (Špriňarová
et al. 2015; Herman and Stachoň 2016; Klippel, Weaver, and Robinson 2011), in our study we
explored perception of 3D terrain geovisualizations displayed in real 3D (stereoscopic) and pseudo
3D (monoscopic), in both static and interactive forms. General research questions were defined as
follows:
. Does the type of 3D visualization (monoscopic/stereoscopic) influence the performance of the
users?
. Does the level of navigational interactivity influence the usability of 3D visualization?
. What is the role of personal spatial abilities in the process of solving 3D visualization tasks?
For the purpose of the study, we developed a special computer testing interface. In this study, we
tested participants with low levels of map-reading experience.
2. Related work
2.1. Principles of 3D vision
3D visualizations contain a number of visual cues. Some static monocular depth cues are embedded
in the 3D visualizations (e.g. lighting and shading cues, occlusion/interposition, relative size of
objects, linear perspective, texture gradients, and aerial perspective). Dynamic monocular depth
cues, which are considered extremely effective for depth information recognition (Loomis and
Eby 1988), include motion parallax, i.e. movement of objects, the viewer, or both in the scene; in
cartography also referred to as kinetic depth (Willett et al. 2015). Such mutual movement can indicate
depth information in the viewer’’s visual field through the perception of different distance
changes between the viewer and the objects. In some specific visualization environments, we can
also use binocular depth cues, which are represented by binocular disparity, in which case we can
see a slightly different view of the scene with each eye separately. The result is synthesized in our
brain, creating a 3D perception of the scene. Binocular disparity and convergence are usually provided
by 3D glasses or another stereoscopic technology such as head-mounted displays.
INTERNATIONAL JOURNAL OF DIGITAL EARTH 157
Based on the number of visual cues used in the visualization, we can distinguish between real and
pseudo 3D visualization (Bowman et al. 2005). Pseudo 3D (monoscopic) visualizations use only
monocular cues, whereas real 3D (stereoscopic) visualizations include both binocular and monocular
depth cues (Buchroithner and Knust 2013). The added value for real 3D visualization is stereoscopy,
provided by binocular depth cues (namely binocular disparity). Stereoscopy is ensured by the use of
computer graphics and specific peripheral devices for 3D vision such as 3D glasses. Pseudo 3D
(monoscopic) visualization is displayed perspective-monoscopically on flat media or on the computer
screen (Buchroithner and Knust 2013).
These two types of visualizations can be considered informationally equivalent, i.e. they can present
exactly the same amount of spatial information (Larkin and Simon 1987). However, they are not
computationally equivalent, because each of them uses different cognitive processing of the perceived
information. When perceiving binocular depth cues in a real 3D VE, the user is expected
to better estimate spatial placement/distribution in the scene (Qian 1997; Landy et al. 1995). Nevertheless,
previous studies dealing with real 3D visualization (e.g. Ware and Franck 1996; Zanola, Fabrikant,
and Cöltekin 2009; Torres et al. 2013; Špriňarová et al. 2015) reported inconclusive
implications for practical use of real 3D technology.
2.2. Interactivity in three-dimensional visualizations
Interactivity in 3D geovisualizations presents the possibility of manipulating the geovisualization. In
this article, we focus only on navigational interactivity. Navigational or viewpoint interactivity is
defined for example by Roth (2012) and it is the core functionality of applications like Google
Earth. Navigational interactivity in 3D space usually includes functions like rotation, panning,
and zooming. Interactive (movable, navigable) 3D environments provide then the above-mentioned
kinetic depth cues (motion parallax), emphasizing the user’’s 3D perception. From a distributed cognition
theory point-of-view (Hutchins 1995), we can see the interaction process as the behaviour
where both internal (inner mind) and external (in the world, within some external medium) cognitive
processes occur. Regarding this, in applied areas, the user interface should always be designed
with maximal attention to provide optimal information reach, securing the user’’s situational awareness.
Only a few experiments regarding interactive 3D environments (their features and usability)
have been published, and navigational interactivity in 3D environments was not necessarily found
to be advantageous (Keehner et al. 2008). For example, Wilkening and Fabrikant (2013) used Google
Earth to help participants solve practical tasks, and measured and analysed movement types applied
by the user. Bleisch, Dykes, and Nebiker (2008 ) assessed a potential combination of a 3D environment
and abstract data, when comparing differences between reading the height of 2D bar charts and
reading bar charts placed in an interactive 3D environment. Herman and Stachoň (2016) published a
pilot test comparing static and interactive 3D visualization, and brought enriching methodology for
data analysis.
Interactive visualization (i.e. with navigational interactivity) offers more ways to retrieve information
than static visualization (i.e. without navigational interactivity). From this point-of-view,
the navigational interactivity can be understood as the next frontier of visualization. In Figure 1,
we can see the different ways 3D geovisualization offers a different number and variety of visual
cues for four different settings: static pseudo 3D, static real 3D, interactive pseudo 3D, and interactive
real 3D geovisualization. The above-mentioned monocular depth cues are ensured by the use of
computer graphics and represent the base in all four settings. In the interactive setting, kinetic
depth is added, and the real 3D setting includes a cue of binocular disparity. In interactive real
3D environments, it is especially the cues of binocular disparity and navigational options (i.e. panning,
zooming, and rotating the geovisualization) allowing the user to acquire more information
about the terrain (e.g. kinetic depth effect). Although both mentioned types of visual cues promote
effective depth perception when used separately, they can be mutually redundant. We presume that
the use of both these visual cues simultaneously don’’t further improve task performance.
158 P. KUBÍČEK ET AL.
2.3. Spatial abilities of users
The role of individual differences in the context of 2D geovisualization use was explored in several
studies (e.g. Kubíček et al. 2017; Stachoň et al. 2013; Griffin et al. 2006; Allen, Cowan, and Power
2006). Individuals vary not only with respect to the level of map literacy (Rautenbach, Coetzee,
and Cöltekin 2017), domain-specific expertise (Svatoňová 2016), or cognitive style (Kozhevnikov
2007), but also with respect to their cognitive abilities – for instance, their working memory capacity
(Daneman and Carpenter 1980) or spatial ability (Tartre 1990). The role of individual differences
among users increasingly shifts in focus, leading us to seek to understand whether or not all users
profit equally from visualizations and if they do not, how different users can be supported. Klippel,
Weaver, and Robinson (2011) substantiated the role of individual differences for cognitive concepts
and proposed a methodology to measure them, stressing both the research methodology aspect and
the technological push towards the individualization of visual representations. Höffler (2010) developed
a meta-analytical review comparing spatial abilities and learning with visualizations. His
results, based on 27 different experiments from 19 independent studies, supported a slight advantage
of high spatial ability learners in working with visual stimuli, but also demonstrated that this advantage
disappears if users with low spatial abilities work with a dynamic environment and 3D
visualization.
3. Research aim
In this study, we took into account the above-mentioned models (Šašinka 2012; Lokka and Cöltekin
2016) and explored human navigational interaction with the 3D environments, investigating all of
Figure 1. Additional depth cues available in different kinds of environments.
INTERNATIONAL JOURNAL OF DIGITAL EARTH 159
the previously mentioned key factors. We designed an experiment where we measured the individual
participants’ spatial abilities (user issue) before they solved altitude identification tasks in different
types of 3D visualizations (real and pseudo 3D) and with different types of control/manipulation
possibilities (static and interactive – see Figure 1). We expected that in a real (stereoscopic) 3D visualization
participants (1) would more accurately estimate the altitude features of the scene (i.e. altitude
profiles) when they solve tasks in static (non-moveable) virtual 3D environments. The
binocular depth cues included in the real 3D visualization were expected to help ‘real 3D’ users
make more precise estimations. In the interactive (moveable) 3D environments, on the other
hand, (2) the performance of the real 3D group and the pseudo 3D group was expected to be similar
due to the possibility of manipulating the geovisualization. When moving the scene, kinetic depth
cues are available for users to correctly judge altitude features in the scene, and in the case of the
pseudo 3D participants, the kinetic depth cues are expected to fully compensate for missing binocular
cues. Furthermore, users were expected to differ in terms of their individual spatial abilities, which
could influence their performance in particular tasks. We measured participants’ spatial abilities with
the use of a mental rotation test (MRT), and (3) expected a close correlation between individuals’
spatial abilities and their performance in the virtual geographical model.
4. Methods
4.1. Participants
Thirty-nine volunteers (18 females and 21 males; aged 16–18) were recruited from two high schools
in Brno (the Czech Republic). The data were collected during October and November 2016. Before
the testing, all the participants and their parents were introduced to the research proposal and study
design, which was previously approved by the ethics commission of Masaryk University. All but four
participants had some previous basic or intermediate experience with 3D visualization applications,
such as watching 3D movies or playing interactive games on a computer, but none of them had
experience in interacting with 3D geographical data as used in this study. The participants were randomly
divided into two groups with an equal proportion of males and females, in order to balance
out the suggested differences between males and females in mental rotation tasks. The experimental
conditions (including lighting conditions and other external factors) were identical for both groups.
All participants had normal or corrected-to-normal vision, and had no motor/movement limitations.
We did not perform an objective visual impairment testing, but no participants reported colour
vision impairment during the initial test questionnaire or post-test individual interviews. All
participants agreed with the experimental procedure and participated voluntarily, with the open
opportunity to withdraw from the testing at any time.
4.2. Materials and geographical data
The test was administered with the testing platform hypothesis (Štěrba et al. 2015; Popelka et al. 2016).
For the main experiment, a new testing application was developed based on the Unity® game engine.1
The application supports real-time rendering of large 3D geographical models using both monoscopic
and stereoscopic displays, and also automates data collection for further analysis. For each task, the
user’’s answers and task performance time were logged. After the testing session, data were exported
to a structured text file and processed using the R statistical software package.
All 39 participants were tested individually using the same hardware setup: a desktop PC and 27′′
display compatible with NVIDIA 3D Vision technology. Users were instructed to put on/off shutter
glasses before each section of real 3D/pseudo 3D tasks. A generic PC mouse was used as an input
device. In order to keep interaction straightforward for users, we chose an interaction scheme consisting
of only three types of actions: orbiting, dragging, and zooming. These actions were controlled
by the left button, right button, and mouse wheel, respectively. The range of rotation, panning, and
160 P. KUBÍČEK ET AL.
zooming was limited to prevent participants from accidentally moving the terrain completely out of
sight or moving a virtual camera below the terrain. Digital terrain models (DTMs) were used for generating
3D models of terrain. A fourth-generation Digital Terrain Model of the Czech Republic
(DTM 4G) was acquired by airborne laser scanning and processed to a ground resolution of 5 ×
5 m. DTM 4G is now being distributed by ČÚZK (Czech Office for Surveying, Mapping and Cadastre).
Six randomly chosen DTMs from various parts of the Czech Republic were used as an input for
the experiment. Surfaces with textures were computed from the input DTM and corresponding
orthophoto data. 3D models were vertically scaled by variable Z factors (one 3D model had a Z
of factor 1, one model a Z of factor 2, three models had a Z factor of 3, and one model had a Z factor
of 4), in order to highlight relatively small variation in altitude in the landscapes.
4.3. Procedure design
The experimental procedure consisted of three main parts. In the first part, we measured the mental
rotation skills of all participants. The second part was the main experiment and included tasks where
participants were asked to work with 3D geovisualizations to estimate altitude profiles. In this second
part, we measured participants’ abilities to estimate profiles of geographical models in two stages. In
the first stage, both static 3D geovisualizations were presented and participants’ estimation of altitude
profiles was based only on visual stimuli, with no possibility to interact with the visualizations.
The second stage presented interactive environments where participants could manipulate the 3D
geovisualizations to estimate the altitude profile. The experiment finished with a third part, which
followed immediately after the altitude profile estimation. Using rating scales, participants were
briefly interviewed by the experiment operator about their general feelings, previous experience
with 3D interactive environments, potential subjective task-solving improvements, and subjective
preferences about the task-solving environment.
Participants were randomly divided into two groups, which both used the same geographical
models, but in a different task order. Tasks were partially counterbalanced by swapping the order
of pseudo 3D (monoscopic) and real 3D (stereoscopic) tasks. See Figure 2 for an overview of the
experimental design, and see the supplementary video for a detailed understanding of both tests
(the supplementary video is available online at https://youtu.be/ddL6gInQFh4). We used a
within-subjects 2 × 2 factorial controlled experiment examining stereoscopic versus monoscopic displays
(Factor 1: Visualization) and static versus interactive displays (Factor 2: Interaction). All participants
experienced all four combinations (static pseudo 3D, static real 3D, interactive pseudo 3D,
and interactive real 3D).
4.3.1. Pilot study
We initially conducted a pilot study in order to validate our research tools. At a public scientific event
(Researchers’ Night 2016, Brno), 17 participants (11–45 years) tested the application we developed
Figure 2. Design of the experiment.
INTERNATIONAL JOURNAL OF DIGITAL EARTH 161
for testing purposes. We observed users’ interactions with the application and we also interviewed
participants about the usability of the application and level of subjectively experienced difficulty.
Based on this experience, participants found the developed application very user-friendly, stating
that even 11-year-old children were able to go through the entire procedure without difficulty.
The overall accuracy of the answers throughout the pilot testing was 72%, which indicated an appropriate
difficulty level of items and applicability of the research design. After the pilot study, we made
some minor adjustments to the instructions and examples (grammar revision, graphic resolution of
figures, simplifying of instructions). The test tasks (stimuli, navigation, etc.) were used without any
changes.
4.3.2. Task and stimuli
The test arrangement was designed specifically for the purpose of this study. A MRT was used for the
test of spatial abilities. The version of the MRT used in the study was developed by Ivana Žahourová
as part of a new test battery focused on object-spatial cognitive style (Vidláková 2010). The MRT
consists of two sub-tests (see Figure 3). There are eight tasks in the first visual comparison subtest.
In each task, two objects are displayed simultaneously, and the participant should judge whether
the objects are identical or not (see Figure 4). In the second sub-test, the memory recall sub-test, the
model figure is presented for a given time, and then four other figures are presented individually.
Participants should again judge whether the figures that follow are identical to the model figure
or not (see Figure 5). The total number of correct answers is an indicator of spatial orientation ability
(Ekstrom et al. 1976). The MRT developed by Žahourová was inspired by and based on concepts
from two other psychological tests (Vandenberg-Kuse MRT – Vandenberg and Kuse 1978; and
the Measure of the Ability to Rotate Mental Images – Campos 2012), both of which are very
well-grounded in the psychological literature and theories (e.g. Ganis and Kievit 2015).
The second part of the testing battery included a non-interactive (static) stage and an interactive
stage. In the interactive stage, participants were asked to estimate the profiles of digital elevation
models displayed in static real 3D (stereoscopic), and static pseudo 3D (monoscopic) visualizations.
In order to avoid the learning effect, the order of the real 3D and pseudo3D sessions was swapped for
group 1 and group 2 (see Figure 2). As seen in Figure 6, each session consisted first of two training
tasks, followed by six experimental tasks.
All tasks included the 3D geographical model with a given task and an answer sheet in one screen,
as seen in Figure 7. Two types of tasks, which we labelled ‘1:3’ and ‘3:1’, were alternated in the testing
battery. ‘1:3’ tasks displayed a set of two interconnected coloured marks in the geographical terrain
on the left part of the screen (Figure 7). On the right side of the screen, there were three figures with
different profiles. Participants were asked to determine which of the profiles matched the real profile
between the marks in the terrain.
‘3:1’ tasks displayed three pairs of interconnected coloured marks set in the geographical terrain in
the left part of the screen (Figure 8). On the right side, there was a single profile, and participants
Figure 3. The first part of the experiment, the MRT.
162 P. KUBÍČEK ET AL.
were asked to determine which one of the three pairs of displayed interconnected coloured marks
shown in the terrain in the left part of the screen matched the given profile.
After the end of the testing battery, we asked participants a series of questions focused on their
subjective opinions about the different types of interactive 3D visualizations.
5. Results
The recorded information about the users’ performance was statistically analysed. Repeatedmeasures
(within-subjects) ANOVA was used for the evaluation of correct answer rates for the
Figure 4. An example of the task from the visual comparison sub-test.
Figure 5. An example of the model figure and figure that follows it from the memory recall sub-test.
INTERNATIONAL JOURNAL OF DIGITAL EARTH 163
altitude profile identification with factors of static–interactive, pseudo 3D–real 3D, and their combinations
(Figure 9). The effects of the order of the pseudo 3D–real 3D conditions were not significant
in any of the task combinations (four t-tests brought non-significant results), so that we could
analyse the two groups together. Only the difference between the static and interactive tasks was
Figure 6. Detailed schema of the second part of the experiment.
Figure 7. An example of a ‘1:3’ task. The terrain model with a single pair of connected points is rendered on the left side of the
screen. Three different altitude profiles are displayed on the right side – one of them corresponds to the terrain in the model.
Screenshot from the testing application.
164 P. KUBÍČEK ET AL.
found to be statistically significant (F = 17.068; df = 1; p < .001; η2
= 0.130). Neither the difference
between pseudo and real 3D (F = 0.298; df = 1, p = .586; η2
= 0.003) nor the interaction of the two
factors (3D cue type and interactivity) was significant (F = 0.152; df = 1; p = .697; η2
= 0.001). We
repeated the analyses of the main effects with t-tests with the same results both for static and interactive
tasks (t = 2.972, df = 152.3, p-value = .003) and for the pseudo and real 3D conditions (t = –
0.382, df = 151.9, p-value = .703).
Figure 8. Example of a ‘3:1’ task. The terrain model with three pairs of connected points is rendered on the left side of the screen. A
single altitude profile is displayed on the right side. One of the three terrain profiles between the connected points corresponds to
the altitude profile. Screenshot was taken from the testing application.
Figure 9. Number of correct answers in the altitude profile estimation. N = 39 in all conditions. Pseudo 3D (static M = 5.00, SD =
1.34; interactive = 5.74, SD = 1.33) and real 3D (static M = 5.15, SD = 1.66; interactive = 5.77, SD = 1.39).
INTERNATIONAL JOURNAL OF DIGITAL EARTH 165
Repeated-measures ANOVA was also used to analyse response times in the altitude profile task
(Figure 10). Again, only the difference between the static and interactive tasks was found to be statistically
significant (F = 21,393; df = 1; p < .001; η2
= 0.158). Neither the difference between pseudo
3D and real 3D (F = 0.061; df = 1, p = .805; η2
= 0.001) nor the interaction of the two factors (F =
0.547; df = 1; p = .461; η2
= 0.005) were statistically significant. We repeated the analyses of the
main effects with t-tests with the same results both for static and interactive tasks (t = 3.267, df =
132.8, p-value = .001) and for the pseudo and real 3D conditions (t = 0.169, df = 153.9, p-value
= .866).
We also performed additional analyses to explore the dataset and look for some potential trends
or patterns. Among others, we compared alternative types of tasks (e.g. 1:3 and 3:1) in order to determine
whether there were differences in difficulty. We didn’’t find any significant differences in the
response time (1:3 m = 18.72 s; 3:1 m = 20.04 s) or accuracy (1:3, 71.7%; 3:1, 79.7%). It seems that the
variability in the performance depends above all on the concrete scenes and landscape profiles. We
also analysed the data to check for potential learning effects. We addressed this question by means of
comparing the results (response time and correctness) for the first and the second sessions of each
stage. The differences were not statistically significant for either the static or the interactive stage (see
Figure 6). Neither was there a recognizable trend in the response time nor in the accuracy between
sessions, thus we assumed no learning effect was caused by the order of static and interactive stages.
In order to identify potential gender differences, the Welch Two Sample t-test was used for a comparison
of correct answers in altitude profile task between males and females (Table 1).
The Welch Two Sample t-test was also used for the comparison of performance in the MRT
between males and females (Table 2).
The Pearson correlation coefficient was used to assess the relationship between the total score in
the MRT and performance in geographical tasks. Males (n = 21) and females (n = 18) were tested
separately. Among female participants, no correlation was found between their total scores on the
MRT and geographical tasks. A low, positive but insignificant correlation was found in males
(Table 3).
Figure 10. Response time (s) in the altitude profile task. N = 39 in all conditions. Pseudo 3D (static M = 16.30, SD = 5.86; interactive
= 20.85, SD = 8.89) and real 3D (static M = 16.72, SD = 5.79; interactive = 20.01, SD = 8.94).
166 P. KUBÍČEK ET AL.
In the third part of the experiment, with the help of Likert scales, participants rated which type of
environment (visualization and interaction) was most difficult for them (see Table 4). Subsequently,
they chose which type of task – ‘1:3’ or ‘3:1’ – seemed to be easier (see Table 5). The experiment finished
with a short discussion led by the experiment facilitator. Participants gave verbal feedback
about the experiment procedure. They mostly reported that the most difficult part of the cartographic
experiment was the static pseudo 3D environment. Some participants experienced difficulties
with the manipulation (interaction) of the 3D geographical models, while others reported
discomfort using the shutter glasses. They perceived irritating flickering or they experienced a slight
headache. However, nobody reported more extreme physiological states that might have led to premature
interruption of experiments or the failure of specific parts of the experiment.
6. Discussion
This experiment focused on user performance with different types of 3D visualizations within VEs
with different levels of navigational interactivity. The research followed outcomes from recently
Table 1. Comparison of males (N = 21) and females (N = 18) in the number of correct answers in the altitude profile task.
M Males M Females SD Males SD Females t df p
Static pseudo 3D 5.33 4.61 1.24 1.38 1.709 34.59 .096
Static real 3D 5.43 4.83 1.57 1.76 1.108 34.47 .276
Interactive pseudo 3D 5.71 5.78 1.45 1.22 –0.149 36.98 .883
Interactive real 3D 6.14 5.33 1.56 1.03 1.938 34.90 .061
Total number of correct answers 22.62 20.56 4.57 4.30 1.452 36.64 .155
Table 2. Comparison of males (N = 21) and females (N = 18) in the number of correct answers in the MRT.
M Males M Females SD Males SD Females t df p
Visual comparison sub-test 5.95 4.94 1.43 1.86 1.871 31.67 .071
Memory recall sub-test 5.38 4.17 1.63 1.62 2.330 36.16 .025
Total number of correct answers 11.33 9.11 2.56 2.63 2.664 35.74 .012
Table 3. Correlation between MRT (total score) and performance in altitude estimation tasks (correct answers).
MRT total score (males) MRT total score (females)
r p r p
Pseudo 3D and Static 0.06 .8 0.05 .86
Real 3D and Static 0.10 .67 –0.12 .63
Pseudo 3D Interactive 0.27 .24 0.14 .59
Real 3D and Interactive 0.29 .2 –0.10 .69
Total number of correct answers 0.23 .31 –0.02 .93
Table 4. Rating of subjectively experienced difficulty for all conditions in the geographical tasks.
Static Interactive
Mean Median Mean Median
Pseudo 3D 2.74 3 2.08 2
Real 3D 2.13 2 1.49 1
Note: A 5-point Likert scale (1 = very easy, 5 = very difficult) was used.
Table 5. Preference of participants (N = 39) for 1:3 and 3:1 task types, respectively.
Preference of 1:3 type of task No preference 3:1 type of task
Frequency 17 10 12
INTERNATIONAL JOURNAL OF DIGITAL EARTH 167
published articles, e.g. Špriňarová et al. (2015), Herman and Stachoň (2016), and Klippel, Weaver,
and Robinson (2011), and was performed using the Hypothesis software and on a custom-built platform
based on the Unity® game engine.
The results presented in the previous section provide the following answers to the established
research questions.
Our main interest was the level of navigational interactivity. The results showed that even low
levels of interface complexity (pan, zoom, rotate) strongly influence the ability of a particular 3D
visualization to support good performance on an altitude profile task. The evaluation of correct
answers and the response times showed statistically significant differences only between the static
and interactive factors, contrary to the conclusions of previously published papers (e.g. Keehner
et al. 2008). The main implication of Keehner et al.’’s study was the finding that more interactivity
may not help to solve the tasks more effectively because users need not be able to use it an effective
way. The authors also pointed out the individual differences between people and their particular ability
to use interactive visualization. However, we have also found several differences between both
studies which are worth mentioning. First, the level of navigational interactivity and available
tools is different. While Keehner et al. (2008) provided only rotation, we incorporated zoom in
and out, move, and both horizontal and vertical rotation in our study. Second, our interface was
also positively constrained and did not allow the vertical rotation of more than 90° (to avoid the
upside down view on the terrain). This type of ‘hard constraint’ could also allow individual user
to easily disclose the appropriate view and accomplish the task effectively. Moreover, the user
group consisted of teenagers using digital technology on a day-to-day basis and experienced in
3D gaming and other applications. Taking into account the aforementioned differences, the users
were more likely to use the interactivity in a more effective way on a general level. Other interaction
primitives (Roth 2012) used for exploratory geovisualization, such as layer switching, transparency
setting, brushing, or displaying detail-on-demand can be also used in the interactive 3D environment
of spatial data (Herman and Řezník 2015; Sieber et al. 2016).
In addition to the influence of navigational interactivity, we focused onthe influences of spatial abilities
on user performance within the 3D environment. The results from the MRT showed a low but
insignificant correlation only in the case of male participants. In case of female participants, no correlation
was found. It might indicate that men and women use different cognitive strategies when solving
spatially oriented tasks within 3D environments. This interpretation is consistent with the theory
of spatial and object imagery, where Blazhenkova and Kozhevnikov (2009) mention that males are
relatively more frequent ‘spatial imagers’ and vice-versa, such that females could be described relatively
more frequently as ‘object imagers’. The MRT is an indicator of the level of ‘spatial imagery’.
Another issue was devoted to the differences between the two task types. We expected the 1:3 tasks
and the 3:1 tasks (compare Figures 7 and 8) to produce different levels of cognitive load. We assumed
that the 3:1 task would require higher cognitive load because of the necessity to mentally create three
different 3D lines on the terrain and compare them with the suggested altitude profiles. Therefore, we
hypothesized that we would observe longer response times in the case of 3:1 tasks. The results showed
that there were no significant differences in the response time and correctness between the task types.
Finally, an influence of the type of 3D visualization (pseudo (monoscopic)/real (stereoscopic)) on
the performance of users was observed. The differences in the use of real 3D visualizations show
slightly better, but not significantly different, results as measured by rate of correct answers. Therefore,
the influence of the type of 3D visualization is still unclear, with neither method clearly performing
better. This finding is consistent with those of Ware and Franck (1996), Zanola,
Fabrikant, and Cöltekin (2009), Torres et al. (2013), or Špriňarová et al. (2015).
The comparison of objective results with subjective user preferences was also interesting. The
pseudo 3D (monoscopic) static task had the worst subjective evaluation. However, the differences
between real and pseudo 3D tasks were insignificant based on objective results. On the other
hand, the real 3D static and pseudo 3D interactive environments obtained similar subjective evaluations
(2.13 and 2.08 on the Likert scale, Table 4), but the objective results demonstrated
168 P. KUBÍČEK ET AL.
considerable differences in users’ performance (Figure 9). This could indicate that users have a tendency
to overestimate the importance of control method compared with visualization type, which
can be explained by cognitive biases, e.g. illusion of control (Thompson 1999). Such an effect of
incongruence between subjective rating and objective performance stresses the importance of conducting
objective empirical user studies.
7. Conclusion
Based on the results interpreted in the discussion, we can conclude that the major influence on correct
answers was navigational interactivity, which provides additional depth cues. However, the
decisions performed in the static environments were made faster than those made in the interactive
environment.
We did not observe any significant positive effect of real 3D (stereoscopic) in either the static or
the interactive environment. Contrary to this finding, Juřík et al. (2017) provide evidence of a positive
influence of real 3D visualization on relative point altitude evaluation. Both experiments used
almost identical environments and spatial data, but the tasks were different. Based on our results,
it appears that for certain types of tasks (as in our case, working with terrain profiles), it is better
to use an interactive visualization, no matter whether it is real 3D or pseudo 3D. The advantage
of pseudo 3D visualization in this case is that it is literally cheaper because it does not require specific
hardware. An interactive pseudo 3D visualization can be produced simply by developing appropriate
software. It can be easily modified and currently includes a broad spectrum of technologies, ranging
from web-based applications to those available on the desktop. Summarizing our findings from the
3D visualization design viewpoint, we can recommend the researchers concentrate on the (navigational)
interactivity instead of stereoscopic visualization. Our conclusions are in line with the current
report of Roth et al. (2017) mentioning the necessity to investigate the strategies to compare static
and interactive maps and also to evaluate the interactivity in map use cases.
The individual differences among users mentioned by Keehner et al. (2008) are one of the open
issues to be solved. We were not able to log out the view at our interaction data when the participants
found the ‘best’ angle to see what was needed for the actual judgement. However, we are currently
working on tools which enable us to timestamp and measure types of interaction (zoom, move,
rotate) (see, e.g. Herman and Stachoň 2016; Herman et al. 2016). This tool will also enable us to
reconstruct the view used for the final judgement and describe the users’ performance and strategies
on the inter-individual level.
Our conclusions are limited to the particular tasks tested in this experiment – a strong influence of
task dependency – and are also limited by the relatively specific group of participants tested in this
experiment, which consisted of high-school students with limited map-reading expertise. These
‘digital natives’ have frequent experience with 3D visualizations, at least with pseudo 3D ones. So,
it is possible that they had previously learned how to work effectively with 3D applications. We
are planning to focus on different user groups and to extend the complexity of tasks solved by
the participants in future work. We want to perform further experiments with these factors modified
using another two vertices of the previously described triarchic model.
Note
1. The Unity game engine has recently been frequently used for the development of serious games and other
research applications, in addition to commercial games. For more details, see: https://unity3d.com/.
Acknowledgements
We would like to thank our test participants for their valuable time and two anonymous reviewers for their helpful and
constructive comments.
INTERNATIONAL JOURNAL OF DIGITAL EARTH 169
Disclosure statement
No potential conflict of interest was reported by the authors.
Funding
This research was funded by projects ‘Influence of cartographic visualization methods on the success of solving practical
and educational spatial tasks’ [grant number MUNI/M/0846/2015] and ‘Integrated research on environmental
changes in the landscape sphere of Earth II’ [grant number MUNI/A/1419/2016], both awarded by Masaryk University,
Czech Republic.
References
Allen, G. L., C. R. M. Cowan, and H. Power. 2006. “Acquiring Information from Simple Weather Maps: Influences of
Domain-specific Knowledge and General Visual–spatial Abilities.” Learning and Individual Differences 16 (4): 337–
349. doi:10.1016/j.lindif.2007.01.003.
Blazhenkova, O., and M. Kozhevnikov. 2009. “The New Object-spatial-verbal Cognitive Style Model: Theory and
Measurement.” Applied Cognitive Psychology 23 (5): 638–663. doi:10.1002/acp.1473.
Bleisch, S., and J. Dykes. 2015. “Quantitative Data Graphics in 3D Desktop-based Virtual Environments – An
Evaluation.” International Journal of Digital Earth 8 (8): 623–639. doi:10.1080/17538947.2014.927536.
Bleisch, S., J. Dykes, and S. Nebiker. 2008. “Evaluating the Effectiveness of Representing Numeric Information
Through Abstract Graphics in 3D Desktop Virtual Environments.” The Cartographic Journal 45 (3): 216–226.
doi:10.1179/000870408X311404.
Bowman, D. A., E. Kruijff, J. J. LaViola, and I. Poupyrev. 2005. 3D User Interfaces: Theory and Practice. 1st ed.
Redwood City, CA: Addison Wesley Longman.
Buchroithner, M. F., and C. Knust. 2013. True-3D in Cartography – Current Hard and Softcopy Developments. In
Geospatial Visualisation, edited by A. Moore and I. Drecki, 41–65. Heidelberg: Springer. doi:10.1007/978-3-642-
12289-7_3.
Campos, A. 2012. “Measure of the Ability to Rotate Mental Images.” Psicothema 24 (3): 431–434.
Craglia, M., K. de Bie, D. Jackson, M. Pesaresi, G. Remetey-Fülöpp, C. Wang, A. Annoni, et al. 2012. “Digital Earth
2020: Towards the Vision for the Next Decade.” International Journal of Digital Earth 5 (1): 4–21. doi:10.1080/
17538947.2011.638500.
Daneman, M., and P. A. Carpenter. 1980. “Individual Differences in Working Memory and Reading.” Journal of Verbal
Learning and Verbal Behavior 19 (4): 450–466. doi:10.1016/S0022-5371(80)90312-6.
Ekstrom, R. B., J. W. French, H. H. Harman, and D. Dermen. 1976. Manual for Kit of Factor-referenced Cognitive Tests.
1st ed. Princeton, NJ: Educational Testing Service.
Ganis, G., and R. Kievit. 2015. “A New Set of Three-dimensional Shapes for Investigating Mental Rotation Processes:
Validation Data and Stimulus Set.” Journal of Open Psychology Data 3 (1): e3. doi:10.5334/jopd.ai.
Gore, A. 1998. “The Digital Earth: Understanding Our Planet in the 21st Century.” Accessed January 4, 2017. http://
www.isde5.org/al_gore_speech.htm.
Griffin, A. L., A. M. MacEachren, F. Hardisty, E. Steiner, and B. Li. 2006. “A Comparison of Animated Maps with Static
Small-multiple Maps for Visually Identifying Space-time Clusters.” Annals of the Association of American
Geographers 96 (4): 740–753. doi:10.1111/j.1467-8306.2006.00514.x.
Haeberling, C., H. Bär, and L. Hurni. 2008. “Proposed Cartographic Design Principles for 3D Maps: A Contribution to
an Extended Cartographic Theory.” Cartographica 43 (3): 175–188. doi:10.3138/carto.43.3.175.
Hájek, P., K. Jedlička, and V. Čada. 2016. “Principles of Cartographic Design for 3D Maps – Focused on Urban Areas.”
In Proceedings, 6th International Conference on Cartography and GIS (Vols. 1 and 2), edited by T. Bandrova and M.
Konečný, 297–307. Sofia, Bulgaria: Bulgarian Cartographic Association.
Herbert, G., and X. Chen. 2015. “A Comparison of Usefulness of 2D and 3D Representations of Urban Planning.”
Cartography and Geographic Information Science 42 (1): 22–32. doi:10.1080/15230406.2014.987694.
Herman, L., and T. Řezník. 2015. “3D Web Visualization of Environmental Information – Integration of
Heterogeneous Data Sources When Providing Navigation and Interaction.” In ISPRS Archives of the
Photogrammetry, Remote Sensing and Spatial Information Sciences (Vol. XL-3/W3), edited by C. Mallet, et al.,
479–485. La Grande Motte: ISPRS. doi:10.5194/isprsarchives-XL-3-W3-479-2015.
Herman, L., and Z. Stachoň. 2016. “Comparison of User Performance with Interactive and Static 3D Visualization –
Pilot Study.” In ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (Vol. XLIB2),
edited by L. Halounová, et al., 655–661. Prague: ISPRS. doi:10.5194/isprs-archives-XLI-B2-655-2016
Herman, L., Z. Stachoň, R. Stuchlík, J. Hladík, and P. Kubíček. 2016. “Touch Interaction with 3D Geographical
Visualization on Web: Selected Technological and User Issues.” In ISPRS Archives of the Photogrammetry,
170 P. KUBÍČEK ET AL.
Remote Sensing and Spatial Information Sciences (Vol. XLII-2/W2), edited by E. Dimopoulou and P. van Oosterom,
33–40. Athens: ISPRS. doi:10.5194/isprs-archives-XLII-2-W2-33-2016.
Hirmas, D. R., T. Slocum, A. F. Halfen, T. White, E. Zautner, P. Atchley, H. Liu, W. C. Johnson, S. Egbert, and D.
McDermott. 2014. “Effects of Seating Location and Stereoscopic Display on Learning Outcomes in an
Introductory Physical Geography Class.” Journal of Geoscience Education 62 (1): 126–137. doi:10.5408/12-362.1.
Höffler, T. N. 2010. “Spatial Ability: Its Influence on Learning with Visualizations – A Meta-analytic Review.”
Educational Psychology Review 22 (3): 245–269. doi:10.1007/s10648-010-9126-7.
Hunter, J., C. Brooking, L. Reading, and S. Vink. 2016. “A Web-based System Enabling the Integration, Analysis, and
3D sub-Surface Visualization of Groundwater Monitoring Data and Geological Models.” International Journal of
Digital Earth 9 (2): 197–214. doi:10.1080/17538947.2014.1002866.
Hutchins, E. 1995. “How a Cockpit Remembers Its Speed.” Cognitive Science 19: 265–288.
Juřík, V., L. Herman, Č Šašinka, Z. Stachoň, and J. Chmelík. 2017. “When the Display Matters: A Multifaceted
Perspective on 3D Geovisualizations.” Open Geosciences 9 (1): 89–100. doi:10.1515/geo-2017-0007.
Keehner, M., M. Hegarty, C. Cohen, P. Khooshabeh, and D. Montello. 2008. “Spatial Reasoning With External
Visualizations: What Matters Is What You See, Not Whether You Interact.” Cognitive Science: A
Multidisciplinary Journal 32 (7): 1099–1132. doi:10.1080/03640210801898177.
Klippel, A., C. Weaver, and A. C. Robinson. 2011. “Analyzing Cognitive Conceptualizations Using Interactive Visual
Environments.” Cartography and Geographic Information Science 38 (1): 52–68. doi:10.1559/1523040638152.
Konečný, M. 2011. “Cartography: Challenges and Potentials in Virtual Geographic Environments Era.” Annals of GIS
17 (3): 135–146. doi:10.1080/19475683.2011.602027.
Kozhevnikov, M. 2007. “Cognitive Styles in the Context of Modern Psychology: Toward an Integrated Framework of
Cognitive Style.” Psychological Bulletin 133 (3): 464–481. doi:10.1037/0033-2909.133.3.464.
Kubíček, P., Č Šašinka, Z. Stachoň, Z. Štěrba, J. Apeltauer, and T. Urbánek. 2017. “Cartographic Design and Usability
of Visual Variables for Linear Features.” The Cartographic Journal 54 (1): 91–102. doi:10.1080/00087041.2016.
1168141.
Landy, M., L. Maloney, E. Johnston, and M. Young. 1995. “Measurement and Modeling of Depth Cue Combination: in
Defense of Weak Fusion.” Vision Research 35 (3): 389–412.
Larkin, J., and H. Simon. 1987. “Why a Diagram Is (Sometimes) Worth Ten Thousand Words.” Cognitive Science 11
(1): 65–100.
Lin, H., M. Batty, S. Jørgensen, B. Fu, M. Konečný, A. Voinov, P. Torrens, et al. 2015. “Virtual Environments Begin to
Embrace Process-based Geographic Analysis.” Transactions in GIS 19 (4): 493–498. doi:10.1111/tgis.12167.
Lokka, I.-E., and A. Cöltekin. 2016. “Simulating Navigation with Virtual 3D Geovisualizations – A Focus on Memory
Related Factors.” In ISPRS Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (Vol.
XLI-B2), edited by L. Halounová, et al., 671–673. Prague: ISPRS. doi:10.5194/isprs-archives-XLI-B2-671-2016.
Lokka, I.-E., and A. Cöltekin. 2017. “Toward Optimizing the Design of Virtual Environments for Route Learning:
Empirically Assessing the Effects of Changing Levels of Realism on Memory.” International Journal of Digital
Earth 56: 1–19. doi:10.1080/17538947.2017.1349842.
Loomis, J. M., and D. W. Eby. 1988. “Perceiving Structure from Motion: Failure of Shape Constancy.” In Proceedings of
Second International Conference on Computer Vision, 383–391. Los Alamitos. doi:10.1109/CCV.1988.590015.
MacEachren, A. M. 1995. How Maps Work: Representation, Visualization, and Design. 1st ed. New York: Guilford
Press.
Pegg, D. 2011. “Design Issues with 3D Maps and the Need for 3D Cartographic Design Principles.” In Workshop ‘Maps
for the Future: Children, Education and the Internet’. Accessed January 3, 2017. http://lazarus.elte.hu/cet/academic/
pegg.pdf.
Popelka, S., and P. Dědková. 2014. “Extinct Village 3D Visualization and Its Evaluation with Eye-movement
Recording.” In Computational Science and Its Applications – ICCSA 2014, edited by B. Murgante, et al., 786–
795. Switzerland: Springer International. doi:10.1007/978-3-319-09144-0_54.
Popelka, S., Z. Stachoň, Č Šašinka, and J. Doleželová. 2016. “EyeTribe Tracker Data Accuracy Evaluation and Its
Interconnection with Hypothesis Software for Cartographic Purposes.” Computational Intelligence and
Neuroscience 2016 (2016): 1–14. doi:10.1155/2016/9172506.
Qian, N. 1997. “Binocular Disparity and the Perception of Depth.” Neuron 18 (3): 359–368.
Rautenbach, V., S. Coetzee, and A. Cöltekin. 2017. “Development and Evaluation of a Specialized Task Taxonomy for
Spatial Planning: A map Literacy Experiment with Topographic Maps.” ISPRS Journal of Photogrammetry and
Remote Sensing 127: 16–26. doi:10.1016/j.isprsjprs.2016.06.013.
Řezník, T., B. Horáková, and R. Szturc. 2013. “Geographic Information for Command and Control Systems:
Demonstration of Emergency Support.” In Intelligent Systems for Crisis Management: Geo-information for
Disaster Management, edited by S. Zlatanova, et al., 263–275. Heidelberg: Springer. doi:10.1007/978-3-642-
33218-0_18.
Roth, R. E. 2012. “Cartographic Interaction Primitives: Framework and Synthesis.” The Cartographic Journal 49 (4):
376–395. doi:10.1179/1743277412Y.0000000019.
INTERNATIONAL JOURNAL OF DIGITAL EARTH 171
Roth, R. E., A. Çöltekin, L. Delazari, H. F. Filho, A. Griffin, A. Hall, J. Korpi, et al. 2017. “User Studies in Cartography:
Opportunities for Empirical Research on Interactive Maps and Visualizations.” International Journal of
Cartography 1: 1–29. doi:10.1080/23729333.2017.1288534.
Šašinka, Č. 2012. “Interindividuální rozdíly v percepci prostoru a map [Interindividual Differences in Perception of
Space and Maps].” PhD Thesis, Masaryk University, Brno.
Seipel, S. 2013. “Evaluating 2D and 3D Geovisualisations for Basic Spatial Assessment.” Behaviour & Information
Technology 32 (8): 845–858. doi:10.1080/0144929X.2012.661555.
Semmo, A., M. Trapp, M. Jobst, and J. Döllner. 2015. “Cartography-oriented Design of 3D Geospatial Information
Visualization – Overview and Techniques.” The Cartographic Journal 52 (2): 95–106. doi:10.1080/00087041.
2015.1119462.
Sieber, R., M. Serebryakova, R. Schnürer, and L. Hurni. 2016. “Atlas of Switzerland Goes Online and 3D – Concept,
Architecture and Visualization Methods.” In Progress in Cartography. Lecture Notes in Geoinformation and
Cartography, edited by G. Gartner, M. Jobst, and H. Huang, 171–184. Switzerland: Springer International.
doi:10.1007/978-3-319-19602-2_11.
Špriňarová, K., V. Juřík, Č. Šašinka, L. Herman, Z. Štěrba, Z. Stachoň, J. Chmelík, and B. Kozlíková. 2015. “Human–
computer Interaction in Real 3D and Pseudo-3D Cartographic Visualization: A Comparative Study.” In
Cartography – Maps Connecting the World, edited by C. R. Sluter, C. B. M. Cruz, and P. M. L. de Menezes, 59–
73. Switzerland: Springer International. doi:10.1007/978-3-319-17738-0_5.
Stachoň, Z., Č. Šašinka, Z. Štěrba, J. Zbořil, Š. Březinová, and J. Švancara. 2013. “Influence of Graphic Design of
Cartographic Symbols on Perception Structure.” Kartographische Nachrichten 63 (4): 216–220.
Štěrba, Z., Č. Šašinka, Z. Stachoň, R. Štampach, and K. Morong. 2015. Selected Issues of Experimental Testing in
Cartography. 1st ed. Brno, Czech Republic: Masaryk University. doi:10.5817/CZ.MUNI.M210-7893-2015.
Svatoňová, H. 2016. “Analysis of Visual Interpretation of Satellite Data.” In ISPRS Archives of the Photogrammetry,
Remote Sensing and Spatial Information Sciences (Vol. XLI-B2), edited by L. Halounová, et al., 675–681. Prague:
ISPRS. doi:10.5194/isprsarchives-XLI-B2-675-2016.
Tartre, L. A. 1990. “Spatial Orientation Skill and Mathematical Problem Solving.” Journal for Research in Mathematics
Education 21 (3): 216–229. doi:10.2307/749375.
Thompson, S. C. 1999. “Illusions of Control: How We Overestimate Our Personal Influence.” Current Directions in
Psychological Science 8 (6): 187–190. http://www.jstor.org/stable/20182602.
Torres, J., M. Ten, J. Zarzoso, L. Salom, R. Gaitán, and J. Lluch. 2013. “Comparative Study of Stereoscopic Techniques
Applied to a Virtual Globe.” The Cartographic Journal 50 (4): 369–375. doi:10.1179/1743277413Y.0000000034.
Vandenberg, S. G., and A. R. Kuse. 1978. “Mental Rotations, a Group Test of Three-dimensional Spatial Visualization.”
Perceptual and Motor Skills 47 (2): 599–604.
Vidláková, I. 2010. “Object – Spatial Imagery Questionnaire (OSIQ).” Annales Psychologici, Masaryk University, Brno.
Accessed December 29, 2016. http://hdl.handle.net/11222.digilib/114425.
Ware, C., and G. Franck. 1996. “Evaluating Stereo and Motion Cues for Visualizing Information Nets in Three
Dimensions.” ACM Transactions on Graphics 15 (2): 121–140. doi:10.1145/234972.234975.
Wilkening, J., and S. I. Fabrikant. 2013. “How Users Interact with a 3D Geo-browser Under Time Pressure.”
Cartography and Geographic Information Science 40 (1): 40–52. doi:10.1080/15230406.2013.762140.
Willett, W., B. Jenny, T. Isenberg, and P. Dragicevic. 2015. “Lightweight Relief Shearing for Enhanced Terrain
Perception on Interactive Maps.” In Proceedings of the 33rd ACM Conference on Human Factors in Computing
Systems (CHI 2015), 3563–3357. Seoul. doi:10.1145/2702123.2702172.
Zanola, S., S. I. Fabrikant, and A. Cöltekin. 2009. “The Effect of Realism on the Confidence in Spatial Data Quality in
Stereoscopic 3D Displays.” In Proceedings, 24th International Cartography Conference (ICC 2009), Santiago.
Accessed January 3, 2017. http://www.geo.uzh.ch/~sara/pubs/zanola_fabrikant_coeltekin_icc09.pdf.
172 P. KUBÍČEK ET AL.
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles
14679671,2021,26,Downloadedfromhttps://onlinelibrary.wiley.com.ByCzechs-MasarykUniversity-on[22/07/2022].Re-useanddistributionisstrictlynotpermitted,exceptforOpenAccessarticles