Bulletin of the IEEE Technical Committee on Learning Technology, Volume 14, Number 4, October 2012 7
Abstract— This paper proposes the use of augmented reality
(AR) interfaces for the construction of educational applications
that can be used in practice to enhance current teaching methods
as well as for the delivery of lecture material. The interactive AR
interface has been piloted in the classroom at an undergraduate
module of a Bachelor of Science (BSc) degree in Games
Technology at Coventry University, UK. An initial evaluation
was performed with fifteen students and qualitative feedback was
recorded. Results indicate that the adoption of AR technology is
not only a promising and stimulating tool for learning computer
graphics, but it can also be incredibly effective when used in
parallel with more traditional teaching methods.
Index Terms—augmented reality, computer graphics, humancomputer
interaction.
I. INTRODUCTION
lthough current teaching methods work successfully,
Universities are interested in introducing more productive
methods for improving the learning experience and increasing
the level of understanding of the students. The emergence of
new technological innovations such as the Internet,
multimedia, virtual and augmented reality technologies, was
able to demonstrate the weaknesses of traditional teaching
methods but also the potential for improving them. This
evolution allows educationalists to make good use of a number
of multimedia technologies to demonstrate phenomena’s or
explain complex theories to the students in a different way
than the traditional methods can, thus overcoming some
limitations.
Augmented reality (AR) has the ability of enhancing the
real world by using computer-generated information that is
projected onto the user’s virtual environment. Users can
visualize the superimposed information with a selection of
display technologies and can interact with it in a natural
manner by employing software interfaces, physical markers
and hardware interaction devices. One of the earliest examples
of research that applied AR to an educational context is the
‘Classroom of the Future’ [1], which conceptualizes how it
could be possible to enhance interaction between instructor
and students by employing AR technologies. Another example
is the higher education collaborative AR learning system for
Dr. Fotis Liarokapis is the director of Interactive Worlds Applied Research
Group (iWARG), Faculty of Engineering and Computing, Department of
Computing, Coventry University, UK (e-mail: F.Liarokapis@coventry.ac.uk).
mathematics and geometry education Construct3D [2] that
allow teachers and students to interact through various
scenarios. An alternative experimental education application
demonstrates the AR enhanced teaching of undergraduate
geography students about earth-sun [3]. In a similar AR
application educators use AR to explain to students how
specific parts of a computer could work in practice [4].
In another study researchers have compared the use of AR
and physical models in chemistry education [5] and results
showed that some students liked to manipulate AR by rotating
the markers to see different orientations of the virtual objects
whereas others preferred to interact with physical models to
get a feeling of physical contact. Moreover, two modules,
‘Solar System’ and ‘Plant System’, were developed for mixed
reality (MR) in the classroom, “providing support for
classroom teaching and self-learning” [6]. This study was
influenced directly by its perceived usefulness, and indirectly
through perceived ease of use and social influence, and
preliminary results seemed to indicate the participants’
intention to use MR for learning. Elsewhere, researchers have
studied the integration of physical objects that are
computationally-augmented to support and encourage face-toface
interaction between disabled students and virtual objects
[7]. Initial results indicated the importance of inclusion in
novel technology enhanced learning approaches for science
education.
In collaborative AR environments, multiple users may
access a shared space populated with digital information and
thus maximizing the transfer of knowledge [8]. A good
example is the use of a real physical book for the development
of a visually augmented reality book, however with drawbacks
that include the affordances of the book resulting in book-like
interaction by the users, which creates challenges in terms of
technology deployment [9]. Another collaborative approach
for teachers and trainees is an AR system that simulates a
web-based training and teaching environment for distance
education and training. AR may be used successfully to
provide assistance to the user necessary to carry out difficult
procedures [10] or understand complex problems.
Demonstration in lecture and seminar rooms is one of the most
effective means of transferring knowledge to large groups of
people [11].
The main advantage of AR over more traditional teaching
methods is that learners can actually ‘see’ and ‘listen to’
supplementary digital information. Additionally, students can
intuitively manipulate the virtual information, allowing them
Augmented Reality Interfaces for Assisting
Computer Games University Students
Fotis Liarokapis, Member, IEEE
A
Bulletin of the IEEE Technical Committee on Learning Technology, Volume 14, Number 4, October 2012 8
to repeat a specific part of the augmentation as many times as
they want. One of the main aims of this research is to
contribute in resolving the perceptual discontinuities [12]
initiated by scattered sources of information during the
learning process. To better understand these discontinuities we
have developed a prototype AR learning system focused on
higher education with a particular interest in computing
courses. The basic idea is an AR table-top learning
environment that integrates the real teaching environment (i.e.
lecture theatre) with virtual learning scenarios in a studentfriendly
and engaging manner. Tangible interfaces are the
medium used to allow students examine and experiment with
the virtual teaching material in a natural manner.
This paper describes the use of a high-level AR interface for
assisting teaching at University level of 2nd
year computer
science undergraduate students in computer games
technologies. Three-dimensional information can be
superimposed in a student-friendly manner into the learning
environment. The interactive AR interface has been piloted in
the classroom at Coventry University at fifteen undergraduate
students and they were asked to comment on the effectiveness
of the system and whether it should be used as an additional
tool for teaching.
The rest of the paper is structure as follows. Section II
describes the method of operation of the AR interface. Section
III illustrates different case studies of teaching in AR. Section
IV presents initial evaluation results while section V
summarizes conclusions and future work.
II. METHOD OF OPERATION
The technical details of the interactive AR interface have
been previously presented [13], [14]. The AR interface allows
for the natural arrangement of virtual information anywhere
inside the interior of the classroom (lecture or laboratory) or
any other type of indoor environment. An overview of the
system in operation is illustrated in Figure 1.
Fig. 1. Operation of the system.
In this configuration, a laptop computer with a USB webcamera,
a video splitter, a plasma screen and a set of trained
marker cards were employed. Depending on the capabilities of
the splitter different configurations can be supported
depending on the level of immersion and collaboration
required. In terms of software technologies, the interactive
application consists of a set of C++ classes which are in
charge of controlling the tracking, visualization and
interaction. In particular, the AR environment uses the vision
tracking libraries of ARToolKit [15] while the graphics
operations are built on top of the OpenGL API.
The lecturer can control the sequence of the AR
presentation in a table-top environment according the
student’s pace. Pre-designed three-dimensional models can be
selected and superimposed in the table top environment and
students can either interact in their desks or in a collaborative
environment (Figure 1). It is worth mentioning that this
technology is used in conjunction with the traditional methods
used (i.e. lecture notes, PowerPoint presentation, etc).
III. TEACHING IN AUGMENTED REALITY
The scenarios proposed are focused on enhancing the
teaching and learning process for higher education and in
particular for a Bachelor of Science (BSc) degree in ‘Games
Technology’. With this purpose in mind, computer graphics
AR scenarios for a module called “3D Graphics
Programming” have been designed to assist the lecturer to
transfer knowledge to the students in other ways than
traditionally has been the case. The AR interface offer the
ability to use sophisticated techniques to achieve better user
interaction with teaching material and complex tools. The
provision of an interactive augmented presentation allows
students a high degree of flexibility and understanding of the
teaching material. Moreover, students do not need any
previous experience to operate the AR interface technology
successfully. To prove the feasibility of the application,
different teaching scenarios from computer graphics theories
have been investigated and implemented, including: shading,
transparency, hard shadows and environmental mapping.
Shading in computer graphics is used for determining the
pixels’ color based on lighting computations [16]. There are
three basic types of shading including flat, smooth and Phong
shading, but only the first two were implemented because they
are directly supported by OpenGL. In flat shading (Figure 2,
a) the color of one particular vertex is duplicated across all the
primitive vertices. This method is very simple and works very
fast but it does not give a smooth appearance to curved
surfaces because all pixels in the polygon as shaded the same.
On the other hand, smooth shading, also known as Gouraud
shading (Figure 2, b), is one of the most popular shading
algorithms which interpolates light intensities across the face
of a polygon using values taken from its vertices [16].
Gouraud shading is slower than flat shading but it produces a
smoother appearance across polygons as it can be illustrated
from Figure 2.
Bulletin of the IEEE Technical Committee on Learning Technology, Volume 14, Number 4, October 2012 9
Fig. 2. Shading in AR (a) flat shading (b) smooth shading.
Texturing is used in computer graphics and modern games
for increasing realism. Figure 3 illustrates the same textured
object with different levels of transparency. Figure 3, (a),
shows a 3D model with the alpha value close to unity (80% or
0.8) blending while Figure 3, (b) presents the same 3D model
with alpha value close to zero (20% or 0.2) blending. This
effect is very useful when occluding 3D objects are overlaid in
the real environment. Similarly, to 3D games, using
transparency it is possible to make the virtual object that
occludes the others so that all objects are visible.
Fig. 3. Transparency levels in AR (a) less transparent (80% alpha blending)
(b) more transparent (20% alpha blending).
Shadows play a very important role for the generation of a
realistic scene in computer games. In reality, all objects have
their shadows so augmented objects should have them as well
[16]. In this work, only hard shadows were implemented. Two
example screenshot that illustrate a 3D representation of the
shadows generated from a simple AR cube (Figure 4, a) and
an AR tree (Figure 4, b) are illustrated in Figure 4.
Fig. 4. Hard shadows (a) Canonical shape (cube) (b) Non-canonical shape
(tree).
Environmental mapping (also known as reflection mapping)
is a simple and effective method of generating approximations
of reflections in curved surfaces [16]. It is usually classified as
category of texturing technique and can be considered as a
simplification of ray tracing [17]. The main advantages of this
technique are that: (a) it is simple to implement and (b) it
provides a rough estimation of reality. Two example
screenshots illustrate how spherical mapping can be applied
on a textured 3D model (Figure 5).
Fig. 5. Environmental mapping (a) simple texture mapping (b) sphere
mapping.
Figure 5, (a), illustrates an augmented 3D model with
simple texturing applied, while Figure 5, (b) illustrates the
same 3D model with sphere mapping.
IV. INITIAL EVALUATION
The AR application was presented to fifteen second year
undergraduate students, studying the “Games Technology”
degree at Coventry University. Different graphics scenarios
were presented during lectures and laboratory sessions and
feedback was recorded. This can be categorized into three
types including: visualization experience; interaction and
movement; and usefulness in learning.
As far as ‘usefulness in learning’ is concerned, all students
agreed the presented technology is very promising and should
be applied in the classroom. Most students were impressed
with the capabilities of AR and liked using it for exploration
and learning. In particular one said “by far the most interesting
lecture I have ever had and made me want to explore more and
more”. Three students mentioned that 3D perception in a
classroom is much better than 2D because it helps to visualize
the atmosphere in a better way and looks closer to reality.
Another said that the system sometimes was ‘jumpy’ and this
could cause problems to people with visual impairments.
However, the feedback received for visualization and
interaction varied. In terms of interaction, the majority of
students were impressed with the interactivity of the AR
tangible interface and noted that this provides an exciting
means of collaboration between the lecturer and the students.
Most of the students mentioned that interaction seems to be
very easy a very enjoyable compared to other software tools
(i.e. Flash, Authorware, etc) and one said that it is “the most
interactive interface that I have experienced in the classroom”.
One stated that it is much better to interact with 3D objects
using markers in AR rather than interacting using the mouse
and keyboard. Another student said that it is better than
traditional methods since it is closer to reality. Only, one
student said that the interaction is confusing and suggested to
use other manipulation aids (i.e. mouse or keyboard).
On the contrary, for the visualization experience feedback
was mixed. In particular, for shading and environmental
mapping most students preferred other multimedia
technologies (i.e. videos, online tutorials) and they found that
AR could be distracting to use. In the case of shadows and
transparency, they all agreed that this is definitely an excellent
Bulletin of the IEEE Technical Committee on Learning Technology, Volume 14, Number 4, October 2012 10
visualization medium. One particular student pointed out that
the use of AR technology is the best means of teaching
computer graphics principles since it makes it easier to
understand the underlying theories and concepts for three-
dimensions.
V. CONCLUSIONS
In this paper a low-cost AR educational and interactive
environment for assisting teaching in higher education was
presented. The main novelty of the system is that it can
provide students with an interactive augmentation of teaching
material focused on computer graphics principles in a
compelling and engaging way. Three-dimensional objects
illustrating some basic concepts of computer graphics and
computer games were presented to both lectures and
laboratory sessions.
The AR application was presented to fifteen second year
undergraduate students, studying the “Games Technology”
degree at Coventry University and initial evaluation results
indicated that it is very useful to be able to ‘see’ and ‘interact’
with related computer graphics in three dimensions in some
cases but not in all principles of computer graphics. Although
some students were skeptical about the technology, the
majority mentioned that this is a very promising technology
for the future.
In the future, more computer graphics scenarios will be
implemented and another study will be performed. In addition,
the AR interface will be piloted for more computer games
modules such as artificial intelligence and physics for
computer graphics. Finally, the AR interface will be ported in
mobile devices allowing students to enjoy the interactive AR
presentation in alternative locations (i.e. their homes).
ACKNOWLEDGMENTS
The author would like to thank the Interactive Worlds
Applied Research Group (iWARG) members for their support
and inspiration as well as the students who participated in the
evaluation study. Additionally, special thanks need to be
extended to Dr. Panagiotis Petridis for creating part of the
digital content.
REFERENCES
[1] J. R. Cooperstok, “The classroom of the future: Enhancing education
through augmented reality”, in Proc. of the Int’l Conference on HumanComputer
Interaction, 2001, pp. 688-692.
[2] H. Kaufmann and D. Schmalstieg, “Mathematics and geometry
education with collaborative augmented reality”, in SIGGRAPH 2002
Conference Abstracts and Applications: SIGGRAPH 2002 Educators
Program, 2002, pp. 37–41
[3] B. E. Shelton and N. R. Hedley, “Using augmented reality for teaching
earth-sun relationships to undergraduate geography students” in Proc. of
the 1st
IEEE Int’l Augmented Reality Toolkit Workshop, 2002, pp. 8.
[4] B. Fernandes and J. C. Miranda, “Learning how computer works with
augmented reality” in Proc. of the 2nd
Int’l Conference on Multimedia
and Information & Communication Technologies in Education, 2003.
[5] Y.-C. Chen, “A study of comparing the use of augmented reality and
physical models in chemistry education”, in VRCIA ’06: Proc. of the
2006 ACM international conference on Virtual reality continuum and its
applications, ACM, 2006, pp. 369-372.
[6] W. Liu, A. D. Cheok, C. L. Mei-Ling and Y.-L. Theng, “Mixed reality
classroom: learning from entertainment”, in DIMEA ’07: Proc. of the 2nd
Int’l conference on Digital interactive media in entertainment and arts,
ACM, 2007, pp. 65-72.
[7] T. N. Arvanitis, A. Petrou, et al, “Human factors and qualitative
pedagogical evaluation of a mobile augmented reality system for science
education used by learners with physical disabilities”, Personal
Ubiquitous Comput, vol. 13, number 3, 2009, pp. 243-250.
[8] H. Kaufmann, “Collaborative augmented reality in education”, in Proc.
of Imagina03 CDROM - Position paper for keynote speech at
Imagina03, 2003.
[9] R. Grasset, A. Dunser and M. Billinghurst, “Edutainment with a mixed
reality book: a visually augmented illustrative childrens’ book”, in ACE
’08: Proc. of the Int’l Conference on Advances in Computer
Entertainment Technology, ACM, 2008, pp. 292-295.
[10] B. Schwald and B. Laval, “An augmented reality system for training and
assistance to maintenance in the industrial context”, Journal of WSCG
2003, pp. 425-432.
[11] F. Liarokapis, N. Mourkoussis, et al, “Web3d and augmented reality to
support engineering education”, World Transactions on Engineering and
Technology Education, vol. 3, number 1, 2004, pp. 11-14.
[12] E. Dubois, L. Nigay and J. Troccaz, “Consistency in augmented reality
systems”, in Proc. of the 8th
Conference on Engineering for Human
Computer Interaction, 2001, pp. 111-122.
[13] F. Liarokapis and E. F. Anderson, “Using Augmented Reality as a
Medium to Assist Teaching in Higher Education”, in Proc. of the 31st
Annual Conference of the European Association for Computer Graphics
(Eurographics 2010), Education Program, Norrkoping, Sweden, 4-7
May, 2010, pp. 9-16.
[14] F. Liarokapis, “An Augmented Reality Interface for Visualizing and
Interacting with Virtual Content”, Virtual Reality, Springer, vol. 11,
number 1, 2007, pp. 23-43.
[15] H. Kato, M. Billinghurst, et al., “Virtual Object Manipulation on a
Table-Top AR Environment”, in Proc. of International Symposium on
Augmented Reality 2000, Munich, 5-6 Oct., 2000, pp. 111-119.
[16] T. Moller, “Real-Time Rendering”, A K Peters Ltd, 1999, pp. 23-38,
171.
[17] A. Watt, “3D Computer Graphics”, Essex: Addison-Wesley Longman
Ltd, Second Edition, 1993, pp. 1-11, 91, 259-262.
Dr. Fotis Liarokapis holds a DPhil in Computer Engineering at the
University of Sussex, an MSc in Computer Graphics and Virtual
Environments at the University of Hull and a BEng in Computer Systems
Engineering at the University of Sussex. He is the director of Interactive
Worlds Applied Research Group (iWARG) at Coventry University and a
research fellow at the Serious Games Institute (SGI). He is also a visiting
lecturer at the Centre for VLSI and Computer Graphics, University of Sussex
and a visiting research fellow at the giCentre, City University. Since 2000, he
has contributed to more than 65 refereed publications and has more than 450
citations. He has been invited more than 40 times to become member of
international conference committees and has chaired 11 sessions in 7
international conferences. Moreover, he has secured around £150,000 from a
number of research projects. In addition, he is a member of IEEE, IET, ACM
and Eurographics. Finally, he has established VS-Games conference which is
sponsored by IEEE and he is on the editorial advisory board of The Open
Virtual Reality Journal published by Bentham and the editor-in-chief of the
International Journal of Interactive Worlds (IJIW) published by IBIMA.