Diane P. McCarthy
Christchurch Polytechnic Institute of Technology, New Zealand
McCarthy, Diane P. (2008). Where's the bling of the thing?: Ethics, gaming and PBL-I. Journal of Applied Computing and Information Technology, 12(1). Retrieved May 26, 2020 from http://www.citrenz.ac.nz/jacit/JACIT1201/2008McCarthy_EthicsGaming.html
This paper describes an evaluation of PBL-Interactive (PBL-I) as authoring tool for an ethical decision-making game, in the Dip ICT course, ET600 Ethics and professionalism, at Christchurch Polytechnic Institute of Technology (CPIT). While the prototype was developed and the usability testing conducted with student volunteers returned positive results, the limitations of the authoring tool to create a multi-linear ethics game outweighed its affordances. In particular, the authoring mode interface lacked ease of use and functionality for advanced features such as video and audio, desirable options to motivate computing students with advanced gaming experience, in online and face-to-face learning environments.
Scenario-based problem solving can be used in serious games to explore ethical decision making. However, authoring tools for tertiary educators should be easy to use, including for multi-linear scenarios. Groups should be able to work through the ethical game scenario in multiple ways, with the potential for students to create new content for future participants. Otherwise, deep thinking is displaced by astute guessing of a 'right answer' from a list. Ethical decision-making may be constrained, more simplistic, and deterministic. Cross cultural issues may be used stereotypically, without developing insight or empathy. Opportunities for international students to make a contribution as subject matter experts may not eventuate.
The author has persistently used collaborative ways of teaching with students through simulation games and case-based learning in her teaching career. A high school social studies teacher in the 1970s and early 1980s, she used role play to portray such themes as social control and social change in whole-class case-based simulations, and as a drama teacher, explored social themes and children's theatre with students in street theatre settings and primary classrooms.
Employed by the Post Primary Teachers Association from the mid 1980s until 2000 as a field officer, she delivered trade union education for local and regional officials in Canterbury, Marlborough, Nelson, and West Coast of the South Island of New Zealand. She designed and delivered seminars for branch officials at school level for managing branch issues through case-based learning such as negotiation of branch issues at school staff meetings and learning and applying new and existing contractual entitlements. Some branches and regional officials commissioned seminars and workshops to meet their particular needs and whole day simulation games were designed to develop their skills, including conflict resolution.
On being employed as a lecturer in a polytechnic setting, and as a secondary teacher in the Christchurch School of Computing in 2001, the author observed that the use of the Internet and the ubiquity of DVDs for popular culture such as movies appeared to blunt the effectiveness of classroom-based face-to-face role plays. Computing students appeared more likely to resist participation than to get involved. She also encountered resistance to reading scenarios in text for ethical problem solving. Students were often not engaged in analysing the text and relied on a few classmates to carry the discussions forward. However, she was informed by her students that many spent many hours each week playing popular massively multi-player online role playing games (MMORPGs) such as World of Warcraft and Everquest.
The first strategy to more fully engage students in case-based learning was to adapt and script an out of print case study Virtual Success as a short video localised in New Zealand. Student volunteers took the roles of CEO of a gaming company, the R&D Manager, and Lead Programmer in a fictitious Christchurch-based company, Kiwi FX., and the video was directed, edited and produced by a former student. The ethical dilemma revolves around whether to develop virtual reality games to train SIS operatives in counter terrorism. The case study is accessed as a streaming video from the ET600 Moodle site, discussed in class, and then used to assess the students' ethical decision-making skills. Having achieved that goal, the author then considered using case-based games for developing ethical decision-making skills prior to the assessment by adapting further text-based scenarios and reviewed several possible technologies that could be used, including e-boards, before choosing the PBL-I tool for authoring.
The purpose of this research project was, as an early adopter, to use PBL-I as an authoring tool, to develop the ethical decision-making game, Hidden Dragons, based on the Markkula ethical decision-making process. It was developed for use in online and face-to-face delivery, to motivate computing students to engage in developing ethical decision-making skills in an IT business setting.
Integration of technology into the delivery of the curriculum is a powerful tool to engage students in their learning and is widely reported in educational and computer science literature (Cairncross & Mannion, 2001; Flanagan & Nissenbaum, 2007; Mayer, 2003; Oliver & Stoney, 1999).
This practitioner research explored two issues. Firstly, the ease of use and functionality of the PBL-I as an authoring tool to convert a text-based scenario into a multi-linear ethical decision-making game. Secondly, through usability testing, trialling the prototype with student volunteers to assess its effectiveness as a learning object. The students were first year students in the second semester of their two year undergraduate programme.
This research project is located in the course ET600: Ethics and professionalism. It is a semester-based compulsory course within the programme of the two-year Diploma of Information and Communications Technology, delivered throughout New Zealand to business ICT students enrolled in institutes of technology and polytechnics (ITPs). In ET600, it is expected that students are able to justify their decisions, and that their decisions should be ethically sound. Ethical decision making, according to Bloom's Taxonomy, is problem solving: analysis, synthesis and evaluation. It is suggested that case studies and simulated exercises are used as a method of teaching and learning these skills (National Advisory Committee on Computing Qualifications, 2004).
Providing students with engaging problem- and case-based scenarios broadly meets social, economic, and environmental issues of strategic importance to our nation state (Massey University, n.d.). Increasing the capacity of ICT within Aotearoa-New Zealand's economy is part of The Digital Strategy, to meet community and business interests (New Zealand Government, 2005). These interests are reflected in the National Advisory Committee on Computing Qualifications (2004) curriculum guidelines. The authoring tool PBL-I (Massey University, n.d.) was developed as a joint Massey University-University of Queensland project funded through the e-Learning Collaborative Development Fund (eCDF), administered by the Tertiary Education Commission (TEC) according to policy developed by the New Zealand Ministry of Education. The project deliverables included the software, exemplars, training materials, and a web site. The roll out of the 'free' software and manual to all state tertiary institutions occurred in August-December, 2006.
Bruntlett (2001) strongly advocates the innovative use of multimedia in education. Papert and Caperton (1997, cited in Bruntlett, 2001) suggest that educators need to have their own vision for using this powerful technology in ways that far exceed their previous expectations. Papert (1998, cited in de Freitas, 2008) is sceptical about whether 'edutainment' is even educational, or as engaging as leisure games. Bruntlett cautions us that we need to develop our own learning objects for use in our own programmes, rather than use commercially produced products that constrain or dictate to us. He suggests we can use existing multimedia, and develop ways of integrating them into our work, develop our own materials, and produce re-authored materials from resources gathered from the Web.
de Freitas (2008) of the Serious Games Institute, UK, summarizes the progress of educational initiatives in gaming and defines educational games as 'applications using the characteristics of video and computing games to create engaging and immersive learning experiences for delivering specified learning goals, outcomes and experiences' (p. 61). Definitions initially evolved from describing games as 'a series of choices' or 'rule-based play' (Wittenstein, 1958; cited in de Freitas, 2008), to more recently distinguishing education and training games from leisure games (Michael & Chen, 2006, cited in de Freitas, 2008).
However, as technologies proliferate in the 2D and 3D web environments, it is useful to note the ways that games themselves have evolved. Games can be categorized into four main strands (de Freitas, 2008, pp. 61-63):
Technologies are rapidly becoming more easily integrated using open source and interoperability, including setting standards for virtual worlds, and as students gain flexibility and self-empowerment in distributed social technologies, the challenges emerge for educators to incorporate blended learning interactions and activities, share learning resources and outputs, enable deeper reflection and better understand the role of developing content in the context of new approaches to curriculum design and pedagogy. In particular, the ways that educational institutions may reinforce rigidity of both curriculum and the management of IT networks as closed systems focused on security, rather than enabling distributed learning and learning communities, may need to be reassessed, to remain relevant to meeting the needs of diverse students in tertiary institutions (de Freitas, 2008).
Recent research indicates that game-based learning, while not universally liked by all students, assists in changing attitudes, engages and motivates learners, especially those who are younger, more visual and underserved, and can be used effectively with groups mixed by age, gender and culture. Learning transfer relies on the degree of fidelity with the real work experience, but the high uptake in vocational training reinforces that the costly development of this area of education has benefits that warrant this investment (de Freitas, 2008; Garris et al., 2002, Mitchell & Savill-Smith, 2005, Hayes, 2005, all cited in de Freitas, 2008).
Orngreen (2004) considers the possibilities and problems of casebased elearning (cbe-L). Students do not usually carry out the kind of high level analysis of a case that this teaching paradigm claims to encourage. Whichever model is used, either the Harvard model, based on a real case, or the interpretative European model, reporting events, decisions and first results, case-based learning analyses the situation, and devises viable strategies or evaluates decisions taken. Students work as individuals, in small groups, and then discuss it in class, with the final part being considered vital to the learning process (Heath, 1998; Lenders & Erskine, 1989; Orngreen & Bielli, 2001, all cited in Orngreen, 2004).
Orngreen (2004) describes cbe-L as a constructivist and experiential approach to learning. Students acquire knowledge through analysis and active discussion with peers, in ill-structured and complex situations, and relate the case to their own work experience, which may or may not be limited. Cases motivate students to analyse using a model and discussing how the theory applies. Often educators expect active participation, discussion and analysis but seldom give students directions on how to do this.
Students need to learn how to analyse the content. They benefit from collaborative analysis in small groups. If students draw mainly from their own experience, they may not critically reflect on the case. They may comprehend the content but not engage in deep thinking at the level of analysis or evaluation required by the curriculum. Cross-cultural issues may be stereotyped, and not develop insight or empathy or enable international students to make a contribution as subject matter experts (Orngreen, 2004).
An authoring tool should support these learning objectives and teaching processes, enabling high-level collaborative analysis of cases and a sound practice-theory relationship. The tool should be developed as a strategy to promote empirical research-based education. Orngreen suggests that cases are developed iteratively, with design specifications, revised design, and prototypes user tested in a simulated walk-through of case development and case analysis situations, prior to the final production phase.
The tool should coexist with the currently used platform and be used with synchronous chat and asynchronous forums. As teachers vary in their ability and motivation to develop cases, and the cases vary in size and complexity, the tool should be flexible enough to meet these needs. Students and teachers are rarely trained to use the case method, so the tool could give guidelines to students and enable a systematic and reflective analysis as well as collaboration with peers (Orngreen, 2004).
Role-play and simulation through gaming technology is being used to encourage the adoption and understanding of gender-inclusive and wider socio-cultural values (Flanagan & Nissenbaum, 2007; Khaled, Biddle, Noble, Barr, & Fischer; 2006). The learning paradigm is shifting towards more action-based objectives such as just-in-time actions in a simulated dynamic world rather than analysis-based on models and theories of cases (Beach & March, 2006; Hailey, 2004).
Downes (2006) encourages educators to see that new forms of learning are emerging for 'net generation' learners, and to work within this new paradigm, connectivism. Learning takes place as ongoing conversations and interaction sharing, with creation and participation embedded in meaningful activities such as games.
In summary, a learning object for cbe-L should ideally have the following affordances:
The research project used two methods to gather and analyse data. Firstly, the ease of use and functionality of the tool in authoring mode were initially appraised through the manual-based training with an exemplar. Some gaps were discovered in the documentation, and the author had to clarify matters with the developer. Once the text-based scenario was converted into a written draft scenario, the ease of use and functionality were evaluated by authoring a prototype of the ethical decision-making game.
Secondly, the Hidden Dragons Game was evaluated as a learning object. The prototype was usability tested using student volunteers. Usability testing was conducted in two stages. Firstly, following Barnum (2002) the draft scenario was evaluated as a 'prototype walkthrough', before authoring. The shell prototype was then usability tested using groups of 'tolerant users', in authoring mode, to evaluate the game's potential to engage students in ethical decision making, and to identify any authoring issues that needed to be addressed.
PBL-Interactive (PBL-I) is a web-based authoring tool and player, developed for use with distance students to participate in problem solving scenarios from within their learning management system, such as Blackboard or Moodle. It can also be distributed by DVD, and used by face-to-face students in either delivery mode. The tool was selected to gain experience of the software in Semester 2, 2006, as an early adopter, and to encourage its use at CPIT with the e-learning and web development team which supports the professional development of academic staff.
The scenario was adapted with permission from the case The Million Dollar Decision, from the Markkula Centre, University of Santa Clara, California. It used the Markkula Ethical Decision Making Model. Both documents are available from www.scu.edu/ethics. The scenario overview localised the episodes, locations, and tasks in a New Zealand setting for an interactive role-playing game renamed Hidden Dragons.
In brief, Pegasus International, a wireless communications company, is planning to expand its business into China, a country in which there is a history of bribe taking that had been considered a normal part of doing business. The case considers the ethical responsibilities of the CEO in developing company policy, in the face of conflicting values and ethical positions that can be adopted. This can be updated to consider the prevailing ethical issues of doing business with China within any contemporaneous context, such as human rights and Tibet, for example.
The game's draft concept plan consisted of the visuals, text, and actions to be interacted with by the players in each episode. Planned resources included background documents on the company, and its values, and links to web sites providing business intelligence. It was planned so that students could work in teams to consider the issues in each episode and make ethically-based decisions from the viewpoint of the manager, in response to the pressures from colleagues and the CEO.
Before beginning authoring, the author conducted usability testing of the concept plan. The purpose of the initial testing was to collect information about how the students related to the Hidden Dragons Project scenario, and whether they experienced difficulties using it with the Markkula process.
Barnum (2002) suggests the most useful approach with an existing product is to assess the ease or difficulty of performing specified tasks. Because the software was already developed, the focus of the usability testing was on a summative evaluation of its use by students in the game scenario using the Markkula method (Barnum, 2002). The objective was to see how the users responded subjectively to the specific tasks. The following tasks were identified for evaluation:
The first task was evaluated by the focus group, before authoring. Then all the tasks were evaluated by tolerant user testing groups after the shell of the prototype was authored in PBL-I. The prototype was tested in a lecture theatre setting on a laptop in player mode.
Nielsen (cited in Barnum, 2002) suggests that small groups can effectively evaluate usability of a product, as long as the sample reflects the user profile. Classes typically comprise 70-90% male, 33% NESB, with an age range of 17 to 25, with about 10% as mature students.
A focus group was recruited from one of the 2006 Semester 2 faceto-face classes in ET600, for ease of access and feedback. A protocol offering confidentiality, and explaining the purpose of the project was signed by the student volunteers. Students then filled in a pre-evaluation questionnaire, to establish their personal profile. Three students who represented the class profile were selected. They were then emailed, offering them a place in the group. All selected students accepted.The focus group comprised one male student aged under 20, and two male students aged 21 to 25. One student did not have English as a first language (NESB). The group did not contain a female student, as it was an all male class.
Five voluntary groups of tolerant users from the two classes tested the shell prototype. Group A, male Africans and a male Fijian, tested before a class. Group B, two mature women, 25+, tested during a class activity and break. Group C, a Chinese student from the focus group and his Chinese friend, and Groups D and E, male European-Pakeha students, including the leader from the focus group, tested during a class activity.
The first evaluation of the draft scenario was a prototype walkthrough (Barnum, 2002). The focus group met for an hour after their scheduled class. Students gave feedback on the paper prototype, before it was authored in PBL-I. They gave feedback on the planned visuals, text, and expected student responses. They gave a response to the overall scenario, and rated how subjectively engaging it was. An older student emerged as a leader, with consensus on points raised being confirmed and elaborated by the other students. Responses were recorded in writing.
The game was authored and prepared for tolerant user testing. The issues raised were considered, the plan adapted, and the game authored to the point of creating a shell prototype, with all the locations, environments, tasks (choices) and text complete. Some of the episodes had full audio. Links to supporting documents and web sites, and the creation of final visuals and audio were still under development. Tolerant user testing involved sympathetic users working through a prototype scenario to deliberately find the flaws (Barnum, 2002). They may be prompted or not by the tester, as the users see fit.
The PBL-I authoring tool was difficult to use, time consuming because of its limitations, due to poor functionality and lack of documentation. The project was abandoned and the game was not completed past the final prototype stage.
The Hidden Dragons Game received positive feedback from the male students, at both draft and prototype stages, regardless of age, and language background, and engaged them in collaborative ethical decision-making of a cross-cultural nature. The mature women students were less positive, and asked for direction.
The students encouraged the author to explore multi-linear pathways through the scenario, so that it was interactive, and not deterministic. They suggested that the tool should be robust enough to enable ongoing development of the scenario, as collaboration between the students and the author. International students gained confidence and verbal fluency through their participation and valued their role as subject matter experts.
The scenario is developed in author mode (see Figure 1), and then tested in player mode. This enables the pathways through the scenario to be checked and corrected. The opening screen is shown above. The tool uses a MUD like environment of four panes, which enables the creation of locations, environments, actions, and text for each episode within the game. Actions are embedded within each location. Actions are set as prerequisites, linked to actions and locations. The tool enables video and audio to be used within the episodes.
To edit a new episode, and check that it is functioning correctly, the authoring tool has to be reset to the start of the whole scenario, which is time consuming. The authoring should be able to be resumed at any point within the scenario. This lack of functionality adds unnecessary time to authoring and acts as a disincentive to its adoption by tertiary educators and e-learning teams who are the target audience. To create multiple pathways through the scenario, actions are linked to prerequisites. However, the lists of actions are created in common to all the locations, regardless of where they are used, and are displayed one by one in a small slotted authoring window.
The more actions, the longer the list, and the harder it is to ensure that cause and effect are properly sequenced. It is very difficult to set sequences and correct errors. The linking of actions and consequences, a key element in ethical decision making, should be embedded within their location. The interface for authoring tasks and their prerequisites needs to be redesigned as a large box, or a button linking tasks. Multiple pathways could be facilitated by having the actions and consequences set in a less binary manner, so resulting options are less rigidly set as a linear cause and effect. Having actions and consequence settings of 'either', 'or', 'and' could enable such a progression through a scenario. Audio files have to be located in a pathway so that they can be embedded, and have to be written in HTML code. This is a cumbersome approach, and unsophisticated. A button to embed audio and video would make this feature more useable.
To discover advanced features, each exemplar needs to be read, and the features listed by hand. The PBL-I manual lacks technical documentation, and advanced features are learned by trial and error, without online help. Apart from the simple practice scenario in the manual, which has a video, there is no description of how to use all of the features effectively. This needs addressing in any future editions.
The focus group feedback, in summary, was as follows:
The responses of the tolerant user groups varied, by gender and age as follows.
The males actively engaged with the game. They had a high tolerance for making mistakes, approached the task as trial and error, were not concerned with incompleteness, were intrigued and highly motivated, with a sense of humour. They asked about the purpose of each pane, experimented, restarted when shut out of the game, persisted, and completed the whole game within 19 to 21 minutes.
They looked for visual cues, discussed which action to take and why, using logic and problem solving, rather than randomness and guesswork. They made positive suggestions, to synchronise actions, experienced the testing as looking for flaws to help develop the game, rather than as testing their gaming competence.
The NESB students did not appear to experience any difficulty interpreting the text, the sequence, or the way to interact with the game, and did not need prompting. Cultural differences were explored in the scenario, with the Chinese students able to make a contribution in mixed groups as subject experts.
The focus group leader, having suggested that students who acted unethically were shut out of the game, was nonplussed when this happened to his group. The issue that arose was related to 'face' and ensuring the working relationship between different cultures could continue unimpaired.
In personal communication later, by email, the student agreed that decisions were contestable, and that the students should be able to comment on their reasons, and perhaps invent other scenarios that could be incorporated into the game.
The male groups were intrigued by the use of Voice Cloak software, and found the game enriched by the use of audio for the characters. In contrast, the women students were slower, more critical, repetitive, and cautious, did not express enjoyment, and saw error as a sign of their personal failure rather than within the game. However, after 28 minutes, they accounted for their personal failure in not progressing past two locations by blaming perceived shortcomings in the game. They were hesitant to read the content pane for cues, said they didn't understand the text, wanted an explanatory screen that described the panes, how to interact with them, and what they were doing and why: “How can I know what to do if you don't tell me?”
In spite of the constraints of the PBL-I authoring tool, this was a valuable experience exploring the potential for using gaming for teaching ethical decision-making. The outcomes of the tolerant user tests of the Hidden Dragons Game were illuminating, and led to further adaptations of the prototype, before the project was abandoned due to technical difficulties with the authoring tool.
The potential for using gaming for teaching ethical decision-making was clearly demonstrated with computing students in a polytechnic setting at Level 6 diploma level, equivalent to an associate degree. Male students, including NESB students, showed a high level of motivation and engagement with the prototype, and showed that interactive scenario-based games could be used effectively to explore the ethical issues and to learn to apply an ethical decision-making model within a cross cultural scenario.
Chinese international students were able to be subject matter experts, and there was a high level of student acceptance of the notion of an ethical decision-making game.
Women students, who were mature students, expressed more caution and less enjoyment, which may have reflected their relative inexperience with gaming related to their age, rather than their gender. This needs further research, as these findings may not apply to younger female students with gaming experience.
Using usability testing for collaborative development of interactive scenario-based games
Usability testing (Barnum, 2002) is an effective, collaborative, learning object developmental tool to gauge student interaction and gain feedback about improving its ease of use and functionality. This was applied throughout the project, as authoring proceeded from a paper scenario to a shell prototype for testing with students. Had the project been completed, the final version would have been user tested with specific goals related to its purpose, ease of use and functionality in computer suites, and in learning management systems such as Blackboard, with online and face-to-face students.
Focus group and tolerant user testing provide rich and relevant data to assist in this feedback and improve its functionality and student subjective learning experiences. It is suggested that practitioners can recruit their students and use simple usability testing in classroom settings to ensure that complex learning objects such as scenario-based games are engaging, with full functionality and ease of use.
The authoring limitations outweighed the affordances, particularly in the setting of navigation sequences and the use of advanced features such as audio (and video). These limitations made it too hard to ensure that the game was not predetermined, and was multi-linear, as is befitting the complexities of collaborative human interaction and choice in ethical decision making.
While the game or scenario may be complex in its pedagogical goals, and the level of thinking that the learning object requires, the authoring tool should be easy to use to foster a strong uptake among tertiary educators. Teachers need encouragement to adapt or create effective learning objects for scenario-based learning for distance and flexible delivery modes, with online and face-to-face students. The ease of use of the tool needs to be upgraded. The lack of technical documentation for PBL-I needs to be addressed in any amended editions of the manual. Otherwise, it will sit unused as a wasted resource.
The potential for developing and using engaging locally-based ethical decision-making games for teaching ethics at tertiary level in ICT, applied sciences, engineering, and other fields such as business studies, medical and nursing studies, and teacher education, for example, is huge. The author is exploring New Zealand open source tools and modes for creating ethical games for face-to-face and online students, especially in serious virtual worlds, collaboratively with staff and students at CPIT. The cyberformance medium, Upstage, version 2, a New Zealand funded and developed open source server application, is being trialled. It enables interaction through the flash player plugin in a browser using graphical avatars, web-cams, props, backdrops, drawing and text-tospeech functionality.
These will enable students to interact with each other within a scenario. Text-to-speech can blend scripted and improvised interaction with comments in chat mode from an audience. This medium may be used to enable teams of face-to-face students to engage in ethical decision making with distance students, within a case-based scenario that is dynamic and multi-linear, using existing computers with an Internet connection, and head sets, with a firewall workaround. Graphics and scripting can be collaboratively prepared in multi-disciplinary teams. Access to this tool, training from experienced practitioners, walkthroughs, manuals and documentation of past performances, and invitations to future events can be found at www.upstage.org.nz.
The author gratefully acknowledges the assistance of Dawn Coburn, School of Education, University of Otago, for supervising the original project, Dr Terry Stewart, Massey University, for technical advice on PBL-I, Lesley Smith, Otago Polytechnic, and Dr Donald Joyce, Unitec New Zealand, for editorial assistance, and Vicki Smith of Avatar Body Collision for advice, guidance and training in Upstage V2. This is an expanded and updated version of a paper presented at the 20th Annual Conference of the National Advisory Committee on Computing Qualifications.
Barnum, C. M. (2002). Usability testing and research. NewYork: Longman.
Beach, B. & March, V. (Presenters). (2006, October 20). Beyond text: Using your voice online. Vidcast retrieved March 2, 2008, from http://blackboard.cpit.ac.nz/webapps/portal/frameset.jsp?tab=courses&url=/bin/comm/course.pl?course_id=1272_1
Bruntlett, S. (2001). Making and using multimedia. In M. Leask (Ed). Issues in teaching using ICT (pp. 158-178). London: Routledge/ Falmer. New Zealand Journal of Applied Computing and Information Technology. Volume 12, Issue 1, 2008
Cairncross, S. & Mannion, M. (2001). Interactive multimedia and learning: Realising the benefits. Innovations in Education and Teaching International, 38(2), 156-164.
de Freitas, S. (2008). Emerging trends in serious games and virtual worlds. Emerging Technologies for Learning, 3. Retrieved April 11, 2008, from http://publications.becta.org.uk/display.cfm?resID=35877&page=1835
Downes, S. (2006). Learning networks and connective knowledge. Retrieved October 13, 2006, from http://www.downes.ca/
Flanagan, M. & Nissenbaum, N. (2007). A game design methodology to incorporate social activist themes. Proceedings of the Conference on Human Factors in Computing Systems. Politics and Activism (pp. 181-190). San Jose, CA: ACM.
Hailey, D. E. (2004). A next generation of digital genres: Expanding documentation into animation and virtual reality. Proceedings of the 22nd Annual International Conference on Design of Communication: The Engineering of Quality Documentation (pp. 19-26). Memphis, TN: ACM.
National Advisory Committee on Computing Qualifications. (2004). New Zealand Institutes of Technology and Polytechnics Qualifications in Information and Communication Technology. Hamilton, New Zealand: NACCQ.
Khaled, R., Biddle, R., Noble, J., Barr, P., & Fischer, R. (2006). Persuasive interaction for collectivist cultures. Proceedings of the Seventh Australasian User Interface Conference (pp. 73-80). Retrieved February 11, 2008, from http://portal.acm.org/
Massey University (n.d.). Tools for delivering scenario-based e-learning both locally and across the Internet. Retrieved March 12, 2007, from http://pbl.massey.ac.nz/
Mayer, R.E. (2001). Multimedia learning. Cambridge, UK: Cambridge University Press.
New Zealand Government. (2005). The Digital Strategy- Creating our digital future. Retrieved July 10, 2006, from http://www.digitalstrategy.govt.nz/Media-Centre/Publications/The-Digital-Strategy/ .
Oliver, R. & Stoney, S. (1999). Can higher order thinking and cognitive engagement be enhanced with multimedia? Interactive Multimedia Electronic Journal of Computer Enhanced Learning, 1(2). Retrieved June 15, 2006, from http://www.imej.wfu.edu/articles/1999/2/07/index.asp.
Orngreen, R. (2004). CaseMaker: An environment for case-based e-Learning. Electronic Journal on e-Learning, 2(1), 167-180. Retrieved July 2, 2006, from http://www.ejel.org/volume-2/vol2-issue1/issue1-art7-orngreen.pdf