Eastern Institute of Technology, Hawke's Bay, New Zealand
Verhaart, M. (2008). The virtualMe, an integrated virtual teacher framework. Journal of Applied Computing and Information Technology, 12(1). Retrieved January 28, 2022 from http://www.citrenz.ac.nz/jacit/JACIT1201/2008Verhaart_VirtualMe.html
With an ever increasing integration of web technologies into our lives, there is a need to look at how to capitalise on this technology. In a teaching and learning environment students, teachers and administrators are increasingly embedding web technology into their daily work. This paper describes research into an Internet-based, teacher-centred framework that can be used in a teaching and learning environment. A web-based prototype is described that has been used to investigate what features are desirable in a web-based teacher-focused information and knowledge framework, and a survey of existing and potential users is discussed in order to identify the perceived usefulness of the proposed framework.
Over the past few years there has been an increasing reliance on the Internet as a content and communication provider. In a teaching and learning environment students are relying on the Internet for their research, teachers are using the technology to prepare and deliver their content and online learning management systems are used to provide an environment that allows content organisation and student administration. In recent times this has been extended to providing social structures where students can blog (e.g., blogger, wordpress), post videos (e.g., youTube), share images (e.g., Flickr) and create a web presence (e.g., mySpace, BeBo). Alongside this is the advancing integration of mobile technologies with SMS text messaging, sound file sharing and podcasting (Cochrane, 2007).
How can information technology be capitalised on in a blended (faceto-face and online) teaching and learning environment? The research question addressed in this paper is "What features are desirable in a web-based teacher focused information and knowledge system?" Information and knowledge
In order to provide content, a mechanism is required to acquire, organise and disseminate information and knowledge. While the definition of information is clear, and can be thought of as "data invested with meaning" or contextualised data, there is still debate as to what constitutes knowledge (Davidson & Voss, 2002, pp. 52-53).
Central to the definition of knowledge is that it belongs to an individual. For example, Nonaka (1994, p. 15) includes "the commitment and beliefs of its holder". Meisenberger and Seiwald (2002, p. 10) emphasise the individual and include a social dimension: "only a small part of individual knowledge is generated through the process of individual experience. Most parts are socially derived."
Therefore, a knowledge acquisition, management and dissemination system should belong to an individual (as it is based on beliefs) and include mechanisms not only to represent information but also to allow for the social construction of knowledge.
What features are desirable for a web-based teacher-focused information and knowledge system? Content delivery is a major component and as such there is a requirement to acquire, organise and disseminate it. In face-to-face delivery, content is often viewed as a series of overhead projections (commonly in the form of PowerPoint slides). Notes may be provided that support the content of the slides, since the slides are usually created as a means to focus the face-to-face delivery and discussion.
In order to have a context, individual slides need to be organised within a structure. The presentation is placed in a learning domain, while the individual slides are placed in a predefined sequence. An identifiable issue arises when content is required in more than one place, for example, the ability to resize a graphic is relevant in all the domains where images are utilised, such as multimedia, web page development, and word processing.
The content on each slide is composed of a variety of media elements, or digital assets. These can include unstructured media such as text, images, sounds, animations and video or structured elements such as documents (doc or pdf files), flash animations, or links to web sites (URL). These assets may be reused in many places in the knowledge system. In a face-to-face session, students and teachers share their knowledge, possibly by using a data projector to focus on a small fragment of content.
In a web-based system, sharing can be achieved via a feedback system, and may be referred to 24 hours a day, 7 days a week.
In order to explore which features would be desirable, research was undertaken using an action research approach and several prototypes developed and trialled. Initial prototypes developed an overall structure, and these have evolved into the current prototype, called the "virtualMe" (Verhaart & Kinshuk, 2005). The virtualMe used the same user file as the prior V/2-Online prototype and became operational on January 1, 2006. The virtualMe embodies the concept of an environment that organises information and knowledge centred on an individual, in this case a teacher, and provides mechanisms for social interaction.
Based on feedback from the prototypes, a framework has been developed and four research areas were identified as desirable in a knowledge acquisition, management and dissemination system, and this forms the framework of the virtualMe: overall structure (manages content, its structure and presentation), annotations, media elements (multimedia objects), and environment (teaching and learning). Within this structure is an interrelationship between components and this is illustrated in Figure 1.
As information is "contextualised data" it is important to place content in a structure. The overall structure or framework is the virtualMe itself, that is, the information and knowledge important to the owner. At the second level, the content needs to be organised into logical domains. In a teaching system these would generally be the areas of expertise such as multimedia, or web construction, or may include interest areas such as research, hobbies, and so forth. A domain can further be broken down into topics, topics to lessons, lessons to slides, and slides into digital assets.
Therefore, to manage context a structure is desirable. In the case of the virtualMe prototype a backbone taxonomy (Guarino, & Welty, 2002) approach has been used to manage the overall structure. A multimedia taxonomy, or topic list, is illustrated by the centre column in Figure 2.
The concept of a slide of content is a fundamental building block of the virtualMe, and has proven to be a useful and usable size for managing and organising content. In order to provide a consistent reference the slide of content needed to be defined and in order to simplify discussion the term "sniplet" was coined. It has been defined as "a piece of knowledge or information that could be represented by one overhead transparency" (Verhaart, 2003).
A sniplet is illustrated in Figure 3 and shows some of the core dimensions of the virtualMe. The content is represented in a format suitable for projecting on a data-projector. The structure is displayed as the breadcrumb sequence (top left) and as a position in a sequence (balls on top right).
The sniplet (Figure 3) is composed of several digital assets. These include some text, an image (collage of screen shots), a link to an audio file, and a reference to the source of the information, and a URL link.
The ability to be able to manage digital assets is an important part of any knowledge system. A digital asset can be used by multiple sniplets, and this creates an interesting contextualisation issue. For example, if we look at an image of a waterfall (Figure 4), and separate it from the text attached it will lose context.
In order to manage the digital assets, a significant amount of research has been carried out. By attaching metadata, context can be retained and this concept has been used for many years. Unfortunately unless it is perceived to be useful, in many instances metadata is not created (Goldfarb & Prescod, 2002; Philips, 2000). The virtualMe uses meta-data attached to each digital asset to manage the context, for example, to create referenceing information or a descriptive tool-tip, and to provide referencing information (refer to the photo album image shown in the left panel of Figure 2).
Another desirable feature that emerged from the prototypes was the ability of the digital asset to contain multiple representations. For example, if an image is a photo of a person, it would be useful to additionally have a business card, a curriculum vitae, an audio clip with the correct pronunciation, and so forth.
To address this issue a multimedia object (MMO) package was designed and implemented containing a meta-data file that describes the object, and the multiple files (Verhaart, Jamieson & Kinshuk, 2004). In the virtualMe each media element is represented by an MMO, and when clicked on will hyperlink to the MMO package. This is illustrated in Figure 5 for a photo and Figure 6 where an image of a web page links to storyboard design elements including a navigation map, page description and element schedule, as well as an APA-style reference.
To allow for the acquisition of the knowledge of users (or site visitors) the virtualMe includes an annotation dimension (Verhaart & Kinshuk, 2006). This is becoming common on many web sites where user input and comment is requested. For example, the help system of Adobe LiveDocs (Adobe, 2008) is replicated online and allows users to enter comments, and Microsoft's support network (Microsoft, 2007) solicits responses in both Likert scale and formative styles.
Shown on the left of Figure 2 are a series of panels that allow for the titles of annotations that have been entered in the system to be displayed. This includes the title of in-context annotations that have been attached to a sniplet, as displayed in a panel at the bottom of Figure 3. The prototypes identified two annotation types: 'out of context' (or global) annotations, and 'in context' annotations. 'Out of context' annotations are those that are not directed at a specific piece of content and may be general comments to all users, for example, a comment that gave a URL to information about comet McNaught that was overhead in January 2007. 'In context' annotations are those attached to content, such as a sniplet (Figure 3).
The fourth and final area in the framework is the environment in which it is to operate. As the researcher is involved in teaching and learning the framework has been designed and implemented as a support tool for this context. Over the past two years a prototype has been used to deliver content to students at the researcher's institute. Modifications have been ongoing, based on issues found when using the system and from student comments. The prototype collected usage information and tracked user movements through the system, though technical issues did affect the user experience.
Two prototypes shared the same user files and the number of users that accessed either of the prototypes since January 1, 2006, that is those most likely to have accessed the virtualMe, was 223 (221 excluding researcher and guest account). Users identified their relationship to the researcher when they created their initial profile, and this is illustrated in Figure 7.
In order to assess the efficacy of the virtualMe framework, a survey was developed that would investigate four areas: the overall framework (context, structure and content); the annotation framework; resource acquisition, management and sharing (media elements); and teaching and learning (overall context).
A well-respected method used to evaluate information technology-based systems is the Technology Acceptance Model (TAM), as conceived by Davis (1989). The Institute for Scientific Information Social Science Citation Index listed 335 journal citations between 1999 and 2004 (Money & Turner, 2004). It has proven to be a reasonably accurate predictor of both users' intentions and system usage, and is centred on two primary belief constructs: ease of use and perceived usefulness.
Based on this model, a survey was constructed with questions using ordinal values on a Likert scale with an opportunity for respondents to provide free-form responses. Ethical approval was obtained. Requests to complete the survey have been circulated to a wide audience of people involved in teaching and learning. As this is a technology-related project a certain level of computer expertise is necessary, and since the system uses web technologies, this automatically limits responses. Once these elements were identified, a survey into the perceived usefulness and easy of use has been carried out.
To be able to complete the survey a respondent must first have used the system. For those unfamiliar, a walkthrough was provided as a PowerPoint presentation, a Word document with written instructions and a video. Respondents were then encouraged to "play" on the system, and once they felt concepts were understood, were asked to complete the survey.
Limitations were identified and questions to determine bias were included. A confounding variable is the ease of use of the system, that is, whether the system itself influenced the responses. Hence, the second part of the TAM model solicits feedback on ease of use.
The survey instrument was then pre-tested using colleagues and existing students and modified, based on the feedback given. Apart from past students that had used the system, the requirement to learn then play with the system required a significant time commitment from survey respondents. Hence, responses from a large number of people were solicited.
The on-line survey form continues to collect data, but a full analysis has been conducted on results retrieved form the database as at the 19th September 2007. Fifty-seven responses were received from a wide variety of information technology competent respondents. The survey respondents were made up of a wide cross section of the teaching and learning community. To determine whether the sample size was appropriate and the responses can be considered valid, a reliability analysis was conducted on the data. The values of Cronbach's alpha for perceived usefulness and ease of use were 0.847 and 0.875 respectively. A reliability coefficient of 0.7 or higher is considered "acceptable" in most social science research situations (UCLA, n.d.).
Of a total of 57 respondents, 41 identified themselves as students, 13 as educators, 5 as visitors, and 6 as 'other', with 6 of them indicating they were in two or more of the categories. There was a good gender balance, with males making up 51% of the respondents and females 46% (3% left this question blank). The ages of respondents were also representative of the potential users of virtualMe, with 38% between 20 and 29, and 23% between 30 and 39.
In terms of expertise, approximately 1.8% indicated novice, 3.5% ok, 19.3% confident, 45.6% proficient and 29.8% expert, which could be expected from respondents who would be prepared to complete a technology-based survey, and this is illustrated in Figure 8.
There is some necessary bias inherent in surveys related to technology, particularly those implemented in an online system, as the respondents require literacy in the use of this technology. As the ubiquitous nature of the Internet increases, the number of people who will have an understanding of this technology will reduce this bias. In order to provide meaningful feedback, respondents needed to be reasonably proficient in using web technologies.
To ascertain the efficacy of the virtualMe, perceived usefulness was measured. An ordinal Likert scale was used with options of 'useless', 'not useful', 'useful', 'very useful' and 'don't know', and it was possible for respondents to leave the response blank. 'Don't know' and blank responses are excluded from the analysis since they do not provide a measurable response either for or against the question posed. The option 'useless' was almost never selected and in order to facilitate analysis has been combined with 'not useful'.
In order to assess whether the usability of the system was a factor in the respondents' replies, ease of use was also measured. An ordinal Likert scale was used with options of 'difficult', 'not easy', 'easy', 'very easy' and 'don't know', and it was possible for respondents to leave the response blank.
Table 1 summarises some of the findings regarding the overall framework (as percentages).
It can be observed from the results that the way in which content is presented is perceived as important. Providing referencing information was generally perceived as being important and this probably reflects the teaching and learning focus of the respondents. The feeling that a user was interacting with a person was deemed not useful by 18% of the respondents, though this needs to be balanced with the 82% who thought otherwise.
The ability to customize was not rated as highly as other capabilities, and this is consistent with research described by Bush and Tiwana (2005, p. 69) who found "that personalization only affects stickiness after the knowledge network has established itself".
Responses related to ease of use for the overall framework are summarised in Table 2.
For the overall framework, at least 83% of respondents found using the system was easy or very easy in response to all four questions, which indicates that using the system itself was not a significant factor for survey participants when responding to the questions.
Reviewing the written comments some users felt it would be useful to create a messaging system so that users were aware of who was online and could initiate dialog. Comments were also made regarding the importance of the navigation system, including the breadcrumbs, and were generally favourable. Providing the ability for a user to highlight sections would be useful, as would a hide/unhide capability for annotations. The main negative comments revolved around the amount of activity presented, that is, the cognitive loading and some difficulty in navigation. Additional comments were generally positive regarding the overall system functionality and what it was attempting to achieve.
In addition the question "does gender affect the perceived usefulness of the overall framework?" was considered and a correlation between the male and female scores was investigated. Of the participants, 29 indicated they were male and 26 female (2 left this blank). Kendall's tau (t) is a suitable measure for looking at the strength of association between the values, and the hypothesis is that "the variables are dependent". With Kendall's tau coefficient ranging between -1 and 1, 1 giving agreement and 0 implying independence, in this case t = .479 (n=9) so it can be deduced that there is a moderate correlation between the male and the female responses. As there is not a strong correlation this could be an area for future research.
Results from selected questions relating to the annotation framework are displayed in Table 3.
Text-based annotations were considered useful and very useful by 100% of respondents, with this falling to 94% for multimedia-based annotations.
Annotations were perceived as being useful and significantly as very useful with their ability to influence a change in content over time. One of the respondents commented "cannot say whether and how often I would really use it also. But it's nice to have the option!" Reviewing the actual annotations entered into the prototype, 158 annotations were added to the system by users, and 51 by the owner of the virtualMe site. With 221 users registered in the system, and excluding the owner of the virtualMe, this equates to almost 1 annotation for every 2 users with 55% of them being entered voluntarily.
Interestingly in the years that both the earlier V/2-Online and current virtualMe prototypes were in use, there have been no inappropriate or spam-type comments. From a usability point of view, annotations can be "hidden" by the owner of the virtualMe should this be necessary and this has been used where an annotation has become out of date (e.g. comments about the state of marking in an assessment) or entered in error twice.
Apart from adding annotations out of context, in excess of 90% of the respondents felt that adding an annotation to the system was easy or very easy. Although, with 17% of the respondents indicating that they found adding an out of context annotations difficult or not easy, this would need to be reviewed in any future development of the prototype. User comments did not highlight any obvious issues.
Resource acquisition, management and sharing at source Responses relating to "can a model be developed that has the ability to retain context while transferring the content from one person to another and from one place to another?" are summarised in Table 4.
Drilling down into a media element is seen as useful or very useful by 94% of participants, and all participants felt that the ability to display media elements is a variety of ways was useful or very useful. Only a few written responses were completed by participants that elaborated on their answer. In all cases at least 90% of respondents indicated that the implementation was easy or very easy to use, indicating that their responses to perceived usefulness were not influenced by the implementation.
There were several features in the virtualMe system designed to be used in a blended (mixture of face-to-face and online) teaching and learning environment and Table 5 summarises responses from this perspective.
From a teaching and learning perspective, 98% of the respondents believed that virtualMe would be useful or very useful, with only 2% of respondents indicating that using the virtualMe for teaching and study would not be useful. In the case of productivity, only 11% of respondents felt that it would not be useful. From using the prototype to deliver actual content to students, the researcher found that this indeed was the case, with a considerable effort required to build the MMOs and restructure courses into logical sniplets. Indeed the user interface to add these features was not well developed since it was not something of concern to the users.
As in the previous areas of the research, in three out of the four cases in excess of 90% of the respondents felt that the prototype was easy or very easy. Results from the survey indicate that displaying content to learn theory would be an area that should be developed further in the future. Respondents were asked to comment on whether there were any issues that could have affected their responses. Apart from a problem with the system running slowly, there were no other major issues identified.
This research question considered in this paper was "What features are desirable for a web-based teacher-focused information and knowledge system?" and a framework that can be used in a teaching and learning environment was described. Four areas were outlined: overall framework, annotations, media elements, and environment. On-line prototypes that explored the viability and feasibility of such a framework have been developed to support the framework and were described. The current iteration has been called the virtualMe, as it is centred on the educator.
A survey was conducted to ascertain the perceived usefulness of the framework and the features. Results indicate that the proposed virtualMe framework and the four areas are perceived to be useful or very useful by a significant number of the respondents, in most cases at least 90%. The prototype and survey can be accessed at http://www.virtualmv.com.
Adobe (2008). Flash 8 LiveDocs. Retrieved February 6, 2008, from http://livedocs.adobe.com/flash/8/index.html.
Bush, A. A. & Tiwana, A. (2005). Designing sticky knowledge networks. Communications of the ACM, 48 (5), 66-71.
Cochrane, T. (2007). Welcome to the Unitec Learning Technologies WIKI! Retrieved February 6, 2008, from http://ltxserver.unitec.ac.nz/mediawiki/index.php/Main_Page
Davidson, C., & Voss, P. (2002). Knowledge management. An introduction to creating competitive advantage from intellectual capital. Auckland, New Zealand: Tandem Press.
Davis, F. D. (1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319-340.
Goldfarb, C. F., & Prescod, P. (2002). XML Handbook (4th ed.). Upper Saddle River, NJ: Prentice Hall.
Guarino, N., & Welty, C. (2002). Evaluating ontological decisions with OntoClean. Communications of the ACM, 45(2), 61-65
Meisenberger, S., & Seiwald, J. (2002). Institutionalised knowledge - Properties of knowledge "in" and "of" institutions. Retrieved June 26, 2002, from http://www.sses.com/public/events/euram/complete_tracks/epistemology/meisenberger_seiwald.pdf
Microsoft (2007). Help and support. Retrieved March 11, 2007, from http://support.microsoft.com/
Money, W., & Turner, A. (2004) Application of the technology acceptance model to a knowledge management system. Proceedings of the 37th Hawaii International Conference on System Sciences, Vol. 8 (p. 80237b). Honolulu: IEEE.
Nonaka, I. (1994). A dynamic theory of organizational knowledge creation. Organization Science, 5(1), 14-37.
Phillips, L. A. (2000). Using XML: Special Edition. Indianapolis, IN: Que.
UCLA (n.d.) SPSS FAQ: What does Cronbach's alpha mean? Retrieved October 1, 2007 from http://www.ats.ucla.edu/STAT/SPSS/faq/alpha.html
Verhaart, M. (2003). Developing a capture of knowledge system based on sharable and self documenting learning objects. Educational Technology & Society, 6(3), 1-16.
Verhaart, M., Jamieson J., & Kinshuk (2004). Collecting, organizing and managing non-contextualised data, by using MVML to develop a HumanComputer Interface. In M. Masoodian, S. Jones, & B. Rogers (Eds.), Lecture Notes in Computer Science, 3101 (pp. 511-520). Heidelberg: Springer-Verlag.
Verhaart, M., & Kinshuk (2005). Developing a user centered model for creating a virtual learning portfolio. In C. Ghaoui, M. Jain, V. Bannore, & L.C. Jain (Eds.), Studies in Fuzziness and Soft Computing, 178 (pp. 203-232). Heidelberg: Springer-Verlag.
Verhaart, M., & Kinshuk (2007). The virtualMe: An integrated teaching and learning framework. In G. Richards (Ed.), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2007 (pp. 6644-6653). Chesapeake, VA: AACE.