The purpose of this paper is to illustrate the development and testing of an innovative mobile application using design-based research.
This paper reports on the process of transformation of existing printed course material into digitized content through design-based research where design, research and practice were concurrently applied through several iterations of the mobile application. For this transformation, one session each from BSc in Nursing, Bachelor of Pharmacy and Bachelor of Medical Laboratory Sciences was selected. In the first phase of the design-based research, the main research question was formulated. In the second phase, a mobile learning application (OUSL MLearn) was designed and developed to address the research question. In the third phase, this application was evaluated by five groups of stakeholders: content experts to validate the content; educational technologists to check the alignment of technical and pedagogical features; novice users to check the overall effectiveness of the application; developer to develop the application, to check the ease of usage; and researchers to identify the impact of this innovation. These stakeholders were closely involved throughout the whole process which lasted over a period of four months. At the end of this development phase, the results were reflected upon and used for further enrichment.
It was observed that the developed mobile application was accessible, appealing and pedagogically constructive for users. However, optimization, development time, technical and organizational issues, workload of academics and production costs were identified as major challenges.
This study was based on the findings of a small sample of potential users.
The findings have implications for designing culturally adaptive interactive mobile applications.
This study will benefit practitioners to design culturally sensitive mobile learning courses and researchers to conduct design-based research.
Emerald Publishing Limited
Copyright © 2018, Buddhini Gayathri Jayatilleke, Gaya R. Ranawaka, Chamali Wijesekera and Malinda C.B. Kumarasinha
Published in Asian Association of Open Universities Journal. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode
Mobile technology is an exceptionally fast-growing field that is closely connected with our work and day-to-day lives. There are new developments added to its growth every day with emerging new patterns of usage, having both positive and negative implications.
In the twenty-first century, higher education institutions had to be reconstructed to adapt to changes with the increasing global competition, the growing need for higher education, the changing nature of information, rapid developments in Information and Communication Technologies (ICT) and the varying expectations and demographic features of learners (Kukulska-Hulme, 2005a). The changes in the dynamics of ICT, institutions and learners influence the academics working in higher education institutions to change their teaching approaches and strategies.
However, we have not seen a noteworthy adoption of these technologies in the education sector even though they are available everywhere (ubiquitous) and have tremendous potential in addressing needs of the individual learner through their unique capabilities. Furthermore, owing to the rapid changes of mobile technologies, including devices and communication technologies have opened up new research opportunities and even change the focus of research (Parsons, 2014). Krull and Duart (2017) reported that in higher education, mobile learning is a growing field of research as evidenced by reviewing journal publications between 2011 and 2015. The major results of their study were that the most researched theme was on m-learning applications and systems, used both quantitative and qualitative studies and were targeted at students. As both faculty and student adoption play a crucial part in the success of mobile learning initiatives, they recommend future studies to look into the implications for both faculty and students.
However, there is a scarcity of research articles related to mobile learning and methodological frameworks for designing sustainable mobile learning activities (Nouri et al., 2016). The purpose of this research study was to address this gap by applying design-based research in designing a mobile learning solution for the undergraduates of the Faculty of Health Sciences of the Open University of Sri Lanka (OUSL). It reports on the findings of the testing phase of the mobile solution by five groups of stakeholders: content experts, educational technologists, developer, novice users and researchers prior to the delivery of the first cycle.
The first section of the paper defines briefly the mobile learning, design-based research and employing design-based research in mobile applications, and stresses the importance of conducting design-based research for future technological innovations. The second section briefly describes the context. The third section examines the methodology adopted for the design-based research for new technological innovations in teaching learning using mobile applications. The fourth section is dedicated to the findings which were collected from all the stakeholders illustrating the potential for innovative teaching practices through mobile learning. The final section is a critical examination of the viewpoints expressed by all stakeholders and formulating guiding principles for both designing mobile learning solutions and on how to conduct design-based research in mobile applications.
Mobile devices are portable, lightweight devices such as mobile phones (cellphones, or handphones), smartphones, palmtops and handheld computers (Personal Digital Assistants or PDAs), tablet PCs, laptop computers and personal media players. These devices can be carried around easily and used for communication and collaboration, and for teaching-learning activities that are different from what is possible with other media.
Traxler (2009) has pointed out that mobile devices together with mobile communication technologies have influenced all fields including education and currently undergoing a transformation. In fact, he called this transformation period as mobile era. He further stressed that most of these mobile devices are not designed specifically for education or training but designed for personal or individual usage which mainly used for one-to-one social interaction.
In the context of education, these mobile devices offer diverse learning opportunities such as portability, social interactivity, context sensitivity, connectivity, individuality and affordance to people in academic settings or non-academic settings (Crompton, 2013). Therefore, these mobile technologies are very useful for learners where they could engage in educational activities and learn by themselves without the constraints of having to come into the institution.
Since the introduction of the term mobile learning in 2005 (Crompton, 2013), many scholars and practitioners have attempted to define it and initial definitions were focused solely on devices and technologies or techno-centric (Crompton, 2013; Keskin and Kuzu, 2015). Most widely accepted mobile learning definition is “learning across multiple contexts, through social and content interactions, using personal electronic devices” (Crompton, 2013, p. 4). This definition encompasses four central constructs associated in mobile learning: pedagogy, technology, context and social interactions. Table I illustrates the categorization of the attributes of mobile learning into these central constructs.
With these attributes, it has much in common with other types of e-learning on desktop computers but allows more diverse and changing locations, more immediate (anytime) interaction, and connect through smaller, often wireless devices (Kukulska-Hulme, 2005a) enabling both advantages and drawbacks.
Learners can choose their own learning path to achieve their learning goal by using their own private mobile device. Hence, mobile learning can take place when the learner is not at a fixed, predetermined location or learning that happens when the learner takes advantage of the learning opportunities offered by mobile technologies (O’Malley et al., 2005, p. 7). As a result, they have spontaneous personal accesses to the large number of learning resources via the internet.
However, in order to create this kind of ambient technology, providers need to design “learning enhanced” buildings and public spaces, by providing devices or establishing systems to respond to on-the-spot interactions.
Several researchers have studied the development of the theoretical frameworks of mobile learning: mobile education (FRAME) based on activity theory (Koole, 2009), construction of knowledge through the exchange of knowledge via pervasive mobile devices (Sharples et al., 2007) based on the conversation theory, developed by Pask (1975), modification of transactional distance education theory for mobile learning (Park, 2011) and educational research on mobile communities (Frohberg, 2003).
Still there is a lack of transferable design frameworks on mobile learning (Cochrane, 2013) irrespective of those developed earlier. There is a greater need by the practitioners, instructional designers and trainers on design theories, so that they may in a better position to integrate mobile technologies into learning environments in an effective manner and to make these technologies more beneficial to users (Koszalka and Ntloedibe-Kuswani, 2010; Park, 2011; Rajasingham, 2011).
When designing instruction, generally learning content is regarded as the most important factor. Thomas et al. (2002) stressed that culture has to be considered as a “dimension” of instructional design capturing, three layers: purposeful intention, interaction with learners to involve them in the design process and for introspection to identify one’s own cultural values and biases. The importance of language, culture and context is also highlighted in teaching and learning (Gunawardena et al., 2009). Chen et al. (2006) emphasized the importance of paying attention to provide support (technical, learning and social) and resources (language, culture and context, and learning content), so that learners can make effective connections between resources and the support. Therefore, designing effective learning for global audiences requires not cultural neutrality but cultural “inclusivity” (Frechette et al., 2014; Henderson, 1996; Powel, 1997) as the online medium (internet) itself is a culturally derived phenomenon (Bowers, 2000) and functions as an incubator for a shared cultural experience (Selvin, 2000). Therefore, it is crucial to explore the “cultural understanding” of learners (Rogers et al., 2007). Furthermore, online tutors and mentors should be more sensitive to culture when facilitating knowledge construction to global audience via online where both learners and tutors bring their own cultural identities and they have to use face-saving and negotiation strategies to build trust to develop online communities (Gunawardena and Jayatilleke, 2014; Gunawardena et al., 2009).
abbreviations (e.g. lol: “laugh out loud,” yolo: “you only live once”);hieroglyphics (e.g. xoxo: “hug kiss hug kiss,” T-T: “tears,”:-: “smiling eyes”), add a new bullet emotions; and
special characters, which are unique visual representations and difficult to classify as wither text or image.
It also is necessary to get the views from learners on the designed product to shape the design itself (McLoughlin and Oliver, 2000) through evaluation studies.
The aim of the design-based research is to improve educational practices through systematic but flexible methodology through iterative analysis throughout the design, development and implementation of the product (Wang and Hannafin, 2005, pp. 6-7). These educational practices are based on the views gathered between researchers and practitioners in a normal setup and these practices would lead to formulate contextually sensitive design principles and theories (Wang and Hannafin, 2005). Thus, fulfilling the ultimate goal of design-based research by building stronger connection between educational research and real-world problems. In this scenario, researchers and practitioners are integral part of the research process and they are closely involved from the initial phase of design and development of the product to the final phase of implementation. However, research validity of design-based research is criticized by some due to the involvement of researchers where they felt that researchers may not be reliable and faithful in their judgments (Barab and Squire, 2004). Further, the intervention may not be replicated in other settings as design-based research is contextually dependent (Design-Based Research Collective, 2003).
Review conducted by Zheng (2015) showed that design-based research studies were conducted with diverse sample groups but mainly with students in higher education (29 percent), under various learning environments including distance learning (14 percent) and blended learning settings (12 percent). Natural science (38 percent) was selected as the most researched learning domain while medical science was the least selected learning domain (2 percent). The technological intervention was the major type of intervention used in the design-based research (53 percent) followed by the integrated teaching models (16 percent), other models (16 percent) and instructional methods (14 percent).
uncertainty about how it differs from other forms of research; uncertainty about how it differs from design, or why design is not research; and uncertainty about what might make it effective (if it is).In view of these uncertainties, they described the design process consisted of six iterative phases: focus the problem, understand the problem, define goals, conceive the outline of a solution, build the solution and test the solution (Easterday et al., 2014, p. 320) which are recursively nested within each other. These phases could be compared with the well-researched four-step framework of Reeves (2006), where he explains design-based research as a process that consists of four steps (Table II).
It is clearly evident from the table that the fourth step, that is documentation and reflection to produce design principles of Reeves’s framework, was not identified in Easteday et al.’s six-phase/step framework.
Ma and Harmon (2009) pointed out that the guidelines and the process presented by Reeves’s (2006) four-step framework provided valuable guidance on how to conduct design-based research from the long-term perspective; however, they felt the inadequacy of the framework to conduct design-based research at the individual level as it was not clear about the conduct of research activities in each step. They further enriched the Reeves’s framework by providing a more detailed and comprehensive development process incorporating research elements to the framework with specific guidance (Ma and Harmon, 2009, pp. 77-78). The main steps in the framework are connected linearly from steps 1 to 4 with connecting loops to each step. They mentioned that “by no means as clean and liner as it might appear” and “researchers may examine their own context to make appropriate modifications” (p. 80). These guidelines enable researchers who are new to design-based research to conduct design-based research systematically and logically.
Both design-based research and action research share many epistemological, ontological and methodological foundations thus sharing a common “meta-paradigm” pragmatism (Cole et al., 2005) and many find it difficult to distinguish the two (Easterday et al., 2014). Generally, design-based research is conducted by a research and design team whereas action research is carried out by a single teacher (practitioner), guided by a theory (Anderson and Shattuk, 2012), focused primarily on already designed product/process and its application into an everyday context (Ørngreen, 2015) and contributes toward theory building (Cole et al., 2005). However, collaboration between practitioners and researchers is not clearly described in the design-based research literature (Kolmos, 2015) and needs further investigation.
However, some researchers have combined design-based research with action research (Keskin and Kuzu, 2015). They feel that the research methodology most appropriate to the third step of the Reeves’s framework is through action research. It can make the product highly effective, efficient and useful by allowing repeated development of the product until all the identified errors of the product are resolved during the testing phase (Susman and Evered, 1978). In view of this notion, Keskin and Kuzu (2015) developed the model by combining the design-based research model put forward by Ma and Harmon (2009), and the action research cycle suggested by Susman and Evered (1978) regarding information systems. The model also has the same four steps proposed in the Reeve’s framework; however, all the steps in the Keskin and Kuzu’s model are interactive with each other. The design principles and the theory can be developed in the fourth step, based on the analyses of the data collected in each step (Ma and Harmon, 2009). By following iterative research process in the design-based research, it attempts to refine the innovation systematically while also proposing design principles unlike in evaluating innovative product or intervention at the end of the development phase.
Many researchers have conducted reviews related to the application of design-based research as a research methodology in conducting various research studies (Anderson and Shattuk, 2012; Krull and Duart, 2017; Zheng, 2015). Major findings of these studies indicated that the majority used design-based research for technological interventions and applications. Anderson and Shattuk (2012) reported that the majority of interventions (68 percent) involved the use of online and mobile technologies. According to Zheng’s study reviewing research articles and publication from 2004 to 2013, technological interventions applications were the most researched area (53 percent) and were to test the effectiveness of the learning environment or a particular tool. However, the nature or the type of the tool was not specifically mentioned in his study. In reviewing journal publications between 2011 and 2015, Krull and Duart (2017) revealed that mobile learning applications and systems were the most researched area conducted using design-based research in higher education. Keskin and Kuzu (2015) combined design-based research and action research to conduct professional development program for academics using M-learning system.
The OUSL is unique in its teaching methodology as it is the only national university in Sri Lanka which is dedicated to open and distance learning. Unlike in conventional universities, the OUSL mediates instructions mainly through print course materials. With the advent of various technologies, the OUSL has gone through generations of technology integrating audio-visual, multimedia and online learning into the core print course materials (Jayatilleke et al., 2009).
Having faced with many challenges with respect to distributing printed course material on time and to reduce production and delivery costs of the course materials, there have been many suggestions from time to time to use other technologies. However, print has remained as the core medium of instruction even though many such initiatives have been taken to promote offering courses entirely online.
Aligned with this notion and also considering the immense potential of using mobile technologies for learning, the OUSL has recently proposed an alternative option to address these challenges. Providing course materials in PDF format loaded on a tablet computer would be a viable option as tablet computers are becoming cheaper by the day, harnessing the potential of improving the learning experience and thereby effect institutional change.
Hence, Faculty of Health Sciences of the OUSL took the initiative to investigate the viability of transforming the existing print course material, and offer them through mobile learning for the undergraduates of the Faculty. This project was carried out from a research grant of the OUSL which enabled to experiment with novel mediating mobile technologies.
Three Bachelor’s degree programs are offered by the Faculty of Health Sciences; Nursing; Medical Laboratory Sciences and Pharmacy. One session each from a degree program was transformed retaining the already existing content and the original framework as these courses are still being offered by the OUSL.
In this study, design-based research model put forward by Ma and Harmon (2009) was used as the framework as it provided the processes clearly (Figure 1). The “analysis of practical problems” is the first phase as in the Reeves’s model. In this phase, a practical problem is identified and the related literature about the practical problem is reviewed. The second phase is “development of solutions” for the practical problem identified in the first phase by conceptualizing a solution within theoretical framework, identifying research purpose and development method, developing a prototype that serves to address the research problem. The third phase is “evaluation and testing of solutions in practice.” The final phase is “documentation and reflection” where design principles are generated and documented in order to provide guidance for practitioners and researchers who are interested in conducting design-based research.
This study was also influenced by the design-based action research model put forward by Keskin and Kuzu (2015) where phase 3 is an iterative cycle rather than a linear process. In this phase, problems related to the prototype are recognized and action plans are developed. At the implementation, these plans are implemented and the consequences of the action are evaluated and reflected. This process continues until all problems are solved.
In this study, development and testing of the mobile application (phase 3) was carried out concurrently with the phase 2 through formative evaluation. Phase 2 and phase 3 were closely linked and phase 3 was incorporated in the phase 2 of the cycle (Figure 1). These two phases were not separated cycles as in Keskin and Kuzu’s model. Since phase 4 (documentation and reflection) was also closely connected with these two phases through reflection, the connection between phase 3 and 4 was illustrated in a two way arrow.
Since design-based research is a multi-phase study, the present study involved five groups of stakeholders. In this study, researchers took the initiative and were involved from the beginning of the design process together with the developer, content experts/practitioners and educational technologists. All these stakeholders were closely involved throughout the whole process which lasted over a period of four months. The formative evaluation was carried out with four content experts, four educational technologists, six novice users, four researchers and one developer.
The analysis of a practical problem by researchers and practitioners is the first phase of this design-based research. Researchers in this study are also practitioners: two are teaching zoology/health courses while other two are training academic staff on online learning/educational technology. They have experienced the practical problem faced by the OUSL for many years; that is the difficulty in producing timely printed course materials to OUSL students with increasing student numbers. This practical problem is equally important to both OUSL students and the institution (OUSL). Thus, researchers took the initiative to conduct this research study through design-based research, where they felt the problem was significant to the learning community of the OUSL. They also believe that the findings will provide evidence to inform decision makers on the viability of providing a tablet computer loaded with the content to students so that decision makers may be in a position to take data-driven decisions rather than taking ad hoc decisions.
An earlier study carried out with the students of the British Open University showed that the majority preferred e-books as a complementary technology and still would like to receive print course materials (Kukulska-Hulme, 2005b, p. 130). Researcher further reported that learners faced difficulties in downloading e-books, getting satisfactory page and font size, navigation and cursor control, etc. With many technological advancements over the years, still students perceive printed texts are easier to read, understand and navigate, and have long-term access even though the digital texts are becoming cheaper (Baglione and Sullivan, 2016). Comparative studies have been conducted to investigate the effects of digital reading (e.g. reading a word or PDF file on screen) with print reading; however, not much research has been carried out into examining learners reading behaviors and the educational benefits of recent, more flexible visually presented texts (Rha, 2014, p. 51). Rogers-Estable (2018, p. 48) reported that many faculty members commented that if electronic texts (eTexts) are purely PDF files (or glorified PDFs) then there is no advantage in using them with students.
Based on the literature review, the decision was taken by the research team, not to provide a digitized text as a PDF (or as a glorified PDF) to learners (as an e-book) but to provide a mobile learning application with enhanced version of the already existing print material with additional pedagogical, technological (interactive), contextual and social interactive attributes associated with mobile learning with innovative strategies and tools. Social interactive attributes inherent to mobile learning was used; however, less priority was given to design peer/tutor interactions in this mobile application as it was designed as a stand-alone package to study offline considering the specific requirements of the target group, that is health professionals with demanding work pressure. However, learners have the opportunity to use the mobile application either online or offline. They can also use other channels such as e-mail and social media to discuss the content if they wish to collaborate socially.
According to Ma and Harmon (2009), identifying the purpose and research questions for a development iteration was discussed as the third step in the second phase (p. 77) even though they have highlighted the importance of it before commencing the development (p. 80).
In this study, research questions were formulated in the first phase as it was felt necessary to identify the purpose and the research questions before starting the development as they will direct and guide the development process through research.
The following research questions were formulated after analyzing the practical problem:
How to design a mobile application using an existing print course material?
What was the process carried out when transforming the existing print materials into mobile application?
What types of interactivity features were added to the mobile application?
What were the challenges faced by content experts, developer and educational technologists when designing mobile application?
In the second phase, a mobile learning application called “OUSL mobile learning” (OUSL MLearn) was designed and developed specifically for the Android mobile devices to address the principle research question within the theoretical framework.
The existing print course materials were originally designed based on teaching and learning theories such as Guided didactic conversation in distance education (Holmberg, 1983). However, transformation of existing printed course material into digitized content requires additional research related to mobile learning such as designing content with in-built interactive features for mobile devices. Furthermore, learning is situated and contextual. Thus, research and practice were concurrently applied through design-based research with several iterations of the mobile application.
Having conceptualized the solution, next step was to decide whether research should be conducted while developing the solution. Since the solution in this study was to develop a mobile application through several iterations, research was an integral part and relevant research studies with respect to the needs and the requirements of the stakeholders (teachers and students), learning preferences of students, cultural propensities were considered when designing user-interface, content development and system technical design.
This mobile application was based on the existing content of the printed material thus, restricting the design and development of the mobile application. Therefore, first step was to develop a prototype that was adequate for the purpose. Having reviewed the literature on various types of prototypes, an icon-based prototype was selected for the development of mobile application as the majority learners are visual learners (Rha, 2014).
In order to develop the mobile application, the first meeting was conducted with the content developers, educational technologists, researchers and the developer and discussed the overall objectives of this project. Three sessions (one session per a degree program) of the existing print materials were handed over to the developer highlighting the requirements, providing necessary information and devices (tablet computers). Developer was given the freedom to select appropriate technologies and tools to develop the mobile application to use with the specified tablet computer. This decision was based on the assumption that the OUSL will provide the standard tablet computers to all OUSL learners rather than requesting them to purchase or use their own tablet devices in order to minimize the technical problems. HTML was the main tool for the development and other tools were also used to enhance the capabilities of the mobile application.
First iteration was to discuss the prototype for the development of the session structure and subsequent iterations developed the prototypes for the course structure and the system architecture. Interactive features such as self-assessments questions and embedding videos were incorporated later.
The following section will describe in detail the design of the OUSL MLearn mobile application.
The system architecture was designed for the entire university which serves as the mobile platform (OUSL MLearn) for the OUSL. In order to make the system user friendly, the unique icon-based system was designed (Figure 2(a) and (b)).
The navigational structure for this system was connected to the main pages forming a semantic network as illustrated in Figure 3. The icon-based system was connected to the home page of the system, then to the Programmes Page where students can select their own program, followed by the Course Page and the Session Page, respectively.
Each course was designed in such a way to make the course as a stand-alone module which can be studied offline. This decision was made considering the baseline survey of the undergraduates of the Faculty of Health Sciences and their past experience of not accessing learning resources through internet (Jayatilleke, Wijesekara and Ranawaka, 2017).
The existing OUSL print materials were designed as self-instructional materials with intermittent activities to facilitate “guided didactic conversation” with text (Holmberg, 1983), incorporating advance organizers at the beginning and summaries at the end (Melton, 1997; Rowntree, 1990). These materials are designed using pedagogical features based on learning theories, research and practice.
These pedagogical features were retained in the mobile application. For instance, advance organizer at the beginning and summary at the end of each session were designed based on the pedagogical features of the original print course materials. Research shows that an advance organizer serves as a schema for the learner to associate new concepts with the already known concepts and to connect them meaningfully (Ausubel, 1960), whereas a summary (post organizer) provides a synopsis that helps the learner to get a holistic picture of the concepts learned in the session. In addition, advance organizers help diverse learners, in particular FD and FI learners, respectively. Research studies have revealed that FD learners are holistic in nature and need external guidance to solve problems while FI learners are serialistic and use their own cues to solve problems (Witkin et al., 1977). Since OUSL learners are diverse, course materials have to provide provision for these two groups to learn the content without the help of the teacher. However, content was re-designed as smaller chunks to suit the mobile screen to avoid overload of information based on Sweller’s (2011) cognitive load theory.
Additional features were also were incorporated to accommodate specific requirements necessary to learn using mobile devices. Each session was transformed considering the four pedagogical aspects of instructional design, namely information design, instruction design, interface design and interaction design. Findings of the earlier research studies on online learning were also considered in designing this mobile application (Table III).
Figure 4 illustrates the screen casts of the animated instructions proving study guidance on how to use mobile device.
Printed course materials use icons in front of the major pedagogical components such as learning outcomes, self-assessment activities (activities), online/video integration, etc. and use them as access devises. These icons were specially designed as authentic learning objects to maintain the OUSL identity across all OUSL materials. These icons were also used in this mobile application to maintain the OUSL identity. These features could be considered as contextual attributes of mobile learning. Certain new icons were added to represent functional specific requirements associated with the mobile application (e.g. Menu, Note, etc.). Typical icons generally represented in global community were used to represent images and settings. The layout of the program control, learner control and specific icons is illustrated in Figure 5.
This mobile application integrated a video, enriching the existing content using the affordances of mobile technologies and designed as an activity activity, based on the video. Generally, OUSL students do not watch videos, unless they are compulsory or integrated in the course materials. Thus, research and practice were considered in the design and development of this mobile application in line with the guidelines of the design-based research (Figure 6).
In addition to the instructional design features, adaptive technologies were also incorporated in the mobile application considering the needs of the heterogeneous nature of OUSL learners (Jayatilleke, 2016). Table III illustrates these features. Learner has the opportunity to adjust the size of the font (Figure 7) and images, taking notes, highlighting the text and copying and pasting facility were some of the adaptive technologies used in this application.
The next section provides the detail account of the evaluation and testing phase of the mobile application.
In the third phase, this mobile application was regularly tested through formative evaluation which was an integral part of the design methodology. It helped to judge strengths and weakness of the innovation while still at its developing stage, for the purpose of revising the instruction. As mentioned earlier, second and third phases conducted concurrently and could not be separated during the research process. Certain features were added after getting the feedback from various stakeholders during the testing phase.
In this phase, appropriate research methods were identified, collected and analyzed data to answer the research questions. Qualitative methods were used in gathering data since design and development of the innovation need in-depth analysis of the innovation. A research diary, committee meeting records, observational sheets of the users while using the tablet interview schedule for users, and checklists for error identification were used as data collection tools.
Content developers of these three sessions (four females), educational technologists (two females and two males), researchers (three females and one male) and one developer (male) were the members of the research and development team from the inception of the research project.
Purposeful sample was used to select the subjects as novice users where they have not followed these courses before to test this innovative mobile application. All the novice users were graduates in different disciplines (BSc in Natural Science −3, BSc in Information Technology −2 and BA in Social Science −1) consisting of four females and two males representing age range of 25–35 years. All of them have smartphones and comfortable of using tablet computers.
The second and third phases were carried out simultaneously with regular meetings with the developer and other stakeholders through testing of the mobile application. In these meetings, the application was evaluated by five groups of stakeholders: content experts to validate the content, educational technologists to check the alignment of technical and pedagogical features, novice users to check the overall effectiveness of the application for learning purposes, developer to develop the application, modify it with the feedback and check the ease of usage and researchers to identify the impact of this innovation.
Novice users were briefed about the purpose and asked them to go through the mobile application. The lead researcher observed and made notes using observational sheets while users were explored the mobile application. At the end of the product evaluation, researcher asked questions using a structured interview schedule about their perceptions of this mobile application, their likes, dislikes, challenges and suggestions for improvement. These viewpoints were categorized using content analysis to identify the major themes.
The entire development of the application was through eight iterations where feedback from different stakeholders at different stages was integrated to the OUSL MLearn system. First iteration was to develop the initial prototype for a session. Then three iterations were focused to develop the framework to design the entire system architecture for the entire university, considering faculty, departments and program requirements. Last four iterations were testing the developed prototype with additional requirements, adding pedagogical features along with the interactive features to the course structure and testing with novice users and content developers.
At the end of the formative evaluation, conclusions were drawn based on the findings. All the stakeholders perceived benefits of the mobile application as an effective tool for learning. Many challenges were expressed by different stakeholders and will be discussed in the results and discussion section.
This is very important phase in design-based research. Unless documentation and reflection, the generation of design principles and guidelines could not be constructed and the purpose of using design-based research is not fully achieved. Ma and Harmon (2009) recommended to provide two sets of principles based on the research study. One set of principles for the practitioners on the research findings specifically related to the instructional innovation/solution/product to improve their practices. The other set of principles for researchers who are interested in conducting design-based research on how to conduct design-based research based on the reflections on the research methodology.
In this current study, reflections were part of the whole process and not only restricted to the documentation phase. Researchers reflected the research processes in all phases from phases 2 to 4 and went back and forth while documenting the process in order to generate principles. At the end of each development phase, the results were re-examined, reflected upon and used for further enrichment, producing a continuous cycle of design-reflection-design. So, formative evaluation was integrated in the testing phase of the design-based research and the results were used to improve the system to make the instruction more effective and efficient. In this study, phases 2, 3 and 4 were all connected and could not be separated as distinct phases.
The current interface of mobile application and its functionalities are the result of revisions based on the suggestions/reflections during the formative evaluation of all three phases.
Having gone through the reflections, the researchers felt the design-based research is very appropriate in designing and developing technology based innovations as user testing is part of the development process. Since both researchers and practitioners were involved from the beginning, their contributions were very useful in conceptualizing the solution within a theoretical framework. The following design principles were derived from the findings of the research study for mobile application.
Research team should have open discussions with all the stakeholders including the developer so that diverse strategies/solutions will emerge as a result. Team can discuss these strategies and identify best solutions in order to reduce the development time of the innovation.
Research team should consider the existing research and practices in the local context in order to develop cultural sensitive solutions as learning is contextual and situated.
Some of the research studies conducted in western world may not directly applicable to eastern cultures. This study was influenced by the research findings of earlier studies conducted at the OUSL with three groups of culturally diverse groups of learners (Sri Lankans, Pakistanis and Mauritians), where they interact via learning management system for seven weeks (Jayatilleke and Gunawardena, 2016; Jayatilleke, Kulasekara, Kumarasinha and Gunawardena, 2017).
Instructions should be integrated in the mobile application as animated learning objects considering the user needs; especially, if the application is designed for open and distance learners.
When developing mobile solutions, alternative technological strategies and adaptive technologies should be designed in order to accommodate diverse learners.
Adaptive technologies should also be integrated in the mobile application to accommodate differently abled learners to empower them while making them more inclusive in the mainstream education.
Institutional leadership for direction, guidance and providing mechanism for establishing support structures are crucial in order for the sustenance and adoption of innovative mobile solution. Otherwise, diffusion of innovations will be observed only at the individual level and gradually die down.
This study adapted the model proposed by Ma and Harmon (2009) and was also influenced by the research of Keskin and Kuzu (2015). The detailed development and research procedure in the Ma and Harmon’s (2009) model was very useful in designing the procedure to conduct design-based research. However, researchers of the current study had to modify the order of certain guidelines to suit the context and the user needs. Even Ma and Harmon (2009, p. 90) stated that researchers may examine their own context to make appropriate modifications to their model. Hence, following guiding principles are proposed for the researchers who are interested in conducting design-based research based on the reflections of this research and development team.
Identifying the purpose and research questions for a development iteration are very crucial in the design-based research as they provide the focus for the study. Thus, they should be included in the first phase of the research study – analysis of a practical problem (refer Section 4.1.3).
Identifying the importance of research at the beginning of the development of innovation of project upfront and decision should be taken to integrate research while developing solutions at the beginning prior to the development of the solution and give fullest attention to the research methodology along with the development phases of the solution.
Educational technologist should be included as a researcher in the design-based research team to provide guidance, direction and to facilitate theory-driven research process and thereby enabling theory-building outcomes of the innovation in an effective manner.
The views expressed by the novel users indicated that the developed mobile application was generally efficient, simple to learn, easy to navigate, appealing and engaging. It was also pedagogically constructive as the content and the tools used in the application were useful from the perspective of both the content experts and the educational technologists. Thus, accomplishing the primary goal of this research study by providing effective instruction through mobile learning. It was also found that the developed mobile learning system was appropriate to the overall purpose of the university, could be served as a mobile learning system for the entire university and also could be used as an academic support system for the OUSL from the perspective of the developer.
Having gone through the reflection process and analyzing the qualitative data obtained by all the stakeholders using various tools, the challenges in implementing the MLearn for the entire university were identified using content analysis of the data. The categorized themes are illustrated in Table IV.
Having gone through this process, it was felt that the design-based research build on the principles of stakeholder centredness was effective in developing mobile learning application. This was due to the fact that the researchers and the practitioners were actively involved throughout the whole process and supported each other to produce an effective mobile application. The framework used in this study embeded the evaluation and testing of the solution phase (Phase 3) within the development of the solution phase (Phase 2) as these two phases are interconnected and run concurrently. Owing to the iterative cycles of the design-based research enabled the development of an effective mobile solution through several refinements based on existing research and practices.
Cowling and Birt (2018) also showed how the process of incremental reflection and refinement of the design-based research enabled the development of a mixed reality simulation to improve skills for students studying paramedic science at a distance.
The findings of the evaluation of the mobile application showed challenges with respect to development time, high production costs, technical and organizational issues, workload of academics and necessity of providing technical support both to remote students and faculty. Therefore, establishing adequate support structures for both teachers and students are essential for the sustenance of these innovative practices. This finding is in line with Montreux et al.’s (2015, p. 10) study where they also emphasized the importance of technical and pedagogical support to “stimulate teacher and student recognition of tablet devices’ potential in education.”
This application will be further evaluated through summative evaluation with actual students to assess the effectiveness of the mobile learning system to complete the design of the system fully.
The design and development of any instructional material depend on the target audience, the subject content and the organizational culture of the institution (context). As such, the findings of this study may not have a universal value; however, these findings may throw light on some of the pedagogical, technological, social interaction and contextual attributes including cultural dimensions that have to considered when designing mobile applications. It also provides guiding principles for designing both mobile solutions and on how to conduct design-based research in mobile learning.
Design-based research model of this study