Research Methods in Learning Design and Technology: A Historical Perspective of the Last 40 Years
Enilda Romero-Hall [eromerohall@ut.edu]
This chapter aims to provide a historical perspective of the evolution of educational research and, more specifically, research methods in the instructional design & technology field. It starts with discussions in educational research that were often documented in the 1980s related to the persistence of positivistic views and post-positivistic thoughts. The chapter then continues to further explore the educational research literature into the 1990s. By then, it was clear that educational research was evolving, researchers were full of desire to immerse and investigate in the real world of teaching and learning. To understand learners and their experiences (cognitive, physical, and emotional), researchers needed to evolve from traditional ideologies of research to a more intricate research method that carefully considers “context.” For the learning design and technology researchers, in the early 2000s, this became particularly critical as complete, complex, and interactive learning environments became tokens of interest by researchers in the educational technology field. The chapters finish with a perspective of the current state of research in the learning design and technology field.
This chapter aims to provide a historical perspective of the evolution of educational research and, more specifically, research methods in the instructional design & technology field. It starts with discussions in educational research that were often documented in the 1980s related to the persistence of positivistic views and post-positivistic thoughts. The chapter then continues to further explore the educational research literature into the 1990s. By then, it was clear that educational research was evolving, researchers were full of desire to immerse and investigate in the real world of teaching and learning. To understand learners and their experiences (cognitive, physical, and emotional), researchers needed to evolve from traditional ideologies of research to a more intricate research method that carefully considers “context.” For the learning design and technology researchers, in the early 2000s, this became particularly critical as complete, complex, and interactive learning environments became tokens of interest by researchers in the educational technology field. The chapters finish with a perspective of the current state of research in the learning design and technology field.
Interpretive and Postmodern Phenomenological Research Approaches: Opportunities for New Lines of
Inquiry in the Field of Learning Design and Technology
Inquiry in the Field of Learning Design and Technology
Keri Duncan Valentine [kevalentine@mail.wvu.edu]
Phenomenology has a long history as both a philosophical and methodological approach, with variations of each emerging over the past century. This chapter seeks to support researchers in the field of Learning Design and Technology (LDT) to consider critical aspects and affordances of variant forms of phenomenological inquiry. First, the philosophical foundations of phenomenology are traced to demonstrate the plurality of methodological approaches. Next, the chapter situates and advocates for interpretive and postmodern phenomenological approaches by considering foci that these forms of phenomenology might open up within LDT. Key considerations are explicated regarding data sources, analysis, and other research practices within these approaches.
Phenomenology has a long history as both a philosophical and methodological approach, with variations of each emerging over the past century. This chapter seeks to support researchers in the field of Learning Design and Technology (LDT) to consider critical aspects and affordances of variant forms of phenomenological inquiry. First, the philosophical foundations of phenomenology are traced to demonstrate the plurality of methodological approaches. Next, the chapter situates and advocates for interpretive and postmodern phenomenological approaches by considering foci that these forms of phenomenology might open up within LDT. Key considerations are explicated regarding data sources, analysis, and other research practices within these approaches.
Mobile Eye-tracking for Research in Diverse Educational Settings
Yong Ju Jung [yongju@psu.edu]
Heather Toomey Zimmerman
Koraly Pérez-Edgar
Mobile eye-tracking is a technology that captures visual information, such as gaze, eye-movements, and pupil dilations, when learners are mobile. Traditional eye-tracking helps researchers to obtain precise, moment-by-moment information about learners’ engagement, interactions, and learning processes. Still, it has some weaknesses due to its structural and stationary nature. Mobile eye-tracking can complement such weaknesses by allowing researchers to collect eye-tracking data when learners move around and interact with multiple targets. This chapter demonstrates how mobile eye-tracking can add more authenticity and nuanced information into Learning Design and Technology research, and then introduces potential research themes that can use mobile eye-tracking. This chapter also overviews the overall processes of applying mobile eye-tracking in a research study and provides an example analysis.
Heather Toomey Zimmerman
Koraly Pérez-Edgar
Mobile eye-tracking is a technology that captures visual information, such as gaze, eye-movements, and pupil dilations, when learners are mobile. Traditional eye-tracking helps researchers to obtain precise, moment-by-moment information about learners’ engagement, interactions, and learning processes. Still, it has some weaknesses due to its structural and stationary nature. Mobile eye-tracking can complement such weaknesses by allowing researchers to collect eye-tracking data when learners move around and interact with multiple targets. This chapter demonstrates how mobile eye-tracking can add more authenticity and nuanced information into Learning Design and Technology research, and then introduces potential research themes that can use mobile eye-tracking. This chapter also overviews the overall processes of applying mobile eye-tracking in a research study and provides an example analysis.
Treating Research Studies as our Primary Subject: Using Meta-Analysis and Meta-Synthesis to Conduct Systematic Reviews
Heather Leary [heather.leary@byu.edu]
Andrew Walker
For a variety of reasons, education research can be challenging to summarize. Varying contexts, designs, levels of quality, measurement challenges, the definition of underlying constructs and treatments as well as the complexity of research subjects themselves can result in variability. Education research is voluminous and draws on multiple methods, including quantitative as well as qualitative approaches to answer key research questions. With increased numbers of empirical research in Instructional Design and Technology (IDT), using various synthesis methods can provide a means to more deeply understand trends and patterns in research findings across multiple studies. The purpose of this article is to illustrate structured review or meta-synthesis procedures for qualitative research as well as novel meta-analysis procedures for the kinds of multiple treatment designs common to IDT settings. Sample analyses are used to discuss critical methodological ideas as a way to introduce researchers to these techniques.
Andrew Walker
For a variety of reasons, education research can be challenging to summarize. Varying contexts, designs, levels of quality, measurement challenges, the definition of underlying constructs and treatments as well as the complexity of research subjects themselves can result in variability. Education research is voluminous and draws on multiple methods, including quantitative as well as qualitative approaches to answer key research questions. With increased numbers of empirical research in Instructional Design and Technology (IDT), using various synthesis methods can provide a means to more deeply understand trends and patterns in research findings across multiple studies. The purpose of this article is to illustrate structured review or meta-synthesis procedures for qualitative research as well as novel meta-analysis procedures for the kinds of multiple treatment designs common to IDT settings. Sample analyses are used to discuss critical methodological ideas as a way to introduce researchers to these techniques.
Considerations for Using Social Media Data in Learning Design and Technology Research
| chapter_5_research_methods_in_learning_design_and_technology_pre-print.pdf |
Spencer P. Greenhalgh [spencer.greenhalgh@uky.edu]
Matthew J. Koehler
Joshua M. Rosenberg
K. Bret Staudt Willet
Social media platforms have firmly established themselves as phenomena of interest for Learning Design and Technology (LDT) researchers, and the data accessible from these platforms provide new methodological possibilities. Given the number, diversity, and constant evolution of both social media platforms and research tools, it would be inappropriate to suggest that there is a single correct wayto carry out LDT research with social media data. In contrast, this chapter introduces six broad, interconnected steps to social media research: Conducting Ethical Research; Framing the Research; Organizing the Research Process; Collecting Data; Analyzing Data; and Writing, Sharing, and Publicizing Research. Each step is associated with several considerations, careful attention to which will guide researchers toward a particular correct wayto accomplish their specific research objectives.
Matthew J. Koehler
Joshua M. Rosenberg
K. Bret Staudt Willet
Social media platforms have firmly established themselves as phenomena of interest for Learning Design and Technology (LDT) researchers, and the data accessible from these platforms provide new methodological possibilities. Given the number, diversity, and constant evolution of both social media platforms and research tools, it would be inappropriate to suggest that there is a single correct wayto carry out LDT research with social media data. In contrast, this chapter introduces six broad, interconnected steps to social media research: Conducting Ethical Research; Framing the Research; Organizing the Research Process; Collecting Data; Analyzing Data; and Writing, Sharing, and Publicizing Research. Each step is associated with several considerations, careful attention to which will guide researchers toward a particular correct wayto accomplish their specific research objectives.
Becoming Action Researchers: Crafting the Curriculum and Learning Experiences for Scholarly
Practitioners in Educational Technology
Practitioners in Educational Technology
Ismahan Arslan-Ari [ARSLANAI@mailbox.sc.edu]
Fatih Ari
Michael M. Grant
Lucas Vasconcelos
Hengtao Tang
William S. Morris
The purpose of this chapter is to describe the process for preparing doctoral students as action researchers in an educational technology program at the University of South Carolina. First, an overview of action research and its key characteristics are discussed. Then, we describe doctoral students' experiences with action research and educational theory, and we report some descriptive, evaluative findings, gauging the success of these experiences. In addition, this chapter includes examples of dissertation topics to demonstrate the use of action research in the educational technology field. Finally, we offer some conclusions and reflections with the modifications we have made to serve our students better.
Fatih Ari
Michael M. Grant
Lucas Vasconcelos
Hengtao Tang
William S. Morris
The purpose of this chapter is to describe the process for preparing doctoral students as action researchers in an educational technology program at the University of South Carolina. First, an overview of action research and its key characteristics are discussed. Then, we describe doctoral students' experiences with action research and educational theory, and we report some descriptive, evaluative findings, gauging the success of these experiences. In addition, this chapter includes examples of dissertation topics to demonstrate the use of action research in the educational technology field. Finally, we offer some conclusions and reflections with the modifications we have made to serve our students better.
Making Data Science Count In and For Education
Joshua M. Rosenberg [jmrosenberg@utk.edu]
Michael Lawson
Daniel J. Anderson
Ryan Seth Jones
Teomara Rutherford
New data sources and analytic techniques have enabled educational researchers to ask new questions and work to address enduring problems. Yet, there are challenges to those learning and applying these methods. In this chapter, we provide an overview of a nascent area of both scholarship and teaching, educational data science. We define educational data science as the combination of capabilities related to quantitative methods in educational research, computer science and programming capabilities, and teaching, learning, and educational systems. We demonstrate that there are two distinct—but complementary—perspectives on educational data science in terms of being both in education (as a research methodology) and for education (as a teaching and learning content). We describe both of these areas in light of foundational and recent research. Lastly, we highlight three future directions for educational data science, emphasizing the synergies between these two perspectives concerning designing tools that can be used by both learners and professionals, foregrounding representation, inclusivity, and access as first-order concerns for those involved in the growing community, and using data science methodologies to study teaching and learning about data science. We highlight the potential for the growth of educational data science within learning design and technology as situated with the broader data science domain and in education more broadly.
Michael Lawson
Daniel J. Anderson
Ryan Seth Jones
Teomara Rutherford
New data sources and analytic techniques have enabled educational researchers to ask new questions and work to address enduring problems. Yet, there are challenges to those learning and applying these methods. In this chapter, we provide an overview of a nascent area of both scholarship and teaching, educational data science. We define educational data science as the combination of capabilities related to quantitative methods in educational research, computer science and programming capabilities, and teaching, learning, and educational systems. We demonstrate that there are two distinct—but complementary—perspectives on educational data science in terms of being both in education (as a research methodology) and for education (as a teaching and learning content). We describe both of these areas in light of foundational and recent research. Lastly, we highlight three future directions for educational data science, emphasizing the synergies between these two perspectives concerning designing tools that can be used by both learners and professionals, foregrounding representation, inclusivity, and access as first-order concerns for those involved in the growing community, and using data science methodologies to study teaching and learning about data science. We highlight the potential for the growth of educational data science within learning design and technology as situated with the broader data science domain and in education more broadly.
Ethnographic Considerations Within Instructional Design Research Practices
Jill E. Stefania [Jill.stefaniak@uga.edu]
Many instructional design studies include elements of ethnography, and researchers are often interacting with their learning and research audience. Ethnography requires the researcher to immerse themselves in the environment and gather data through direct observations, interactions with participants, and recordings of everyday life. The purpose of this chapter is to provide instructional designers with an overview of the various types of ethnographic methodologies used in instructional design research and explain how ethnographic methodologies align with instructional design practices, particularly taking into account the influence that contextual factors may have on a design intervention or study.
Many instructional design studies include elements of ethnography, and researchers are often interacting with their learning and research audience. Ethnography requires the researcher to immerse themselves in the environment and gather data through direct observations, interactions with participants, and recordings of everyday life. The purpose of this chapter is to provide instructional designers with an overview of the various types of ethnographic methodologies used in instructional design research and explain how ethnographic methodologies align with instructional design practices, particularly taking into account the influence that contextual factors may have on a design intervention or study.
Complex, Multiple, Interdependent Layers (C-MIL):A Conceptual Model For Usability Studies in
3-Dimensional Virtual Worlds
3-Dimensional Virtual Worlds
Sarah Espinosa
Peter Leong [peter.leong@hawaii.edu]
Usability studies on simulations within 3-Dimensional Virtual Worlds (3D VW) have challenges and features that are unique compared to a traditional usability study on a website, mobile app, or even a 3D VW itself. The locus of control is not the 3D VW, but rather the content created within the 3D environment, i.e., the simulation. This chapter reviews the challenges and features that are unique to usability studies of 3D VWs. Drawing from previous literature and a study to evaluate the usability of the Crafter's Ear simulation within Minecraft (Espinosa, 2019), this chapter examines concepts and examples from usability studies done within 3D VWs to propose a conceptual model for usability studies designed for 3D VW simulations.
Peter Leong [peter.leong@hawaii.edu]
Usability studies on simulations within 3-Dimensional Virtual Worlds (3D VW) have challenges and features that are unique compared to a traditional usability study on a website, mobile app, or even a 3D VW itself. The locus of control is not the 3D VW, but rather the content created within the 3D environment, i.e., the simulation. This chapter reviews the challenges and features that are unique to usability studies of 3D VWs. Drawing from previous literature and a study to evaluate the usability of the Crafter's Ear simulation within Minecraft (Espinosa, 2019), this chapter examines concepts and examples from usability studies done within 3D VWs to propose a conceptual model for usability studies designed for 3D VW simulations.
Learning User Experience Design (LUX): Adding the “L” to UX research using biometric sensors
Quincy Conley [quincyconley@gmail.com]
As an emerging research methodology, biometrics is applied using a mix of biometric sensors including, eye-tracking, facial expression recognition, galvanic skin conductance, EEG, and more. These tools provide an extra layer of data for tracking user’s attention as well as their emotions during a learning experience. With proven success, advances in biometric technology give a line of sight into the interrelationships between affective components, all while making it more feasible to minimize biases issues. This chapter is about providing a comprehensive overview of biometric research methods. Furthermore, it will be discussed how learning design and technology (LDT) researchers and designers can use this new approach to help create more efficacious digital learning interventions. This chapter will also cover the connection between a user’s emotional state and the learning process and how biometrics can be used to unveil the hidden affective learning drivers such as attention, motivation, and cognition behaviors.
As an emerging research methodology, biometrics is applied using a mix of biometric sensors including, eye-tracking, facial expression recognition, galvanic skin conductance, EEG, and more. These tools provide an extra layer of data for tracking user’s attention as well as their emotions during a learning experience. With proven success, advances in biometric technology give a line of sight into the interrelationships between affective components, all while making it more feasible to minimize biases issues. This chapter is about providing a comprehensive overview of biometric research methods. Furthermore, it will be discussed how learning design and technology (LDT) researchers and designers can use this new approach to help create more efficacious digital learning interventions. This chapter will also cover the connection between a user’s emotional state and the learning process and how biometrics can be used to unveil the hidden affective learning drivers such as attention, motivation, and cognition behaviors.
Exploring the Evolution of Instructional Design and Technology Disciplinary Knowledge through
Citation Context Analysis
Citation Context Analysis
Wendy Ann Gentry [wendy.gentry@bakeru.edu]
Barbara Lockee
This chapter situates discourse and citation within the more extensive system of Instructional Design and Technology (IDT) disciplinary knowledge. It introduces a mixed-method approach called citation context analysis to explore content that is transferred from one document to another through citation. A better understanding of the application of concepts through citation will alert IDT researchers to concepts and arguments that may have become overlooked over time and the mix of those that have come in and out of favor. This provides an opportunity for researchers, experienced and novice alike, to reflect on their practice and consider how their efforts have influenced the direction of the discipline and what changes they can make going forward to benefit IDT research and practice. To achieve these objectives, this chapter provides an introduction to CCA, an overview of related research, and a seven-step framework to help researchers apply the method in their work.
Barbara Lockee
This chapter situates discourse and citation within the more extensive system of Instructional Design and Technology (IDT) disciplinary knowledge. It introduces a mixed-method approach called citation context analysis to explore content that is transferred from one document to another through citation. A better understanding of the application of concepts through citation will alert IDT researchers to concepts and arguments that may have become overlooked over time and the mix of those that have come in and out of favor. This provides an opportunity for researchers, experienced and novice alike, to reflect on their practice and consider how their efforts have influenced the direction of the discipline and what changes they can make going forward to benefit IDT research and practice. To achieve these objectives, this chapter provides an introduction to CCA, an overview of related research, and a seven-step framework to help researchers apply the method in their work.
Learning Environments Visual Mapping
Sonia Tiwari [sut224@psu.edu]
Yu-Chen Chiu
This chapter demonstrates a method of analyzing learning environments for students in the K-12 space, which foregrounds the visual representations of students and the multi-dimensions of learning spaces. The technique relies on collecting Visual Maps from children (their drawings of learning activities at home, school, and other places), and questionnaires from their teachers (information about the class as a group) and parents (information about the individual child) - to collectively inform research on children's learning environments across settings in everyday life. Example data presented to demonstrate how this method could be used to study children's learning environments across spaces. The data is analyzed by mapping keywords from all three sources on an Analysis Map - to generate an "at-glance" profile of each student's learning environment. Findings from all participants then act as a basis to design guidelines for creating the most effective learning environments.
Yu-Chen Chiu
This chapter demonstrates a method of analyzing learning environments for students in the K-12 space, which foregrounds the visual representations of students and the multi-dimensions of learning spaces. The technique relies on collecting Visual Maps from children (their drawings of learning activities at home, school, and other places), and questionnaires from their teachers (information about the class as a group) and parents (information about the individual child) - to collectively inform research on children's learning environments across settings in everyday life. Example data presented to demonstrate how this method could be used to study children's learning environments across spaces. The data is analyzed by mapping keywords from all three sources on an Analysis Map - to generate an "at-glance" profile of each student's learning environment. Findings from all participants then act as a basis to design guidelines for creating the most effective learning environments.
Learning analytics: The Emerging Research Method for Enhancing Teaching and Learning
Tiantian Jin [jin.tiantian121@gmail.com]
Learning Analytics, as an emerging discipline, has exhibited extraordinary impact and drawn attention within the field of Learning Design and Technology (LDT). This chapter discusses what learning analytics is, what methodologies and techniques involve in learning analytics, how learning analytics can be applied in the field of LDT, and what benefits and challenges may be met during the application process. In addition, after reading this chapter, people can better understand the connection, similarities, and distinctions between learning analytics and educational data mining.
Learning Analytics, as an emerging discipline, has exhibited extraordinary impact and drawn attention within the field of Learning Design and Technology (LDT). This chapter discusses what learning analytics is, what methodologies and techniques involve in learning analytics, how learning analytics can be applied in the field of LDT, and what benefits and challenges may be met during the application process. In addition, after reading this chapter, people can better understand the connection, similarities, and distinctions between learning analytics and educational data mining.
Futurama: Learning, Design and Technology Research Methods
Enilda Romero-Hall [eromerohall@ut.edu]
Ana Paula Correia
Robert Maribe (Rob) Branch
Yasemin Demiraslan Cevik
Camille Dickson-Deane
Bodong Chen
JuhongChristie Liu
Hengtao Tang
Lucas Vasconcelos
Nicola Pallitt
Briju Thankachan
This chapter serves as an examination and projection of developments in learning design and technology research methods in the next decade and beyond, as provided by scholars and practitioners in the field. These members of the community were asked to focus on the probable nature of learning design research methods in the near future to outline their perspectives on where learning design and technology research is going or ought to be going. Why do we engage in this prediction of the future? Perhaps, we share these perspectives because it allows us to feel optimistic about what is to come, it will enable reflection on what has happened and how it can improve, and it is an outlet for progressive ideas that have gone unexplored. Historical comments, current strengths, and weaknesses, trends, or overlapping statements were not edited or removed.
Ana Paula Correia
Robert Maribe (Rob) Branch
Yasemin Demiraslan Cevik
Camille Dickson-Deane
Bodong Chen
JuhongChristie Liu
Hengtao Tang
Lucas Vasconcelos
Nicola Pallitt
Briju Thankachan
This chapter serves as an examination and projection of developments in learning design and technology research methods in the next decade and beyond, as provided by scholars and practitioners in the field. These members of the community were asked to focus on the probable nature of learning design research methods in the near future to outline their perspectives on where learning design and technology research is going or ought to be going. Why do we engage in this prediction of the future? Perhaps, we share these perspectives because it allows us to feel optimistic about what is to come, it will enable reflection on what has happened and how it can improve, and it is an outlet for progressive ideas that have gone unexplored. Historical comments, current strengths, and weaknesses, trends, or overlapping statements were not edited or removed.