CHAT AND EXPLORE - Bvekennis

CHAT AND EXPLORE

The role of support and motivation in collaborative scientific discovery learning

Cover design: Rieme Gleijm. Printed by PrintPartners Ipskamp B.V., Amsterdam. ISBN-13 978-90-7808-702-1 ISBN-10 90-7808-702-1 © Nadira Saab, Amsterdam, 2005 Alle rechten voorbehouden. All rights reserved.

CHAT AND EXPLORE

The role of support and motivation in collaborative scientific discovery learning

ACADEMISCH PROEFSCHRIFT

ter verkrijging van de graad van doctor aan de Universiteit van Amsterdam op gezag van de Rector Magnificus

Prof. mr. P.F. van der Heijden ten overstaan van een door het college voor promoties ingestelde

commissie, in het openbaar te verdedigen in de Aula der Universiteit

op vrijdag 14 oktober 2005, te 14.00 uur

door

Nadira Saab

Geboren te Leiden

Promotiecommissie Promotor: Prof. dr. B. H. A. M. van Hout-Wolters Co-promotor: Dr. W. R. van Joolingen Overige leden: Prof. dr. J. J. Beishuizen (Vrije Universiteit Amsterdam) Prof. dr. A. J. M. de Jong (Universiteit Twente) Dr. M. Elshout-Mohr (Universiteit van Amsterdam) Dr. G. Erkens (Universiteit Utrecht) Prof. dr. S. P. Lajoie (McGill University, Montreal) Prof. dr. B. J. Wielinga (Universiteit van Amsterdam) Faculteit: Maatschappij- en Gedragswetenschappen

Het promotie-onderzoek is voorbereid aan het Instituut voor de Lerarenopleiding van de Universiteit van Amsterdam

TABLE OF CONTENTS

PREFACE CHAPTER 1 Introduction

1

CHAPTER 2 Communication in collaborative discovery learning

7

CHAPTER 3 Supporting communication in a collaborative scientific discovery learning environment: The effect of instruction

27

CHAPTER 4 Supporting collaborative scientific discovery learning, a tool and its difficulties

45

CHAPTER 5 Motivation and collaborative scientific discovery learning

61

CHAPTER 6 Discussion

83

REFERENCES 93 AUTHOR INDEX 101 APPENDIX A 105 APPENDIX B 106 SUMMARY 109 SAMENVATTING 119 CURRICULUM VITAE 129

PREFACE

Collaboration and discovery learning match me in different ways. They have my scientific interest, but they also fit into my life. Chatting and exploring are two things that describe my way of living. From early childhood, talking is one of my favorite activities. Furthermore, I like to satisfy my curiosity by exploring every-thing I meet on my path. All happened in what one might call a collaborative envi-ronment, my parental home and my world of education. The fact that I followed my primary and secondary education at a Montessori school supported me in this col-laborative discovery way of life. The way of learning at the Montessori school ex-cited my interest for education. After finishing my secondary schooling, studying educational psychology was a logical step. Even better was doing research in this direction… A lot of people supported me and helped me in different ways with the process of writing this dissertation and finishing my PhD study. First of all, I would like to thank my promotor and co-promotor for guiding me and giving me all their exper-tise they thought necessary. Bernadette gave me support all the way. Also, she was always very excited and sympathized with me in every important moment. Wouter, as an initiator of this PhD project, supported me with the planning of the research, giving it a firm basis. He also gave feedback and supported me in the writing proc-ess. I would like to thank Gert for all his input and the time he spend helping me and giving me feedback, especially regarding the structure, the lay-out, and the printing procedures. I would like to thank Hein for his nice company and his patience to give answer to all my questions. I would like to thank Marielle for her support and pleas-ant talks. I would like to thank Martine, for offering me support every time she thought I needed it (which I did). I would like to thank Joost who helped me with loads of statistical analyses. Marie, I am going to miss you when you’ll leave Hol-land. Who will help me with my English now? And of course thanks for the nice train trips. I would like to thank all the PhD students who made the process of doing research very pleasant. Not only was it nice to talk about all aspects of work, but the lunches, the dinners and the drinks were making work enjoyable! Simone, Monique, Stan, Annemieke, Marleen, Anousjka, Anne, Lap, Irma, Hélène (as a postdoc) and especially the guys in the corner room Jaap en Patrick, thank you! I would like to thank the members of the OZO for their thoughtful suggestions on my research. I would like to thank all the members of the HogCog. They helped me reflect on my articles in a very agreeable way. I would like to thank Pauline and Petra for all the nice talks, drinks and dancing. During work they reminded me of the nice life next to work. I would like to thank Lisa and Noor for giving me feedback on my analysis

scheme and for coding all those protocols. I would like to thank Inez, Jeroen, Marcelle, Glenn, Bert, Bas, Rob, Nico and Anita for making the ILO a nice place and working there easy going. I would like to thank Elwin and Saskia for being such fine roommates. I would like to thank the members of the VPO board, Hanneke, Bregje, Jan-Willem, PJ, Rosanne, Jaap, and Liesbeth. We were always enthusiastic about thinking about new ideas and projects. Having meetings was fun! I would like to thank Susanne for her hospitality and for having me at McGill University, that was a great experience! I would like to thank Anca, Minou, Noura, Ronald, and Jan-Willem for their inter-est, support, and feedback that helped me evaluating this dissertation. I would like to thank Rieme for all her efforts and for making such a nice cover. I would like to thank Manja, who introduced me to education in a very sweet way. I am very grate-ful to all my friends and family who were listening to my stories about this research, supporting me whenever they could, and who made me realize that there is much more besides research! I would like to thank my parents, Amir and Wil, very much. They helped me in many different ways, from making suggestions for the cover to reading this dissertation thoroughly and giving me very elaborated feedback. Every moment of the day I could ask them for support or help. Lus dilanti! Most of all I would like to thank Luciano for his love and support. I am so glad that he was there and stood beside me in every pleasant or hard moment. This makes life, and, since this is a preface for my dissertation, doing research super! Thanks everybody! Nadira Saab Den Haag, 2005.

CHAPTER 1

INTRODUCTION

It is no longer possible to imagine a Western society without technology. Children grow up with different kinds of media, like television, computers and cell phones, and learn from childhood how to interact with these media. In schools, computers are also being used more and more. Principals of Dutch schools have predicted at the end of 2002 that in 2006 66% of all secondary school teachers in the Netherlands will make use of ICT applications in their lessons (Stichting Ict op School, 2003). Thus, in a society where almost every aspect of life is controlled by technology, it is important for young learners to learn and to be taught how to work effectively with technological devices.

Just as society has changed with respect to the use of media, educators and theo-reticians have changed their view of learning. In the last few decades, a shift has taken place from a traditional, teacher-centered view of learning in which informa-tion is transferred to the learner to a more constructivist view that is more learner-centered. In the constructivist view of learning (Duffy & Jonassen, 1991; Jonassen, 1991, 2000), knowledge is constructed by the learner in an active way. Learners restructure their prior knowledge in order to construct new knowledge. An approach to learning which is compatible with constructivism is scientific discovery learning. In this type of learning, learners work in a learning environment where they generate hypotheses, conduct experiments by manipulating variables, draw conclusions, and integrate experimental outcomes as new, collectively-gathered or constructed knowledge.

This dissertation is concerned with the topic of collaborative scientific discovery learning in a computer environment. Discovery learning and collaborative learning have separately been studied thoroughly (e.g., De Jong & Van Joolingen, 1998; Van der Linden, Erkens, Schmidt, & Renshaw, 2000). In the following sections, several aspects with respect to collaborative discovery learning in a computer-supported environment will be discussed.

2 CHAPTER 1

1 COLLABORATIVE SCIENTIFIC DISCOVERY LEARNING

In scientific discovery learning learners construct knowledge by means of discovery and experimentation. Many researchers have focused on the process of discovery learning (e.g., De Jong & Van Joolingen, 1998; Klahr & Dunbar, 1988; Kuhn, Black, Keselman, & Kaplan, 2000; Simon & Lea, 1974; Van Joolingen & De Jong, 1997; White, Shimoda, & Frederiksen, 1999). All these studies identify basically the same kind of processes, which resemble phases in the scientific process of discovery (Burns, Okey, & Wise, 1985; Klahr & Dunbar, 1988; Löhner, Van Joolingen, Savelsbergh, & Van Hout-Wolters, 2005; Van Joolingen & De Jong, 1997). These discovery learning processes include orientation, generating ideas and hypotheses, testing these hypotheses, and drawing conclusions. These processes are labeled by De Jong and Njoo (1992) as transformative discovery learning processes, and in-volve the generation of new information and knowledge. In addition to transforma-tive discovery learning processes, De Jong and Njoo (1992) distinguish regulative discovery learning processes, such as planning and monitoring, that help the learner to control the execution of the transformative processes.

Previous research has revealed aspects that learners find difficult about the dis-covery learning process. Learners can have difficulty stating hypotheses (De Jong & Van Joolingen, 1998; Njoo & De Jong, 1993), designing suitable experiments to test these hypotheses (Gorman, 1986; Gorman, Stafford, & Gorman, 1987; Mynatt, Do-herty, & Tweney, 1978; Reimann, 1991; Schauble, Glaser, Raghavan, & Reiner, 1991; Wason, 1960), executing these experiments in an effective way (Shute & Glaser, 1990; Quinn & Alessi, 1994), interpreting the collected data, and drawing conclusions (Chinn & Brewer, 1993; Klahr & Dunbar, 1988; O’Brien, Costa, & Overton, 1986).

As many aspects of the discovery learning process are found to be difficult, sup-port seems to be needed. In this dissertation the focus lies on providing support by putting the discovery processes in the context of collaboration between learners.

Several authors have argued that collaborative learning can improve the quality of the learning process and learning performance (e.g., Cohen, 1994; Springer, Stanne, & Donovan, 1999; Van der Linden et al., 2000). In collaborative learning, learners work together by communicating with each other. When learners are col-laborating, they are stimulated to explain (Dekker & Elshout-Mohr, 1998), discuss, negotiate and, ultimately, to create new constructed knowledge (Baker, 1999). By verbalizing and proposing new ideas, asking questions (Chi, Bassok, Lewis, Re-imann, & Glaser, 1989; King, 1997), or asking for and giving explanations (Ploetzner, Dillenbourg, Preier, & Traum, 1999; She, 1999; Wegerif, 1996; Weiss & Dillenbourg, 1999) in an elaborated manner, learners exchange ideas and, in this way, externalize their thoughts (Marshall, 1995; Roelofs, Van der Linden, & Erkens, 1999; Webb, & Farivar, 1994). By externalizing their thoughts, they may become conscious of the quality of their ideas. They reflect on their own and their partner’s ideas and can see both the strengths and flaws in their own reasoning and that of their peers. Such reflection can lead to conceptual change or to new constructed knowledge (Van Boxtel, 2000; Van der Linden et al., 2000). In addition, the airing of conflicting views in discussions and the confrontation this may involve can pro-

INTRODUCTION 3

voke a socio-cognitive conflict that also can lead to conceptual change when learn-ers try to solve their conflicts by reflecting on the proposed ideas (Doise, Mugny, & Pérez, 1998; Limón, 2001; Nastasi & Clements, 1992).

Specifically for discovery learning, research has indicated that collaboration can make the discovery learning process more accessible and more effective (Okada & Simon, 1997; Salomon & Globerson, 1989; Whitelock, Scanlon, Taylor, & O’Shea, 1995). However, it is not clear which communicative activities in collaborative learning contribute to the discovery learning process. This dissertation will focus on the relationship between communicative activities and discovery activities.

2 COMPUTER-SUPPORTED COLLABORATIVE DISCOVERY LEARNING

For both discovery learning and collaborative learning computerized learning envi-ronments are used frequently. Among the learning environments that are suitable for discovery learning are computerized simulation environments in which learners can collect data and carry out experiments (De Jong & Van Joolingen, 1998). Simulation environments replicate real situations in which learners can actively acquire infor-mation. Learners can carry out experiments that would have been impossible to con-duct otherwise. For example, situations that are dangerous in real life can be mod-eled in a simulation environment. Situations in which certain variables are fixed so that other variables can be investigated need computerized simulation environments. Examples of simulations are Smithtown (Shute & Glaser, 1990), which focuses on microeconomics, and Stat lab (Veenman & Elshout, 1995; Veenman, Elshout, & Meijer, 1997), in which the domain of statistics can be explored. Examples of au-thoring tools where different simulation applications can be made and used are Sim-Quest (Van Joolingen & De Jong, 2003), Co-Lab (Van Joolingen, De Jong, Lazon-der, Savelsbergh, & Manlove, 2005), and Agentsheets (Repenning, Ioannidou, & Ambach, 1998).

Computer-supported collaborative learning (CSCL) involves collaboration sup-ported by a technology-enriched environment. Shared tools are provided to support the collaborative learning process. In a CSCL environment, learners communicate through means of computer-mediated communication (CMC). There are asyn-chronic communication tools, such as e-mail and discussion forums, and synchronic tools, such as chat. Chat is a fast way to communicate. By typing in text messages, learners can react quickly to each other’s generally short messages. Communication through a chat channel has some obvious limitations, for instance, the fact that ges-tures and other non-verbal signals are not visible. However, these limitations can sometimes also be advantageous. The absence of visual cues makes it necessary to formulate and explicate messages more precisely (Henri, 1992). Moreover, social differences between learners related to factors such as gender and speech (e.g. re-gional or foreign accents) are less visible (Dubrovsky, Kiesler, & Sethna, 1991), which may allow learners to focus more on the task (Walther, 1992). Finally, chat is logged, which implies that learners can look back at their conversation that repre-sents the history of the learning process (Baker & Lund, 1997).

4 CHAPTER 1

Although a beneficial effect of CSCL as been found in many situations, more spe-cific research is needed to see in which way collaboration by means of CSCL tools such as chat and shared environments can contribute to the process of discovery learning. A central focus in this thesis will be the relation between the communica-tion between learners and the processes of discovery.

3 SUPPORTING COLLABORATIVE DISCOVERY LEARNING

When collaborative learning is not guided or scaffolded, several interactional prob-lems can hinder the quality of the collaborative learning process. For instance a problem that can occur is that learners work individually (Ettekoven, 1997) instead of checking with others whether everything is understood (Baker, Hansen, Joiner, & Traum, 1999; Erkens, Andriessen, & Peters, 2003). The flipside of this problem is the ‘free rider effect’, where one participant does all the work, while the other learn-ers do nothing (Salomon & Globerson, 1989; Wasson, 1998). Another problem that can occur is that learners ignore information given by other learners, or reject pro-posed ideas without discussion (Chan, 2001; Barron, 2003). This means that guid-ance of the collaborative learning process is needed. Providing support for cognitive processes has proven to be effective in both face-to-face learning situations (e.g., Elshout-Mohr, 1992; Hoek, 1998) and computerized environments (e.g., Van Jool-ingen & De Jong, 2003; Veenman, Elshout, & Busato, 1994). Therefore, it seems likely that providing support for the collaborative discovery learning process will improve the learning process.

In this dissertation, two ways of supporting learners are investigated: instruction in effective communication, and provision of cognitive tools to support collaborative discovery learning.

Positive results of providing instruction to learners to promote effective commu-nication have been found (e.g., Hoek, 1998; Mercer, 1996; Rojas-Drummond & Mercer, 2003; Swing & Peterson, 1982; Wegerif, Mercer, & Dawes, 1999), resulting in several important characteristics of effective communication. These results can form the basis of instruction in effective communication and collaboration. Learners should exchange relevant ideas and take every idea into consideration (Barron, 2003; King, 1997; Wegerif et al., 1999). They should discuss different alternatives (Veerman, Andriessen, & Kanselaar, 2000), and search for common ground or joint agreement (Baker et al., 1999; Erkens, et.al, 2003) before making decisions by, for example, asking verification questions (Baker et al., 1999; Bandura, 2001; Erkens et al., 2003; Mercer, 1996; Van Boxtel, Van der Linden, & Kanselaar, 2000; Wegerif & Mercer, 1996; Wegerif et al., 1999). Other essential communicative activities are asking elaborated questions and giving elaborated help (King, 1997; Ploetzner et al., 1999; She, 1999; Webb & Farivar, 1994; Wegerif, 1996; Weiss & Dillenbourg, 1999); keeping on asking clear and elaborated questions after incomprehension (Chi et al., 1989; Veerman et al., 2000; Wegerif & Mercer, 1996), or in situations in which it is necessary to identify the differences between various ideas that have been generated. Accepting and confirming each other’s propositions to show that they are listening to each other (Barron, 2003) is also important for learners. Furthermore,

INTRODUCTION 5

encouraging and giving evaluative feedback promotes the collaboration process (King, 1997). Also important in collaborative learning situations is that everybody should have the same responsibility for the actions taken (Ebbens, Ettekoven, & Van Rooijen, 1997; Wegerif et al., 1999), and everybody should participate actively in the interaction and problem solving process (Johnson & Johnson, 1999) with more or less the same number of actions (Damon & Phelps, 1989). Instructing learners to communicate effectively should take account of the importance of the activities de-scribed above.

Apart from supporting students’ collaborative learning process by means of instruction in effective communication, collaborative discovery learning can also be supported by the provision of cognitive tools. Cognitive tools (Lajoie, 1993; Van Joolingen, 1999) can support the learning process by providing instruments aimed at the direct support of specific cognitive or collaborative tasks. They do not deliver instruction, but instead scaffold the processes and actions that lead to knowledge construction (Jonassen, 2000; Salomon, 1993). Examples of cognitive tools that support discovery learning are tools that support hypothesis generation, such as a structured scratchpad to create hypotheses (Lajoie, Lavigne, Guerrera, & Munsie, 2001; Van Joolingen, & De Jong, 1991) as well as tools that support the design of experiments and the collection and interpretation of data (Reimann, 1991; Veermans, De Jong, & Van Joolingen, 2000).

Examples of cognitive tools that support collaboration are sentence-openers found in work by Baker and Lund (1997) and Lazonder, Wilhelm, and Ootes, (2003). In one of the studies presented in this thesis a cognitive tool is introduced, which not only supports the discovery learning process, but introduces support for collaboration as well.

4 MOTIVATION AND COLLABORATIVE DISCOVERY

A necessary condition for all learning, and specifically for collaborative discovery learning, to be successful is that learners are motivated. Learners need to be moti-vated to engage in the discovery learning processes itself and in the interaction with their peers. Motivation is important when working collaboratively (Strijbos, 2004) in a CSCL environment (Jones & Issroff, 2005). Few studies have investigated motiva-tion in collaborative learning environments (e.g., Järvelä, Rahikainen, & Lehtinen, 2001), or in collaborative discovery environments.

The motivation of students will be assessed in this dissertation, through the use of an expectancy-value model of motivation (Pintrich, Smith, Garcia, & McKeachie, 1993; Pintrich, 2000; Pintrich & De Groot, 1990; Wigfield, 1994; Wigfield & Eccles, 2000). In particular, the relationship between students’ motivational beliefs and the process of collaborative scientific discovery learning will be investigated.

5 OVERVIEW

The goal of this dissertation is to investigate the relationship between collaborative learning and discovery learning and to provide support for the communication proc-

6 CHAPTER 1

ess and the discovery process. When learners learn by means of discovery, it is as-sumed that the knowledge gathered will be more flexibly applied and more deeply anchored than when other pedagogical methods are used (Swaak & De Jong, 1996). The dissertation focuses on both the collaborative discovery learning process and knowledge as a product of learning. Three general research questions are posed: Which communicative activities and discovery activities are frequently used together in a collaborative scientific discovery learning environment?

In Chapter 2, this first research question is investigated in an explorative study. The aim of this study is to find out which communicative activities and discovery activi-ties are used in a collaborative discovery learning environment. Correlational analy-ses, factor analyses, and analyses of variance are used to explore the relationship between the communicative activities and the discovery activities of the students, as well as the relationship between these activities and learning outcomes. Can specific support of the communication process and the discovery learning proc-ess improve collaborative scientific discovery learning?

In Chapter 3 and Chapter 4, two specific kinds of support for collaborative discovery learning processes are presented. In Chapter 3, instruction in efficient and effective communication is provided in the form of the RIDE rules (Respect, Intelligent col-laboration, Deciding together, and Encouraging). The effects of this instruction on the collaborative discovery learning process and learning product are investigated. Chapter 4 presents the results of a study in which a cognitive tool, the Collaborative Hypothesis Tool (CHT), is introduced to support the collaborative discovery learn-ing process. The effects of the use of CHT on the collaborative discovery learning process and learning product are studied. What is the role of motivation in collaborative scientific discovery learning?

Chapter 5 reports the results of an analysis with respect to the role of motivation in learning of aggregated data of the participants of the studies performed in Chapters 2 to 4. In this chapter, the relationships between learners’ motivation and the individ-ual learning process and learning product are analyzed. In addition, the question of whether team heterogeneity in terms of motivation influences the collective collabo-rative discovery learning process and learning product is addressed. Finally, Chapter 6 presents a review of the results and conclusions of the four stud-ies presented in Chapters 2 to 5, followed by a discussion of general findings, in which the similarities and differences in the findings of the four studies are high-lighted. Limitations of the four studies are also discussed. The chapter concludes with a discussion of the educational implications of the studies.

The studies that are presented in Chapters 2 to 5, have been submitted or ac-cepted for publication in international journals. Since each chapter is based on a separate article, some overlap may occur.

CHAPTER 2

COMMUNICATION IN COLLABORATIVE DISCOVERY LEARNING∗

In this study, a computer-based learning environment is introduced, in which discovery learning and collaborative learning are implemented simultaneously. It is investigated which communicative activities are frequently used in the discovery learning process and which communicative and discovery activities co-occur. Learners (15-17-years-old) worked in dyads on separate screens in a shared discovery learning environment. They communicated using a chat box. In order to find a possible relationship between communicative activities and discovery learning processes, correlational analyses and principal compo-nent analysis were performed. Significant relations were found between communicative and discovery activities, as well as five factors combining the communicative process and the discovery learning proc-esses. Communicative activities are performed most frequently during the activities in generating hy-potheses, experimental design, and conclusion construction. Argumentation occurs less than expected and is associated with the construction of conclusions, instead of generating hypotheses. High-performing dyads proposed more answers, generated more hypotheses, and confirmed and accepted ideas more fre-quently than less successful dyads. Further research should focus on means to augment communicative and discovery activities that are related to positive learning outcomes.

1 INTRODUCTION

Constructivist approaches to learning (Duffy & Jonassen, 1991; Jonassen, 1991, 2000) focus on learning environments in which learners have the opportunity to construct knowledge and negotiate this knowledge with others. Discovery learning and collaborative learning are examples of learning contexts that cater for knowledge construction processes. In discovery learning (De Jong & Van Joolingen, 1998), learners construct knowledge based on new information and data collected by the learners in an explorative learning environment. In collaborative learning (Van der Linden, Erkens, Schmidt, & Renshaw, 2000), learners construct knowledge through mutual communication and the joint use of instruments and shared representations. Separately, these new forms of learning (Simons, Van der Linden, & Duffy, 2000) have been studied extensively (e.g., De Jong & Van Joolingen, 1998; Van der Linden et al., 2000). In this study, a computer-based learning ∗ Saab, N., Van Joolingen, W. R., & Van Hout-Wolters, B. H. A. M. (in press). Communica-tion in collaborative discovery learning. The British Journal of Educational Psychology.

8 CHAPTER 2

environment is introduced, in which the two forms of learning are implemented simultaneously. The focus lies on the interaction between the processes of discovery learning and collaborative learning.

In discovery learning, learners manipulate a domain by conducting experiments, i.e. by manipulating variables and parameters in a domain and observing the effects of these manipulations. Learners can build their own knowledge using this informa-tion (De Jong & Van Joolingen, 1998; Njoo & De Jong, 1993; Njoo, 1994). Discov-ery learning requires inductive processes, in which information and knowledge is generated from the experiments performed (Swaak, 1998). The skills necessary for the successful performance of this learning process are similar to scientific skills (Klahr & Dunbar, 1988; Van Joolingen & De Jong, 1997). To investigate the influ-ence of collaboration on discovery learning, it is necessary to start to form an analy-sis of the learning process needed in discovery learning. Earlier work has revealed that we can distinguish between regulative and transformative processes (De Jong & Njoo, 1992; Njoo, & De Jong, 1993). Regulative processes control the performance of the discovery activities, and include planning and monitoring. Transformative processes involve the generation of new information and knowledge, and reflect the empirical cycle as identified by many researchers, although categorization and terms vary (Burns, Okey, & Wise, 1985; Klahr & Dunbar, 1988; Van Joolingen & De Jong, 1997). Transformative processes include orientation (identifying variables and possible features, getting to know the domain), generating hypotheses (generation of new ideas on possible solutions), experimentation or testing hypotheses (gathering data from the domain), and conclusion (using information to decide whether a hy-pothesis must be rejected or not).

These theoretical considerations form a starting point in the analysis of discovery processes. Orientation can be further decomposed into identifying variables and parameters, collecting data, and interpreting these data and graphics exploratively. During the generating hypotheses process describing and recognizing relations is important, as are proposing answers, thinking of alternatives and formulating hy-potheses. The process of testing hypotheses involves designing experiments, predict-ing, and collecting data. Finally, conclusion involves interpreting the data, rejecting hypotheses, and making conclusions (see Table 1). The discovery processes are part of the inquiry cycle. This means that these processes are visited repeatedly. Previous studies (e.g., De Jong & Van Joolingen, 1998; Njoo & De Jong, 1993) have shown that, although there seems to be a preferred order in the inquiry processes, often this order is not used systematically.

Computer simulations are a flexible way of representing a learning domain. The learners who work with a simulation are engaged in active investigation while con-structing knowledge. Simulations can be used to create situations that are not possi-ble to investigate in reality (De Jong, 1991). Simulations can be used especially in learning domains like chemistry or physics, where students, while engaged in the discovery learning process, often have to perform experiments to find rules or for-mulas.

In this study we are interested in the mutual influences between collaborative learning and discovery learning. This interaction is interesting because different forms of cross-fertilization may be possible between the two forms of learning. Col-

COMMUNICATION IN COLLABORATIVE DISCOVERY LEARNING 9

laboration can have a positive influence on the discovery learning processes (Salo-mon & Globerson, 1989) and vice versa.

Table 1. Discovery learning processes and activities representing the inquiry cycle

Discovery learning processes and activities

Examples

Orientation

Identifying parameters and variables Collecting data Interpreting data and graphics

‘Time is in seconds’ A simulation run ‘The line goes through x=2’

Generating hypotheses Describing and recognizing relations Thinking of alternatives Proposing an answer Formulating hypotheses

‘Velocity is distance divided by time’ ‘It can be 7, too’ ‘I think it is 5’ ‘If t=2, then x=7’

Testing hypotheses Experimental design Predicting Collecting data

‘We have to do a simulation first’ ‘I think it will be t=3’ A simulation run

Conclusion Interpreting data and graphics Rejecting hypotheses Concluding

‘The ball is moving faster now’ ‘No, this idea was not right’ ‘So, as we can see it must be 1’

Note. The subcategories within the four general discovery processes are labeled ‘activities’.

In collaborative learning, two or more people try to share and construct knowledge in working towards a solution of an assignment or problem. Springer, Stanne, and Donovan (1999) report in a meta-analysis that collaborative learning can have a positive effect on learning. We see collaboration as a promoter of elaboration and explication (Dekker & Elshout-Mohr, 1998), for instance, by asking questions (Chi, Bassok, Lewis, Reimann, & Glaser, 1989), or giving meaning to concepts or prob-lems by explanation (Ploetzner, Dillenbourg, Preier, & Traum, 1999; She, 1999; Wegerif, 1996; Weiss, & Dillenbourg, 1999). By externalizing their thoughts, learn-ers can become conscious of their ideas and cognitive and metacognitive processes, and thus of the possible defects in these processes (Van Boxtel, 2000; Van der Lin-den et al., 2000). When learners verbalize their thoughts, planning and decision making, and again internalize these in an elaborative manner (Marshall, 1995; Roelofs, Van der Linden, & Erkens, 1999), by asking questions and giving explana-tions, this can lead to organization and attunement of the knowledge and, in the end, to extension of that knowledge (Chan, Burtis, & Bereiter, 1997; Wegerif, 1996; Wegerif, & Mercer, 1997). Communicative processes contributing to the joint con-struction of knowledge and building of common ground are argumentation and in-formation checking, such as asking for verification (Van Boxtel, Van der Linden, &

10 CHAPTER 2

Kanselaar, 2000; Veerman, 2000). Aspects of communication (e.g., explaining, checking, or asking questions) will be analyzed, partly based on the analysis scheme that Van Boxtel (2000) used in her study of concept development. Table 2 shows the communicative processes that are searched for in this study.

Table 2. The communicative process with examples

Communicative activities

Examples

Informative

‘I think the answer is three’

Argumentative Evaluative

‘Because the other time we did it also like this’ ‘That’s really good!’

Elicitative (asking the other for response) Verification (checking) Critical (checking)

‘What do you think about this question?’ ‘Do you also think it is 4?’ ‘Do you really think so?’

Responsive Confirmation/Acceptance

Answer to a question ‘Yes, I agree’

Directive ‘Try the other one’ Off task Off task technical

‘Yesterday I went to the beach’ ‘Can you move the upper window, please’

As collaboration triggers learners to elaborate their thoughts as part of the communication, it can be assumed that, as learners in collaborative learning search for a common way of working, the learners make the discovery learning processes explicit, for which one will expect a positive contribution to these processes. Okada and Simon (1997) conducted a study in which they compared singles and pairs working in a discovery learning environment. They found that pairs performed better than singles, because pairs used explanatory activities, such as generating hypotheses and generating alternative ideas more often than singles. These activities were effective for the discovery learning process only when the learners conducted experiments. This leads to the assumption that communicative activities in collaboration will contribute to discovery learning. In a study of Whitelock, Scanlon, Taylor, and O’Shea (1995) pairs of students worked together, face tot face, with a simulation environment about collisions. The hypothesis that pairs performed better than singles was confirmed in this study. The researchers concluded that the reasons for this better performance could be subscribed to peer interaction and, moreover, peer presence. Pairs, compared to singles, adopted a “predict, observe, explain” kind of approach and shared their meanings which could lead to better learning results. Peer presence could have had a motivational effect and anxiety could have been lowered because of working in pairs.

In addition, another relation between discovery and collaboration is possible, af-fecting the design of collaborative discovery learning environments. Knowledge about discovery learning processes and interaction data from learners with the dis-


covery environment can be used to support the communicative process. For in-stance, when, based on theoretical or empirical considerations, it is supposed to be beneficial to engage in argumentation during a part of the discovery process, say, generating hypotheses, the learning environment could issue prompts for the learner to start exchanging arguments. This would take the form of including communica-tive support into the learning environment used for supporting discovery learning. In the current study, we concentrate on the first of these two relations, the possible rela-tion between the communicative process and the discovery process, and possible indications that the first supports the second. However, information gathered in our study may provide input to designs of the learning environment as indicated in our second possible relation between discovery and collaboration.

If collaboration would contribute to discovery, it should be able to influence the performance of the processes of discovery learning. Certain communicative actions would trigger specific discovery processes and vice versa. This would mean that the elaborative aspects of collaboration would improve the quality and quantity of dis-covery learning processes. This will be elaborated for the transformative discovery learning processes.

In orientation, learners have to build common ground before they can work con-structively together (e.g., Baker, 1999). During orientation, building common ground would be the main function or goal of communication. For example, this means that learners have to check whether they are talking about the same thing be-fore they start solving the problem. This includes agreeing on the definition of vari-ables, initial ideas on relevant relations, and synchronizing background knowledge. Communicative activities that support the building of common ground include in-formative activities, such as issuing statements, elicitative activities, such as asking questions, and responsive activities such as answering questions. These communica-tive processes allow for a basic exchange of ideas, and hence establish what is common and what is not. In generating hypotheses, learners have to share hypothe-ses and alternative ideas, which can lead to co-construction of knowledge (Chi et al., 1989; Okada & Simon, 1997; Tao & Gunstone, 1999). They have to establish com-mon ideas by means of informative, argumentative and elicitative activities. The difference with orientation is that instead of exchanging existing ideas, new ideas and conjectures have to be exchanged. So, in addition to elicitative and informative activities that are shared with orientation, argumentative activities that focus on the reasonability and plausibility of hypotheses are expected. In testing hypotheses, learners have to agree on the design of the experiment and to perform experiments. Again, informative activities are expected. Directive activities are also needed, which is inherent to the nature of discovery learning environments, where typically one learner at a time can control the exploratory tools, such as a simulation, and other learners can only influence these actions by providing directions. Finally, in conclusion, learners have to agree on the decisions they made by means of argumen-tation, asking for response and, in the end, agreement. Decisions whether or not to reject a hypothesis, how to read and interpret graphs, and finally, whether or not there is agreement are activities that are needed in this process of conclusion. These activities require communication focused on the content and validity of arguments.

12 CHAPTER 2

Checking whether ideas are common or not by asking for response and agreement also is important.

Table 3. Discovery processes and activities with their corresponding communicative support and communicative activities

Discovery processes and activities

Communicative support

Communicative activity

Orientation

Identifying parameters and variables Collecting data Interpreting data and graphics

Build common ground Informative Elicitative

Generating hypotheses Describing and recognizing relations Thinking of alternatives Proposing an answer Formulating hypotheses

Exchange ideas Establish common ideas

Argumentative Elicitative Informative


Doing it (testing) Agree on design

Directive Informative


Establish common conclusions

Argumentative Elicitative Acceptance

We have shown that communicative activities can be linked to discovery activities. Table 3 summarizes the possible relations between collaboration and discovery. In the current chapter an exploratory study is presented, in which evidence is sought for a relation between collaboration and discovery. The research questions are: 1) Which communicative activities between two learners, working collaboratively

in a computer-based discovery environment, are frequently used in the discov-ery learning process?

2) Which communicative and discovery activities co-occur during this learning process?

According to the model presented in Table 3 it is expected that several communica-tion activities and discovery processes will align in the performance of dyads that engage in a discovery environment.


2 METHOD

In the current study, learners were engaged in a discovery learning environment on particle collisions (physics). Subjects worked in dyads on separate screens in a shared environment. They communicated using a chat box. The analysis of the chat communication and the discovery learning processes taking place in the environ-ment, as well as an analysis of the learning results yield the answers to our research questions.

2.1 Learning environment

The learning environment that is used in this study was based on a computer simula-tion called Collisions1 developed in SimQuest (Van Joolingen & De Jong, 2003). Dyads of students worked collaboratively on two computers with a shared interface, communicating through an unstructured chat channel (using Microsoft Netmeeting).

Figure 1. Screenshot of the learning environment used. Shown are the simulation window, assignment windows, and chat window.

Figure 1 displays the complete learning environment, as it appears to the students while working together on the assignments. The window in the upper right corner is

1 Collisions was developed by Hans Kingma and Koen Veermans (University of Twente). SimQuest was developed in the SERVIVE project coordinated by University of Twente.

14 CHAPTER 2

the chat window, by which the learners communicate. The window in the lower right corner shows the assignments. At this moment, the learners are working with assignment A1 of the topic “Eenparige Beweging” (“Uniform Motion”). The as-signment itself is shown in the window in the lower left corner. The learners can experiment in the window “Eenparige Beweging” in the upper right corner, where they can change the mass (‘massa’) and the velocity (‘snelheid’) of the ball. Discov-ering the rules behind the simulations was the main learning task of the environ-ment. The environment consisted of four simulations with matching assignments. Every assignment presented the learners with a small multiple-choice research ques-tion. By using the simulation, learners could gather data for answering the assign-ments. Explanations of each of the variables present in the simulation were available on request.

2.2 Subjects

The study involved 25 dyads of tenth-grade learners who were following pre-university education. Four of these dyads were not included in the analysis, because of technical problems. Their age ranged from 15 to 17 years. The learners were re-cruited from two secondary schools in Amsterdam (three classes). For their partici-pation, subjects received € 20.

2.3 Procedure

Learners individually completed a pretest and a posttest. Dyads of learners working in the learning environment were assigned randomly. The dyads consisted of learn-ers from the same class, so every dyad member knew each other before they worked together. Before the learners started working with the learning environment, an ex-planation was given plenary on how to work with the environment. An example as-signment was discussed and the learners were instructed to work together on each assignment in the learning environment. A complete session, including the pre- and posttests, took three hours. The learners worked for one hour on the pretests, for one-and-a-half hours they worked collaboratively with the learning environment, followed by half an hour for the posttests.

2.4 Measuring learning outcomes

Two types of learning outcome are identified. One is related to the performance within the learning environment, the other is a measure of what is learned from working with the learning environment (pretests and posttests). For the performance within the learning environment the students could get one point for each assign-ment if their answer was right the first time. So if they tried again after giving a wrong answer, no points were attributed. The amount of points gained by a team is taken as measure, which is labeled SWLE (score within learning environment). To measure what is learned from working with the learning environment the results of a domain knowledge posttest and the gain in score related to the domain knowledge


pretest is used as a measure. SWLE is measured on dyads, while the pre-posttest measure is taken individually.

In the pretest, learners were tested on domain knowledge. The domain knowledge was measured by means of an Explicit Knowledge Test and a WHAT-IF Test (Swaak, 1998; Veermans, De Jong, & Van Joolingen, 2000). These tests were developed specifically for the domain of Collisions2. Explicit Knowledge Test items test the learners for declarative, conceptual knowledge such as formulas and facts. The WHAT-IF Test presents the learners with situations before a collision, describes a change in the situation, such as a change of initial velocity, and asks the learners to predict the effect of the change. Both tests were administered on-screen.

2.5 Data analysis

The interaction between the learners -their communication on the chat channel and their activities in the learning environment- was logged. Chat messages were logged for each individual. Actions within the shared Collision environment were also re-corded, however, no information was available on which learner did what within Collisions. The chat messages were coded on the communication scale (see Table 2, section 1) whereas the scales on discovery learning activities was applied to both the chat communication and to running the simulation, which is represented by the codes collecting data for orientation and collecting data for testing hypotheses (see Table 1, section 1). For analyzing the protocols, the computer program MEPA (Mul-tiple Episode Protocol Analysis) was used (Erkens, 1998). Two independent raters rated 10% of the protocols. Cohen’s Kappa of inter-rater reliability between the two raters was on the communication dimension κ=.79 and on the discovery dimension κ=.84, which are both acceptable.

The purpose of this analysis is to identify cross-relations between discovery and communication activities and vice versa. A chat action or utterance is defined as one verbalization typed in a message box (Lebie, Rhoades, & McGrath, 1996). Each chat action was scored on both the communicative and the discovery aspect, sometimes as communicative or discovery activity only, sometimes as both communicative and discovery activity.

3 RESULTS

The learners in the experiment used the chat channel with ease. All subjects stated they had previous chat experience and therefore communicating through this chan-nel was no problem. 15,7% of the chat was off-task, which is comparable to a study of coordination processes in collaborative writing, where chat was divided in epi-sodes (compared to utterances) and 13% and 8% off task talk was found (Erkens, Jaspers, Prangsma, & Kanselaar, 2005). Off-task chat is characterized as interactions between learners that have nothing to do with the task, in contrast with off-task tech-

2 Both tests were developed by Janine Swaak (Swaak, 1998).

16 CHAPTER 2

nical talk, which doesn’t cover the content of the task, but is talk about the learning environment features.

Table 4 displays an overview of the frequencies of each communicative and dis-covery activity that are found in the protocols. The minimum of nine activities is zero. In most of the cases there are just one or two dyads of students that did not perform a certain activity, except for the activity describing and recognizing vari-ables in the generating hypotheses process, where seven pairs did not perform this activity. The average frequency of this activity for the group as a whole was M=2.33 (SD=2.63), just like the frequency of the activity identifying parameters and vari-ables in the orientation process (M=2.38; SD=4.06), which can be seen as a similar activity.

Table 4. Absolute frequencies of communicative and discovery activities for each protocol

Frequencies

Activity

M SD Min. Max.

Discovery learning

Orientation Identifying parameters and variables Collecting data Interpreting data and graphics

2.38

33.29 3.52

4.06

14.58 3.76

0 9 0

15 59 12

Generating hypotheses Describing and recognizing of relations Thinking of alternatives Proposing an answer Formulating hypotheses

2.33

25.19 28.19 7.43

2.63

17.95 17.08 5.75

0 0 2 0

9

75 71 20


6.52

.62 20.52

7.40 1.02

10.02

0 0 2

30

4 42


3.95 0.57 5.38

6.09 1.03 4.60

0 0 1

25

4 15

Communication

Informative 45.67 28.14 5 143 Argumentative Evaluative

14.33 26.29

11.19 12.27

0 12

36 60

Elicitative 34.62 15.50 12 75 Responsive Confirmation/Acceptance

16.14 28.29

8.00 20.84

4 4

35 88

Directive 22.10 12.80 7 50 Off task Off task technical

43.62 47.95

37.86 24.09

3 11

160 103


The discovery activities collecting data, thinking of alternatives, and proposing an-swers are performed the most. For the communicative activities, all of the activities are employed frequently, except for the argumentative and the responsive process, although there is a large variance over dyads.

3.1 Relation between communication and discovery learning activities

In order to find a possible relation between communicative and discovery learning activities a correlational analysis with the discovery and communicative activities was performed. A non-parametric correlational analysis is performed because of the skewness of the distribution of some of the variables. Several significant correlations between the communicative and the discovery activities are found (Table 5).

Table 5. Spearman’s rho correlations between discovery and communicative activities in different discovery learning processes (see Table 1 and Table 2 for definitions of the commu-

nicative and discovery activities)


Discovery processes and activities

Info

rmat

ive

Arg

umen

tativ

e

Eva

luat

ive

Elic

itati

ve

Res

pons

ive

Acc

epta

nce

Dir

ectiv

e

Off

task

Orientation

Identifying parameters and variables Collecting data Interpreting data and graphics .57**

Generating hypotheses Describing and recognizing relations Thinking of alternatives .78** .52* .62** .53* Proposing an answer .86** .54* .64** Formulating hypotheses .44*

Testing hypotheses Experimental design .46* .61** .58** .70** .61** -.55* Predicting .46* Collecting data .61**

Conclusion Interpreting data and graphics Rejecting hypotheses Concluding .82** .46* .51* .48*

Note. Only significant correlations are shown. *p<0.05. **p<0.01. Although we expected informative and elicitative activities to be the communicative activities that would correlate with orientation, a significant correlation was only

18 CHAPTER 2

found between argumentative activities and interpreting data and graphics (r=.57; p<.01).

In generating hypotheses we expected argumentative, elicitative and informative activities to co-occur. There are significant relations between informative activities with three activities in generating hypotheses: thinking of alternatives (r=.78; p<.01), proposing answers (r=.86; p<.01), and formulating hypotheses (r=.44; p<.05). In addition, elicitative activities and confirmation/acceptance co-occur with proposing answers and thinking of alternatives. Responsive activities correlate with thinking of alternatives (r=.62; p<.01).

In testing hypotheses we expected directive and informative activities to have a correlation with the discovery activities in this process. Significant correlations be-tween the informative activities and all the discovery activities in this process are found. Positive significant correlations between experimental design and evaluative (r=.61; p<.01), responsive (r=.58; p<.01), acceptance (r=.70; p<.01), and directive (r=.61; p<.01) activities are also found. There is a negative correlation with experi-mental design and off-task chat (r=-.55; p<.05).

In conclusion we predicted that argumentative, elicitative, and acceptance activi-ties would correlate with the discovery activities in this process. Concluding corre-lates significantly with the argumentative activity (r=.82; p<.01), the elicitative ac-tivity (r=.46; p<.05), the responsive activity (r=.51; p<.05), and acceptance (r=.48; p<.05).

To gain more insight into the relation between communicative and discovery learn-ing activities, the total group of activities was subjected to principal component analysis followed by Promax with Kaiser Normalization rotation with eigenvalue > 1.00 as a criterion for determining the number of factors. This analysis was per-formed for the pairs and not for individual members of teams. The activities sub-jected to the exploratory factor analysis resulted in five factors, which together ac-counted for 84.2% of variance. Table 6 shows the Pattern matrix. The boldface numbers indicate which activities show the highest load on the different factors.

Correlations between factors were computed to examine the relations among the resulting factors. Table 7 shows the Factor correlation matrix. The correlations among the factors ranged between 0.06 and 0.58, indicating sufficient independence of each factor, except for the correlation between factor 1 and 4 (r=.58).


Table 6. Pattern matrix with the loading of activities on different factors. For each activity, the highest load is boldface

Factor

Activity 1 2 3 4

5


Informative .12 .65 -.05 .08 .41 Argumentative .13 .10 .91 -.09 .03 Evaluative 1.17 -.04 -.10 -.39 -.16 Elicitative .00 .23 .11 .81 .06 Responsive _a _a _a _a _a Confirmation/acceptance .90 .12 .24 -.07 -.22 Directive .59 -.15 -.11 .40 .06 Discovery learning activities Identifying parameters and variables _a _a _a _a _a Collecting data for orientation -.16 -.28 .16 -.51 .93 Interpreting data and graphics in the orientation process .04 -.32 .86 .10 .18 Describing and recognizing of relations -.20 -.24 .08 .98 -.21 Thinking of alternatives .28 .47 0.01 .08 .36 Proposing an answer .26 .73 -.09 .02 .18 Formulating hypotheses -.22 1.10 -.09 -.04 -.16 Experimental design .82 -.25 .09 .27 -.08 Predicting _a _a _a _a _a Collecting data for testing hypotheses -.18 .13 -.03 .07 .92 Interpreting data and graphics in the conclusion _a _a _a _a _a Rejecting hypotheses -.17 .82 .24 -.32 -.18 Concluding -.08 .19 .86 .20 -.10

aNot all activities are represented in this analysis, because the correlation matrix was not positive definite or the Kaiser-Meyer-Olkin Measure of Sampling Adequacy was very low.

Table 7. Factor correlation matrix

Factor

1 2 3 4 5

1

1.00

.33

.11

.58

.38

2 1.00 .20 .27 .35 3 1.00 .10 .06 4 1.00 .31

Table 8 shows a description of the factors in terms of the activities that load highest on them. Looking at the components of a factor reveals the relation between the dis-

20 CHAPTER 2

covery processes and communication processes that constitute it. The first factor indicates that experimental design is related with a more directive pattern of giving orders and accepting these. Evaluative also loads high on this factor. The second factor shows that the typical processes of generating new ideas (propose answers, give alternative answers, generate and reject hypotheses) associate with informative communication. Interpreting data and concluding involves argumentation, as indi-cated by the third factor. Elicitative communication is performed together with de-scribing and recognizing or relations. This can be seen as an orientation factor. Fi-nally, collecting data is not associated with any particular communication activity.

Table 8. Description of factors

Factor 1 Factor 2 Factor 3 Factor 4 Factor 5


Evaluative Directive Confirmation/acceptance

Informative Argumentative Elicitative

Discovery activities

Experimental design Thinking of alternatives Formulating hypotheses Rejecting hypotheses Proposing an answer

Interpreting data and graphics in the orientation process Concluding

Describing and recog-nizing of relations

Collecting data for orientation Collecting data for testing hypotheses

In Table 9 and Table 10, two protocol fragments are shown. In Table 9, factor 1 and factor 4 are represented. In this protocol, the students are trying to establish common ideas and are trying to agree on the design of the experiment. These are two of the communicative supports that are mentioned in Table 3 (see section 1). In this learn-ing environment, one of the learners has the control. First, learner A has the control and learner B gives the orders, until line 12 where learner B takes over the control and learner A gives the orders. This protocol gives an example of the non-chronological structure of the discovery learning processes. The learners switch from the generating hypotheses process –italic- to the testing hypotheses process -bold.


Table 9. Piece of protocol with factor 1(bold) and factor 4 (italic) being represented

Learner Action Communication Discovery

A

Which one was the x(t)

Elicitative

Describing and recognizing parameters and variables in the generating hy-potheses process

A The first one, isn’t it?

Elicitative Describing and recognizing parameters and variables in the generating hy-potheses process

B Yes Confirmation/ acceptance

B Oh no, just try it again

Directive

A Doesn’t work Informative B Try some different

things now Directive Experimental design

A OK Confirmation/ acceptance

B Change the mass Directive Experimental design A OK Confirmation/

acceptance

B What was the velocity again?

Elicitative Describing and recognizing parameters and variables in the generating hy-potheses process

A I don’t remember Responsive A Leave it like this Directive Experimental design B Just leave it on 10? Elicitative Experimental design A Yes Confirmation/

acceptance

B OK Confirmation/ acceptance

SIMULATION Collecting data for testing hypotheses A Enlarge the mass Directive Experimental design

SIMULATION Collecting data for testing hypotheses

22 CHAPTER 2

Table 10 shows a protocol fragment where factors 2, 3 and 5 are represented. In this protocol the students are exchanging ideas, building common ground, and testing their hypotheses.

Table 10. Piece of protocol with factor 2 (bold), factor 3 (italic), and factor 5 (underlined) being represented

Learner Action Communication Discovery

C

But I don’t see a horizontal line

Argumentative

Interpreting data and graphics for orientation

D If x-t is horizontal, then is the answer 3

Argumentative

D I think hypothesis 5 is the right answer

Informative Proposing an answer

SIMULATION Collecting data for testing hypotheses

C Hypotheses 1, 2, 3 and 6 are not true.

Informative Thinking of alternatives

C 4 and 7 are true Informative Thinking of alternatives SIMULATION Collecting data for testing

hypotheses D 4 is wrong Informative Thinking of alternatives D 7 is the same as 5 Informative Thinking of alternatives

D Only 5 is a better answer Informative Thinking of alternatives D Do you see it? Elicitative

C Yes Confirmation/ acceptance

C Because they don’t always multiply

Argumentative Concluding

3.2 Learning outcome

The results of a paired t-test, as shown in Table 11, indicate a significant progress in scores at the posttests (the Explicit Knowledge Test and the WHAT-IF Test), compared to the pretest-scores. The Pearson correlation between the pre Explicit Knowledge Test and the post Explicit Knowledge Test is significant (r=.30, p<0.05). The Pearson correlation between the pre WHAT-IF Test and the post WHAT-IF Test (r=.35, p<0.05) is significant.


Table 11. Results of paired t-test of the pre- and posttests

Pretest

Posttest

M SD M SD t Df p

N

Explicit Knowledge Test

6.40

2.56

7.62

2.72

-2.766

49

.008

50

WHAT-IF Test

9.88 4.58 13.76 4.30 -5.425 49 .000 50

The results show that the learners improved between pretest and posttest. This means that the learners gathered information from the domain and built their own knowledge with this information (De Jong & Van Joolingen, 1998; Njoo & De Jong, 1993; Njoo, 1994). Correlational analysis between the pre- and posttests show that part of the progress of the learners on the tests can be explained by their prior knowledge. However, the other part of the progress can be attributed to the proc-esses of communication and discovery taking place while the students collaborated in the learning environment.

The teams were divided in ‘high-performing dyads’ and ‘low-performing dyads’ by using the median on the basis of the SWLE scores of the teams. A Mann Whitney U test was conducted to find possible differences between these kinds of teams. Sig-nificant differences are found between the two groups (Table 12) for proposing an answer (p=.015), formulating hypotheses (p=.049), and confirmation/acceptance (p=.036).

Table 12. Results of a Mann Whitney U test between ‘high performing dyads’ and ‘low per-forming dyads’

Good teamsa

Weak teamsb Mann Whitney U

M SD M SD U z

p1

Proposing an answer

35.82

16.85

19.80

13.53

24.5

-2.148

.015

Formulating hypotheses 9.09 5.87 5.60 5.30 31.5 -1.659 .049 Confirmation/Acceptance 36.09 24.82 19.70 11.13 29.5 -1.797 .036

1 One tailed significance. aN=11 bN=10.

24 CHAPTER 2

4 CONCLUSIONS AND DISCUSSION

In this chapter, a study into the contribution of communication to processes as discovery learning is presented. We look for relations between modes of communication and modes of discovery by analyzing the communication between members of a dyad involved in a discovery learning activity. We look for relations from two perspectives: one focused on communication and one on the discovery processes. As we see communication as promoting elaboration and explication (Dekker & Elshout-Mohr, 1998), which can lead to organization and attunement of knowledge and to the extension of that knowledge (Roelofs et al., 1999; Wegerif & Mercer, 1996), we expect different communicative activities to co-occur with several discovery learning activities.

In the introduction of this chapter, the expected co-occurrence of discovery ac-tivities and communication activities is listed. We predicted that communication would fulfill certain functions (grounding, information exchange, establish common ideas, etc.) in the discovery process, and, moreover, we aligned these communica-tive activities with the various transformative activities of discovery learning. The correlations between the individual processes and their loading on the factors identi-fied indicate whether these expected alignments are found.

In the orientation process of discovery, we expected that informative and elicita-tive activities would contribute to the grounding between learners needed in this process. However, it was found that learners exposed little orientation activities, other than collecting data. No significant relations of informative or elicitative communication activities with orientation activities were found, although we did expect such relations. The correlational analysis did show an unexpected correlation between argumentation and the interpretation of data and graphics in the orientation process. This indicates that, as part of the orientation, learners already argue about the data they have collected, i.e. data collected in a way that is not driven by a pre-stated hypothesis or theory.

For generating hypotheses we expected argumentative, elicitative and informa-tive activities to contribute to the exchange of ideas and to establish common ideas, and hence to co-occur in the analysis. Both factor analysis and correlational analysis indicate that the informative activities are predominant here. Learners exchange ideas but do not strive for agreement in the stage where they are generating hypothe-ses. The correlational analysis confirms the absence of argumentation, but it does show elicitation, responsiveness, and acceptance associated with sub-activities of generating hypotheses.

In testing hypotheses we expected directive and informative activities to contrib-ute to agreement on the design of experiments. In the learning environment, always one learner is in control of the cursor and the other learner gives orders. In the ex-perimentation stage, evaluative, directiveness, and acceptance are in the same factor as one of the discovery activities in this process, experimental design. Significant correlations were found between the informative communication activity and all the discovery activities in this process. A significant correlation between the responsive activity and experimental design is also found. The relationship between directive communication and experimentation can be explained from the nature of the learn-


ing environment, where doing experiments involves many operations on the simula-tion in the learning environment. These operations can be done by one learner who is in control (although learners can switch control, always one learner has the lead). This means that the other learner is at that moment limited to giving directions in order to control the design of experiments.

For conclusion we predicted that argumentative, elicitative, and acceptance ac-tivities would correlate with the discovery activities in this process, as these activi-ties would help learners to establish agreement on the conclusions with respect to their hypotheses. Both the argumentative and elicitative activities do correlate sig-nificantly with conclusion. In addition, the responsive activity correlates with con-clusion.

Collecting data in either discovery process is carried out independently of any communicative process. It should be noted that the protocols in this study are coded with a two dimensional scale, on communication and discovery activities, except for the activity collecting data. This activity, in both the discovery processes orientation and testing hypotheses, can only be coded as a discovery activity. Even though cor-relations between collecting data and communicative activities are possible, a prob-able cause of this result can be the method of coding.

It has been shown that communicative activities are performed most frequently with generating hypotheses, experimental design, and concluding. It has also been shown that argumentation occurs little and is most strongly associated with conclu-sion, instead of with generating hypotheses, as we expected. When learners generate hypotheses, they appear to just exchange ideas rather than arguing about them. Some argumentation takes place when interpreting graphics that result from exploratory data collection, but most argumentation is associated with concluding. So, not until hypotheses are tested, they are really discussed. Viewing the dyads as a unit, this means that units shows data driven or experimenter behavior (e.g., Klahr & Dunbar, 1988). Usually this is associated with lack of prior knowledge of the domain investi-gated. An alternative explanation, specific to collaborative work, could be that learners lack skills and insight in how to discuss their knowledge with their co-workers, and that they need a concrete trigger, such as the request to answer a ques-tion together, to exchange arguments.

Although limited, there are some relations between process and product. Dyads that perform well on working within the learning environment (SWLE score) also propose each other more answers and generate more hypotheses. This result is in line with the results in the study of Okada and Simon (1997) where pairs performed better than singles because of using explanatory activities. On the communicative side they confirm and accept more, indicating that they establish agreement more often. Again, there is no visible relation with argumentation. Agreement thus pays of in this situation. One explanation for this result that not can be excluded is that, within the domain chosen, the better couples did not disagree, either because they had the same ideas beforehand, or because one of the two learners in the dyad was dominant. Further studies, with more controlled group composition could provide more information on this issue.

The exploratory nature of this study only allows us to show correlations, no cause-effect relations. Hence, the question what comes first, the communication or

26 CHAPTER 2

the discovery process cannot be answered. Van der Linden et al. (2000) stress the relevance of aspects of collaboration such as maintaining common ground, co-responsibility, verbalization, and mutual support and criticism. Other research, in-cluding the ones leading to our communicative analysis (Van Boxtel, et. al, 2000; Veerman, 2000), stressed the importance of specific communicative activities for these aspects. Our contribution is linking these communicative activities to cognitive processes, associated with the goals of working within the collaborative learning environment, in our case scientific discovery. Social constructivist theory of learning stresses the importance of both discovery learning, where learners can construct their own knowledge actively, and collaborative learning, where learners can share their meanings. The results of this study enable us to connect these two approaches of learning. Further research must reveal the causal structure of these relations, which can be done by influencing one part, for instance elicitating specific commu-nicative processes, and observing the effects on the learning process and learning product. Such a line of research can help defining the context in which collaborative learning can be utilized most fruitfully.

CHAPTER 3

SUPPORTING COMMUNICATION IN A COLLABORATIVE SCIENTIFIC DISCOVERY

LEARNING ENVIRONMENT

The effect of instruction∗

In this Chapter, a study on the effect of instruction on collaboration in a collaborative scientific discovery learning environment is presented. The instruction we used, called RIDE, is built upon four principles identified in the literature on collaborative processes: Respect, Intelligent collaboration, Deciding to-gether, and Encouraging. In an experimental study, a group of learners (age 15-17) receiving this instruc-tion was compared to a control group. The learners worked in dyads on separate computers in a shared discovery learning environment in the physics domain of collisions, communicating through a chat chan-nel. Analyses of the logged actions in the learning environment and the chat protocols showed that the RIDE instruction can lead to more constructive communication, and more effective discovery learning activities, as expected, although no direct effect on discovery learning results was found. This study shows the benefits of providing instruction on effective communication and the learning process in a collaborative discovery learning situation.

1 INTRODUCTION

Research has shown that collaboration between learners may improve learning (e.g., Springer, Stanne, & Donovan, 1999; Van der Linden, Erkens, Schmidt, & Renshaw, 2000). In a collaborative learning situation, two or more learners working together, for example, to solve a problem or create a product for an assignment, construct knowledge through communication and the shared use of tools and representations. Collaboration requires learners to externalize their reasoning by means of communi-cation, which may make them aware of possible deficits in their thoughts (Van Box-tel, 2000; Van der Linden et al., 2000). When they then internalize their thoughts in an elaborative way (asking questions and giving explanations) (Marshall, 1995; Roelofs, Van der Linden, & Erkens, 1999), this can lead to a better organization of

∗ Saab, N., Van Joolingen, W. R., & Van Hout-Wolters, B. H. A. M. (submitted). Supporting communication in a collaborative scientific discovery learning environment: The effect of instruction.

28 CHAPTER 3

existing knowledge or the constructing of new knowledge (Chan, Burtis, & Bereiter, 1997; Wegerif, 1996; Wegerif & Mercer, 1997). Collaborative learning can contrib-ute to better learning in problem solving situations (e.g. Mercer, 1996), as well as in discovery learning environments (Saab, Van Joolingen, & Van Hout-Wolters, in press; Chapter 2; Salomon, & Globerson, 1989). Collaboration triggers learners to elaborate their thoughts as part of the communication (Dekker & Elshout-Mohr, 1998). Learners, working in a collaborative environment can make the scientific discovery learning processes explicit, which can lead to a positive contribution to these processes. Okada and Simon (1997) showed in their research that collaborative learning enhanced scientific discovery learning, because, for instance, generating hypotheses was made explicit by the participants, which resulted in a better per-formance compared to the learners who worked individually. In this chapter, we focus on computer-based learning environments where learners are collaboratively involved in the discovery learning process by means of chatting with each other. Among the advantages of using chat as a medium is the possibility for learners to see the history of their communication so that the discussion becomes explicit (Veerman, Andriessen, & Kanselaar, 2000), and that they need to explicitly formu-late their thoughts before sending a message.

It is not self-evident that learners know how to collaborate constructively. Sev-eral studies have shown that collaboration without instruction or support on how to collaborate does not lead automatically to effective knowledge construction (Chan, 2001; Mercer, 1996; Ross & Cousins, 1995; Webb & Farivar, 1994). For example, some learners in a team can do all the work while the other participants do little or even nothing, the so-called free-rider effect (Wasson, 1998). Another example is that learners work individually (Ettekoven, 1997) and do not check with their partners if everything is understood (Baker, Hansen, Joiner, & Traum, 1999).

In a previous study (Saab et al., in press; see Chapter 2), we found a relation be-tween the communication process of argumentation and the discovery process of drawing conclusions. Informative communication processes and the discovery proc-esses hypothesis generation were also related in this study. Based on the findings of our research, we can hypothesize that if it were possible to induce these communica-tive processes in learners, a more successful discovery process may be achieved.

In the study we present in this chapter, we try to achieve this by providing learners with instructions on how to collaborate. Where learners do not know what is expected, or where they work individually instead of collaborating (Ettekoven, 1997) instruction on how to behave effectively in collaborative settings may have a positive effect on the collaborative processes. Mercer and colleagues conducted several experiments where they gave instructions in effective communication to children in aged between 8 and 11 years old (e.g., Mercer, 1996; Rojas-Drummond & Mercer, 2003; Wegerif, Mercer, & Dawes, 1999). They found an increase of “exploratory talk” after the children received instructions. Exploratory talk is defined as a kind of discussion in which learners talk through their problems and investigate ideas together. It may be characterized by giving statements and new ideas, and reacting constructively and critically on these statements by offering justifications and alternative hypotheses. They also found that exploratory talk leads to better problem solving, both for a group and for the individual. The instructions

SUPPORTING COMMUNICATION 29

that elicited explorative talk were based on ground rules (Mercer, 1996; Wegerif et al., 1999). These rules included, for example, that the groups have to seek agreement; that alternative ideas have to be discussed before the groups reach agreement; and that the participants ask for reasons when appropriate.

A basic question is what the contents of such instruction should be. To give ef-fective instruction in collaboration, we need to know the important communicative activities for effective collaboration. In the literature, several characteristics of effec-tive collaboration are mentioned: • Learners should allow all participants to have a chance to join the communica-

tion process (Wegerif & Mercer, 1996); • Learners should share relevant information and consider ideas brought up by

every participant thoroughly (King, 1997; Wegerif et al., 1999); • Learners should provide each other with elaborated help and explanations

(King, 1997; Ploetzner, Dillenbourg, Preier, & Traum, 1999; She, 1999; Webb & Farivar, 1994; Wegerif, 1996; Weiss & Dillenbourg, 1999);

• Learners should strive for joint agreement by, for example, asking verification questions (Baker et al., 1999; Bandura, 2001; Erkens, Andriessen, & Peters, 2003; Mercer, 1996; Van Boxtel, Van der Linden, & Kanselaar, 2000; Wegerif & Mercer, 1996; Wegerif et al., 1999);

• Learners should discuss alternatives before a group decision is taken or action is undertaken (cf. Veerman, et al., 2000; Wegerif et al., 1999);

• All learners should take responsibility for the decisions and action taken (Ebbens, Ettekoven, & Van Rooijen, 1997; Wegerif et al., 1999);

• Learners should ask each other clear and elaborated questions until help is given (Chi, Bassok, Lewis, Reimann, & Glaser, 1989; King, 1997; Veerman et al., 2000; Webb & Farivar, 1994; Wegerif & Mercer, 1996; Wegerif et al., 1999);

• Learners should encourage each other (King, 1997); • Learners should provide each other with evaluative feedback (King, 1997).

Although these characteristics are found in several studies on how to support or instruct effective communication, none of them was aiming to support synchronous distance communication, such as text-based chat. For our study, we developed a computerized instruction to assist learners to collaborate by means of chat, based on the findings in the literature described above. The rules we extracted were grouped under four principles: Respect, Intelligent collaboration, Deciding together, and Encouraging, which we labeled as the “RIDE rules”. Table 1 lists how this grouping was constructed. For the form of instruction we chose the principles of the cognitive apprenticeship model or situated cognition (Brown, Collins, & Duguid, 1989; De Jager, Reezigt, & Creemers, 2002; Hendricks, 2001; Masterman & Sharples, 2002): activating prior knowledge of the learners, modeling of skills, coaching or supporting, scaffolding, articulation by the learners, and evaluative and reflection by the learners (Webb & Farivar, 1994). The detailed design of the instructional material is described in the Method section below.

30 CHAPTER 3

Table 1. Rules and sub-rules taught to the learners by means of a computerized instruction

RIDE rules

Sub-rules

(R) Respect

Everyone will have a chance to talk Everyone’s ideas will be thoroughly considered

(I) Intelligent collaboration Sharing all relevant information and suggestions Clarify the information given Explain the answers given Give criticisms

(D) Deciding together Explicit and joint agreement will precede decisions and actions Accepting that the group (rather than an individual member) is responsible for decisions and actions

(E) Encouraging Ask for explanations Ask till you understand Give positive feedback

We expect that instruction based on the RIDE rules will lead to more communica-tive activities that contribute to successful collaboration. The study presented here investigates the effects of this instruction. It is expected that the instruction will lead to an increase in relevant communication activities, which represent behavior ac-cording to the RIDE rules. Based on the results of our previous study (Saab et al., in press), we hypothesize that this will in turn lead to more effective discovery learning activities, in particular for the Intelligent Collaboration part of the RIDE rules, as this category encompasses argumentation and informative activities for which a positive effect was found. Hence, the research questions are: Can instruction in effective communication in a discovery learning environment lead to: 1) more effective communicative activities during the discovery learning process? 2) more effective discovery learning activities? 3) improved discovery learning results?

2 METHOD

2.1 Subjects and design

This study involved 38 pairs of tenth-grade learners of a secondary school who were following pre-university education, and who had chosen physics as a topic. Their age ranged from 15 to 17 years. The learners were recruited from three secondary schools in Amsterdam. For their participation, subjects received €20. The design of the study was a pretest-posttest control group design. The learners were randomly assigned to an experimental group and a control group. Due to technical problems and the fact that some learners did not show up in the second session 9 pairs dropped


out. As a result, the experimental group contained 17 and the control group 12 pairs of learners.


2.2 Learning environment and task

The learners worked together with a learning environment that was based on a com-puter simulation, Collisions, developed in SimQuest (Van Joolingen & De Jong 2003)3. The main learning task was to discover the rules of physics behind the simu-lation. In Collisions, learners are presented with assignments that focus their atten-tion to a specific part of the model they investigate. Assignments present the learners with a short term research question, and were presented in the form of an open ques-tion or a multiple-choice question. In total there were 35 assignments, of which sev-enteen open questions. Open questions were used to orientate on the level, multiple choice questions could be used by learners to test their own knowledge. Learners were free in choosing whether or not to use assignments and were also free in the order to do them, although the naming of assignments suggested an order, as as-signment names began with a number. Data for answering the assignments could be

3 Collisions was developed by Hans Kingma and Koen Veermans (University of Twente). SimQuest was developed in the SERVIVE project coordinated by the University of Twente.

32 CHAPTER 3

collected using the simulation. The environment also contains explanations for each of the variables present in the simulation. These explanations are available on re-quest for the learners. Pairs of learners worked collaboratively on two computers with a shared interface, communicating through a chat channel (Figure 1).

The learning environment was different for the experimental group and the con-trol group. Whenever an assignment is opened in the experimental version, a prompt (Wegerif, 1996) will pop up reminding the learners of one of the rules of the instruc-tion (i.e. the RIDE rules). An example of such a prompt is given in Figure 2. The control group did not receive any kind of pop-ups.

Figure 2. Example of a prompt with the rule Deciding together with sub-rules.

Before working with the application Collisions, learners in both groups received an instruction. The goal was to acquaint them with the learning environment by work-ing with a similar environment (also made with SimQuest). The control group re-ceived an instruction on logical thinking problems, which had nothing to do with the experiment, and the experimental group received an instruction on collaboration, i.e. the RIDE instruction. Instruction in the RIDE rules consisted of a recorded computerized presentation, where the rules were explained by both audio and visual means. After the presenta-tion, the learners practiced the rules by working together in the SimQuest environ-ment, where they were reminded to pursue the rules while collaborating. The design of this instruction is shown in Table 2.

The scheme of Table 2 was repeated for each of the four RIDE rules. The data of the evaluation forms (see Table 2) were collected and showed that 83 % of the learners mentioned they applied the rules with ease. The learners found that the fol-lowing aspects of the rules went well: explaining, asking for opinions or explana-tions, making decisions together and respecting each other (for example: listening to each other), although explaining was sometimes found difficult.


Table 2. Design of the computerized instruction

Presentation on screen

Rule introduction

The students can read the rule and the rule is being read out loud by a voice recorded in the computer.

Rule clarification

Every general rule has sub-rules. The students can read the sub-rules with every general rule. An example of every sub-rule is being read out loud.

Examples The students see several chats with good and bad examples of every rule.

Every example starts with the announcement of the rule read out loud, then an assignment is shown, and finally in a chat window it can be seen how two people chat, while solving the problem. The students are stimulated by a ques-tion (audio) to consider whether the communication between the people in the chat window is good or bad. Then the example is being evaluated (audio).

Practicing the rules

Concrete application

The students can think together (they are working together already) about assignments with regard to the different rules. The students are asked to think about examples for the different rules.

Practice The students have to solve different problems together. All the time the rules

are shown in a window. In addition, they sometimes pop up together with assignments.

Evaluation The students have to fill in together (on screen) an evaluation form. They are

asked whether the rules are applied in the practice session, where they have to pay attention to in the future, how the collaboration went, and if they liked it.


In this experiment, we identified two types of learning outcome. One is associated with the performance within the learning environment; the other is a measure of what is learned from this performance. For the performance within the learning envi-ronment we assessed the answers given to the assignments. Three points were awarded for a good answer on each assignment. For the multiple-choice assignments this was all or nothing, whereas for open-answer assignments scores of one or two were possible. Three points were awarded for a correct answer that was supported by an argument. We labeled this score SWLE, score within learning environment. Given 35 assignments, the maximum SWLE score was 105 points. As learners were free to choose from the set of assignments we computed an absolute and a relative SWLE, consisting of the total number of points scored and the percentage of points scored on assignments that were actually opened respectively. The pretest-posttest measure was taken individually, while SWLE was measured on dyads.

34 CHAPTER 3

The domain knowledge pretests and posttests each consisted of two domain knowl-edge tests, an explicit knowledge test, which tests the learners for declarative knowl-edge, such as facts and formulas, and a WHAT-IF test (Swaak, 1998; Veermans et al., 2000), which asks the learners to predict an effect of a change after showing them situations before a collision, and presenting a change in the situation. Both tests were developed specifically for the domain of Collisions4 and were adminis-tered on screen. The pretests and posttests were both parallel versions of the tests. In earlier experiments (Saab et al., in press), learners could not finish all the levels of the Collisions application in the time given. That is why the last of the four levels in the application were removed as well as the corresponding items in the pre- and posttests. The explicit knowledge test consisted of 15 items (originally 20 items) and the WHAT-IF test consisted of 13 items (originally 24).

2.4 Procedure

For making up dyads, we chose a group composition that is heterogeneous with respect to school achievement, since research has shown that learners working in such groups are more successful working together than learners working in homogeneous groups (Blatchford, Kutnick, Baines, & Galton, 2003). The reason is that the brighter learner can learn from giving elaborated explanations (Webb & Farivar, 1994), while the weaker learner can learn from the explanation given (Van der Linden et al., 2000). However, the difference in level between learners should not be too large. We used the grades of the learners, provided by the teachers beforehand, to identify the levels of achievement of the learners. To compose heterogeneous groups we used a method based on the one Pijls, Dekker and Van Hout-Wolters (2003) used in their study. All learners received a token: a colored triangle or circle. The color stood for grade (red was either high or low graded, yellow was middle grade), and the shape stood for condition. Subjects were not informed about this meaning. Subjects were free to choose a partner who had a token with the same shape but different color. This assured that the dyads were composed of either a low and a middle graded learner or a middle and a high graded learner, and that learners could choose someone that they liked to work with.

Participants in the study attended two sessions. In the first session, the partici-pants in the experimental group received individual instruction on collaboration ac-cording to the RIDE rules (see Table 1). In the first session, the control group re-ceived instructions on how to solve logical reasoning problems, which had nothing to do with the content of the experiment. This session dealt with problems such as: “Tomorrow it is Wednesday. What day is it four days before yesterday?” After the instruction sessions, all learners practiced with the simulation learning environment SimQuest on an application with logical thinking problems, in a collaborative setting similar to the Collision environment. In this way, the learners from both groups be-came acquainted with working together in this learning environment. Furthermore, the learners in the experimental group could practice applying the rules they had learned earlier in the session. 4 Both tests were developed by Janine Swaak (Swaak, 1998).


The second session started with the pre-domain knowledge tests for all learners. Then, they worked together for 90 minutes with the application Collisions in the learning environment SimQuest, and it ended with the post-domain knowledge tests.

2.5 Data collection and analysis.

All communicative and discovery learning activities were logged, as well as the chats, and were put together in a single protocol for each dyad. A three-dimensional analyses scheme (Table 3) is used to analyze the protocols. The dimensions are: (a) communicative activities; (b) discovery transformative learning activities, which promote the generation of information (Njoo & De Jong, 1993); and (c) discovery regulative learning activities, which support and guide the learning process (Njoo & De Jong, 1993). The communication dimension of the analysis scheme is partly based on the analysis scheme that Van Boxtel (2000) used in her study of collabora-tive concept learning. The scheme has been developed and used before in a descrip-tive study about the frequency of communicative activities used in the discovery learning process and the co-occurrence of communicative and discovery activities (Saab et al., in press; see Chapter 2).

Each chat utterance in the protocols was scored on one or more of the dimen-sions communicative activities, transformative discovery activities, and regulative discovery activities. The codes for the discovery activities collecting data for orien-tation as well as collecting data for hypothesis testing were not assigned to chat ac-tions, but to a SimQuest action, i.e. simulation running. A chat utterance was defined as a verbalization typed in the chat window (Lebie, Rhoades, & McGrath, 1996). Two independent raters rated 10% of the protocols, after both raters were trained in using the analysis scheme. Cohen’s Kappa of inter-rater reliability between the two raters was κ=.89 for the communicative dimension, κ=.83 for the transformative discovery dimension, and κ=.97 for the regulative discovery dimension, which can be considered as good agreement (Fleiss, 1981).

36 CHAPTER 3

Table 3. Scheme to analyze student’s actions and interactions


Discovery transformative activities

Discovery regulative activities

Orientation on environment

Identification of parameters and variables Collecting data Interpreting data and graphics

Generating hypothesis Describing and recognizing of relations Thinking of alternative answers Proposing an answer Formulating hypotheses Thinking of alternative hypotheses

Hypothesis testing Experimental design Predicting Collecting data

Informative Argumentative Evaluative Elicitative (asking for the others response) Responsive

Informative Confirmation/acceptance Negative response

Directive Asking for action Off task

Technical Coordinated Social


Orientation Planning Evaluation Monitoring

The analysis scheme for communicative actions is quite generic. In order to see the specific effects of the RIDE instruction, we need to zoom in on a subset of the communication dimension, namely those communicative actions that represent the activities that are promoted in RIDE. As mentioned in the introduction, instruction in the RIDE rules can lead to specific communicative activities. Table 4 shows the RIDE rules and the communicative activities we expect will be elicited by these rules. Respect for each other (R) is represented by a balanced amount of utterances between participants in a team, and little negative individual evaluation. Intelligent collaboration (I) is represented by informative responses, asking for understanding, argumentative, and informative activities. Deciding together (D) is represented by asking for action, confirmation/acceptance, asking for agreement, and coordinated off task talk. Encouraging (E) should lead to an increased occurrence of asking open questions, asking critical questions, asking after incomprehension, and positive indi-vidual evaluation. Table 3 shows only the main categories of the communicative activities. The communicative activities that we expect to be elicited by the RIDE rules, and are shown in Table 4, are, in a few cases, specified to subcategories of these main categories. The activity asking for open questions that is connected to the


rule Encouraging, for example, belongs to the main category elicitative activities (see Appendix A for a complete analysis scheme for communicative activities).

Table 4. Communicative activities that represent application of the RIDE rules

RIDE rules


(R) Respect

Less negative individual evaluation More symmetry in communication

(I) Intelligent collaboration Informative responses Asking for understanding Argumentative activities Informative activities

(D) Deciding together Asking for action Confirmation/acceptance Asking for agreement Coordinated off task talk

(E) Encouraging Asking open questions Asking critical questions Asking after incomprehension Positive individual evaluation

3 RESULTS

3.1 Communicative activities

To investigate whether the instruction leads to more effective communicative activi-ties, we compared the frequencies of communicative activities between the experi-mental and control group, using a Mann-Whitney U test with one-tailed testing to see possible significant differences between the two groups. We chose the Mann-Whitney U test, because of the skewness and the broad distribution of some of the variables.

From Table 5, it can be seen that four groups of communicative activities are significantly more frequently used in the experimental group than in the control group: elicitative, responsive, confirmative/acceptance activities, and asking for ac-tion.

38 CHAPTER 3

Table 5. Frequencies of communicative activities and the results of a Mann Whitney U test for differences

Experimental group

Control group

Mann Whitney U


M

SD

M

SD

U

z

p1

Informative

35.00

16.59

28.25

18.22

76.0

1.152

.132

Argumentative Evaluative

24.76 9.24

13.82 5.87

17.33 8.33

10.55 10.12

66.5 75.5

1.574 1.177

.059

.123 Elicitative 37.29 15.28 27.83 14.59 58.5 1.927 .027* Responsive Confirmation/acceptance Negative response

37.18 34.59 4.24

14.60 13.79 2.95

26.75 25.00 4.25

13.79 20.40 3.25

53.0 54.0 98.0

2.174 2.127 .179

.015*

.017*

.440 Directive 10.59 7.92 10.58 5.60 89.5 .556 .293 Asking for action 16.12 10.22 8.58 7.08 54.5 2.108 .017* Off task Technical Coordinated Social TOTAL

10.88 14.06 43.35 62.47

9.78 8.39 27.85

81.54

9.58

15.75 48.08

217.42

8.22 19.01 26.85

104.73

95.0 89.0 86.0 67.0

.312 .577 .709 1.550

.389 .293 .250 .064

1 One-tailed significance *p<0.05. **p<0.01.

We conducted a Mann Whitney U test with one-tailed testing to detect possible dif-ferences in frequencies of the RIDE rules activities between the experimental and the control group. As the Respect rule is represented by symmetry in communication and the lack of occurrence of a process, a collective test for this rule is not possible in this way. For this rule, the measures on the components are given. Asymmetry in communication is the difference in amount of utterances between the participants in one team, presented as a percentage of all utterances. Table 6 shows that the com-municative activities elicited by the rules Deciding together and Encouraging were used significantly more often in the experimental group.


Table 6. Frequencies of RIDE related communicative activities and the results of a Mann Whitney U-test for differences

Experimental group

Control group

Mann Whitney U

RIDE Rules

M

SD

M

SD

U

z

p1

(R) Respect for each other

Negative individual evaluation .59 .94 1.58 2.43 80.5 1.069 .174 Asymmetry in communication (%)

7.75 5.36 10.28 6.24 76.5 1.129 .132

(I) Intelligent collaboration 69.12 32.29 52.17 29.84 72.5 1.307 .098 (D) Deciding together 79.94 26.20 62.08 49.09 49.0 2.349 .009** (E) Encouraging 24.88 11.34 15.25 10.98 48.5 2.374 .008**


3.2 Discovery activities

Table 7 shows the frequency of the discovery learning activities (transformative and regulative) and the results of a Mann-Whitney U test between the experimental group and the control group. The transformative activities describing and recogniz-ing of relations, and concluding were used significantly more often by the experi-mental group than by the control group. Compared to the control group, the experi-mental group performed significantly more often evaluation actions. Also the total number of regulative activities was significantly higher for this group.

40 CHAPTER 3

Table 7. Frequencies of discovery learning activities and the results of a Mann Whitney U-test for differences

Experimental group

Control group

Mann Whitney U

Activities

M

SD

M

SD

U

z

P1

Discovery transformative learning

Orientation Identifying parameters and variables Collecting data Interpreting data and graphics

1.12

17.82 2.65

1.05 8.22 3.43

.92

14.58 1.33

1.24 5.50 2.60

87.5 83.5 70.5

.680 .821

1.506

.264 .210 .083

Hypothesis generating Describing and recognizing of relations Thinking of alternative answers Proposing an answer Formulating hypotheses Thinking of alternative hypotheses

7.00 2.18 5.12

21.53 3.00

5.29 2.92 3.12

10.78 3.04

3.08 1.83 4.75

15.50 3.67

2.64 1.95 3.86 6.71 2.39

54.0

102.0 84.0 73.0 84.5

2.138 0.000 .806

1.286 .787

.017* .500 .222 .106 .222


8.76 .65

20.88

8.33 .86

11.73

6.92 .25

22.58

7.43 .87

12.00

84.5 72.0 91.5

.779

1.691 .466

.222 .098 .324


4.12

.82 9.71

3.59 1.29 6.29

3.67 2.25 6.25

6.02 2.49 6.92

70.5 70.5 58.5

1.418 1.489 1.933

.083 .083 .027*

Total 105.35 35.25 87.58 35.04 72.0 1.329 .098 Discovery regulative learning

Orientation 3.88 3.25 3.25 3.36 91.0 .492 .324 Monitoring 50.41 24.64 38.58 26.92 71.0 1.373 .090 Planning 22.29 10.45 15.83 11.09 66.0 1.596 .059 Evaluation 4.94 3.54 3.58 5.28 60.5 1.871 .033* Total 81.53 41.79 61.25 39.77 60.0 1.860 .033*


3.3 Instruction and discovery activities

Table 8 shows the significant Spearman correlations that were found between the frequencies of the communicative activities associated with the Intelligent collabora-tion, the Deciding together, and the Encouraging rule (see Table 4) and the discov-ery learning activities for the experimental and the control group. Due to the reasons


given earlier in this section, such analysis is not possible for the Respect rule. The communicative activities coupled to the (R)IDE rules correlate with several discov-ery learning activities in both the experimental and control group, especially the rule Intelligent collaboration. The regulative discovery learning activities monitoring and planning have significant correlations with all the three rules.

3.4 Score within the learning environment

A one-way between-groups analysis of variance (ANOVA) was conducted to detect differences in group scores within the learning environment (SWLE) and number of assignments completed between the experimental group and the control group (Ta-ble 9). We did not find any significant differences between the groups.

3.5 Learning results on domain knowledge

For technical reasons (i.e. not everything was completely logged), the scores of six participants were not included in the analyses of the pre- and post-domain knowl-edge tests. The reliability of the pre- and post-domain knowledge tests was consid-erably low. Using a co-variance analysis, no significant differences were found be-tween the groups.

42 CHAPTER 3

Table 8. Significant Spearman correlations between the frequencies of RIDE rule communica-tive activities and discovery learning activities for the experimental and the control group

Intelligent collaboration

Deciding together

Encouraging

Activities

Ea

Cb

Ea

Cb

Ea

Cb

Discovery transformative learning

Orientation Identifying parameters

and variables Collecting data Interpreting data and

graphics

.534*

.581*

Hypothesis generating Describing and

recognizing of relations Thinking of alternative

answers Proposing an answer Formulating hypotheses Thinking of alternative

hypotheses

.780** .757** .661*

.901** .608* .708*

.662*

.604*


.496*

.504*

.607*

.859**

Conclusion Interpreting data and

graphics Rejecting hypotheses Concluding

.545* .854**

.734** .682* .655*

.485*

Discovery regulative learning Orientation Monitoring .523* .546* .583* .695* Planning .542* .742** .634** .578* .523* Evaluation

Note. Only significant correlations are shown. aExperimental group bControl group. *p<0.05. **p<0.01.


Table 9. ANOVA on SWLE and amount of questions answered between the experimental group and the control group

Experimental group

Control group

M

SD

M

SD

F

Df

p

SWLE

27.59

10.28

29.33

14.60

.142

1,27

.709

No. assignments completed

16.82 5.42 18.17 5.08 .454 1,27 .506

4 CONCLUSION AND DISCUSSION

In this chapter we present an attempt to support communicative processes in a collaborative discovery learning environment by introducing instruction before and during the learning process. The hypothesis was that this instruction would lead to more effective communication processes which, in turn, would lead to more productive discovery processes and better learning results. The instruction is centered on the RIDE rules: Respect, Intelligent collaboration, Deciding together, and Encouraging. Learners receiving the RIDE instruction used more communicative activities associated with the RIDE rules, especially those associated with Deciding together (D) and Encouraging (E). These learners asked more questions than the learners not receiving the RIDE instruction. Among the questions asked by the experimental group were requests for agreement, open questions, critical questions and questions after incomprehension. The experimental group also gave more informative answers, agreed more often, and asked their partner more often to perform an action in the learning environment. As mentioned by several researchers (Baker et al., 1999; King, 1997; Mercer, 1996; Van Boxtel et al., 2000; Veerman et al., 2000; Webb & Farivar 1994; Wegerif et al., 1999) these activities should contribute to more effective learning. This indicates that that these learners were working more collaboratively than the control group. Thus, the first part of the research question, whether the instruction leads to improved communication, can be answered positively.

The second research question, whether this improved communication leads to a more productive discovery process, requires a more complicated answer. We see an increase in a few transformative discovery processes (describing and recognizing of relations, and concluding) and an overall increase of regulative processes in the ex-perimental group, indicating that the improved communication resulting from the instruction leads to more regulation of the learning process.

Also another important effect in the data is found. We found a number of corre-lations between communicative activities associated to the RIDE rules with trans-

44 CHAPTER 3

formative and regulative discovery learning activities, especially for Intelligent col-laboration (I). These correlations are found for learners in both groups. The correla-tions indicate that there is a positive relationship between Intelligent collaboration, which includes informative responses, learners asking for understanding, argumenta-tive activities and informative activities, and the occurrence of productive discovery processes. In an earlier study of Saab et al. (in press) almost the same correlations were found between the communicative activities represented by the rule Intelligent collaboration and the discovery activities mentioned. Relations between the rules Deciding together and Encouraging on the one hand and the transformative discov-ery activity concluding and the regulative discovery activities monitoring and plan-ning on the other hand were found, too. The use of these activities was also induced by the instruction, since they were significantly more used in the group that received the instruction. However, while the instruction has the greatest effect on the D and E part of the RIDE rules, the I part seems to have the most influence on discovery processes.

We did not find significant differences between the experimental and the control group for results of working within the learning environment (the SWLE score). A possible explanation of this may be that the learners did not spend a sufficiently long time within the learning environment to realize to the full the potential of the learn-ing environment. Moreover, learning to apply the RIDE rules may have increased the load on the learners who received this instruction. The learners that received the RIDE rule were given prompts during the learning process. Although prompting rules can have a positive effect (cf. Howe & Tolmie, 1998), it takes time to read them and follow up the instruction, which can result in finishing fewer assignments in a fixed amount of time.

It can be concluded from this study that the RIDE instruction leads to more con-structive communication, and more and effective discovery learning activities, but not directly to better discovery learning results. The RIDE instruction supported the communication leading to a more productive discovery learning process.

This study explored the potential of providing instruction on communication in order to improve the performance in collaborative discovery learning environments. It was found that instructing learners in how to communicate effectively can result in improved communication and that this may give rise to better discovery learning, but still these effects on discovery processes and results are indirect and somewhat limited. Therefore, a possible next step to take is to design a learning environment in such a way that the beneficial communicative activities are elicited by the communi-cation instruction, for example, by letting the learners practice more often with the RIDE rules, as well as by cognitive tools that elicit communicative actions in the specific discovery learning context.

CHAPTER 4

SUPPORTING COLLABORATIVE SCIENTIFIC DISCOVERY LEARNING, A TOOL AND ITS

DIFFICULTIES∗

In this study, a Collaborative Hypothesis Tool (CHT) is introduced to guide learners engaged in a col-laborative scientific discovery learning task. The tool offered a scratchpad that assisted in the construction of relations, as well as hints on the hypothesis generation process, the hypothesis testing process, and the conclusion process. In an experimental study the effects of offering the tool were determined by contrast-ing groups with or without the tool. No effects on learning product or learning process were found due to little usage of the tool by dyads in the experimental group. Analysis of the cases in which the tool was used shows that the CHT can influence the use of communicative and discovery activities.

1 INTRODUCTION

In collaborative scientific discovery learning, learners communicate and work together in a shared environment gathering data and using these data for joint knowledge construction. By altering variables and parameters and observing the effects, learners can uncover the rules that hold in a phenomenon they investigate, and in doing so, build new knowledge. Computer simulations are often used as safe and easily accessible phenomena to investigate (De Jong & Van Joolingen, 1998; Njoo, 1994; Njoo & De Jong, 1993). When they investigate a phenomenon, learners need to employ learning processes that resemble the inquiry cycle (De Groot, 1969). Njoo & De Jong (1993) distinguish transformative learning processes, such as orientation, hypothesis generation, experimentation and concluding, that contribute to the generation of new knowledge and regulative processes, such as planning and monitoring. A vast body of research has shown that learners need support in the successful employment of these learning processes. A review of this research can be found in De Jong and Van Joolingen (1998).

∗ Saab, N., Van Joolingen, W. R., Van Hout-Wolters, B. H. A. M. (submitted). Supporting collaborative scientific discovery learning, a tool and its difficulties.

46 CHAPTER 4

In a collaborative setting, such support can arise naturally from the environment. As scientists benefit from collaboration and discussion with colleagues (Dunbar, 2000), we may also expect learners engaged in a scientific task to profit from interaction with peers (Salomon & Globerson, 1989). Okada and Simon (1997) found such a beneficial effect when comparing students working in pairs with students working alone on a discovery task in the domain of molecular genetics. Compared to singles, pairs were more effective in discovering rules, because they were using more ex-planatory activities, such as generating alternative ideas and generating hypotheses. Also Whitelock, Scanlon, Taylor, and O’Shea (1995) found a beneficial effect of collaboration on scientific discovery learning. Working with a simulation environ-ment about collisions, they found that dyads of learners were more successful com-pared to single learners.

In the study presented in Chapter 2 (Saab, Van Joolingen, & Van Hout-Wolters, in press), we found that communicative activities can contribute to essential stages in a collaborative discovery process. For example, directive activities can contribute to the testing of hypotheses, whereas argumentation can lead to a successful process of concluding. This leads to the conjecture that when learners are encouraged to use these communicative activities, a more successful discovery process can be the re-sult. But, even though the collaborative environment can contribute to the discovery process, specific process support remains necessary.

In the current study, we are interested in providing such support. Based on the concept of cognitive tools (Lajoie, 1993; Van Joolingen, 1999), we will investigate the role of scaffolds for discovery processes in a collaborative environment.

In general, cognitive tools support learners to carry out cognitive tasks. The tools do not deliver direct instruction, but instead offer scaffolds that may help learners to learn from working with them (Jonassen, 2000; Salomon, 1993). According to Lajoie (1993), cognitive tools can serve several scaffolding functions: • The support of cognitive processes, such as memory and metacognitive proc-

esses; • To share the cognitive load by providing support for lower-level cognitive skills

so that resources are left over for higher-order thinking skills; • To allow the learners to engage in cognitive activities that would be out of their

reach otherwise; • To allow learners to generate and test hypotheses in the context of problem

solving. (Lajoie, 1993, p.261). There is a wide range of examples of cognitive tools, virtually for all processes of discovery learning, ranging from tools to support monitoring (Veermans, De Jong, & Van Joolingen, 2000; Veermans, Van Joolingen, & De Jong, submitted), to state hypotheses (Shute & Glaser, 1990; Van Joolingen & De Jong, 1991, 1993) and or-ganize empirical evidence (Lajoie, Lavigne, Guerrera, & Munsie, 2001; Reimann, 1991).

The tool we used in our study is based on the hypothesis scratchpad, that was first used by Van Joolingen and De Jong (1991, 1993), as well as by Gijlers (2005). Stating hypotheses is a recognized difficult discovery process, as learners have prob-

A TOOL AND ITS DIFFICULTIES 47

lems to state syntactically correct hypotheses (Njoo & De Jong, 1993) and to state hypotheses that are testable (Van Joolingen & De Jong, 1997). The hypothesis scratchpad offers a template that helps learners in stating hypotheses, ensuring that they are syntactically correct. The tool is adapted in two ways. The first was to em-bed the tool into the discovery process as a whole, by adding prompts that reminded learners of actions to undertake, such as stating a hypothesis or to gather experimen-tal data. The second adaptation was towards using the tool in a collaborative envi-ronment, by adding specific opportunities for argumentation about the hypotheses stated and whether they should be tested or not. The tool is shared by two learners and learners can make their individual opinion visible about the hypotheses stated on the scratchpad. Meanwhile they have to reach agreement and formulate a common answer to assignments given in the learning environment. This setup is intended to serve as a trigger for argumentation between collaborating learners. We label this new tool version the Collaborative Hypothesis Tool (CHT). A detailed description will be given in the method section below.

It is hypothesized that the tool will contribute to learning in the collaborative sci-entific discovery environment. Offering the hypothesis template and prompting learners to perform the discovery learning process should lead to well-stated hy-potheses, as well as to a higher quality discovery process, expressed in more trans-formative discovery activities. Asking learners to make their opinions on hypotheses explicit should lead to more argumentative communication processes between learn-ers. In total this should lead to improved learner outcomes.

In the experiment presented below, we compared a group of dyads of learners who worked in a collaborative discovery environment with the CHT, with a group that worked in the same environment but without the CHT. Learning outcomes are measured with pre- and posttests as well as with the result of working in the envi-ronment. Learning processes were measured by analyzing logs of both learning ac-tivities and chat between learners. We focused on communicative activities, discov-ery activities, and, more in particular, on use of the CHT.

2 METHOD


Research participants were 32 dyads of tenth-grade students (15-16 years old) of six secondary schools who were enrolled in pre-university education and who all had physics as a topic. The mean age was 15.6 years (SD=0.85). The participants volun-teered to participate in the study and were paid €20. The students were randomly divided in two groups; one experimental group that used a collaborative environ-ment that included the Collaborative Hypothesis Tool (CHT) and one control group using the same environment but without the CHT. Because of technical reasons (i.e. chat not completely logged or problems with the connection between the two com-puters) and one chat log with utterances in a language other than Dutch, we had to exclude seven dyads from the data analysis, which resulted in the group with the added cognitive tool containing 15 dyads and the instruction-only group containing 10 dyads of students.

48 CHAPTER 4

Figure 1 Screenshot of the learning environment used. Shown are the simulation window, assignment windows, and chat window.


All students worked with a learning environment called Collisions, developed in SimQuest (Van Joolingen & De Jong 2003)5. Collisions consists of a computer simulation of colliding particles, embedded in an environment offering instructional support (Figure 1). The main learning task is to discover the main laws underlying the domain: the conservation laws of momentum and energy, as well as their corol-laries for one-dimensional collisions. Learners can manipulate the simulation, the results of which are shown in an animation and graphs. The version of Collisions that was used contains three levels of model progression, increasing in difficulty: Uniform motion, Fixed wall, and Elastic Collisions. The first two levels introduce the basic concepts of motion and collision, whereas in the third level, the real colli-sions come into play. Each level opened with a window showing the main learning goals for that level. Furthermore, assignments were presented on each level, to direct the focus to specific sub questions within the level. Assignments presented the learn-ers with a small research question, and were presented in the form of an open ques-tion or a multiple-choice question. In total there were 35 assignments, of which sev-enteen open questions. Open questions were used to orientate on the level, multiple

5 Collisions was developed by Hans Kingma and Koen Veermans (University of Twente). SimQuest was developed in the SERVIVE project coordinated by the University of Twente.


choice questions could be used by learners to test their own knowledge. Learners were free in choosing whether or not to use assignments and were also free in the order to do them, although the naming of assignments suggested an order, as as-signment names began with a number. Assignments stimulated learners to collect data that could help them find an answer. Collisions also contains textual explana-tions for each of the variables present in the simulation, which could be opened at any time.

Table 1. RIDE Rules, Sub-rules, and communicative activities

RIDE rules

(R) Respect

(I) Intelligent

collaboration

(D) Deciding together

(E) Encouraging

Sub-rules

Everyone will have a chance to talk Everyone’s ideas will be thoroughly considered

Sharing all relevant information and suggestions Clarify the information given Explain the answers given Give criticisms

Explicit and joint agreement will precede decisions and actions Accepting that the group (rather than an individual member) is responsible for decisions and actions

Ask for explanations Ask till you understand Give positive feedback


Less negative individual evaluation More symmetry in communica-tion

Informative responses Asking for understanding Argumentative activities Informative activities

Asking for action Confirmation /acceptance Asking for agreement Coordinated off task talk

Asking open questions Asking critical questions Asking after incomprehension Positive individ-ual evaluation

2.3 Support for collaboration: the RIDE rules

Dyads of students worked collaboratively on two computers with a shared interface, communicating through a chat channel. Both learners saw the same state of the learning environment and could take turns in performing actions within it.

As part of the procedure, all students were instructed in effective communica-tion. This instruction was based on a set of rules we call the RIDE rules (see Chapter 3), which stands for Respect, Intelligent collaboration, Deciding together, and En-couraging. Table 1 shows the RIDE rules with their sub-rules and communicative activities that represent these rules. An earlier study (Saab, Van Joolingen, & Van Hout-Wolters, submitted-a; Chapter 3) showed that instructing these rules can lead to more effective communication. The RIDE instruction consists of a recorded pres-entation, which was given before a practice session with Simquest. During the prac-tice, students also practiced with these rules. During working with Collisions, each time the learners opened an assignment, a prompt reminded them of one of the rules.

50 CHAPTER 4

2.4 The collaborative hypothesis tool

Learners in the experimental group received a version of Collisions that contained the CHT as an extra means of instructional support. The CHT offers learners a tem-plate (see Figure 2) that is designed to facilitate the users in stating hypotheses. By selecting variables and a relation, learners construct an expression that yields a test-able hypothesis, guaranteeing that the resulting hypothesis is a testable statement. Moreover, by letting students indicate whether a hypothesis needs to be tested, the hypothesis scratchpad allows for planning. In addition, learners could indicate their trust in a hypothesis on a scale from 0 to 100%. Learners individually indicate their trust and opinions about testing, but have to use the results to formulate an answer to an assignment together. This setup is intended to support collaboration, in particular argumentation.

The CHT also provides prompting windows with instructions on how to use the collaborative scratchpad together (the Hypothesis window), how to plan the way to the answer (the Planning window), and a reminder to check whether the answer confirmed or rejected the hypotheses in the scratchpad (the Conclusion window). Each of these windows provided hints for a specific stage of the discovery process. The Hypothesis window suggested filling in the template and indicating trust. The Planning window suggested thinking of which and how many experiments to do and, finally, the Conclusion window indicated that the answer given should be consistent with the activities recorded on the hypothesis scratchpad.

The CHT popped up whenever one of the seventeen open-answer assignments was opened. The collaborative hypothesis scratchpad was visible the whole time the assignment was open. The prompting windows were visible at turns, starting with the Hypothesis window. In order to prevent an overload of prompts for learners in the experimental condition, the amount of RIDE prompts was lowered in this ver-sion of Collisions.

Figure 2. Collaborative hypothesis scratchpad.



In this experiment, two types of learning outcome are identified. One is associated with the performance within the learning environment; the other is a measure of what is learned from this performance. For the performance within the learning envi-ronment the answers given to the assignments are assessed. Three points were awarded for a good answer on each assignment. For the multiple-choice assignments this was all or nothing, whereas for open-answer assignments scores of one or two were possible. Three points were awarded for a correct answer that was supported by an argument. We label this score SWLE, score within learning environment. Given 35 assignments, the maximum SWLE score was 105 points. As learners were free to choose from the set of assignments, we computed an absolute and a relative SWLE, consisting of the total number of points scored and the percentage of points scored on assignments that were actually opened respectively.

For measuring what is learned from working in the environment a domain knowledge pre- and the posttest were used. Two consisting tests were used devel-oped for this domain by Swaak (1998), an Explicit Knowledge Test, which tests the learners for declarative knowledge, such as facts and formulas, and a WHAT-IF Test (Swaak, 1998; Veermans, De Jong, & Van Joolingen, 2000). The WHAT-IF Test shows the learners a situation before a collision, presents them with a change in the situation (for example, a collision against a fixed wall) and asks the learners to pre-dict an effect of this change. Both tests were administered on-screen. The pretest and posttest were both parallel versions of the tests, meaning that the same types of items were presented in the same order. The Explicit Knowledge Test consisted of 15 multiple-choice items and the WHAT-IF Test consisted of 13 multiple-choice items. It should be noted that the domain knowledge tests were taken on individual subjects, whereas the SWLE is a score for the dyad as a whole.

2.6 Measuring learning process

All communicative and discovery learning activities were logged and were put to-gether in a single protocol for each dyad. A three-dimensional analysis scheme (Ta-ble 2) was used to analyze the protocols. The scheme has been developed and used in previous studies (Saab et al., in press; Saab et al., submitted-a). The dimensions are: a) communicative activities, b) discovery transformative learning activities, which promote the generation of information (Njoo & De Jong, 1993), and c) dis-covery regulative learning activities, which support and guide the learning process (Njoo & De Jong, 1993).

52 CHAPTER 4

Table 2. Scheme to analyze students' actions and interactions





Identification of parameters and vari-ables Collecting data Interpreting data and graphics









In the protocols, each chat utterance was scored on all three dimensions. The SimQuest action simulation running was coded only as a transformative discovery activity, collecting data. The computer program MEPA (Multiple Episode Protocol Analysis) was used for analyzing the protocols (Erkens, 1998). Two independent raters rated 10% of the protocols, after both raters were trained in using the analysis scheme. Cohen’s kappa of inter-rater reliability between the two raters was .94 for the communicative dimension, .89 for the transformative discovery dimension, and .95 for the regulative discovery dimension, which can be considered as good agreement (Fleiss, 1981). For the communicative actions we computed the totals for actions associated with three of the RIDE rules, viz., the actions associated with Intelligent collaboration, Deciding together and Encouraging, as specified in Table 1 (section 2.3).

Besides the analyses of the protocols, the use of the Collaborative Hypothesis Tool was investigated for the group that was presented with this tool. The use of the CHT is measured by the following variables, taken from the log files:


• The total amount of time (s) using the CHT; • The number of hypotheses generated within the tool with each question where

the CHT was presented to the learners; • The total number of hypotheses generated while using the CHT; • The proportion of planning, defined as the percentage of CHT uses in which

learners checked one or more planning actions. For each open-answer assign-ment, entering a hypothesis on the CHT counted as one CHT use;

• The proportion of checking, defined as the percentage of CHT uses where learners checked whether the answer given is the same as one of the hypotheses constructed with the CHT;

• The proportion correctly answered questions after using the CHT.

2.7 Procedure

The experiment took place in two sessions. In the first session, dyads were com-posed and assigned to the experimental group, the RIDE instruction was given and learners practiced with a Simquest practice environment on simple logical problems. The second session started with a pretest, then 90 minutes of interaction with Colli-sions, and concluded with the administration of the posttest.

Dyads were made up heterogeneously (Blatchford, Kutnick, Baines, & Galton, 2003), based on students average school grades. The reason is that brighter students can learn from giving elaborated explanations (Webb & Farivar, 1994), and weaker students learn from the explanation given (Van der Linden, Erkens, Schmidt, & Renshaw, 2000). However, the difference in levels between students should not be too large. All students received a token: a colored triangle or circle. The color stood for grade (red was either high or low graded, yellow was middle grade), and the shape stood for condition. Subjects were not informed about this meaning. Subjects were free to choose a partner who had a token with the same shape but different color. This assured that the dyads were composed of either a low graded student and a middle graded student or a middle graded student and a high graded student and that, to increase their motivation, students could choose someone who they liked to work with.

3 RESULTS

First, the differences in learning process and results between the two groups are ana-lyzed, then we analyze the use of the CHT, after that the analysis of the relation be-tween CHT use and the learning process and learning product is presented, and fi-nally two fragments of protocols are analyzed.

3.1 Learning processes and results

A Mann Whitney U test with two-tailed testing on the frequencies of communicative activities found was conducted. No significant differences were found. In both groups, the informative, elicitative and confirmation/acceptance communicative

54 CHAPTER 4

activities were the most frequently used. Also no differences between groups were found on the frequencies of use of discovery activities.

A one-way between-groups analysis of variance was conducted to detect differ-ences in SWLE, both absolute and relative (See Section 2.5). The average absolute SWLE was 19.06, whereas the average relative SWLE was 58.59%. No significant differences between groups were found.

The internal consistency was measured with Cronbach’s alpha. The reliability of the domain knowledge tests was low, ranging from .32 to .45. Furthermore, no sig-nificant differences between groups were found on these tests.

Table 3. Frequencies of the use of the CHT

Assignment Number

M

SD

A4 a

1.93

1.58

A5 a 0.73 0.70 A6 a 0.53 0.83 A7 a 0.13 0.35 B6 a 0.07 0.26 B7 a 0.07 0.26 B8 a 0.07 0.26 B9 a 0.07 0.26

3.2 Use of the Collaborative Hypothesis Tool

Table 3 shows that the CHT was rarely used, and mainly in the beginning of work-ing with the environment (all learners performed the assignments in numerical or-der). Only in eight out of seventeen possible cases where the CHT popped up it was actually used, only by a few learners. As learners progress in the environment, CHT use dies out. The average amount of time using the CHT is 249 seconds. The mean amount of hypotheses generated in the CHT is 3.6. The mean proportion of planning is 24%, and the mean proportion of checking is 45%. The mean proportion of cor-rectly answered assignments after using CHT is 44 %. The relation between CHT use with communicative and discovery activities was investigated using Spearman correlation analysis (Table 4). Significant positive cor-relations were found between the total number of hypotheses generated in CHT and the total number of communicative activities (r=.58; p<.05). Also, significant corre-lations were found between planning in the CHT and Deciding together activities (r=.53; p<.05), and total number of regulative activities (r=.54; p<.05). Moreover, significant positive correlations were found between the proportion correctly an-swered questions after using CHT and Deciding together activities (r=.68; p<.01),


total number of transformative activities (r=.58; p<.05), and total number of regula-tive activities (r=.61; p<.05).

Table 4. Correlations between measures of CHT use and communicative and discovery activities

Total amount of time using

CHT

Total number of hypotheses generated in CHT

% plan-ning while using CHT

% checking

while using CHT

% correctly answered

assignments after using

CHT


(I) Intelligent collaboration -.10 .40 .47 .41 .51 (D) Deciding together -.23 .36 .53* .14 .68** (E) Encouraging -.23 .27 .29 -.13 .30 Total -.08 .58* .40 .21 .51 Discovery activities Transformative total -.19 .42 .44 .18 .58* Regulative total

-.23 .35 .54* .14 .61*

*p<0.05. **p<0.01. To relate learning process with learning outcome, we computed Spearman correla-tions analyses between, on one side, the total number of communicative activities, transformative activities and regulative activities and, on the other side, SWLE. For communication, we also computed separate correlations for actions related to the latter three RIDE rules. We computed these correlations separately for both groups and computed Fisher’s Z’ scores to compare correlations between groups. The re-sults are shown in Table 5. There are several significant positive correlations for the experimental group. Deciding together activities (r=.62; p<.05), transformative ac-tivities (r=.55; p<.05), and regulative activities (r=.53; p<.05) correlate significantly with SWLE. No significant correlations are found between SWLE and the frequen-cies of activities used in the control group. Significant differences in correlation be-tween both groups were found, with significantly greater correlation coefficients for the experimental group for Deciding together activities (p<.05) and total number of regulative activities (p<.05).

56 CHAPTER 4

Table 5. Correlations between frequencies of communicative activities and transformative and regulative discovery activities and learning results within the learning environment

(SWLE) for both experimental and control group, as well as a Fisher’s z-test on the differences between these correlations

SWLE

Experimental group

Control group

Za

p


(I) Intelligent collaboration .39 -.13 1.29 .20 (D) Deciding together .62* -.18 2.10 .04* (E) Encouraging .19 -.25 1.05 .29 Total .40 -.14 1.32 .19 Discovery activities Transformative total .55* -.07 1.61 .11 Regulative total .53* -.39 2.35 .02*

aZ scores computed for the differences of Fisher’s Z’ scores for both sets of correlations. *p<0.05. **p<0.01.

3.2.1 Two Examples of CHT use

In the following section, two fragments of protocol are presented to provide more insight in how the dyads worked with the CHT. These protocols were selected from the experimental group. In the first protocol the dyads worked with the CHT and scored high on SWLE. In the second protocol the dyads worked with the CHT and scored rather low on SWLE.


Protocol 1: Cedric and Andrew; working collaboratively 191. Cedric: I think the smaller the mass, the smaller the momentum 192. Andrew: the bigger the mass, the smaller the momentum 193. SIMULATION 194. Cedric: No, it increases 195. Andrew: I think 196. Cedric: It doesn’t matter, minus, just as with the velocity 197. Cedric: I think 198. HYPOTHESIS “If v increases quickly, then distance decreases” 199. HYPOTHESIS removed 200. Andrew: then your hypothesis is right 201. Cedric: ;) 202. Cedric: what is the symbol for momentum? 203. Andrew: Wait, I’ll check it out 204. HYPOTHESIS “If m increases, then p decreases” 205. Andrew: ready 206. Cedric: Are you sure that it is p? 207. Cedric: p=pressure 208. Andrew: yes, smarty 209. Andrew: It is written up there 210. Andrew: momentum (p) 211. Cedric: It doesn’t decrease 212. Cedric: It increases 213. CHANGE HYPOTHESIS “If m increases, then p increases” 214. Cedric: write it there 215. Andrew: Iz cool In the first protocol, the learners started with stating two different hypotheses. It seems like Cedric does an experiment (turn 193), as Andrew adds in turn 195 “I think” to his earlier stated hypothesis. Cedric then interprets the graph and explains that the minus does not influence their line of thinking. One of them fills in a hy-pothesis in the CHT, but removes it right away. Andrew confirms that Cedrics’ hy-pothesis is right. The next step for them is to fill in the hypothesis with abstract terms. Andrew finds the symbol for momentum and has to convince Cedric, which he does by indicating where he found the explanation about momentum. Cedric no-tices that the hypothesis Andrew filled in is not completely correct, and he tells An-drew what should be different before changing the hypothesis in the CHT. Andrew is going to write the answer in the assignment window and Cedric encourages him to do so.

The learners in this dyad are both working with the learning environment. They gather data by doing experiments, interpret these data and fill in hypotheses in the CHT. They conclude with giving an answer to the assignment. They are sharing ideas, explaining and evaluating their own and each others ideas.

58 CHAPTER 4

Protocol 2: Peter and Omar; Peter gets desperate 151. Peter: Why did you use a hypothesis? 152. Omar: First read the storyyyyyyyyyyyyyyyyyyyyyyyyyyy 153. Peter: Why are you yelling 154. Omar: can you hear me than? 155. Peter: I don’t get this table 156. HYPOTHESIS “If slope decreases, then v is steady” 157. Peter: What are you doing???? 158. Peter: ??????? 159. Omar: Just say: if slope increases, then v is steady 160. Peter: Now I get it 161. Peter: now what? 162. Omar: Wait 163. Peter: What are you doing? 164. SIMULATION 165. Peter: With this hypothesis I agree 166. SIMULATION 167. Peter: What are you doing now? 168. Peter: I wonder 169. SIMULATION 170. SIMULATION 171. Peter: You make me crazy 172. Peter: What are you doing????????????? 173. Omar: Next question 174. Peter: did you answer assignment 4 correctly? 175. Omar: Next 176. Omar: Yeeeees 177. Peter: did you really? 178. Omar: YES The second protocol is an example of a learning situation in which learners are not communicating or collaborating. One learner, Omar, is in control of the learning environment and stays in control. Peter is trying to start a discussion, but Omar re-acts with irritation, or does not react at all. In turn 151, Peter wants to know why Omar is filling in a hypothesis in the CHT. Omar does not respond to his question and tells him to read the instruction. Peter does not like the way Omar communi-cates with him, but Omar does not care. Peter (turn 155) gives a signal that he does not understand the table (simulation window). Omar just goes on working in the environment; he first fills in a hypothesis in the CHT (turn 156) and checks this hy-pothesis by doing several simulations (turn 164, 166, 169, and 170) and does not pay attention to Peter. Peter is persistent in trying to get into contact with Omar. When he sees the hypothesis Omar has filled in the CHT, Peter states that he agrees (turn 165), as a first step in reaching agreement and common ground. Omar just goes on doing simulations, while Peter gets more and more desperate. Finally, Omar com-municates with Peter again by saying that they have to go to the next assignment


(turn 173). Peter, in a last attempt to solve this problem together, asks if Omar an-swered the assignment correctly. Omar responds annoyed that he did.

In summary, Omar was completely self-focused and showed that he was not in-terested in collaborating in finishing the assignment. He was working with the envi-ronment and used the tools, but alone. Peter had a different expectation about work-ing collaboratively. He tried to get in touch with Omar, and let Omar know what he thinks. At the end, he gave up and asked if Omar at least has filled in the answer correctly. This dyad did not work collaboratively and was not successful in perform-ing in the learning environment. Omar really did not want to work collaboratively. If Peter would have had another partner that was trying to seek contact with him the same way Peter has done this time, more effective collaboration could have oc-curred.

In both examples the CHT is used. In the first protocol, Cedric and Andrew use the CHT to fill in their ideas. When they see the hypothesis in the CHT, they check in their newly gathered data whether the hypothesis is correct. The filled in hypothe-sis is also used as a drive to talk about the data in the graph. This dyad uses the CHT for what it is made for: as a promoter of generating new ideas and interpreting data together. The way Cedric and Andrew work is completely in contrast with the way Peter and Omar are working. Peter and Omar are not using the CHT collaboratively at all. Omar considers filling in a hypothesis in the CHT as part of the task, which he does alone. Peter tries to react on the hypothesis stated by Omar on the CHT (which is a desirable action), but Omar does not respond.

4 CONCLUSION AND DISCUSSION

In this study, a Collaborative Hypothesis Tool (CHT) was introduced, meant to sup-port the collaborative learning process of learners working in a computer-supported environment. Our hypothesis that this tool would result in more argumentation, bet-ter discovery learning processes as well as improved learning results could not be confirmed. No significant differences between groups on any of our output measures were found.

The main cause for this lack of success must be sought in the scarce use made of the CHT by students in the experimental group. Students were free in choosing whether to use the tool or not and, after some initial use, all students in the experi-mental group refrained from using the tool. Effectively this annihilated our experi-mental intervention to a great extent.

However, when looking more closely at the rare cases the tool was used it turned out that it can have some effect. Correlational analyses shows that students who stated more hypotheses on the CHT communicated more, and that students who used the planning boxes more also decided more together and scored higher on regu-lative processes. This indicates that the purpose of the planning indicator served its goal. For the assignments for which learners used the CHT, Deciding together and regulation were positively correlated with the performance on these assignments. A hypothesis that can be formulated, based on these findings, is that the CHT did in-

60 CHAPTER 4

duce effective communicative processes for those learners that used it, and that this yielded better performance on the assignments.

This hypothesis is supported by the separate correlational analysis of process measures with SWLE for the two groups. Learners in the experimental group showed significantly higher correlations between process measures and SWLE. So even though the tool was not used often, its presence seems to have some influence on the way learners employ learning processes. Results seem to indicate that these activities, especially communicative activities connected to the Deciding together rule and regulative activities, were used more effectively by the experimental group, so when they decided more together or when they regulated their learning process more, this led to a higher score on SWLE. One must remind that even though stu-dents may not use the tool, each time they opened an assignment they saw the tool as well as the accompanying prompts.

From the analysis of protocols it is found that learners can be able to use the CHT to generate new ideas together. However, they did not always do so. Motiva-tion can play a role in this lack of working together. In one of the cited protocols, it became clear that one of the learners did not collaborate for such lack of motivation.

A likely explanation for the little use that was made of the CHT in the experimental group is that learners themselves did not see obvious benefit in using it. One cause may be that use of the CHT costs time and resources. Also in a study by Lazonder, Wilhelm, and Ootes (2003) where learners could choose to work with the presented tool or not, most of the time they did not. Apparently, learners are inclined to use a tool only when they see direct benefit or when there is pressure to use it. The prompts that were used to stimulate use did not have the desired effect.

Combining the finding that the tool was little used and the small indications that its use may have some benefit, we must seek for situations where a new version of the tool will more likely be used. There are two candidate options for trying to achieve this. The first option is to place the scratchpad more prominent in the basic working procedure of the students, effectively making its use mandatory. For in-stance, instead of offering the scratchpad as support for trying to answer assign-ments, learners could be required to state their answers through the hypothesis scratchpad. That is, the correct answer should be in the form of a statement entered on the hypothesis scratchpad, together with an indication from both learners that they accept that one as the answer. In doing so, the attention will be more drawn towards the functionality of the hypothesis tool, and the role of the tool in the work-ing process will be much clearer.

The second option is to provide learners with instruction and an opportunity to practice with the tool before working with the learning environment. This will in-crease their knowledge of how and when to use the tool effectively. This may be effective as in a study by Gijlers and De Jong (submitted), who used a very similar tool and compared it to providing lists of predefined hypotheses, it was found that the scratchpad was difficult to use, yielding inferior results when compared to using the given proposition list.

Future research should investigate the possible results of applying these options, which, of course, can also be combined.

CHAPTER 5

MOTIVATION AND COLLABORATIVE SCIENTIFIC DISCOVERY LEARNING∗

In this study, we investigate the relation between motivation and the individual collaborative scientific discovery learning process and learning product, and whether team heterogeneity in terms of motivation can influence the collective collaborative scientific discovery learning process and learning product. 73 dyads of learners (15 to 17 years) worked together in a collaborative discovery learning environment, sharing the interface and communicating by means of a chat channel. The results show that motivation is related to individual learning rather than to the learning at group level, except for task value beliefs. Fur-thermore, it appeared that motivation was not positively related to learning results. This can be a result of the nature of the learning product measures used.

1 INTRODUCTION

Apart from cognitive factors, learning is also influenced by learners’ motivation. Motivation can be considered to be a domain-specific and situation-bound concept, which can be measured by asking students about their beliefs concerning their own motivation and behavior as well as by observing their behavior (Linnenbrink & Pintrich, 2002; Zusho, Pintrich, & Coppola, 2003). In this chapter, the role of learners’ motivation in relation to the learning process and learning product in collaborative scientific discovery learning environments will be addressed. We base ourselves on an existing inventory for assessing motivation, on observed behavior for measuring the learning process, and on measured learning outcomes of a collaborative scientific discovery learning task.

1.1 Motivation and collaborative scientific discovery learning

In collaborative scientific discovery learning, learners explore a learning environment, by doing experiments, collecting data, and drawing conclusions.

∗ Saab, N., Van Joolingen, W. R., & Van Hout-Wolters, B. H. A. M. (submitted). Motivation and collaborative scientific discovery learning.

62 CHAPTER 5

Collaborative discovery learning requires that learners engage in the processes of scientific inquiry (De Jong & Van Joolingen, 1998; Van Joolingen, De Jong, Lazonder, Savelsbergh, & Manlove, 2005). By working in dyads or small groups with a common learning goal, learners need to externalize ideas (Van Boxtel, 2000; Van der Linden, Erkens, Schmidt, & Renshaw, 2000), negotiate with one another, and construct new shared knowledge (Chan, Burtis, & Bereiter, 1997; Wegerif, 1996; Wegerif & Mercer, 1997). Therefore, collaboration can serve as support for the performance of the processes of scientific discovery learning (Saab, Van Joolingen, & Van Hout-Wolters, in press; see Chapter 2). Saab, Van Joolingen, & Van Hout-Wolters (submitted-a; Chapter 3) introduced four communication “rules” with a positive contribution to the collaborative discovery learning process: Respect, Intelligent collaboration, Deciding together, and Encouraging. These rules group effective communicative activities, such as asking for action or argumentative activities. Learners who were instructed to use these rules showed more effective communicative activities and processes of scientific discovery learning, such as drawing conclusions and regulative activities, than learners who were not instructed to do so.

In complex learning environments, motivation is a crucial factor. An unmotivated learner will not engage deeply in complex learning processes. This will be true for individual learners as well as for learners working together in groups. This means that learners have to be motivated to regulate both the discovery and the collaborative process. Jones and Issroff (2005) stress the importance of social affinity between learners working in technology enhanced collaborative environments. Bandura (2001) hypothesized that the more learning groups see themselves as highly efficacious, the better their collective performance outcome will be. In this hypothesis, Bandura considers self-efficacy to be a motivational variable at group level instead of individual level, in which he assumes that all members of the group have the same high level of self-efficacy. However, lowly-motivated and highly-motivated learners can be part of one single group. In the current study, levels of motivation of participants in dyads will be investigated in order to explore the relation between heterogeneous or homogenous group composition, and collective and individual learning process and learning product.

1.2 Expectancy-value model

Motivation, as seen from a social cognitive perspective, is domain-specific and lin-ked to a learning situation (Linnenbrink & Pintrich, 2002). This means that learners can be motivated in one discipline or situation, but not necessarily in another. For instance, in one situation a learner can find it very important to show to his friends how high his grades are, while in another situation the same learner is interested in mastering the subject. Most research see motivation as a combination of several components (Linnenbrink & Pintrich, 2002; Pintrich, 2000; Ryan & Deci, 2000). The constructs of motivation on which this study will be focused are based on a ge-neral expectancy-value model of motivation (Pintrich, 2000; Pintrich & De Groot, 1990; Pintrich, Smith, Garcia, & McKeachie, 1993; Wigfield, 1994; Wigfield &

MOTIVATION AND COLLABORATIVE DISCOVERY LEARNING 63

Eccles, 2000). This model distinguishes the following components in motivation: a) value, b) expectancy, and c) affect. It presumes that these components relate to the learning process and performance outcomes (Pintrich & De Groot, 1990). Eccles and Wigfield (2002) reviewed theories linked to expectancy-value models. In the current study, the motivation scales of the ‘Motivated Strategies for Learning Ques-tionnaire’ (MSLQ) (Pintrich & De Groot, 1990; Pintrich et al., 1993) will be used as a measure for motivation.

Value involves students’ goals and beliefs about a task and what aspects of the task are interesting and important to them (Pintrich & De Groot, 1990). In the MSLQ (Pintrich & De Groot, 1990; Pintrich et al., 1993), three value components are distinguished: intrinsic goal orientation, extrinsic goal orientation, and task value beliefs. Learners who are intrinsically goal-oriented focus on learning and mastering. Extrinsically goal-oriented learners give importance to the approval of others, such as receiving good grades. Learners with a high level of task value beliefs are interested in the task presented to them and see those tasks as useful and important. Intrinsic goal orientation and task value beliefs have been found to be related to self-regulation (the use of deep information processing strategies) (Bruinsma, 2004; Covington, 2000; Greene & Miller, 1996), and school achievement (Pintrich & De Groot, 1990, Pintrich et al., 1993). Other theories, such as achievement goal theory, conceptualize these components differently. Achievement goal theory makes a distinction between learning goals (also called mastery or task goals), which can be compared to intrinsic goal orientation, and performance goals, (e.g., Covington, 2000; Dweck & Leggett, 1988; Elliot, 1999; Greene & Miller, 1996; Linnenbrink & Pintrich, 2002), which resembles extrinsic goal orientation. Learning goals represent the interest in learning and the mastering of new skills, whereas performance goals embody the interest of looking capable to others in order to compete with others and outperform them. There is empirical evidence that adopting mastery goals can lead to better school achievement. For performance goals, no uniform results are found (Linnenbrink & Pintrich, 2002). For this reason, theoreticians have divided performance goals into performance approach goals, in which learners are focused on performing better than their peers, and performance avoidance goals, in which learners try to avoid failure (Elliot, 1999).

Expectancy focuses on the beliefs learners have in their own ability to perform the task (Bandura, 1986). Self-efficacy for learning and performance, which is also known as perceived competence or perceived ability, and control of learning beliefs belong to the expectancy construct (Pintrich & De Groot, 1990; Pintrich et al., 1993). Self-efficacy is not a stable trait, just as the other motivational components are not, and must not be confused with self-esteem (Linnenbrink & Pintrich, 2002) or self-concept (Choi, 2005; Zimmerman, 2000). Learners’ self-efficacy has been found to influence their self-regulation (Greene, Miller, Crowson, Duke, & Akey, 2004; Schunk & Zimmerman, 1997; Zimmerman, 1998), their use of deep information processing strategies (Bruinsma, 2004; Covington, 2000; Greene & Miller, 1996) and school achievement in experiments in which the participants differ from high school to college students (Choi, 2005; Greene et al., 2004; Linnenbrink & Pintrich, 2002; Pintrich & De Groot, 1990; Pintrich et al., 1993; Schunk & Zimmerman, 1997; Zimmerman, 1998; Zimmerman, 2000). Pintrich et al. (1993)

64 CHAPTER 5

found that control of learning beliefs was positively related to self-regulation and school achievement. Schunk and Zimmerman (1997) point out that high self-efficacy beliefs in combination with high values can influence the learning process in a positive way. Greene and Miller (1996) found that self-efficacy was indirectly related to school achievement. Self-efficacy seems to influence cognitive engagement, which in its turn seems to influence school achievement. Learners with more positive self-efficacy beliefs think they can do the task and will show more cognitive strategies which as a consequence will lead to better learning results. On the other hand we will stress that effective self-efficacy beliefs must be based on reality (Linnenbrink & Pintrich, 2002). That is, learners who overestimate their capabilities in having high self-efficacy beliefs that are not based on what they are capable of, do not necessarily have to perform well in school. Learners can acquire self-efficacy beliefs by prior experiences on performance or through interaction with the environment, such as receiving feedback or observing peers. Successful prior experiences can lead to higher self-efficacy beliefs, whereas failure experiences can lower the beliefs learners have (Bandura, 1986).

Affect can be distinguished in positive components such as joy, hope and pride, and negative ones, with components such as boredom and hopelessness. In this study, we will focus on the negative component of test anxiety. Pekrun, Goetz, Titz and Perry (2002) see test anxiety, or the amount of fear or worry learners have when taking a test, as a negative prospective task-related and self-related emotion, com-pared to retrospective emotions, such as the positive emotion pride, or the negative emotion jealousy. In their explorative research of emotions of learners in school and university, they found that anxiety was negatively related to intrinsic and extrinsic motivation (which can be compared to intrinsic and extrinsic goal orientation), but positively to extrinsic (performance) avoidance motivation. They also found that anxiety was negatively related to self-regulation strategies and achievement. Other empirical research has confirmed their finding that test anxiety is negatively related to self-regulation and school achievement (Pintrich & De Groot, 1990, Pintrich et al., 1993). In sum, research has shown that several constructs of motivation are positively re-lated to achievement and regulative cognitive processes. Intrinsic goal orientation, task value beliefs, self-efficacy, and control of learning beliefs are positively associ-ated, while test anxiety is negatively associated with achievement and learning proc-esses. Because different types of extrinsic goal orientation can be distinguished, it is more difficult to draw conclusions about this motivation component.

1.3 Context of study

In the current study, a computer-supported collaborative discovery learning envi-ronment is introduced, in which learners collaboratively generate hypotheses, test these hypotheses by performing experiments, and construct knowledge by interpret-ing the data of these experiments together. Since motivation is an individual trait, it is interesting to find out how the motivation of an individual learner relates to the


individual behavior of this learner in the collaborative learning process. But it is also interesting to know how the level of motivation in the group relates to a dyad of learners’ collaborative learning process and learning product, especially when moti-vation differs between partners. All motivation studies mentioned in the Introduction of this chapter used self-report questionnaires when measuring aspects of the learn-ing process, such as self-regulation (off-line measures). Most of them stress the im-portance of using other types of measures such as thinking-aloud protocols or obser-vation. In this study a self-report questionnaire will be used to measure the motiva-tional beliefs of learners, but we will use on-line measures to measure communica-tive and discovery activities of the learners, including regulative activities. Instead of school grades more specific measures are used as well, linked to the performance on the learning task. So, in the present study we will investigate how motivation is related to the observed collaborative scientific discovery learning activities. We will study both the process and the product of collaborative discovery learning, for indi-viduals in a team as well as for dyads itself. Our research questions are: 1) In what way do separate motivation variables of individuals relate to their indi-

vidual communicative and scientific discovery learning process and learning product?

2) In what way does a heterogeneous or homogeneous group composition of dyads with respect to motivation, relate to the group’s communicative and scientific discovery learning process and learning product?

With respect to the first research question, we expect that the motivation variables intrinsic goal orientation, task value beliefs, control of learning beliefs and self-efficacy will positively relate to the individual communicative and discovery learn-ing process and learning product of learners. Furthermore, we expect that the moti-vation variable test anxiety will negatively relate to the individual communicative and discovery learning process and learning product of learners. Due to the fact that little research has been conducted into motivation and learning of dyads, we have no specific expectations with regard to the second research question. This question will be investigated in a more explorative manner.

2 METHOD


A total of 73 dyads (N=146) from twelve classes of six secondary schools in Am-sterdam participated in this study. Their age ranged from 15 to 17 years. The partici-pants received €20. All subjects volunteered to enroll in the study. Data were col-lected on three occasions in the context of the three studies presented in the preced-ing chapters (Saab et al., in press; Chapter 2; Saab et al., submitted-a; Chapter 3; Saab, Van Joolingen & Van Hout-Wolters, submitted-b; Chapter 4) and aggregated into one data set. From the 73 dyads, 21 dyads are from the first study (Saab et al., in press), 27 dyads from the second study (Saab et al., submitted-a), and 25 dyads from

66 CHAPTER 5

the third study (Saab et al., submitted-b). The first, descriptive, study had a pretest-posttest design. The designs of the second and third study were pretest-posttest-control-group-designs. In these two studies the subjects were randomly divided in a control and an experimental group (see the following sections for more information about these studies).


All students worked collaboratively with a learning environment named Collisions6. Collisions was developed in SimQuest (Van Joolingen & De Jong, 2003). It is based on a computer simulation of colliding particles (Figure 1). The main learning task was to discover the underlying physics rules. For example, learners had to uncover the relation between the momentum of a particle before and after it hits a wall. Learners could see the effect on the other variables of a performed simulation by varying the variables mass and initial velocity of the ball. The environment also con-tains explanations for each of the variables present in the simulation.

In Collisions, students were presented with assignments that focused their atten-tion on a specific part of collisions, such as uniform motion, which is presented by the simulation. Assignments presented the learners with small research questions that could guide the discovery process. The assignments were completed either by choosing an answer to the question from a list or by formulating the answer as text. The environment contained four content levels of increasing complexity: Uniform motion, Fixed wall, Elastic collisions, and Completely inelastic collisions. During the first study, learners did not use the fourth level. We therefore decided to remove this level in the second and third study. In our analyses, we have taken this differ-ence in learning environment between the studies into account.

When students opened a level, a window with learning goals for that level was shown. For example: ‘In this level, you will find the relation between the mass (m) and the size of the momentum (p) of the ball’. In the first study the assignments con-sisted of multiple-choice questions. In the second and third study, the assignments were a combination of multiple-choice and open questions. The purpose of the first assignments of a level, which were multiple-choice questions, in the second and third study was to get the learners acquainted with the topic of the level. Then, when the learners had got acquainted with the topic, they were presented with open-question assignments in which they were supposed to type in an answer. To com-plete the level, they were presented with multiple-choice assignments with content that was related to open questions they had received earlier. In all studies, data to answer the assignments could be collected by means of the simulation. Dyads of students worked collaboratively on two computers with a shared interface, communicating through a chat channel. Students were not familiar with Collisions, but were acquainted with the variables presented in the environment.

6 Collisions was developed by Hans Kingma and Koen Veermans (University of Twente). SimQuest was developed in the SERVIVE-project which was coordinated by University of Twente.



The goal of the first study was to discover which communicative activities are fre-quently used in the discovery learning process, and to find out which communicative and discovery activities co-occur during this learning process. The students were not divided into an experimental or control group. In the second and third study, an in-tervention was carried out. In the second study, we investigated whether instruction in effective communication in a discovery learning environment can lead to more effective communicative activities, more effective discovery learning activities, and improved discovery learning results. The learners were randomly divided into a con-trol and experimental group, of which the latter received an instruction on collabora-tion, the RIDE rules (Saab et al., submitted-a). In the third study, all learners re-ceived the RIDE rules instruction, and the learners in the experimental group were additionally presented with a cognitive tool, the Collaborative Hypothesis Tool (CHT), which guided them through the processes of collaborative discovery learning (Saab et al., submitted-b).

In the current study three conditions of dyads can be distinguished (see Table 1). The first condition is the one with no intervention; this condition exists in dyads in the first study and the control group of the second study (N=31). The second condi-tion is the one in which the learners received the RIDE rules instruction; this condi-tions exists in the experimental group of the second study and the control group of the third study (N=32). The third condition exists in the experimental group of the

68 CHAPTER 5

third study (N=10), in which the dyads are, in addition to the RIDE rules, presented with the CHT.

Table 1. Conditions of dyads with corresponding amount of dyads

Conditions

Number of dyads

1) No intervention

31

2) RIDE instruction 32 3) CHT + RIDE instruction 10


We identified two types of learning outcome: the individual learning outcome and the collective performance within the learning environment.

The individual learning outcome is measured with the domain knowledge pretests and posttests. The domain knowledge pretests and posttests each include two domain knowledge tests, an explicit knowledge test, which tests the learners for declarative knowledge, such as facts and formulas, and a WHAT-IF test (Swaak, 1998; Veermans, De Jong, & Van Joolingen, 2000), which asks the learners to predict an effect of collision, after showing them situations before a collision, and presenting a change in the situation. Both tests were developed specifically for the domain of Collisions7 and were administered on screen. Both kinds of tests are used as declarative knowledge tests. The pretests and posttests were both parallel versions of the tests. As mentioned above, learners could not finish all the levels of the Collisions application in the time given in study 1. That is why the last of the four content levels in the application as well as the corresponding items in the pre- and posttests were removed for study 2 and 3. For this study the explicit knowledge test consisted of 14 items (originally 20 items) and the WHAT-IF test consisted of 12 items (originally 24). The internal consistency was measured with Cronbach’s alpha. The Cronbach’s alpha of the pre explicit knowledge test was considerably low (α=.24), but the internal consistency of the pre WHAT-IF test and the posttests were reasonable. Table 2 shows the Cronbach’s alpha coefficients for the pretests and posttests.

7 Both tests were developed by Janine Swaak (Swaak, 1998).


Table 2. Domain knowledge tests and Cronbach’s alphas

Domain knowledge tests

α

Number of items

Pre Explicit knowledge test

.24

14

Pre WHAT-IF test .44 12 Post Explicit knowledge test .47 14 Post WHAT-IF test .57 12

The collective learning outcome was measured by the groups’ performance on as-signments executed while working in the learning environment. For each assign-ment, a maximum of three points could be earned, one or two for providing a good answer and three points for providing an additional explanation. For multiple-choice assignments, the three points were awarded for the good answer. For 35 assign-ments, learners could therefore earn 105 points. As the nature of the assignments was different for the first study (which only contained multiple-choice assignments), that study was excluded from this part of the analysis. In studies 2 and 3, 17 assign-ments were open-ended and 18 were multiple choice. The total score is labeled as SWLE which stands for “score within learning environment”. Dyads could decide which assignments they wanted to do and whether they wanted to skip certain as-signments. Dyads could, for example, explore the environment instead of doing as-signments. This means that the amount of assignments was different for each dyad. That is why the relative SWLE, the proportion of the score within the learning envi-ronment, in addition to the absolute SWLE as described above, has been analyzed, as well.

2.4 Measuring motivation

For measuring motivation, six scales of the ‘Motivated Strategies for Learning Questionnaire’ (MSLQ) (Pintrich & De Groot, 1990)8 are used. A seven-point Likert scale was used for the items in the MSLQ, from 1 (not at all true of me) to 7 (very true of me). Table 3 shows the motivation scales with corresponding number of items for this study. Other psychometric details can be found in the section ‘Re-sults’. The score per scale is measured by taking the mean score of the items in that scale. Appendix B gives an overview of the questionnaire. The questionnaire was aimed at the subject “physics”, wherever a domain topic was mentioned in the ques-tionnaire, reference was made to physics.

8 Partly translated by Sabine Severiens.

70 CHAPTER 5

Table 3. Scales used to measure motivation

Motivation scales

Number of items

Intrinsic Goal orientation

4

Extrinsic Goal orientation 6 Task Value beliefs 4 Control of Learning beliefs 5 Self-Efficacy for Learning and Performance 8 Test Anxiety 5

2.5 Procedure

In the first study the learners were randomly assigned to the dyads. In the second and the third study, we opted for a heterogeneous group composition (Saab et al., submitted-a; Saab et al., submitted-b), since research has shown that groups of stu-dents with different levels of school grades are more successful working together than groups of students with similar learning results (Blatchford, Kutnick, Baines, & Galton, 2003).

Participants in the second and third study attended two sessions; the participants in the first study only attended one session, which is similar to the second session of study 2 and 3. In the first session of study 2 and 3, the participants received individ-ual instruction on collaboration (the RIDE rules). After the instruction, the students practiced collaboratively with logical thinking problems. In this way, the students could practice applying the rules they had learned earlier in the session.

The second session of studies 2 and 3, as well as the single session of study 1, started with the Motivation questionnaire (paper and pencil) and the domain knowledge pretests (on screen) for all students, which they filled in individually. They then worked together for 90 minutes with the application Collisions in the learning environment SimQuest. The experimental and control groups worked with different versions of the learning environment (see section 2.3). The session ended with the individually administered domain knowledge posttests (on screen) for all students.

So, the learners of all three studies were administered to the Motivation ques-tionnaire before the start of the experiment, i.e. working together with the applica-tion Collisions. Table 4 presents an overview of the features of the three studies.


Table 4. Overview of features of the three studies

Study 1

Study 2

Study 3

Number of sessions

1

2

2

Motivation questionnaire Yes Yes Yes Time on task 90 min. 90 min.a 90 min.a

Intervention No RIDE instruction RIDE instruction + CHT Condition 1 1+2 2+3 Control group No Yes Yes N (dyads) 21 27 25

aSecond session.

2.6 Data collection

All communicative and discovery learning activities were logged and were put to-gether in a single protocol for each dyad. A three-dimensional analysis scheme (Ta-ble 5) was used to analyze the protocols. The scheme was used in all studies. The dimensions are: a) communicative activities, b) discovery transformative learning activities, which promote the generation of information (Njoo & De Jong, 1993), and c) discovery regulative learning activities, which support and guide the learning process (Njoo & De Jong, 1993).

In the protocols, each chat utterance (defined as a verbalization typed in a chat window; Lebie, Rhoades, & McGrath, 1996) was scored on the dimensions commu-nicative activities, transformative discovery activities, and regulative discovery ac-tivities. Chat utterances were coded on all three dimensions. For all studies, two in-dependent researchers rated 10% of the protocols, after they both were trained in using the analysis scheme. Cohen’s kappa of inter-rater reliability between the two raters was for all dimensions in all studies between .79 and .97, which is reasonable to good (see Saab et al., in press; Saab et al., submitted-a; Saab et al., submitted-b).

72 CHAPTER 5

Table 5. Scheme to analyze students' actions and interactions





Identification of parameters and vari-ables Collecting data Interpreting data and graphics










3 RESULTS

3.1 Motivation questionnaire: psychometric data

Table 6 shows the means, standard deviations, and internal reliability coefficients for the Motivation scales for all learners.

Table 6. Descriptives of the Motivation scales

Motivation scales

M

SD

α

N


4.77

0.97

.49

146

Extrinsic Goal orientation 4.91 0.89 .59 146 Task Value beliefs 3.79 1.21 .81 146 Control of Learning beliefs 5.08 1.00 .61 146 Self-Efficacy for Learning and Performance 4.52 1.12 .90 146 Test Anxiety 3.55 1.40 .83 146

The Cronbach’s alpha’s for the motivation scales are moderate to good, with half of the alphas above .8. Intrinsic Goal orientation had the lowest alpha (.49). These measures are consistent to those found by Pintrich et al. (1993).

In Table 7, Pearson correlations between scales are reported. These correlations correspond to the correlations that were found by Pintrich et al. (1993).

Table 7. Pearson correlations between all motivation scales

Motivation scales

1.

2.

3.

4.

5.

1. Intrinsic Goal orientation

2. Extrinsic Goal orientation .32** 3. Task Value beliefs .41** .47** 4. Control of Learning beliefs .33** .12 .18* 5. Self-Efficacy for Learning and Performance .41** .37** .38** .55** 6. Test Anxiety -.12 -.02 -.08 -.24** -.48**

*p<0.05 **p<0.01.

74 CHAPTER 5

3.2 Motivation related to the communicative and discovery learning process and learning product

3.2.1 Individuals

To investigate whether the motivation of individuals is related to their behavior in collaborative discovery learning tasks, a Spearman correlational analysis with one-tailed testing was performed (see Table 8). Spearman correlations are used because of the skew distribution of some of the activities. The correlational analysis is per-formed for all learners. To be able to use all learners from all conditions in one analysis, no significant differences should occur between these conditions. To check whether there are significant differences between conditions, an ANOVA was car-ried out for every motivation scale and communicative and discovery learning activ-ity. When the interaction between a motivation scale and conditions was significant, this variable was used in the analysis.

Table 8. Spearman correlations between Motivation scales and total number of communicative and discovery learning activities

Activities

Motivation scales

Communicative

Transformative

Regulative


.10

---a

.22**

Extrinsic Goal orientation -.04 .14* .07 Task Value beliefs .14 .14* .17* Control of Learning beliefs ---a .07 .07 Self-Efficacy for Learning and Performance

.11 .19* .19*

Test Anxiety

-.22** -.28** -.15*

aLeft out of the analysis, because significant differences between conditions were found for this variable *p<0.05. **p<0.01 one sided. Significant positive correlations are found between Transformative activities and Extrinsic Goal orientation (r=.14; p<.05), Task Value beliefs (r=.14; p<.05), and Self-efficacy (r=.19; p<.05), and between Regulative activities and Intrinsic Goal orientation (r=.22; p<.01), Task Value beliefs (r=.17; p<.05), and Self-efficacy (r=.19; p<.05). Significant negative correlations are found between Communicative activities and Test Anxiety (r=-.22; p<.01), Transformative activities and Test Anxi-ety (r=-.28; p<.01), and Regulative activities and Test anxiety (r=-.15; p<.05). A paired-samples t-test was conducted to evaluate the impact of working with the learning environment on the learning product, measured with the domain knowledge


tests. There was a statistically significant increase in scores on the explicit knowl-edge test between the pretest (M=6.37, SD=2.03) and the posttest (M=7.82, SD=2.41), t(138)=6.08, p<.0005. The eta squared statistic (η2=.21) indicates a large effect size. We also found a significant increase in scores on the WHAT-IF test be-tween the pretest (M=4.89, SD=2.10) and the posttest (M=7.23, SD=2.36), t(144)=11.69, p<.0005. The eta squared statistic (η2=.49) indicates a large effect size for these tests as well. Partial Pearson correlations were used to explore the relation-ship between the individual Motivation scales and the learning products, while con-trolling for scores on the pretests (Table 9).

Table 9. Partial Pearson correlations of domain knowledge learning products and Motiva-tion scales, controlling for pretests scores

Post Explicit knowledge test

Post WHAT-IF test


-.06

-.04

Extrinsic Goal orientation .04 .02 Task Value beliefs -.07 -.04 Control of Learning beliefs .07 .09 Self-Efficacy for Learning and Performance

.05 .10

Test Anxiety -.21** -.17*

*p<0.05. **p<0.01.

There were significant negative partial correlations between the Explicit knowledge test and Test Anxiety (r=-.21; p<.01) and the WHAT-IF test and Test Anxiety (r=-.17; p<.05).

3.2.2 Dyads

We were interested in finding possible differences in communicative and discovery learning process and learning product between dyads with two highly motivated participants, dyads with two low motivated participants, and dyads with one highly motivated and one lowly motivated participant. To compose these dyads, we calcu-lated the median for every motivation scale. Table 10 shows the minimum, maxi-mum, and median of all scales. Using this median, dyads were classified as two high scoring participants, dyads with two low scoring participants, and dyads with one high scoring and one low scoring participant. For the latter group, we also required that the difference in score between dyad members was at least one standard devia-tion, in order to exclude dyads with both average motivation. Thus for every scale a different classification of dyads was used.

76 CHAPTER 5

Table 10. Minimum, Maximum, and Median of Motivation scales

Motivation scales

Minimum

Maximum

Median


1.00

6.75

5.00

Extrinsic Goal orientation 2.67 6.83 5.00 Task Value beliefs 1.00 6.50 3.75 Control of Learning beliefs 2.00 7.00 5.00 Self-Efficacy for Learning and Performance 1.59 6.63 4.50 Test Anxiety 1.00 6.80 3.50

To detect significant differences between the three kinds of dyads on the different motivation scales, multivariate analyses of variance (MANOVA) were conducted. Three dependent variables were used: total number of communicative activities, total number of transformative discovery activities, and total number of regulative discovery activities. Preliminary assumption testing was conducted to check for ho-mogeneity of variance-covariance matrices by using the Box’s M Test of Equality of Covariance Matrices, and for multicollinearity, with no serious violations noted.

As with the analysis of individual learners an ANOVA was carried out for every motivation scale and dependent variable for the dyads to check whether there are significant differences between conditions. When the interaction between the moti-vation scale and conditions was significant, this variable was not included in the MANOVA, because in that case the frequency of activities was significantly differ-ent for the conditions. From all the motivation scales, the MANOVA for Task Value beliefs was signifi-cant. The MANOVA revealed the main effect of the group for the motivation scale Task Value beliefs, F(6,126)=2.50, p<.05; Wilks’ Lambda=.80, η2=.11. The two univariate results concerning Task Value beliefs that were significant are Total number of communicative activities, F(2, 65)=5.82, p<.01, η2=.15, and Total num-ber of regulative activities, F(2, 65)=5.14, p<.01, η2=.14. Table 11 shows the means and standard deviations of the frequency of the dependent variables of the different kind of dyads for Task Value beliefs. As can be seen for Total number of communi-cative activities and Total number of regulative activities, dyads with two high scor-ing participants on Task Value beliefs had the highest frequency, respectively M=299.90, SD=91.47, and M=133.85, SD=721.08, compared to the dyads with two low scoring participants on Task Value beliefs, which had the lowest frequency of respectively M=202.21, SD=69.86, and M=67.79, SD=34.42.


Table 11. Mean, SD, and F-value of the three kinds of dyads for Task Value beliefs and fre-quencies of the total number of collaborative discovery learning process activities

Two high scorers on Task Value

beliefs N=20

Two low scor-

ers on Task Value beliefs

N=14

One high and

one low scorer on Task Value

beliefs N=34

M

SD

M

SD

M

SD

F


299.90

91.47

202.21

69.86

286.03

92.57

5.82**

Transformative activities 108.70 41.03 91.07 32.25 118.71 53.51 1.77 Regulative activities

133.85 72.08 67.79 34.42 109.94 63.33 5.14**

*p<0.05. **p<0.01. To get a closer look at which communicative and regulative activities are different in the three kinds of dyads, a second MANOVA was performed which included all communicative and regulative activities (see Table 5). This MANOVA, with as in-dependent variable dyad types for task value beliefs and dependent variables all separate communicative and regulative activities (see Table 5, see section 2.6), was significant, F(26,106)=1.78, p<.05; Wilks’ Lambda=.48, η2=.30. Significant differ-ences between these groups were found for three communicative activities and one regulative activity: Directive communicative activities, F(2, 65)=4.57, p<.05, η2=.12, Elicitative communicative activities, F(2, 65)=6.24, p<.01, η2=.16, Techni-cal/coordinated Off-task talk, F(2, 65)=3.45, p<.05, η2=.10, and Monitoring (regula-tive) activities, F(2, 65)=4.12, p<.05, η2=.11. The means and standard deviations of the frequency of the significant univariate results for the different kind of dyads for Task Value beliefs are presented in Table 12. For all activities, two high scoring participants on Task Value beliefs had the highest frequency compared to the dyads with two low scoring participants.

78 CHAPTER 5

Table 12. Mean, SD, and F-value of the three kinds of dyads for Task Value beliefs

Two high scor-

ers on Task Value beliefs

N=20

Two low scorers on Task Value

beliefs N=14

One high and one low scorer on Task Value

beliefs N=34

M

SD

M

SD

M

SD

F


Directive activities 30.10 12.35 16.71 11.54 24.18 13.37 4.57* Elicitative activities 40.30 16.54 23.64 10.18 39.50 16.14 6.24** Technical/coordinated off task talk

45.25 27.89 23.93 12.18 43.06 27.20 3.45*

Regulative activity Monitoring 73.90 53.50 32.50 22.07 56.24 39.16 4.12*

*p<0.05. **p<0.01. To measure the learning product of the dyads, the absolute and relative score within learning environment (SWLE) were used. As mentioned in the method section, data from Study 1 were excluded from this analysis.

To detect significant differences between the three kinds of dyads on SWLE and the proportion correctly answered assignments, MANOVAs were conducted. Pre-liminary assumption testing was conducted to check for homogeneity of variance-covariance matrices by using the Box’s M Test of Equality of Covariance Matrices, and for multicollinearity, with no violations noted. The MANOVA for the scale In-trinsic goal orientation was significant, F(4,84)=3.25, p<.05; Wilks’ Lambda=.75, η2=.13. Differences between kinds of dyads were significant only for SWLE, F(2, 43)=6.26, p<.01, η2=.23. As shown in Table 13, the dyads with two low intrinsic goal orientated participants had the highest score on SWLE (M=34.20, SD=13.72), compared to the dyads with two high intrinsic goal orientated participants which had the lowest score (M=19.54, SD=11.69). No significant differences were found for the relative SWLE.


Table 13. Mean, SD, and F-value of the three kinds of dyads for Intrinsic goal orientation

Two high intrinsic

goal orientated participants

N=13

Two low intrinsic goal orientated

participants N=10

One high and one low intrinsic goal orientated partici-

pant N=23

SWLE

M

SD

M

SD

M

SD

df

F

Absolutea

19.54

11.69

34.20

13.72

21.35

8.69

2,43

6.26**

Relativeb

.57 .12 .61 .17 .54 .18 2,43 0.412

*p<0.05. **p<0.01. aFrequency of score within the learning environment. bProportion of score within the learning environment.

4 DISCUSSION

In this study, we examined the relation between motivation and the observed learn-ing processes and learning product of collaborative scientific discovery learning process and learning product. We examined this for individuals and for dyads of learners.

For individuals, we expected intrinsic goal orientation, task value beliefs, control of learning beliefs, and self-efficacy to be positively related to the communicative and discovery learning process and learning product of learners. The expected posi-tive relations with regulative activities were found, except for control of learning beliefs. Moreover, we also found positive relations between extrinsic goal orienta-tion and transformative discovery activities. In this analysis, it should be taken into account that all individual actions were performed in the context of collaboration (Stahl, 2005). This may have consequences for the occurrence of activities, as some take place in reaction to the partner’s behavior. In addition, task value beliefs and self-efficacy related positively to transformative activities. None of these motiva-tional variables was related to communicative activities. Contrary to what we ex-pected, none of these motivational variables was positively related to the individual learning product, the score on the posttests, either. In line with the second hypothe-sis, test anxiety did indeed relate negatively with all aspects of the individual col-laborative discovery learning process and learning product. Learners with fear or worries about the task communicated less and performed less transformative and regulative activities than learners with little or no fear. They were less involved in the learning process than learners with a low score on test anxiety. Furthermore, the individual learning product is also negatively related to anxiousness of the learners.

80 CHAPTER 5

This may be a result of a less effective learning process. Another explanation could be that higher test anxiety has a direct negative influence on the performance on the tests.

For dyads of learners, it was investigated whether team heterogeneity in terms of motivation can influence the collective collaborative discovery learning process and learning product. Dyads existing of participants who both scored high on task value beliefs used the most communicative and regulative activities, compared to dyads with two low scoring participants, which used the fewest communicative and regulative activities. Especially, they used more technical and coordinated off-task talk, directive, and elicitative activities, and also more monitoring. These communicative activities are connected to two of the RIDE rules (Saab et al., submitted-a), viz. Deciding together and Encouraging. This means that dyads that found working on the task important engaged in more effective communication and regulation.

An unexpected result is the finding that two low scoring intrinsic goal oriented participants in a dyad scored higher on absolute number of correct assignments within the learning environment than two high intrinsic goal oriented ones. How-ever, this was not the case for the relative number of correct assignments. These higher scores were probably an effect of the lower motivated dyads to complete more assignment rather than performing better on each assignment chosen. This seems indicative of a situation where highly intrinsically motivated learners spend more time on exploring the environment, resulting in less completed assignments.

To sum up, three general conclusions can be drawn. Firstly, for our context, the motivational beliefs of the learners seem to influence learning at the individual rather than at the group (dyad) level. Secondly, test anxiety is found to be related to all aspects of the individual learning process and learning product, and not to the collective learning process or learning product. Thirdly, task value belief is the only motivational variable that is related to both the individual learning process and col-lective learning process. In future research, it would be interesting to investigate the relation between task value beliefs and the collaborative discovery learning process more thoroughly.

The design of the current study differs in several ways from earlier research on motivation and learning processes. Instead of using learners’ self-reports on learning processes, we used on-line measures of the learning process by analyzing the com-munication between students in a dyad. The fact that learning took place in a col-laborative setting allowed us, on one hand, to have a natural observation of learning processes and, on the other hand, to examine the effects of interaction between simi-larly or differently motivated students. To obtain firm results, triangulating studies (such as self-reports or observations) with on-line data within one study are needed, especially when the retrospective nature of self-reports is taken into account (Richardson, 2004).

Despite the fact that relations between motivation and learning process are found, no positive relation was found between motivational variables and learning results. A possible explanation for this lies in the difference in achievement meas-ures between the study at hand and other research (Choi, 2005; Greene & Miller, 1996; Greene et al., 2004; Linnenbrink & Pintrich, 2002; Pintrich & De Groot,


1990; Pintrich et al., 1993; Schunk & Zimmerman, 1997; Zimmerman, 1998, 2000). Most studies measure school achievement as grades, acquired over a longer time. We measured achievement using domain knowledge posttests (individual achieve-ment) and performance on assignments (SWLE, collective achievement). Especially the latter is not a usual one, as the assignments were primarily meant as instructional support and learners could select the assignments they wanted to pursue. The inter-pretation of results therefore should be done carefully and one should look at both the number of assignments done as well as the number of assignments that were completed successfully. Lowly intrinsically motivated dyads completed more as-signments, but did not do better relatively speaking than highly intrinsically moti-vated dyads (proportion SWLE). This indicates a difference in task perception be-tween lowly and highly intrinsically motivated learners. Lowly intrinsically moti-vated learners seem to process the assignments more superficially.

Because we had access to general school achievement data of the learners, which we used in the process of group composition, we could contrast our results with a more traditional approach to performance measuring. One-tailed Pearson correlation analyses were conducted after checking for possible significant differences between conditions, and found significant relations between school grades on physics (N=92) and intrinsic goal orientation (r=.33; p<.01), self-efficacy (r=.34; p<.01), and control of learning beliefs (r=.20; p<.05). Furthermore, it was found that regulative activi-ties were positively related to school grades (r=.30; p<.01). These findings are in line with results of other research (Choi, 2005; Greene & Miller, 1996; Greene et al., 2004; Linnenbrink & Pintrich, 2002; Pintrich & De Groot, 1990; Pintrich et al., 1993; Schunk & Zimmerman, 1997; Zimmerman, 1998, 2000), in which motiva-tional beliefs relate to learning process and learning product. No significant correla-tions between the school grades and the individual and collective learning results were found. It can therefore be concluded that both the individual measure, the do-main knowledge posttests, and the collective outcome measure (SWLE) measure a different construct than school grade. The learning result measures used in this study assess learning performance over a short period of time, whereas school grades measure performance over a longer period of time. The measurement of school grades also includes doing homework properly in addition to performing on tests and exams. Furthermore, the domain knowledge posttests consisted of few items, which can make the tests less reliable.

There was one exception to the findings that school grade correlates better with motivation than our knowledge measures. Task value beliefs do not correlate with school grades, and do correlate with our learning result measures. One can speculate that task value beliefs have a strong relation with the properties of specific learning tasks.

More detailed measures of motivational beliefs and repeated measures of moti-vation to check whether change over time occurs as a consequence of working in dyads are needed to assess the interaction of motivation between partners in a group. Furthermore, since these results are found in one learning environment, it would be interesting to see what happens with motivational beliefs and learning processes and learning product of learners in other kinds of collaborative learning environments.

82 CHAPTER 5

Overall, our results are in line with our expectations and theory, but, contrary to our hypotheses, no positive connections were found between motivation and individual learning results measured with the posttests. Furthermore, task value seems to be an important motivational construct with regard to the composition of dyads. Commu-nicative and regulative activities are more used in dyads where the task is experi-enced to be important. This effect of group composition with respect to task value beliefs should be an issue in future research.

CHAPTER 6

DISCUSSION

In this dissertation the relationship between collaborative and discovery learning processes has been investigated. It was assumed that a collaborative environment would serve as a support for scientific discovery learning. However, as learners do not always engage in effective collaborative learning, the collaborative learning processes have been supported as well as the processes of discovery by instructing learners in effective communication skills (Chapter 3) and also by means of provid-ing a cognitive tool aimed at assisting hypothesis generation and guiding learners through the discovery process (Chapter 4). In addition, the role of motivation in the collaborative discovery process is investigated (Chapter 5). The following research questions are central to this dissertation: Which communicative activities and discovery activities are frequently used in a collaborative discovery learning environment? Can specific support of the communication process and the discovery learning process improve collaborative discovery learning? What is the role of motivation in collaborative discovery learning? In the previous chapters, four studies were presented in which these research ques-tions were addressed. In these studies, secondary school students worked with a col-laborative simulation-based learning environment called “Collisions”. Students worked collaboratively in dyads on two different computers, using a shared screen and a chat channel to communicate. In this final chapter, the main findings of the four studies are summarized and integrated.

84 CHAPTER 6

1 RESULTS AND DISCUSSION OF THE FOUR STUDIES

1.1 Which communicative activities and discovery activities are frequently used in a collaborative discovery learning environment?

Chapter 2 reports an explorative study in which the collaborative discovery learning process was investigated by measuring the kinds of communicative activities and discovery activities in which the learners engaged. It was concluded that most com-municative activities co-occurred with the discovery activities of proposing answers, thinking of alternative answers, designing experiments, and concluding. It was found that the communicative activity ‘argumentation’ occurred little and, contrary to our expectations, that it co-occurred mostly with the discovery activity, ‘conclud-ing’, rather than with ‘generating hypotheses’. When generating hypotheses, learners appeared to simply exchange ideas, rather than arguing about them. Real discussion was postponed until the phase in which the learners drew conclusions. The post-ponement of discussion may be indicative of data-driven, inductive ‘experimenter’ behavior, in contrast to more deductive ‘theorist’ behavior, in which learners use their prior knowledge to generate hypotheses before carrying out an experiment (e.g., Klahr & Dunbar, 1988). Another finding was that high-performing dyads pro-posed more answers and generated more hypotheses than dyads with poorer per-formance. This finding is in line with the results of a study by Okada and Simon (1997), in which pairs performed better than singles because they used more ex-planatory activities. Thus, explanatory activities, such as proposing answers and generating hypotheses, seem to be of importance in the learning process, as they result in good performance. The dyads that performed well in this first study con-firmed and accepted ideas more frequently than less successful dyads. This is an indication that these dyads established agreement more often. This finding is cor-roborated in a study by Barron (2003) in which successful groups (triads) were more involved with the ideas of the other learners, did not reject as many proposed ideas, and confirmed more than groups that performed less well on a mathematics prob-lem-solving task.

Because of the use of correlational analyses, it is not possible to establish cause-effect relationships between collaboration and discovery learning. Future research needs to focus on the probable causal relationship between the two. Collaboration may have a positive influence on the discovery learning process (Okada & Simon, 1997; Salomon & Globerson, 1989).

1.2 Can specific support of the communication process and discovery learning process improve collaborative discovery learning?

As learners need support with both the collaborative and the discovery learning process, two kinds of support were presented to them in the same learning environment. In Chapter 3, computerized communication instruction on effective and efficient communication was designed and presented to the learners in the form of the RIDE rules (Respect, Intelligent collaboration, Deciding together, and Encouraging). Offering the RIDE rules proved to be effective, as students who

DISCUSSION 85

received instruction in these rules used the activities associated with the rules more, in particular Deciding together (D) and Encouraging (E). Students instructed in the RIDE rules asked for agreement more frequently, asked more open and critical questions and asked more frequently for clarification of lack of understanding. They also more frequently gave informative answers, agreed more often, and asked their partner more often to perform an action in the learning environment. The dyads that received instruction also performed more discovery learning activities, such as generating hypotheses, concluding, and performed more regulative activities. In other studies, these activities were found to contribute to more effective learning (e.g., De Jong & Njoo, 1992; King, 1997; Mercer, 1996; Njoo & De Jong, 1993; Webb & Farivar, 1994; Wegerif, Mercer, & Dawes, 1999).

In Chapter 4, the Collaborative Hypothesis Tool (CHT) was added to the appli-cation Collisions that guided the learners through the processes of hypothesis gen-eration, planning and concluding. This tool consisted of a Hypothesis scratchpad (Van Joolingen & De Jong, 1991; 1993) adapted to the collaborative environment. It stimulated both learners in a dyad to generate hypotheses, evaluate each other’s hy-potheses and to indicate whether they wanted to test a hypothesis. It included win-dows with instructions on how to use the collaborative scratchpad together, how to plan the way to the answer, and a reminder to check whether the answers confirmed or rejected the hypotheses in the scratchpad. The fact that use of the tool was not mandatory explains that it was used very little, which again explains the lack of dif-ferences in learning results between experimental groups. However, in the cases where the tool was used, learners used more effective communicative and discovery learning activities. In a study by Gijlers and De Jong (submitted) a similar tool, which they labeled a ‘Shared proposition scratchpad’, was compared to a tool known as a ‘Shared proposition list’ that supported the discussion of hypotheses learners had already been given. This latter tool proved to be more effective than the proposition scratchpad, where the students had to compose their own hypotheses. This result, together with the findings of the present study, indicates that formulating hypotheses - even when a tool is provided - is still a bottleneck in the discovery learning process.

No difference in collective learning results were found between the groups that received the extra supports and the groups that did not (Chapter 3 and Chapter 4). In both experimental conditions, the students were provided with prompts during the learning process. Although prompts can have a positive effect on learning (cf. Howe & Tolmie, 1998), it takes time to read them and to follow the instructions, which can result in fewer assignments being finished in a certain amount of time. Kozma (1991) found that not all learners make use of or respond to prompts in a similar way. In this study, in which Kozma examined the impact of computer-based tools and embedded prompts on college writers, it was found that novices responded to the prompts differently from advanced writers. Thus, how learners make use of prompts is also dependent on their existing skills and knowledge. This means differ-ences in learners’ prior skills and prior knowledge should be taken into account when designing supportive measures for collaborative scientific discovery learning. Not all support is likely to be equally effective for all kinds of learners. In our stud-ies, it is possible that learners were given not enough time to acquaint themselves

86 CHAPTER 6

with the support offered, meaning that during the experimental session, they could give less attention than needed to construct new knowledge.

1.3 What is the role of motivation in collaborative discovery learning?

The first objective of the study reported in Chapter 5 was to investigate the relation-ship between the level of motivation and both the individual communication and discovery learning processes and learning products. The second objective was to examine whether team heterogeneity in terms of motivation influences the collective learning process and learning product. The questionnaire that was used to measure motivation was based on an expectancy value model and included the motivational constructs: value, expectancy and affect.

The results showed that level of motivation for the constructs value and expec-tancy was related to individual learning, rather than to learning at group level, ex-cept in the case of task value beliefs. Dyads with two participants with ‘high’ task value beliefs communicated more and used more regulative discovery activities than dyads with two participants with ‘low’ task value beliefs, or than heterogeneously composed dyads.

In the current study motivation did not appear to be positively related to learning results. In this study, performance was measured in terms of performance on the specific task, rather than in terms of general grades. Hence, the relation between school achievement and motivation found in earlier research (Choi, 2005; Greene & Miller, 1996; Greene, Miller, Crowson, Duke, & Akey, 2004; Linnenbrink & Pintrich, 2002; Pintrich & De Groot, 1990; Pintrich, Smith, Garcia, & McKeachie, 1993; Schunk & Zimmerman, 1997; Zimmerman, 1998, 2000) could not be replicated in this study at the smaller scale of task performance. However, similar positive relationships between motivation and school achievement were also found in this study, when looking at the relationship between motivation and school grades. The measurement of school grades includes a number of tasks carried out over an extended period of time, like doing homework and completing various tests, whereas the learning product measures used in this dissertation assessed the learning outcomes of a single task carried out within a limited period of time.

The results from our motivational analysis must be interpreted with care. Pairing of students was not based on motivation but on school achievement instead. Also, motivation was assessed only on one point, namely before the actual experiment took place, which makes it impossible to detect changes in motivation over time. Another noteworthy point is that level of motivation was only assessed before learn-ers worked with the learning environment. However, it is possible for learners to change their motivational beliefs while working collaboratively, which can influence the learning process and learning results. Järvelä, Rahikainen, and Lehtinen (2001) studied learners working together in the collaborative inquiry environments, CSILE and Knowledge Forum (Scardamalia & Bereiter, 1994). They found that learners working in a computer-supported environment had different motivational goals over time from learners in a traditional classroom context. They could not specify, how-ever, how motivation was related to the learning process or learning product. Future

DISCUSSION 87

study is recommended in which motivation is measured at different times while learners are working with the learning environment, and in which the relationship between motivation at these different times and the activities in the collaborative discovery learning process are investigated.

2 GENERAL DISCUSSION

2.1 Measuring learning process and learning product

In this dissertation, the collaborative discovery learning process was analyzed. In this analysis, communicative activities and discovery activities were distinguished. These activities were logged as protocols in which the communicative activities, the chats, and the discovery activities performed in the learning environment could be distinguished. These protocols were analyzed using the analysis scheme described in Chapter 2. The protocols were scored on three dimensions: communicative activi-ties, transformative discovery activities and regulative discovery activities. A single utterance could thus be scored on all three dimensions. This allowed us to combine the communicative and the discovery perspective in a single instrument of analysis. One utterance could be coded as both a communicative and a transformative discov-ery activity, or as both a communicative and a regulative discovery activity. For example, formulating a hypothesis, which is a transformative discovery activity, can be carried out in different communicative ways. Learners can formulate the hy-pothesis as a checking question: ‘The momentum is twice the velocity, right?’ The hypothesis can also be formulated as a declarative statement: ‘I think that the mo-mentum is twice the velocity’. Using this analysis scheme to code the protocols could have been problematic if the communicative and discovery process dimen-sions were dependent on each other, for instance, if a particular communicative ac-tion was always scored as the same discovery activity. However, inspection of the data showed that no such couplings occurred, and thus that the scoring dimensions really were independent of each other.

Two kinds of measurements were used to measure the learning results in the studies of this dissertation. The first kind was an individual off-line measurement, and the second was a collective on-line measurement called ‘score within the learn-ing environment’ (SWLE). The latter measurement, the SWLE, is not a real learning result measure, since it was assessed during the intervention. As SWLE is a learning result measure obtained during the learning process, it can, therefore, be seen as part of the learning process. The individual off-line learning results were measured using domain knowledge posttests, controlling for the pretest level of domain knowledge. The pre- and posttests were parallel versions of an Explicit Knowledge test and a What-if test.

The domain knowledge tests suffered from low reliability in the studies reported in Chapters 3 and 4, though not in the studies reported in Chapters 2 and 5. Other uses of the same tests (Swaak & De Jong, 2001; Swaak, De Jong, & Van Joolingen, 2004; Veermans, 2003) also produced varying reliabilities. Two differences between the studies in this dissertation could explain the differences in reliability results. Firstly, in the studies reported in Chapters 3 and 4 the number of items in the tests

88 CHAPTER 6

was smaller than in the study in Chapter 2. Secondly, in Chapter 5, more subjects participated in the study, as data from the three studies of Chapters 2 to 4 were used in the analyses. The number of items and the number of participants are features that may have affected test reliability. Another possible explanation for the low reliabil-ities can be found in the rather low scores on the tests, especially for the pretests, which in most cases were just above chance level.

2.2 Cause-effects relations

A significant body of research has already established the benefits of collaborative learning. Research has shown that collaborative learning can support the learning process and, as a result, also the learning product. Moreover, it has been found that pairs perform better than singles in discovery learning situations (Okada & Simon, 1997; Whitelock, Scanlon, Taylor, & O’Shea, 1995), which led to the formulation of the hypothesis that carrying out communicative activities in collaboration can con-tribute to discovery learning. Did collaboration have a positive effect on discovery learning in the research reported in this dissertation? In Chapter 2, it was hypothe-sized that communicative support of the discovery learning process, such as building common ground or establishing common conclusions, can enhance the learning process. The results of this study showed that, in general, the expected communica-tive activities were positively related to the discovery activities. When learners per-formed many discovery activities associated with, for example, the discovery proc-ess ‘concluding’, they carried out a lot of confirming, argumentative, and elicitative activities, which are important communicative activities that aid the process of con-cluding. From these results it can be concluded that a positive relationship exists between communicative activities and discovery activities. However, as correla-tional analyses were used rather than analyses that investigate causal relationships, it is not possible to draw the conclusion that communication actually supported dis-covery learning. Nevertheless, in Chapter 3, the learners who were instructed in us-ing the RIDE rules had more effective communication and used more discovery ac-tivities, such as drawing conclusions and regulative activities, than the learners who did not receive instruction. Thus, there is some indication that, by using more effec-tive communication in the collaborative process, the discovery learning process was supported.

3 GENERAL CONSIDERATIONS

3.1 Salience of activities

The use of communicative activities and discovery activities was compared in all the studies, in order to ascertain which kinds of activities were used most frequently. Both regulative activities and the activities associated with the RIDE rules Deciding together and Encouraging turned out to be the most conspicuous. In Chapter 3, the experimental group (the group that received the RIDE rules) used these activities significantly more often than the dyads in the control group. In Chapter 4, these ac-tivities (except for activities associated with Encouraging) were positively related to

DISCUSSION 89

activities involving use of the Collaborative Hypothesis Tool (CHT), namely, the proportion of planning within the CHT and the proportion of correctly answered assignments after using the CHT. In Chapter 5, dyads with two participants who scored high on task value beliefs used significantly more of these activities. Al-though in the study of Chapter 2 the RIDE rules had not yet been introduced, indica-tions were found that corroborate the findings of the other studies in this disserta-tion. The communicative activity, confirmation and acceptance, which is connected to the Deciding together rule, was used significantly more by teams that scored high on collective learning results, than by low scoring teams. A general finding of this dissertation is that regulative activities and communicative activities associated with the RIDE rules Deciding together and Encouraging are essential activities in the computer-supported collaborative discovery environment used in this dissertation. Future research examining the use of these specific activities in collaborative dis-covery learning environments is recommended. Moreover, a potentially fruitful di-rection for future research into the support of collaborative discovery learning proc-esses could involve instruction designed to stimulate the use of these specific activi-ties.

3.2 Implicit communication in computer-supported environments

Some communicative and discovery activities assumed to be necessary in the collaborative discovery learning process turned out not to be as effective as expected. Other activities turned out to be used rather infrequently in the studies reported in this dissertation. In particular, hypothesis generation and experimental design were not encountered frequently in the logs of the chat conversations. A possible explanation for the infrequent occurrence of these activities is that learners not only communicated by chatting with each other, but that they also communicated through their manipulations of the environment (cf. Van Drie, Van Boxtel, Jaspers, & Kanselaar, 2005; De Vries, Lund, & Baker, 2002). For instance, instead of communicating through chat about designing an experiment or formulating a hypothesis, a learner can change the variables on the simulation, which is seen by the other learner and subsequently accepted or rejected by that other learner. Consequently, there may be no need to type an utterance in addition to carrying out an action, meaning that ideas and utterances remain implicit. Sins, Van Joolingen, Savelsbergh, and Van Hout-Wolters (submitted) found that students who communicated by means of chat made different use of actions than students who communicated face-to-face. For these students, the use of certain activities led to better performance, whereas the use of these activities by the students in the face-to-face group did not, which could indicate that for the learners engaged in chat part of the communication occurred via the use of these activities. In this dissertation in terms of communicative activities, argumentative activities were used less frequently than expected. This communicative activity did not appear to be necessary in the learning environment used, which could once again be the result of using features in the learning environment other than the chat channel as a medium of communication. Lack of use of the chat channel for argumentation may relate to

90 CHAPTER 6

an important characteristic of chat, namely, that it invites learners to use short messages, which makes it more difficult to argue or discuss.

3.3 Knowledge and skills learned

In the study reported in Chapter 5, the data from all the studies were merged. The results of this study showed that, in overall terms, working with the learning envi-ronment did lead to an increase in domain knowledge or declarative knowledge. The difference between the pre- and the posttests was significant, which means that working with the learning environment had a positive effect on level of domain knowledge. However, although these effects were significant, they were only small. A possible explanation for the small size of the improvements is that, rather than acquiring a large amount of declarative knowledge, the learners may have primarily acquired discovery skills, collaborative skills or skills relevant only to the specific environment. This could be a consequence of the nature of the learning environment, or it could be a consequence of the limited time the learners worked with the envi-ronment. It is possible that if learners had worked longer with the subject matter, they would have acquired more declarative knowledge. Another issue is that what learners learn may depend on their perceptions on the learning environment. Learn-ers can approach a learning environment in different ways, which can have an effect on the knowledge they gather or skills they learn. If they see the learning environ-ment as a place where the emphasis is on collaboration rather than on discovery learning, they will probably focus on the collaborative process, rather than on ex-ploring the environment. Similarly, if they focus solely on working with added cog-nitive tools, they will not acquire the deeper ideas or knowledge that can be learned from the environment.

To assess whether the learners learned things other than declarative knowledge, other instruments are needed. A relevant question for future research concerns the kinds of skills and knowledge learners acquire when working with the collaborative discovery environment used in this dissertation. In future studies, learners could also be interviewed to obtain information concerning their perception of the learning environment.

4 EDUCATIONAL IMPLICATIONS

Participating effectively in today’s society requires a sophisticated level of general development, and also requires people to possess well-developed social and com-municative skills that enable them to work collaboratively. The Dutch government has recognized the importance of paying attention to the development of social and practical skills in their policies and introduced a new pedagogical and didactical approach into Dutch secondary schools in 1999, which they called the ‘Studiehuis’ Ministerie van OCenW, 2005-a) The pedagogical aim of this approach is to enable students to work more actively and independently in order to promote the develop-ment of both practical skills and social skills. Additionally, the use of ICT (Informa-tion and Communication Technology) as a support for learning processes is also

DISCUSSION 91

considered important (Ministerie van OCenW, 2005-b). These ideas with respect to active, collaborative and independent learning are compatible with the constructivist view of learning (Duffy & Jonassen, 1991; Jonassen, 1991, 2000).

In this dissertation, three aspects of learning considered important in the ‘Studiehuis’ were examined: collaborative learning, discovery learning and learning in a computerized simulation environment. Specifically, the relationship between collaborative learning and discovery learning was investigated in a computerized simulation environment. As students can have problems with collaborative learning and/or discovery learning (e.g., De Jong & Van Joolingen, 1998; Mercer, 1996; Webb & Farivar, 1994), it was necessary to support the collaborative discovery learning process. In this dissertation, two kinds of support were provided: communication-oriented instruction in the form of the RIDE rules, and a cognitive tool, the CHT. The RIDE rules turned out to promote effective communicative and discovery activities. This finding is in line with other studies in which students benefited from instruction in effective communication, but in which communication took place face-to-face, rather than through chat (e.g., Hoek, 1998; Mercer, 1996; Rojas-Drummond & Mercer, 2003; Swing & Peterson, 1982; Wegerif et al., 1999). The CHT was not used very often, but when it was used, it was positively related to the use of effective activities. Furthermore, the relationship between motivation and collaborative discovery learning was investigated. Although motivation is important when working collaboratively (Strijbos, 2004) in a computer-supported learning environment (Jones & Issroff, 2005), there is few research with respect to motivation in collaborative discovery learning.

The RIDE rules that learners were taught were general communication and col-laboration rules. Hence, they could also be applied in a face-to-face learning situa-tion or in a different collaborative learning environment in which learners have to communicate by means of chat. However, it should be remembered that instruction was given in the form of a computerized presentation in which examples of possible situations specific to the learning environment were presented to serve as a model for the learners. The design of the instruction was based on the principles of the cognitive apprenticeship model, in which modeling is part of the instruction method. Consequently, the instruction provided is quite specific to the learning environment used in this dissertation. Thus, for use in collaborative learning situations other than the current one, the instruction would need to be constructed in such a way that it models the situation of the learning environment used.

Another implication of the research in this dissertation is that when support for learning processes is provided in a collaborative discovery environment, extensive training with sufficient practice opportunities should precede working with the learning environment, in order to reduce the likelihood of learners experiencing cognitive overload.

The research reported in this dissertation showed that there was a relationship between communicative activities and discovery activities. However, because of the use of correlational analyses, no firm conclusions with respect to the causal direction of the support can be drawn. A crucial question for educational practice and future research that still needs to be answered is: which learning processes need to be sup-ported in order to enhance the collaborative discovery learning process? Does col-

92 CHAPTER 6

laboration need support or does the discovery learning process need support, or maybe both? Future research in which the attention is directed towards possible causal relationships between communicative and discovery activities is highly rec-ommended. The results of the research reported here can serve as a basis for the generation of new hypotheses.

As found in previous research and as pointed out in a recent report of the Inspectorate of education (Inspectie van het Onderwijs, 2003), there is a need for training in practical and social skills. The research in this dissertation has tried to take a step in the right direction by providing some insight into how students’ learning processes can be supported. More research could contribute to gaining a clearer picture of how students can be guided in developing an independent, active, and collaborative learning process so that they can function effectively and with more ease in a society in which these skills are essential.

REFERENCES

Baker, M. (1999). Argumentation and constructive interaction. In: J. Andriessen & P. Coirier (Eds.), Foundations of argumentative text processing (pp. 179-202). Amsterdam: University of Amsterdam Press.

Baker, M. J., Hansen, T., Joiner, T. & Traum, D. (1999). The role of grounding in collaborative learning tasks. In P. Dillenbourg (Eds.), Collaborative learning: Cognitive and computational approaches, (pp. 31-63). Amsterdam: Pergamon / Elsevier Science.

Baker, M. J. & Lund, K. (1997). Promoting reflective interactions in a CSCL environment. Journal of Computer Assisted Learning, 13, 175-193.

Bandura, A. (1986). Social foundations of thought and action: A social cognitive theory. Englewood Cliffs, NJ: Prentice-Hall.

Bandura, A. (2001). Social cognitive theory of mass communication. Mediapsychology, 3, 265-299. Barron, B. (2003). When smart groups fail. The Journal of the Learning Sciences,12 (3), 307-359. Blatchford, P., Kutnick, P., Baines, E., & Galton, M. (2003). Toward a social pedagogy of classroom

group work. International Journal of Educational Research, 39, 153-172. Brown, J. S., Collins, A., & Duguid, P. (1989). Situated cognition and the culture of learning. Educa-

tional Researcher, 18 (1), 32-42. Bruinsma, M. (2004). Motivation, cognitive processing and achievement in higher education. Learning

and Instruction, 14, 549-568. Burns, J. C., Okey, J. R., & Wise, K. C. (1985). Development of an integrated science process skills test.

TIPS II. Journal of Research in Science Teaching, 2, 169-177. Chan, C. (2001). Peer collaboration and discourse patterns in learning from incompatible information.

Instructional Science, 29, 443-479. Chan, C., Burtis, J., & Bereiter, C. (1997). Knowledge-building as a mediator of conflict in conceptual

change. Cognition and Instruction, 15, 1-40. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How

students study and use examples in learning to solve problems. Cognitive Science, 13, 145-182. Chinn, C. A, & Brewer, W. F. (1993). The role of anomalous data in knowledge acquisition: A theoretical

framework and implications for science education. Review of Educational Research, 63, 1-49. Choi, N. (2005). Self-efficacy and self-concept as predictors of college students’ academic performance.

Psychology in the Schools, 42 (2), 197-204. Cohen, E. G. (1994). Restructuring the classroom: Conditions for productive small groups. Review of

Educational Research, 64, 1-35. Covington, M. V. (2000). Goal theory, motivation, and school achievement: An integrative review. An-

nual Review of Psychology, 51, 171-200. Damon, W., & Phelps, E. (1989). Critical distinctions among three approaches to peer education. Interna-

tional Journal of Educational Research, 13, 9-19. De Groot, A. D. (1969). Methodology, foundations of inference and research in the behavioural sciences.

The Hague: Mouton. De Jager, B., Reezigt, G. J., & Creemers, B. P. M. (2002). The effects of teacher training on new instruc-

tional behaviour in reading comprehension. Teaching and Teacher Education, 18, 831-842. De Jong, T. (1991). Learning and instruction with computer simulations. Education and Computing, 6

(3,4), 217-229. De Jong, T., & Njoo, M. (1992). Learning and instruction with computer simulations: Learning processes

involved. In E. De Corte, M. Linn, H. Mandl & L. Verschaffel (Eds.), Computer-based learning en-vironments and problem solving (pp. 411-429). Berlin: Springer-Verlag.

De Jong, T. & Van Joolingen, W. R. (1998). Discovery learning with computer simulations of conceptual domains. Review of Educational Research, 68, 179-201.

94

De Vries, E., Lund, K., & Baker, M. (2002). Computer-mediated epistemic dialogue: Explanation and argumentation as vehicles for understanding scientific notions. The Journal of the Learning Sciences, 11 (1), 63-104.

Dekker, R. & Elshout-Mohr, M. (1998). A process model for interaction and mathematical level raising. Educational Studies in Mathematics, 36, 303-314.

Doise, W., Mugny, G., & Pérez, J. A. (1998). The social construction of knowledge: Social marking and socio-cognitive conflict. In: U. Flick (Eds.), The psychology of the social, (pp. 77-90). New York: Cambridge University Press.

Dubrovsky, V. J., Kiesler, S., & Sethna, B. N. (1991). The equalization phenomenon: Status effects in computer-mediated and face-to-face decision making groups. Human Computer Interaction, 6, 119-146.

Duffy, T. M. & Jonassen, D. H. (1991). Constructivism: New implications for instructional technology? Educational Technology, 7-12.

Dunbar, K. (2000). What scientific thinking reveals about the nature of cognition. In K. Crowley, C. D. Schunn, & T. Okada (Eds.), Designing for Science: Implications from Everyday, Classroom, and Professional Settings (pp. 115-140). Hillsdale, NJ: Lawrence Erlbaum Associates.

Dweck, C. S. & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psy-chological Review, 95, 256-273.

Ebbens, S., Ettekoven, S. & Van Rooijen, J., (1997). Samenwerkend leren. Praktijkboek. [Collaborative learning. Practical book.] Groningen: Wolters-Noordhoff.

Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values and goals. Annual Review of Psychol-ogy, 53, 109-132.

Elliot, A. J. (1999). Approach and avoidance motivation and achievement goals. Educational Psycholo-gist, 34 (3), 169-189.

Elshout-Mohr, M. (1992). Metacognitie van lerenden in onderwijsleerprocessen. [Metacognition of learn-ers in educational learning processes.] Tijdschrift voor onderwijsresearch, 17 (5), 273-289.

Erkens, G. (1998). Multiple Episode Protocol Analysis (MEPA 3.0) (Internal publication). Utrecht, The Netherlands: Department of Educational Science, Utrecht University.

Erkens, G., Andriessen, J. E. B., & Peters, N. (2003). Interaction and performance in computer supported collaborative tasks. In H. Van Oostendorp (Eds.), Cognition in a digital world (225-251). Mahwah, NJ: Erlbaum.

Erkens, G., Jaspers, J., Prangsma, M., & Kanselaar, G. (2005). Coordination processes in computer sup-ported collaborative writing. Special issue: The pedagogical use of information and communication technology in education: A Dutch perspective, Computers in Human Behavior, 21 (4), 575-602.

Ettekoven, S. (1997). Samenwerkend leren in de praktijk. Valkuilen of uitdagingen? [Collaborative learn-ing in practice. Pitfalls or challenges?] In P. Leenheer, P. R. J. Simons & J. Zuylen (Eds.), Didac-tische verkenningen van het studiehuis. [Didactic explorations of the studiehuis.] (pp. 89-97). Til-burg: MesoConsult B.V.

Fleiss, J. L. (1981). Statistical methods for rates and proportions. NY: John Wiley & Sons. Gijlers, A. H. (2005). Confrontation and co-construction: Exploring and supporting collaborative scien-

tific discovery learning with computer simulations. Unpublished doctoral dissertation, University of Twente, Enschede, The Netherlands.

Gijlers, A. H., & De Jong, T. (submitted). Sharing and confronting propositions in collaborative scien-tific discovery learning.

Gorman, M. E. (1986). How the possibility of errors affects falsification on a task that models scientific problem solving. British Journal of Psychology, 77, 85-96.

Gorman, M. E., Stafford, A., & Gorman, M. E. (1987). Disconfirmation and dual hypotheses on a more difficult version of Wason’s 2-4-6 task. The Quarterly Journal of Experimental Psychology, 39A, 1-28.

Greene, B. A., & Miller, R. B. (1996). Influences on achievement: Goals, perceived ability, and cognitive engagement. Contemporary Educational Psychology, 21, 181-192.

Greene, B. A., Miller, R. B., Crowson, H. M., Duke, B. L., & Akey, K. L. (2004). Predicting high school students’ cognitive engagement and achievement: Contributions of classroom perceptions and moti-vation. Contemporary Educational Psychology, 29, 462-482.

Hendricks, C. C. (2001). Teaching causal reasoning through cognitive apprenticeship: What are results from situated learning? The Journal of Educational Research, 94 (5), 302-311.

REFERENCES 95

Henri, F. (1992). Computer conferencing and content analysis. In: A. R. Kaye (Eds.), Collaborative learning through computer conferencing. The Najaden papers. (pp. 117-136). Berlin Heidelberg: Springer-Verlag.

Hoek, D. J. (1998). Social and cognitive strategies in cooperative groups: Effects of strategy instruction in secondary mathematics. Unpublished doctoral dissertation, University of Amsterdam, Amsterdam, The Netherlands.

Howe, C., & Tolmie, A. (1998). Computer support for learning in collaborative contexts: Prompted hy-pothesis testing in physics. Computers & Education 3-4, 223-235.

Inspectie van het Onderwijs [Dutch Inspectorate of Education] (2003).Tweede fase vierde jaar. Een over-zicht van de stand van zaken vier jaar na invoering van de tweede fase havo/vwo. [An overview of the state of affairs four years after introducing the tweede fase havo/vwo.] Utrecht: Inspectie van het Onderwijs.

Järvelä, S., Rahikainen, M., & Lehtinen, E. (2001, August). Meaningful participation in a process of computer-supported inquiry: The analyses of students’ motivational involvement. Paper presented at the 9th European conference for Research on Learning and Instruction, Freiburg, Switzerland.

Johnson, D. W., & Johnson, R. T. (1999). Making cooperative learning work. Theory in practice, 38 (2), 67-73.

Jonassen, D. H. (1991). Objectivism versus constructivism: Do we need a new philosophical paradigm? Educational Technology Research & Development, 39, 5-14.

Jonassen, D. H. (2000). Computers as mindtools for schools. NJ: Prentice-Hall, Inc. Jones, A., & Issroff, K. (2005). Learning technologies: Affective and social issues in computer-supported

collaborative learning. Computers and Education, 44, 395-408. King, A. (1997). ASK to THINK-TEL WHY®©: A model of transactive peer tutoring for scaffolding

higher level complex learning. Educational Psychologist, 32 (4), 221-235. Klahr, D., & Dunbar, K. (1988). Dual space search during scientific reasoning. Cognitive Science, 12, 1-

48. Kozma, R. B. (1991). The impact of computer-based tools and embedded prompts on writing. Cognition

& Instruction, 8 (1), 1-27. Kuhn, D., Black, J., Keselman, A., & Kaplan, D. (2000). The development of cognitive skills to support

inquiry learning. Cognition & Instruction, 18 (4), 495-524. Lajoie, S. P. (1993). Cognitive tools for enhancing learning. In S. P. Lajoie & S. J. Derry (Eds.), Com-

puters as cognitive tools (pp. 261-289). Hillsdale, NJ: Erlbaum. Lajoie, S. P., Lavigne, N. C., Guerrera, C., & Munsie, S. (2001). Constructing knowledge in the context

of BioWorld. Instructional Science, 29 (2), 155-186. Lazonder, A. W., Wilhelm, P., & Ootes, S. A. W. (2003). Using sentence openers to foster student inter-

action in computer-mediated learning environments. Computers & Education, 41 (3), 291-308. Lebie, L., Rhoades, J. A,. & McGrath, J. E. (1996). Interaction process in computer-mediated and face-to-

face groups. Computer Supported Cooperative Work, 4 (2-3), 127-152. Limón, M. (2001). On the cognitive conflict as an instructional strategy for conceptual change: a critical

appraisal. Learning and Instruction, 11 (4), 357-380. Linnenbrink, E. A., & Pintrich, P. R. (2002). Motivation as an enabler for academic success. School Psy-

chology Review, 31 (3), 313-327. Löhner, S., Van Joolingen, W. R., Savelsbergh, E. R., & Van Hout-Wolters, B. H. A. M. (2005). Students'

reasoning during modeling in an inquiry learning environment. Computers in Human Behavior, 21, 441-461.

Marshall, S. P. (1995). Schemas in Problem Solving. Cambridge: Cambridge University Press. Masterman, L., & Sharples, M. (2002). A theory-informed framework for designing software to support

reasoning about causation in history. Computers and Education, 38, 165-185. Mercer, N. (1996). The quality of talk in children’s collaborative activity in the classroom. Learning and

Instruction, 6 (4), 359-377. Ministerie van OCenW [Dutch Department of Education, Culture, and Science] (2005-a). Dossier Tweede

Fase. Retrieved July 25, 2005 from Ministerie van OCenW [Dutch Department of Education, Cul-ture, and Science] Web site: http://www.minocw.nl/tweedefase.

Ministerie van OCenW [Dutch Department of Education, Culture, and Science] (2005-b). Dossier ICT in het Onderwijs. Retrieved July 25, 2005 from Ministerie van OCenW [Dutch Secretary of Education, Culture, and Science] Web site: http://www.minocw.nl/ict.

96

Mynatt, C. R., Doherty, M. E., & Tweney, R. D. (1978). Consequences of confirmation and disconfirma-tion in a simulated research environment. Quarterly Journal of experimental Psychology, 30, 395-406.

Nastasi, B. K., & Clements, D. H. (1992). Social-cognitive behaviours and higher-order thinking in edu-cational computer environments. International Journal of Educational Research, 18 (3), 215-238.

Njoo, M. K. H. (1994). Exploratory learning with a computer simulation: Learning processes and in-structional support. Unpublished doctoral dissertation, Technische Universiteit Eindhoven, Eindho-ven, The Netherlands.

Njoo, M. K. H., & De Jong, T. (1993). Exploratory learning with a computer simulation for control the-ory: Learning processes and instructional support. Journal of Research in Science Teaching, 30 (8), 821-844.

O’Brien, D. P., Costa, G., & Overton, W. F. (1986). Evaluations of causal and conditional hypotheses. The Quarterly Journal of Experimental Psychology, 38 (3), 493

Okada, T., & Simon, H. A. (1997). Collaborative discovery in a scientific domain. Cognitive Science, 21 (2), 109-146.

Pekrun, R., Goetz, T., Titz, W., & Perry, R. P. (2002). Academic emotions in students’ self-regulated learning and achievement: A program of qualitative and quantitative research. Educational Psy-chologist, 37 (2), 91-105.

Pijls, M., Dekker, R. & Van Hout-Wolters, B. H. A. M. (2003). Mathematical level raising through col-laborative investigations with the computer. International Journal of Computers for Mathematical Learning, 8, 191-213.

Pintrich, P. R. (2000). An achievement goal theory perspective on issues in motivation terminology, theory, and research. Contemporary Educational Psychology, 25, 92-104.

Pintrich, P. R. & De Groot, E. V. (1990). Motivational and self-regulated learning components of class-room academic performance. Journal of Educational Psychology, 82 (1), 33-40.

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1993). Reliability and predictive validity of the motivated strategies for learning questionnaire (MSLQ). Educational and Psychological Measurement, 53, 801-813.

Ploetzner, R., Dillenbourg, P., Preier, M. & Traum, D. (1999). Learning by Explaining to Oneself and to Others. In: P. Dillenbourg (Eds.), Collaborative Learning. Cognitive and Computational Approaches (pp.103-121). Oxford: Elsevier Science Ltd.

Quinn, J., & Alessi, S. (1994). The effects of simulation complexity and hypothesis-generation strategy on learning. Journal of Research on Computing in Education, 27 (1), 75-87.

Reimann, P. (1991). Detecting functional relations in a computerized discovery environment. Learning & Instruction, 1, 45-65.

Repenning, A., Ioannidou, A., & Ambach, J., (1998). Learn to communicate and communicate to learn. Journal of Interactive Media in Education, 98/7. Retrieved September 3, 2005 from http://www-jime.open.ac.uk/98/7/repenning-98-7-paper.html.

Richardson, J. T. E. (2004). Methodological issues in questionnaire-based research on student learning in higher education. Educational Psychology Review, 16 (4), 347-358.

Roelofs, E., Van der Linden, J. & Erkens, G. (1999). Leren in dialoog. Een discussie over samenwerkend leren in onderwijs en opleiding. [Learning in dialog. A discussion of collaborative learning in educa-tion and training.] Pedagogische Studiën, 76 (6), 7-34.

Rojas-Drummond, S., & Mercer, N. (2003). Scaffolding the development of effective collaboration and learning. International Journal of Educational Research 39, 99-111.

Ross, J. A., & Cousins, J. B. (1995). Impact of explanation seeking on student achievement and attitudes. Journal of Educational Research, 89 (2), 109-117.

Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. American Psychologist, 55 (1), 68-78.

Saab, N., Van Joolingen, W. R., & Van Hout-Wolters B. H. A. M. (in press). Communication in collabo-rative discovery learning. British Journal of Educational Psychology.

Saab, N., Van Joolingen, W. R., & Van Hout-Wolters B. H. A. M. (submitted-a). Supporting communica-tion in a collaborative discovery learning environment;The effect of instruction.

Saab, N., Van Joolingen, W. R., & Van Hout-Wolters B. H. A. M. (submitted-b). Supporting collabora-tive discovery learning, a tool and its difficulties.

Saab, N., Van Joolingen, W. R., & Van Hout-Wolters B. H. A. M. (submitted-c). Motivation and collabo-rative discovery learning.

REFERENCES 97

Salomon, G. (1993). On the nature of pedagogic computer tools: The case of the writing partner. In S. P. Lajoie & S. J. Derry (Eds.), Computers as cognitive tools (179-196). Erlbaum, Hillsdale, NJ.

Salomon, G., & Globerson, T. (1989). When teams do not function the way they ought to. International Journal of Educational Research, 13, 89-98.

Scardamalia, M. & Bereiter, C. (1994). Computer support for knowledge building communities. The Journal of the Learning Science, 1, 37-68.

Schauble, L., Glaser, R., Raghavan, K., Reiner, M. (1992). The integration of knowledge and experimen-tation strategies in understanding a physical system. Applied Cognitive Psychology, 6 (4), 321-343.

Schunk, D. H., & Zimmerman, B. J. (1997). Social origins of self-regulatory competence. Educational Psychologist, 32 (4), 195-208.

She, H. C. (1999). Student’s knowledge construction in small groups in the seventh grade biology labora-tory: Verbal communication and physical engagement. International Journal of Science Education, 21 (10), 1051-1066.

Shute, V. J., & Glaser, R. (1990). A large-scale evaluation of an intelligent discovery world: Smithtown. Interactive Learning Environments, 1, 51-77.

Simon, H. A., & Lea, G. (1974). Knowledge and cognition. In L. W. Gregg (Eds.), Knowledge and Cog-nition (pp. 105-127). Lawrence Erlbaum Associates.

Simons, R., Van der Linden, J., & Duffy, T. M. (2000). New Learning. Dordrecht: Kluwer Academic Publishers.

Sins, P., Van Joolingen, W. R., Savelsbergh, E. R., Van Hout-Wolters, B. H. A. M. (submitted). Effects of face-to-face versus chat communication on performance in a collaborative computer-supported mod-eling task.

Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Re-search, 69, 21-51

Stahl, G. (2005). Group cognition in computer-assisted collaborative learning. Journal of Computer As-sisted Learning, 21, 79-90.

Stichting Ict op School [Foundation ICT at School] (2003). Sterke groei computergebruik in de klas.[High increase of computer use in the classroom.] NieuwsBulletin, 8.

Strijbos, J. (2004). The effect of roles on computer-supported collaborative learning. Unpublished Doc-toral dissertation, Open University, Heerlen, The Netherlands.

Swaak, J. (1998). What-if, discovery simulations and assessment of intuitive knowledge. Unpublished doctoral dissertation, University of Twente, Enschede, The Netherlands.

Swaak, J., & De Jong, T. (1996). Measuring intuitive knowledge in science: the development of the what-if test. Studies in Educational Evaluation, 22, 341-362.

Swaak, J., & De Jong, T. (2001). Discovery simulations and the assessment of intuitive knowledge. Jour-nal of Computer Assisted Learning, 20, 225-234.

Swaak, J., De Jong, T., & Van Joolingen, W. R. (2004). The effects of discovery learning and expository instruction on the acquisition of definitional and intuitive knowledge. Journal of Computer Assisted Learning, 20, 225-234.

Swing, S. R., & Peterson, P. L. (1982). The relationship of student ability and small-group interaction to student achievement. American Educational Research Journal, 19 (2), 259

Tao, P. K. & Gunstone, R. F. (1999). Conceptual change in science through collaborative learning at the computer. International Journal of Science Education, 21 (1), 39-57.

Van Boxtel, C. (2000). Collaborative concept learning. Collaborative learning tasks, student interaction, and the learning of physics concepts. Unpublished doctoral dissertation, Utrecht University, Utrecht, The Netherlands.

Van Boxtel, C., Van der Linden, J., & Kanselaar, G. (2000). Deep processing in a collaborative learning environment. In H. Cowie & G. M. Van der Aalsvoort (Eds.), Social Interaction in Learning and In-struction (pp.161-178). Amsterdam: Pergamon.

Van der Linden, J., Erkens, G., Schmidt H., & Renshaw, P. (2000). Collaborative Learning. In P.R.J. Simons, J. Van der Linden & T. Duffy (Eds.), New Learning. Dordrecht: Kluwer Academic Publish-ers.

Van Drie, J.,Van Boxtel, C., Jaspers, J., & Kanselaar, G. (2005). Effects of representational guidance on domain specific reasoning in CSCL. Computers in Human Behavior, 21, (4), 575-602.

Van Joolingen, W. R. (1999). Cognitive tools for discovery learning. International Journal of Artificial Intelligence in Education, 10 (3-4), 385-397.

98

Van Joolingen, W. R. & De Jong, T. (1991). Characteristics of simulations for instructional settings. Education & Computing, 6, 241-262.

Van Joolingen, W. R., & De Jong, T. (1993). Exploring a domain through a computer simulation: Trav-ersing variable and relation space with the help of a hypothesis scratchpad. In D. Towne, T. De Jong, & H. Spada (Eds.), Simulation-based Experiential Learning (pp. 191-206). Berlin: Springer.

Van Joolingen, W. R., & De Jong, T. (1997). An extended dual search space model of learning with com-puter simulations. Instructional Science, 25, 307-346.

Van Joolingen, W. R. & De Jong, T. (2003). SimQuest, authoring educational simulations. In T. Murray, S. Blessing, S. Ainsworth (Eds.), Authoring Tools for Advanced Technology Learning Environments: Toward cost-effective adaptive, interactive, and intelligent educational software (pp. 1-31). Dor-drecht, Kluwer.

Van Joolingen, W. R., De Jong, T., Lazonder, A. W., Savelsbergh, E. R., & Manlove, S. (2005). Co-lab: Research and development of an online learning environment for collaborative scientific discovery learning. Computers in Human Behavior, 21, 671.

Veenman, M. V. J., & Elshout, J. J. (1995). Differential effects of instructional support on learning in simulation environments. Instructional Science, 22, 363-383.

Veenman, M. V. J., Elshout, J. J., & Busato, V. V. (1994). Metacognitive mediation in learning with computer-based simulation environments. Computers in Human Behavior, 10, 93-106.

Veenman, M. V. J., & Elshout, J. J., & Meijer, J. (1997) The generality vs. domain specificity of meta-cognitive skills in novice learning across domains. Learning and Instruction, 7, 197-209

Veerman, A. (2000). Computer-Supported Collaborative learning Through Argumentation. Unpublished doctoral dissertation, Utrecht University, Utrecht, The Netherlands.

Veerman, A. L., Andriessen, J. E. B. & Kanselaar, G. (2000). Learning through synchronous electronic discussion. Computers & Education 34, 269-290.

Veermans, K. H. (2003). Intelligent support for discovery learning. Unpublished doctoral dissertation. University of Twente, Enschede, The Netherlands.

Veermans, K. H., De Jong, T., & Van Joolingen, W. R., (2000). Promoting self directed learning in simu-lation based discovery learning environments through intelligent support. Interactive Learning Envi-ronments, 8, 229-255.

Veermans, K. H., Van Joolingen, W. R., & De Jong, T. (submitted). Using heuristics to facilitate discov-ery learning in a simulation learning environment in a physics domain.

Walther, J. B. (1992). Relational communication in computer-mediated interaction. Human Communica-tion Research, 19 (1), 50-88.

Wason, P. C. (1960). On the failure to eliminate hypotheses in a conceptual task. Quarterly Journal of Experimental Psychology, 12, 129-140.

Wasson, B. (1998). Identifying coordination agents for collaborative telelearning. International Journal of Artificial Intelligence in Education, 9, 275-299.

Webb, N. M., & Farivar, S. (1994). Promoting helping behavior in cooperative small groups in middle school mathematics. American Educational Research Journal, 31 (2), 369-395.

Wegerif, R. (1996). Using computers to help coach exploratory talk across the curriculum. Computers and Education, 26 (1-3), 51-60.

Wegerif, R., & Mercer, N. (1996). Computers and reasoning through talk in the classroom. Language and Education, 10, 47-64.

Wegerif, R., & Mercer, N. (1997). Using computer based text analysis to integrate qualitative and quanti-tative methods in research on collaborative learning. Language and Education, 11 (4), 271-286.

Wegerif, R., Mercer, N., & Dawes, L. (1999). From social interaction to individual reasoning: an empiri-cal investigation of a possible socio-cultural model of cognitive development. Learning and Instruc-tion, 9, 493-516.

Weiss, G. & Dillenbourg, P. (1999). What is ‘multi’ in multi-agent learning? In: P. Dillenbourg, (Eds.), Collaborative learning. Cognitive and computational approaches (pp.246). Oxford, Elsevier Science Ltd..

White, B. Y., Shimoda, T. A., & Frederiksen, J. R., (1999) Enabling students to construct theories of collaborative inquiry and reflective learning: Computer support for metacognitive development. In-ternational Journal of Artificial Intelligence and Education, 10, 151-182.

Whitelock, D., Scanlon, E., Taylor, J., & O’Shea, T. (1995, October). Computer Support for Pupils Col-laborating: A Case Study on Collisions. Paper presented at CSCL ’95 (Computer Support for Col-laborative Learning), Bloomington, Indiana, pp. 380-384.

REFERENCES 99

Wigfield, A. (1994). Expectancy-value theory of achievement motivation: A developmental perspective. Educational Psychology Review, 6 (1), 49-78.

Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25, 68-81.

Zimmerman, B. J. (1998). Academic studying and the development of personal skill: A self-regulatory perspective. Educational Psychologist, 33 (2/3), 73-86.

Zimmerman, B. J. (2000). Self-efficacy: An essential motive to learn. Contemporary Educational Psy-chology, 25, 82-91.

Zusho, A., Pintrich, P. R., & Coppola, B. (2003). Skill and will: The role of motivation and cognition in the learning of college chemistry. International Journal of Science Education, 25 (9), 1081-1094.

AUTHOR INDEX

Akey, K. L., 63; 86 Alessi, S., 2 Ambach, J., 3 Andriessen, J. E. B., 4; 28; 29 Baines, E., 34; 53; 70 Baker M., 2; 3; 4; 5; 11; 28; 29; 43;

89 Bandura, A., 4; 29; 62; 63; 64 Barron, B., 4; 84 Bassok, M., 2; 4; 9; 11; 29 Bereiter, C., 9; 28; 62; 86 Black, J., 2 Blatchford, P., 34; 53; 70 Brewer, W. F., 2 Brown, J. S., 29 Bruinsma, M., 63 Burns, J. C., 2; 8 Burtis, J., 9; 28; 62 Busato, V. V., 4 Chan, C., 4; 9; 28; 62; 111; 121 Chi, M., 2; 4; 9; 11; 29 Chinn, C. A., 2 Choi, N., 63; 80; 81; 86 Clements, D. H., 3 Cohen, E. G., 2 Collins, A., 29 Coppola, B., 61 Costa, G., 2 Cousins, J. B., 28; 121 Covington, M. V., 63 Creemers, B. P. M., 29 Crowson, H. M., 63; 86 Damon, W., 5 Dawes, L., 4; 5; 28; 29; 43 ;85; 91 Deci, E. L., 62

De Groot, A. D., 45 De Groot, E. V., 5; 62; 63; 64; 69; 80; 81; 86; 114; 124 De Jager, B., 29 De Jong, T., 1; 2; 3; 4; 5; 6; 7; 8; 13; 15; 23; 31; 35; 45; 46; 47; 48; 51; 52; 60; 62; 66; 68; 71; 85; 87; 91; 109; 112; 119; 123 De Vries, E., 89 Dekker, R., 2; 9; 24; 28; 34 Dillenbourg, P., 2; 4; 9; 29 Doherty, M. E., 2 Doise, W., 3 Donovan, S. S., 2; 9; 27 Dubrovsky, W., 3 Duffy, T. M., 1; 7; 91; 109; 119 Duguid, P., 29 Duke, B. L., 63; 86 Dunbar, K., 2; 8; 25; 46; 84; 111; 121 Dweck, C. S., 63 Ebbens, S., 5; 29 Eccles, J. S., 5; 63 Elliot, A. J., 63 Elshout, J. J., 3; 4 Elshout-Mohr, M., 2; 4; 9; 24; 28 Erkens, G., 1; 2; 4; 7; 8; 9; 15; 24; 25; 27; 28; 29; 34; 52; 53; 62 Ettekoven, S., 4; 5; 28; 29 Farivar, S., 2; 4; 28; 29; 34; 43; 53;

85; 91; 111; 121 Fleiss, J. L., 35; 52 Frederiksen, J. R., 2 Galton, M., 34; 53; 70 Garcia, T., 5; 62; 63; 64; 81; 83; 86;

114; 124 Gijlers, A. H., 46; 60; 85; 112; 123

102

Glaser, R., 2; 3; 9; 29; 46 Globerson, T., 3; 4; 9; 28; 46; 84; 111; 116; 121; 126 Goetz, T., 64 Gorman, M. E., 2 Greene, B. A., 63; 64; 80; 81; 86; 114; 125 Guerrera, C., 5; 46 Gunstone, R. F., 11 Hansen, T., 4; 28; 29; 43 Hendricks, C. C., 29 Henri, F., 3 Hoek, D. J., 4; 91 Howe, C., 44; 85 Inspectie van het Onderwijs, 92 Ioannidou, A., 3 Issroff, K., 5; 62; 91 Järvelä, S., 5; 86 Jaspers, J., 15; 89 Johnson, D. W., 5 Johnson, R. T., 5 Joiner, T., 4; 28; 29; 43 Jonassen, D. H., 1; 5; 7; 46; 91; 109; 119 Jones, A., 5; 62; 91 Kanselaar, G., 4; 10; 15; 28; 29; 43;

89; 43 Kaplan, D., 2 Keselman, A., 2 King, A., 2; 4; 5; 29; 43; 85 Kiesler, S., 3 Klahr, D., 2; 8; 25; 84; 111; 121 Kozma, R. B., 85 Kuhn, D., 2 Kutnick, P., 34; 53; 70 Lajoie, S. P., 15; 46; 110; 112; 120;

122 Lavigne, N.C., 5; 46 Lazonder, A. W., 3; 5; 60; 62 Lea, G., 2 Lebie, L., 15; 35; 71

Legget, E. L., 63 Lehtinen, E., 5; 86 Lewis, M. W., 2; 4; 9; 11; 29 Limón, M., 3 Linnenbrink, E.A., 61; 62; 63; 64; 80; 81; 86; 114; 125 Löhner, S., 2 Lund, K., 3; 5; 89 Manlove, S., 3; 62 Marshall, S. P., 2; 9; 27 Masterman, L., 29 McGrath, J. E., 15; 35; 71 McKeachie, W. J., 5; 62; 86; 114; 124 Mercer, N., 4; 5; 9; 24; 28; 29; 43; 62; 85; 91; 110; 111; 120; 121 Miller, R. B., 63; 64; 80; 81; 86; 114; 125 Ministerie van OCenW, 90; 91 Mugny, G., 3 Munsie, S., 5; 46 Mynatt, C. R., 2 Nastasi, B. K., 3 Njoo, M. K .H., 2; 8; 23; 35; 45; 47; 51; 71; 85; 93; 109; 112; 119; 123 O'Brien, D. P., 2 Okada, T., 3; 10; 11; 25; 28; 46; 84; 88; 111; 116; 121; 126 Okey, J. R., 2; 8 Ootes, S. A. W., 5; 60 O'Shea, T., 3; 10; 46; 88 Overton, W. F., 2 Pekrun, R., 64 Pérez, J. A., 3 Perry, R. P., 64 Peters, N., 4; 29 Peterson, P. L., 4; 91 Phelps, E. 5 Pijls, M., 34 Pintrich, P.R., 5; 61; 62; 63; 64; 69; 73; 80; 81; 86; 114; 124; 125 Ploetzner, R., 2; 4; 9; 29

AUTHOR INDEX 103

Prangsma, M., 15 Preier, M., 2; 4; 9; 29 Quinn, J., 2 Raghavan, K., 2 Rahikainen, M., 5; 86 Reezigt, G. J., 29 Reimann, P., 2; 5; 9; 29; 46 Reiner, M., 2 Renshaw, P., 1; 7; 27; 53; 62 Repenning, A., 3 Rhoades, J. A., 15; 35; 71 Richardson, J. T. E., 80 Roelofs, E., 2; 9; 24; 27 Rojas-Drummond, S., 4; 28; 91 Ross, J. A., 28; 121 Ryan, R. M., 62 Saab, N., 28; 30; 34; 35; 44; 46; 49;

51; 62; 65; 66; 67; 70; 71; 80; 111; 121

Salomon, G., 3; 4; 5; 9; 28; 46; 84; 111; 116; 121; 126 Savelsbergh, E. R., 2; 3; 62; 89 Scanlon, E., 3; 10; 46; 88 Scardamalia, M., 86 Schauble, L., 2 Schmidt, H., 1; 7; 27; 53; 62 Schunk, D. H., 63; 64; 81; 86 Sethna, B. N., 3 Sharples, M., 29 She, H. C., 2; 4; 9; 29 Shimoda, T. A., 2 Shute, V. J., 2; 3; 46 Simon, H. A., 2; 3; 10; 11; 25; 28; 46; 84; 88; 111; 116; 121; 126 Simons, R., 7 Sins, P., 89 Smith, D. A. F., 5; 62; 86; 114; 124 Springer, L., 2; 9; 27 Stafford, A., 2 Stahl, G., 79 Stanne, M. E., 2; 9; 27 Stichting ICT op School, 1 Strijbos, J., 5; 91

Swaak, J., 6; 8; 15; 34; 51; 68; 87 Swing, S. R., 4; 91 Tao, P. K., 11 Taylor, J., 3; 10; 46; 88 Titz, W., 64 Tolmie, A., 44; 85 Traum, D., 2; 4; 9; 28; 29; 43 Tweney, R. D., 2 Van Boxtel, C., 2; 4; 9; 10; 26; 27;

29; 35; 43; 62; 89 Van der Linden, J., 1; 2; 4; 7; 9; 26; 27; 29; 34; 53; 62 Van Drie, J., 89 Van Hout-Wolters, 2; 28; 30; 34; 45; 44; 46; 49; 51; 62; 65; 66; 67; 70; 71; 80; 89; 111; 121 Van Joolingen, W. R., 1; 2; 3; 4; 5; 7; 8; 13; 15; 23; 28; 30; 31; 34; 35; 45; 46; 47; 48; 49; 51; 62; 65; 66; 67; 68; 70; 71; 80; 85; 87; 89; 91; 109; 110; 111; 112; 119; 120; 121; 122; 123 Van Rooijen, J., 5; 29 Veenman, M. V. J., 3; 4 Veerman, A., 4; 10; 26; 28; 29; 43 Veermans, K. H., 5; 13; 15; 31; 34; 46; 48; 51; 66; 68; 87 Walther, J. B., 3 Wason, P. C., 2 Wasson, B., 4; 28 Webb, N. M., 2; 4; 28; 29; 34; 43; 53;

85; 91; 111; 121 Wegerif, R., 2; 4; 5; 9; 24; 28; 29; 32; 43; 62; 85; 91 Weiss, G., 2; 4; 9; 29 White, B. Y., 2 Whitelock, D., 3; 10; 46; 88 Wigfield, A., 5; 62; 63 Wilhelm, P., 5; 60 Wise, K. C., 2; 8 Zimmerman, B. J., 63; 64; 81; 86 Zusho, A., 61

APPENDIX A

Complete analysis scheme for communicative activities


Informative Argumentative

Clarify Reason (because)/justify Condition (if, when) Consequent/conclusion (then, thus)/interpret Disjunctive (or) Counter (but, no + explanation) But,…

Evaluative (personal opinion towards partner, task or actions) To a person (positive, negative) To a task (positive, negative)

Elicitative (asking for the others response) Verification (checking) Understanding (checking) Agreement (checking) Clarification (checking) Open (new information) Continue after incomprehension

Responsive Informative Confirmation/acceptance Negative response

Directive Order for action Order for attention/focus Order for individual thinking (e.g. “Wait a minute”)

Asking for action Off task


106

APPENDIX B

The MSLQ

Intrinsic goal orientation • In a class like this, I prefer course material that really challenges me so I can

learn new things • In a class like this, I prefer course material that arouses my curiosity, even if it

is more difficult to learn. • When I have the opportunity in this class, I choose course assignments that I

can learn from even if they don't guarantee a good grade. • The most satisfying thing for me in this course is trying to understand the con-

tent as thoroughly as possible. Extrinsic goal orientation • Getting a good grade in this class is the most satisfying thing for me right now. • The most important thing for me right now is improving my overall grade point

average, so my main concern in this class is getting a good grade. • If I can, I want to get better grades in this class than most of the other students. • I want to do well in this class because it is important to show my ability to my

family, friends, employer, or others. • I always study for a good grade, whether I like the course or not. • I want to show my grades for this course to others. Task value beliefs • I like the subject matter in this course. • Understanding the subject matter of this course is very important to me. • It is important for me to learn the course material in this class. • I am very interested in the content area of this course.

Control of learning beliefs • If I study in appropriate ways, I will be able to learn the material in this course. • It is my own fault if I don't learn the material in this course. • If I try hard enough, I will understand the course material. • If I don't understand the course material, it is because I didn't try hard enough. • It doesn’t matter how hard I study, I either understand the subject matter or I

don’t. Self-efficacy of learning and performance • I'm certain I can understand the most difficult material presented in the readings

for this course. • I'm confident I can learn the basic concepts taught in this course.

107

• I'm confident I can understand the most complex material presented by the in-structor in this course.

• I'm confident I can do an excellent job on the assignments and tests in this course.

• I expect to do well in this class. • I'm certain I can master the skills being taught in this class. • Considering the difficulty of this course, the teacher, and my skills, I think I will

do well in this class. • I believe I will receive an excellent grade in this class. Test anxiety • When I take a test I think about items on other parts of the test I can't answer. • When I take tests I think of the consequences of failing. • I have an uneasy, upset feeling when I take an exam. • I feel my heart beating fast when I take an exam. • When I take a test I think about how poorly I am doing compared with other

students.

SUMMARY

These days, media for information and communication are used by almost every-body who is living in the Western society. Children grow up with different kinds of media, like television, computers and cell phones, and learn from childhood in their own natural way how to interact with these media. In schools, computers are also being used more and more. Thus, in a society where almost every aspect of life is controlled by this technology, it is important for young learners to learn and to be taught how to work effectively with technological devices.

Just as the society has changed with respect to the use of media, educators and theoreticians have changed their view of learning. In the last few decades, a shift has taken place from a traditional, teacher-centered view of learning in which informa-tion is transferred to the learner to a more constructivist view that is more learner-centered. Constructivist approaches to learning (Duffy & Jonassen, 1991; Jonassen, 1991, 2000) focus on learning environments in which learners have the opportunity to construct knowledge themselves and negotiate this knowledge with others. Dis-covery learning and collaborative learning are examples of learning contexts that cater for knowledge construction processes.

In collaborative scientific discovery learning, learners communicate and work together in a shared environment gathering data and use these data for joint knowl-edge construction. By altering variables and parameters and observing the effects, learners can uncover the rules that hold in a phenomenon they investigate, and in doing so, build new knowledge. Computer simulations are often used as safe and easily accessible phenomena to investigate (De Jong & Van Joolingen, 1998; Njoo, 1994; Njoo & De Jong, 1993).

In this dissertation, learners work with the simulation environment Collisions. In this environment, learners are presented with various simulations about colliding balls. For example, in one simulation, a ball collides against a wall. In another simu-lation, two balls collide against each other. Learners can conduct experiments by changing the mass or the velocity of such a ball. They can see how the collision takes place and see the effects of the collisions represented in the graphs.

The main aim of this dissertation is to focus on the interaction between the proc-esses of discovery learning and collaborative learning. It was assumed that a col-laborative environment would serve as a support for scientific discovery learning. However, as learners do not always engage in effective collaborative learning, we also supported the collaborative learning processes as well as the processes of dis-covery by instructing learners in effective communication skills and also by means of providing a cognitive tool aimed at assisting hypothesis generation and guiding learners through the discovery process. In addition, we investigated the role of moti-vation in the collaborative discovery process. The following research questions are central to this dissertation:

110

Which communicative activities and discovery activities are frequently used in a collaborative discovery learning environment? Can specific support of the communication process and the discovery learning process improve collaborative discovery learning? What is the role of motivation in collaborative discovery learning?

In this dissertation, four studies are presented in which these research questions were addressed. In these studies, secondary school students worked with a collaborative simulation-based learning environment called “Collisions”. Students worked col-laboratively in dyads on two different computers, using a shared screen and a chat channel to communicate. The learners saw the same windows on their computer screen and could discover the environment together. They could switch the control of the cursor by clicking the mouse.

CHAPTER 1

Chapter 1 gives a general introduction to this dissertation. In this chapter, it is argued that students find many aspects of the discovery learning process difficult and that collaborative learning can be used as a support for the discovery learning process. Furthermore, reasons for providing support of the collaborative discovery learning process are discussed. Different ways of providing support are possible and can be offered in various ways. It can be presented as an instruction that is given before or during the interaction with the learning environment (e.g., Mercer, 1996) or it can be built into the learning environment as cognitive tools (Lajoie, 1993; Van Joolingen, 1999). Moreover, the subject of investigating the relationship between motivation and collaborative discovery learning is introduced.

CHAPTER 2

In Chapter 2, a first explorative study is presented, in which the following research questions are addressed: 1) Which communicative activities between two learners, working collaboratively

in a computer-based discovery environment, are frequently used in the discov-ery learning process?

2) Which communicative and discovery activities co-occur during this learning process?

In Chapter 2, the relation between communication (chat) between dyads and the quality of the discovery learning process is analyzed. It was concluded that the most communicative activities occurred with the following discovery activities: proposing answers, thinking of alternative answers, designing experiments, and concluding. Thus, the most communication took place when, for example, the learners were de-

SUMMARY 111

signing an experiment. It was found that the communicative activity ‘argumentation’ occurred little and, counter to our expectations, that it co-occurred the most with the discovery activity, ‘drawing conclusions’, rather than with ‘generating hypotheses’. When generating hypotheses, learners appeared to simply exchange ideas, rather than arguing about them. Real discussion was postponed until the phase in which the learners drew conclusions. The postponement of discussion may be indicative of data-driven, inductive ‘experimenter’ behavior, in contrast to more deductive ‘theo-rist’ behavior, in which learners use their prior knowledge to generate hypotheses before carrying out an experiment (e.g., Klahr & Dunbar, 1988). Another finding was that high-performing dyads proposed more answers and generated more hy-potheses than dyads with poorer performance. This finding is in line with the results of a study by Okada and Simon (1997), in which pairs performed better than singles because they used more explanatory activities. Thus, exploratory activities, such as proposing answers and generating hypotheses, seem to be of importance in the learning process, as they result in good performance. Another finding was that dyads that performed well in the present study confirmed and accepted ideas more fre-quently than less successful dyads, indicating that they established agreement more often. It is concluded that further research should focus on means to augment com-municative and discovery activities that are related to positive learning outcomes.

CHAPTER 3

Chapter 3 reports a study in which the effect of a computerized instruction on collaboration in a collaborative discovery learning environment was investigated. Collaborative learning can contribute to better learning in problem solving situations (e.g., Mercer, 1996), as well as in discovery learning environments (Okada & Simon, 1997; Saab, Van Joolingen, & Van Hout-Wolters, in press (Chapter 2); Salomon, & Globerson, 1989). Although it may be fruitful to support learning by means of collaborative learning, collaboration itself may need support. It is not self-evident that learners know how to collaborate constructively. Several studies have shown that collaboration without instruction or support on how to collaborate does not lead automatically to effective knowledge construction (e.g., Chan, 2001; Mercer, 1996; Webb & Farivar, 1994). In this chapter the following questions are addressed: Can instruction in effective communication in a discovery learning environment lead to: 1) more effective communicative activities during the discovery learning process? 2) more effective discovery learning activities? 3) improved discovery learning results? The instruction we used, called RIDE, is built upon four principles identified in the literature on collaborative processes: Respect, Intelligent collaboration, Deciding together, and Encouraging. In an experimental study, a group of learners receiving this instruction was compared to a control group. Analyses of the logged actions in

112

the learning environment and the chat protocols showed that offering the RIDE in-struction proved to be effective, as students who received instruction in these rules used the activities associated with rules more, in particular Deciding together (D) and Encouraging (E). Students instructed in the RIDE rules asked for agreement more frequently, asked more openly and critically and asked more frequently for clarification of lack of understanding. They also more frequently gave informative answers, agreed more often, and asked their partner more often to perform an action in the learning environment. The dyads that received instruction also performed more discovery learning activities, such as generating hypotheses, concluding, and performed more regulative discovery activities. The findings in this study showed that offering the RIDE instruction can lead to more constructive communication, and more effective discovery learning activities, as expected, although no direct effect on discovery learning results was found. This study showed the benefits of provid-ing instruction on effective communication and the learning process in a collabora-tive discovery learning situation.

CHAPTER 4

Chapter 4 reports a study in which the effect of a cognitive tool to support the col-laborative discovery learning process was investigated. In a previous study (Chapter 2) it was found that communicative activities can contribute to essential stages in a collaborative discovery process. For example, directive and informative activities can contribute to the testing of hypotheses, whereas argumentation can lead to a suc-cessful process of concluding. This leads to the conjecture that when learners are encouraged to use these communicative activities, a more successful discovery process can be the result. But, even though the collaborative environment can con-tribute to the discovery process, specific process support remains necessary.

In the current study, such support is provided. Based on the concept of cognitive tools (Lajoie, 1993; Van Joolingen, 1999), the role of scaffolds for discovery proc-esses in collaborative learning environments is investigated. In general, cognitive tools help learners to carry out cognitive tasks. The tool presented in this study, the Collaborative Hypothesis Tool (CHT), is based on the hypothesis scratchpad, that was first used by Van Joolingen and De Jong (1991, 1993), as well as by Gijlers (2005). Stating hypotheses is a recognized difficult discovery process, as learners have problems to state syntactically correct hypotheses (Njoo & De Jong, 1993) and to state hypotheses that are testable (Van Joolingen & De Jong, 1997). The hypothe-sis scratchpad offers a template that helps learners in stating hypotheses, ensuring that they are syntactically correct. The tool was adapted in two ways. The first was to embed the tool into the discovery process as a whole, by adding prompts that re-minded learners of actions to undertake, such as stating a hypothesis or to gather experimental data. The second adaptation was towards using the tool in a collabora-tive environment, by adding specific opportunities for argumentation about the hy-potheses stated and whether they should be tested or not. It was hypothesized that the tool would contribute to learning in the collaborative discovery environment.

SUMMARY 113

Offering the hypothesis template and prompting learners to perform the discovery learning process should lead to well-stated hypothesis, as well as a higher quality discovery process, expressed in more transformative discovery activities. Asking learners to make their opinions on hypotheses explicit should lead to more argumen-tative communication processes between learners. In total this should lead to im-proved learner outcomes. The results showed that the hypothesis that this tool would result in more argumentation, better discovery learning processes as well as im-proved learning results could not be confirmed. We found no significant differences between groups on any of these output measures. The use of the tool was not manda-tory, which resulted in its not being used very much. However, in the cases where the tool was used, learners used more effective communicative activities and discov-ery learning activities.

Results seem to indicate that these activities, especially communicative activities connected to the Deciding together rule and regulative activities, were used more effectively by the experimental group, so when they decided more together or when they regulated their learning process more, this led to better collective learning re-sults.

A likely explanation for the little use that was made of the CHT in the experi-mental group is that learners themselves did not see obvious benefit in using it. One cause may be that use of the CHT costs time and resources. Apparently, learners are inclined to use a tool only when they see direct benefit or when there is pressure to use it. The prompts that were used to stimulate use did not have the desired effect.

Combining the finding that the tool was little used and the small indications that its use may have some benefit, we must seek for situations where a new version of the tool will be more likely used. There are two candidate options for trying to achieve this. The first option is to place the scratchpad more prominent in the basic working procedure of the students, effectively making its use mandatory. The sec-ond option is to provide learners with instruction and an opportunity to practice with the tool before working with the learning environment. This will increase their knowledge of how and when to use the tool effectively. Future research should be aimed investigating these two options.

CHAPTER 5

In Chapter 5, a study is reported in which the role of learners’ motivation in relation to the learning process and learning product in collaborative scientific discovery learning environments is addressed. The first objective of the study was to investi-gate the relationship between the level of motivation and both the individual com-munication and discovery learning process and learning product. The second objec-tive was to examine whether team heterogeneity in terms of motivation influences the collective learning process and learning product. In this chapter, the following questions are addressed:

114

1) In what way do separate motivation variables of individuals relate to their indi-vidual communicative and scientific discovery learning process and learning product?

2) In what way does a heterogeneous or homogeneous group composition of dyads with respect to motivation, relate to the group’s communicative and scientific discovery learning process and learning product?

To measure motivation the “Motivated Strategies for Learning Questionnaire” (MSLQ) (Pintrich & De Groot, 1990; Pintrich, Smith, Garcia, & McKeachie, 1993) was used. The MSLQ is based on an expectancy value model and includes the moti-vational constructs: value, expectancy and affect.

The results showed that level of motivation for the constructs value and expec-tancy was related to individual learning, rather than to learning at group level, ex-cept in the case of the motivation variable task value beliefs, which belongs to the motivation construct value. Dyads with two participants with ‘high’ task value be-liefs communicated more and used more regulative discovery activities than dyads with two participants with ‘low’ task value beliefs, or than heterogeneously com-posed dyads. This means that dyads that found working on the task important en-gaged in more effective communication and regulation.

In the current study motivation did not appear to be positively related to learning results. In this study, performance was measured in terms of performance on the specific task, rather than in terms of general grades. Hence, the relation between school achievement and motivation found in earlier research (e.g., Greene & Miller, 1996; Linnenbrink & Pintrich, 2002; Pintrich, et al., 1993) could not be replicated in this study at the smaller scale of task performance. However, similar positive rela-tionships between motivation and school achievement were also found in this study, when looking at the relationship between motivation and school grades. The meas-urement of school grades includes a number of tasks carried out over an extended period of time, like doing homework and completing various tests, whereas the learning product measures used in this dissertation assess the learning outcomes of a single task carried out within a limited period of time.

For the third motivational construct, affect, text anxiety was measured. In line with what was expected, learners with little or no fear or anxiety about the task communicated more and performed more transformative and regulative activities than learners with a higher level of fear or anxiety. Low-anxiety learners also per-formed better on the individual posttests.

To sum up, three general conclusions can be drawn. Firstly, for our context, the motivational beliefs of the learners seem to influence learning at the individual rather than at the group (dyad) level. Secondly, test anxiety is found to be related to all aspects of the individual learning process and learning product, and not to the collective learning process or learning product. Thirdly, task value belief is the only motivational variable that is related to both the individual learning process and col-lective learning process. In future research, it would be interesting to investigate the relationship between task value beliefs and the collaborative discovery learning process more thoroughly.

SUMMARY 115

CHAPTER 6

In Chapter 6, an overview of results of the four studies as well as suggestions for future research are presented, followed by a discussion of general findings, in which the similarities and differences in the findings of the four studies are highlighted. General considerations and limitations of the four studies are also discussed. The chapter concludes with a discussion of the educational implications of the studies.

In the general discussion, firstly, the measurement and coding of the communica-tion process and the discovery process are discussed. The protocols were analyzed using the analysis scheme described in Chapter 2. The protocols were scored on three dimensions: communicative activities, transformative discovery activities and regulative discovery activities. It is argued that using this analysis scheme to code the protocols could have been problematic if the communicative and discovery process dimensions were dependent on each other, for instance, if a particular com-municative action was always scored as the same discovery activity. However, in-spection of the data showed that no such couplings occurred, and thus that the scor-ing dimensions really were independent of each other.

Secondly, the reliability of the measurements of the learning products is dis-cussed. The studies in this dissertation produced varying reliabilities with respect to the individual learning result measure. The number of items and the number of par-ticipants are features that may have affected test reliability. Another possible expla-nation for the low reliabilities can be found in the rather low scores on the tests, es-pecially for the pretests, which in most cases were just above chance level. Another comment that was made on the measurement of learning results is that the collective learning result (Score within the learning environment; SWLE), is a learning result measure obtained during the learning process. Therefore, it can be seen as part of the learning process.

Thirdly, cause-effect relations between communicative activities and discovery activities are discussed. The research reported in this dissertation showed that there was a positive relationship between communicative activities and discovery activi-ties. However, because of the use of correlational analyses, no firm conclusions with respect to the causal direction of the support can be drawn.

Next, three general considerations central to the reported research are discussed. The first one concerns the salience of regulative discovery and communicative ac-tivities in the various studies. A general finding of this dissertation is that regulative activities and communicative activities associated with the rules Deciding together and Encouraging are essential activities in the computer-supported collaborative discovery environment used in this dissertation.

The second consideration concerns the function of communication in a com-puter-supported environment. Some communicative and discovery activities as-sumed to be necessary in the collaborative discovery learning process turned out to be less effective or less frequently used in the studies reported in this dissertation. A possible explanation for the infrequent occurrence of these activities is that learners not only communicated by chatting with each other, but that they also communi-cated through the environment, by performing actions in the environment.

116

The third consideration concerns the knowledge and skills learners may have ac-quired while working with the learning environment. The results of the study in Chapter 5, in which all data were merged, showed that, in overall terms, working with the learning environment did lead to an increase in domain knowledge or de-clarative knowledge. The difference between the pre- and the posttests was signifi-cant, which means that working with the learning environment had a positive effect on level of domain knowledge. However, although these effects were significant, they were only small. A possible explanation for the small size of the improvements is that, rather than acquiring a large amount of declarative knowledge, the learners may have primarily acquired discovery skills, collaborative skills or skills relevant only to the specific environment.

Suggestions for future research are made throughout the chapter. It is suggested that future research needs to focus on the probable causal relationship between collaborative learning and discovery learning. Collaboration may have a positive influence on the discovery learning process (Okada & Simon, 1997; Salomon & Globerson, 1989). The results of the research reported here can serve as a basis for the generation of new hypotheses.

With respect to the relationship between motivation and collaborative discovery learning, future study is recommended in which motivation is measured at different times while learners are working with the learning environment, and in which the relationship between motivation at these different times and the activities in the collaborative discovery learning process are investigated. Furthermore, as it is found that regulative activities and communicative activities associated with the rules Deciding together and Encouraging are essential activities in the computer-supported collaborative discovery environment used in this dissertation, future research examining the use of these specific activities in collaborative discovery learning environments is recommended. Moreover, a potentially fruitful direction for future research into the support of collaborative discovery learning processes could involve instruction designed to stimulate the use of these specific activities.

Another final relevant question for future research concerns the kinds of skills and knowledge learners acquire when working with the collaborative discovery en-vironment used in this dissertation. In future studies, learners could also be inter-viewed to obtain information concerning their perception of the learning environ-ment in order to find out what their learning goal is.

Subsequently, guidelines for educational practice are discussed. The research in this dissertation has tried to take a step in the right direction by providing some in-sight into how students’ learning processes can be supported. The RIDE rules that learners were taught were general communication and collaboration rules. Hence, they could also be applied in a face-to-face learning situation or in a different col-laborative learning environment in which learners have to communicate by means of chat. However, for use in collaborative learning situations other than the current one, the instruction would need to be constructed in such a way that it models the situa-tion of the learning environment used. Another implication of the research in this dissertation is that when support for learning processes is provided in a collaborative discovery environment, extensive training with sufficient practice opportunities should precede working with the learning environment, in order to reduce the likeli-

SUMMARY 117

hood of learners experiencing cognitive overload. In this dissertation, it is posed that in order to be able to participate effectively in today’s Western society, people need to possess well-developed social and communicative skills. This claim is also recog-nized by the Dutch government. A most obvious conclusion is to teach people those necessary skills during the school years.

SAMENVATTING

Tegenwoordig worden media voor informatie en communicatie door bijna iedereen in de Westerse maatschappij gebruikt. Kinderen groeien thuis op met verschillende soorten media, zoals televisie, computers en mobiele telefoons, en leren van jongs- af aan al doende met deze middelen om te gaan. Ook in het onderwijs worden com-puters steeds meer gebruikt ter ondersteuning van het leren. In een maatschappij waar bijna elk aspect van het leven wordt beheerst door technologie, neemt com-putergestuurd onderwijs een steeds belangrijker plaats in.

Net zoals het mediagebruik in de maatschappij is veranderd, hebben theoretici en onderwijzers hun visie op leren veranderd. De laatste jaren is er een verschuiving opgetreden van een traditionele, docentgerichte visie op leren, waarbij informatie wordt overgedragen aan leerlingen, naar een meer constructivistische visie op leren, die meer leerling-gericht is. Deze constructivistische benaderingen van leren (Duffy & Jonassen, 1991; Jonassen, 1991, 2000) zijn gericht op leeromgevingen waarin leerlingen de mogelijkheid hebben zelf kennis te construeren en deze kennis te be-discussiëren met anderen. Ontdekkend leren en samenwerkend leren passen bij deze visie op leren.

In dit proefschrift worden samenwerkend leren en ontdekkend leren samenge-voegd en richt het onderzoek zich op samenwerkend ontdekkend leren in een com-putergestuurde leeromgeving. In samenwerkend ontdekkend leren communiceren en werken leerlingen met elkaar in een gedeelde leeromgeving, terwijl ze gegevens (hierna: data) verzamelen en deze data gebruiken voor gezamenlijke kennisconstruc-tie. Door middel van het veranderen van variabelen en parameters en het observeren van de effecten kunnen leerlingen de regels ontdekken achter het model dat zij aan het onderzoeken zijn, en op deze manier nieuwe kennis construeren. Computersimu-laties zijn vaak gebruikt als een veilige en gemakkelijke omgeving om te onder-zoeken (De Jong & Van Joolingen, 1998; Njoo, 1994; Njoo & De Jong, 1993).

In dit proefschrift werken leerlingen met de computersimulatie Botsingen. In de-ze simulatie worden verschillende simulaties over balletjes die botsen aangeboden. Er is bijvoorbeeld een simulatie waarin een balletje tegen een muur botst en een an-dere simulatie waar twee balletjes tegen elkaar botsen. Leerlingen kunnen experi-menten uitvoeren door de massa of de snelheid van zo’n balletje te veranderen. Ver-volgens kunnen zij zien hoe de botsing plaatsvindt en kunnen zij de effecten van die verandering terugzien in de weergegeven grafieken.

Het hoofddoel van dit proefschrift is het onderzoeken van de interactie tussen ontdekkend leren en samenwerkend leren. Aangenomen wordt dat een samenwerk-end leeromgeving ondersteuning biedt aan ontdekkend leren. Omdat leerlingen niet altijd effectief samenwerken, worden in de beschreven studies zowel samenwerkend leren als ontdekkend leren ondersteund. Enerzijds met behulp van een instructie in effectief communiceren, en anderzijds door het aanbieden van een cognitive tool die

120

de leerlingen kan ondersteunen bij het genereren van hypothesen en die leerlingen kan leiden door het ontdekkend leerproces. Daarnaast is de functie van motivatie in het samenwerkend ontdekkend leerproces onderzocht. De volgende onderzoeks-vragen staan centraal in het proefschrift: Welke communicatieve activiteiten en ontdekkend leeractiviteiten worden frequent gebruikt in een samenwerkend ontdekkend leeromgeving? Kan specifieke ondersteuning van het communicatieproces en het ontdekkend leer-proces het samenwerkend ontdekkend leren verbeteren? Wat is de functie van motivatie in samenwerkend ontdekkend leren? In dit proefschrift worden vier studies gepresenteerd waarin deze onderzoeksvragen onderzocht worden. In deze studies werkten leerlingen uit 4 VWO in een samen-werkend leeromgeving gebaseerd op een simulatie, Botsingen genoemd. De leer-lingen werkten in tweetallen samen, ieder op een eigen computer. Om te kunnen samenwerken en communiceren deelde ieder tweetal de interface en een chatbox. Op deze manier konden beide leerlingen van een tweetal hetzelfde zien op het com-puterscherm en konden zij dus samen de leeromgeving verkennen. De controle over de cursor kon overgenomen worden door te dubbelklikken op de muis.

HOOFDSTUK 1

Hoofdstuk 1 geeft een algemene inleiding tot dit proefschrift. In dit hoofdstuk wordt uiteengezet dat leerlingen verschillende aspecten van het ontdekkend leerproces ingewikkeld vinden en dat samenwerkend leren dit proces kan ondersteunen. Daar-naast worden er redenen aangedragen waarom ook het samenwerkend leren onder-steund zou moeten worden. Verschillende manieren van ondersteuning zijn mogelijk en kunnen op verschillende manieren worden aangeboden. Ondersteuning kan aan-geboden worden als een instructie vooraf of tijdens het leren in de omgeving (bijv. Mercer, 1996). Ondersteuning kan ook ingebouwd worden in de leeromgeving als een cognitive tool (Lajoie, 1993; Van Joolingen, 1999). Vervolgens wordt in de in-leiding het onderzoek naar de relatie tussen motivatie en samenwerkend ontdekkend leren geïntroduceerd.

HOOFDSTUK 2

In hoofdstuk 2 wordt het eerste exploratieve onderzoek gepresenteerd, waarin de volgende onderzoeksvragen behandeld worden: 1) Welke communicatieve activiteiten worden vaak gebruikt tijdens het ontdekkend

leerproces door twee leerlingen die samenwerken in een computergestuurd ont-dekkend leeromgeving?

2) Welke communicatieve activiteiten en ontdekkend leeractiviteiten komen samen voor in dit leerproces?

SAMENVATTING 121

In dit onderzoek werd de relatie tussen communicatie (chat) tussen leerlingen en de kwaliteit van het ontdekkend leerproces onderzocht. Er kon geconcludeerd worden dat de meeste communicatieve activiteiten samen met de volgende ontdekkend leer-activiteiten gebruikt werden: antwoorden voorstellen, bedenken van alternatieve antwoorden, experimenten ontwerpen en concluderen. Er werd dus het meest ge-communiceerd wanneer de leerlingen bijvoorbeeld een experiment ontwierpen. De communicatieve activiteit ‘argumenteren’ werd niet veel gebruikt, en als de ac-tiviteit wel gebruikt werd, ging dat meestal samen met de ontdekkend leeractiviteit ‘concluderen’, in plaats van met het genereren van hypothesen, zoals verwacht werd. Wanneer leerlingen hypothesen genereerden, wisselden ze eerder ideeën uit dan erover te discussiëren. Het echte discussiëren werd uitgesteld tot het moment dat zij conclusies trokken. Dit uitstellen van discussie kan wijzen op inductief, experi-menter gedrag, in tegenstelling tot meer deductief theorist gedrag, waar leerlingen hun voorkennis gebruiken bij het genereren van hypothesen, voordat zij een experi-ment uitvoeren (bijv. Klahr & Dunbar, 1988). Een ander resultaat was dat tweetallen met betere leerresultaten meer antwoordvoorstellen deden en meer hypothesen genereerden, dan tweetallen met slechtere leerresultaten. Dit resultaat bevestigde de resultaten van een onderzoek van Okada en Simon (1997), waar tweetallen beter presteerden dan leerlingen alleen, omdat zij meer verklarende activiteiten gebruik-ten. Dus, verklarende activiteiten zoals het doen van antwoordvoorstellen en het genereren van hypothesen blijken belangrijk in het leerproces omdat zij kunnen lei-den tot goede leerprestaties. Daarnaast bleek dat tweetallen die goed presteerden in dit onderzoek meer bevestigden en accepteerden dan minder succesvolle tweetallen. Toekomstig onderzoek zou daarom gericht kunnen worden op het bevorderen van communicatieve en ontdekkend leeractiviteiten die gerelateerd zijn aan goede leer-prestaties.

HOOFDSTUK 3

Hoofdstuk 3 beschrijft een studie waarin het effect van een computergestuurde instructie voor effectief samenwerken in een samenwerkend ontdekkend leeromgeving wordt onderzocht. Samenwerkend leren kan leiden tot beter leren tijdens het problemen oplossen (bijv. Mercer, 1996), maar ook tijdens het ontdekkend leren (Okada & Simon, 1997; Saab, Van Joolingen, & Van Hout-Wolters, in press; Salomon & Globerson, 1989). Ook al is het lonend om het leerproces te ondersteunen door middel van samenwerkend leren, samenwerkend leren heeft op haar beurt ook ondersteuning nodig. Het is namelijk niet vanzelfsprekend dat leerlingen bij voorbaat al weten hoe zij constructief kunnen samenwerken. Verschillende studies hebben aangetoond dat samenwerken zonder instructie of ondersteuning in het samenwerken niet automatisch leidt tot effectieve kennisconstructie (Chan, 2001; Mercer, 1996; Ross & Cousins, 1995; Webb & Farivar, 1994). In dit hoofdstuk worden de volgende vragen behandeld:

122

Kan een instructie in effectief communiceren in een ontdekkend leeromgeving leiden tot: 1) het gebruik van meer effectieve communicatieve activiteiten gedurende het ont-

dekkend leerproces? 2) het gebruik van meer effectieve ontdekkend leeractiviteiten? 3) betere ontdekkend leerresultaten? De instructie die gebruikt werd, RIDE, is opgebouwd uit vier regels gebaseerd op de literatuur over samenwerkend leren: Respect, Intelligent samenwerken, Samen bes-lissen en Aanmoedigen. De experimentele groep die de instructie kreeg, werd verge-leken met een controlegroep, die een dummy instructie kreeg over logisch redene-ren. De analyses van de opgeslagen activiteiten in de leeromgeving en de chat proto-collen lieten zien dat het aanbieden van de RIDE instructie effectief was, omdat de leerlingen die de instructie kregen meer communicatieve activiteiten gebruikten die bij de regels hoorden, met name de regels Samen beslissen en Aanmoedigen. Leer-lingen die de instructie kregen, vroegen meer om overeenstemming, stelden meer open en kritische vragen en vroegen meer om verduidelijking wanneer ze iets niet begrepen. Zij gaven ook meer informatieve antwoorden, accepteerden elkaars ideeën vaker en vroegen hun partner vaker iets in de leeromgeving te doen. De tweetallen die de RIDE instructie ontvingen gebruikten ook meer ontdekkend leeractiviteiten, zoals het genereren van hypothesen, concluderen en regulatieve ontdekkend leerac-tiviteiten. De resultaten van dit onderzoek laten zien dat de RIDE instructie kan lei-den tot het gebruik van meer constructieve communicatie en meer effectieve ont-dekkend leeractiviteiten, hoewel de tweetallen die de instructie kregen niet beter scoorden op het gezamenlijke leerresultaat dan de tweetallen die geen RIDE instruc-tie hadden gekregen. Dit betekent dat het leerproces bij instructie van de RIDE re-gels wel verbeterde, maar dat er geen direct effect op het ontdekkend leerresultaat is gevonden.

HOOFDSTUK 4

Hoofdstuk 4 beschrijft een studie waarin het effect van een cognitive tool ter ondersteuning van het samenwerkend ontdekkend leerproces is onderzocht. Eén van de resultaten in een vorige studie (hoofdstuk 2) was dat communicatieve activiteiten een positieve bijdrage kunnen leveren aan essentiële fasen in het samenwerkend ontdekkend leerproces. Directieve en informatieve communicatieve activiteiten bijvoorbeeld, kunnen bijdragen aan het proces van het testen van hypothesen, terwijl argumentatie kan bijdragen aan het proces van concluderen. Dit leidt tot de veronderstelling dat een meer succesvol ontdekkend leerproces kan plaatsvinden wanneer leerlingen gestimuleerd worden deze communicatieve activiteiten te gebruiken. Maar, ook al kan samenwerken voordelig zijn voor het ontdekkend leerproces, specifieke ondersteuning van het leerproces blijft nodig.

In de huidige studie werd de leeromgeving voorzien van zo’n specifieke ondersteuning. Gebaseerd op het concept van cognitive tools (Lajoie, 1993; Van Joolingen, 1999) werd de rol van ondersteuning van het ontdekkend leerproces in

SAMENVATTING 123

samenwerkend leeromgevingen onderzocht. Cognitive tools dienen om leerlingen te helpen cognitieve taken uit te voeren. Het ontwerp van de tool die werd gebruikt in deze studie, de Collaborative Hypothesis Tool (CHT), is gebaseerd op het hypothese kladblok dat voor het eerst in een onderzoek gebruikt werd door Van Joolingen en De Jong (1991, 1993), en daarnaast ook door Gijlers (2005). Hypothese vorming is erkend als een moeilijk proces in het ontdekkend leren, omdat leerlingen zowel moeite hebben met het vormen van syntactisch correcte hypothesen (Njoo & De Jong, 1993) als met het vormen van hypothesen die getest kunnen worden (Van Joolingen & De Jong, 1997). Het hypothese kladblok biedt de leerlingen een sjabloon aan dat helpt bij het vormen van syntactisch correcte hypothesen. In deze studie is de tool op twee manieren aangepast. Ten eerste was de tool in de leeromgeving ingebed om het hele ontdekkend leerproces te ondersteunen. Dit is gedaan door tijdens het ontdekkend leerproces de leerlingen telkens hints en strategieën aan te bieden in de vorm van zogenaamde prompts die de taak hadden leerlingen te helpen herinneren aan het uitvoeren van bepaalde acties, zoals het vormen van een hypothese of het verzamelen van data. De tweede aanpassing was gericht op het gebruik van de tool in een samenwerkend leeromgeving. De verwachting was dat de leerlingen meer zouden argumenteren over het maken van hypothesen, door iedere leerling van een tweetal de mogelijkheid te geven in het kladblok een hypothese te vormen en vervolgens elkaars hypothesen te beoordelen. De veronderstelling was namelijk dat het gebruik van de tool positief zou bijdragen aan het leerproces in de samenwerkend ontdekkende leeromgeving.

Het aanbieden van het hypothese sjabloon en het prompten van leerlingen tot het uitvoeren van ontdekkend leeractiviteiten zouden kunnen leiden tot goed gefor-muleerde hypothesen, meer argumentatie over de hypothesen en tot een hogere kwaliteit van het ontdekkend leerproces, wat tot uiting zou kunnen komen in gebruik van meer ontdekkend leeractiviteiten. Dit alles zou kunnen leiden tot verbeterde leerresultaten. De resultaten lieten echter zien dat de veronderstelling dat de tool zou leiden tot meer argumentatie, een beter ontdekkend leerproces of betere leerresul-taten, niet bevestigd kon worden. Er werden met betrekking tot deze variabelen geen significante verschillen gevonden tussen de experimentele groep die de tool aange-boden kreeg en de controle groep. Een mogelijke verklaring hiervoor is dat het ge-bruik van de tool niet verplicht was, waardoor leerlingen hiervan weinig gebruik hebben gemaakt. Desondanks pasten de leerlingen op momenten dat zij de tool wel gebruikten, meer effectieve communicatieve activiteiten en ontdekkend leerac-tiviteiten toe.

De resultaten lieten vervolgens zien dat deze activiteiten, met name de commu-nicatieve activiteiten gerelateerd aan de regel Samen beslissen en de regulatieve activiteiten, effectiever werden gebruikt door de experimentele groep. Dus wanneer tweetallen meer samen beslisten of wanneer zij hun leerproces meer reguleerden, scoorden zij beter op het gezamenlijke leerresultaat.

Een voor de hand liggende verklaring voor het geringe gebruik van de CHT in de experimentele groep is dat de leerlingen er geen voordeel in zagen om de tool te gebruiken. Een reden kan zijn dat het gebruik van de tool extra tijd kost. Blijkbaar zijn leerlingen alleen bereid een tool te gebruiken wanneer zij direct voordeel zien of wanneer er druk wordt uitgeoefend van buitenaf om de tool te gebruiken. De

124

prompts die waren ingezet om de leerlingen te stimuleren het collaboratieve hy-pothese kladblok te gebruiken hadden niet het gewenste effect.

Het feit dat de tool weinig werd gebruikt gecombineerd met kleine aanwijzingen dat het gebruik van de tool kan leiden tot een beter leerproces, leidt tot de conclusie dat er gezocht moet worden naar situaties waarin het waarschijnlijk is dat een nieu-we versie van de tool wel gebruikt zal worden. Om zo’n situatie te bereiken, zijn twee opties mogelijk. De eerste optie is om het gebruik van het kladblok verplicht te maken door op een meer prominente manier de tool te presenteren in de leerom-geving, bijvoorbeeld door de leerlingen te verplichten een hypothese te maken voor-dat zij een experiment kunnen uitvoeren. De tweede optie is om de leerlingen meer instructie te geven en ze te laten oefenen met de tool voordat ze met de leerom-geving aan de slag gaan. Dit zal de leerlingen meer bewust maken van wanneer en hoe zij de tool effectief kunnen gebruiken. Toekomstig onderzoek zal deze twee opties kunnen bestuderen.

HOOFDSTUK 5

In hoofdstuk 5 wordt een studie beschreven waarin de functie van de motivatie van de leerlingen wordt geanalyseerd in relatie tot het leerproces en het leerproduct in samenwerkend ontdekkend leeromgevingen. Het eerste doel van de studie was het onderzoeken van de relatie tussen aan de ene kant motivatie van de leerlingen en aan de andere kant zowel het gebruik van hun communicatieve en ontdekkend leerac-tiviteiten als de scores op de (individuele) natoets. Het tweede doel van de studie was het onderzoeken of heterogeniteit in teams met betrekking tot motivatie het gezamenlijke leerproces en leerproduct beïnvloedt. In dit hoofdstuk worden de vol-gende vragen behandeld: 1) Wat is de relatie tussen de verschillende motivatievariabelen van individuen en

hun individuele communicatieve en ontdekkend leerproces en leerproduct? 2) Wat is de relatie tussen een heterogene of homogene groepssamenstelling van

tweetallen met betrekking tot motivatie en het communicatieve en ontdekkend leerproces en leerproduct van de groep?

De vragenlijst die gebruikt werd om motivatie te meten was de ‘Motivated Strategies for Learning Questionnaire’ (MSLQ) (Pintrich & De Groot, 1990; Pintrich, Smith, Garcia, & McKeachie, 1993). Deze vragenlijst is gebaseerd op een expectancy-value model en bevat de volgende motivatie-elementen: value (waarde), expectancy (verwachting) en affect (emotie). De motivatie-elementen bevatten weer motivatievariabelen. Het motivatie-element waarde bestaat bijvoorbeeld uit de motivatievariabelen intrinsieke doeloriëntatie, die de mate van inhoudelijke interesse voor de stof weergeeft, extrinsieke doeloriëntatie, die de mate van behoefte aan externe waardering door medeleerlingen, docenten of familie, of van behoefte aan externe waardering in de vorm van cijfers of geld weergeeft, en taakwaardering.

De resultaten lieten zien dat het niveau van motivatie met betrekking tot de ele-menten waarde en verwachting eerder gerelateerd was aan het individuele leren dan aan het leren op groepsniveau, met uitzondering van de variabele taakwaardering.

SAMENVATTING 125

Tweetallen waarvan beide leerlingen hoge taakwaarderingen hadden, communiceer-den meer en gebruikten meer regulatieve activiteiten dan tweetallen met lage taak-waarderingen of heterogeen samengestelde tweetallen. Dit betekent dat tweetallen die de taak interessant en belangrijk vonden, meer effectieve communicatie gebruik-ten en hun leerproces meer reguleerden.

Motivatie bleek in deze studie niet positief gerelateerd te zijn aan leerresultaten. Dit betekent dat de relatie tussen schoolresultaten en motivatie gevonden in eerder onderzoek (bijv. Greene & Miller, 1996; Linnenbrink & Pintrich, 2002; Pintrich et al., 1993) niet gerepliceerd konden worden in deze studie op het gebied van taak-gerelateerde resultaten. Echter, vergelijkbare positieve relaties tussen motivatie en schoolresultaten werden ook in deze studie gevonden, nadat de relatie tussen moti-vatie en schoolcijfers werd bekeken. Het meten van leerresultaten met behulp van algemene cijfers omvat, naast het presteren op de taak zelf, de uitvoering van meer-dere taken verspreid over een langere periode, zoals het huiswerk maken en preste-ren op meerdere toetsen. De meetinstrumenten die in deze studie gebruikt waren, maten daarentegen de leerresultaten van één taak die uitgevoerd was in een korte periode.

Het derde motivatie-element, emotie, is gemeten door testangst te meten. Zoals verwacht werd, communiceerden leerlingen met weinig of geen testangst meer en gebruikten zij meer ontdekkend leeractiviteiten dan leerlingen met meer testangst. Leerlingen met weinig testangst presteerden ook beter op de individuele natoets.

Samenvattend kunnen drie conclusies getrokken worden. Ten eerste blijkt moti-vatie van de leerlingen eerder het leren op individueel niveau dan op groepsniveau te beïnvloeden. Ten tweede is testangst gerelateerd aan alle aspecten van het individu-ele leerproces en leerproduct en niet aan het gezamenlijke leerproces en leerproduct. Ten derde is taakwaardering de enige motivatievariabele die gerelateerd is aan zo-wel het individuele leerproces gemeten met de natoets als het gezamenlijke leer-proces. In toekomstig onderzoek zou de relatie tussen taakwaardering en het samen-werkend ontdekkend leerproces uitgebreider onderzocht kunnen worden.

HOOFDSTUK 6

In hoofdstuk 6 wordt zowel een overzicht gegeven van de resultaten van de vier stu-dies als suggesties voor toekomstig onderzoek. Tevens worden de overeenkomsten en verschillen tussen de studies benadrukt en beperkingen besproken. Het hoofdstuk eindigt met implicaties voor de onderwijspraktijk.

Nadat een overzicht van de resultaten is gegeven, wordt ten eerste het coderen van het communicatieproces en het ontdekkend leerproces besproken. De protocollen zijn geanalyseerd met een analyseschema dat besproken is in hoofdstuk 2. De protocollen zijn gescoord op drie dimensies: communicatieve activiteiten, transformatieve ontdekkend leeractiviteiten, zoals het genereren van hypothesen, en regulatieve ontdekkend leeractiviteiten, zoals plannen en monitoren. Ten tweede wordt de betrouwbaarheid van de meetinstrumenten besproken, die de leerproducten meten. In de studies in dit proefschrift werden variërende resultaten gevonden met betrekking tot de betrouwbaarheid van de testen die het individuele leerresultaat

126

gemeten hebben. Het aantal items en het aantal deelnemers zijn factoren die de testbetrouwbaarheid kunnen hebben beïnvloed. Minder items en minder deelnemers kunnen zorgen voor een lagere betrouwbaarheid. Bovendien zijn de scores op de toetsen (voornamelijk de voortoetsen) relatief laag en vaak maar net iets hoger dan de scores die volgens de regels van kansberekening met gokken verkregen kunnen worden. Een andere bemerking met betrekking tot het meten van de leerresultaten is dat het collectieve leerresultaat gedurende het leerproces gemeten is en niet erna. Daarom kan dit leerresultaat gezien worden als onderdeel van het leerproces. Ten derde worden oorzaakgevolg relaties tussen communicatieve activiteiten en ontdekkend leeractiviteiten besproken. Het onderzoek in dit proefschrift heeft een positieve relatie tussen communicatieve activiteiten en ontdekkend leeractiviteiten aangetoond. Er kunnen echter geen conclusies getrokken worden met betrekking tot de causale richting van de ondersteuning, aangezien gebruik is gemaakt van correlatieanalyses.

Vervolgens worden drie algemene bevindingen uit dit proefschrift besproken. De eerste bevinding heeft betrekking op de meest opvallende communicatieve en regu-latieve activiteiten in de verschillende studies. Een algemene bevinding is dat regu-latieve activiteiten en communicatieve activiteiten gerelateerd aan de regels Samen beslissen en Aanmoedigen essentiële activiteiten zijn in de computergestuurde leeromgeving die in dit proefschrift gebruikt is.

De tweede bevinding heeft betrekking op de functie van communicatie in een computergestuurde leeromgeving. Sommige communicatieve en ontdekkend leerac-tiviteiten waarvan werd aangenomen dat zij essentieel waren in deze leeromgeving, bleken minder effectief of minder vaak gebruikt te zijn in de studies dan verwacht. Een mogelijke verklaring voor het minder vaak voorkomen van deze activiteiten is dat de leerlingen niet alleen communiceerden door te chatten, maar dat zij ook communiceerden door activiteiten in de leeromgeving uit te voeren.

De derde bevinding heeft betrekking op de kennis en vaardigheden die de leer-lingen mogelijk kunnen hebben vergaard tijdens het werken met de leeromgeving. De resultaten van de studie besproken in hoofdstuk 5, waarin alle data bij elkaar gevoegd waren, toonden aan dat het werken met de leeromgeving in het algemeen leidt tot een toename in domeinkennis of declaratieve kennis. Hoewel de leerlingen beter scoorden op de natoets dan op de voortoets, zijn de verschillen erg klein. Een mogelijke verklaring voor de kleine vooruitgang in domeinkennis is dat leerlingen in plaats van het vergaren van domeinkennis andere kennis en vaardigheden hebben vergaard, zoals ontdekkend leervaardigheden, samenwerkingsvaardigheden of vaar-digheden die specifiek gerelateerd zijn aan de leeromgeving.

Door het hele slothoofdstuk heen worden suggesties voor toekomstig onderzoek gedaan. Gesuggereerd wordt om onderzoek te doen naar de mogelijke causale relatie tussen samenwerkend leren en ontdekkend leren. Samenwerken kan een positief effect hebben op het ontdekkend leerproces (Okada & Simon, 1997; Salomon & Globerson, 1989). Ontdekkend leeractiviteiten kunnen echter ook het samenwerken beïnvloeden. De resultaten van het gepresenteerde onderzoek kunnen als basis die-nen voor het genereren van nieuwe hypothesen.

In toekomstig onderzoek waar de relatie tussen motivatie en samenwerkend ontdekkend leren wordt onderzocht, zou motivatie gemeten moeten worden op

SAMENVATTING 127

verschillende momenten in het leerproces. Vervolgens zou onderzocht kunnen worden hoe deze metingen in verhouding staan met de activiteiten die tijdens het samenwerkend ontdekkend leren worden uitgevoerd. Daarnaast is geconcludeerd dat regulatieve activiteiten en communicatieve activiteiten gerelateerd aan de regels Samen beslissen en Aanmoedigen essentiële activiteiten in de gebruikte leeromgeving zijn. Onderzoek zou het gebruik van deze activiteiten in computergestuurde samenwerkend ontdekkend leeromgeving verder kunnen analyseren. Bovendien zou onderzoek naar stimulatie van het gebruik van deze essentiële activiteiten nuttig kunnen zijn.

Een laatste relevante vraag voor toekomstig onderzoek heeft betrekking op de kennis en vaardigheden die leerlingen vergaren als zij werken met de leeromgeving zoals geïntroduceerd in dit proefschrift. Leerlingen zouden ook geïnterviewd kunnen worden over hun percepties met betrekking tot de leeromgeving, zodat achterhaald kan worden wat hun leerdoel is.

Als laatste worden implicaties voor de onderwijspraktijk besproken. Met het on-derzoek in dit proefschrift is getracht meer inzicht te geven in de wijze waarop de leerprocessen van leerlingen ondersteund kunnen worden. De RIDE regels die de leerlingen werden geleerd, zijn algemene communicatie- en samenwerkingsregels. Dit betekent dat deze regels ook gebruikt kunnen worden in een face-to-face situatie of in een andere samenwerkingsomgeving waarin leerlingen communiceren door te chatten. Als de instructie echter gebruikt wordt in een andere samenwerkingsleer-situatie dan die in dit proefschrift, moet de instructie wel op zodanige wijze gecon-strueerd worden dat de gebruikte leeromgeving als voorbeeld dient. In de com-putergestuurde instructie die in dit proefschrift aan de leerlingen werd aangeboden, werden namelijk voorbeelden gebruikt van situaties uit de leeromgeving. Een ander implicatie van het onderzoek in dit proefschrift is dat leerlingen eerst uitgebreid moeten oefenen met nieuw aangeboden ondersteuning van het leerproces, om een mogelijkheid tot cognitive overload te vermijden. Tot slot wordt in dit proefschrift gesteld dat mensen goed ontwikkelde sociale en communicatieve vaardigheden no-dig hebben om in de huidige Westerse samenleving effectief te kunnen functioneren. Dit is ook erkend door de Nederlandse regering. Een voor de hand liggende conclu-sie is dan ook om mensen deze benodigde vaardigheden aan te leren als ze nog op school zitten, want jong geleerd is oud gedaan.

CURRICULUM VITAE

Nadira Saab was born on January 23rd 1975 in Leiden (The Netherlands) and com-pleted her secondary schooling in 1992 at the Haags Montessori Lyceum in The Hague. She studied Psychology at the University of Leiden and graduated in 1999, specializing in Educational and Developmental Psychology. Her Master’s thesis focused on bilingualism and language proficiency of Papiamento and Dutch speak-ing children in Aruba and in The Netherlands.

Nadira took her PhD at the Graduate School of Teaching and Learning of the University of Amsterdam during the period 2000-2005 in the field of collaborative scientific discovery learning in computer-supported environments. She combined research with teaching presentation skills and writing skills at the Graduate School of Teaching and Learning. From 2002, she also worked as a coordinator of the courses in these communication skills. In 2003, Nadira worked for three months at the McGill University (Montreal, Canada), where she designed the tool that was used in the third study of her PhD research project. Furthermore, she was a member of the board of the PhD organisation of the national association of educational re-search, the VPO (Vereniging voor onderwijsresearch Promovendi Overleg), and also a member of the Educational committee of the national research school ICO.

Next to her research, teaching, and coordination at the Graduate School of Teaching and Learning, Nadira currently works on a project for the Stichting ICT op School.

Documents

CHAT AND EXPLORE - Bvekennis