sotl spotlight: reflections from a sotl project
by andrew greer, cft graduate teaching fellow
learning happens (or doesn’t), and the processes often remain hidden. teachers tend to modify their teaching with trial and error, and base these modifications on assumptions about the problems with student learning. however, instead of being guided by hunches, scholarship of teaching and learning (sotl) researchers strive to frame problems in the classroom just as they would research problems in their home disciplines.
for example, students might learn better when teachers offer repeated low-stakes testing (more on that later), and sotl studies might investigate such a problem with as much methodological rigor as evaluations in non-classroom settings. furthermore, sotl embraces an interdisciplinary approach to analyzing the problems, often with a team of researchers across departments. finally, sotl investigations publicize problems and often solutions, so that teachers across universities can learn from each other’s work, as in their disciplinary research. in 2012-2013, i worked with an incredible team of peers in the sotl scholars program led by nancy chick at the 2022年世界杯中国小组赛积分 (cft), and the following is an attempt to share my project.
what might a sotl project look like?
i became interested in sotl when i started thinking about processes that increase students’ retention and transfer (or later application) of knowledge—or students’ abilities to maintain and retrieve what they’ve learned and then apply it in new situations. after applying to the sotl scholars program, i teamed up with andy van schaack (hod) to investigate the effects of intermittent testing on long-term retention and transfer of knowledge. in a class about research methods, we randomly assigned approximately 150 students into two groups: one group that received weekly practice tests on concepts of experimental design and another group that received weekly practice tests on concepts of qualitative design. after finding no difference across groups at baseline, we began the intervention and measured retention and transfer for both concepts across students at the middle and end of the semester.
as expected from the literature, students retained and transferred knowledge better when they practiced a particular concept, for the most part. students who received weekly practice tests on qualitative design scored significantly higher on retention and transfer of qualitative design to novel settings than students who received practice tests on experimental design. however, even though students who received practice tests on experimental design scored significantly higher on the transfer of their concept than the comparison group, they did not score significantly higher on retention of experimental design. we were perplexed. why would practice testing work for the retention of one concept better than another concept? were there differences between the concepts that might have decreased the effect of practice tests for the concept of experimental design? we were determined to find out, and now, for my favorite part of sotl projects: collaboration.
a major part of the sotl scholars program at the cft involves work-shopping each other’s projects, and i found this activity to be extremely useful. i brought my sotl project to the group of scholars at the cft, and we collaborated on possible approaches to answer our questions. we agreed that it would be best to include a qualitative component in the study, so i asked a subsample of the undergraduate students for feedback on the concepts. for students who practice tested the qualitative concept, the majority thought that the definition and description of the qualitative question (i.e., demonstrating their retention) was more abstract with greater flexibility in their responses than the experimental question, which may have led to greater retention for the more abstract concept.
the following quote from a student in the qualitative group illustrates this finding:
second, students who practice tested the experimental question thought that defining and describing it was easier, suggesting that they may have crammed for the questions before the exam and suppressed any effect from the intervention. one student who practiced the experimental concept said,
so what are the implications of this sotl study?
first, this investigation adds nuance to the large body of literature that suggests that practice testing leads to long-term retention and transfer. this project offers teachers a place to start when considering retention and transfer of concepts. if the goal of the class is to have the students transfer knowledge to novel settings, then low-stakes practice testing might facilitate such a goal. alternatively, if the goal of the class is to have students retain information, practice testing might work better for complex concepts than for concepts that students see as straightforward.
the sotl scholars program helped me to complete the project by keeping me on track to produce most of a journal article. sotl scholars meet an average of twice a month over the course of the academic year, and everyone who signed up for the program completed their projects successfully. see more information about sotl, or email nancy chick or andrew greer.
leave a response