Projects in the Scholarship of Teaching and Learning (SoTL) are never solo endeavors, and the most successful partnerships are those where both parties are flexible and open to trying different approaches to their teaching and learning practices. Working with Dr. Marci Levine at the College of Dentistry has been a wonderful adventure, as we addressed the many challenges involved in preparing four-hundred-plus students for dental practice. We began the process ten years ago by experimenting with lower tech solutions to the problem of tactile approaches to teaching the delivery of anesthesia. Paper cut-outs of the instruments marked an improvement: students no longer jumped directly from lectures on the topic to injecting one another for practice. But we realized that even better simulations were possible, especially through evolving, and increasingly accessible, virtual reality technologies. The journey led us to where we are now, designing and implementing virtual reality solutions with realistic haptic response peripherals. You can learn more about this work in our recent TeachTalk.
Ours was—and is—an on-going iterative design and research process. It starts with identifying and analyzing an educational gap, designing a solution with assessment in mind, building the solution, and finally implementing and/or testing the solution, either in situ or in a lab-like environment. All of the recognition, awards, and grants that Dr. Levine has received testify to her commitment to exploration and to paving a way forward for integrating educational technology into the curriculum.
During our TeachTalk, a question surfaced about how to design a SoTL research study specifically when comparing two interventions. Some suggestions include:
1. Decide how the assessment measurement will evaluate both interventions to see if there is a difference in learning gains favoring one over the other.
2. If you do not have a comparison, then a pre-post test can help determine learning gains for just that intervention. It is not as strong as a comparison but still viable.
3. Always include a pre-post survey to collect background, feedback, and emotional data.
4. Look at prior research publications similar to the research that you want to conduct. These can help you make decisions around the study design, methods, and measurements.
For example, our study comparing a Virtual Reality Simulation (VR Sim) with a Plastic Manikin Simulation (PM Sim) involved a strong comparison of interventions. Our haptic pilot study, however, used a pre-post survey because we were not quite ready to do a formal comparison study—we first needed the feedback and evaluation of our current stage in the development process.
Another question raised during the presentation was how to measure a learning gain. A learning gain can be measured by a reliable and valid traditional test that addresses the same construct as the intervention and meets the same learning objectives. It can also be measured by an assessment of the actual task performed by a student (e.g., perform all the steps to inject a patient with local anesthesia) that is scored with a rubric by an expert or two, and that again addresses the same construct as the intervention.
For these and many other questions on SoTL research methodologies, we recommend the forthcoming SoTL Research Methodologies by Yeo, Miller-Young, and Manarin. Also, consider joining NYU’s online SoTL discussion space, where myself and many other NYU community members participate in an ongoing conversation on SoTL (click this link to join the SoTL google space).