Aerial view of Washington Square Park at sunset

Getting Started with SoTL – Deprecated 1page version

Identifying Problems in Teaching

The first step in the Scholarship of Teaching and Learning (SoTL) typically begins with an idea for helping students. One of NYU’s longest SoTL projects, for example, began with Dentistry faculty asking a simple question: Could we teach dental students to administer local anesthesia without them giving each other injections? 

According to Randy Bass’s “The Scholarship of Teaching: What’s the Problem?” identifying an educational gap should not be seen as something to be fixed, but rather as a topic for continuous investigation, analysis, representation, and discussion with a scholarly community. He explains: 

“In scholarship and research, having a ‘problem’ is at the heart of the investigative process; it is the compound of the generative questions around which all creative and productive activity revolves. But in one’s teaching, a ‘problem’ is something you don’t want to have, and if you have one, you probably want to fix it. Asking a colleague about a problem in his or her research is an invitation; asking about a problem in one’s teaching would probably seem like an accusation. Changing the status of the problem in teaching from terminal remediation to ongoing investigation is precisely what the movement for a scholarship of teaching is all about. How might we make the problematization of teaching a matter of regular communal discourse? How might we think of teaching practice, and the evidence of student learning, as problems to be investigated, analyzed, represented, and debated?” (Bass, 1999)

Some key aspects of choosing your “problem” can include..

  • Is it meaningful and significant to you?
  • Is the problem causing ineffective learning? 
  • Is it possible to research with the time, resources, and students you have?
  • Is it deliberate, narrow, and focused, so that your project will adequately answer your research question?

*Adapted from Taylor Institute for Teaching and Learning’s SoTL Guide (page currently under construction) 

References:

Bass, R. (1998). The Scholarship of Teaching : What’s the Problem?. Inventio, 1(1).

Types of SoTL Questions

Identifying so-called “problems” is one approach to beginning with SoTL. Another is to begin by asking questions. These questions can be asked of students (what is working? What is not?) or they can come from thinking through a new classroom approach, in the hopes of further investigating a new approach or more deeply understanding one that’s ongoing.. 

Hutchings (2000) provided a taxonomy of questions that are frequently posed by scholars in SoTL. The taxonomy comprises four distinct types of questions: “What works?” “What is (happening)?” “Visions of the possible,” and “Formulating new conceptual frameworks.” 

Within the SoTL literature, the most common questions are the first two:

  • What works?
    • This question aims to gather evidence about the effectiveness of different teaching approaches, strategies, or interventions. 
      • Example: Does student comprehension improve more when they collaborate with peers compared to working individually? 
  • What is (happening)?
    • This question focuses on describing what a practice or condition looks like in the classroom, but does not evaluate the effectiveness of different teaching approaches or interventions.  
      • Example: What occurs when students engage with complex literary texts that challenge their understanding?

For further insight into “What works” and “What is (happening)” SoTL projects, explore ‘Examples of SoTL Projects’

References:

Hutchings, P. (2000). Opening lines: Approaches to the Scholarship of Teaching and Learning. The Carnegie Foundation for the Advancement of Teaching.

Conducting a Literature Review

A literature review can provide your SoTL project with a strong critical foundation. It can help situate your project within existing theory and practice and provide you with models to follow for research design, methodology, and data collection. Since SoTL studies aim to expand what we know about teaching and learning within a discipline, it is essential to review as much relevant literature as possible to know the current limits of the field. The sources covered in a literature review may include journal articles, conference proceedings, books, dissertations, and more.

It is also important to note that while a literature review is a great way to understand what has been studied, what methodologies have been used, and where there might be notable gaps in the existing research, it is also important to continually return to and update your reviews, particularly as studies extend over time.

Literature Database

Conducting literature reviews in the field of SoTL may require us to consult  unfamiliar databases in fields other than our own. The Libraries’ Education guide offers a quick start for seeking resources regarding SoTL, including lists of databases, statistics resources, and other related data and information sources. Learning about some of the databases more commonly used in SoTL can ease the process. You may also consider using Google Scholar. And more recently, scholars have started using Generative AI, such as ChatGPT, to generate literature resources, though it does come with risks such as hallucination. 

Two prominent databases are: 

  • Education Source with ERIC. This database provides access to research on all levels of education, from early childhood to adult. It includes peer reviewed journal articles, ebooks, book reviews, dissertations, conference papers and policy reports on all education-related topics.
  • PsycNet (With PsycINFO). This database provides information about and access to journal articles, books, ebooks, dissertations, conference presentations, tests and measures, videos, gray literature, and many other other publication types in the psychological, social, behavioral, and health sciences.

Search Terms

Search terms required for SoTL projects will differ significantly from those employed within our respective disciplines. Familiarity with search terms related to teaching and learning is imperative. 

The SoTL annotated literature database, curated by Nicola Simmons at Brock University in Canada, provides a great starting point. It surveys common search terms (see its table of contents), which can help you identify those pertinent to your specific area of investigation, such as, “active learning,” “project-based learning,” and “student engagement.” 

Some search terms might yield a substantial number (in the tens of thousands) of results. A search for “active learning” in the ERIC database, limited to results from peer-reviewed and academic journals, still produced around 18,000 results as of July 2023. This is likely because “active learning” encompasses elementary teaching strategies, such as think-pair-share, as well as more complex pedagogical approaches, such as case-based learning. Below are some suggestions on how to narrow down your focus:

  • Pedagogy or framework (e.g., case-based learning, problem-based learning, Community of Inquiry framework, etc.)
  • Modality (e.g., online courses, blended courses, etc.) 
  • Qualifiers (e.g., effectiveness, effects, correlation, prediction, challenges, perceptions, etc.)
  • Settings (e.g., higher education, college settings, postsecondary, etc.) 
  • Discipline (e.g., engineering, social science, education, etc.)
  • Target audience (e.g., undergraduates, graduates, first-generation students, etc.)*

*Adapted from the University of Georgia’s Center for Teaching and Learning and revised by the LED team. 

Combining database searches with snowball searches is also very effective for finding SoTL research relevant to your topic. In a snowball search, the researcher tracks down related research studies by reviewing the reference lists in articles or by exploring the “Cited by” feature on the Google Scholar website. You may also link your NYU account to Google Scholar for a more robust search.

Research Methods for SoTL Study

Choosing the appropriate research method is crucial for SoTL projects. We explore three approaches below: quantitative methods, qualitative methods, and mixed methods.

TypeQuantitativeQualitativeMixed Method
Description“An approach for testing objective theories by examining the relationship among variables. These variables, in turn, can be measured, typically on instruments, so that numbered data can be analyzed using statistical procedures” (p.4, Creswell & Creswell, 2018).

Quantitative research studies
“Seek to confirm hypotheses about phenomena
Seek to quantify variation or predict casual relationships about phenomena
Seek to describe characteristics of an educational population” (Hubball & Clarke, 2010, p. 4)		“An approach for exploring and understanding the meaning individuals or groups ascribe to a social or human problem. The process of research involves emerging questions and procedures, data typically collected in the participant’s setting, data analysis inductively building from particulars to general themes and the researcher making interpretations of the meaning of the data.” (p.4, Creswell & Creswell, 2018)

Qualitative research studies
“Seek to explore complex phenomena in educational setting
Seek to describe and explain variation and/or relationships in complex educational settings
Seek to describe individual experiences and/or group norms in complex educational settings” (Hubball & Clarke, 2010, p. 4)		“An approach to inquiry involving collecting both quantitative and qualitative data, integrating the two forms of data, and using distinct designs that may involve philosophical assumptions and theoretical frameworks.” (p.4, Creswell & Creswell, 2018)
ExamplesHere are some examples of quantitative research design methods: Experimental design method and survey.

Experimental design method

“Experimental research seeks to determine if a specific treatment influences an outcome. This impact is assessed by providing a specific treatment to one group and withholding it from another and then determining how both groups scored on an outcome.” (p.12, Creswell & Creswell, 2008) 

Here are examples of SoTL studies that employ experimental design method: 

Hurt, N. E., Moss, G. S., Bradley, C. L., Larson, L. R., Lovelace, M. D., & Prevost, L. B., Riley, N., Domizi, D. P., Camus, M. S. (2012). The ‘Facebook’ effect: College students’ perceptions of online discussions in the age of social networking. International Journal For The Scholarship Of Teaching & Learning, 6(2), 1-24.

Noppe I. C. (2007). PowerPoint presentation handouts and college student learning outcomes. International Journal for the Scholarship of Teaching and Learning, 1(1), 1-13. 

Survey 

“Survey research provides a quantitative or numeric description of trends, attitudes, or opinions of a population by studying a sample of that population.” (p.12, Creswell & Creswell, 2008)

Here is an example of a SoTL study that employs survey method: 

Gaston Ms, S., & Kruger Ms, M. L. (2014). Students perceptions of volunteering during the first two years of studying a social work degree. International Journal for the Scholarship of Teaching and Learning, 8(2), 1-18.

You might find it beneficial to explore surveys and questionnaires provided in Columbia University’s Science of Learning Research Initiative’s Toolkit. These tools are designed to quantify various aspects of student experience. 
Here are some examples of qualitative research methods:

Phenomenology

Consider reading Webb and Welsh’s (2019) ‘Phenomenology as a methodology for Scholarship of Teaching and Learning research’ to delve into phenomenology. 

Here is an example of a SoTL study that employs phenomenology: 

Brackenbury, T. (2012). A qualitative examination of connections between learner-centered teaching and past significant learning experiences. Journal of the Scholarship of Teaching and Learning, 12(4), 12-28.

Resources:
Deakin University’s qualitative study design – Phenomenology 
Vagle, M. D. (2018). Crafting phenomenological research. Routledge.


Case Study

Explore the SoTL Advocate’s blog posting to gain insights into the concept of a case study

Here is an example of a SoTL study that employs case study: 

Fayer, L. (2014). A multi-case study of student perceptions of online course design elements and success. International Journal for the Scholarship of Teaching and Learning, 8(1), 1-27.

Resources:
Deakin University’s qualitative study design – Case Study
Pearson, M. L., Albon, S. P., & Hubball, H. (2015). Case study methodology: Flexibility, rigour, and ethical considerations for the Scholarship of Teaching and Learning. The Canadian Journal for the Scholarship of Teaching and Learning, 6(3). 1-6.
Yin, R. K. (2017). Case study research and applications: design and methods (6th ed.). SAGE Publications.

Autoethnography

Explore Starr’s (2010) paper, ‘The use of autoethnography in educational research: Locating who we are in what we do,’ to gain insights into autoethnography. 

Here is an example of a SoTL study that employs autoethnography: 

Enfield, M., & Stasz, B. (2011). Presence without being present: Reflection and action in a community of practice. Journal of the Scholarship of Teaching and Learning, 11(1), 108-118.

Resources:
Keleş, U. (2022). Writing a “good” autoethnography in educational research: A modest proposal. The Qualitative Report, 27(9), 2026-2046.
Consider reading Kimmon’s (2022) ‘Education research: Across multiple paradigms’ to gain an understanding of mixed-methods.

Here is an example of a SoTL study that employs mixed-method:

Coakley, L. A., & Sousa, K. J. (2013). The effect of contemporary learning approaches on student perceptions in an introductory business course. Journal of the Scholarship of Teaching and Learning, 13(3), 1-22.

References:
Creswell, J. W., & Creswell, J. D. (2008). Research design: qualitative, quantitative, and mixed methods approaches (3rd ed.). SAGE Publications.
Creswell, J. W., & Creswell, J. D. (2018). Research design: qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE Publications.
Hubball, H., & Clarke, A. (2010). Diverse methodological approaches and considerations for SoTL in higher education. Canadian Journal for the Scholarship of Teaching and Learning, 1(1), 1-11.

Identifying Evidence

In the journey of SoTL, the initial exploration through a literature review provides a strong foundation. But that’s only the beginning. To gain deeper insight into your students’ learning and to effectively address your research questions, you will need to identify the right evidence to collect from the classroom.

According to Bernstein and Bass (2005), faculty members engaged in SoTL projects often pose the following questions: “How did they know that their students were learning?” and “Did the students’ learning promise to last?” Bernstein and Bass (2005) explained that “By asking these questions, many faculty discovered early on that what most interested—or eluded—them about their students’ learning could not be answered simply by looking at regularly assigned course work” (p. 39). Therefore, SoTL collects the evidence of learning in a variety of forms.

Evidence of learning may include: 

  • Classroom assessment techniques, or CATs (minute paper, muddies point, clicker data, etc.)
  • Evidence of how students actually think (think-alouds, process logs, reflective journals/reflection papers, concept maps, etc.)
  • Exam scores, or scores on a single exam question
  • Counts (the number of online discussion postings/replies, the number of logins to Brightspace, the number of Brightspace pages read/written, number of videos watched etc.)
  • Samples of students’ work (papers, journals, projects, presentations, performances, online discussion posts, etc.)
  • Institutional research data (grades, GPAs, admission scores, retention rates, etc.)
  • Students’ reports of their learning (surveys, interviews, focus group interviews, etc.)

Adapted from Vanderbilt University Center for Teaching’s guide on identifying evidence.

References:

Bernstein, D., & Bass, R. (2005). The Scholarship of Teaching and Learning. Academe, 91(4), 37-43.

IRB and Ethical Concerns

IRB is typically necessary when conducting “generalizable” research, with findings applicable beyond the study’s specific participants. IRB is also usually necessary when research involves identifiable data such as student names, email addresses, etc. 

However, SoTL research may not always be intended for generalizability, as sometimes we are working to improve teaching and learning within a specific context. Or we may use data that has been stripped of personally identifiable information. Therefore, it is a good practice to contact the IRB office (ask.humansubjects@nyu.edu) to clarify whether your SoTL research project requires IRB approval. 

For further information, please visit the NYU IRB website. If you require assistance in preparing your IRB proposal, you can take advantage of NYU IRB’s drop-in consultation hours. 

Resources

Hutchings, P. (2003). Competing goods: Ethical issues in the Scholarship of Teaching and Learning. Change, 35(5), 26-33. 

Vanderbilt University’s Doing Ethical Research: SoTL and the IRB

IRBs and Research on Teaching and Learning (Society for Teaching of Psychology)

Presenting and Publishing SoTL Works

One essential aspect of the SoTL project is the dissemination of findings and insights to a broader audience. This process of ‘going public’ is pivotal in contributing to the collective knowledge of effective teaching and learning practices. While publication in SoTL journals may be the goal, you can begin by sharing your work at local, national, or international conferences, whether your research is complete or still in-progress. 

For those who may feel apprehensive about publishing their SoTL work in scholarly journals, there are alternative avenues for sharing teaching experiences and reflections. These include non-academic journals, which often adopt a more accessible and informal approach. 

Here are Illinois State University’s tips for publishing SoTL journals.

We recommend getting started finding appropriate SoTL Journals using the two most comprehensive resources. Both are more or less exhaustive, however, some users may find they prefer one navigation interface over the other: 

Next: