An Introduction to Evidence-Based Undergraduate STEM Teaching

Week 1 - Principles of learning, Part 1

Prior knowledge

• The way students make sense of new information is influenced by their prior knowledge and experiences.

• Students' mental organization influences their problem-solving skills.

• Categories of misconceptions (adapted from Chi & Roscoe, 2002):

  • Proposition-level misconceptions (easier to address).

  • Flawed mental models.

  • Ontological miscategorizations.

  • Embedded beliefs (harder to address).

• Assign prior activities to surface and activate prior knowledge. Show a demonstration, ask students why it works, and then provide the explanation.

Knowledge organization

• Deep learning requires much more than memorization.

• Build adaptive expertise, expertise to solve problems outside of textbook problems.

• Experts have rich, meaningful, knowledge structures. Novices/students tend to build sparse, superficial knowledge structures. Students need to develop meaningful structures, and develop nodes and connections in knowledge (graphic syllabus, concept maps).

Week 2 - Principles of learning, Part 2

Practice and feedback

• Learning new concepts or skills requires iterative practice and feedback.

• Designing effective learning environments requires attention to the affective domain (feelings, values, beliefs, and motivations).

• Levels/path to expertise:

  1. Unconscious incompetence.

  2. Conscious incompetence.

  3. Conscious competence.

  4. Unconscious competence.

• It's not teaching that causes learning, but the learners themselves when they attempt to perform, taking into account feedback.

• It's not always possible for one instructor to give individualized feedback, sometimes is more practical and useful for students to receive feedback from their peers.

• Realistically, there's not a lot of time for practice and feedback. A practical approach is to focus class time on hard stuff, and trust the students to figure out the easier stuff outside class.

The affective domain

• The cognitive domain: how we think. The affective domain: how we feel (values, beliefs, and motivations).

• What motivates a student to learn:

  • Intrinsic motivations: interests, feeling of accomplishment, fear of failure, learning.

  • Extrinsic motivations: grades, parents, grad school, jobs, money.

  • Students who are motivated by extrinsic factors tend to engage in strategic learning, they will do just enough to get a good grade (Ken Bain, 2004).

  • Deep learning is learning for mastery, critical thinking, conceptual understanding, be prepared to use what you've learned once the course is over.

  • Have multiple, smaller tests along the way, give opportunities to show what they know in different ways, final assignment could be paper or poster.

  • Don't grade on a curve, creates a competitive environment and shifts mentality to strategic learning.

  • One interesting technique is having an audience for students' work. This introduces purpose.

  • Students want to share interests. Social bookmarking: A shared site where students were bookmarking, saving links to news and articles and websites and resources that they had found that appealed to them. They see each other as people they can learn from.

  • Four ideas to look for, to introduce in activities and class: Competency, autonomy, purpose, and community.

Week 3 - Learning objectives

Learning objectives

• Learning objective: what a student should be able to do after instruction. You need to be able to assess whether a student actually achieved that goal.

• Learning objectives are different than a syllabus. A syllabus only includes a list of topics, but it doesn't provide information about what's going to be learned.

• It's important for students to see how the ideas grow over time, and each week talk about what they are going to be learning and how those ideas build on each other.

• Adds fairness to the exam, everything evaluated is in the learning objectives.

• Start with the phrase: "At the end of this course, students should be able to..."

Introduction to backwards design

• Where are you, where do you want to go, how are you going to get there.

• Three-step process:

  1. Define learning objectives (desired outcomes).

  2. Decide on assessments (evidence of understanding).

  3. Design instruction (help students achieve results).

• Select activities so that you think everyone should be able to do well on that assessment.

• Learning goal: Broad, course-level. 5-10 per course

• Learning objective: More specific, 2-5 per topic

• Ways to unpack a topic or goal:

  1. Find someone else's objectives

  2. Look at end-of-chapter summaries

  3. Look at education research

  4. Work backwards from an exam question

  5. Talk to clleagues

What should they know and how deeply

• Different types of learning:

  1. Cognitive abilities:

Facts: Terminology, information, details.

Concepts: Classifications, reasoning, principles.

• 2. Procedure and skills Techniques, methods, problem-solving.

  3. Metacognitive Self-awareness about what helps you learn; studying and learning strategies.

  4. Attitudes and beliefs Appreciate, enjoy, value.

• Bloom's Taxonomy pyramid (lower-order skills on the bottom and higher skills on the top):

  1. Create

  2. Evaluate

  3. A n a l y z e

  4. A p p l y

  5. U n d e r s t a n d

  6. R e m e m b e r

• Drawback of the pyramid: It's not specific to particular disciplines.

• Check-list for refining topic-scale learning objectives:

  1. Is the goal expressed in terms of what the student will achieve or be able to do?

  2. Is the goal well-defined? Is it clear how you would measure achievement?

  3. Do chosen verbs have a clear meaning?

  4. Is terminology familiar/common? If not, is the terminology itself a goal?

  5. Does the goal align with your course-scale goals?

  6. Is the Bloom’s level of the goal appropriate? (Is it aligned with your actual expectations, and with student ability?) Do your goals cover a range of levels?

  7. Do your goals cover a range of types of knowledge?

• It's important to talk to your colleagues when your course is:

  1. Core to the major

  2. Pre-requisite to other courses

  3. Part of a sequence

  4. Has goals related to certification

  5. Or you just need their expertise and guidance.

Week 4 - Assessment

Formative and summative assessments:

• • Formative: information that students and/or instructors can use to improve at shorter timescales (e.g. clicker questions, minute papers, etc.)

  • Summative: information that assesses whether students and/or instructors have succeeded in meeting a goal (e.g. final exams, research-based surveys, etc.). Usually comes at the end of a course or sections of the course.

• "Bloomify questions": Make them harder by choosing verbs higher on Bloom's taxonomy triangle.

• Multiple option questions are not the best because explaining things triggers knowledge.

Importance of open-ended problems

• They involve multiple solution paths

• They are more qualitative in nature

• Make students reflect on their learning process, ask how they are working on things and why they are doing certain things.

• Peer Assisted Reflection (PAR): Students analyze the work of their peers, this teaches them reflection skills because often it's much easier for us to look at and be objective about somebody else's work rather than trying to look at our own work which is sometimes too close to us to really see the flaws in it. One PAR problem in every assignment, generally difficult.

• Establish norms at the beginning of the semester on how to criticize other students' work.

• Teach them how to give feedback

Adapting traditional problems

• Students underestimate the amount of time a problem takes to solve.

• Adapt traditional textbook problems

• Ask students to take an idea they've been practicing and apply it to a new context that they're interested in studying.

Focus on the instructor

• Open-ended questions check a deeper level of understanding.

• A student might articulate an idea that is wrong but the underlying logic is right (and vice versa)

• Give the proper definition and rearticulate a few weeks later

• Use talk moves: ask students "say more" when you are confused about their idea. Or ask another student "who can explain what so and so said".

• Hold back your first impulse to make an evaluative statement of the first idea.

• Be ready to harvest ideas from open-ended questions.

• Create a safe space and support for students to share their ideas.

Introduction to mindset

• Some praise comments can have a negative impact, for example, "you are so brilliant, you got an A without studying", has the potential to suggest that if students can't get an A without studying, they are not brilliant.

• Normalize student struggle. STEM learning can be hard when students haven't developed certain skills.

• Don't mistake student background with natural ability.

Research-based surveys

• Content related surveys, attitudinal and belief surveys, student affect.

• Conceptual assessment or concept inventories, there are assessment tools for each course.

• Use research based pre/post tests in every class where there's one available, even if is not 100% directly aligned with the course.

• Can be used to compare semester after semester how changes impact and what works.

• Feedback codes: Graders go through a set of papers first to identify common mistakes and determine feedback codes. Facilitates feedback.

Week 5 - Active Learning

• In the year 2000, the attention span of humans was 12 seconds, in the year 2013 it was 8 seconds.

• Active learning:

  1. Encourages engagement.

  2. Reinforces concepts and skills.

  3. Provides immediate feedback.

  4. Fosters a community of learners.

  5. Creates personal connections to the materials.

• Active learning broken into two topics:

  1. Critical thinking: Problem based learning (PBL), inquiry based labs

  2. Teamwork: Cooperative learning (small groups), peer instruction

• Chinese proverb: Tell me and I forget, teach me and I remember, involve me and I learn.

• The Differentiated Overt Learning, by Michelene Chi 2009, divides active learning methods into modes, depending on students' cognitive engagement:

  1. Passive method: Traditional lecture.

  2. Active: Involves learners in hands-on manipulation of the materials.

  3. Constructive: Students are expected to facilitate the generation of new ideas beyond those directly presented.

Critical Thinking

• Critical thinking entails examination of those structures or elements of thought implicit in all reasoning:

  • Purpose, problem, or question-at-issue.

  • Assumptions, concepts, and frame of reference.

  • Empirical grounding.

  • Reasoning leading to conclusions.

  • Implications and consequences.

  • Objections from alternative viewpoints.

  • IT'S NOT: The retention of information alone, the possession of a set of skills, the use of skills without acceptance of their results.

• Problem-based learning: It's a teaching approach that challenges students to learn concepts/principles by applying them to real-life problems. Students must use their problem-solving skills and their disciplinary knowledge to address problems they may encounter in the future.

• Inquiry-Based Labs: Students engage in many of the same activities and thinking processes as scientists. What do scientists do?:

  • Ask questions.

  • Propose hypotheses and models.

  • Design, carry out, and analyze studies to evaluate hypotheses and models.

  • Communicate results and interpretations.

  • Revise results and interpretations in response to critiques.

• Elements of inquiry:

  • Observing, questioning

  • Reporting

  • Repeating

  • Analyzing data

  • Collecting data

  • Designing experiments

  • Peer review

• Key elements to include in inquiry-based labs:

  • Instructure guidance to help students develop inquiry skills. Defining a research problem, formulating hypotheses, planning an experiment.

  • Opportunity for peer-to-peer interactions and teaching.

  • Opportunity for students' summative presentation of work (oral or written).

Teamwork

• Cooperative learning: It's the instructional use of small groups so that students work together to maximize their own and each other's learning.

• Principles of cooperative learning:

  • Positive interdependence "sink or swim".

  • Individual and group accountability.

  • Face-2-face interactions.

  • Teamwork skills.

  • Group processing and facilitation.

• It's very similar in some regards to peer instruction, but it's a little bit less formal in its mechanism. We still always want students to be able to articulate their own thoughts, and then within that group, defend and refine them and work together towards a larger goal.

Active learning

• Active learning: Clicker questions with small-group discussion before answering questions.

• The best questions to task in class are the questions that come up in students' minds, not in the minds of the instructors.

• Keep track of common mistakes that students make on assignments, homework, midterm examinations, etc.

• Think of the question as the beginning of four or five conversations you want your students to have.

• Measuring active learning in the classroom. GORP: Generalized Observation and Reflection Protocol, it's an online protocol-based observation platform. It's a way of looking at what's happening in class, how much time is being spent in active learning, in asking questions, in answering questions. TAs can be trained to do it and collect the data.

• Group testing to enhance collaborative learning

Week 6 - Inclusive teaching

• Have students introduce themselves. Making sure that everyone has a voice in the classroom and has been heard at least once, even just through introducing themselves, makes them more likely to be able to ask questions later on.

• Stereotype threat: People from communities, particularly white women and women and men of color, who know that they are stereotyped as not being good at science and math. This affects results.

• Fixed mindset: E.g. The idea that you are a "math person", and if you are not, you will not do well.

• Growth mindset: E.g. You can learn to do mathematics well. You can design a diagnostics pretest within a context of a growth mindset, based on regular problems. This would help set students into the growth mindset.

• Six impactful teaching practices to improve the academic achievement of underrepresented minority and first generation students (Kesha Wilkinson and Janaina Saad, Delta Program in Research, Teaching and Learning, University of Wisconsin-Madison, 2014):

  • Integrate culturally inclusive and relevant content

  • Decrease the potential intimidation students feel toward instructors

  • Get students involved with supplemental instruction

  • Be intentional about how students groups and project teams are formed

  • Work with TAs and other instructors in the class.

  • Use inclusive teaching practices.

• Microinequities (Hall and Sandler, 1982, 1993). Male students:

  • tend to get more eye contact.

  • are called on more.

  • get more praise for answers.

  • are asked more follow-up questions.

  • have their names used more.

  • and are more regularly given credit for their constributions.

• Guiding principles and strategies:

  • Examine your assumptions.

  • Learn and use students' names.

  • Model inclusive language.

  • Use multiple and diverse examples.

  • Establish ground rules for interaction.

  • Strive to be fair.

  • Be mindful of low-ability cues.

  • Don't ask people to speak for an entire group.

  • Be careful about microinequities.