Productive Persistence: Part Two of Our Interview with Learning Expert Rachel Beattie

Welcome back! Today we present the final section of our talk with the Carnegie Foundation's Director of Productive Persistence: Rachel Beattie! This particular portion of our conversation centers on the testing and application of new strategies in the classroom. The Carnegie Foundation, one of our nation's most prominent educational  research centers, has developed a fascinating model for applied research, and Rachel does an amazing job here breaking down exactly what teachers should and should not do when conducting research or applying new strategies in class. Enjoy! 

---

ASP Blog: Moving on to the next topic, will you talk a little bit about the non-cognitive factors that affect students while they are learning math?
 
Beattie: Sure! This could easily turn into quite the list, as it really is a complex landscape—all of the different factors that affect math learning.
 
Once again, the knowledge that intelligence is malleable and not fixed is really important. When talking with students, I often hear them talk as if there are two races of people: math people and non-math people. We want students to see that this isn’t really the case. This is usually an uphill battle. Over two-thirds of our students answer this way. For students in developmental mathematics, you can probably guess which group they think they are in. That is why we make sure our pathways address productive persistence—to make sure students think about creating new mindsets.
 
But we also believe that many other non-cognitive factors are important: belonging being one of the biggest. The sense that one belongs in a math course is incredibly predictive of math success. It is really a problem in math because of all of the negative stigmas and stereotypes attached to mathematics learning—that girls, for instance, can’t learn math. Even positive stereotypes present problems.
 
The point is that if you believe that others are judging you, or that stereotypes dictate that your classmates don’t think highly of you, this can drastically change your feelings about your ability to succeed in a particular setting. You might constantly question whether or not you belong—especially if something negative happens [in class].
 
Emotional regulation is another main non-cognitive factor. What is really bad here is how we sometimes mistake stress—sometimes your body has the same physiological reaction to things you want to approach as it does to things you want to run away from: heart-beating, sweaty palms, intake of oxygen, all of these are related to both of these reactions. So what we do is have students replay these physiological reactions and test their bodies’ ability to get ready for stress. We have seen that this significantly improves performance on exams—it doesn’t help students develop better study strategies—but it does reduce anxiety enough to help students perform better on exams. 
 
The last factor I’ll bring up is homework systems. We want students to become self-regulated learners. We actually build in opportunities for students to practice these self-regulating skills directly within their homework. This helps students increase confidence in their knowledge of how to accomplish the task at hand. We also make sure that homework assignments always build in complexity and are applied in different situations. This is really essential to students becoming experts. It also helps students keep engaged.

Reflection is another major part of self-regulated learning—so we also include a phase after each homework assignment for students to reflect on the strategies they used and any next steps they want to take. Many of our students, in the past, have only learned shallow study and homework strategies. We want to create an environment in which they can flesh out their meta-cognitive skills.   
 
ASP Blog:  It seems like you are very much in the business of applied psychology. You use whatever you can figure out about the way people think and learn and apply this knowledge directly into the classroom. What is interesting about this to me is that one both learns and applies information in the same exact setting: the classroom. How can universities use the classroom as a test case, while still not losing focus on the main function of the class itself: teaching math?
 
Beattie: That is a great question. Our goal is to never interfere with the teaching of mathematics. For this reason, we use a methodology called “improvement science.” This involves small paths of change, not huge overhauls. The goal is to conduct tests that do not take more than ten minutes or so from a class. It is based upon six different principles.
 
1. You should be user-centered and focused. You really listen to students’ questions and concerns, and you are extremely focused in on a direct problem. 
 
2. Next, you really need to attend to students’ abilities. We want to understand what works for whom, and under what conditions. Across our pathways, there has never been this “one-size-fits-all” mentality. There are adaptations that people need to make, in terms of setting, for something to work.
 
3. It is really important that if you do make a change, you figure out how this change interacts with everything else in your particular setting—otherwise whatever you are trying to apply might become overwhelming.
 
4. We also document what we learn. What are my hypotheses, what are my learning questions? What is my data? This helps you to reflect on an action afterwards and prepare for the next test. We can’t improve what we can’t measure.
 
5. You don’t just want to look at test scores. You want to take note of the everyday behavior in the classroom: are students turning in their homework, are they asking questions, how are groups working? In order to improve a classroom, you really need to understand what is happening there every day. 
 
6. We help create what we call Network Improvement Communities to make sure faculty, teachers and students are working together. Creating a networked community keeps you from having to do everything yourself. If everybody is working together, then someone might have already tried something that will work well in your particular setting—they have already tested it out, and it just needs to be adapted a bit.
 
All of this helps to really reduce the load of testing so that it doesn’t seem too unwieldy.
 
ASP Blog: Can you talk a little bit specifically about how to properly measure strategy results within a classroom?
 
Beattie: Outcome measures are always really important, so I don’t want to say never measure your outcomes—but you do want to look at the larger goals and aims of your project. If your goal is to improve success in one math classroom, then you definitely want to look at grades, as well as the credits students get in mathematics after that class. At the whole college level, you want to look across your whole department: grades, credits, transfer rates, etc.
 
With outcome data, you want to make sure it closely mirrors your aims: why you are doing this work in the first place?

What is really important for our work is what we call “process measures.” These actually measure the process of improvement, the daily processes in class. For us, attendance is a big one. We have developed a couple processes that we know work because we measure them daily. We were able to meet semester long attendance rates between 85 percent and 93.3 percent with just one small change that we tried out. So it is really powerful to put these day-to-day measures to work. Another one we look at is group functioning. Are students being supported in their groups, do they feel like they are being productive and that everyone is contributing? That is really important to form a community.
 
You also want to look at “balancing measures.” Sometimes changes unintentionally affect classroom learning—I love that you already brought up time, that new strategies take up a lot of faculty members’ time and keeps them from doing other things that were already affective. This means that it is important to think about “if I do this change, what might be affected in my classroom either short term or long term and how can I measure that?”
 
Finally, another part of our data collection system is what we call “practical measures.” It is hard to measure a student’s mindset in three minutes. If you are familiar with psychological research, we aren’t big on short test batteries. Questionnaires and interviews are always LONG, because you really want to explore the contextual frameworks, you want to understand students’ beliefs on multiple levels. However, with a practical measure, you have to cut it down drastically and really focus in on the variables that are causally related to student success. With practical measures, we measure students at week one and week four, only because we do a lot of intervention and we are setting up a lot of classroom norms and expectations—this allows us to measure a before and an after. It is really wonderful to see [this dynamic] in our data system, because it helps us understand the different elements that lead to student success. This is true from our point of view, but also our faculty’s. We actually give each of our faculty members a report about their classrooms and about productive persistence. This gives them classroom aggregated data on individual and group mindsets—on how well students are transitioning.
 
 

And that' wraps it up! Once again, we want to thank Rachel Beattie and the Carnegie Foundation for their participation. Tune in next Monday for another exciting post! 

---