Impacts of OGAP

A number of studies have been conducted on the impacts of OGAP on teachers and students. The key findings of these studies are summarized below.

Impacts on Teachers   

OGAP increases teachers’ comfort and confidence level with mathematics:

  • A survey study of 107 OGAP participants identified several important benefits to teachers. Respondents reported better understanding of evidence of student work, stronger use of evidence to inform instruction, better ability to utilize their mathematics program or curricula, and stronger understanding of fractions concepts (Petit, Laird & Marsden, 2010).
  • In a survey study of 56 OGAP teacher leaders and teachers, participants were asked to estimate on a scale of 1 to 5 their “expertise in content knowledge related to teaching fractions.” On average, teacher leaders rated themselves at 2.74 in expertise prior to OGAP, and 4.33 after participating in OGAP professional development and support. Teachers indicated similar progress, growing from 2.65 to 4.03 in their fraction content knowledge expertise. The pre-post differences for both groups were statistically significant. More importantly, these self reported results were externally validated, as both groups offered concrete examples of the ways in which their content knowledge had grown and deepened (Riggan & Ebby, Forthcoming).

OGAP increases teachers’ pedagogical content knowledge:

  • A study of OGAP teachers’ “mathematical knowledge for teaching” (MKT) found that, after one year of OGAP training and follow up development, teachers made positive and statistically significant gains (p<.05) in performance, with average scores increasing from 57% on the pre-assessment to 65% on the post. (The MKT survey is designed to produce a median score of 50%.) (Riggan & Ebby, Forthcoming)
  • In another curriculum use study, the MKT was administered to 16 teachers, half of whom had participated in OGAP. While sample sizes were too small to test for statistical significance between the groups, the average score for OGAP teachers was 70%, with a range of 57-93%. For non-OGAP teachers the average score was 58%, with a range of 39-86%. (Oettinger, 2014).

Taken together, these two findings suggest that participating in OGAP may strengthen teachers’ mathematical knowledge for teaching—a finding consistent with self-reported survey responses.

OGAP teachers use a learning trajectory framework to analyze student work conceptually and develop instructional responses based on that analysis.

  • Results from an analysis of pre- and post-assessments given to OGAP teachers showed that OGAP teachers demonstrated statistically significant (p<.001) gains in their ability to identify both appropriate strategies for solving fractions problems, and student errors or misconceptions (Petit-Cunningham, 2008).
  • A study of teacher logs from 55 OGAP teachers found that OGAP teachers displayed a strong tendency to analyze student work for conceptual understanding, doing so in nearly half the instances in which they recorded item results. More tellingly, there was a strong correlation (r =.585, p<.01) between teachers’ tendency toward conceptual analysis and their ability to plan and execute new instructional strategies specifically based on those analyses. (Riggan & Ebby, Forthcoming)

On a test of grades 4-5 teachers’ \ formative assessment practices called the TASK (Teacher Assessment of Student Knowledge), OGAP teachers consistently provided responses that were learning-trajectory based; they explained the student work in relation to their understanding of multiplicative reasoning, they ranked the student work based on this understanding, and they suggested instructional next steps that were intended to build up from students’ current understanding towards more sophisticated strategies. (Oettinger, 2013; Riggan & Ebby, Forthcoming).  Other studies of the TASK show teachers often work at lower, more procedural, levels.

Impacts on Students

OGAP students show gains in math performance

  • A study of 936 students in grades 2-5 found that on average, student scores on pre- and post-assessments at the beginning and end of an instructional unit in which OGAP was used, increased 38 percentage points;  a six-month follow-up study showed that the students retained those gains. Further, students in the follow-up study who were were new to their schools (i.e., did not benefit from OGAP) had significantly lower mean scores (Vermont Mathematics Partnership, 2005)

OGAP students outperform non-OGAP students

  • A comparison between two fourth grade classes taught by OGAP-trained teachers and two fourth grade classrooms of teachers who did not participate in OGAP showed similar pre-assessment scores across the two groups. After participating in OGAP, students instructed by the OGAP teachers had statistically significantly larger gains on the post-assessment (Bunker, 2007)

References

Ambrose, R., Baek, J.-M., & Carpenter, T. P. (2003). Children’s invention of multidigit multiplication and division algorithms. In A. J. Baroody & A. Dowker (Eds.), The development of arithmetic concepts and skills: Constructing adaptive expertise (pp. 305-336). Mahwah, NJ: Lawrence Erlbaum Associates.

Bunker, J. (2007). The case for ongoing assessment based on cognitive research. Burlington, VT: University of Vermont.

Carpenter, T.P., Fennema, E. , Peterson, P.L. & Carey, D. A. (1988). Teachers pedagogical content knowledge of students’ problem solving in elementary arithmetic. Journal for Research in Mathematics Education, 19, 385-531.

Greer, B. (1992). Multiplication and division as models of situations. In D. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp 276-295) New York: Macmillan Publishing Company

Kaput, J. (1989). Supporting concrete visual thinking in multiplicative reasoning: Difficulties and opportunities. Focus on Learning Problem in Mathematics, 11, 35-47.

Lamon, S. (1994). Ratio and proportion: Cognitive foundations in unitizing and norming in multiplicative reasoning. In G. Harel and J. Confrey (Eds),  The development of multiplicative reasoning in the learning of mathematics  (pp.89-120). Albany: State University of New York Press.

Nesher, P. (1988). Multiplicative school word problems: Theoretical approaches and empirical findings. In J. Hiebert & M. Behr (Eds.), Number concepts and operations in the middle grades (pp. 19–40). Hillsdale, NJ: Erlbaum.

Oettinger, A.R. (2014). Curricular decisions: Using knowledge and information to guide practice (Doctoral dissertation). University of Pennsylvania, Philadelphia, PA.

Petit, M. (2011). Learning trajectories and adaptive instruction meet the realities of practice. In Daro, P., Mosher, F., & Corcoran, T. (2011). Learning trajectories in mathematics: A foundation for standards, curriculum, assessment, and instruction (Research Report #68). Philadelphia, PA: Consortium for Policy Research in Education.

Petit, M., Laird, R., & Marsden, E. (2010). A focus on fractions: Bringing research to the classroom. New York, NY: Routledge.

Petit, M., Laird, R., & Marsden, E. (2010). They get fractions as pies–but now what? Mathematics Teaching in the Middle School, 16(1), 5-10.

Petit, M. & Zawojewski, J.  (2010). Formative Assessment in Elementary Classrooms. Teaching and Learning Mathematics: Translating Research for Elementary School Teachers. Reston, VA: National Council of Teachers of Mathematics.

Petit-Cunningham, E. (2008). Preliminary analysis of the impact of OGAP fraction professional development and use on teacher knowledge. Burlington, VT: University of Vermont.

Riggan, M. & Ebby, C.B. (Forthcoming). Enhancing formative assessment in elementary mathematics:  The Ongoing Assessment Project. (Research Report). Philadelphia, PA: Consortium for Policy Research in Education.

Vermont Mathematics Partnership’s Ongoing Assessment Project (VMP OGAP). (2005). Unpublished student work samples and data. Montpelier, VT.