This literature review is excerpted from my work on a research proposal in partial fulfillment of EDUC 800 from the University of Kansas. I am posting it to this blog to highlight alignment between current research and the ANIE & SNAP assessments.

The larger research proposal is based on this theme:

What if an assessment can be formative and summative in nature, while at the same time providing learning experiences for the students through doing it? This assessment would be “for, of, and as learning” (Earl, 2006) – a unicorn of sorts in the educational world. This research proposal will delve into the impact and validity of such an assessment, one that asserts its potential as an important technological change in mathematics instruction.

Literature Review – The Underpinnings of ANIE & SNAP

ANIE was based on a theory that a balanced approach to assessing and teaching math could and should mirror a balanced literacy approach. In essence, the good work that had been done in the teaching of reading could be used in the math context.

Dual Lens Model

To illustrate this idea, the Dual Lens model was developed and presented in The ANIE. This model showed that literacy and numeracy both had Skill, Fluency and Comprehension as building block components of a balanced program (Figure 1).

Figure 1 – Dual Lens Model (Bird & Savage 2014, pg. 55)

• Number Work includes the skill development of traditional calculations and algorithms of math.
• Number Fluency is based on the concept of automaticity – if one is able to do something fluently, then brain processing power can be used for other executive functioning. Schneider (n.d.) suggests that automaticity in cognitive function frees up 90% of working memory for higher-order skills (Schneider and Shiffrin, 1977; Shiffrin and Schneider, 1977). The mathematical tasks in this area include rapid recall of time tables, math facts, and practice in all areas of the math curriculum.
• Comprehension includes the ability to analyze, apply and interpret math
• Communication is the ability to share thinking through oral or written means.

Additionally, key literacy strategies such as making predictions (estimating in math), visualization (creating images or models), connecting via text to text, text to self or text to world (real life connections), and metacognition (reflection) were all woven into the framework of ANIE to create a numeracy assessment that borrowed heavily from literacy teaching practices.

Assessment – For, Of and As Learning

The educational theory behind the premise of The ANIE (2014) is that an assessment can be used three main ways (Stiggins, 2002, 2015):

1. As a formative assessment – for learning
2. As a summative assessment – of learning
3. As a learning experience – as learning

“Assessment as teaching and learning. Assessments and instruction can be one and the same, if and when we want them to be.” (Stiggins, 2015)

What is special about The ANIE, (and by association SNAP), is the assertion that an assessment can do all three of these things at the same time if it is designed correctly. A good design is both open and authentic in nature. An open assessment is one where students have the ability to show their learning in a variety of non-predetermined ways. Authenticity is achieved when students can make personal and/or real-life connections to their learning tasks (Shepard, 2000).

An assessment that is open by design can be an effective learning tool to assist students and teachers to find learning gaps, practice skills and finally demonstrate mastery. This idea that an assessment can be practiced over and over again – providing feedback through formative assessment practices, until a student “gets it” is one that is not well used in traditional math classes.

The power of this approach is that a formative assessment, upon successful demonstration of learning, becomes a summative assessment – simultaneously. This idea was discussed by Bell and Cowie (2000) and later by Dunn and Mulvenon (2009). They assert that it is how the assessment data are used that determines the label we give the assessment. For instance, a test that students do poorly on can be used to inform the teacher that further teaching is needed, and/or that students are not learning. Acting on this information would make the assessment formative in nature. If the teacher used the same data to fail students, the assessment would be summative in practice. The same assessment can be used in multiple ways to achieve differing ends. Bell and Cowie go on to discuss the how formative assessment can morph and be used for teaching “conceptual development”.

“In taking into account students’ thinking in their teaching, teachers are responding to and interacting with the students’ thinking that they have elicited in the classroom. They are therefore undertaking formative assessment while teaching for conceptual development.” (p. 538)

Their idea of conceptual development is closely aligned with the third main way that ANIE can be used as an assessment tool – as a learning experience termed ‘as learning’.

History of ANIE

The research contained in The ANIE was limited primarily to action research in two elementary schools. These pilot schools showed incredible results in both improved student achievement (as gauged against the BC Foundational Skills Assessment) and teacher practice. The action research (unpublished school-based studies) showed that the classrooms that used ANIE extensively had significantly better achievement results on the FSA than classrooms that did not. Building upon this early success, a district team of educators refined the ANIE approach and rebranded it as the Student Numeracy Assessment & Practice (SNAP) to begin the district initiative.

The ANIE was born from evidence-based practices in literacy, but there is a lot of research supporting the benefits of such an approach in mathematics as well (Roepke & Gallagher, 2015; Zollman, 2009).

“Literacy activities, designed to help students negotiate and create text, can be adapted for use in math classrooms.”

“It is not a matter of conceptualizing entirely new methods for teaching mathematics, but rather of incorporating the established methods described by literacy educators with math instruction.” (Draper, 2002; p. 528)

This connection is especially evident when the math skills are broken into the following distinct literacy strategies:

Making Predictions (Estimating Strategies)

Making and justifying a prediction in math is a key strategy in understanding the mathematical concept as well as self-checking the math calculation. Estimating strategies assist students in making close guesses that add to conceptual understanding and real-life connections. Students who are able to use estimation strategies are more likely to enjoy math and engage in alternative solutions. Key estimation strategies are found in strong mathematics students and impact number skills, cognitive processes, and affective attributes (Reys, 1982; Simms et. al., 2016).

Imaging Strategies

Taking the abstract and creating concrete representation is an important skill for mathematicians. Comprehension is enhanced with concrete representation. This can be in the form of a drawing, a manipulative, a graph, or even a mental image. Indeed, conceptual comprehension is not possible without the ability to image, as imaging is an important stage of comprehension. (Bird & Savage, 2015; Bower & Morrow, 1990) First, if one can image (create concrete from abstract) in their mind; second, they can make inferences from their mental model and finally justify an interpretation. Additionally, it is important that students are able to represent their work and understanding in a multi-representational fashion. This approach caters to students with different skills and dispositions, and; providing that the result is reasonable, the pathway and mode of representation is valued (Zevenbergen et. al., 2008).

Open Ended Questions

It is clear that open ended questioning in math can support higher achievement levels as students are required to consider multiple approaches and options to find solutions (Lee et. al., 2012). Open ended questions “induce more cognitive strategy” use than closed questions such as multiple choice or right/wrong answers (O’Neil 1998). A 3-year study involving students from two different schools found that students who are taught regularly from a pedagogy of open-ended activities performed better than students who were taught a more traditional, textbook based approach (Boaler 1998). A similar study by Çakır and Cengiz (2016) also showed improved engagement and achievement of students when taught using open ended questioning techniques in elementary classrooms.

Real Life Connections

Helping students make real life connections to math is another foundational piece of comprehension acquisition. For math concepts to stick, it is important that students are able to apply and connect them to their lives. Keene and Zimmerman (1997) concluded that this is best done when students make text to text, text to self, and text to world connections. Making connections is not only important for student engagement and understanding but is also a way to address issues of confidence and self-efficacy (Stoehr, 2015). “The pedagogical practice of connecting mathematical content to real world contexts, particularly contexts relevant to students’ knowledge and experiences, can positively impact student motivation as well as promote conceptual understanding.” (Sugimoto et al., 2017).

Metacognitive Strategies

Teaching students how to improve their metacognitive skills has a significant impact on their understanding and achievement in all teaching domains, including mathematics (Hattie, 2009). The term metacognition refers to an individual’s awareness and ability to control their thinking processes. It can also be defined as ‘thinking about thinking’. In multiple studies by Ozsoy and Ataman (2009, 2011), grade 5 students who were explicitly taught metacognitive strategies performed significantly better in mathematical problem-solving achievement than control groups that received no such training.

Other Key Features

Cultural Sensitivity

The SNAP is designed to be culturally sensitive. Because the students are required to create their own “real life connections” to the math, they are able to draw upon their life experiences. This means that the assessment is not imposing on the students an unconnected mathematical situation or story that requires a background knowledge to access the context. Students use their home language to show their understanding of mathematical concepts and learning, an important principal of creating equitable practice in diverse classrooms (Zevenbergen et. al., 2008).

Focus on Math – Not Reading

Because the SNAP is focused on the students bringing their own background knowledge to the assessment, students are not required to read contextual material. They simply interact with the math. This not so subtle difference is a potential game changer for students who have relatively high math proficiency and low literacy skills. As such, international students, bilingual students, or any student that struggles with reading should have improved achievement when utilizing this assessment.

One Question

SNAP supports a deep dive into one question. The assessment is only one page long and is organized graphically so as to not overwhelm the student.

Discourse

The literature, although overwhelming in the support of the foundational pieces surrounding SNAP/ANIE, did raise a number of cautionary flags. One concern is related to the concept of real-world connections to math and teacher efficacy. Teachers often lack the skills, abilities, and time needed to implement such connections to their lesson planning. Combined with limited professional development opportunities and scarce resources, it is difficult for teachers to make these connections on an ongoing basis (Sugimoto et al., 2017).

This complexity should not be minimized. The SNAP is on the surface a simple one page numeracy assessment, but it is a doorway into the complex minds of students. Teachers struggle with this adjustment, and some cannot make it without support – making this initiative difficult to implement on a large scale.

Additionally, who benefits more from this approach? Who might benefit less? Will lower socioeconomic schools show greater benefit?

References

Barton, C. (2018). On formative assessment in math: How diagnostic questions can help. American Educator, 42(2), 33-38.

Bird, K. & Savage, K. The ANIE. (2014) Markham, ON: Pembroke Publishers

Bird, K. & Savage, K. PLAN for Better Learning. (2015) Markham, ON: Pembroke Publishers

Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86, 8–21.

Boaler, J. (1998). Open and closed mathematics: Student experiences and understanding.Journal for Research in Mathematics Education, 29(1), 41.

Boaler, J., Munson, J., & Williams, C. (2018). Mindset mathematics: Visualizing and investigating big ideas, grade 3 Jossey-Bass, An Imprint of Wiley. 10475 Crosspoint Boulevard, Indianapolis, IN 46256.

Bower, G., & Morrow, D. (1990). Mental models in narrative comprehension. Science, 247(4938), 44-48.

Çakır, H., & Cengiz, Ö. (2016). The use of open ended versus closed ended questions in turkish classrooms. Open Journal of Modern Linguistics, 6, 60-70. http://dx.doi.org/10.4236/ojml.2016.62006

Chinnappan, M. (2006). Using the productive pedagogies framework to build a community of learners online in mathematics education. Distance Education, 27(3), 355-369.

Draper, R. (2002). School mathematics reform, constructivism, and literacy: A case for literacy instruction in the reform-oriented math classroom. Journal of Adolescent & Adult Literacy,45(6), 520-529. Retrieved from http://www.jstor.org/stable/40014740

Dunn, Karee E and Mulvenon, Sean W. (2009). A critical review of research on formative assessments: The limited scientific evidence of the impact of formative assessments in education. Practical Assessment Research & Evaluation, 14(7). Available online: http://pareonline.net/getvn.asp?v=14&n=7

Earl, L. M., & Division, M. S. P. (2006). Rethinking Classroom Assessment with Purpose in Mind : Assessment for Learning, Assessment as Learning, Assessment of Learning: Government of Manitoba.

Faruk Çetin, Ö. and K. Köse (2015). The relationship between eighth grade elementary students’ operational and measurable prediction skills and mathematical literacy. American Journal of Educational Research 3(2): 142-151.

Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London: Routledge.

Jackson, K., Garrison, A., Wilson, J., Gibbons, L., & Shahan, E. (2013). Exploring relationships between setting up complex tasks and opportunities to learn in concluding whole-class discussions in middle-grades mathematics instruction. Journal for Research in Mathematics Education, 44(4), 646-682.

Keene, E. & Zimmerman, S. (1997). Mosaic of Thought. Portsmouth, NH: Heinemann.

Lee, Y., Kinzie, M. B., & Whittaker, J. V. (2012). Impact of online support for teachers’ open-ended questioning in pre-k science activities. Teaching and Teacher Education, 28(4), 568-577.

Lucangeli, D. and Cornoldi, C. (1997). Mathematics and metacognition: What is the nature of the relationship? Mathematical Cognition, 3 (2), 121-139.

Mottram, D. R. (1995). A comparative study of computational estimation ability and strategies used in estimation problems. Unpublished doctoral dissertation, Faculty of the Graduate School, Colorado University, Colorado, USA.

O’Neil, H. F., Jr., & Brown, R. S. (1998). Differential effects of question formats in math assessment on metacognition and affect. Applied Measurement in Education, 11(4), 331-51.

Ozsoy, G. & Ataman, A. (2009). The effect of metacognitive strategy training on problem solving achievement. International Electronic Journal of Elementary Education, 1(2), 67–82.

Oszoy, G. (2011). An investigation of the relationship between metacognition and mathematics achievement. Asia Pacific Education Review, 12, 227–235.

Reys, R., Rybolt, J., Bestgen, B., & Wyatt, J. (1982). Processes used by good computational estimators. Journal for Research in Mathematics Education, 13(3), 183-201. doi:10.2307/748555

Reys, B. J. (1986). Teaching computational estimation: Concepts and strategies. In H. L. Shoen and W. J. Zweng (Eds.), Estimation and mental computation-1986 year book (pp. 31-44). VA: National Council of Teachers of Mathematics.

Reys, B. J., Reys, R. E., and Penafiel, A. F. (1991). Estimation performance and strategy use of Mexican 5th and 8th grade student sample. Educational Studies in Mathematics, 22, 353-375.

Reys, R. E. and Yang, D. (1998). Relationships between the computational performance and number sense among sixth and eighth grades in Taiwan. Journal for Research in Mathematics Education, 29 (2), 225-237

Rohrer, D., Dedrick, R. F., & Stershic, S. (2015). Interleaved practice improves mathematics learning. Journal of Educational Psychology, 107(3), 900-908.

Roembke, T. C., Hazeltine, E., Reed, D. K., & McMurray, B. (2019). Automaticity of word recognition is a unique predictor of reading fluency in middle-school students. Journal of Educational Psychology, 111(2), 314-330.

Tena L. Roepke, & Debra K. Gallagher. (2015). Using Literacy Strategies to Teach Precalculus and Calculus. The Mathematics Teacher, 108(9), 672-678. doi:10.5951/mathteacher.108.9.0672

Schneider, W., & Shiffrin, R. M. (1977). Controlled and automatic human information processing: I. detection, search, and attention. Psychological Review, 84(1), 1-66.

Schneider, W. (n.d.). Automaticity in Complex Cognition. Retrieved May 19, 2019, from https://web.archive.org/web/20080509061115/http://coglab.psy.cmu.edu/index_main.html
Tracking the development, performance and biology of automatic component skills.

Schneider, W., & Chein, J. M. (2003). Controlled & automatic processing: behavior, theory, and biological mechanisms. Cognitive Science, 27(3), 525-559.

Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), 4-14.

Shiffrin, R. M., & Schneider, W. (1977). Controlled and automatic human information processing: II. perceptual learning, automatic attending and a general theory. Psychological Review, 84(2), 127-190.

Simms, V.M., Clayton, S., Cragg, L., Gilmore, C., & Johnson, S. (2016). Explaining the relationship between number line estimation and mathematical achievement: The role of visuomotor integration and visuospatial skills. Journal of experimental child psychology, 145, 22-33 .

Stiggins, R.J. (2015) 7 Principles of Student Centered Classroom Assessment. Retrieved May 19, 2019 from http://rickstiggins.com/2014/03/04/7-principles-of-student-centered-classroom-assessment/

Stiggins, R. J. (2002). Assessment crisis: The absence of assessment for learning. Phi Delta Kappan, 83(10), 758–765. https://doi.org/10.1177/003172170208301010

SNAP (Student Numeracy Assessment and Practice). (April 19, 2019) Retrieved April 19, 2019 from https://snap.sd33.bc.ca

Sowder, J. (1992). Estimation and number sense. In D, A, Grouws (Ed.), Handbook of Research on Mathematics Teaching and Learning, pp. 371-389, New York: Macmillan.

Stoehr, K., Turner, E., & Sugimoto, A. (2015). One teacher’s understandings and practices for real-world connections in mathematics. In T. G. Bartell, K. N. Bieda, R. T. Putnam, K. Bradfield, & H Dominguez (Eds.), Proceedings of the 37th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 1150 –1153). East Lansing, MI: Michigan State University.

Sugimoto, A. T., Turner, E. E., & Stoehr, K. J. (2017). A case study of dilemmas encountered when connecting middle school mathematics instruction to relevant real world examples.Middle Grades Research Journal, 11(2), 61-82.

Suurtamm, C., Thompson, D. R., Kim, R. Y., Moreno, L. D., Sayac, N., Schukajlow, S., . . . Vos, P. (2016). Assessment in Mathematics Education. In C. Suurtamm, D. R. Thompson, R. Y. Kim, L. D. Moreno, N. Sayac, S. Schukajlow, E. Silver, S. Ufer, & P. Vos (Eds.), Assessment in Mathematics Education: Large-Scale Assessment and Classroom Assessment (pp. 1-38). Cham: Springer International Publishing.

Swan, M., & Burkhardt, H. (2012). A designer speaks: Designing assessment of performance in mathematics. Educational Designer: Journal of the International Society for Design and Development in Education, 2(5), 1–41.

Zevenbergen, R, Niesche, R, Grootenboer, P, Boaler, J. (2008). Creating equitable practice in diverse classrooms: Developing a tool to evaluate pedagogy.

Zollman, A. (2009b). Students using graphic organizers to improve problem solving, Middle School Journal, 41(3), 4-12.