Data-driven instruction

Data-driven instruction is an educational approach that relies on information to inform teaching and learning. The idea refers to a method teachers use to improve instruction by looking at the information they have about their students. It takes place within the classroom, compared to data-driven decision making. Data-driven instruction works on two levels. One, it provides teachers the ability to be more responsive to students’ needs, and two, it allows students to be in charge of their own learning. Data-driven instruction can be understood through examination of its history, how it is used in the classroom, its attributes, and examples from teachers using this process.

History

Prior to the current emphasis on data and accountability in schools, some school leaders and education researchers focused on standards-based reform in education. From the idea of creating standards comes accountability, the idea that schools should report on their ability to meet the designated standards.[1] Late in the last century and in the early 2000s, an increased emphasis on accountability in public organizations made its way into the realm of education. With the passing of the No Child Left Behind (NCLB) Act in 2001 came laws requiring schools to provide information to the public concerning the quality of education provided to students. To be able to provide such data, states were mandated to create accountability measures and yearly assessments to gauge the effectiveness of schools in meeting those measures.[2][3] Following NCLB, more recent legislation under the Race to the Top Act further pushed states to use data gathering and reporting to demonstrate school’s ability to meet the demands of the public. Embedded in both NCLB and the Race to the Top Act is an assumption that the collection and use of data can lead to increased student performance.[4]

Attributes

Data in the classroom is any information that is visible during instruction that could be used to inform teaching and learning. Types of data include quantitative and qualitative data, although quantitative data is most often used for data-driven instruction. Examples of quantitative data include test scores, results on a quiz, and levels of performance on a periodic assessment.[5] Examples of qualitative data include field notes, student work/artifacts, interviews, focus groups, digital pictures, video, reflective journals.[6]

Quantitative and qualitative data is generally captured through two forms of assessments: formative and summative. Formative assessment is the information that is revealed and shared during instruction and is actionable by the teacher or student.[7] Paul Black and Dylan Wiliam offer examples of classroom assessment that is formative in nature, including student observations and discussions, understand pupils’ needs and challenges, and looking at student work.[7] Conversely, summative assessments are designed to determine whether or not a student can transfer their learning to new contexts, as well as for accountability purposes.[7] Formative assessment is the use of information made evident during instruction in order to improve student progress and performance. Summative assessments occur after teaching and learning occurred.

Examples

Understanding the differences between quantitative data vs. qualitative data, as well as formative assessment vs. summative assessment that tease out this data can be defined as assessment literacy.[5] Building assessment literacy also includes knowing when to use which type of assessment and the resulting data to use to inform instruction. The purpose of data driven instruction is to use information to guide teaching and learning. Dylan Wiliam offers examples of data driven instruction using formative assessment:

• Clarifying, sharing, and understanding learning intentions and criteria
• Eliciting evidence of learners’ achievement
• Providing feedback that moves learning forward
• Activating students as instructional resources for one another
• Activating students as owners of their own learning[8]

Because of the lack of timely feedback regarding the results plus the inability to personalize the approach, summative assessments are not readily used for data driven instruction in the classroom. Instead, a variety of information gleaned from different forms of assessments should be used to make decisions about student progress and performance within data-driven instruction. The use of multiple measures of different forms and at different times to make instructional decisions is referred to as triangulation.[5]

Data-Driven Instructional Systems

Background and origins

Data-Driven Instructional Systems refers to a comprehensive system of structures that school leaders and teachers design in order to incorporate the data into their instructions.[9] Building on organizational and school change literature, Richard Halverson, Jeffrey Grigg, Reid Prichett, and Chris Thomas developed a DDIS framework in an attempt to describe how relevant actors manage school-level internal accountability to external accountability.[9] Specifically, high-stakes external accountability policies such as No Child Left Behind Act (NCLB) was implemented to hold schools accountable for the reported standardized, summative assessment metrics. However, schools already had active internal accountability systems that place high emphasis on an ongoing cycle of instructional improvement based on the use of data including formative assessment results and behavioral information. Therefore, when the high-stakes accountability was implemented, schools naturally go through process of alignment between different types of data different purposes and the corresponding tension. Richard Halverson and his colleagues, employing case study approaches, explore leaders’ effort of coordination and alignment process which occurs between extant “central practices and cultures of schools” and “new accountability pressure” in a pursuit of improving student achievement score.[9]

Key concepts

In their article, Richard Halverson, Jeffrey Grigg, Reid Prichett, and Chris Thomas suggest that the DDIS framework is composed of six organizational functions: data acquisition; data reflection; program alignment; program design; formative feedback; test preparation.[9]

Data Acquisition

Data acquisition includes the data collection, data storage, and data reporting functions. “Data” in DDIS model is broadly conceptualized as any type of information that guides teaching and learning. In practice, schools collect academic data standardized assessment test scores, as well as non-academic data like student demographic information, community survey data, curricula, technological capacity, and behavioral records. In order to store such data, some schools develop their own local collection strategies using low-tech printouts and notebooks, whereas other schools rely on high-tech district storage systems, which provide tremendous amounts of reports. School leaders have discussions around which data needs to be reported and how to report the data in a way that they can use to guide teaching practices.

Data Reflection

In the DDIS model, data reflection refers to collectively making sense of the reported data.[9] District-level data retreats provide key opportunities for the schools within districts to identify the school-level strengths and weaknesses in terms of achievement data. Retreats help districts to develop district-level visions for instruction. In contrast, through local data reflection meetings, teachers have conversations focused on the individual students’ progress by examining each student’s performance on the assessed standards.

Program Alignment

Richard Halverson and his colleagues states that program alignment function refers to “link[ing] the relevant content and performance standards with the actual content taught in classroom.”[9] For example, the benchmark assessment results, as “problem-finding tools,” help educators to identify the curricular standards that are not aligned well with the current instructional programs.

Program Design

After identifying the main areas in relation to students learning needs and school goals, leaders and teachers design interventions: faculty-based programs; curriculum-based programs; and student-based programs. In an effort to improve the faculty’s data literacy, educators are provided with a variety of professional development opportunities and coaching focused on professional interaction (faculty-based programs). In addition, educators modify their curriculum as a whole-classroom approach (curriculum-based programs) or develop customized instructional plans taking into account individual students’ needs (student-based programs).

Formative Feedback

Educators interact with each other around the formative feedback on the local interventions implemented across classrooms and programs. Formative feedback systems are made of three main components: intervention, assessment, and actuation. Intervention artifacts here include curriculum materials like textbooks and experiments, or programs such as individualized education programs (Intervention). The effect of these intervention artifacts can be evaluated through formative assessments, either commercial or self-created, from the perspective that they had brought intended changes to teaching and learning (Assessment). In the actuation space, educators interpret the assessment results in relation to the initial goals of the intervention, and discuss how to modify the instruction delivery or assessment as measurement tools, which lays groundwork for the new interventions (Actuation).

Test Preparation

This function is not intended for teachers to “teach to the test.” Rather, it points to the following activities: curriculum-embedded activities, test practice, environmental design, and community outreach. Teachers incorporate the content of standardized assessment into their day-to-day instructions (curriculum-embedded activities), assist students to practice or be accustomed to test-taking with similar types of tests (test practice), and establish a favorable test-taking environment (environmental design). Further, teachers communicate with parents and the community members on the topics ranging from test implementation to interpreting the test results (community outreach).

Implications

For school districts

The primary implication for school districts is in ensuring high quality and relevant data is gathered and available. Beyond creating systems to gather and share the data, the school district must provide the expertise, in the form of data expert personnel and/or the access to professional development resources to ensure school building leaders are able to access and use the data.[10]

Another critical component of the responsibility of the district is to provide the leadership and vision to promulgate the use of information about student performance to improve teaching practice. Zavadsky and Dolejs suggest two areas for school districts to consider:

“The first is data collection and analysis. Districts and schools must carefully consider what data they need to collect, develop instruments with which to collect the data, and make the data available as soon as possible. The second component is data use. Principals and district leaders must give teachers sufficient time and training to understand the data and learn how to respond to what the data reveal”.[11]

While the literature shows the vital importance of the role of the district in setting the stage for data driven instruction, more of the work of connecting student performance to classroom practices happens at the school and classroom level.

For schools

Schools have a major role in establishing the conditions for data-driven instruction to flourish. Heppen, et al. indicate a need for a clear and consistent focus on using data and a data-rich environment to support teachers’ efforts to use data to drive instruction. When the leadership creates and maintains an environment which promotes collaboration and clearly communicates the urgency to improve student learning, teachers feel supported to engage in data use. The additional scaffold of modeling the use of data at the school level increases teachers’ expertise in the use of data.[12]

For teachers

Data-driven instruction is created and implemented in the classroom. Teachers have the most direct link between student performance and classroom practices. Through the use of data, teachers can make decisions about what and how to teach including how to use time in class, interventions for students who are not meeting standards, customizing lessons based on real-time information, adapting teaching practice to align to student needs, and making changes to pace, scope and sequence.[13]

To be able to engage in data-driven instruction, teachers must first develop the knowledge, skills, and dispositions required. Working in a school culture and climate in which data-driven instruction is valued and supported, teachers have the ability to increase student achievement and potentially reduce the achievement gap. Additionally, teachers must have access to learning opportunities or professional development which helps them understand the pedagogical framework and technical skills required to obtain, analyze, and use information about students to make instructional decisions.[14]

For students

A significant new growth in data-driven instruction is in having students shape their lessons using data about their own progress. Younger learners who are able to self-report regarding grades and other assessments can experience high levels of achievement and progress within instruction.[15] To embed data analysis by students into classroom practices, it requires time, training, and action.[16] The strategies that students use to evaluate their own learning vary in effectiveness. In a meta-analysis, Dunlosky, Rawson, Marsh, Nathan & Willingham ranked ten learning strategies based on the projected impact each would have on achievement:

Highly Effective Strategies:

• Practice testing - self-testing, solving practice problems
• Distributed practice - repetitive practice over specific intervals of time

Moderately Effective Strategies:

• Elaborative interrogation - explaining the 'why'
• Self-explanation - explaining how new information relates to what is already known
• Interleaved practice - mixing different kinds of problems in a practice session

Less Effective Strategies:

• Summarization - writing summaries of the material being learned
• Highlighting - marking parts of text that are important
• Keyword mnemonic - various mnemonic techniques
• Imagery use for text learning - attempt to form mental images of the things being read
• Rereading - read material again after initial reading[17]

It is worth noting that the less effective strategies may be more commonly used in K-12 classrooms than the moderately effective and highly effective strategies. The authors suggest that students should be taught how to use more effective techniques and when they are most helpful in guiding their learning. When these strategies become internalized, students will have developed techniques in order to learn how to learn. This is critical as they move into the secondary level and are expected to be more independent in their studies.

Criticisms

A major criticism of data driven instruction is that it focuses too much on test scores, and that not enough attention is given to the results of classroom assessments. Data driven instruction should serve as a “road map through assessment” that helps “teachers plan instruction to meet students’ needs, leading to better achievement”.[18] Summative assessments should not be used to inform the day-to-day teaching and learning that is supported by data-driven instruction. Additional problems associated with perceptions of data driven instruction include the limitations of quantitative data to represent student learning, not considering the social and emotional needs or the context of the data when making instructional decisions, and a hyperfocus on the core areas of literacy and mathematics while ignoring the encore, traditionally high-interest areas such as the arts and humanities.

Citations

1. Elmore, Richard F. Building a New Structure for School Leadership. Albert Shanker Institute.
2. Moriarty, Tammy Wu (May 2013). Data-driven decision making: Teachers' use of data in the classroom (Thesis). ProQuest 1432373944.
3. Larocque, M (2007). "Closing the Achievement Gap: The Experience of a Middle School". Clearing House. 80 (4): 157–162. doi:10.3200/tchs.80.4.157-162.
4. Kennedy, Brianna L.; Datnow, Amanda (December 2011). "Student Involvement and Data-Driven Decision Making: Developing a New Typology". Youth & Society. 43 (4): 1246–1271. doi:10.1177/0044118X10388219.
5. Boudett, K. P.; City, E. A.; Murname, R. J. (2013). Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning. Cambridge, MA: Harvard Education Press.
6. Dana, N. F.; Yendol-Hoppey, D. (2014). The Reflective Educator's Guide to Classroom Research: Learning to Teach and Teaching to Learn Through Practitioner Inquiry (3rd ed.). Thousand Oaks, CA: Corwin.
7. Black, P; Wiliam, D. (1998). "Inside the Black Box: Raising Standards Through Classroom Assessment". Phi Delta Kappan. 80 (2): 139–148.
8. Wiliam, Dylan (2011). Embedded Formative Assessment. Bloomington, IN: Solution Tree.
9. Halverson, Richard; Grigg, Jeffrey; Prichett, Reid; Thomas, Chris (March 2007). "The New Instructional Leadership: Creating Data-Driven Instructional Systems in School" (PDF). Journal of School Leadership. 17 (2): 159–194. doi:10.1177/105268460701700202.
10. Swan, G.; Mazur, J. (2011). "Examining data driven decision making via formative assessment: A confluence of technology, data interpretation heuristics and curricular policy". Contemporary Issues in Technology and Teacher Education. 11 (2): 205.
11. Zavadsky, H.; Dolejs, A. (2006). "DATA: Not Just Another Four-Letter Word". Principal Leadership, Middle Level Ed. 7 (2): 32–36.
12. Heppen, Jessica; Faria, Ann-Marie; Thomsen, Kerri; Sawyer, Katherine; Townsend, Monika; Kutner, Melissa; Stachel, Suzanne; Lewis, Sharon; Casserly, Michael. Using Data to Improve Instruction in the Great City Schools: Key Dimensions of Practice. Urban Data Study. Council of the Great City Schools.
13. Hamilton et al. - 2009 - Using student achievement data to support instruct.pdf. (n.d.). Retrieved from http://files.eric.ed.gov/fulltext/ED506645.pdf
14. Furlong-Gordon, Jean Marie (November 2009). Driving classroom instruction with data: From the district to the teachers to the classroom (Thesis). ProQuest 250914319.
15. Hattie, J. (2012). Visible Learning for Teachers: Maximizing Impact on Learning. New York: Routledge.
16. Depka, Eileen (March 29, 2019). Letting Data Lead: How to Design, Analyze, and Respond to Classroom Assessment. Bloomington, IN: Solution Tree. p. 106.
17. Dunlosky, J.; Rawson, K. A.; Marsh, E. J.; Nathan, M. J.; Willingham, D. T. (2013). "Improving students' learning with effective learning techniques promising directions from cognitive and educational psychology". Psychological Science in the Public Interest. 14 (1): 4–58. doi:10.1177/1529100612453266. PMID 26173288. S2CID 1621081.
18. Neuman, Susan B. (November 2016). "Code Red: The Danger of Data-Driven Instruction". Educational Leadership. 74 (3): 24–29.

Additional References

• Black, P.; Wiliam, D. (1998). "Inside the Black Box: Raising Standards Through Classroom Assessment". Phi Delta Kappan. 80 (2): 139–148.
• Boudett, K. P., City, E. A., Murname, R. J. (2013). Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning. Cambridge, MA: Harvard Education Press.
• Dana, N. F. & Yendol-Hoppey, D. (2014). The Reflective Educator’s Guide to Classroom Research: Learning to Teach and Teaching to Learn Through Practitioner Inquiry (3rd Ed.). Thousand Oaks, CA: Corwin.
• Datnow, A., & Park, V. (2014). Data-Driven Leadership. San Francisco, CA: Jossey-Bass.
• Dunlosky, J.; Rawson, K. A.; Marsh, E. J.; Nathan, M. J.; Willingham, D. T. (2013). "Improving students' learning with effective learning techniques promising directions from cognitive and educational psychology". Psychological Science in the Public Interest. 14 (1): 4–58. doi:10.1177/1529100612453266. PMID 26173288.
• Eagle, M., Corbett, A., Stamper, J., McLaren, B. M., Baker, R., Wagner, A., ... Mitchell, A. (2016). Predicting Individual Differences for Learner Modeling in Intelligent Tutors from Previous Learner Activities (pp. 55–63). ACM Press. doi:10.1145/2930238.2930255
• Elmore, R. F. (2000). Building a new structure for school leadership. Albert Shanker Institute. Retrieved from http://eric.ed.gov/?id=ED546618
• Gold, S (2005). "DRIVEN by DATA". Technology & Learning. 25 (11): 6, 8–9.
• Halverson, R., Grigg, J., Prichett, R., & Thomas, C. (2007). The New Instructional Leadership: Creating Data-Driven Instructional Systems in School. Journal of School Leadership, 17(March), 159–194.
• Hamilton et al. - 2009 - Using student achievement data to support instruct.pdf. (n.d.). Retrieved from http://files.eric.ed.gov/fulltext/ED506645.pdf
• Hamilton, L., Halverson, R., Jackson, S. S., Mandinach, E., Supovitz, J. A., Wayman, J. C., ... Steele, J. L. (2009). Using student achievement data to support instructional decision making. Retrieved from http://repository.upenn.edu/gse_pubs/279/
• Hattie, J. (2012). Visible Learning for Teachers: Maximizing Impact on Learning. New York: Routledge.
• Heppen, J., Faria, A.-M., Thomsen, K., Sawyer, K., Townsend, M., Kutner, M., ... Casserly, M. (2010). Using Data to Improve Instruction in the Great City Schools: Key Dimensions of Practice. Urban Data Study. Council of the Great City Schools. Retrieved from http://eric.ed.gov/?id=ED536737
• Johnson, L. (2009). Randi Weingarten, President Antonia Cortese, Secretary-Treasurer. Retrieved from http://eric.ed.gov/?id=ED511575
• Jones, A (2005). "The Myths of Data-Driven Schools. Principal Leadership". Middle Level Ed. 6 (2): 37–39.
• Kennedy, B. L.; Datnow, A. (2011). "Student Involvement and Data-Driven Decision Making Developing a New Typology". Youth & Society. 43 (4): 1246–1271. doi:10.1177/0044118X10388219.
• Larocque, M (2007). "Closing the Achievement Gap: The Experience of a Middle School". Clearing House. 80 (4): 157–162. doi:10.3200/tchs.80.4.157-162.
• Melucci, L. (2013, August). TEACHER PERCEPTIONS AND USE OF DATA-DRIVEN INSTRUCTION.pdf. Capella University.
• Mokhtari, K.; Rosemary, C. A.; Edwards, P. A. (2007). "Making Instructional Decisions Based on Data: What, How, and Why". Reading Teacher. 61 (4): 354–359. doi:10.1598/rt.61.4.10.
• Pella, S (2012). "What Should Count as Data for Data-Driven Instruction? Toward Contextualized Data-Inquiry Models for Teacher Education and Professional Development". Middle Grades Research Journal. 7 (1): 57–75.
• Rogers, L. N., & Tyndall, P. D. (2001). Teachers’ Perspectives: Developing Instructional Leadership through Classroom Inquiry. Retrieved from http://eric.ed.gov/?id=ED465596
• Schmidt, W. H.; Burroughs, N. A.; Zoido, P.; Houang, R. T. (2015). "The Role of Schooling in Perpetuating Educational Inequality An International Perspective" (PDF). Educational Researcher. 44 (7): 371–386. doi:10.3102/0013189x15603982.
• Schmoker, M. (1996). Results: the key to continuous school improvement. Alexandria, VA: Association for Supervision and Curriculum Development.
• Shanahan, T., Callison, K., Carriere, C., Duke, N. K., Pearson, P. D., Schatschneider, C., & Torgesen, J. (2010). Improving Reading Comprehension in Kindergarten through 3rd Grade: IES Practice Guide. NCEE 2010-4038. What Works Clearinghouse. Retrieved from http://eric.ed.gov/?id=ED512029
• Stamper, J., Ed, Pardos, Z., Ed, Mavrikis, M., Ed, McLaren, B. M., Ed, & International Educational Data Mining Society. (2014). Proceedings of the Seventh International Conference on Educational Data Mining (EDM) (7th, London, United Kingdom, July 4–7, 2014). International Educational Data Mining Society. http://www.educationaldatamining.org
• Swan, G.; Mazur, J. (2011). "Examining data driven decision making via formative assessment: A confluence of technology, data interpretation heuristics and curricular policy". Contemporary Issues in Technology and Teacher Education. 11 (2): 205.
• Wiliam, D. (2011). Embedded Formative Assessment. Bloomington, IN: Solution Tree.
• Zavadsky, H., & Dolejs, A. (2006). DATA: Not Just Another Four-Letter Word. Principal Leadership, Middle Level Ed., 7(2), 32–36.