In an age of boundless technological advances and increased accountability, school districts nationwide increasingly rely upon innovative tools that allow administrators and educators to use high-quality student data to inform decisions. Organizations, such as the following, have developed resources to help school districts adopt the best data governance practices and technological solutions to adequately protect student privacy and improve student achievement.
The U.S. Department of Education Privacy Technical Assistance Center (PTAC), established as “a ‘one-stop’ resource for education stakeholders to learn about data privacy, confidentiality, and security practices related to student-level longitudinal data systems;”
The Data Quality Campaign (DQC), a partnership of approximately 100 organizations “committed to realizing the vision of an education system in which all stakeholders—from parents to policymakers—are empowered with high-quality data from the early childhood, K–12, postsecondary, and workforce systems;” and
Closing the Gap: Turning SIS/LMS Data into Action, which exists as a collaboration of the American Association of School Administrators (AASA), the Consortium for School Networking (CoSN), and Gartner Inc., a technology research and advisory firm.
Data Use
Research shows that most data management systems for education function as either student information systems (which focus on the collection, organization, and management of student data) or learning management systems (which are used for planning, delivering and managing, tracking, and reporting learner events, programs, records, and training content). While in the past, districts have found basic student information systems sufficient to serve reporting and administrative purposes, in the current era of increased accountability, districts increasingly use student data to improve instruction and guide decision making. As such, leaders must adequately plan for data use, selectively choose data sets, and adequately train all data users.
Planning for data use
Research recommends districts develop a “data collection and use plan” for each type of data the district intends to evaluate. The data collection and use plan should include a description of the data, the source where the data is stored prior to the implementation of the new data collection system, the data extraction method and frequency of extraction, the “data owner” or “point of contact” responsible for that type of data, the groups that will use the data, and intended uses for the data once it has been collected.
During the planning process, district leaders may also take careful steps to ensure stakeholders throughout all levels of the district have the tools and support to effectively use data.
Data selection
Because districts increasingly collect student data to serve a variety of purposes, leaders may find the data collected for reporting purposes to be insufficient for guiding decisions on the district, school, and classroom levels. As research shows that there is often “a fundamental misalignment between the types of data that districts deem to be imperative for student achievement and the types of data that are being requested by the state for federal accountability purposes,” Closing the Gap recommends that districts choosing to collect additional categories of data consider restraints on data collection and the district’s ability to store and analyze data. In assessing data collection feasibility, this recommends that district leaders consider the following questions:
Is the data you wish to collect protected by laws and government mandates (e.g. Health Insurance Portability and Accountability Act – HIPAA, Children’s Online Privacy Protection Act – COPPA, Family Educational Rights and Privacy Act – FERPA, etc.)?
Do you have the resources or technical bandwidth and capacity to store the data? Once collected from every student within the district, will the data be too large to store in your current database/data warehouse (e.g. photos of every student and their parents/guardians)?
Do you have the staff to update and maintain the data for accuracy (e.g. you may want daily trend analysis of how assessments [go] in every classroom in every school but teachers may not have the time to conduct such an analysis on a daily basis)?
Do you have the ability to display the data on different types of displays (e.g., computer screens, smart phones, tablets, etc.)?
Is the data currently collected somewhere? How accurate is that data?
Although districts may find they have access to a large amount of data, Closing the Gap warns against the collection of more data than the district will have the “capacity or capability to store, mine, and analyze.” Accordingly, it is recommended that district leaders consider the following questions to prioritize which pieces of data they choose to collect:
Does the data element answer a question that directly supports current and future school, district, state, and national education goals?
Does the district currently have a means for accurately collecting and displaying the data element across the district?
Does the district have a means of securely and accurately storing/maintaining the data?
Finally, it is recommended that district leaders select data after consideration of the eventual use of the data, using questions such as these:
What are the instructional questions the data should answer?
Which professional learning resources are needed to support the effective use of the data to strengthen classroom practices?
Which evidence-based instructional practices will the data further enable (e.g., the role of feedback and assessment for learning)?
Training and Professional Development
In most cases, district leaders, school leaders, and teachers will require training and professional development to effectively collect and use student data. Research finds multiple training practices commonly in place in best-practice districts:
Best-practice districts provide twice as much data-related training and professional development for new teachers and 50 percent more data-related training and professional development for returning teachers than other districts.
Best-practice districts devote a greater proportion of training and professional development to data analysis than to technology. The result of this practice has been “tangible differences, such as the use of advanced data analysis techniques including correlations, regression, analysis of variance and multivariate analysis.”
Educators with data analysis training more frequently use available analytical tools. In best-practice districts that provided training in data analysis training, 63 percent of teachers and principals used analytical tools, in comparison with six percent of teachers and 14 percent of principals employed by other districts.
Data collection efforts, such as the U.S. Department of Education Civil Rights Data Collection (CRDC), use innovative technologies and web-based tools to facilitate the collection of data. As we increasingly use data to support student achievement in our schools, guiding principles of data collection and use will help shape the way that data is ultimately used to support achievement in our schools.
Expert insight
To provide additional insight into compelling questions and topics around the use of data in schools, we spoke with Dr. Todd Johnson, the Director of Research, Capital Region ESD 113, Tumwater, Washington. Below we’ve included some highlights from our conversation with Dr. Johnson.
Hanover: In your work, have you found that there are certain components of applying data that are the most challenging for district administrators and regional service agencies? Do you have any advice for overcoming these challenges?
Todd Johnson: Data is ubiquitous, and with that ubiquity comes the need to somehow filter, slice, and synthesize this stuff to understand and maximize program effectiveness. Whether you are looking at quantitative, qualitative, or even the mix of the two data layers many folks get pretty overwhelmed. Getting to the essential data that maximizes the program effectiveness is what most folks in the field are finding the most challenging. To overcome this challenge, my mind quickly goes to how we need to “TRIM” things for better understanding and informed decision making.
In my thinking, “TRIM” is an acronym that helps me organize what needs to be done. First is “T” which stands for “Time”. Time is needed for the leader and/or team to get together and identify the goal, objective(s), and all the other essential ingredients to maximize the effectiveness of the program. Sadly, without the “T” for time you will always stay on the outer edge of your ability to maximize data informed effectiveness. Next thing is the “R” and this is “Resources” and these are the leader and/or team assets to help them function effectively to maximize program effectiveness. These include space for meetings, materials and supplies budget, etc. Next is the “I” for “Information” and this is where that data ubiquity comes into play. This is the place by which folks gather facts and vital insights to what all the data is telling them and also helps establish where they may want to go. Okay, the last one is the “M” and this stand for “Management”. This is where the leader and/or team actually need to have some sort of mechanism to continually keep focused on making decisions and focusing in on their goal and problem using the essential data elements.
Hanover: Are there specific actions that district offices can take to play a stronger role in program evaluation and evaluating resource-spend?
Todd Johnson: District Offices {DO} should consider the use of more embedded evaluations. Essentially embedded evaluations could be regarding the implementation of anything related to curriculum, instruction, assessment, and even an environmental change that is being considered to maximize program effectiveness. Districts can look at these embedded evaluations as opportunities to Discover, Uncover, & Evaluate {DUE} what works for their districts and schools. For some they may see this as action research, and this is okay also, but for many district offices and school leaders they will find the term “evaluation” being better received by those whose children will be a part of opportunity.
Hanover: In your experience, what is the most-underused application of data in the K12 industry
Todd Johnson: Student experiential data is the most-underused data element informing the K-12 industry. It’s important to also keep in mind that “student” in this “sense/cents” also includes the educators in the system who are also learning. It “seems/seams” our educational system is truly still challenged by how best to capture and address student voice, both verbal and no-verbal to maximize program effectiveness in real-time.
Hanover: How did your experience working in Higher Education prepare you for your role as a Director of Research and Data Analysis in K12? What do you see as being the same and different between PK-12 and Higher Education?
Todd Johnson: My experience was that higher education and the K-12 system both have the same goal of maximizing the learner’s experiences. Given those experiences, the transition from the higher education to k-12 system was pretty seamless. The mission of the Capital Region ESD 113 focused on “Our collective purpose is to ensure excellent & equitable education for all students through service and collaboration” also made the transition easier as it aligned with my own personal goals. However, probably one of the biggest advantages that the university offered was the connectivity to some of the biggest and brightest thinkers in the world. Seems like a lot of my work at the university was learning and connecting with many of these national and international thinkers to keep learning from them and transferring this into the classroom with the university students. This international and national connectivity was priceless and even today I continue to reach out to this brilliance. K-12 education, and the emerging Birth to 12 educational field is much more action oriented and focused.
Hanover: Please share any additional thoughts on ways that the district offices can advance research, evaluation, measurement and its practices.
Todd Johnson: District offices really need to continue with the training and education of staff regarding data informed decisions, privacy, confidentiality, and security practices. As the data starts to accumulate in longitudinal systems there is an ever increasing opportunity to learn from it, but there is also an ever increasing responsibility to protect those in it. Developing agreements with researchers, evaluators, and measurement folks on the front-end regarding the topics of why, who, when, and for how long are super important. District offices would also do well to schedule yearly reviews of any results of their participation in research, evaluation, and measurement activities to continually seek to understand how these outputs and outcomes can inform instructional and systems practices and operations.