COVID-19 Special Report

The COVID-19 Special Report attempts to provide essential context for understanding and interpreting 2021 NSCAS Results. By providing a brief overview of the history, response, and efforts to overcome the impacts of COVID-19 on Nebraska schools, the NDE hopes to provide important insights into the ongoing efforts of school renewal and acceleration. The NDE encourages anyone interested in reviewing 2021 NSCAS data to first read the context provided by the COVID-19 Special Report.

COVID-19 Special Report: Nebraska Student-Centered Assessment System (NSCAS) Results 2020-2021

Introduction

 

Nebraska administered assessments in the spring of 2021 as one of a number of strategies to better understand how students who tested are performing academically as part of our COVID-19 academic recovery efforts.

 

Information gleaned from Nebraska Student Centered Assessment System (NSCAS) helps identify areas where we most need to accelerate learning opportunities for our students. Similarly, participation data provides demographic information about who did and did not take the assessment in light of COVID-19 learning conditions and should be considered in context when reviewing the results.

 

Direct comparisons of assessment data with previous years is not appropriate for many reasons:

  • Changing nature of NSCAS Summative (movement to through-year adaptive)
    • Shortened assessment to preserve instructional time
    • Large changes in enrollment 
    • Large differences in participation rates
    • Increase of non-participants were not representative of the whole population
  • COVID-19 impact on 2019-20 and 2020-21 school years:
    • Decrease in opportunity to learn in the spring of 2020
    • Inability to accurately administer assessments to test students remotely
    • Uncertainty of impacts on opportunity to learn in 2020-21
    • Mixture of remote, in-person, and hybrid learning
    • Lack of standardization for educational opportunities across the state and across years
    • Impact of trauma on the ability of individual students to demonstrate what they know and can do

 

However, the data provides an important temperature check that allows the state to better track and address student recovery in the short and long term.

 

Note to parents: For parents looking at their child’s individual report, we encourage consideration of their child’s results within the context of the variety of potential learning disruptions they may have encountered as well as the modifications that were made to the assessment itself.

 

The COVID-19 Special Report attempts to provide essential context for understanding and interpreting 2021 NSCAS Results. By providing a brief overview of the history, response, and efforts to overcome the impacts of COVID-19 on Nebraska schools, the NDE hopes to provide important insights into the ongoing efforts of school renewal and acceleration.

 

There are many factors at play when considering this year’s results.  Even though our state’s participation rates are lower than in past years, the assessment results present an opportunity (in combination with other data) to inform the type of supports students, educators, and schools need to be successful. We owe it to students to consider every piece of information we have to help them succeed.

COVID-19 Pandemic Spring 2020

 

On March 13, 2020, Governor Pete RIcketts provided guidance for school closures in a press conference with NDE Commissioner, Matthew Blomstedt. By April 1, 2020, all Nebraska schools were directed to operate without students in the building through May 31, 2020, effectively ending in person learning for the 2019-2020 school year. Districts across the state used a variety of strategies to continue to engage students in the spring of 2020. Some districts with enough devices were able to offer instruction to students remotely. Other districts provided learning packets for students to work on while at home. Extracurricular activities were canceled for the spring. Efforts were made to compensate for the interrupted learning of COVID-19 Pandemic.

 

Governor Pete Ricketts signed Executive Order No. 20-20 on April 20, 2020 waiving certain accountability and assessment requirements.

 

On March 25, the state of Nebraska received a waiver from the U.S. Department of Education for 2019-2020 assessments, accountability, school identification, and reporting requirements as originally planned, due to extensive school closures. On March 18, the NDE suspended all statewide assessments (NSCAS) for the 2019-2020 school year. This included NSCAS – General, Alternate, and ACT.

 

The NDE provided opportunities in the fall of 2020 for high school seniors that missed the chance to take the ACT in spring of their junior years due to the cancellation of testing during the COVID-19 Pandemic. The makeup opportunity was provided by NDE at no cost to the families.

Launch Nebraska

 

Over the summer, the NDE coordinated efforts to provide guidance for districts on how to return to school for the 2020-2021 school year. Those efforts were a part of the Launch Nebraska initiative and included Planning a Safe Return to School in Nebraska and Return to Schools Roadmap among other resources and professional learning opportunities.

 

Return to School 2020-2021

Teacher of the year with class all wearing masks

2022 Nebraska Teacher of the Year Lee Perez and his class at Alice Buffett Magnet Middle School.

Unlike many other states, in Nebraska the vast majority of students in most schools learned in-person for the majority of the school year. While many schools offered students virtual options, 100% of Nebraska public school districts also offered in-person learning options for their students during the 2020-21 school year. 

 

This was in large part due to the monumental efforts of school leaders, teachers, communities, and students rallying to ensure safe return to school and continuity of learning. 

 

Local districts worked with local health departments to contact trace known exposures, and exposed students were quarantined at home to limit additional exposure. Many districts offered remote learning in addition to in-person learning. Local districts made decisions based on the circumstances of the pandemic and available resources. Oftentimes, the strategies changed or adapted based on ever changing circumstances. The local situation had implications for students’ opportunities to learn. Assessment results should be understood with the local context in mind about how the pandemic impacted individuals and groups of students’ opportunity to learn. 

 

Still, all students were likely impacted by the pandemic whether learning in person or remotely. It is easy to understand some of the large-scale impacts on learning such as the remote versus in person learning, but schools and students were impacted in subtle ways as well. These impacts varied by district, by school, and by student, but all students were impacted. The mitigation strategies were necessary because they helped to protect students and teachers and because they made in-person learning possible. The list below is not exhaustive nor is it meant to demonize the efforts of schools and health departments to keep the public at large as safe as possible. The list is intended to help everyone understand the challenges that students faced in learning during the pandemic and help us better understand why simple or direct comparisons of data from 2021 to previous years is problematic.

 

  • Lost instructional time due to mitigation strategies
    • Additional hand washing
    • Wiping down furniture between groups
    • Mask breaks
    • Social distancing in the hallways that caused transitions between classes to be extended
    • Social distancing at the beginning and end of day which caused classes to start later or end earlier for some students
  • Inability to socialize with friends outside of an established grade/class grouping
  • Lower levels of autonomy during typically less restrictive activities such as who to sit with at lunch or what to play during recess
  • Limitations of other interactions with adults in order to limit potential exposures
  • Impacted schedules and learning opportunities due to lack of substitute teachers
  • Sense of loss over canceled or modified traditions
  • Shortened courses such as art or music
  • Canceled extracurricular activities/clubs
  • Canceled performances or competitions
  • Fear of COVID19 exposure for themselves or loved ones
  • Stress from the rigid guidelines to minimize potential exposure
  • Uncertainty caused by the ever changing circumstances of the pandemic
  • Fatigue from the constant and pervasive impacts of the pandemic
  • Isolation due to canceled opportunities beyond school (camps, clubs, sleep overs, parties, etc.)
  • Impacts of family quarantine on jobs and interpersonal relationships

 

Students react differently to these types of stressors and it is difficult to say how circumstances may have impacted individual students. There is ample evidence to say that there were impacts and they were widespread even when communities had low levels of perceived impacts from the pandemic. Nationally, research by NWEA and by ACT both indicate that student learning was impacted. NDE’s own research using MAP Growth data from fall of 2020 indicated that students continued to learn but at slower rates than what was typically expected.

 

The NDE Reports: MAP Growth Test Scores Across Years

Enrollment in 2020-2021

The ongoing pandemic impacted how students learned in 2020-2021, but there were also shifts in who was responsible for teaching. Applications to home school students surged over the summer and resulted in nearly a 70% increase in the number of Nebraska students enrolled in home-school. An estimated 14,780 students were homeschooled in 2020-2021, which was up from an estimated 9,450 students in 2019-2020. Students that are enrolled in homeschool programs do not take statewide assessments. This number has leveled off to near pre-pandemic numbers in preliminary enrollment data from the 2021-22 school year. The shift in enrollment is another reason that directly comparing 2021 assessment data is problematic.

 

The NDE Report: 2021 Enrollment Analyses Across Years

NSCAS Assessments

Standardized Summative assessments work best when conducted in stable environments. The purpose of the assessments is to measure change in achievement at the school and district level. When the environment is stable the assessments are meant to detect programmatic improvements in both curriculum and/or instruction.

  • Interpreting data from large scale standardized assessments is always difficult because of the number of variables in an education system. Even in typical circumstances knowing the exact reason for score increases or decreases is problematic.
  • Analyzing assessment data should always be done with as much context as possible, with multiple sources of data, and with deliberate thoughtfulness. Due to the number of variables, educators rely on trend data to track how programs change over time.
  • The COVID-19 Pandemic introduced many additional variables that may impact achievement both within the school system but also across communities.
  • The additional variables make comparing 2021 data to previous years particularly problematic due to the wildly different circumstances. Any analysis which includes direct comparisons to previous years, which is not recommended, would need multiple sources of data and full accounting of environmental context to have even marginal usefulness.

Opportunity to Learn

Standardized Summative assessments also assume that students have an opportunity to learn (OTL) the content being assessed. Districts are responsible for providing all students with an opportunity to learn the assessed content. Ideally, educators would provide an OTL and then employ formative assessment processes to check for understanding. Educators would then use the results of the formative assessment processes to determine if students needed additional opportunities to learn specific content.

  • The COVID19 Pandemic impacted all students’ opportunity to learn content at the end of the 2019-2020 school year when districts stopped meeting in person.
  • The COVID19 Pandemic likely impacted most students’ opportunity to learn some assessed content during the 2020-2021 school year.
  • Direct comparisons of the 2020-21 assessment data to previous years are not recommended since summative assessments assume students have opportunity to learn while opportunity to learn in 2020-2021 was considerably different from pre-pandemic years.

2020-2021 Statewide Assessment Plan

The Statewide Assessment Plan for 2020-2021 was designed for several purposes.

  •  Allow Nebraska to continue to transition to a new assessment design to better inform instruction.
  • Maximize the amount of instruction time was available for learning by shortening the assessments.
  • Provide data to help educators and the public better understand the impacts of the COVID-19 Pandemic on teaching and learning.

 

 The NDE announced the transition to a new innovative through-year assessment model in October of 2019. The COVID-19 Pandemic interrupted the transition when statewide assessments were canceled in spring of 2020.

  • The NDE decided to continue to prioritize the transition to the new innovative model instead focusing on the existing model when planning for assessments in 2020-2021.
  • The original plan included a pilot assessment to gather important data for the transition and did not include public reporting of the results. Results would have been available for districts to better understand impacts on learning of the COVID-19 Pandemic.
  • The US Department of Education insisted that public reporting occur despite the transitional pilot and despite the likely limitations of results.
  • Statewide Assessment plans were adjusted to allow for public reporting and to accomplish the originally stated goals. One such adjustment included the creation of this COVID-19 Special Report: Nebraska Student-Centered Assessment System (NSCAS) Results 2020-2021 to provide important context to help the public better understand the results.

 

The NSCAS ELA and Mathematics assessments were shortened to preserve as much instructional time as possible.

  • Shorter assessments are typically less statistically reliable and less precise than the longer assessments used in the past.
  • The shorter 2021 assessments were classified as highly reliable even though they were slightly less reliable and slightly less precise than early versions of the assessments.
  • The 2021 assessments were based on the same blueprints as earlier versions but had fewer total items. The proportion of items were maintained for each of the standard areas included in the assessment
    • English Language Arts include Vocabulary, Reading Comprehension, and Writing Skills
    • Mathematics include Number, Algebra, Geometry, and Data/Probability

 

As the 2020-2021 school year progressed, it became apparent that some students would not be assessed on the spring summative assessment due to the COVID-19 Pandemic. The possibility of assessing students remotely was vetted for feasibility but ultimately rejected for multiple reasons including, but not limited to, concerns about test security, student privacy, and logistical complexities.

  • The NDE in consultation developed a COVID Waiver to track which students did not test due to ongoing and continued concern about exposure to COIVD-19.
  • Guidelines attempted to ease the burden on districts to track the waiver while attempting to accurately determine the extent of students not tested due to COVID-19 as opposed to typical reasons such as Emergency Medical Waiver or Out of State Placement.
  • Statewide 8,553 unique students did not take required statewide assessments and were designated as COVID Wavier as the reason. These numbers represent between 5.64% and 7% of students statewide, depending on the specific NSCAS assessment as well as grade and content levels.
  • The large number of students not tested due to COVID-19 was responsible for a significant surge in the number and percentage of students that did not participate in NSCAS assessments.
  • The distributions of students not tested utilizing the COVID Waiver was not evenly distributed across the state. Larger districts accounted for the majority of the COVID Waivers. Larger numbers of COVID Waivers in larger districts were to be expected because larger districts were more likely to have remote options for learning whereas many smaller districts were completely in-person only.
  • COVID Waivers accounted for a large increase in the percentage of students that did not count as participants.

Participation

Federal education accountability guidelines require that at least 95% of students participate in statewide summative assessments. Schools and districts that have lower participation rates are penalized in federal accountability initiatives. Pre-pandemic Nebraska districts typically had high levels of participation and almost never approached the 95% threshold.

  •  Statewide pre-pandemic participation in grades 3-8 was almost always 99.5+%
  • Statewide pre-pandemic participation for high school NSCAS ACT was typically 98% or higher.
  • Statewide participation for NSCAS grades 3-8 ELA was 94.36%
  • Statewide participation for NSCAS grades 3-8 Mathematics was 94.28%
  • Statewide participation for high school NSCAS ACT was 94% for ELA and 93% for Mathematics and Science
  • Overall, Nebraska had high levels of participation in 2021 when compared nationally, but the change in percentage represents thousands of students.
  • Slight random fluctuations in participation have little impact on state level data. Adding or subtracting a few scores will not change averages or the percent of students that are determined to be proficient for the state scores. The smaller the district, the larger the impacts of having missing data.
  • The students that did not test and had COVID Waivers do not represent a random sample of students. The families of those students made a conscious decision to limit the risk of exposure to their child and by extension other members of their family. Families that made these decisions did not necessarily match the overall demographics of a school, district, or state. Since the students were not random, the impacts on data can be noteworthy.
  • Who tests and who does not test can have important impacts on averages. Large differences in participation make direct comparison of the 2021 NSCAS data to previous years problematic. Understanding who tested and who did not can have an impact on how a single year’s results are interpreted and make it difficult to understand changes when there are large shifts in participation.
  • Changes in participation can make interpreting data difficult at the state level. These difficulties are amplified when those changes are at the district or school level.

State Results

  • Measurement experts strongly discourage direct comparison of the 2021 assessment results to previous years. Direct comparisons that fail to be informed by context of the COVID-19 Pandemic are likely to be misinterpreted and misused.
  • Measurement experts strongly discourage comparing districts and schools using 2021 assessment results. Circumstances during the pandemic varied greatly between schools and districts.
  • Typical comparisons are problematic and are likely to be misconstrued since circumstances for districts and schools were far from standardized.
  • Any analyses or interpretation of 2021 assessment results should only be conducted with a full understanding of the context of the school and district. Such analyses or interpretation should be conducted with multiple sources of data. Even with a full understanding of the context and multiple sources of data, interpretations and decisions should be made with extreme caution and should be verified with new data when available.
  • There is strong evidence that Nebraska students continued to learn despite interruptions from the COVID-19 Pandemic, due to the extraordinary efforts of students, parents, educators, and communities across the state. While students did continue to learn, it is likely that most students did not grow as much as would have been expected during a normal school year.
  • Educators across the state continue to leverage resources to compensate for unfinished learning caused by the COVID-19 Pandemic. Efforts are underway to better understand the unfinished learning. Resources are being utilized to renew and accelerate learning for all students.
  • NSCAS Assessment results are a one time snapshot of student achievement. Districts have multiple sources of data for students. The combination of data and trends are more important than any single assessment result.

 

Nebraska Student-Centered Assessment System (NSCAS) Summative Grades 3-8 Statewide Data for 2021

        Percent Proficient      Participation
      English Language Arts         48%      94% (7,922 Non-participants)
     Mathematics         46%      94.28% (8,032 Non-participants)
     Science*         NA      NA

*NSCAS Science was a field test for 2021 for which there is no reportable data.  A field test is used to ensure test items are ready for an operational assessment that will be given in spring of 2022.

 

Nebraska Student-Centered Assessment System (NSCAS) ACT High School Statewide Data for 2021

     Percent Meeting Expectation      Participation
     English Language Arts      46%      94% (1,516 Non-participants)
     Mathematics      44%      93% (1,530 Non-participants)
     Science      50%      93% (1,559 Non-participants)

Click here for Additional State Data and Access to District and School results.

Renewal and Acceleration and ARP Investments

The NDE adapted Launch Nebraska in February 2021 to refocus on Renewal and Acceleration, outlining the key shifts, components, and core actions for 2021 and beyond. Read more about Launch Nebraska 21-22.

 

Additionally, as part of the historic federal investment in education, the American Rescue Plan provided $546 million to schools in the state, with 90% flowing directly through to districts. The NDE reserved 10% of the funds for statewide activities. You can read more about ARP ESSER and the NDE’s statewide investments.