Executive Summary
The Middlebury Interactive Language Program (MIL) was launched in a pilot in the Hartford Public Schools (Connecticut) in 2013-2014 with a goal of building English language proficiency in an interactive project based learning experience. The results have been positive from the pilot year through the next two years of program expansion. In the most recent full year of program implementation in 2015-2016 in 32 schools across the Hartford district. Students who were continuously improved demonstrated growth in English Language Proficiency and many achieved the criteria for existing the program.
The Purpose of the Report
The purpose of this report is to provide background on the efficacy of the Middlebury Interactive Language Program (MIL) since its pilot in 2013-2014 through the 2015-2016 school year implementation. The initial research was conducted by The Johns Hopkins University. Subsequent research has been conducted by K12 Inc. This report includes background on the program and the important role that the Hartford Public Schools (HPS) has played in refining MIL to best meet the needs of school-age English Language Learners (ELL).
The Role of the Hartford Public Schools
Hartford Public Schools in Connecticut is a unique school choice district. Families and students can choose from a range of school types including charter, magnet, district, and open choice schools in neighboring townships. Eighteen percent (3,700) of students are English language learners. These students represent over 70 different native languages. The commitment of Hartford Public Schools is that the English language (EL) acquisition students develop proficiency in the English Language while achieving proficiency on state content-area standards as they continue to value their own native language and culture.
Hartford Public Schools is committed to providing rigorous instruction for EL acquisition. For this reason, this district sought to co-develop a blended learning program with Middlebury Interactive Languages, aimed at supporting EL students primarily in grades 5-10. The program, which began with a pilot at eight school sites in 2013-2014, is currently in its fourth year[1] of implementation, now serving students in grades 3-12 at 29 school sites.
The overarching goal of the Hartford Public Schools/Middlebury Interactive Partnership EL Program is to provide an option for non-English native speakers to be college and career ready by supporting them in their academic language development. Professional development, and frequent opportunities for district and MIL collaboration, are also important elements contributing to the success of the partnership.
Research Approach
Research on the successful acquisition of English among elementary and secondary school students demonstrates that this is not an easy nor “one size fits all” process.[2] Students eligible for English Language Learning programs often represent a wide range of native languages, cultural backgrounds, and family characteristics. Further, many of these students experience family and community interaction that reinforces fluency in the home language, with school being the primary location for experience in formal English instruction.
The Middlebury Interactive (MIL) Program for English Language Learners was specifically developed to provide opportunities for students to engage in age and grade level relevant academic language development via an engaging project-based blended learning model. The project-based learning experiences ensure that students learn and practice English in an environment that includes experience in building peer communication, collaboration and engagement with the school as well as the extended community. Class periods vary but are typically 45-50 minutes long and students engage with the Middlebury curriculum three to five times each week. The learning environments promote student collaboration with the teacher being a mentor and guide for the project-based program.
English is the dominant language used in instruction and in the classroom, although students often have access to online translators or tutors in the classroom, especially those at the entering level. The goal of this approach is to accelerate English academic language development and support students in building confidence in their ability to participate in regular or sheltered core content classrooms.
Figure 1: Prior years’ English proficiency levels for students currently enrolled in 2016-17 implementation schools
Data Description
Students were selected for this dataset if they are currently enrolled in 2016-2017 in a school site that uses the Middlebury program. Student data are grouped by the students enrolled in 2016-17. Student data are split into school groups defined by initial implementation of the Middlebury program: (1) in the 2013-2014 pilot year, (2) in the 2014-15 school year, or (3) in the 2015-16 school year. In this report, students who are currently enrolled in a pilot school will be referred to as “pilot students,” students who are currently enrolled in a school that began programming in 2015 will be referred to as “MIL 2015 students” and students who currently attend schools that began programming in 2016 will be referred to as “MIL 2016 students.”
All data are connected to the school through the students who are currently enrolled in the 2016-17 schoolyear. This means that the Las Links (LAS) English proficiency levels for prior years are reported by the 2016-17 school the students are currently attending. There is no prior year enrollment information to allow the connection of students to schools other than attended in the 2016-2017 school year.
Data Collection: 2014-16
2014:15: Pilot Study Conducted by The Johns Hopkins University Center for Research and Reform in Education (CRRE)
During the pilot phase of the HPS/MIL EL program, The Johns Hopkins University Center for Research and Reform in Education (CRRE) studied the effectiveness of this program as piloted in eight Hartford Public Schools, Connecticut. Using the individual student scores from the Las Link interim Benchmarks Assessments, CRRE reported that students increased their English Language fluency as measured by reading comprehension, increased at a steady rate across the school year, and eventually exceeded the Reading Achievement levels of comparable ELL students not in the MIL program.[3] This means that students grew their scale score performance across five proficiency levels.
In addition to these achievement results, surveys summarized indicated that teachers, MIL teacher coaches, principals and parents perceived the MIL program to be both engaging and supportive of their students’ English language learning needs. However, because the focus of this research was within year improvement in English language proficiency, no data were available to measure English language proficiency and growth year over year on the same measure.
Year over Year Growth Research Study
Following the positive experience in the pilot year, Hartford Public Schools expanded the pilot to include additional grades 4-8 site implementations, plus high school newcomer programs in the 2015-2016 school year. Hartford Public Schools also agreed to use the mid-year summative Las Links assessments required by the State as the measure of proficiency and growth. This assessment is a standardized research-based assessment administered throughout the state of Connecticut. These data were available for three school years: 2013-2014, 2014-2015 and 2015-2016.
Using the summative Las Links data, it is possible to examine the cumulative effect of the MIL program year over year in building English proficiency in non-native speakers. This descriptive research is not intended to establish a causal relationship between the structure of the program and the outcomes. However, it was possible to follow students over a three-year exposure to the MIL program relative to proficiency in English.
Because there are many variables that may have impacted student learning outside the program, we cannot assert causality. However, the exploratory data analyses do highlight some valuable findings.
District-wide Analyses
Grade Level Concentration of MIL program participants
Building on the CRRE study conducted during the initial pilot year of the MIL Program, it is important to explore growth year over year of students’ English language proficiency. At the end of the 2015-2016 school year, a total of 287 students had Las Links scores from each of the three years they participated in the program. These data were used to address the analysis of students’ improvement in English language proficiency year over year.
Middlebury Results for Students with Three Years of Data
In the third year of implementation in the Hartford Public Schools, students in 32 schools were enrolled in the MIL Program. Across all three years, more students in Elementary grades participated than in either Middle School or High School. Figures 2A and 2B show the growth in proficiency year over year of all enrolled students.
Figure 2A: Overall number of students with three years of data and their change in English proficiency levels from the 2013-14 SY to the 2014-15 school year
Figure 2B: Overall number of students with three years of data and their change in English proficiency levels from 2014-15 to 2015-16 school year
When looking at student growth in aggregate, 287 students were enrolled in schools implementing the Middlebury program for all three years (2013-2014, 2014-2015, and 2015-2016). While the median English proficiency level stayed the same at 2 for all three years, the mean[4] English proficiency level increased year over year, from 1.7 in 2013-2014 to 2.24 in 2014-2015 to 2.40 in 2015-2016. Note that these are the students currently enrolled in a pilot school and who had three years of data.
Middlebury Results for Students with Two Years of Data
Similar to those students with three years of data, the 470 students who had two years of data (2014-2015 and 2015-2016), also had a consistent median English proficiency score of 2 for both years. Yet there was an increase in the English proficiency mean from 1.94 in 2015 to 2.22 in 2016. This supports the conclusions that positive change/growth is occurring year over year. Students who were in the program for two years, 2014-2015 and 2015-2016, showed a decrease in the number of students who were in the lowest level (1) of English proficiency and an increase in higher English proficiency categories (levels 3 and 4). (See Figure 3)
Figure 3: Overall number of students with two years of data and their change in English proficiency level from 2014-15 to 2015-16
Like students with three years of data, students with two years of data had forty-three percent of students increase their English proficiency by one or two levels from the 2014-2015 school year to the 2015-2016 school year.
Are students’ English proficiency levels increasing over time in the program?
This research shows that students enrolled in the MIL program improve their English proficiency as measured by Las Links. For students who have been continuously enrolled since the MIL Hartford pilot in 2013-2014, Figure 4 shows the growth in aggregate student performance over three years. Note the majority of students enrolled in MIL for three years increase from 21 students at or above Level 3 on Las Links to 56 students in 2015-2016. The same results were found for students continuously enrolled for two years (see Figure 5). Note the drop in Level 1 72 percent of students in the 2014-2015 school year to 18 percent of students in the 2015-2016 school year. Also the increase in Level 3 and above from 31 percent of students in the 2014-2015 school year to 43 percent of students in the 2015-2016 school year.
Figure 4: Overall growth of students enrolled in the pilot schools’ MIL for three consecutive years
Figure 5: Overall growth of students enrolled in the MIL 2015 schools for two years
For students participating in MIL in non-pilot schools, the growth in proficiency is also evident. In 2014-2015; only 6 students were at level 3 and none were above levels 4 or 5. With one year of instruction, 19 students were at level 3 or higher.
How Long Does It Take for Students to Exit the Program?
Figure 6: Change in English proficiency levels over three years for pilot school students with level 1 English proficiency levels in 2013-14
For students in the pilot schools, the growth was notable. From 2013-2014 when no students were proficient as measured by the Las Links, in each subsequence year more and more students moved up the proficiency scale. This means that 29 percent of students met the exit criteria within three years of ELL instruction in MIL.
Figure 7: Change in English proficiency levels over three years for MIL 2015 students with level 1 English proficiency levels in 2013-14
Does MIL Impact Student ELL equally effectively for different grade level groups?
We have seen that students participating in the MIL program have increased their English proficiency levels year over year. This third research question examines if these increases in English proficiency levels year over year occur for students in grade 5-8 and for students in grades 9-12 (high school).
Figure 8: Change in English proficiency over two years for MIL 2015 students in the grades 5-8 cohort
Figure 9: Change in English proficiency over two years for MIL 2015 students in the grades 9-12 cohort
Educators often ask whether an ELL program is equally effective for older and younger students. What we see in the MIL program is that of the 24 students below the proficiency cut score in 2014-2015 in grades 5-8, 65 percent moved up a level in one year. For high school students, 22 percent moved up a level. While these findings are based on relatively small sample sizes, we do have some evidence that older students are more difficult to impact in one year than are younger students. However, there is growth among the high school students in their English language proficiency.
Case Study: Bulkeley High school
Bulkeley High schools is an interesting case because it was an early adopter. In 2013-2014 this high school had no students achieving proficiency on the LAS. In each subsequent year that school had both a drop in the numbers of students in Level 1 but also growth to students level the first proficiency level (3) in year two and to level 4 in year three of MIL implementation, And, as reported in Figure 11, only one students slipped back a level.
Figure 10: Change in English proficiency levels over three years for students currently enrolled at Bulkeley High School
Figure 11: Overall number of students currently enrolled at Bulkeley High School and their change in English proficiency level from 2013-14 to 2014-15
Conclusion
The Middlebury Interactive Language (MIL) Program was successfully implemented in the Hartford Public Schools over the past three years. Beginning with the 8 pilot schools and then growing district wide, the teachers have been effective in growing the proficiency of students enrolled. And, students tended to remain enrolled over multiple years, thereby improving their likelihood of becoming proficient in English. While the numbers of students enrolled are relatively low and the available data are limited to proficiency category, not scale scores, there is evidence of positive impact of MIL on student English language acquisition.
This report describes two different research studies. The first early study by CRRC found growth within the year in reading achievement scores for students enrolled in the MIL Program using the three benchmark interim Las Links assessments. The larger study conducted over two years of data (from 2014-2015 to 2015-2016) shows that students continuously enrolled in the MIL Program progressed each year they were enrolled in the program. The increase in overall mean score year over year indicates continuous achievement growth for the cohort of students participating in the program, with the largest growth in achievement occurring in the first year for both those participating across three years and those participating across two years.
As noted earlier, an important element of the MIL/HPS partnership is the ongoing coaching, professional development and collaboration among the MIL product developers and the HPS EL team. HPS coaches report that teachers, especially those with one or two years’ experience in using the program, are employing differentiated instructional strategies to meet student needs, and requesting professional learning communities to continue their development of the program. Further, while the program was intended for students in 4th through 10th grades, teachers and administrative staff in multiple sites have requested that the program include 3rd grade students as well as upper level high school students.
As of the fall of 2016-17, the program expansion of the MIL/HPS partnership includes all elementary, middle and high schools demonstrating sufficient enrollment of EL students for the blended learning program implementation in one or more sections at the school site. Data collection priorities for the 2016-17 school year include implementation of the newcomer online progress assessments for each entering level theme, continued monitoring of student progress using the annual LAS Links assessment, and qualitative data collection via teacher and coach surveys.
Recommendations
Use of English assessment scale scores instead of or along with English proficiency levels to examine change/growth between proficiency levels
Comparison group data to show that students are increasing English proficiency levels by participating in the MIL program
Confirm the LAS Links criteria for exiting the program is Level 3 and if exiting criteria is a combination of requirements then include all in the dataset
[1] The 13-14 SY was considered a pilot year
[2] A User Study: Technology to Increase Teachers’ Linguistic Awareness to Improve Instructional Language Support for English Language Learners
[3] Reading is one of four sub scores reported on each of the three Benchmarks. The others are Speaking, Listening and Writing. Brochure-Las Links, CTB, 2015.
[4] The rough means presented here are a directional proxy for the scale scores. The English proficiency level is a categorical outcome, making the median the statistically correct measure of central tendency instead of the mean/average. Yet the use of the means was pertinent to both visually examining the increase in English proficiency levels over time as well as opening the discussion for the need for scale scores.