Barbour, M. K. (2021, February). What does the research tell us? How should research shape policy? [Online Presentation]. League of Women Voters Lansing Area.
League of Women Voters Lansing Area (2021) - What Does the Research Tell Us? How Should Research Shape Policy?
1. What Does the Research Tell
Us? How Should Research
Shape Policy?
Michael K. Barbour
Touro University California
2.
3.
4.
5.
6. What Does the Research Tell
Us? How Should Research
Shape Policy?
Michael K. Barbour
Touro University California
H istory of K- 12 Distance Education
History of K-12 Distance Learning
10. State of K-12 Online Learning
• exists in all 50 states in some form or another
• at least 2 million K-12 student engage in
supplemental online learning
• approximately 750,000 K-12 students
engaged in full-time online learning
14. “Unfortunately, there is almost a complete
absence of research into the nature or quality of
curriculum and student experience, nor is there
research examining the unstated assumptions
about the type of learning provided by the virtual
education experience. What we do know about
the daily life of a student attending a virtual school
is both dated and often based on material
provided by either the corporate educational
management organizations themselves or
secondhand reviews.” (pp. 52-53)
The Virtual School Experience
15. The Virtual School Experience
• the education process begins when the company
“provides each eligible student with a textbook and
instructional materials, computer, printer, and
reimbursement for Internet connection.” (Klein, 2005)
• “In a recent court case in Missouri, a student was
assigned to a district-based virtual school, but was not
provided with a computer, Internet access, or
technology of any kind. The student in question was
simply told to travel by bus to the public library where
their time on the computer was restricted to two
hours per day.” LW v. Special Administrative Board of
the St. Louis Public School District. No. 1522-CC00605.
(2018 Missouri Div No. 20)
16. The Virtual School Experience
“students, under the direction of their parents, study the materials
and complete various assignments to demonstrate their
understanding. The parents are provided with instructor’s materials to
assist the student’s learning. The parents check the students’ work on
their assignments to determine whether the students have mastered
the topic. A parent is required to devote four to five hours per day to
the student’s education…. certified teachers… review samples of
students’ work to assess progress, and hold one to two 20-30 minute
telephone conferences per month with each student and parent,
during which they discuss and assess student progress. They
correspond with students via email, and respond to parental requests
for assistance via email and telephone. Certified teachers also conduct
30-40 minute interactive online classes using online conferencing
software; students participate in such classes two to four times per
month.” Johnson v. Burmaster, 2006AP1380 (2008 WI APP 4)
17. The Virtual School Experience
• “a public school, using public funds, that
relied upon unlicensed individuals as the
primary teachers of the pupils” Johnson v.
Burmaster, 2006AP1380 (2008 WI APP 4)
• “although enrolled in ‘virtual charter schools,’
most children receiving the K12 curriculum
are homeschooled” (Bracey, 2004)
19. Tell me about the research…
• “based upon the personal experiences of
those involved in the practice of virtual
schooling” (Cavanaugh et al., 2009)
• “a paucity of research exists when examining
high school students enrolled in virtual
schools, and the research base is smaller still
when the population of students is further
narrowed to the elementary grades” (Rice,
2006)
20. The Research
1
3 3 4 3
7 6 6
9
11
5
14
12
16
22
33
25
20
23
31
40
27
35
1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016
Number
of
Articles
Year
Articles per Year
21. The Research?
Top
journal
published
7% of the
total
articles.
132
journals
published
four or
fewer
articles.
102
journals
published
one
article.
0 5 10 15 20 25 30
Journal of Online Learning Research
American Journal of Distance Education
International Journal of E-Learning & Distance…
Journal of Open Flexible and Distance Learning*
Journal of Technology and Teacher Education
TechTrends
The Morning Watch
Distance Learning
International Review of Research in Open and…
Quarterly Review of Distance Education
NUMBER OF ARTICLES
JOURNALS Top 10 Journals
22. • Author Analysis—384 distinct authors; ranked by
number of articles and position of authorship.
• Top 11 authors: Michael Barbour (57), Cathy
Cavanaugh (19), Ken Stevens (18) Elizabeth Murphy
(16), Charles Graham (15), Margaret Roblyer (14), Jered
Borup (14), Leanna Archambault (12), Diana Greer (11),
Dennis Beck (10), Niki Davis (10)
• Of note: 276 authors (just under 75% of the authors)
published only one article; more than half of these
articles were published from 2011 though 2016,
perhaps indicating a growth in interest in K-12 online
learning and newer scholars.
The Research?
23. “indicative of the foundational descriptive work
that often precedes experimentation in any
scientific field. In other words, it is important to
know how students in virtual school engage in
their learning in this environment prior to
conducting any rigorous examination of virtual
schooling.” (Cavanaugh et al., 2009)
The Research?
24. What Does That Research Say?
1. Comparisons of student performance based upon
delivery model (i.e., classroom vs. online)
2. Studies examining the qualities and characteristics of
the teaching/learning experience
• characteristics of
• supports provided to
• issues related to isolation of online learners
(Rice, 2006)
1 Effectiveness of virtual schooling
2 Student readiness and retention issues (Cavanaugh
et al., 2009)
25. Student Performance
Report Finding
2013 While 52% of brick-and-mortar district and charter schools met AYP, only 23.6% of virtual schools did
the same.
2014 “Virtual schools’ Adequate Yearly Progress results were 22 percentage points lower than those of
brick-and-mortar schools… The on-time graduation rates for full-time virtual schools was close to
half the national average: 43.8% and 78.6%, respectively.”
2015 “Full-time virtual schools continued to lag significantly behind traditional brick-and-mortar schools…
The on-time graduation rate (or four-year graduation rate) for full-time virtual schools was nearly
half the national average: 43.0% and 78.6%, respectively.”
2016 “Of the 121 virtual schools for which data were available, 22 (18.2%) had proficiency rates above the
state average; 82 percent had proficiency rates below state averages… The on-time graduation rate
(or four-year graduation rate) for full-time virtual schools was half the national average: 40.6% for
virtual schools and 81.0% for the nation as a whole.”
2017 “[Only] 37.4 percent of full-time virtual schools received acceptable performance ratings… The
graduation rate of 43.4% in virtual schools [compared to a national average of 82.3%].”
2018 “Virtual schools continued to underperform academically, …36.4% of full-time virtual schools
received acceptable performance ratings. The graduation rate of 50.7% in virtual schools… fell far
short of the national average of 83%.”
2019 “Overall, 48.5% of full-time virtual schools were rated acceptable performance ratings… the on-time
graduation rates for full-time virtual schools (50.1%)… fell far short of the national average of 84%.”
26. Student Performance
Study Finding
Ohio (2019) “Students attending online charter schools have substantially weaker growth in both reading and math than the
average TPS VCRs. The gaps translate to 47 fewer days of learning in reading and 136 fewer days of learning in
math for online charter students.”
North Carolina (2017) “For the 2015-16 school year, both VCS received an overall School Performance Grade (SPG) of D which
translates numerically to a 52 for Connections and 45 for NCVA respectively. Both VCS received a SPG of C in
Reading and an F SPG in Mathematics. Comparatively, during the 2015-16 school year, traditional public schools
had a lower percentage of schools with D and F (22.9%) than public charter school (27.7%).”
Ohio (2017) The students who started e-schools in the lower baseline academic distribution scored lower on state testing and
had lower likelihoods of meeting high school graduation standards. Students with prior levels of high
achievement also scored lower than their traditional public and charter school peers, but the difference was not
as stark as those with lower prior levels of academic achievement.
National (2016) “Compared to traditional public school students, full-time virtual charter school students have much weaker
academic growth overall. Full-time virtual charter schools perform worse than traditional public schools in most
states. All subgroups of students have weaker academic growth in full-time virtual charter schools than in
traditional public schools. The vast majority of full-time virtual charter schools perform worse than traditional
public schools.”
Ohio (2016) “Across all grades and subjects, students who attend e-schools perform worse on state tests than otherwise-
similar students who attend brick-and-mortar district schools, even accounting for prior achievement”
Tennessee (2016) “The scores are generally lower [for the full-time cyber schools] than the scores of the districts that established
the schools.”
National (2015) “Across all tested students in online charters, the typical academic gains for math are -0.25 standard deviations
(equivalent to 180 fewer days of learning) and -0.10 (equivalent to 72 fewer days) for reading.”
Georgia (2015) “In 2013–14, none of Georgia’s three statewide fully online schools: A) met all of the standardized assessment
goals included in their respective charter contracts; or B) outperformed the state on the CCRPI ‘achievement’
component.”
Kansas (2015) Online students (which included a combination of full-time and supplemental students) performed at similar
levels in reading before and after controlling for student demographics, but that online students performed at
lower levels in mathematics compared to their face-to-face counterparts.
Colorado (2014) “Online school performance on state assessments had been lower across all grade levels and content areas than
that of its brick and mortar counterparts.”
Ohio (2014) “… [virtual] schools experienced lower student performance than their traditional counterparts.”
27. Student Performance
Study Finding
Michigan
(2020)
Students enrolled in full-time virtual schools operated by local education authorities had a
pass rate of 51%, while students enrolled in full-time virtual schools operated by public
school academies had a pass rate of 49%; compared to the students’ non-virtual course
pass rate of 76%.
Michigan
(2019)
Students enrolled in full-time virtual schools operated by local education authorities had a
pass rate of 47%, while students enrolled in full-time virtual schools operated by public
school academies had a pass rate of 53%; compared to the students’ non-virtual course
pass rate of 79%.
Michigan
(2018)
Students enrolled in virtual schools had a pass rate of 49%, compared to the students’ non-
virtual course pass rate of 78%.
Michigan
(2017)
Students enrolled in virtual schools had a pass rate of 53%, compared to the students’ non-
virtual course pass rate of 78%.
Michigan
(2017)
Students enrolled in virtual schools had a pass rate of 53%, compared to the students’ non-
virtual course pass rate of 78%.
Michigan
(2016)
Students enrolled in virtual schools had a pass rate of 52%, compared to the students’ non-
virtual course pass rate of 87%.
Michigan
(2015)
“Cyber enrollments had a ‘Completed/Passed’ rate of 54%... whereas Non-Virtual Learners
had an 89% ‘Completed/Passed’ rate.”
28. “AYP is not a reliable measure of school
performance…. There is an emerging consensus
to scrap AYP and replace it with a better system
that measures academic progress and growth.
K12 has been measuring student academic
growth on behalf of its partner schools, and the
results are strong with academic gains above the
national average.”
Jeff Kwitowski - K12, Inc. Vice President
of Public Affairs
34. “Media are mere vehicles that
deliver instruction but do not
influence student achievement
any more than the truck that
delivers our groceries cause
changes in our nutrition.” (Clark,
1983)
38. State of K-12 Online Learning Research
1. Possesses a confusing terminology
2. Has a lack of historical perspective
3. Exists is an absence of construct
validity
4. Suffers from a tension due to a
postmodern turn in research
5. Is American-centric
41. Michigan Virtual Learning Research Institute
1. Test, evaluate, and recommend as appropriate new technology-based instructional tools and resources;
2. Research, design, and recommend digital education delivery models for use by pupils and teachers that include age-appropriate multimedia instructional content;
3. Research, develop, and recommend annually to the department criteria by which cyber schools and virtual course providers should be monitored and evaluated to ensure a quality education
for their pupils.
4. Based on pupil completion and performance data reported to the department or the center for educational performance and information from cyber schools and other virtual course
providers operating in this state, analyze the effectiveness of virtual learning delivery models in preparing pupils to be college- and career-ready and publish a report that highlights
enrollment totals, completion rates, and the overall impact on pupils. The report shall be submitted to the house and senate appropriations subcommittees on state school aid, the state
budget director, the house and senate fiscal agencies, the department, districts, and intermediate districts not later than March 31 of each year.
5. Provide an extensive professional development program to at least 30,000 educational personnel, including teachers, school administrators, and school board members, that focuses on the
effective integration of virtual learning into curricula and instruction. The Michigan Virtual Learning Research Institute is encouraged to work with the MiSTEM advisory council created under
section 99s to coordinate professional development of teachers in applicable fields. In addition, the Michigan Virtual Learning Research Institute and external stakeholders are encouraged to
coordinate with the department for professional development in this state. Not later than December 1 of each year, the Michigan Virtual Learning Research Institute shall submit a report to
the house and senate appropriations subcommittees on state school aid, the state budget director, the house and senate fiscal agencies, and the department on the number of teachers,
school administrators, and school board members who have received professional development services from the Michigan Virtual University. The report shall also identify barriers and other
opportunities to encourage the adoption of virtual learning in the public education system.
6. Identify and share best practices for planning, implementing, and evaluating virtual and blended education delivery models with intermediate districts, districts, and public school academies
to accelerate the adoption of innovative education delivery models statewide.
7. Develop and report policy recommendations to the governor and the legislature that accelerate the expansion of effective virtual learning in this state’s schools.
8. Provide a clearinghouse for research reports, academic studies, evaluations, and other information related to virtual learning.
9. Promote and distribute the most current instructional design standards and guidelines for virtual teaching.
10. In collaboration with the department and interested colleges and universities in this state, support implementation and improvements related to effective virtual learning instruction.
11. Pursue public/private partnerships that include districts to study and implement competency-based technology-rich virtual learning models.
12. Create a statewide network of school-based mentors serving as liaisons between pupils, virtual instructors, parents, and school staff, as provided by the department or the center, and
provide mentors with research-based training and technical assistance designed to help more pupils be successful virtual learners.
13. Convene focus groups and conduct annual surveys of teachers, administrators, pupils, parents, and others to identify barriers and opportunities related to virtual learning.
14. Produce an annual consumer awareness report for schools and parents about effective virtual education providers and education delivery models, performance data, cost structures, and
research trends.
15. Provide an internet-based platform that educators can use to create student centric learning tools and resources for sharing in the state’s open educational resource repository and facilitate
a user network that assists educators in using the content creation platform and state repository for open educational resources. As part of this initiative, the Michigan Virtual University shall
work collaboratively with districts and intermediate districts to establish a plan to make available virtual resources that align to Michigan’s K-12 curriculum standards for use by students,
educators, and parents.
16. Create and maintain a public statewide catalog of virtual learning courses being offered by all public schools and community colleges in this state. The Michigan Virtual Learning Research
Institute shall identify and develop a list of nationally recognized best practices for virtual learning and use this list to support reviews of virtual course vendors, courses, and instructional
practices. The Michigan Virtual Learning Research Institute shall also provide a mechanism for intermediate districts to use the identified best practices to review content offered by
constituent districts. The Michigan Virtual Learning Research Institute shall review the virtual course offerings of the Michigan Virtual University, and make the results from these reviews
available to the public as part of the statewide catalog. The Michigan Virtual Learning Research Institute shall ensure that the statewide catalog is made available to the public on the
Michigan Virtual University website and shall allow the ability to link it to each district’s website as provided for in section 21f. The statewide catalog shall also contain all of the following: (A)
The number of enrollments in each virtual course in the immediately preceding school year. (B) The number of enrollments that earned 60% or more of the total course points for each virtual
course in the immediately preceding school year. (C) The completion rate for each virtual course.
17. Support registration, payment services, and transcript functionality for the statewide catalog and train key stakeholders on how to use new features.
18. Collaborate with key stakeholders to examine district level accountability and teacher effectiveness issues related to online learning under section 21f and make findings and
recommendations publicly available.
19. Provide a report on the activities of the Michigan Virtual Learning Research Institute.
42.
43. Student Performance - Michigan
Study Finding
Michigan
(2020)
Students enrolled in full-time virtual schools operated by local education authorities had a
pass rate of 51%, while students enrolled in full-time virtual schools operated by public
school academies had a pass rate of 49%; compared to the students’ non-virtual course
pass rate of 76%.
Michigan
(2019)
Students enrolled in full-time virtual schools operated by local education authorities had a
pass rate of 47%, while students enrolled in full-time virtual schools operated by public
school academies had a pass rate of 53%; compared to the students’ non-virtual course
pass rate of 79%.
Michigan
(2018)
Students enrolled in virtual schools had a pass rate of 49%, compared to the students’ non-
virtual course pass rate of 78%.
Michigan
(2017)
Students enrolled in virtual schools had a pass rate of 53%, compared to the students’ non-
virtual course pass rate of 78%.
Michigan
(2017)
Students enrolled in virtual schools had a pass rate of 53%, compared to the students’ non-
virtual course pass rate of 78%.
Michigan
(2016)
Students enrolled in virtual schools had a pass rate of 52%, compared to the students’ non-
virtual course pass rate of 87%.
Michigan
(2015)
“Cyber enrollments had a ‘Completed/Passed’ rate of 54%... whereas Non-Virtual Learners
had an 89% ‘Completed/Passed’ rate.”
44. • in 1999 Michigan banned virtual charter schools after a
case of extreme corruption between one school district and
a for-profit provider
• a decade later, the legislature passed Public Act 205, which
lifted the ban on virtual charter schools and allowed two
companies to each create one full-time program
– each of these virtual charter schools was limited to 400 students
in the first year and an additional 1000 students in second year
– but for each regular education student that registered in year
two, they were required to enroll one student from the State’s
drop out roll
• at the end of two years, the Department of Education
would determine future enrollment limits based on the
performance of the programs in those first two years
The Michigan Experience
45. Student Performance - Michigan
MEAP MICA 2010 MVCA 2010
Statewide
2010
MICA 2011 MVCA 2011
Statewide
2011
Gr 3 – Math 44.0% 14.3% 35% 42.2% 26.3% 36%
Gr 3 – Reading 75.0% 66.7% 63% 64.4% 55.3% 62%
Gr 4 – Math 23.7% 40.0% 40% 37.8% 20.5% 40%
Gr 4 – Reading 71.0% 66.7% 64% 82.2% 56.4% 68%
Gr 4 – Writing 36.8% 48.4% 47% 37.8% 25.6% 45%
Gr 5 – Math 13.9% 32.0% 30% 33.3% 36.8% 40%
Gr 5 – Reading 72.2% 68.0% 65% 77.8% 60.5% 69%
Gr 5 – Science 8.3% 8.0% 17% 18.5% 19.4% 15%
Gr 6 – Math 18.9% 20.0% 36% 19.0% 22.0% 37%
Gr 6 – Reading 75.7% 66.7% 63% 83.3% 70.7% 67%
Gr 6 – Social
Studies
21.6% 20.0% 28% 21.4% 26.2% 28%
Gr 7 – Math 34.6% 14.7% 36% 36.2% 34.4% 37%
Gr 7 – Reading 73.1% 47.1% 56% 59.6% 57.4% 60%
Gr 7 – Writing 50.0% 35.3% 48% 38.3% 34.4% 47%
Gr 8 – Math 18.8% 19.1% 29% - - 29%
Gr 8 – Reading 65.6% 66.7% 56% - - 61%
Gr 8 – Science 12.5% 9.6% 15% - - 16%
Gr 9 – Social
Studies
34.7% - 33% 28.1% 24.6% 29%
46. Legislative Oversight
Report Finding
2013 “Although there have been some recent legislative efforts to clarify expectations in such
areas as accountability and standards, states are struggling to establish accountability
mechanisms appropriate for both guiding and auditing virtual schools – even as they
allow them to expand…. A continuing challenge for states will be to reconcile traditional
funding mechanisms, governance structures, and accountability demands with the
unique organizational models and instructional methods found in virtual schools.”
2014 “…policymakers have been struggling to reconcile traditional funding structures,
governance and accountability systems, instructional quality, and staffing demands with
the unique organizational models and instructional methods of virtual schooling.”
2015 “Our analysis revealed that state legislatures have proposed bills that attempt to
increase oversight of virtual schools; however, we found little evidence to indicate that
legislative actions are being informed by the emerging research on virtual schools.”
2017 State legislatures continue to respond to challenges raised by virtual schooling, as
evidenced by proposed bills that attempt to increase oversight of virtual schools;
however, as we discuss below, fewer than 40% of proposed bills have been enacted.”
2019 “Our analysis revealed a decrease in legislative activity in 2017 and 2018, yet state
legislatures have continued to propose bills similar to previous years that attempt to
increase oversight of virtual schools.”
47. • in the spring of 2010, only months before the
review from the Department of Education would
have occurred, the legislature moved to remove
all meaningful restrictions on the number and
enrollment levels of virtual schooling in the state
• Senate Bill 619 removed the cap on the number
of virtual charter schools and limited enrollment
for each virtual charter school to 2,500 students
in the first year, 5,000 students in the second year
and 10,000 students after the second year
The Michigan Experience
48.
49. Rank School district State/Territory Students in 2016
1 New York City New York 984,462
2 Los Angeles Unified California 633,621
3 City of Chicago (SD 299) Illinois 378,199
4 Miami-Dade County Florida 357,249
5 Clark County Nevada 326,953
6 Broward County Florida 271,852
7 Houston ISD Texas 216,106
8 Hillsborough County Florida 214,386
9 Orange County Florida 200,674
10 Palm Beach Florida 192,721
11 Fairfax County Virginia 187,467
12 Hawaii Department of Education Hawaii 181,550
13 Gwinnett County Georgia 178,214
14 Wake County North Carolina 160,467
15 Montgomery County Maryland 159,010
16 Dallas ISD Texas 157,886
17 Charlotte-Mecklenburg North Carolina 147,428
18 Philadelphia City Pennsylvania 133,929
19 Prince George's County Maryland 130,814
20 Duval County Florida 129,479
21 San Diego Unified California 128,040
22 Cypress-Fairbanks ISD Texas 114,868
23 Cobb County Georgia 113,151
24 Baltimore County Maryland 112,139
25 Shelby County Tennessee 111,403
50.
51. “The well-documented, disturbingly low performance
by too many full-time virtual charter schools should
serve as a call to action to state leaders and authorizers
across the country. It is time for state leaders to make
the tough policy changes necessary to ensure that this
model works more effectively than it currently does for
the students it serves. It is also time for authorizers to
close chronically low-performing virtual charter public
schools.”
(National Alliance for Public Charter Schools, the 50-State Campaign for
Achievement Now, & the National Association of Charter School Authorizers,
2016, p. 2)
A Call to Action to Improve the Quality of
Full-Time Virtual Charter Public Schools
52. “We urge authorizers to work within existing state
policy frameworks to close chronically low-performing
full-time virtual charter schools. Authorizers can take
the necessary steps to close such schools without any
changes to state law. Authorizers have a legal and
moral responsibility to close chronically low-performing
charter schools of any kind, including full-time virtual
charter schools.”
(National Alliance for Public Charter Schools, the 50-State Campaign for
Achievement Now, & the National Association of Charter School Authorizers,
2016, p. 6)
A Call to Action to Improve the Quality of
Full-Time Virtual Charter Public Schools
53. 1. We recommend that states only permit authorizers that have been granted statewide or regional chartering
authority to oversee full-time virtual charter schools that enroll students from more than one district, while
still allowing districts to authorize full-time virtual charter schools that enroll students only from within their
districts. In order to curb the temptation to authorize for financial gain, we also recommend that states cap the
amount of authorizing fees that an authorizer can withhold from a full-time virtual charter school.
2. We prefer that states initially maintain a core principle that full-time virtual charter schools, like all other types
of charter schools, must serve all students. However, should it be shown that other interventions prove unable
to make full-time virtual charter schools successful with all students, states should study the creation of
criteria for enrollment, a change which in many states may require that full-time virtual charter schools
operate as something other than charter schools.
3. We recommend that states require authorizers and schools to create desired enrollment levels for the full-time
virtual charter schools in their states for each year of their charter contracts, not to exceed a certain number of
students per school in any given year, and allow schools to grow – or not – based on performance.
4. We recommend that states require authorizers and schools to jointly determine additional, virtual-specific
goals regarding student enrollment, attendance, engagement, achievement, truancy, attrition, finances, and
operations and to include these goals in the schools’ charter contracts. These goals are in addition to the
rigorous goals that every charter school contract should contain. We recommend that authorizers make
renewal and closure decisions based upon schools’ achievement of the goals in their contracts.
5. We recommend that states require full-time virtual charter school operators to propose and justify a price per
student in their charter school applications. We also call on states to seek guidance from experts and
researchers in determining responsible levels of funding based on the real costs of full-time virtual charter
schools.
6. We recommend that as states establish valid cost levels for operating full-time virtual charter schools, they
also fund full-time virtual charter school students via a performance-based funding system.
A Call to Action to Improve the Quality of
Full-Time Virtual Charter Public Schools
55. Associate Professor of Instructional Design
College of Education & Health Services
Touro University California
mkbarbour@gmail.com
http://www.michaelbarbour.com