Preparing high school students for college-level math

>>SHANNA RUSS: Thank you
so much for joining us today for Setting Up for Success:
Preparing High School Students for College‑level Math. This webinar is hosted by the
New York Research Partnership for Alternative Pathways and
the Rhode Island Pipelines to College
and Career Research Partnership. At this time, I would like to
introduce Dr. Katherine Shields who will
be moderating today’s webinar. Good afternoon, Katherine,
and have a great session.>>KATHERINE
SHIELDS: Thank you, Shanna. Welcome, everyone,
and thank you for joining us. I’m Katherine Shields,
a researcher with the Regional Educational
Laboratory Northeast & Islands. This webinar grew out of
interest from two of our state research partnerships, both are
looking at ways to reduce the number of high school graduates
who place into remedial education in math, sometimes
referred to as developmental education. The New York Partnership for
Alternative Pathways is focused on research that informs its
policies related to graduation pathways, as well
as college and career readiness, and the Rhode Island Pipelines
to College and Career Research Partnership uses research to
increase public college access and degree completion
rates, especially for high‑needs students, and uses
data to strengthen students’ trajectories
into growth industries. Today our webinar will begin
with an overview of findings from an impact evaluation of a
high school‑based program which aims to reduce
college math remediation rates. Then we’ll hear from
a practitioner’s perspective on what it takes to implement such
a program at a statewide level. And you’ll have a chance to
ask questions at the end of our presenters. It’s my pleasure
to introduce our presenters. Our guest researcher is
Dr. Angela Boatman of Vanderbilt University, where she
is Assistant Professor of Public Policy and Higher Education. Her research explores the
outcomes of policies designed to increase college completion
for populations traditionally underrepresented
in higher education. She’s currently conducting
several large‑scale studies on the impact of innovations in the
delivery of remedial courses. Dr. Boatman is a faculty
affiliate of the Center for the Analysis of Postsecondary
Readiness housed at the Community College Research
Center at Teachers College. And she’s also an affiliate of
the Center for Education Policy Research at Harvard University. Our discussant is Dr.
Marla Davis from the Mississippi Department of Education
where she is Bureau Director for Secondary
Curriculum and Instruction. In that role, she
provides support to educators, administrators, and curriculum
coordinators on the common core state standards, as well
as on math assessments and STEM education. For her work implementing
the math readiness program that she’s going to talk about
today, Dr. Davis was awarded the President’s Trailblazer
Award from the Southern Regional Education Board. She also has many years’
experience in the classroom as a secondary math teacher. So, I’ll
turn it over to Dr. Boatman.>>ANGELA BOATMAN: Thank you. I am
really happy to be here today. I’m going to share some
background on the innovative college remediation
effort known as SAILS, and I’ll talk about the
SAILS program in more detail, which is taking place in
high schools across Tennessee. And I’m also going to share some
of the findings from our recent research study that was looking
at the impacts of SAILS on students’ college
outcomes two years later. So, the research I’m going to
discuss was a collective effort across several partners,
primarily in the state of Tennessee. It
was myself here at Vanderbilt, along with a graduate student,
along with some coauthors from Harvard
Center for Ed Policy Research, but it also involved multiple
state agencies across Tennessee. That included the Tennessee
Department of Education representing the K‑12 sector,
the Tennessee Higher Education Commission, Tennessee Board
of Regents, Measure Tennessee, which is a consortium that
provided the data for the study, the SAILS program itself, which
stands for Seamless Alignment and Integrated Learning Support,
they were an instrumental partner in this work,
ACT, which I will talk about a little later. And then the entire research
project was funded by the Bill and Melinda Gates Foundation. I may
not need to tell this audience, but college remediation touches
a pretty significant number of college students each year. Over two‑thirds of students
attending community colleges report taking
at least one remedial course, typically those
courses are in math or English. That’s about 40% of students
attending four‑year colleges. In the past couple of years,
we have seen some really good research, rigorous, strong
research out there that has largely concluded when
looking at the effects of these courses overall. We’ve sort of seen these null
or negative effects on student persistence, students’ degree
completion when enrolling in remedial courses. There’s been a lot of criticism
of the effectiveness of remedial courses in college, but, again,
they are affecting such large numbers of students. However, we haven’t seen
evaluations looking at whether remedial courses are actually
improving students’ content knowledge ‑‑ math knowledge
in the case of a math course. And so that’s really
largely what our study set out to examine. But, of course, colleges
who continue to serve students regardless of their level of
academic preparation have begun already to experiment with the
ways they offer these courses, and that
has taken many different forms. I’m sure you’ve heard of a
lot of these different types of interventions, including
blended learning courses, corequisite courses where
the remediation class is taught alongside the college‑level
course simultaneously. Some are bridge programs. The goal of all of
these innovations is to improve students’ academic success. And so the Tennessee SAILS
program is one example of an alternative model
to traditional math remediation. So it’s another really nice
example of an innovation taken on by a state. And it is in many ways
a collaboration between the K‑12 and higher ed systems
in the state of Tennessee. So the SAILS math course is
a college remedial math course that’s offered in
the senior year of high school, and it has a very different
format than a traditional high school course. The instruction is largely ‑‑
it’s delivered entirely online, and the students work at their
own pace at computers either in a computer lab, at
laptops in their math classroom. The teachers are available
in the classroom and they’re providing more individual
assistance or tutoring not in a traditional lecture‑style course
typically seen in high school math classes. The program has five modules. Once students complete all five
of the modules with an exam at the end of each module that’s
proctored by their teacher, they are considered
to have completed the course. And a really key component of
this program is a position known as the field coordinator. And the field coordinators
are people who are based in the region’s community colleges
who visit high school classrooms regularly and have a set of high
schools they visit on a rotation and they help to train the high
school math teachers to ensure consistent implementation
of the curriculum, and then also to help the
teachers in monitoring student progress,
so that if there is an issue, they’re able to note this
earlier than later and there can be some kind of intervention
or support provided. So, already that classroom
description should sound pretty different than a typical
high school math course because you’ve got college liaisons
essentially visiting high school classes on a regular basis. But it’s
also structurally different. So the SAILS course operates
quite differently when it comes to the longer trajectory
towards college‑going. So in the traditional
high school‑college pathway, a student would take their
ACT score in Tennessee in their junior year of high school,
almost all students in Tennessee take the ACT in 11th grade, and
they receive a score on the math portion of that test. And that score then determines
a student’s need for remediation once in college. A student gets a score in the
traditional model in 11th grade and that then goes —
the student then goes on to take their 12th grade math course. In this
case, in a traditional pathway, they would take just a
typical 12th grade math course. The student would then,
if he or she goes on to college, they would then be placed in
remedial math in college because their 11th grade score was below
where it needed to be to pass into college‑level math. So that student would then,
in the traditional pathway, need to take the remedial course
either before college math or alongside college math. However, if a student attends
a high school where SAILS is available, the student would
then instead take SAILS during their
senior year of high school. And so the SAILS course
effectively exempts the student from needing
remediation once in college. So that student would then
enroll in college directly into their college‑level math course
and would not need to first take a remedial math course or
take a corequisite math course, thus freeing up time to take
a different course, for example. And the SAILS
program began with a pilot, began in 2012-13
and scaled up significantly the following year. This picture just shows the
scale‑up on the left set of bars over time. And then you can see that there
are still a little over 150 high schools that do not have SAILS
offered in the high school. The SAILS program has been very
intentional about its scale‑up because they wanted to ensure
that they can provide support necessary for
the successful implementation. So they have been
intentional about the amount, the number of new schools
they bring on board each additional year. So, of course, in order to
isolate the effects of the SAILS program, this would
be a challenge anyway given that there are issues of selection
as to who enrolls in the SAILS course in the first place, but
it turns out to be especially difficult in a state
like Tennessee that has so many exciting, interesting higher
ed policy changes going on. Many of you may be familiar
with the Tennessee Promise, which is a statewide policy
that impacts recent high school graduates, allowing them to
enroll directly into Tennessee’s community colleges and
a handful of four‑year colleges tuition‑free. That policy was adopted in
2014‑15 for the graduating class of 2015, which is kind of right
in the middle of this large scale‑up of the SAILS program. And then also that same
fall, in the fall of 2015‑16, the state
adopted corequisite remediation. This is where they moved from
having a standalone remedial math course that preceded
the college-level math course to co-requisite courses where those
two courses were taken alongside one another. These are two pretty significant
changes that happened in state policy right in the middle of
the years of our study looking at evaluating
the impact of the SAILS program. So we had to do some fancy
methodological work here to try to isolate that impact in light
of some other pretty big policy changes that were going on. And if you’re interested in more
of the technical details of how we did this and how
that interacted with other state changes, I would refer you to
the full research report that we wrote. It’s referenced in the policy
brief that is available on this webinar. You can find all of the
more technical details there. But, essentially, in summary, we
created two different comparison groups to address this issue. The first was we isolated a
comparison group of students who had the same ACT math scores,
being below the cutoff for SAILS, who both did and did not
attend high schools that offered the program. So we looked at the years before
there ever even was a SAILS program all the way up until the
most recent year of data we had and then we used the fact that
there were high schools in the state that
did not yet adopt the program. We had a comparison group there. And then we also created a
second comparison group looking at students who scored just
above and below the 19 cutoff on the ACT math test. We used the students who scored
just above the cutoff as the control group for the
students who scored just below. For that, we only used one year
of data due to the partnership with ACT that I will describe
a little bit more in a minute. These were our two
comparison groups that we used in our analysis. Largely, what we found was
pretty interesting in the sense that the effects were different
based on the context of the overall state at the time. So we first examined the effects
of SAILS in the 2013‑14 cohort of seniors. And this was before
the Tennessee Promise existed, before there was statewide
corequisite remediation. And this was actually a state
context more similar to the majority of states out there who
don’t yet have a statewide free community college program, or
statewide corequisite programs. So we thought there was value
in studying the early years of SAILS in a context that might
mirror more of the states that are out there. And we found that we didn’t
see any effects really on SAILS eligibility on students
completing high school or on enrolling in college, but we did
see this pretty dramatic drop in the percent of students that
needed to take remedial math in college, which
is exactly what we would expect. That’s sort of by definition how
the program should be working. And then we see subsequently
an increase in the percent of students that took college‑level
math and passed it during their first year of college. We also saw that SAILS students
earned about 2.2 more college credits by their second year. We didn’t see a significant
effect on college persistence or on earning a credential
within two years, but, again, we only had two
years of follow-up data here. That might not be enough time
to observe degree attainment or credential attainment. But, generally, in this early
sort of pre‑Tennessee Promise context, we see what we
would consider pretty positive findings of the SAILS program
on students enrolling in college math and
earning college‑level credits. We also looked at some subgroups
here for each of our different cohorts of students. And I think this
is a really important note here. We actually saw the
strongest improvements for the lowest‑scoring
students when it came to SAILS. So we saw the largest effects
here for students with ACT math scores that were at or below 16. This is, I think, really
important given how difficult it can be to see success with
students with these really low levels of
incoming academic preparation. We also looked by gender, and
it appears that the improvements that we were seeing in student
outcomes were largely being driven by women. Also, interesting to note given
that this is a math intervention in remedial math. So then when
we looked at the later cohorts, those cohorts that were impacted
also by corequisite remediation and the Tennessee Promise,
we see a similar drop here. Actually,
exactly the same estimate, a 28 percentage point decrease
in taking remedial math within the first year. So, again, largely
what we might expect to see. But we no longer saw the
gains here in taking and passing college‑level math. But digging
in further to these results, this was largely due to the
corequisite policy superseding the SAILS program here
because that, by definition, that policy allowed all students
to take college‑level courses. So regardless of what
your incoming ACT score was or whether or
not you participated in SAILS, you now were
allowed to enroll and, in fact, encouraged to enroll
directly in college‑level math. So some of those effects
that we saw in the early cohort, they essentially
disappear for this later cohort. And then we didn’t see
significant subgroup differences here really
by ACT score, gender, or race. So one of the big research
questions that was driving our inquiry here was on whether or
not students were ‑‑ if any of this was related to an increase
in retaining math content that came as
a result of the SAILS program. And
so in order to estimate this, we needed to re‑administer an
assessment at the end of their senior year math course. And so we worked closely with
ACT who helped us to design a shortened assessment. This was an ACT math test that
would fit into a class period, but we worked closely with them
to make sure that these tests were aligned so that we could
make the comparisons we wanted to make. It was a 35-question assessment. We administered it to students
at 119 schools across Tennessee, ultimately received
responses from about almost 70% of the students. And then
as part of that ACT post‑test, we also included a short survey. It was just a 15‑question survey
asking students about their attitudes about
math and about college‑going. And this was a pretty unique
aspect of our study because we were really trying to see if
there was anything being gained when it came to math content
from the junior year ACT through the end of
their senior year math course. And we did not find that
SAILS participation improved performance on the ACT math test
any more than students taking other senior‑year math courses,
and that’s evident here in this picture where you can see the
cutoff here of the ACT cutoff. It’s been rescaled
to be just a finer measure. But that red bar represents
the 19 on the ACT math test. And you can see there’s
not really a jump here in either direction between the students
just below the cutoff that were eligible for SAILS
and the students just above. So we didn’t see improved
math performance for SAILS. But again, this
is just for students, again, right at the cutoff for
assignment to the SAILS program. But on those student survey
responses, at the same time, students were reporting positive
impacts on their perceptions of math. So SAILS students were 7
percentage points more likely to report that their senior year
math course content would be useful in their career or 10
percentage points more likely to report being better
prepared for college‑level math. They reported
being more interested in math. Interestingly, they were
less likely to report that their class stayed busy. But having visited a number of
SAILS classrooms and talked with field coordinators as
well as math course instructors, this could also be due to the
work at your own pace nature of the courses and that high school
students aren’t typically used to having this free time in
which to structure their course. They could perceive
this as not being quite as busy. But, generally, we saw these
positive impacts on students’ attitudes towards taking
math as a result of SAILS. So at the end of this study,
we concluded that SAILS was definitely effective in shifting
remediation back to high school. I got a lot of students ‑‑ a
large number of students out of remediation in college, and that
is exactly what the program was intended to do. In the 2013‑14 cohort, prior
to these other policy changes in Tennessee, it increased the
proportion of students that were taking college
math by 14 percentage points. And
by the end of their second year, they had
completed more college credits. In later years, we continued
to see that the remediation need went down. But, again, the effects largely
disappear due to the other larger policy changes
that were going on in the state. But regardless of cohort, the
SAILS students did report more positive
experiences in the course. There are a couple of important
caveats of this research. We are only looking at students
coming right out of high school. Those were the ones that
were eligible for SAILS in high school but then went
right on into college courses. So we’re missing students
who may take some time off in between high school and college. We looked only at
community college students only, as this was largely the
population ‑‑ that was largely where
this population was attending. But we also have a pretty
limited timeframe. Two years. The hope is to continue
following students into years three and four to see if there’s
maybe a longer‑term impact of the SAILS program. So,
there were a couple, I think, just final
notes here about this research, and then I’m interested
in the conversation to follow. So, of the additional students
that took college‑level math, only about half
of those students passed the college‑level math course. So while we saw that this
program opened up college math to a new group of students, and
if we believe getting to college math is barrier, an unnecessary
barrier for some students, this would be a success. It did open up that course
to a new group of students. But it does raise the question
as to what are the barriers for students once they are in that
college‑level math course with only about half of the new
students passing that course. What are some of the other
barriers that might be getting in their way? And it also raises some pretty
important questions about how well college‑level math
might be aligned with students’ subsequent
course‑taking or their major. The best
way to assess remedial needs, calling up questions of is the
ACT math the assessment tool, how about considering multiple
measures or assessments when students get to college. That’s a larger
conversation that our study didn’t delve into. But, you know, Tennessee was
largely using the ACT math for assessment. And also, you know, I think
it’s important to consider the long‑term
outcomes of a corequisite model, which a lot of research
recently is attempting to do, so I will be paying
close attention to some of the long‑term impacts that we
see from the corequisite model. And I’ll just conclude by saying
I really think it was clear that leaders in Tennessee were to be
applauded not only for adopting a pretty innovative K‑12‑higher
ed partnership program in college remediation, but really
also for taking the risk to assess its impact. There are really, really
important questions in remedial education out there where we
still don’t have answers as to whether these programs
work, who they work for, and how they work. And so states like Tennessee who
are willing to take the risk to assess one
of their own programs, I think, are really
to be commended. Thank you.>>KATHERINE SHIELDS: 
Great. Thank you, Dr. Boatman. If you have any questions for
Dr. Boatman about the Tennessee SAILS program or about the
findings from this evaluation, please post them in the chat. I see some folks have already
put some questions there that we’ll get to at the end. So now we’d like to
take a moment and hear from our audience. So there’s going to be
a poll question on the screen. Just please click to answer. What strategies have you used
in your institution to prepare students for college math? And
you can select all that apply. So some of these might be things
that happen in high school, like
the Tennessee SAILS program, or they might be things like
a summer bridge program between high school and college. And then there’s a lot of
exciting innovations going on at the college level, such
as corequisite or accelerated remedial
classes and other models. So I see
we have some folks chiming in. It looks like a few people have
used high school prerequisite courses, and a few administering
a college placement test during high school. It looks like a handful
of people have also worked with corequisite
college remediation courses. So that’s
great to see. Thank you. So I know we’ve got a good range
of people represented who work at
both schools, districts, states, and institutions of higher
education here in the audience. So, thank you for sharing
what you’re working on now. Great. So I’d like to now turn this
over to Dr. Marla Davis who will be talking with us about her
experience as a practitioner. And Marla, you can take it away.>>MARLA DAVIS: Good afternoon. Again, Marla Davis from
the Mississippi Department of Education. And I’ll be sharing with you all
some information as it relates to our ‑‑ we call
them transitional courses in the state of Mississippi. We have actually tried to move
away from using the connotation of remediation as it relates to
supporting our students that are struggling below grade
level and those that are having deficiencies in
math and also in literacy ELA. And so what we’ve actually
put in place in the state of Mississippi is a continuum
to support our students that are transitioning from
middle school into high school, and also for those students
that are transitioning from high school into college that we have
deemed and identified as having very minor to some of
the most profound deficiencies, if you will, as it relates to
math and ELA in preparation of being college and career‑ready. And so what I’ll do is
I’ll spend a little bit of time talking to you all a little bit
about our Ready for High School Math and Literacy courses. We’ve actually begun
implementing these courses at the ‑‑ excuse me, at the middle
school level going into our second year. So this is our second year of
implementation for our Ready for High School Math and Literacy. At the
onset of offering these courses, we initially wanted to open
these courses up only to 8th grade students. It’s been within the last year
that we’ve decided to actually open these courses up also for
9th grade students that may not be ready
for English I or Algebra I. And so our Ready for High School
Math and Literacy courses are now open for any student
that is in grades eight or nine. We’ve also put some parameters
in place as it relates to our teacher training. So during the teacher training,
which is a three‑day recommended training for any teacher in the
state of Mississippi that wants to ‑‑ that is being ‑‑
that is offering these courses, these particular course
trainings allow our teachers to actually spend time learning
about what are the instructional practices that are needed for
our Ready for High School Math and Literacy Courses. And the reason that these
particular course trainings are so long is because we want to
make sure that our teachers have the instructional strategies
they need, for example, in the math course to dig deep
into what we’ve identified as the biggest areas of weakness
that many of our students are coming into high school with. And so our middle school course,
the 8th and 9th grade course, excuse me, for math spent
some time looking at operations with fractions. There’s
some time for the number system, ratio and proportional
relationships, expressions, equations, and inequalities,
and then, of course, functions
and linear relationships. As a result of reviewing
our data at the state level, we were fully aware
that our students were having difficulties in these main areas
as they transition from middle school into high school. As it relates to our Ready
for High School Literacy course, this particular course allows
our teachers to spend time identifying lessons and
very rich tasks that allow our students to complete
authentic written assignments. And then also, how do they
begin to explore a variety of complex text. So there is some time in the
training where our students –- excuse me, our teachers are able
to begin developing lesson plans around our
Ready for High School courses. For our students that are not
ready to enter credit‑bearing courses at the
high school level ‑‑ excuse me, at the college level, we have
implemented two separate courses at the 12th grade. And so we’re one of a host of
states that offer the SREB math ready and literacy ready
courses for 12th‑grade students. Now, for this particular course,
our students must be classified as 12th grade students or they
could be an 11th grader that has also received a number of credit
hours possible for them to be classified as 12th graders. But this particular course has
an ACT requirement in place for our students. And so over the last few years,
the state of Mississippi has mandated that all students
at the junior year take the ACT. And so all schools and districts
are required to begin reviewing their ACT data for any
student that has an ACT English sub-score
or math sub-score below 15. Any student at the 12th‑grade
level that has that particular score range on the ACT is
required to take the SREB Math Ready
and Literacy Ready courses. This particular
course, because of the rigor, also has a three‑day teacher
training implemented as well. And so any teacher that
is assigned the SREB course must attend a three‑day training and
earn an additional endorsement. Those 900-level endorsements
are a requirement that have also been placed in our
state board policies and our accreditation standards. We also have an Essentials
for College Math and Literacy courses that are fashioned
a little bit –- they’re very comparable to the SREB course,
but the difference actually is the enrollment piece
as it relates to our students. So students that have
an ACT sub‑score 15 or above are required to take the Essentials
for College Math or College Literacy course. When I say 15 or above, we’re
thinking any students between the score ranges of 15 to 18. Any student that
has those particular sub‑scores, as I’ve
already previously mentioned, are required to take
the SREB and Essentials courses. And so I wanted to talk to
you all a little bit around our implementation and training as
it relates to the three courses that I’ve just mentioned, the
three courses for math and the three courses for literacy. So, any teacher in the
state that wants to teach these courses ‑‑ excuse me, any person
that wants to offer training around these courses must
receive a master credentialing where they are identified
as a master teacher training. Those particular certifications
are typically offered outside of the state by national experts. And that particular training is
usually around three days to one week long. We also require that any
facilitator or trainer for these courses participate in a series
of classroom observations. Those classroom observations
are done in classrooms of other master teacher trainers. And we also require that
they spend some time looking at teacher data as it relates
to their implementation of the course and how they
actually work with students. And then finally, our
implementation and training also require that we spend
a considerable amount of time looking at the
feedback after each one of our training sessions. And so we’re looking at
the types of questions that our trainers
are asking during the training, in addition to the types
of questions our teachers are asking. We spend some time also looking
at the types of questions that may arise as a result of the
types of texts that are being used in the training, and
also the types of tasks that are being engaged
in throughout the training. And so implementation
and training in the state of Mississippi is very robust. We don’t take it lightly because
we’re also thinking about the caliber of students that
are going to be enrolled in these courses. And
so our master teacher training, classroom observations,
and feedback are actually on a continuous cycle throughout
the entire implementation year. So, one of the things that I’ve
mentioned to you all is that we have an actual state
board policy in place around our transitional courses. Again, we try to move
away from the word remediation, but we have an actual
state board adopted policy as it relates
to our transitional courses. So beginning with the 2017‑2018
school year, any LEA ‑‑ again, our high schools that offer the
Essentials course ‑‑ must adhere to the following requirements. The teachers must have above
the 7 through 12 math or ELA endorsement, earning a separate
credentialing at the 900 level. In addition to that, we’ve also
put in place that any school or district that is in violation of
these particular series of six requirements will result
in an accreditation violation of accountability
standards 2 and 26. And we just feel that given the
caliber of students that these courses are intended to support,
it was very important for us to put a policy in place
around what was required for the teachers, what’s
required for our students. And then we also have a policy
in partnership with our IHLs. One of the things
that was really important for implementation in the state
is that not only did our IHLs ‑‑ some states call them IHEs,
Institutions of Higher Education — but in addition to offering
the training for teachers, it was very important that we
had a partnership with our IHLs where they also sit
in on the three‑day training. This was one of
the requirements, if you will, that we put in place with our
IHLs so that they would be able to see the rigor that’s
involved in each of these transitional courses. And as a result of IHL faculty,
from the math, and the English, and/or reading
departments at each of our IHLs, sitting in on this training, we
were able to craft a policy in partnership with them
around our remedial courses, our transitional courses
for our 12th grade students. And so what you see here on your
screen is an excerpt from policy 608 which basically indicates
that any student that takes our transition courses
for 12th graders, that, again, you have not pretty much met the
benchmark for ACT and college readiness, as long as that
student completes our Essentials course with an 80 or above, they
will not be required to take remedial courses at
the collegiate level for college algebra and English composition. This particular policy is in
place in all eight of our public universities
in the state of Mississippi. Really quickly,
before my time runs out, I wanted to share with you all
the impact that these courses have had on our 12th
grade performance on the ACT. So what you will see here
is a breakout of data for our Literacy Ready courses
and our Math Ready courses. This is actually combined data
from our SREB course with our Essentials course. And so what I would like for you
all to kind of see is that we’ve made tremendous gains for those
students that have enrolled in our SREB and/or Essentials
courses at the 12th grade. Our
biggest gain, as you can see, has been right here in
science with a typical student’s improvement score on the science
section of the ACT has increased 3.2 points,
and that’s a tremendous gain. Again, all of our ACTs ‑‑ our
juniors are required to take the ACT at the junior year. And then we also see some gains
as it relates to students that were enrolled in the Literacy
Ready course and then retook the ACT
post‑enrollment in our courses. We’re very excited to see this
uptick in improvement at the high school level for students
that are enrolling in remedial and transitional courses
prior to entering college. Overall,
we have seen a tremendous gain, not only as a result of
our essentials and SREB courses, but we’ve seen tremendous gains
as it relates to ACT achievement overall
in the state of Mississippi. Our 11th graders are meeting all
four ACT benchmarks remaining at 9%. Our grade 11 ACT composite score
has increased across the entire state from 17.6 in the year
2015 to 17.8 in the year 2018. And then we have held pretty
steady for the class of 2018 composite scores, while the
national average dropped a small percentage in the year 2017. And so we just thought it was
very important as a practitioner to share with you all the
steps that we’ve put in place to ensure a continuum of support
from our students that are transitioning from middle school
to high school and to college. And one piece that I would love
to definitely bring out before my time runs out is the fact
that we spent a lot of time at the state department
working with classroom teachers, working with practitioners,
working with consultants, and also working with
IHL to ensure that the readiness courses provide a beautiful
alignment with our college and career readiness standards. And so any student that
has enrolled in our remedial transitional courses
does not see a break in content. And so we believe that these are
some of the practices that have led to the improved achievement
that we began to see over the last few years
in the state of Mississippi. Of course, our overall
goal being to ensure that our students are college and career
ready and that they have the skills that are
necessary to perform outside of the classroom.>>KATHERINE SHIELDS: Thank
you so much, Dr. Davis, for that presentation. We’d now like to
take on some of these questions. Thank you for all the questions
that have come in on the chat. I’ll start with this one about
the Tennessee SAILS program for Dr. Boatman. Did the treatment group include
all the students in the SAILS program, or only those students
who were successful in passing the course?>>ANGELA BOATMAN: This
is a really good question. We included all students
who took the SAILS course, but we did not restrict it to
just the students who passed the course. We did look at the
pass rates of the SAILS program, and they’re actually
very high across the state. There is some
differentiation by high school. But across the state, it’s
upwards of 90% of students that enroll in the
course pass the SAILS course. So the estimates
wouldn’t be that different, but we did not restrict it to
the students that just passed. We ran some descriptive numbers
for the state just to provide the state with some data about
the differences between the students
who passed versus did not pass. The state was obviously
very interested in that. But for our technical analysis
we included students regardless of if they
completed the course or not.>>KATHERINE SHIELDS: Thank you. So, for Dr. Davis about
the program in Mississippi, a couple of people have asked
did students get a credit for the remedial math course? Did it count toward graduation? Did
they get a high school credit?>>MARLA DAVIS: I
just want a point of clarity. Are we speaking about our 8th
and 9th grade course or our 12th grade? Let me
give a little point of clarity. Our grade eight and 9th grade
Ready for High School course does offer a math and
an ELA reading Carnegie unit. The answer to that is yes. Those particular credits are
considered lower than Algebra I and lower than English I. So any student that earns those
Carnegie units is still required to earn three additional
maths or three additional English credits. For the 12th grade course,
they do offer a Carnegie unit as well, and those are considered
higher than Algebra and higher than English II. For some students, it does serve
as their fourth‑year math or fourth-year English depending
on how they earn their credits. So both
courses do offer Carnegie units. Great question.>>KATHERINE SHIELDS: Thank you. So looking back at
the SAILS program in Tennessee, Dr. Boatman, are you collecting
data on which college math course
students took following SAILS?>>ANGELA BOATMAN: Yes.
We worked closely with the folks within the colleges to
identify the four most popular math courses, I believe, the three
to four most common math courses that students enrolled
in for college‑level math. We have not yet linked students’
remediation requirements with any particulars regarding their
proposed field of study and how that impacts
the math courses that they take. This would be a next step of
this work is to think about the individual math course and
how that impacts the student’s trajectory in either a STEM
field or progress through other types of degrees. But, yes, we have broken out the
math courses by the different course options
that students might take. I don’t believe we report
on this in the full report, but it is something that we have
looked at at the request of the state as well. They were interested in
differences by different types of courses.>>KATHERINE SHIELDS: Great.
So now another question, Dr. Davis, for
you about Mississippi’s program. So, to clarify,
do students take the ACT twice?>>MARLA DAVIS: No. Students
are only required by state board policy to take it as juniors. The students that are in
the study that I shared in the presentation were offered
an opportunity to retest after taking the state‑mandated
ACT test as juniors. And that opportunity to retest
was actually ‑‑ we called them pilot sites, of course, because
we were looking at study data. Any of those schools that opted
in to have their students retest after taking our 12th grade
readiness courses were provided a voucher to retest.>>KATHERINE SHIELDS: And,
Dr. Davis, one other clarifying question about the model. Someone asked who
is training the master trainer? That was a reference
to somebody outside the state.>>MARLA DAVIS:
Yes. Really great question. Any teacher that’s receiving
the master trainer credentialing actually does go through
training with the Southern Region Educational Board.>>KATHERINE SHIELDS: 
Great. Thanks. So, Dr. Boatman, there’s a question
about the SAILS program. Is there any sunsetting of
the SAILS student’s eligibility in college?>>ANGELA BOATMAN: I’m assuming
that just means that if a student doesn’t enroll
in college immediately or takes some time in taking their math
requirement, does that sunset? I guess that’s how
I’m interpreting that question. And the answer is that a student
who completes the SAILS program successfully in high school is
supposed to be exempt from the remediation requirement
regardless of when that is. So, to my knowledge, that would
stay with the student throughout different semesters, unless I
am not understanding the use of sunsetting there.>>KATHERINE
SHIELDS: Great. Thank you. And another question for
you about more the study design. Was there a comparison
group for the improvements in pre‑ to post‑tests?>>ANGELA BOATMAN: So our
assessment of learning gains was looking at students
above and below the cutoff. So we ‑‑
and we did not see improvement. That might make me
wonder if that’s a question for Mississippi. We did not observe
improvements at that cutoff. But, again, I would offer the
clarification that we were just able ‑‑ given that study design,
we were only able to look at students with a point or two
to the left of the cutoff and a point or two to the right. So that’s a pretty narrow band
of students that were eligible for the SAILS program. And so we’re not able to talk
about learning gains that may or may not have
existed below that ACT cutoff. And I think that’s
an area for future research, especially given that we saw
some of the positive impacts of SAILS were being captured
by the lower‑scoring students. But just given the study
methodology and the way that we could isolate that cutoff, we
weren’t able to draw conclusions about students
further below that cutoff.>>KATHERINE SHIELDS: I
see we’ve got a clarification. That was actually a question
intended for Mississippi. So, Dr. Davis, for those pre‑ to
post‑test improvements that you shared
from Mississippi students, was there
a comparison group for that?>>MARLA DAVIS: Okay. I’m sorry. I just
heard the end of that question. Was there a comparison group?>>KATHERINE SHIELDS: When you
showed the improvements in pre‑ to post‑test, was there a
comparison group for that study?>>MARLA DAVIS: No, this
was actually one of the first studies
of this type for the state, so
the answer is not at this point. But we do have plans to actually
duplicate this study with a larger population
of students in the state.>>KATHERINE
SHIELDS: Great. Thank you. So this is a question
about the SAILS program. And I believe
this may have been answered. But
was there a teacher in the room, or was it entirely students
working self‑paced on computers?>>ANGELA BOATMAN: Yeah. No.
There is always a teacher in the room. So the teacher is — in a lot
of the SAILS classrooms that I visited, this teacher
is usually roaming the room, speaking
with individual students, answering questions
in a more one‑on‑one basis. But it is taking place
during a typical class session, and so
the teacher is in the classroom, getting the class started,
adjourning it when it’s over. It’s just more the sort of
middle portion of the class is where the teacher sort of takes
a backseat and the computer is largely
providing the instruction.>>KATHERINE SHIELDS: Thanks.
We had another question about that program. Who wrote the online math SAILS
program and how are students selected for it?>>ANGELA BOATMAN: 
Yeah. That’s a great question. It was originally conceived by
some members of the math faculty at
one of the community colleges, Chattanooga State
Community College in Tennessee, who partnered with Pearson to
development the course itself. So the software was
a Pearson product, My Math Lab. And they began piloting that. That was what was being
used in the community college. And so then the
SAILS pilot came out of that, where they used that same
software to then move that into the high school. And
the program since has, you know, evaluated the use of Pearson and
thought about different models. But when it
originally was developed, it was a Pearson product.>>KATHERINE SHIELDS: So,
Dr. Davis, in looking at Mississippi again,
someone is asking is any work being done in the state
on deemphasizing algebra at the college level and
replacing it with discrete math or statistics?>>MARLA DAVIS: We’re not
actually engaged in any conversations along those lines. Not that that’s
not a priority conversation. It’s just not one
that we’re having at the moment. But we actually had ‑‑ in our ‑‑
we actually have standalone discrete math and standalone
statistics courses that we actually absorbed into our
new framework across grades ten, eleven, and twelve. And so I don’t know if
that kind of gets to the answer, but I just would probably
say at this moment in time, it’s not one
that we’ve had at this moment.>>KATHERINE SHIELDS: 
And another question for you for Mississippi. What does
the teacher training look like? Does it focus on pedagogy,
instructional strategies, or content?>>MARLA DAVIS: It actually ‑‑
all of our trainings actually deal with both. And so the very, very first day
of the three‑day training spends a considerable amount of time on
the first half of day one really understanding the depth of
knowledge of the teachers that are actually at the training. The second part of day
one spends a lot of time talking about how do you ensure
the depth of knowledge is at the level it needs to be for the
teachers that are going to be going
into day two and day three. Day two and day three actually
spend a lot of time around instructional strategies. It allows the teachers
the actual engagement with the activities and the tasks that
are going to be put in front of their students. And then the latter part of day
three is actually something that we probably
would coin as a demonstration, where we place our ‑‑ a lot
of states do something similar, but we actually allow
our teachers to get into small groups and do a demonstration of an actual lesson. That’s an opportunity for the
facilitators to provide feedback around how to differentiate
instruction for the caliber of student that’s in these courses. While many of our students
in these courses will have a similar ACT sub‑score, we
provide some tips on how do you differentiate instruction
around particular deficiencies. And so I hope that kind of
answers the question around how we do our three‑day trainings.>>KATHERINE SHIELDS: 
Going back to Tennessee, there’s a question asking
for clarification, I believe, about the findings
you were discussing previously, Dr. Boatman. He’s asking was the 50% pass
rate for the students just below the cutoff –- that passed the
test — different than the pass rate for those comparison
students just above the cut score in terms of
pass rate on the SAILS course?>>ANGELA BOATMAN: Right. So the 50% pass rate was largely
related to the students below the cutoff. So it’s sort of saying of
the SAILS program did bring more students
into college‑level math. So with more students
entering college‑level math, there was an increase in overall
numbers of students passing college‑level math because
there were more students in college‑level math. But that of the students that
were brought in as a result of the SAILS program, only about
half of those students that were brought in as a result of
the program passed the course. So it’s related to the
SAILS‑eligible students that are below the cutoff.>>KATHERINE SHIELDS: 
Okay, great. Thank you. And then this was a question
for, again, for Tennessee. What is a typical student if the
change in the average scores is 1.8 points? I don’t know if we need
some clarification maybe on that question.>>ANGELA BOATMAN: Also,
I wonder if that question might also
be for Mississippi, could it be?>>KATHERINE SHIELDS: Oh,
sorry. Yeah. It sounds like it. Maybe you can clarify on that. Dr. Davis, does that
sound like that’s one for you? What is a typical student, given
where your average score was and the change in
average scores was 1.8 points?>>MARLA DAVIS: And so
I’m trying to –- I’m trying to understand
the question one more time.>>KATHERINE SHIELDS: Why don’t
we come back to that one while Anita types that back in. I think she
was typing up a clarification. Okay. Here we go. In looking at the Mississippi
scores on the ACT scores, there’s a column that
says change for typical student, but there’s another one
that refers to the change in average score. I think she’s trying to
understand what does that next column indicate.>>MARLA DAVIS: 
Gotcha. Gotcha. Gotcha. So what we did is we
did an actual aggregate where we evaluated our scores
across the entire subgroup, and then we actually
took an actual approach where we identified the average
improvement just across each one of the groups. And so there is some extra data
that was not included in the slide that we shared with you
all that looks at subgroup data across gender, across race,
across socioeconomic status that’s not included, and that’s
where that data comes from. And I’m sorry.>>KATHERINE
SHIELDS: Okay, thank you.>>MARLA DAVIS: We didn’t
include it in the data point on the slide. Great question.>>KATHERINE
SHIELDS: Okay, thank you. These were all great questions. So at the end of this webinar,
your browser is going to redirect you to a survey. We really encourage you to just
take a minute and send us some feedback so that we
can keep improving our webinars. I want to thank Dr. Angela
Boatman and Dr. Marla Davis so much for sharing
their insights and experience. As a reminder, you can download
the Tennessee SAILS evaluation report as well as today’s
slides in the download box. There are also two reports there
from our colleagues at other regional educational
laboratories about remediation work done in Oregon and Florida. So, thank you very much. I’ll turn it over to Shanna.>>SHANNA RUSS: Thank you all
so much for joining us today. Today’s webinar recording
will be archived and uploaded to IES’s YouTube channel
as well as to our website. If you
registered for today’s webinar, you’ll receive an email from
us with a link to the recording once it’s ready. Additionally, don’t forget
to download any of those files because it will close. And
have a great rest of your day. Thank you all so much.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *