>>SHANNA RUSS: Thank you

so much for joining us today for Setting Up for Success:

Preparing High School Students for College‑level Math. This webinar is hosted by the

New York Research Partnership for Alternative Pathways and

the Rhode Island Pipelines to College

and Career Research Partnership. At this time, I would like to

introduce Dr. Katherine Shields who will

be moderating today’s webinar. Good afternoon, Katherine,

and have a great session.>>KATHERINE

SHIELDS: Thank you, Shanna. Welcome, everyone,

and thank you for joining us. I’m Katherine Shields,

a researcher with the Regional Educational

Laboratory Northeast & Islands. This webinar grew out of

interest from two of our state research partnerships, both are

looking at ways to reduce the number of high school graduates

who place into remedial education in math, sometimes

referred to as developmental education. The New York Partnership for

Alternative Pathways is focused on research that informs its

policies related to graduation pathways, as well

as college and career readiness, and the Rhode Island Pipelines

to College and Career Research Partnership uses research to

increase public college access and degree completion

rates, especially for high‑needs students, and uses

data to strengthen students’ trajectories

into growth industries. Today our webinar will begin

with an overview of findings from an impact evaluation of a

high school‑based program which aims to reduce

college math remediation rates. Then we’ll hear from

a practitioner’s perspective on what it takes to implement such

a program at a statewide level. And you’ll have a chance to

ask questions at the end of our presenters. It’s my pleasure

to introduce our presenters. Our guest researcher is

Dr. Angela Boatman of Vanderbilt University, where she

is Assistant Professor of Public Policy and Higher Education. Her research explores the

outcomes of policies designed to increase college completion

for populations traditionally underrepresented

in higher education. She’s currently conducting

several large‑scale studies on the impact of innovations in the

delivery of remedial courses. Dr. Boatman is a faculty

affiliate of the Center for the Analysis of Postsecondary

Readiness housed at the Community College Research

Center at Teachers College. And she’s also an affiliate of

the Center for Education Policy Research at Harvard University. Our discussant is Dr.

Marla Davis from the Mississippi Department of Education

where she is Bureau Director for Secondary

Curriculum and Instruction. In that role, she

provides support to educators, administrators, and curriculum

coordinators on the common core state standards, as well

as on math assessments and STEM education. For her work implementing

the math readiness program that she’s going to talk about

today, Dr. Davis was awarded the President’s Trailblazer

Award from the Southern Regional Education Board. She also has many years’

experience in the classroom as a secondary math teacher. So, I’ll

turn it over to Dr. Boatman.>>ANGELA BOATMAN: Thank you. I am

really happy to be here today. I’m going to share some

background on the innovative college remediation

effort known as SAILS, and I’ll talk about the

SAILS program in more detail, which is taking place in

high schools across Tennessee. And I’m also going to share some

of the findings from our recent research study that was looking

at the impacts of SAILS on students’ college

outcomes two years later. So, the research I’m going to

discuss was a collective effort across several partners,

primarily in the state of Tennessee. It

was myself here at Vanderbilt, along with a graduate student,

along with some coauthors from Harvard

Center for Ed Policy Research, but it also involved multiple

state agencies across Tennessee. That included the Tennessee

Department of Education representing the K‑12 sector,

the Tennessee Higher Education Commission, Tennessee Board

of Regents, Measure Tennessee, which is a consortium that

provided the data for the study, the SAILS program itself, which

stands for Seamless Alignment and Integrated Learning Support,

they were an instrumental partner in this work,

ACT, which I will talk about a little later. And then the entire research

project was funded by the Bill and Melinda Gates Foundation. I may

not need to tell this audience, but college remediation touches

a pretty significant number of college students each year. Over two‑thirds of students

attending community colleges report taking

at least one remedial course, typically those

courses are in math or English. That’s about 40% of students

attending four‑year colleges. In the past couple of years,

we have seen some really good research, rigorous, strong

research out there that has largely concluded when

looking at the effects of these courses overall. We’ve sort of seen these null

or negative effects on student persistence, students’ degree

completion when enrolling in remedial courses. There’s been a lot of criticism

of the effectiveness of remedial courses in college, but, again,

they are affecting such large numbers of students. However, we haven’t seen

evaluations looking at whether remedial courses are actually

improving students’ content knowledge ‑‑ math knowledge

in the case of a math course. And so that’s really

largely what our study set out to examine. But, of course, colleges

who continue to serve students regardless of their level of

academic preparation have begun already to experiment with the

ways they offer these courses, and that

has taken many different forms. I’m sure you’ve heard of a

lot of these different types of interventions, including

blended learning courses, corequisite courses where

the remediation class is taught alongside the college‑level

course simultaneously. Some are bridge programs. The goal of all of

these innovations is to improve students’ academic success. And so the Tennessee SAILS

program is one example of an alternative model

to traditional math remediation. So it’s another really nice

example of an innovation taken on by a state. And it is in many ways

a collaboration between the K‑12 and higher ed systems

in the state of Tennessee. So the SAILS math course is

a college remedial math course that’s offered in

the senior year of high school, and it has a very different

format than a traditional high school course. The instruction is largely ‑‑

it’s delivered entirely online, and the students work at their

own pace at computers either in a computer lab, at

laptops in their math classroom. The teachers are available

in the classroom and they’re providing more individual

assistance or tutoring not in a traditional lecture‑style course

typically seen in high school math classes. The program has five modules. Once students complete all five

of the modules with an exam at the end of each module that’s

proctored by their teacher, they are considered

to have completed the course. And a really key component of

this program is a position known as the field coordinator. And the field coordinators

are people who are based in the region’s community colleges

who visit high school classrooms regularly and have a set of high

schools they visit on a rotation and they help to train the high

school math teachers to ensure consistent implementation

of the curriculum, and then also to help the

teachers in monitoring student progress,

so that if there is an issue, they’re able to note this

earlier than later and there can be some kind of intervention

or support provided. So, already that classroom

description should sound pretty different than a typical

high school math course because you’ve got college liaisons

essentially visiting high school classes on a regular basis. But it’s

also structurally different. So the SAILS course operates

quite differently when it comes to the longer trajectory

towards college‑going. So in the traditional

high school‑college pathway, a student would take their

ACT score in Tennessee in their junior year of high school,

almost all students in Tennessee take the ACT in 11th grade, and

they receive a score on the math portion of that test. And that score then determines

a student’s need for remediation once in college. A student gets a score in the

traditional model in 11th grade and that then goes —

the student then goes on to take their 12th grade math course. In this

case, in a traditional pathway, they would take just a

typical 12th grade math course. The student would then,

if he or she goes on to college, they would then be placed in

remedial math in college because their 11th grade score was below

where it needed to be to pass into college‑level math. So that student would then,

in the traditional pathway, need to take the remedial course

either before college math or alongside college math. However, if a student attends

a high school where SAILS is available, the student would

then instead take SAILS during their

senior year of high school. And so the SAILS course

effectively exempts the student from needing

remediation once in college. So that student would then

enroll in college directly into their college‑level math course

and would not need to first take a remedial math course or

take a corequisite math course, thus freeing up time to take

a different course, for example. And the SAILS

program began with a pilot, began in 2012-13

and scaled up significantly the following year. This picture just shows the

scale‑up on the left set of bars over time. And then you can see that there

are still a little over 150 high schools that do not have SAILS

offered in the high school. The SAILS program has been very

intentional about its scale‑up because they wanted to ensure

that they can provide support necessary for

the successful implementation. So they have been

intentional about the amount, the number of new schools

they bring on board each additional year. So, of course, in order to

isolate the effects of the SAILS program, this would

be a challenge anyway given that there are issues of selection

as to who enrolls in the SAILS course in the first place, but

it turns out to be especially difficult in a state

like Tennessee that has so many exciting, interesting higher

ed policy changes going on. Many of you may be familiar

with the Tennessee Promise, which is a statewide policy

that impacts recent high school graduates, allowing them to

enroll directly into Tennessee’s community colleges and

a handful of four‑year colleges tuition‑free. That policy was adopted in

2014‑15 for the graduating class of 2015, which is kind of right

in the middle of this large scale‑up of the SAILS program. And then also that same

fall, in the fall of 2015‑16, the state

adopted corequisite remediation. This is where they moved from

having a standalone remedial math course that preceded

the college-level math course to co-requisite courses where those

two courses were taken alongside one another. These are two pretty significant

changes that happened in state policy right in the middle of

the years of our study looking at evaluating

the impact of the SAILS program. So we had to do some fancy

methodological work here to try to isolate that impact in light

of some other pretty big policy changes that were going on. And if you’re interested in more

of the technical details of how we did this and how

that interacted with other state changes, I would refer you to

the full research report that we wrote. It’s referenced in the policy

brief that is available on this webinar. You can find all of the

more technical details there. But, essentially, in summary, we

created two different comparison groups to address this issue. The first was we isolated a

comparison group of students who had the same ACT math scores,

being below the cutoff for SAILS, who both did and did not

attend high schools that offered the program. So we looked at the years before

there ever even was a SAILS program all the way up until the

most recent year of data we had and then we used the fact that

there were high schools in the state that

did not yet adopt the program. We had a comparison group there. And then we also created a

second comparison group looking at students who scored just

above and below the 19 cutoff on the ACT math test. We used the students who scored

just above the cutoff as the control group for the

students who scored just below. For that, we only used one year

of data due to the partnership with ACT that I will describe

a little bit more in a minute. These were our two

comparison groups that we used in our analysis. Largely, what we found was

pretty interesting in the sense that the effects were different

based on the context of the overall state at the time. So we first examined the effects

of SAILS in the 2013‑14 cohort of seniors. And this was before

the Tennessee Promise existed, before there was statewide

corequisite remediation. And this was actually a state

context more similar to the majority of states out there who

don’t yet have a statewide free community college program, or

statewide corequisite programs. So we thought there was value

in studying the early years of SAILS in a context that might

mirror more of the states that are out there. And we found that we didn’t

see any effects really on SAILS eligibility on students

completing high school or on enrolling in college, but we did

see this pretty dramatic drop in the percent of students that

needed to take remedial math in college, which

is exactly what we would expect. That’s sort of by definition how

the program should be working. And then we see subsequently

an increase in the percent of students that took college‑level

math and passed it during their first year of college. We also saw that SAILS students

earned about 2.2 more college credits by their second year. We didn’t see a significant

effect on college persistence or on earning a credential

within two years, but, again, we only had two

years of follow-up data here. That might not be enough time

to observe degree attainment or credential attainment. But, generally, in this early

sort of pre‑Tennessee Promise context, we see what we

would consider pretty positive findings of the SAILS program

on students enrolling in college math and

earning college‑level credits. We also looked at some subgroups

here for each of our different cohorts of students. And I think this

is a really important note here. We actually saw the

strongest improvements for the lowest‑scoring

students when it came to SAILS. So we saw the largest effects

here for students with ACT math scores that were at or below 16. This is, I think, really

important given how difficult it can be to see success with

students with these really low levels of

incoming academic preparation. We also looked by gender, and

it appears that the improvements that we were seeing in student

outcomes were largely being driven by women. Also, interesting to note given

that this is a math intervention in remedial math. So then when

we looked at the later cohorts, those cohorts that were impacted

also by corequisite remediation and the Tennessee Promise,

we see a similar drop here. Actually,

exactly the same estimate, a 28 percentage point decrease

in taking remedial math within the first year. So, again, largely

what we might expect to see. But we no longer saw the

gains here in taking and passing college‑level math. But digging

in further to these results, this was largely due to the

corequisite policy superseding the SAILS program here

because that, by definition, that policy allowed all students

to take college‑level courses. So regardless of what

your incoming ACT score was or whether or

not you participated in SAILS, you now were

allowed to enroll and, in fact, encouraged to enroll

directly in college‑level math. So some of those effects

that we saw in the early cohort, they essentially

disappear for this later cohort. And then we didn’t see

significant subgroup differences here really

by ACT score, gender, or race. So one of the big research

questions that was driving our inquiry here was on whether or

not students were ‑‑ if any of this was related to an increase

in retaining math content that came as

a result of the SAILS program. And

so in order to estimate this, we needed to re‑administer an

assessment at the end of their senior year math course. And so we worked closely with

ACT who helped us to design a shortened assessment. This was an ACT math test that

would fit into a class period, but we worked closely with them

to make sure that these tests were aligned so that we could

make the comparisons we wanted to make. It was a 35-question assessment. We administered it to students

at 119 schools across Tennessee, ultimately received

responses from about almost 70% of the students. And then

as part of that ACT post‑test, we also included a short survey. It was just a 15‑question survey

asking students about their attitudes about

math and about college‑going. And this was a pretty unique

aspect of our study because we were really trying to see if

there was anything being gained when it came to math content

from the junior year ACT through the end of

their senior year math course. And we did not find that

SAILS participation improved performance on the ACT math test

any more than students taking other senior‑year math courses,

and that’s evident here in this picture where you can see the

cutoff here of the ACT cutoff. It’s been rescaled

to be just a finer measure. But that red bar represents

the 19 on the ACT math test. And you can see there’s

not really a jump here in either direction between the students

just below the cutoff that were eligible for SAILS

and the students just above. So we didn’t see improved

math performance for SAILS. But again, this

is just for students, again, right at the cutoff for

assignment to the SAILS program. But on those student survey

responses, at the same time, students were reporting positive

impacts on their perceptions of math. So SAILS students were 7

percentage points more likely to report that their senior year

math course content would be useful in their career or 10

percentage points more likely to report being better

prepared for college‑level math. They reported

being more interested in math. Interestingly, they were

less likely to report that their class stayed busy. But having visited a number of

SAILS classrooms and talked with field coordinators as

well as math course instructors, this could also be due to the

work at your own pace nature of the courses and that high school

students aren’t typically used to having this free time in

which to structure their course. They could perceive

this as not being quite as busy. But, generally, we saw these

positive impacts on students’ attitudes towards taking

math as a result of SAILS. So at the end of this study,

we concluded that SAILS was definitely effective in shifting

remediation back to high school. I got a lot of students ‑‑ a

large number of students out of remediation in college, and that

is exactly what the program was intended to do. In the 2013‑14 cohort, prior

to these other policy changes in Tennessee, it increased the

proportion of students that were taking college

math by 14 percentage points. And

by the end of their second year, they had

completed more college credits. In later years, we continued

to see that the remediation need went down. But, again, the effects largely

disappear due to the other larger policy changes

that were going on in the state. But regardless of cohort, the

SAILS students did report more positive

experiences in the course. There are a couple of important

caveats of this research. We are only looking at students

coming right out of high school. Those were the ones that

were eligible for SAILS in high school but then went

right on into college courses. So we’re missing students

who may take some time off in between high school and college. We looked only at

community college students only, as this was largely the

population ‑‑ that was largely where

this population was attending. But we also have a pretty

limited timeframe. Two years. The hope is to continue

following students into years three and four to see if there’s

maybe a longer‑term impact of the SAILS program. So,

there were a couple, I think, just final

notes here about this research, and then I’m interested

in the conversation to follow. So, of the additional students

that took college‑level math, only about half

of those students passed the college‑level math course. So while we saw that this

program opened up college math to a new group of students, and

if we believe getting to college math is barrier, an unnecessary

barrier for some students, this would be a success. It did open up that course

to a new group of students. But it does raise the question

as to what are the barriers for students once they are in that

college‑level math course with only about half of the new

students passing that course. What are some of the other

barriers that might be getting in their way? And it also raises some pretty

important questions about how well college‑level math

might be aligned with students’ subsequent

course‑taking or their major. The best

way to assess remedial needs, calling up questions of is the

ACT math the assessment tool, how about considering multiple

measures or assessments when students get to college. That’s a larger

conversation that our study didn’t delve into. But, you know, Tennessee was

largely using the ACT math for assessment. And also, you know, I think

it’s important to consider the long‑term

outcomes of a corequisite model, which a lot of research

recently is attempting to do, so I will be paying

close attention to some of the long‑term impacts that we

see from the corequisite model. And I’ll just conclude by saying

I really think it was clear that leaders in Tennessee were to be

applauded not only for adopting a pretty innovative K‑12‑higher

ed partnership program in college remediation, but really

also for taking the risk to assess its impact. There are really, really

important questions in remedial education out there where we

still don’t have answers as to whether these programs

work, who they work for, and how they work. And so states like Tennessee who

are willing to take the risk to assess one

of their own programs, I think, are really

to be commended. Thank you.>>KATHERINE SHIELDS:

Great. Thank you, Dr. Boatman. If you have any questions for

Dr. Boatman about the Tennessee SAILS program or about the

findings from this evaluation, please post them in the chat. I see some folks have already

put some questions there that we’ll get to at the end. So now we’d like to

take a moment and hear from our audience. So there’s going to be

a poll question on the screen. Just please click to answer. What strategies have you used

in your institution to prepare students for college math? And

you can select all that apply. So some of these might be things

that happen in high school, like

the Tennessee SAILS program, or they might be things like

a summer bridge program between high school and college. And then there’s a lot of

exciting innovations going on at the college level, such

as corequisite or accelerated remedial

classes and other models. So I see

we have some folks chiming in. It looks like a few people have

used high school prerequisite courses, and a few administering

a college placement test during high school. It looks like a handful

of people have also worked with corequisite

college remediation courses. So that’s

great to see. Thank you. So I know we’ve got a good range

of people represented who work at

both schools, districts, states, and institutions of higher

education here in the audience. So, thank you for sharing

what you’re working on now. Great. So I’d like to now turn this

over to Dr. Marla Davis who will be talking with us about her

experience as a practitioner. And Marla, you can take it away.>>MARLA DAVIS: Good afternoon. Again, Marla Davis from

the Mississippi Department of Education. And I’ll be sharing with you all

some information as it relates to our ‑‑ we call

them transitional courses in the state of Mississippi. We have actually tried to move

away from using the connotation of remediation as it relates to

supporting our students that are struggling below grade

level and those that are having deficiencies in

math and also in literacy ELA. And so what we’ve actually

put in place in the state of Mississippi is a continuum

to support our students that are transitioning from

middle school into high school, and also for those students

that are transitioning from high school into college that we have

deemed and identified as having very minor to some of

the most profound deficiencies, if you will, as it relates to

math and ELA in preparation of being college and career‑ready. And so what I’ll do is

I’ll spend a little bit of time talking to you all a little bit

about our Ready for High School Math and Literacy courses. We’ve actually begun

implementing these courses at the ‑‑ excuse me, at the middle

school level going into our second year. So this is our second year of

implementation for our Ready for High School Math and Literacy. At the

onset of offering these courses, we initially wanted to open

these courses up only to 8th grade students. It’s been within the last year

that we’ve decided to actually open these courses up also for

9th grade students that may not be ready

for English I or Algebra I. And so our Ready for High School

Math and Literacy courses are now open for any student

that is in grades eight or nine. We’ve also put some parameters

in place as it relates to our teacher training. So during the teacher training,

which is a three‑day recommended training for any teacher in the

state of Mississippi that wants to ‑‑ that is being ‑‑

that is offering these courses, these particular course

trainings allow our teachers to actually spend time learning

about what are the instructional practices that are needed for

our Ready for High School Math and Literacy Courses. And the reason that these

particular course trainings are so long is because we want to

make sure that our teachers have the instructional strategies

they need, for example, in the math course to dig deep

into what we’ve identified as the biggest areas of weakness

that many of our students are coming into high school with. And so our middle school course,

the 8th and 9th grade course, excuse me, for math spent

some time looking at operations with fractions. There’s

some time for the number system, ratio and proportional

relationships, expressions, equations, and inequalities,

and then, of course, functions

and linear relationships. As a result of reviewing

our data at the state level, we were fully aware

that our students were having difficulties in these main areas

as they transition from middle school into high school. As it relates to our Ready

for High School Literacy course, this particular course allows

our teachers to spend time identifying lessons and

very rich tasks that allow our students to complete

authentic written assignments. And then also, how do they

begin to explore a variety of complex text. So there is some time in the

training where our students –- excuse me, our teachers are able

to begin developing lesson plans around our

Ready for High School courses. For our students that are not

ready to enter credit‑bearing courses at the

high school level ‑‑ excuse me, at the college level, we have

implemented two separate courses at the 12th grade. And so we’re one of a host of

states that offer the SREB math ready and literacy ready

courses for 12th‑grade students. Now, for this particular course,

our students must be classified as 12th grade students or they

could be an 11th grader that has also received a number of credit

hours possible for them to be classified as 12th graders. But this particular course has

an ACT requirement in place for our students. And so over the last few years,

the state of Mississippi has mandated that all students

at the junior year take the ACT. And so all schools and districts

are required to begin reviewing their ACT data for any

student that has an ACT English sub-score

or math sub-score below 15. Any student at the 12th‑grade

level that has that particular score range on the ACT is

required to take the SREB Math Ready

and Literacy Ready courses. This particular

course, because of the rigor, also has a three‑day teacher

training implemented as well. And so any teacher that

is assigned the SREB course must attend a three‑day training and

earn an additional endorsement. Those 900-level endorsements

are a requirement that have also been placed in our

state board policies and our accreditation standards. We also have an Essentials

for College Math and Literacy courses that are fashioned

a little bit –- they’re very comparable to the SREB course,

but the difference actually is the enrollment piece

as it relates to our students. So students that have

an ACT sub‑score 15 or above are required to take the Essentials

for College Math or College Literacy course. When I say 15 or above, we’re

thinking any students between the score ranges of 15 to 18. Any student that

has those particular sub‑scores, as I’ve

already previously mentioned, are required to take

the SREB and Essentials courses. And so I wanted to talk to

you all a little bit around our implementation and training as

it relates to the three courses that I’ve just mentioned, the

three courses for math and the three courses for literacy. So, any teacher in the

state that wants to teach these courses ‑‑ excuse me, any person

that wants to offer training around these courses must

receive a master credentialing where they are identified

as a master teacher training. Those particular certifications

are typically offered outside of the state by national experts. And that particular training is

usually around three days to one week long. We also require that any

facilitator or trainer for these courses participate in a series

of classroom observations. Those classroom observations

are done in classrooms of other master teacher trainers. And we also require that

they spend some time looking at teacher data as it relates

to their implementation of the course and how they

actually work with students. And then finally, our

implementation and training also require that we spend

a considerable amount of time looking at the

feedback after each one of our training sessions. And so we’re looking at

the types of questions that our trainers

are asking during the training, in addition to the types

of questions our teachers are asking. We spend some time also looking

at the types of questions that may arise as a result of the

types of texts that are being used in the training, and

also the types of tasks that are being engaged

in throughout the training. And so implementation

and training in the state of Mississippi is very robust. We don’t take it lightly because

we’re also thinking about the caliber of students that

are going to be enrolled in these courses. And

so our master teacher training, classroom observations,

and feedback are actually on a continuous cycle throughout

the entire implementation year. So, one of the things that I’ve

mentioned to you all is that we have an actual state

board policy in place around our transitional courses. Again, we try to move

away from the word remediation, but we have an actual

state board adopted policy as it relates

to our transitional courses. So beginning with the 2017‑2018

school year, any LEA ‑‑ again, our high schools that offer the

Essentials course ‑‑ must adhere to the following requirements. The teachers must have above

the 7 through 12 math or ELA endorsement, earning a separate

credentialing at the 900 level. In addition to that, we’ve also

put in place that any school or district that is in violation of

these particular series of six requirements will result

in an accreditation violation of accountability

standards 2 and 26. And we just feel that given the

caliber of students that these courses are intended to support,

it was very important for us to put a policy in place

around what was required for the teachers, what’s

required for our students. And then we also have a policy

in partnership with our IHLs. One of the things

that was really important for implementation in the state

is that not only did our IHLs ‑‑ some states call them IHEs,

Institutions of Higher Education — but in addition to offering

the training for teachers, it was very important that we

had a partnership with our IHLs where they also sit

in on the three‑day training. This was one of

the requirements, if you will, that we put in place with our

IHLs so that they would be able to see the rigor that’s

involved in each of these transitional courses. And as a result of IHL faculty,

from the math, and the English, and/or reading

departments at each of our IHLs, sitting in on this training, we

were able to craft a policy in partnership with them

around our remedial courses, our transitional courses

for our 12th grade students. And so what you see here on your

screen is an excerpt from policy 608 which basically indicates

that any student that takes our transition courses

for 12th graders, that, again, you have not pretty much met the

benchmark for ACT and college readiness, as long as that

student completes our Essentials course with an 80 or above, they

will not be required to take remedial courses at

the collegiate level for college algebra and English composition. This particular policy is in

place in all eight of our public universities

in the state of Mississippi. Really quickly,

before my time runs out, I wanted to share with you all

the impact that these courses have had on our 12th

grade performance on the ACT. So what you will see here

is a breakout of data for our Literacy Ready courses

and our Math Ready courses. This is actually combined data

from our SREB course with our Essentials course. And so what I would like for you

all to kind of see is that we’ve made tremendous gains for those

students that have enrolled in our SREB and/or Essentials

courses at the 12th grade. Our

biggest gain, as you can see, has been right here in

science with a typical student’s improvement score on the science

section of the ACT has increased 3.2 points,

and that’s a tremendous gain. Again, all of our ACTs ‑‑ our

juniors are required to take the ACT at the junior year. And then we also see some gains

as it relates to students that were enrolled in the Literacy

Ready course and then retook the ACT

post‑enrollment in our courses. We’re very excited to see this

uptick in improvement at the high school level for students

that are enrolling in remedial and transitional courses

prior to entering college. Overall,

we have seen a tremendous gain, not only as a result of

our essentials and SREB courses, but we’ve seen tremendous gains

as it relates to ACT achievement overall

in the state of Mississippi. Our 11th graders are meeting all

four ACT benchmarks remaining at 9%. Our grade 11 ACT composite score

has increased across the entire state from 17.6 in the year

2015 to 17.8 in the year 2018. And then we have held pretty

steady for the class of 2018 composite scores, while the

national average dropped a small percentage in the year 2017. And so we just thought it was

very important as a practitioner to share with you all the

steps that we’ve put in place to ensure a continuum of support

from our students that are transitioning from middle school

to high school and to college. And one piece that I would love

to definitely bring out before my time runs out is the fact

that we spent a lot of time at the state department

working with classroom teachers, working with practitioners,

working with consultants, and also working with

IHL to ensure that the readiness courses provide a beautiful

alignment with our college and career readiness standards. And so any student that

has enrolled in our remedial transitional courses

does not see a break in content. And so we believe that these are

some of the practices that have led to the improved achievement

that we began to see over the last few years

in the state of Mississippi. Of course, our overall

goal being to ensure that our students are college and career

ready and that they have the skills that are

necessary to perform outside of the classroom.>>KATHERINE SHIELDS: Thank

you so much, Dr. Davis, for that presentation. We’d now like to

take on some of these questions. Thank you for all the questions

that have come in on the chat. I’ll start with this one about

the Tennessee SAILS program for Dr. Boatman. Did the treatment group include

all the students in the SAILS program, or only those students

who were successful in passing the course?>>ANGELA BOATMAN: This

is a really good question. We included all students

who took the SAILS course, but we did not restrict it to

just the students who passed the course. We did look at the

pass rates of the SAILS program, and they’re actually

very high across the state. There is some

differentiation by high school. But across the state, it’s

upwards of 90% of students that enroll in the

course pass the SAILS course. So the estimates

wouldn’t be that different, but we did not restrict it to

the students that just passed. We ran some descriptive numbers

for the state just to provide the state with some data about

the differences between the students

who passed versus did not pass. The state was obviously

very interested in that. But for our technical analysis

we included students regardless of if they

completed the course or not.>>KATHERINE SHIELDS: Thank you. So, for Dr. Davis about

the program in Mississippi, a couple of people have asked

did students get a credit for the remedial math course? Did it count toward graduation? Did

they get a high school credit?>>MARLA DAVIS: I

just want a point of clarity. Are we speaking about our 8th

and 9th grade course or our 12th grade? Let me

give a little point of clarity. Our grade eight and 9th grade

Ready for High School course does offer a math and

an ELA reading Carnegie unit. The answer to that is yes. Those particular credits are

considered lower than Algebra I and lower than English I. So any student that earns those

Carnegie units is still required to earn three additional

maths or three additional English credits. For the 12th grade course,

they do offer a Carnegie unit as well, and those are considered

higher than Algebra and higher than English II. For some students, it does serve

as their fourth‑year math or fourth-year English depending

on how they earn their credits. So both

courses do offer Carnegie units. Great question.>>KATHERINE SHIELDS: Thank you. So looking back at

the SAILS program in Tennessee, Dr. Boatman, are you collecting

data on which college math course

students took following SAILS?>>ANGELA BOATMAN: Yes.

We worked closely with the folks within the colleges to

identify the four most popular math courses, I believe, the three

to four most common math courses that students enrolled

in for college‑level math. We have not yet linked students’

remediation requirements with any particulars regarding their

proposed field of study and how that impacts

the math courses that they take. This would be a next step of

this work is to think about the individual math course and

how that impacts the student’s trajectory in either a STEM

field or progress through other types of degrees. But, yes, we have broken out the

math courses by the different course options

that students might take. I don’t believe we report

on this in the full report, but it is something that we have

looked at at the request of the state as well. They were interested in

differences by different types of courses.>>KATHERINE SHIELDS: Great.

So now another question, Dr. Davis, for

you about Mississippi’s program. So, to clarify,

do students take the ACT twice?>>MARLA DAVIS: No. Students

are only required by state board policy to take it as juniors. The students that are in

the study that I shared in the presentation were offered

an opportunity to retest after taking the state‑mandated

ACT test as juniors. And that opportunity to retest

was actually ‑‑ we called them pilot sites, of course, because

we were looking at study data. Any of those schools that opted

in to have their students retest after taking our 12th grade

readiness courses were provided a voucher to retest.>>KATHERINE SHIELDS: And,

Dr. Davis, one other clarifying question about the model. Someone asked who

is training the master trainer? That was a reference

to somebody outside the state.>>MARLA DAVIS:

Yes. Really great question. Any teacher that’s receiving

the master trainer credentialing actually does go through

training with the Southern Region Educational Board.>>KATHERINE SHIELDS:

Great. Thanks. So, Dr. Boatman, there’s a question

about the SAILS program. Is there any sunsetting of

the SAILS student’s eligibility in college?>>ANGELA BOATMAN: I’m assuming

that just means that if a student doesn’t enroll

in college immediately or takes some time in taking their math

requirement, does that sunset? I guess that’s how

I’m interpreting that question. And the answer is that a student

who completes the SAILS program successfully in high school is

supposed to be exempt from the remediation requirement

regardless of when that is. So, to my knowledge, that would

stay with the student throughout different semesters, unless I

am not understanding the use of sunsetting there.>>KATHERINE

SHIELDS: Great. Thank you. And another question for

you about more the study design. Was there a comparison

group for the improvements in pre‑ to post‑tests?>>ANGELA BOATMAN: So our

assessment of learning gains was looking at students

above and below the cutoff. So we ‑‑

and we did not see improvement. That might make me

wonder if that’s a question for Mississippi. We did not observe

improvements at that cutoff. But, again, I would offer the

clarification that we were just able ‑‑ given that study design,

we were only able to look at students with a point or two

to the left of the cutoff and a point or two to the right. So that’s a pretty narrow band

of students that were eligible for the SAILS program. And so we’re not able to talk

about learning gains that may or may not have

existed below that ACT cutoff. And I think that’s

an area for future research, especially given that we saw

some of the positive impacts of SAILS were being captured

by the lower‑scoring students. But just given the study

methodology and the way that we could isolate that cutoff, we

weren’t able to draw conclusions about students

further below that cutoff.>>KATHERINE SHIELDS: I

see we’ve got a clarification. That was actually a question

intended for Mississippi. So, Dr. Davis, for those pre‑ to

post‑test improvements that you shared

from Mississippi students, was there

a comparison group for that?>>MARLA DAVIS: Okay. I’m sorry. I just

heard the end of that question. Was there a comparison group?>>KATHERINE SHIELDS: When you

showed the improvements in pre‑ to post‑test, was there a

comparison group for that study?>>MARLA DAVIS: No, this

was actually one of the first studies

of this type for the state, so

the answer is not at this point. But we do have plans to actually

duplicate this study with a larger population

of students in the state.>>KATHERINE

SHIELDS: Great. Thank you. So this is a question

about the SAILS program. And I believe

this may have been answered. But

was there a teacher in the room, or was it entirely students

working self‑paced on computers?>>ANGELA BOATMAN: Yeah. No.

There is always a teacher in the room. So the teacher is — in a lot

of the SAILS classrooms that I visited, this teacher

is usually roaming the room, speaking

with individual students, answering questions

in a more one‑on‑one basis. But it is taking place

during a typical class session, and so

the teacher is in the classroom, getting the class started,

adjourning it when it’s over. It’s just more the sort of

middle portion of the class is where the teacher sort of takes

a backseat and the computer is largely

providing the instruction.>>KATHERINE SHIELDS: Thanks.

We had another question about that program. Who wrote the online math SAILS

program and how are students selected for it?>>ANGELA BOATMAN:

Yeah. That’s a great question. It was originally conceived by

some members of the math faculty at

one of the community colleges, Chattanooga State

Community College in Tennessee, who partnered with Pearson to

development the course itself. So the software was

a Pearson product, My Math Lab. And they began piloting that. That was what was being

used in the community college. And so then the

SAILS pilot came out of that, where they used that same

software to then move that into the high school. And

the program since has, you know, evaluated the use of Pearson and

thought about different models. But when it

originally was developed, it was a Pearson product.>>KATHERINE SHIELDS: So,

Dr. Davis, in looking at Mississippi again,

someone is asking is any work being done in the state

on deemphasizing algebra at the college level and

replacing it with discrete math or statistics?>>MARLA DAVIS: We’re not

actually engaged in any conversations along those lines. Not that that’s

not a priority conversation. It’s just not one

that we’re having at the moment. But we actually had ‑‑ in our ‑‑

we actually have standalone discrete math and standalone

statistics courses that we actually absorbed into our

new framework across grades ten, eleven, and twelve. And so I don’t know if

that kind of gets to the answer, but I just would probably

say at this moment in time, it’s not one

that we’ve had at this moment.>>KATHERINE SHIELDS:

And another question for you for Mississippi. What does

the teacher training look like? Does it focus on pedagogy,

instructional strategies, or content?>>MARLA DAVIS: It actually ‑‑

all of our trainings actually deal with both. And so the very, very first day

of the three‑day training spends a considerable amount of time on

the first half of day one really understanding the depth of

knowledge of the teachers that are actually at the training. The second part of day

one spends a lot of time talking about how do you ensure

the depth of knowledge is at the level it needs to be for the

teachers that are going to be going

into day two and day three. Day two and day three actually

spend a lot of time around instructional strategies. It allows the teachers

the actual engagement with the activities and the tasks that

are going to be put in front of their students. And then the latter part of day

three is actually something that we probably

would coin as a demonstration, where we place our ‑‑ a lot

of states do something similar, but we actually allow

our teachers to get into small groups and do a demonstration of an actual lesson. That’s an opportunity for the

facilitators to provide feedback around how to differentiate

instruction for the caliber of student that’s in these courses. While many of our students

in these courses will have a similar ACT sub‑score, we

provide some tips on how do you differentiate instruction

around particular deficiencies. And so I hope that kind of

answers the question around how we do our three‑day trainings.>>KATHERINE SHIELDS:

Going back to Tennessee, there’s a question asking

for clarification, I believe, about the findings

you were discussing previously, Dr. Boatman. He’s asking was the 50% pass

rate for the students just below the cutoff –- that passed the

test — different than the pass rate for those comparison

students just above the cut score in terms of

pass rate on the SAILS course?>>ANGELA BOATMAN: Right. So the 50% pass rate was largely

related to the students below the cutoff. So it’s sort of saying of

the SAILS program did bring more students

into college‑level math. So with more students

entering college‑level math, there was an increase in overall

numbers of students passing college‑level math because

there were more students in college‑level math. But that of the students that

were brought in as a result of the SAILS program, only about

half of those students that were brought in as a result of

the program passed the course. So it’s related to the

SAILS‑eligible students that are below the cutoff.>>KATHERINE SHIELDS:

Okay, great. Thank you. And then this was a question

for, again, for Tennessee. What is a typical student if the

change in the average scores is 1.8 points? I don’t know if we need

some clarification maybe on that question.>>ANGELA BOATMAN: Also,

I wonder if that question might also

be for Mississippi, could it be?>>KATHERINE SHIELDS: Oh,

sorry. Yeah. It sounds like it. Maybe you can clarify on that. Dr. Davis, does that

sound like that’s one for you? What is a typical student, given

where your average score was and the change in

average scores was 1.8 points?>>MARLA DAVIS: And so

I’m trying to –- I’m trying to understand

the question one more time.>>KATHERINE SHIELDS: Why don’t

we come back to that one while Anita types that back in. I think she

was typing up a clarification. Okay. Here we go. In looking at the Mississippi

scores on the ACT scores, there’s a column that

says change for typical student, but there’s another one

that refers to the change in average score. I think she’s trying to

understand what does that next column indicate.>>MARLA DAVIS:

Gotcha. Gotcha. Gotcha. So what we did is we

did an actual aggregate where we evaluated our scores

across the entire subgroup, and then we actually

took an actual approach where we identified the average

improvement just across each one of the groups. And so there is some extra data

that was not included in the slide that we shared with you

all that looks at subgroup data across gender, across race,

across socioeconomic status that’s not included, and that’s

where that data comes from. And I’m sorry.>>KATHERINE

SHIELDS: Okay, thank you.>>MARLA DAVIS: We didn’t

include it in the data point on the slide. Great question.>>KATHERINE

SHIELDS: Okay, thank you. These were all great questions. So at the end of this webinar,

your browser is going to redirect you to a survey. We really encourage you to just

take a minute and send us some feedback so that we

can keep improving our webinars. I want to thank Dr. Angela

Boatman and Dr. Marla Davis so much for sharing

their insights and experience. As a reminder, you can download

the Tennessee SAILS evaluation report as well as today’s

slides in the download box. There are also two reports there

from our colleagues at other regional educational

laboratories about remediation work done in Oregon and Florida. So, thank you very much. I’ll turn it over to Shanna.>>SHANNA RUSS: Thank you all

so much for joining us today. Today’s webinar recording

will be archived and uploaded to IES’s YouTube channel

as well as to our website. If you

registered for today’s webinar, you’ll receive an email from

us with a link to the recording once it’s ready. Additionally, don’t forget

to download any of those files because it will close. And

have a great rest of your day. Thank you all so much.

## Recent Comments