The centrist think tank Third Way released a report in August that uses the first year of data from the U.S. Department of Education’s new College Scorecard to understand how well four-year public colleges and universities serve low-income students. In particular, What Free Won’t Fix: Too Many Public Colleges are Dropout Factories tried to quantify how effectively those institutions are improving the overall economic circumstances of their students, and it came up with a composite score that Third Way calls the “mobility metric.” This analysis was a follow-up to a similar look at four-year private colleges released last spring.
As one of the report’s authors, Tamara Hiler, senior policy advisor for education at Third Way, points out below, the College Scorecard may be frustratingly incomplete in places but its data provides new opportunities for insight that never existed before.
In this analysis, Third Way looked at a subset of low-income students coming from families making $48,000 or less and evaluated 535 colleges and universities on the following measures:
Net price of college for that group.
Six-year completion rate for first-time/full-time students in that group.
Percentage of students in that group making progress toward loan repayment after three years.
Percentage of students in that group who, six years after enrollment, were making more than the median income of working adults with only a high school diploma.
The analysis also looked at the percentage of all students the school admits who receive Pell grants.
Third Way then assigned weights to each of those factors and ranked the schools, ultimately making the case that students, their families, and taxpayers aren’t getting a strong return on their investment in higher education from many public four-year schools. For example, the higher the concentration of Pell-funded students a university has, the less likely it is to do well on the other mobility measures. For the average school, at six years after enrollment 35% of students were not earning more than the median income for high-school graduates.
The report also highlights several schools that lay outside the mean in their analysis. For example, University of California-Santa Cruz, whose student body is 44.6% Pell funded (versus as an average of 38%) has a six-year graduation rate of 73.5% (versus an average of 48.3%).
Recognizing that many of Third Way’s conclusions and the data it is based on — which is available for download with the report — will be of interest to educators working to extend access and improve educational outcomes, we asked Hiler for the following interview, which has been condensed and edited for clarity.
What were you trying to discover with this analysis?
We noticed, and probably many of your readers noticed, that anybody talking about the “crisis in higher education today” is almost exclusively talking about college costs. When we go to speak to members of Congress, the first question we’re almost inevitably asked is, “What should we be doing about financing?”
It’s undeniable that college cost is an important factor, but we felt like there was this missing conversation about quality. We wanted to refocus the conversation on value.
Do you feel like the College Scorecard is changing the conversation?
Definitely. A student can type in a college and see graduation rates, average amount of debt, net price for students, what percentage of low or moderate income students that school is taking in. When I went to college, the only thing I looked at was the U.S. News and World Report rankings, which focused much more on inputs such as SAT scores.
It’s changing the conversation around data and around transparency. It’s helpful that institutions are saying, “This data is not perfect.” The Higher Education Act bans reporting a student unit record so that we can follow an individual student from school to school. The only thing the federal government can collect is institution level data.
Even institutions that have historically prohibited information from coming forward say, “We need to make sure that, as we’re moving into a new accountability era, the data being used to make some of those determinations is as complete a picture as possible.”
What responses have you gotten?
When we talk about graduation rates being low, people in higher education are not surprised whatsoever. But for the majority of Americans who don’t think about this all the time, there has been a significant amount of shock at how low a lot of these numbers are. We’ve generated a lot of interest at the state and local levels from local press.
We’ve also seen some interesting op-eds from college presidents saying, “We know our graduation rate is not as good as it could be, and here are the things that we are trying as an institution to improve.”
What are your thoughts on some institutions saying this analysis doesn’t account for the different populations that they serve?
We’re not trying to say every college should have the same mission. We are trying to highlight notable exceptions. Demographics is not necessarily destiny, and there are plenty of examples of schools taking in very high percentage of Pell students and getting really strong results for them. At two schools in Chicago, each has 50% Pell Grant students, which is an incredibly high number. One has a graduation rate of 20%, and one has a graduation rate of 60%.
Schools taking in harder to serve populations should be applauded and supported for that. In K-12 we recognize through Title I that schools that take in higher concentrations of kids in poverty need more resources to get the same outcomes. That basic principle doesn’t exist in higher education. But that’s not necessarily an excuse for graduating 20% of students.
There has to be conversation around what minimum floor there should be for putting in place some support plan, the way that we’ve seen in K-12. Right now, we have absolutely nothing. We have schools graduating students in the single digits and we’re still allowing students to take out student loans to attend those institutions. [We should] think about what those supports would look like and make sure we’re incentivizing schools to change or rewarding the ones doing a really good job.
Why do some schools with a large number of Pell funded students get comparatively high graduation rates?
We were looking at a very top level for this report. Our hope next is to do some comparison studies into what these institutions are doing. The anecdotal evidence is that it has a lot to do with the leadership making the commitment to focus, first and foremost, on completion as the driver for everything. A lot of times it ends up being small nudges that make a huge difference in students’ lives.
How much do non-academic aspects of campus contribute to graduation rates?
Some schools have done interesting things around predictive analytics that show which students are set up for success. The University of Nebraska used predictive analytics to identify students [not re-enrolling] and told professors to reach out to those students and say, “Hey, what’s going on?” Most of the time, it was things like “I got a $50 parking ticket I can’t pay, and now I have a hold on my registration.”
Then the professor was actually the one advocating to lift the parking ticket and let the student register. They saw a huge spike in re-enrollment just by empowering leaders and professors to feel the shared responsibility for students coming to school and finishing.
I don’t think anyone is acting in a nefarious way to hurt these kids. But we’re talking about students enrolling with the anticipation that they will graduate with a four-year degree. The worst-case scenario is these students take out loans and then not finish. Often the student defaulting on loans is not the student who has $100,000 plus in debt. It’s usually the student with $10,000 or less, which in most cases means students who didn’t complete college.
Why did you choose the term “mobility metric” when so much of the discussion is about graduation rates?
We are trying to show the percentage of Pell students this institution takes in. For most students, this is going to be your ticket to the middle class. It’s going to be the one way you will be able to have mobility in today’s economy.
Another aspect of the data is that for the first time colleges in 2018 have to report their Pell graduation rate, which was just mind-blowing to me when I came into higher ed. Why weren’t we asking schools to report on how well they do with taxpayer subsidized students?
What’s the thinking about the term “dropout factory”?
It is a loaded term. That was not a popular term for a lot of readers. It’s interesting because in the private nonprofit report we used the term “dropout factory” multiple times but we didn’t have it in the title. I don’t know if that’s what is generating much more interest.
We spent a year working on No Child Left Behind Reauthorization bill. One provision added that any high school graduating fewer than two-thirds of students would get labeled informally as a dropout factory which triggered the district or state to intervene and put in some sort of support plan.
So, as we had the [college] data in front of us, we said, “How many would be considered dropout factories if we had a similar standard? How many are graduating fewer than two-thirds?” That helped ground our thinking. Do we have some sort of bar when it comes to saying this is unacceptable to be allowing students to take out Title IV aid to go to this school?
As I said at the beginning, our main purpose for these reports is really to change the conversation. And I think it helped to generate that, drawing attention to what institutions are doing as a whole around completion given the importance that having a degree means in today’s economy.