Rolla = Cambridge in new college rankings that try to show how schools add value | St. Louis Public Radio

Rolla = Cambridge in new college rankings that try to show how schools add value

Apr 29, 2015

No one has ever mistaken Rolla, Mo., for Cambridge, Mass. But new college rankings place the schools in both towns on just about the same level.

The report from a unit of the Brookings Institution in Washington, D.C., is an attempt to determine how well colleges prepare students for high-paying careers.

It’s designed to go beyond the rankings published in places like U.S. News and Forbes by using a so-called “value added” approach that determines not just how well graduates will do in their careers but how much their college experience contributed to that success.

If the rankings are any indication, students who graduate from Missouri University of Science and Technology, the Rolla campus of the University of Missouri system, are getting added value on a par with students at Harvard or MIT.

Missouri colleges do well in the Brookings Institution rankings on value-added schools.
Credit j.o.h.n. walker | Flickr

The rankings consist of three key numbers, on a 1-100 scale, measuring three components of the career of a typical graduate: mid-career earnings, occupational earnings power and loan repayment rate.

Based on these categories, MIT was at the top of the pack, with scores of 100, 99 and 99. But Missouri S&T was practically on the same level, with scores of 98, 100 and 96. Harvard came in at 97, 84 and 88.

Other schools at the top of the list included Caltech; Colgate University in New York; Carleton College in Minnesota; and Rose-Hulman Institute in Indiana.

Among schools in the St. Louis area, Washington University had the highest scores, with 83, 83 and 72. Fontbonne University trailed the others with scores of 3, 29 and 16.

_

The authors of the report, titled “Beyond College Rankings: A Value-Added Approach to Assessing Two and FourYear Schools,” stressed that it is designed to provide information that other ratings do not take into account. Ideally, they said, prospective college students will be able to use the new information to help them decide where they might want to enroll.

“College is a major investment for individuals and the taxpayers who subsidize it,” Brookings Fellow Jonathan Rothwell said in a statement. “So, the public has a huge stake in promoting quality. No ratings system can capture everything about a college that matters, but these data can shed some light on how colleges compare in their contributions to student success and, hopefully, spark further research with even better data.” 

He added, “These college-specific data can be used to learn about, evaluate, and improve college performance. Colleges serve very diverse populations. The advantage of measuring value-added is that it adjusts a school’s rankings based on the type of college and the characteristics of its student body.”

Five factors

In Rolla, the news of the good showing by Missouri S&T was welcome.

“The term value added really isn’t one that we use," said spokesman Andrew Careaga. "But it seems to resonate well. The idea that Missouri S&T education pays off — pays great dividends in the future — is one that resonates well with prospective students and their families.”

Careaga noted that campuses that stress so-called STEM disciplines — science, technology, engineering and math — seemed to do well in the Brookings rankings. But he said that S&T also has good programs in business and the humanities. Still, he added, the school's history, back to its beginning as the Rolla School of Mines and Metalurgy, has focused on engineering.

"Our focus has been as a technological research university for years," Careaga said. "But the name Missouri S&T, with that emphasis on science and technology, was really designed to help us with our recruitment and to help prospective students identify and know the type of college that we are."

He said the school will be able to use the new rankings as part of its recruitment strategy, which also emphasizes opportunities like an average starting salary for graduates of $61,000 a year.

"Rankings provide a great third-party perspective on universities and the value that they bring," he said.

In an interview Wednesday, Rothwell called Rolla's ranking "spectacular" and just the kind of track record that can help graduates stand out in the job market.

“That sort of knowledge is usually rewarded in the labor market," he said of the STEM disciplines. "The students who come out with engineering or computer software skills are able to find jobs. There's a shortage of workers around the country in those sorts of positions.”

He noted that some liberal arts schools such as Carleton also did well in the new study, and he plans further research to figure out why.

In addition to being useful for students and their families in determining the best schools to attend, Rothwell said the new rankings also can have a broader impact on the communities where colleges are located.

"What we're finding os that a large percentage of graduates stay in their region to work after they finish school," he said. "And if they're earning higher salaries they're paying more revenue to the local tax base, they're spending more on housing and local goods and services. That drives local GDP.

"So once state and local leaders start thinking about the quality of schools as having direct implications for economic prosperity, I'm optimistic there's going to be greater effort to work on solutions, making sure that schools are doing as good a job as they can for their students."

On its relatively low ranking, Fontbonne released this statement via email:

"Without having access to the numbers and full methodologies of the recent Brookings report, it’s difficult to assess its accuracy. We were not contacted about this study, nor did we contribute to it in any way. At Fontbonne University, we have always been committed to providing value to students. We define value not just in terms of salary, but by our graduates’ ability to think critically, act ethically, serve their communities and make a positive impact on society — as well as their ability to bring home a paycheck and contribute to the greater economy. 

"Many of our degrees, such as education, social work, deaf education, special education and others, are important to the fabric of a community, but are not going to weigh heavily in a career earnings assessment. Developing well-rounded individuals, able to make both a living and a life, is our ultimate goal. While this study provides interesting numbers that we will continue to assess, we understand that it speaks to only a piece of what it means to be valuable as an institution of higher education."

The Brookings report accompanying the data pointed to five factors that Brookings said “are strongly associated with more successful economic outcomes for alumni in terms of salary, occupational earnings power and loan repayment:

  •  Curriculum value: The amount earned by people in the workforce who hold degrees in a field of study offered by the college, averaged across all the degrees the college awards
  • Alumni skills: The average labor market value, as determined by job openings, of skills listed on alumni resumes
  • STEM orientation: The share of graduates prepared to work in STEM occupations 
  • Completion rates: The percentage of students finishing their award within at least twice the normal time (four years for a two-year college, eight years for a four-year college)
  • Student aid: The average level of financial support given to students by the institution itself.

Brookings explained that a low score in the rankings does not necessarily mean that a student will do poorly after graduation. The scores are designed only to indicate how much a school contributed to a student’s performance beyond what the student would have been expected to do otherwise.

It said that such a system can help students and their parents determine whether the cost of attending any particular college is a good investment, by showing whether the money spent will result in improved performance expected after graduation.

That kind of data, Brookings said, can be more helpful than typical rankings of schools that don’t look at value-added expectations. Those other rankings, based only on the performance of graduates and not compared with how they would have done no matter what college they attended, fail to taken into account the actual school experience.

And, Brookings added, many colleges rank highly elsewhere mostly because they admit only students who are expected to do well in any situation, so the contribution that the school makes could play a minimal role in that success.

“Highly selective research universities admit only the most highly prepared students as measured by high school grades and admission test scores, while many two-year colleges have open admissions policies, accepting students who struggled to finish high school and have very low test scores." a fact sheet  released by the institution stated.

“Because the most prepared students tend to earn higher salaries than the least prepared students, evaluations of college quality should consider student characteristics and adjust predicted outcomes and final ratings accordingly.”

The three measures that the study included – salary, earning power and loan repayment – were included for three main reasons, Brookings said:

  • They are important to individual and collective well-being
  • They can be measured with precision
  • They are available for a large number of colleges

“Of course,” Brookings added, “there are other economic outcomes that individuals and elected officials care about, such as the prospects for becoming a great leader, or accomplished artist, scientist, or entrepreneur. All of these are extremely difficult to measure, and it would be easy to mistakenly attribute rare individual accomplishments (like Academy Awards or Nobel prizes) to the institutions they happened to attend.”

And, the institution said, other measures beyond those it used are valid ways of gauging a college, but they are more difficult to quantify.

“Educators may be more interested in how well the students acquire knowledge,” it said. “Presently, however, there are no reliable post-alumni exams administered at a scale wide enough to assess individual colleges, and it would be very difficult to determine what alumni should be tested in or expected to know.…

“Others may prefer to know how alumni contribute to social justice or their likelihood of living good lives. In principle, the method used here could be applied to such outcomes, if they could be measured, but a great many practical limitations make that unlikely.”

How can the new rankings be used, besides by future college students and their families? Brookings has two suggestions:

“College administrators and trustees could use these data to evaluate their institution’s broad strengths and weaknesses so as to target further investigation and inquiry in to how to best serve their students. In some cases, poor results may be due entirely to an institutional legacy or mission that is largely incommensurate with the graduation of many high-earning alumni. Other schools may find there is more they can do without sacrificing their core mission.

“Public officials could use these data to broadly observe which schools are failing to deliver and which are outperforming their peers. It would be a mistake to allocate public resources (or even private donations) based entirely on econometric results such as these, but these data can provide initial guidance into which schools bear further scrutiny and may lead to targeted support of and new investments in failing schools so they can better serve the public. Likewise, high-performing colleges may offer important lessons as to what institutional-specific programs and initiatives can be replicated elsewhere.”

The data used to measure the value added by the schools came from government sources as well as private sources, including the websites Payscale and LinkedIn.