<< One Academic Writer's Perspective on the Emetic Prose of his Shallow, Supercilious & Unblocked Peers | Front Page | Guest Post: An MFA Student Ponders His Navel and Life After the Degree >>
Sunday, June 05, 2005
Getting Ranked
The Research Assessment Exercise, which ranks departments and universities by the quantity of their publications, has had an unhappy effect on British academics. When one’s own paycheck depends on the number of publications extruded by one’s department, the colleague who publishes three mediocre monographs in five years is a hero; the one who publishes one great book every ten years is not pulling her weight. I have noticed that British academics often prefer to publish a collection of articles as a ninety-dollar book rather than as a special issue of a journal with a wide circulation. Why? A book counts for more in the RAE.
One major public university in the U.S. has a slightly more nuanced system of evaluation. Each journal is assigned a numerical rank based on its quality and prestige. Raritan might be a four and Postmodern Culture a 3. Of course, members of the department do the ranking. And how much does the name of the journal ever tell you about the quality of the article? “Bring out number, weight, and measure,” Blake said, “in a year of dearth.”
At the same time, there is nothing especially admirable about a department where no one publishes anything. Some professors say that they don’t publish much because they invest more in their pedagogy, but in my experience the most active scholars are often also the most inventive and committed teachers. And it is understandable that administrations, legislatures, and prospective students would want some tools for evaluating departments and comparing them to each other. As distasteful as it may be to think about assessment methodologies, I would rather design a rational one than have a stupid one imposed on me.
What, then, should those methodologies be? One possibility is that they could be based on peer review. Most departments are already reviewed every five years by external evaluators; perhaps departments could make portions of these documents publicly available. Then, instead of looking at a virtually meaningless numerical ranking in the style of U.S News and World Report, we could see thoughtful descriptions of a department’s strengths and weaknesses. Departments could also offer a self-evaluation. Such a document might attempt to conceal weaknesses, but it would probably offer a reasonably trustworthy map of a department’s most noteworthy accomplishments.
I am tempted to elevate this suggestion to a general principle. If we are all being sorted, ranked, and evaluated with increasing frequency, perhaps the best response is not to stonewall but to offer richer information about ourselves.
Take, for example, the writing component of the new SAT exams. Rather than assigning numerical ratings to student essays, it is infinitely preferable to transmit the essays themselves to college admissions departments. Then, instead of relying on ETS graders, college admissions officers can make up their own minds about the quality of a student’s writing.
Comments
I want to hear your rankings of departments, journals, all of it. We’re going to duplicate the Leiter Report right here and now.
Jonathan, I find it hard to understand the target of your sarcasm. I am not advocating a numerical ranking of departments and journals in the manner of Leiter. I am pointing out that academic institutions are being pressured into performing more and more assessment and evaluation.
In the last year, I have spent more time than I would like trying to get the NCTE to accredit my department to train teachers. I just wrote a small chunk of the self-study for the college’s accreditation review. I have been asked to sit on a task force for the development of a rubric for evaluating term papers from our composition classes. I was asked to speak on a panel on outcomes assessment. I am going to a conference on outcomes assessment later this week. My department was asked by a CUNY dean to provide a tally of our books, articles, grants, and editorial commitments for the previous year. Meanwhile, the CUNY system as a whole has just added a set of critical thinking exams that students take after their first sixty credits of coursework. If they don’t pass, they don’t get to move on to their junior years. The assessments are descending on us like locusts. Since academic workers cannot simply opt out, the question is whether we passively submit. If we must participate, I think we should try to design the best systems we can. It is our moral obligation to do this for our students, and it is prudent for us to do this for ourselves when someone insists on evaluating us. Would you disagree with this argument? If so, why? If not, do you have any ideas about how to do smarter evaluations of students, teachers, scholars, departments, or colleges?
I wasn’t being sarcastic at all. I’ve long advocated a Leiter-style ranking of English departments. I say we fire it up, right here and now.
Some professors say that they don’t publish much because they invest more in their pedagogy, but in my experience the most active scholars are often also the most inventive and committed teachers.
Mmm, not in my experience. If you want to have some sound basis for evaluating performance, you’re going to have to do better than that.
Departments could also offer a self-evaluation. Such a document might attempt to conceal weaknesses, but it would probably offer a reasonably trustworthy map of a department’s most noteworthy accomplishments.
Wow! Do you really think a department is going to risk its reputation on an honest self-evaluation? No way! It would be advertisement.
As I thought I had made clear, I agree with you on both points, blah. If one wants to evaluate a department’s teaching, one needs things like teaching evaluations and possibly exit interviews with graduating students. I was just saying that when a professor produces no scholarship, a commitment to teaching is not a sufficient excuse. The reason why college professors spend less time in the classroom than high school teachers is that they are expected to do original research. And I think that professors who are involved in research do teach a different set of skills, even in entry-level courses.
A self-evaluation can sometimes be surprisingly honest. Have you ever asked students what grades they think they deserve? Some are too lenient but many are way too harsh. But the point I was making in my post was that a self-evaluation would give a fairly accurate picture of a department’s major successes: Professor A got a Guggenheim, Professor B now writes for both Harper’s and the New Yorker, and Professor C just published an article in PMLA. Right now, it can be surprisingly hard to extract such information from a department’s web site. You usually have to go through individual faculty biographies, if they are even up to date. When I was applying to graduate school, before the advent of the web, it was even harder to determine what a department’s strengths were. I had to go to the library and look at microfiche catalogues just to figure out who the faculty in a department were, then look them up in the MLA Bibliography, which was on a CD-ROM in a separate room, and then hunt down books and articles.
Is a 4/4 load less time in the classroom than a high school teacher? It may be, but I was thinking they didn’t teach quite that much on average.
A 4/4 load usually is 12 hours of classroom time. High school teachers usually spend at least four hours a day in the classroom, which is 20 hours. And colleges with 4/4 loads, if they have any interest in research at all, usually give course releases to productive scholars so that their actual loads are more like 3/3.
My colleagues in the music department do over twenty hours a week of teaching, if one counts their one-on-one lessons as well as their classes. They are known as fabulous teachers. And they somehow find time to write books and articles, memorize long, complex works and perform them, and serve on numerous committees. Yikes!
"And colleges with 4/4 loads, if they have any interest in research at all, usually give course releases to productive scholars so that their actual loads are more like 3/3.”
Are you absolutely sure about that?
There are many sad exceptions, but administrators at schools with heavy course-loads do seem to have the ability to lighten the burdens of at least a minority of their faculty. I know many schools where almost no one teaches the official course-load: even a tiny amount of research gets one the course release. Administrators can give course releases for research or for administrative responsibilities, some of which might not be very demanding. Sometimes during the hiring process they are not allowed to spell this out.
A lot of my time, and that of my colleagues, is already spent on such assessment activities. Whether I am reading an article for a journal whose editorial board I belong to, evaluating a book ms. for a university press, evaluating a candidate for tenure at another university (or my own), reviewing grant proposals for the NEH, serving on a commitee that reviews graduate programs within the university, evaluating my own colleagues on the merit commmittee, preparing my own documents to be evaluated by others. When I’m not doing that, I’m grading student papers or exams or evaluating job candidates for positions in my department. So much time spent passing judgment: that’s the academic life. I’m not complaining. All these judgments need to be made, after all. I’m just not sure we need more of it than we already have. It would be nice to have more time to teach and do research.