Welcome to The Valve
Login
Register


Valve Links

The Front Page
Statement of Purpose

John Holbo - Editor
Scott Eric Kaufman - Editor
Aaron Bady
Adam Roberts
Amardeep Singh
Andrew Seal
Bill Benzon
Daniel Green
Jonathan Goodwin
Joseph Kugelmass
Lawrence LaRiviere White
Marc Bousquet
Matt Greenfield
Miriam Burstein
Ray Davis
Rohan Maitzen
Sean McCann
Guest Authors

Laura Carroll
Mark Bauerlein
Miriam Jones

Past Valve Book Events

cover of the book Theory's Empire

Event Archive

cover of the book The Literary Wittgenstein

Event Archive

cover of the book Graphs, Maps, Trees

Event Archive

cover of the book How Novels Think

Event Archive

cover of the book The Trouble With Diversity

Event Archive

cover of the book What's Liberal About the Liberal Arts?

Event Archive

cover of the book The Novel of Purpose

Event Archive

The Valve - Closed For Renovation

Happy Trails to You

What’s an Encyclopedia These Days?

Encyclopedia Britannica to Shut Down Print Operations

Intimate Enemies: What’s Opera, Doc?

Alphonso Lingis talks of various things, cameras and photos among them

Feynmann, John von Neumann, and Mental Models

Support Michael Sporn’s Film about Edgar Allen Poe

Philosophy, Ontics or Toothpaste for the Mind

Nazi Rules for Regulating Funk ‘n Freedom

The Early History of Modern Computing: A Brief Chronology

Computing Encounters Being, an Addendum

On the Origin of Objects (towards a philosophy of computation)

Symposium on Graeber’s Debt

The Nightmare of Digital Film Preservation

Richard Petti on Occupy Wall Street: America HAS a Ruling Class

Bill Benzon on Whatwhatwhatwhatwhatwhatwhat?

Nick J. on The Valve - Closed For Renovation

Bill Benzon on Encyclopedia Britannica to Shut Down Print Operations

Norma on Encyclopedia Britannica to Shut Down Print Operations

Bill Benzon on What’s an Object, Metaphysically Speaking?

john balwit on What’s an Object, Metaphysically Speaking?

William Ray on That Shakespeare Thing

Bill Benzon on That Shakespeare Thing

William Ray on That Shakespeare Thing

JoseAngel on That Shakespeare Thing

Bill Benzon on Objects and Graeber's Debt

Bill Benzon on A Dirty Dozen Sneaking up on the Apocalypse

JoseAngel on A Dirty Dozen Sneaking up on the Apocalypse

JoseAngel on Objects and Graeber's Debt

Advanced Search

Articles
RSS 1.0 | RSS 2.0 | Atom

Comments
RSS 1.0 | RSS 2.0 | Atom

XHTML | CSS

Powered by Expression Engine
Logo by John Holbo

Creative Commons Licence
This work is licensed under a Creative Commons License.

 


Blogroll

2blowhards
About Last Night
Academic Splat
Acephalous
Amardeep Singh
Beatrice
Bemsha Swing
Bitch. Ph.D.
Blogenspiel
Blogging the Renaissance
Bookslut
Booksquare
Butterflies & Wheels
Cahiers de Corey
Category D
Charlotte Street
Cheeky Prof
Chekhov’s Mistress
Chrononautic Log
Cliopatria
Cogito, ergo Zoom
Collected Miscellany
Completely Futile
Confessions of an Idiosyncratic Mind
Conversational Reading
Critical Mass
Crooked Timber
Culture Cat
Culture Industry
CultureSpace
Early Modern Notes
Easily Distracted
fait accompi
Fernham
Ferule & Fescue
Ftrain
GalleyCat
Ghost in the Wire
Giornale Nuovo
God of the Machine
Golden Rule Jones
Grumpy Old Bookman
Ideas of Imperfection
Idiocentrism
Idiotprogrammer
if:book
In Favor of Thinking
In Medias Res
Inside Higher Ed
jane dark’s sugarhigh!
John & Belle Have A Blog
John Crowley
Jonathan Goodwin
Kathryn Cramer
Kitabkhana
Languagehat
Languor Management
Light Reading
Like Anna Karina’s Sweater
Lime Tree
Limited Inc.
Long Pauses
Long Story, Short Pier
Long Sunday
MadInkBeard
Making Light
Maud Newton
Michael Berube
Moo2
MoorishGirl
Motime Like the Present
Narrow Shore
Neil Gaiman
Old Hag
Open University
Pas au-delà
Philobiblion
Planned Obsolescence
Printculture
Pseudopodium
Quick Study
Rake’s Progress
Reader of depressing books
Reading Room
ReadySteadyBlog
Reassigned Time
Reeling and Writhing
Return of the Reluctant
S1ngularity::criticism
Say Something Wonderful
Scribblingwoman
Seventypes
Shaken & Stirred
Silliman’s Blog
Slaves of Academe
Sorrow at Sills Bend
Sounds & Fury
Splinters
Spurious
Stochastic Bookmark
Tenured Radical
the Diaries of Franz Kafka
The Elegant Variation
The Home and the World
The Intersection
The Litblog Co-Op
The Literary Saloon
The Literary Thug
The Little Professor
The Midnight Bell
The Mumpsimus
The Pinocchio Theory
The Reading Experience
The Salt-Box
The Weblog
This Public Address
This Space: The Fire’s Blog
Thoughts, Arguments & Rants
Tingle Alley
Uncomplicatedly
Unfogged
University Diaries
Unqualified Offerings
Waggish
What Now?
William Gibson
Wordherders

Sunday, June 05, 2005

Getting Ranked

Posted by Matt Greenfield on 06/05/05 at 11:59 PM

The Research Assessment Exercise, which ranks departments and universities by the quantity of their publications, has had an unhappy effect on British academics.  When one’s own paycheck depends on the number of publications extruded by one’s department, the colleague who publishes three mediocre monographs in five years is a hero; the one who publishes one great book every ten years is not pulling her weight.  I have noticed that British academics often prefer to publish a collection of articles as a ninety-dollar book rather than as a special issue of a journal with a wide circulation.  Why?  A book counts for more in the RAE. 

One major public university in the U.S. has a slightly more nuanced system of evaluation.  Each journal is assigned a numerical rank based on its quality and prestige.  Raritan might be a four and Postmodern Culture a 3.  Of course, members of the department do the ranking.  And how much does the name of the journal ever tell you about the quality of the article?  “Bring out number, weight, and measure,” Blake said, “in a year of dearth.”

At the same time, there is nothing especially admirable about a department where no one publishes anything.  Some professors say that they don’t publish much because they invest more in their pedagogy, but in my experience the most active scholars are often also the most inventive and committed teachers.  And it is understandable that administrations, legislatures, and prospective students would want some tools for evaluating departments and comparing them to each other.  As distasteful as it may be to think about assessment methodologies, I would rather design a rational one than have a stupid one imposed on me.

What, then, should those methodologies be?  One possibility is that they could be based on peer review.  Most departments are already reviewed every five years by external evaluators; perhaps departments could make portions of these documents publicly available.  Then, instead of looking at a virtually meaningless numerical ranking in the style of U.S News and World Report, we could see thoughtful descriptions of a department’s strengths and weaknesses.  Departments could also offer a self-evaluation.  Such a document might attempt to conceal weaknesses, but it would probably offer a reasonably trustworthy map of a department’s most noteworthy accomplishments.

I am tempted to elevate this suggestion to a general principle.  If we are all being sorted, ranked, and evaluated with increasing frequency, perhaps the best response is not to stonewall but to offer richer information about ourselves.

Take, for example, the writing component of the new SAT exams.  Rather than assigning numerical ratings to student essays, it is infinitely preferable to transmit the essays themselves to college admissions departments.  Then, instead of relying on ETS graders, college admissions officers can make up their own minds about the quality of a student’s writing. 


Comments

I want to hear your rankings of departments, journals, all of it. We’re going to duplicate the Leiter Report right here and now.

By Jonathan on 06/06/05 at 11:39 PM | Permanent link to this comment

Jonathan, I find it hard to understand the target of your sarcasm.  I am not advocating a numerical ranking of departments and journals in the manner of Leiter.  I am pointing out that academic institutions are being pressured into performing more and more assessment and evaluation.

In the last year, I have spent more time than I would like trying to get the NCTE to accredit my department to train teachers.  I just wrote a small chunk of the self-study for the college’s accreditation review.  I have been asked to sit on a task force for the development of a rubric for evaluating term papers from our composition classes.  I was asked to speak on a panel on outcomes assessment.  I am going to a conference on outcomes assessment later this week.  My department was asked by a CUNY dean to provide a tally of our books, articles, grants, and editorial commitments for the previous year.  Meanwhile, the CUNY system as a whole has just added a set of critical thinking exams that students take after their first sixty credits of coursework.  If they don’t pass, they don’t get to move on to their junior years.  The assessments are descending on us like locusts.  Since academic workers cannot simply opt out, the question is whether we passively submit.  If we must participate, I think we should try to design the best systems we can.  It is our moral obligation to do this for our students, and it is prudent for us to do this for ourselves when someone insists on evaluating us.  Would you disagree with this argument?  If so, why?  If not, do you have any ideas about how to do smarter evaluations of students, teachers, scholars, departments, or colleges?

By Matthew Greenfield on 06/07/05 at 12:25 AM | Permanent link to this comment

I wasn’t being sarcastic at all. I’ve long advocated a Leiter-style ranking of English departments. I say we fire it up, right here and now.

By Jonathan on 06/07/05 at 12:30 AM | Permanent link to this comment

Some professors say that they don’t publish much because they invest more in their pedagogy, but in my experience the most active scholars are often also the most inventive and committed teachers.

Mmm, not in my experience.  If you want to have some sound basis for evaluating performance, you’re going to have to do better than that. 

Departments could also offer a self-evaluation.  Such a document might attempt to conceal weaknesses, but it would probably offer a reasonably trustworthy map of a department’s most noteworthy accomplishments.

Wow!  Do you really think a department is going to risk its reputation on an honest self-evaluation?  No way! It would be advertisement.

By on 06/07/05 at 12:51 AM | Permanent link to this comment

As I thought I had made clear, I agree with you on both points, blah.  If one wants to evaluate a department’s teaching, one needs things like teaching evaluations and possibly exit interviews with graduating students.  I was just saying that when a professor produces no scholarship, a commitment to teaching is not a sufficient excuse.  The reason why college professors spend less time in the classroom than high school teachers is that they are expected to do original research.  And I think that professors who are involved in research do teach a different set of skills, even in entry-level courses.

A self-evaluation can sometimes be surprisingly honest.  Have you ever asked students what grades they think they deserve?  Some are too lenient but many are way too harsh.  But the point I was making in my post was that a self-evaluation would give a fairly accurate picture of a department’s major successes: Professor A got a Guggenheim, Professor B now writes for both Harper’s and the New Yorker, and Professor C just published an article in PMLA.  Right now, it can be surprisingly hard to extract such information from a department’s web site.  You usually have to go through individual faculty biographies, if they are even up to date.  When I was applying to graduate school, before the advent of the web, it was even harder to determine what a department’s strengths were.  I had to go to the library and look at microfiche catalogues just to figure out who the faculty in a department were, then look them up in the MLA Bibliography, which was on a CD-ROM in a separate room, and then hunt down books and articles.

By Matthew Greenfield on 06/07/05 at 11:02 AM | Permanent link to this comment

Is a 4/4 load less time in the classroom than a high school teacher? It may be, but I was thinking they didn’t teach quite that much on average.

By Jonathan on 06/07/05 at 11:21 AM | Permanent link to this comment

A 4/4 load usually is 12 hours of classroom time.  High school teachers usually spend at least four hours a day in the classroom, which is 20 hours.  And colleges with 4/4 loads, if they have any interest in research at all, usually give course releases to productive scholars so that their actual loads are more like 3/3.

My colleagues in the music department do over twenty hours a week of teaching, if one counts their one-on-one lessons as well as their classes.  They are known as fabulous teachers.  And they somehow find time to write books and articles, memorize long, complex works and perform them, and serve on numerous committees. Yikes!

By Matthew Greenfield on 06/07/05 at 12:03 PM | Permanent link to this comment

"And colleges with 4/4 loads, if they have any interest in research at all, usually give course releases to productive scholars so that their actual loads are more like 3/3.”

Are you absolutely sure about that?

By Jonathan on 06/07/05 at 01:31 PM | Permanent link to this comment

There are many sad exceptions, but administrators at schools with heavy course-loads do seem to have the ability to lighten the burdens of at least a minority of their faculty.  I know many schools where almost no one teaches the official course-load: even a tiny amount of research gets one the course release.  Administrators can give course releases for research or for administrative responsibilities, some of which might not be very demanding.  Sometimes during the hiring process they are not allowed to spell this out.

By Matthew Greenfield on 06/07/05 at 07:28 PM | Permanent link to this comment

A lot of my time, and that of my colleagues, is already spent on such assessment activities.  Whether I am reading an article for a journal whose editorial board I belong to, evaluating a book ms. for a university press, evaluating a candidate for tenure at another university (or my own), reviewing grant proposals for the NEH, serving on a commitee that reviews graduate programs within the university, evaluating my own colleagues on the merit commmittee, preparing my own documents to be evaluated by others.  When I’m not doing that, I’m grading student papers or exams or evaluating job candidates for positions in my department.  So much time spent passing judgment:  that’s the academic life.  I’m not complaining.  All these judgments need to be made, after all. I’m just not sure we need more of it than we already have.  It would be nice to have more time to teach and do research.

By Jonathan Mayhew on 06/07/05 at 11:12 PM | Permanent link to this comment

Add a comment:

Name:
Email:
Location:
URL:

 

Remember my personal information

Notify me of follow-up comments?

Please enter the word you see in the image below: