January 09, 2009
January 08, 2009
Reviewers raise concerns about RAE gameplaying
The strategic games played by universities to inflate their ratings in the research assessment exercise have been laid bare by the peer-review panels that judged the submissions.
In a series of subject overview reports published by the Higher Education Funding Council for England this week, a number of the panels, which were responsible for assessing research quality in 67 disciplines across 159 institutions, raise concerns. They say that some universities excluded research-active staff from the exercise to artificially "exaggerate" their strengths, while others drafted in research stars on "unusual" contracts who were not fully integrated members of the research team.
David Otley, who chaired the panel that assessed economics, accounting and business and management studies, says in his report: "Some very strong units chose to submit only a proportion of their staff for assessment, despite the encouragement in the criteria for all research-active staff to be submitted."
Because "the panels had no information on the proportion of staff submitted", this meant that some departments "appeared stronger than others solely for this reason".
The business and management studies subpanel says in its report that it "remained concerned ... about the varying degree of selectivity that was apparent in the submissions".
"Although this knowledge was not used in making assessments, it is clear that some submissions included a very small proportion of academic staff from some institutions. This selectivity probably exaggerates the strengths of some institutions."
Unlike in previous RAEs, the Higher Education Statistics Agency was not able to release data showing the proportion of eligible academics left out of the 2008 RAE because of complaints that the guidance on eligibility for submission was unclear. This meant that there was no measure of "research intensity" in departments, prompting claims that the RAE results failed to show the true picture of research.
Times Higher Education understands that some RAE panels penalised perceived gamesmanship by giving a low ranking in the "esteem" and "research environment" assessment categories to departments they believed had submitted a low proportion of their staff. These categories will be published in RAE "subprofiles" in the spring.
The panels' reports also contain concerns about the recruitment of research stars. The subpanel on sociology is "struck by the fact that a few departments placed a great reliance on the presence of highly esteemed academics on fractional contracts or other unusual contractual arrangements".
It adds: "There was not always evidence that these academics ... made an effective contribution to the research culture."
The Asian studies panel's report notes that "the recent appointment of several short-term contract staff" in some institutions "puts a question mark over the sustainability of certain areas of research in the longer term".
Paul Marshall, executive director of the 1994 Group of small research-intensive universities, which submitted a higher proportion of their researchers than the larger research-intensive universities to previous RAEs, said: "Comments emerging from the panels are of great interest and will, we believe, be reflected in the subprofiles achieved for esteem and the research environment."
Les Ebdon, vice-chancellor of the University of Bedfordshire and head of the Million+ think-tank, said: "Some universities have selected their most active staff for submission to gain reputational advantage; whether this will also maximise funding remains to be seen."
---------
The games they play... and these are the ones that have come out. There are others...
From: http://www.timeshighereducation.co.uk
In a series of subject overview reports published by the Higher Education Funding Council for England this week, a number of the panels, which were responsible for assessing research quality in 67 disciplines across 159 institutions, raise concerns. They say that some universities excluded research-active staff from the exercise to artificially "exaggerate" their strengths, while others drafted in research stars on "unusual" contracts who were not fully integrated members of the research team.
David Otley, who chaired the panel that assessed economics, accounting and business and management studies, says in his report: "Some very strong units chose to submit only a proportion of their staff for assessment, despite the encouragement in the criteria for all research-active staff to be submitted."
Because "the panels had no information on the proportion of staff submitted", this meant that some departments "appeared stronger than others solely for this reason".
The business and management studies subpanel says in its report that it "remained concerned ... about the varying degree of selectivity that was apparent in the submissions".
"Although this knowledge was not used in making assessments, it is clear that some submissions included a very small proportion of academic staff from some institutions. This selectivity probably exaggerates the strengths of some institutions."
Unlike in previous RAEs, the Higher Education Statistics Agency was not able to release data showing the proportion of eligible academics left out of the 2008 RAE because of complaints that the guidance on eligibility for submission was unclear. This meant that there was no measure of "research intensity" in departments, prompting claims that the RAE results failed to show the true picture of research.
Times Higher Education understands that some RAE panels penalised perceived gamesmanship by giving a low ranking in the "esteem" and "research environment" assessment categories to departments they believed had submitted a low proportion of their staff. These categories will be published in RAE "subprofiles" in the spring.
The panels' reports also contain concerns about the recruitment of research stars. The subpanel on sociology is "struck by the fact that a few departments placed a great reliance on the presence of highly esteemed academics on fractional contracts or other unusual contractual arrangements".
It adds: "There was not always evidence that these academics ... made an effective contribution to the research culture."
The Asian studies panel's report notes that "the recent appointment of several short-term contract staff" in some institutions "puts a question mark over the sustainability of certain areas of research in the longer term".
Paul Marshall, executive director of the 1994 Group of small research-intensive universities, which submitted a higher proportion of their researchers than the larger research-intensive universities to previous RAEs, said: "Comments emerging from the panels are of great interest and will, we believe, be reflected in the subprofiles achieved for esteem and the research environment."
Les Ebdon, vice-chancellor of the University of Bedfordshire and head of the Million+ think-tank, said: "Some universities have selected their most active staff for submission to gain reputational advantage; whether this will also maximise funding remains to be seen."
---------
The games they play... and these are the ones that have come out. There are others...
From: http://www.timeshighereducation.co.uk
January 07, 2009
The Ten Recommended Administrative Measures
1. “Focus on the situation, issue, or behaviour, not the person.”
2. Replace quasi-judicial campus tribunals with administrative decision-making.
3. Unless evidence compels them, avoid forensic words like allegations and charges.
4. Keep the rules clear, fair, and simple; keep policy and procedure manuals short.
5. In the face of demands that a professor be punished, entertain not just the null hypothesis but the mobbing hypothesis.
6. Seek proximate, specific, depersonalized explanations for why some professor is on the outs, as opposed to distant, general, personal explanations.
7. Encourage mindfulness of all the bases on which academic mobbings occur.
8. Defend free expression and encourage dialogic outlets for it on campus.
9. Keep administration open and loose.
10. Answer internal mail.
From: http://arts.uwaterloo.ca/~kwesthue/waterloostrategy.htm
2. Replace quasi-judicial campus tribunals with administrative decision-making.
3. Unless evidence compels them, avoid forensic words like allegations and charges.
4. Keep the rules clear, fair, and simple; keep policy and procedure manuals short.
5. In the face of demands that a professor be punished, entertain not just the null hypothesis but the mobbing hypothesis.
6. Seek proximate, specific, depersonalized explanations for why some professor is on the outs, as opposed to distant, general, personal explanations.
7. Encourage mindfulness of all the bases on which academic mobbings occur.
8. Defend free expression and encourage dialogic outlets for it on campus.
9. Keep administration open and loose.
10. Answer internal mail.
From: http://arts.uwaterloo.ca/~kwesthue/waterloostrategy.htm
January 06, 2009
Retribution for asserting (or trying to assert one's rights)
Graduate students are at the mercy of academic bul... Graduate students are at the mercy of academic bullies just as professors are. It appears to be a phenomenon on both sides of the Atlantic, as well. In the US, a system of regional accrediting agencies is supposed to ensure students' rights, but in reality these agencies are run by the institutions and do little or nothing to ensure fair processes. In my case, the university administrator in charge of the doctoral program made substantive changes to the program and prohibited doctoral candidates from pursuing their dissertation research until long after advancement to candidacy. She claimed the delay (two full semesters) was for educational purposes, but she provided no rationale and the regional accrediting agency refused to intervene even though such a delay was a clear violation of the agency's standards (WASC, in this case). The university ombudsman offered no relief just as in the case of the post, above. It's a sad commentary on the state of higher education when educated individuals act in such a medieval manner. I'm not a fan of anonymous posts, but you will see that I have posted anonymously. Why? Retribution for asserting (or trying to assert one's rights).
Anonymous
Anonymous
December 26, 2008
The 'best' for 2008...
- News from Kingston University
- A favorite tactic of bullies...
- Not only in America...
- Staff give sector managers low marks
- Sticks, Stones and Semantics: The Ivory Tower Bully's Vocabulary of Motives
- A letter to Sally Hunt
- The canary down the mine: what whistleblowers' health tells us about their environment
- Warning: chronic bullying is hazardous to the academy's health
- Academic bullies
- Why does [academic] mobbing take place?
- The Fundamental Question: Do You Side With Bullied Targets or With Perpetrators?
- Avoiding Academe's Ax Murderers
Vote/nominate for the oppressive University of 2008:
The shortlist included: the University of Leicester, Leeds Metropolitan, Oest. Akademie der Wissenschaften (Austria), the University of Nottingham, the University of Wales College, and Kingston University.
And the winner is: Kingston University - the facts speak for themselves.
December 23, 2008
42% consider leaving!
When 42% consider leaving their job at a certain university, then you do know there is a problem - a managerial problem.
December 22, 2008
A senseless system graduates without honours
The 2008 university Research Assessment Exercise (RAE), whose results have been announced with a mixture of fear, loathing and exhaustion, is a classic example of the self-defeating performance-management drive that is overwhelming the public sector.
RAE results determine the research funding allocated to institutions by the Higher Education Funding Council, according to a formula that changes each time. The official line is that the assessment - 2008's is the sixth since 1986 - is a success. It is "important and valuable", to quote one vice-chancellor, in providing an accepted quality yardstick and a means of promoting UK universities abroad. Others argue that it helps to ensure accountability for £8bn of public funding, the largest single chunk of university income. That sounds plausible: but as usual it conveniently airbrushes out other costs and consequences.
The first and most obvious of these is colossal bureaucracy. Government blithely assumes that management is weightless; but the direct cost of writing detailed specifications and special software, and assembling 1,100 panellists to scrutinise submissions from 50,000 individuals in 2,500 submissions, high as it already is, is dwarfed by the indirect ones - in particular, the huge and ongoing management overheads in the universities themselves. As with any target exercise, the RAE has developed into a costly arms race between the participants, who quickly figure out how to work the rules to their advantage, and regulators trying to plug the loopholes by adjusting and elaborating them.
The result is an RAE rulebook of staggering complexity on one side and, on the other, the generation of an army of university managers, consultants and PR spinners whose de facto purpose is not to teach, nor make intellectual discoveries, but to manage RAE scores. As in previous assessments, a lively transfer market in prolific researchers developed before the submission cut-off date at the end of 2007, while, under the urging of their managers, many university departments have been drafting and redrafting their submissions for the past three years.
Meanwhile, the figures themselves can be interpreted in so many different ways that even insiders find them hard to comprehend. How many parents will know that, because the rules and ranking system has changed so much since 2001, it's difficult to identify performance trends? That departments nominally teaching the same subject may figure under different assessment panels, so here too direct comparison is difficult? That some numbers are bafflingly rounded, while from the figures given it is impossible to calculate how many of a department's staff have been submitted for the assessment exercise, and thus its "real" research strength?
Not surprisingly, as the monster has become increasingly unwieldy, the intervals between the ever more onerous audits has steadily lengthened. After a gap that has stretched to seven years this time, RAE 2008, the last of the present format, is expiring exhausted - although it will rise again in 2013 as a system based on 'metrics', or citations, that promises to be equally controversial.
In the meantime, though, many thoughtful academics believe that much damage has been done. On a systems view, you can't optimise one part of a system without affecting others. In the university context, what suffers from the research obsession ("publish or perish") is teaching, especially undergraduate teaching. It's not much use students choosing a university with internationally known researchers if the researchers are too busy to teach. A teaching assessment exercise turned out to be too nightmarishly bureaucratic even for this government and has been abandoned.
Within research, there is little doubt that target pressure has distorted priorities, forcing researchers to work within the tight guidelines of a few established publications, discouraging unconventional views and making unpredictable discovery all but impossible.
Somewhat ironically, the narrow horizons have a particularly perverse effect in economics and business studies, where, judging by today's melted-down financial sector, "paradigm shifts" are needed more than anywhere else. They are unlikely to emerge, however, from learned journals that effectively privilege research for research's sake over usable knowledge and are light years away from the concerns of inquiring managers.
Finally, the RAE is a potent symbol and vehicle for the bullying top-down managerial culture that has steadily eroded both the quality of working life and results in much of the public sector. This management style has given us Baby P and HM Revenue and Customs on the one hand, and General Motors and the financial collapse on the other. Universities should be part of the search for alternatives, not a reinforcement for today's bankrupt model.
From: http://www.guardian.co.uk/business/2008/dec/21/rae-university-funding
December 20, 2008
+GraphicShrunk_1.jpg)
+GraphicShrunk_2.jpg)
+GraphicShrunk_3.jpg)
+GraphicShrunk_4.jpg)
+GraphicShrunk_6.jpg)
+GraphicShrunk_5.jpg)
+GraphicShrunk_7.jpg)
+GraphicShrunk_8.jpg)
+GraphicShrunk_9.jpg)
+GraphicShrunk_10.jpg)
December 19, 2008
About academic management...
I recently attending a bullying and harassment policy briefing session for academics in management roles at my university. The point of the session was to ensure that managers stayed just on the right side of the law, given that (in the words of the session leader) "there is a very thin line between bullying and management."
From: Poll points to distrust of 'petty' managers
From: Poll points to distrust of 'petty' managers
Subscribe to:
Posts (Atom)