ALCUNI ESEMPI DI VALUTAZIONE DELLA RICERCA SCIENTIFICA IN EUROPA Pierangelo Marcati Dipartimento di Matematica Pura e Applicata marcati@univaq.it L Aquila 23 maggio 2008
RAE del Regno Unito quale esempio di valutazione nazionale DFG - Deutsche Forschungsgemeinschaft quale esempio di valutazione nazionale indiretta Universita di Oulu in Finlandia quale esempio di valutazione interna Il CIVR e l ANVUR (?)
A 5 panels MEDICINA 1 * 1 Cardiovascular Medicine * 2 Cancer Studies * 3 Infection and Immunology * 4 Other Hospital Based Clinical Subjects * 5 Other Laboratory Based Clinical Subjects B 4 panels MEDICINA 2 * 6 Epidemiology and Public Health * 7 Health Services Research * 8 Primary Care and Other Community Based Clinical Subjects * 9 Psychiatry, Neuroscience and Clinical Psychology C 4 panels MEDICINA 3 * 10 Dentistry * 11 Nursing and Midwifery * 12 Allied Health Professions and Studies * 13 Pharmacy D 3 panels BIO 1 * 14 Biological Sciences * 15 Pre-clinical and Human Biological Sciences * 16 Agriculture, Veterinary and Food Science E 3 panels FISICA CHIMICA GEO * 17 Earth Systems and Environmental Sciences * 18 Chemistry * 19 Physics
F 4 panels MAT. e INFORMATICA * 20 Pure Mathematics * 21 Applied Mathematics * 22 Statistics and Operational Research * 23 Computer Science and Informatics G 6 panels INGEGNERIA e MATERIALI * 24 Electrical and Electronic Engineering * 25 General Engineering and Mineral & Mining Engineering * 26 Chemical Engineering * 27 Civil Engineering * 28 Mechanical, Aeronautical and Manufacturing Engineering * 29 Metallurgy and Materials H 4 panels ARCHITETTURA * 30 Architecture and the Built Environment * 31 Town and Country Planning * 32 Geography and Environmental Studies * 33 Archaeology I 4 panels ECONOMIA e FINANZA * 34 Economics and Econometrics * 35 Accounting and Finance * 36 Business and Management Studies * 37 Library and Information Management J 6panels LEGGE, Sc.POLITICHE, SOCIOLOGIA, ANTROPOLOGIA * 38 Law * 39 Politics and International Studies * 40 Social Work and Social Policy & Administration * 41 Sociology * 42 Anthropology
* 43 Development Studies
K 3panels PSICOLOGIA, Sc. FORMAZIONE, SPORT * 44 Psychology * 45 Education * 46 Sports-Related Studies L M N 4 panels STUDI INTERNAZIONALI * 47 American Studies and Anglophone Area Studies * 48 Middle Eastern and African Studies * 49 Asian Studies * 50 European Studies 8panels LINGUA E LETTERATURA * 51 Russian, Slavonic and East European Languages * 52 French * 53 German, Dutch and Scandinavian Languages * 54 Italian * 55 Iberian and Latin American Languages * 56 Celtic Studies * 57 English Language and Literature * 58 Linguistics 4panels STORIA, FILOSOFIA, RELIGIONE * 59 Classics, Ancient History, Byzantine and Modern Greek Studies * 60 Philosophy * 61 Theology, Divinity and Religious Studies * 62 History O 5panels ARTE, MUSICA, TEATRO, DANZA, Sc. COMUNICAZIONE * 63 Art and Design * 64 History of Art, Architecture and Design * 65 Drama, Dance and Performing Arts * 66 Communication, Cultural and Media Studies * 67 Music
Come funziona il RAE? Il RAE analizza con il sistema del peer review combinato con study sessions la ricerca scientifica e le attività connesse Quale e l oggetto della valutazione? I dipartimenti universitari Quale e lo scopo? Distribuire I finanziamenti
Quali sono le informazioni che entrano in valutazione? Ogni istituzione fornisce informazione su: Staff di ricerca attivo (RA1) Prodotti della ricerca fino a 4 pubblicazione per persona attiva (RA2) Research assistants (gia in RA1), research students, borse di ricerca (RA3) Entrate esterne per ricerca (RA4) Descrizione dell ambiente della ricerca e pianificazione strategica (RA5) (in aggiunta informazioni confidenziali )
Cosa e un prodotto della Ricerca? Referee article in Category A journal Monograph (publisher significant), scholarly edition Chapter in book Also: creative writing,coauthored works, applied research, teaching material IF there s a research component, electronic publications, databases, etc.
Livelli di Qualita 4* Quality that is world-leading in terms of originality, significance and rigour. 3* Quality that is internationally excellent in terms of originality, significance and rigour but which nonetheless falls short of the highest standards of excellence 2* Quality that is recognised internationally in terms of originality, significance and rigour. 1* Quality that is recognised nationally in terms of originality, significance and rigour. u/c Quality that falls below the standard of nationally recognised work.
GLI ESTEEM INDICATORS SONO INDICATORI PARAMETRICI ESTERNI - PER ESEMPIO L IMPACT FACTOR
Come viene usato? Guardiamo alcuni risultati del RAE 2001 1 Chemical Engineering 2 Physics
Chemical Engineering 2001 Research Assessment Exercise Unit of Assessment : 27 Chemical Engineering Number of submissions: 17 2001 Rating Proportion of Staff Selected Category A and A* Research Active Staff (FTE) University of Bath 4 A 24,1 University of Birmingham 5* A 24,0 University of Bradford 3a D 6,5 University of Cambridge 5 A 20,0 Imperial College of Science, Technology and Medicine 5* A 37,9 Loughborough University 4 A 19,6 University of Manchester Institute of Science & 5 A 33,9 Technology University of Newcastle 5 B 13,0 University of Sheffield 4 B 19,0 South Bank University 3a E 5,0 University of Surrey A - Centre For Environmental Strategy 5 A 9,0 Z - Chemical Engineering 4 B 11,2 University College London 5* A 25,0 University of Edinburgh 4 C 12,0 Glasgow Caledonian University 3b D 6,0 Heriot-Watt University 4 A 11,3 The Queen's University of Belfast 4 A 16,0
Physics 2001 Research Assessment Exercise Unit of Assessment : 19 Physics Number of submissions: 50 Proport ion of Staff Selecte d Category A and A* Research Active Staff (FTE) 2001 Rating University of Bath 4 A 24,0 University of Birmingham 5 B 52,6 University of Brighton 3a A 1,0 University of Bristol 5 A 47,0 University of Cambridge 5* A 138,9 University of Central Lancashire 4 B 16,2 City University 4 A 2,0 University of Durham 5 A 55,0 University of Exeter 5 A 20,0 University of Hertfordshire 4 B 22,0 Imperial College of Science, Technology and Medicine 5* A 99,8 Keele University 3a B 16,5 University of Kent at Canterbury 3a C 17,2 King's College London 4 B 16,5 Lancaster University 5* A 20,8 University of Leeds 5 A 39,6 University of Leicester 5 A 41,0 University of Liverpool 5 A 35,2 Liverpool John Moores University 4 A 16,0 Loughborough University 4 C 7,0 University of Manchester 5 A 59,0 University of Manchester Institute of Science & Technology 4 A 31,3 University of Newcastle 4 B 14,5 University of Nottingham 5 A 31,8 Open University 3a C 20,0 University of Oxford 5* A 155,6 University of Plymouth 3a C 3,0 Queen Mary, University of London 5 B 49,0 University of Reading 4 B 18,3 Royal Holloway, University of London 5 B 21,0 University of Sheffield A - Physics and Astronomy 5 B 24,0 B - Medical Physics & Clinical Engineering 5 A 4,0 Sheffield Hallam University 3a F 2,0 University of Southampton 5* B 30,7
University of Surrey 5 A 28,0 University of Sussex 5 A 21,7 University College London 5 B 84,4 University of Warwick 5 A 30,3 University of York 4 B 22,0 University of Edinburgh 5 B 64,8 University of Glasgow 5 B 40,7 Heriot-Watt University 4 A 31,3 University of Paisley 3a B 7,0 University of St Andrews 5 B 27,0 University of Strathclyde 4 A 45,7 University of Wales, Swnsea 5 A 11,6 University of Wales, Aberystwyth 4 A 15,3 Cardiff University 5 A 23,0 Armagh Observatory 4 A 8,0 The Queen's University of Belfast 5 A 55,0
COME VENGONO USATI I RISULTATI? Il Dipartimento contribuisce a determinare il Finanziamento dell Universita in proporzione al numerino nella terza colonna
Problemi Costo della valutazione (= un intera nuova universita ), oltre 75 milioni di sterline!! Tempo usato per preparare il materiale e per i 359 membri dei panel Distorsione per acquisto di superstar (come nel calcio), dove tutte le pubblicazione seguono il prof. Distorsione per seguire gli indicatori delle riviste, etc
Altri problemi Le scienze umanistiche non permettono in generale uso di indicatori bibliometrici Rottura della collegialita > prime donne vs staff docente. Valutazioni bloccate per 5 6 anni I professori senior spesso impongono chi e cosa debba essere valutato
La DFG usa invece un metodo di inverse mapping ricostruisce il rank dai finanziamenti Opera per Aree e per Universita
FONDI ESTERNI 2001-2003 PER AREA per persona Serve per rinormalizzare le scale di valutazione.
Ranking
Le Universita finlandesi applicano il sistema inglese in modo autonomo ciascuna per suo conto Esempio importante: 1- panel internazionale 2- autovalutazione per assunzioni e internal funding
Esempio di panel Panel 2: Biosciences Tim Hunt, UK Cancer Research, UK, Chair Per Andersen, University of Oslo, Department of Physiology, Norway Michael Ashburner, University of Cambridge, Department of Genetics, UK Paul M. Brakefi eld, Leiden University, Institute of Biology, The Netherlands S. Dusko Ehrlich, Génétique Microbienne, INRA, France Hans-Peter Lipp, University of Zürich, Institute of Anatomy, Switzerland Celestina Mariani, University of Nijmegen, Plant Cell Biology, The Netherlands Ian P. F. Owens, Imperial College London, Department of Biological Sciences, UK Lisa Sennerby Forsse, The Swedish Research Council for Environment, Agricultural Sciences and Spatial Planning, Sweden Anna Tramontano, University of Rome, Department of Biological Sciences, Italy