There was a session on bibliometrics, including a hands-on bit looking at citation records.
The consensus seems to be that the REF for the education area won’t use bibliographic metrics extensively in the way that other subjects will. I assume this is partly because the data isn’t well recorded and possibly that publication patterns are just not sufficiently rich to make the statistics reliable. So indications the REF is likely to be similar to the RAE in that each personal submission will be based on four world-class publications (I wish!). But to keep effort manageable the reviewing panel may only look in detail at one of these and they may use bibliometrics to choose.
Earlier Bridget had mentioned external grant income as important REF metric – again, avoids REF panel looking at detail since they can use the fact that external funding is acquired through peer review process, so existence of grant is some statement of quality.
Journal rankings were discussed. The impact factors given by databases didn’t seem to reflect views of people in the room. People are also submitting to journals that aren’t high impact or even to those that aren’t given a rating at all. (I wasn’t at all clear what this means – implication was that impact was so poor that not worth calculating. But I don’t think that follows, though it might mean that data was so thin that couldn’t be calculated and that would imply too few citations to measure.) Also emerged that some places people publish aren’t covered by databases – some open access journals may not yet be included and book chapters aren’t indexed. So such publications appear invisible.
Message appears mixed. ‘Best to publish in journals where you wish to be part of the conversation’. But on the other hand, given a choice, better to publish in prestigious journal.
Mention was made again that book chapters don’t appear in citations. (Actually, the big databases index journals and so only collect citations between journals and not out to books. But the new kid Google Scholar does count citations to books, although its journal citations probably are nothing like as complete.) I’ve been looking for my own publications (vanity, vanity) and found that the book chapters are all but invisible on the web, though journal publications much easily found. So putting them in ORO is a good thing – Google Scholar indexes ORO and demonstrably increases chance of finding stuff.
Still on a personal note, my citation record actually appears pretty good – if you look at it the right way. Papers I published twenty years ago are still being cited, the total number of citations is pretty good and some journals have high impact factors. Slightly unfortunate that it includes no recent papers and none in the CREET area! So I guess none of it will count
[BTW I also noticed that Bridget Heywood doesn’t appear to feature in ORO ]