Category Archives: Metrics

Metrics: what they are and how to use them

David and I ran the session on metrics on Wednesday and had a good attendance – so thanks to all who came out.

metrics-what they are and how to use them-December-2016- Blog

The session was an overview of metrics, researcher profiles, the application of metrics and the problematic behaviours these applications have created.

The slides are available: metrics-what they are and how to use them-December-2016- Blog and so are our notes:  Metrics-what-they-are-and-how-to-use-them-script-December-2016.

 

 

ORO Downloads – June 2016

I’m a bit reluctant to publish lists of top downloads from ORO as they only tell the story of those items that get an exceptional number of downloads from ORO.  Sometimes these numbers are questionable and can be the result of non-human downloads that haven’t been filtered out by either the e-prints software (on which ORO runs) or the Jisc service IRUS which we also use to capture download counts.  But more importantly it doesn’t capture the more modest downloads accruing on ORO – the repository has, if you like, a long tail of downloads where the majority of downloads are actually gained by lots and lots of outputs getting smaller amounts of downloads.

So I’ve expanded our list from 15 to 50 to see what we capture.  It’s still the exceptions (the top 50 makes up about 0.6% of the total Open Access items in ORO) but there are some interesting stories to tell.

ORB-2016-06-monthly-downloadsFirst and foremost is the top of the list: Petre, Marian (2013). UML in practice. In: 35th International Conference on Software Engineering (ICSE 2013), 18-26 May 2013, San Francisco, CA, USA.  Marian’s paper has been a very popular paper in ORO for a very long time but last month was extraordinary with 1,784 downloads. Fortunately, I think we can explain this, ORO is telling me that it the item is being referred to from Wikipedia and it appears the paper was added as a reference to the Wikipedia page on UML sometime in August last year.  Additionally, in June traffic to the ORO item appears to have referred from social media sites like Twitter and Facebook, and sites like Y combinator (a start-up incubator) and feedly (an aggregator service).  I think it’s fair to say the presence of the paper on Wikipedia has led to it’s greater dissemination across various web platforms – maybe it’s time for the OU to have a Wikipedian in Residence!

Secondly we have the fifth item on the list: Sharples, M.; Adams, A.; Alozie, N.; Ferguson, R.; FitzGerald, E.; Gaved, M.; McAndrew, P.; Means, B.; Remold, J.; Rienties, B.; Roschelle, J.; Vogt, K.; Whitelock, D. and Yarnall, L. (2016). 创新教学报告2015 —探索教学、学习与评价的新形式 [Innovating Pedagogy 2015]. 开放学习研究 [Open Learning] (2016.1) pp. 1–18.  Now this is the Chinese translation of the 2015 Innovating Pedagogy Report which appears in the Chinese language journal Open Learning.  Last year we wouldn’t have accepted this item in ORO as is.  Rather than a discrete item with associated metadata in the source language we would have just added the file alongside the original English language version.  However, last year during Open Access week we were challenged to accept these items as discrete records in ORO to support discoverability and make ORO a more global resource.  We changed our policy and the benefit is evident here.  Interestingly, it seems that the majority of downloads, at least in June, are coming from the U.S.A. (187) rather than China (43).

Thirdly, there are three theses on the list.  Theses occasionally get a bit higher ranked than this and make the top 15 – but they are consistently highly downloaded.  Institutional Repositories have a major role supporting the dissemination of materials that do not get published via the standard routes of academic publishing – I’m thinking particularly about theses and reports that may not get a good platform for dissemination elsewhere.  So whilst there has been such an emphasis recently on ORO and the HEFCE Open Access Policy – we shouldn’t lose sight of the key function ORO can play in the dissemination of these other research outputs.

Finally, and perhaps most fundamentally, this Top 50 draws into stark contrast the benefits different faculties get from ORO.  There is only one item from the Science Faculty on this list.  The route to Open Access for Science is well supported by disciplinary repositories and Gold Open Access publishing (frequently, but not always, funded by the RCUK block grant).  Notwithstanding the requirement for a university to be aware of, and showcase, all its published research outputs, the value of the institutional repository can be discipline specific – and we need to pay close attention to that when advocating its usage.

 

Does using ORO provide a citation advantage?

There have been several studies indicating that deposit in an institutional repository (like our own ORO) does provide a citation advantage.  The one I usually refer to when I’m promoting ORO is the one conducted by Lars Kullman “The Effect of Open Access on Citation Rates of Self archived Articles at Chalmers” which finds that “self-archived articles have a 22% higher citation rate than articles that were not self-archived“.  

It’s something I’ve often wanted to do in relation to ORO, but haven’t got round to before now, not least because it’s a tricky thing to investigate.  There are a number of things to bear in mind when evaluating the impact deposit in a repository may have on citations of any particular paper, they include:

  1. The disciplinary variance of citations – some disciplines cite heavier than others and this might skew the results if a certain discipline is more heavily represented in the dataset being analysed.
  2. Citations increase over time – obviously older papers that have been in circulation longer will accrue more citations.
  3. Items deposited in the institutional repository as Open Access may also be Open Access in another repository (e.g. arXiv) or published Gold Open Access on the publisher’s website.  So it might not be their presence in ORO that is making the difference!
  4. Author’s may self select which papers are made Open Access in a repository i.e. only the better papers are made openly available.  And these (you would hope!) are the ones that will get more citations.

And at this point I normally give up and go back to the nitty gritty of running ORO – I’m not a researcher.  However, we’ve recently been using the bibliometrics tool SciVal (an Elsevier product based on Scopus data) and evaluating how that can support individuals, research groups and the University to benchmark their research.  So I decided to run some data through it and see what came out, and this is what I found.

Total Outputs Citations per Publication Field Weighted Citation Impact
ORO research outputs with DOI 10,008 20.9 1.88
OU research outputs with DOI 11,684 16.1 1.65
OU research outputs 16,627 14.9 1.48
  • Date range: 1996-2015
  • Field Weight Citation Impact (FWCI) – a relative measure of citations normalised by discipline, publication type and published date where 1 is the average e.g. a FWCI of 1.48 means it has been 48% more cited than expected
  • All tables ranked by FWCI
  • Data available here: ORB_OROSciValData

OK, let’s unpick that a bit.

  • OU research outputs is everything in Scopus/Scival that has been affiliated to an OU researcher.  SciVal/Scopus is only a subset of everything that is published and is skewed towards the sciences – coverage of the Arts and Social Sciences is not great!
  • OU research outputs will capture all outputs from authors with an OU affiliation – these may not be research contracted staff (e.g. Academic-related staff or Associate Lecturers) e.g. some of these people may not be able to deposit in ORO.
  • ORO currently captures around two thirds of everything that gets published by OU researchers.
  • ORO captures research outputs of current OU researchers published before they joined the OU – so some ORO items will not have an OU affiliation and therefore not be in the SciVal/Scopus dataset.
  • The ORO research outputs with DOI are only those items in ORO with a DOI – there are a lot of items in ORO that don’t have DOIs – but that’s the only way I could import them into SciVal to do the analysis!  I don’t know but maybe there would be a correlation between possession of a DOI and citability of an output.

So that’s some context, but the data there does seem to indicate that items in ORO get a citation advantage compared to all OU research outputs indexed in SciVal/Scopus.  So I dug a bit deeper and tried to get some data on those publications that were only on ORO or only in the SciVal/Scopus OU dataset.

Total Outputs Citations per Publication Field Weighted Citation Impact
ORO outputs only (with DOI) 3,003 27.0 2.03
OU research outputs not in ORO (with DOI) 4,679 13.0 1.41

That appears to be even more emphatic, but it may well be self selecting – people may only be putting on ORO the stuff they want people to see!

My appetite well and truly wetted I tried to see whether being Open Access in ORO differed from just being in ORO (i.e. metadata only).

Total Outputs Citations per Publication Field Weighted Citation Impact
ORO Open Access with DOI 2,926 16.1 1.98
ORO research outputs with DOI 10,008 20.9 1.88
ORO Metadata only with DOI 7,116 22.8 1.84

Note: you’ll see the numbers don’t add up – that’s because there were a number of duplicates that appeared in both sets (i.e. they had been deposited in ORO as both open access and metadata only). 

Not much at first glance, it might appear that just getting the metadata visible to Google is enough to get the research output noticed and citable.  However,  the FWCI value for Open Access is a bit higher than metadata only, especially when considered in relation with Citations per Publication.  I’ve written here before how Arts and Social Sciences full text gets downloaded more than STEM and, given differences in citation practices, I wonder if that is happening here too?

Finally, I tried to tie this up by seeing whether appearance in high impact factor journals had a determining influence in the varying citation measures across these datasets.  So I used one of the metrics in SciVal (Publications in Top 10 Journal Percentiles% – SNIP) to see whether this might be the case.

Total Outputs Citations per Publication Field Weighted Citation Impact Publications in Top 10 Journal Percentiles % – SNIP
ORO outputs only (with DOI) 3,003 27.0 2.03 25.4
ORO Open Access with DOI 2,926 16.1 1.98 22.1
ORO research outputs with DOI 10,008 20.9 1.88 24.3
ORO Metadata only with DOI 7,116 22.8 1.84 25.2
ORO and OU with DOI 2,926 18.2 1.81 23.9
OU research outputs with DOI 11,684 16.1 1.65 22.0
OU research outputs 16,627 14.9 1.48 20.1
OU research outputs not in ORO (with DOI) 4,679 13.0 1.41 18.6

And generally yes, there is a trend that those outputs published in journals in the top percentiles in their subject area do get a higher FWCI value.

However, there is one notable exception: those items that are Open Access items in in ORO have a higher FWCI value despite having a lower Journal percentile ranking than metadata only deposits . They also have very similar Journal percentile ranking but significantly higher FWCI value than all OU research outputs with DOI.

So I think there are a couple of things to take away:

  • All items deposited in ORO do appear to get higher citations than those that aren’t, however this may be due to the fact that these items are more likely to be published in more prestigious journals in the field, rather than the fact they have been deposited in ORO.  So it’s likely that there is a self selecting element here – researchers may be more inclined to deposit items in ORO if they appear in more prestigious journals.
  • Open Access items in ORO appear to contradict this general trend. Open access deposit in ORO may lead to more citations irrespective of the apparent quality of the journal they appear in.

 

Open Research Online – Quarterly Update

Please find below quarterly update for ORO Q3 2015/16.  Key headlines for the last quarter are:

  • Increase in deposits on previous quarter and quarter 3 2014/15
  • Increase in Open Access items and items with Full text 

PRO-2015-6-q3-report

Deposits

702 deposits over the last quarter is a significant rise over the previous quarter (381) and on the same quarter of the previous year (545).  ORO deposits have historically fluctuated with increased faculty engagement, internal drivers (research audits), external requirements (REF) and the ebb and flow of new staff joining the OU. (Deposit counts can be influenced by the enthusiasm of one or two new members of staff especially if they are in a discipline that generates a lot of research outputs!)  However, the requirement to deposit in a timely fashion for the HEFCE Open Access Policy should see deposits remain at a consistent level… we’ll see.

% Open Access & % Full text

I’ve included both % Open Access and % Full text data for the first time.  % Open Access indicates the % of items with freely available documents, % full text indicates the % of items with any attachment – these attachments may, or may not, be freely downloadable and will include those papers that are under a publisher embargo.  There has been a steep increase in the % full text indicator over the last 2 quarters (72% and 69% of items include full text) and I associate that directly with an increase in embargoed full text being archived in ORO.  Again, I’m seeing this as an indicator of engagement with the HEFCE Open Access Policy.

PDF of the quarterly report is available: ORO Quarterly Report Q3 2015-16

ORO Annual Update 2014-15

I’ve collected the data for the ORO Annual Report 2014-15. ORO 2014-15 Annual Report Highlights include:

  • 15% increase in deposits over previous year
  • Over 1 million downloads from ORO for the second year in succession
  • Open Access deposits reaching 73% in KMi

ORO continues to punch above its weight compared to other Institutional Repositories. The latest ranking from The Ranking Web of Repositories puts ORO 7th of 146 repositories in the UK.  In this list we are behind only 3 other University Repositories (University of Southampton, LSE & UCL). Additionally the full text figure (i.e. publicly available open access paper and papers restricted by a publisher’s embargo) reached 51% in 2014-15. This is significant as it indicates OU researchers are engaging with the HEFCE Open Access policy and depositing papers earlier in the publication lifecycle when they are likely to be subject to a publisher’s embargo. The HEFCE policy comes into force in April 2016.  It’s going to be a busy year for the ORO service! PDF version of the Annual Report is here: ORO 2014-15 Annual Report

Quarterly ORO Report

ORO 2015 Q4 Report

 

I’ve completed the quarterly ORO report for final quarter of 2014-15.  Highlights include:

Deposits

The last quarter saw a significant increase in the number of deposits to ORO; over 750 items were deposited in the quarter which is one of the highest counts we’ve ever had.

Open Access

Even though the percentage open access fell from the previous quarter we still see an extraordinarily high level of Open Access deposits from both KMi (71%) and IET (62%).

In fact our overall level of 27% of content being fully Open Access compares well with the percentage of Open Access content in similar repositories.  A quick survey of a set of similar repositories gives an average Open Access level of 24 % (based on data from 39 hybrid UK eprint repositories).

 

PDF of the quarterly report is available here: ORO 2015 Q4 Report

 

The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management

Over the last few years there has been lots of discussion about the use of metrics in research assessment.  An Independent review was commissioned by the government in April 2014 to investigate the current and potential future roles that quantitative indicators can play in the assessment and management of research. Through 15 months of consultation and evidence-gathering, the review looked in detail at the potential uses and limitations of research metrics and indicators, exploring the use of metrics within institutions and across disciplines. Its report, ‘The Metric Tide’, was published in July 2015

One of the main findings of the report is that no metric can provide a like-for-like replacement for REF peer review though carefully selected indicators can complement decision making.

The report also recommends that ORCID ID’s should be mandatory for all researchers submitted to the REF.  ORCID’s are permanent identifiers for researchers and enable researchers to protect their unique scholarly identity. Many funders and journals have adopted ORCID’s so it is worthwhile getting one if you haven’t already.

 

Top ORO Downloads for June

Here are the top ORO downloads for June.

Again certain faculties are very well represented in the list, which indicates to me the value of an institutional repository to particular disciplines not served by well established subject repositories.  I’ll pick this up next month when I produce some statistics for the University year and I’ll post findings to the blog.

OROTop15DownloadsJune2015-1

PDF is available here: OROTop15DownloadsJune2015

 

Ranking Altmetrics

Thanks to a handy post on DIY altmetrics I’ve put together a list of top OU publications as ranked by their Altmetric score.  An Altmetric score is calculated by the number of mentions a paper receives in the social media and news outlets (e.g. blogs, Twitter, Facebook, Google+, message boards and mainstream newspapers).  The basic claim of altmetrics is that it measures the reach of research in the wider world beyond academia.  

I’ve posted the top 20 papers (co-)authored by an OU researcher below.  A long list (Digital Object Identifiers, titles and ORO IDs only) is here: ORO Altmetrics Full List.

Fascinating reading, the top 20 is dominated by multi-authored Physical Science papers, frequently in high impact factor journal titles.  And that’s got to be no great surprise given the strength of OU research in this field.  But there are interesting exceptions both in the Top 20 and in the long list!

In order for a publication to appear it has to have a DOI and it has to be archived in ORO – that’s where i got the DOIs!  The raw data is in this google sheet.

The Top 20 OU Publications as ranked by Altmetrics

Sager, William W.; Zhang, Jinchang; Korenaga, Jun; Sano, Takashi; Koppers, Anthony A. P.; Widdowson, Mike and Mahoney, John J. (2013). An immense shield volcano within the Shatsky Rise oceanic plateau, northwest Pacific Ocean. Nature Geoscience, 6 pp. 976–981.

McLennan, S. M.; Anderson, R. B.; Bell III, J. F.; Bridges, J. C.; Calef III, J. F.; Campbell, J. L.; Clark, B. C.; Clegg, S.; Conrad, P.; Cousin, A.; Des Marais, D. J.; Dromart, G.; Dyar, M. D.; Edgar, L. A.; Ehlmann, B. L.; Fabre, C.; Forni, O.; Gasnault, O.; Gellert, R.; Gordon, S.; Grant, J. A.; Grotzinger, J. P.; Gupta, S.; Herkenhoff, K. E.; Hurowitz, J. A.; King, P. L.; Le Mouélic, S.; Leshin, L. A.; Léveille, R.; Lewis, K. W.; Mangold, N.; Maurice, S.; Ming, D. W.; Morris, R. V.; Nachon, M.; Newsom, H. E.; Olilla, A. M.; Perrett, G. M.; Rice, M. S.; Schmidt, M. E.; Schwenzer, S. P.; Stack, K.; Stolper, E. M.; Sumner, D. Y.; Treiman, A. H.; VanBommel, S.; Vaniman, D. T.; Vasavada, A.; Wiens, R. C.; Yingst, R. A. and Science Team, MSL (2014). Elemental geochemistry of sedimentary rocks at Yellowknife Bay, Gale Crater, Mars. Science, 343(6169), article no. 1244734.

Braga-Ribas, F.; Sicardy, B.; Ortiz, J. L.; Snodgrass, C.; Roques, F.; Vieira-Martins, R.; Camargo, J. I. B.; Assafin, M.; Duffard, R.; Jehin, E.; Pollock, J.; Leiva, R.; Emilio, M.; Machado, D. I.; Colazo, C.; Lellouch, E.; Skottfelt, J.; Gillon, M.; Ligier, N.; Maquet, L.; Benedetti-Rossi, G.; Ramos Gomes, A.; Kervella, P.; Monteiro, H.; Sfair, R.; El Moutamid, M.; Tancredi, G.; Spagnotto, J.; Maury, A.; Morales, N.; Gil-Hutton, R.; Roland, S.; Ceretta, A.; Gu, S.-h.; Wang, X.-b.; Harpsøe, K.; Rabus, M.; Manfroid, J.; Opitom, C.; Vanzi, L.; Mehret, L.; Lorenzini, L.; Schneiter, E. M.; Melia, R.; Lecacheux, J.; Colas, F.; Vachier, F.; Widemann, T.; Almenares, L.; Sandness, R. G.; Char, F.; Perez, V.; Lemos, P.; Martinez, N.; Jørgensen, U. G.; Dominik, M.; Roig, F.; Reichart, D. E.; LaCluyze, A. P.; Haislip, J. B.; Ivarsen, K. M.; Moore, J. P.; Frank, N. R. and Lambas, D. G. (2014). A ring system detected around the Centaur (10199) Chariklo. Nature, 508(7494) pp. 72–75.

Sierks, Holger; Barbieri, Cesare; Lamy, Philippe L.; Rodrigo, Rafael; Koschny, Detlef; Rickman, Hans; Keller, Horst Uwe; Agarwal, Jessica; A’Hearn, Michael F.; Angrilli, Francesco; Auger, Anne-Therese; Barucci, M. Antonella; Bertaux, Jean-Loup; Bertini, Ivano; Besse, Sebastien; Bodewits, Dennis; Capanna, Claire; Cremonese, Gabriele; Da Deppo, Vania; Davidsson, Björn; Debei, Stefano; De Cecco, Mariolino; Ferri, Francesca; Fornasier, Sonia; Fulle, Marco; Gaskell, Robert; Giacomini, Lorenza; Groussin, Olivier; Gutierrez-Marques, Pablo; Gutiérrez, Pedro J.; Güttler, Carsten; Hoekzema, Nick; Hviid, Stubbe F.; Ip, Wing-Huen; Jorda, Laurent; Knollenberg, Jörg; Kovacs, Gabor; Kramm, J. Rainer; Kührt, Ekkehard ; Küppers, Michael; La Forgia, Fiorangela; Lara, Luisa M.; Lazzarin, Monica; Leyrat, Cédric; Lopez Moreno, Josè J.; Magrin, Sara; Marchi, Simone; Marzari, Francesco; Massironi, Matteo; Michalik, Harald; Moissl, Richard; Mottola, Stefano; Naletto, Giampiero; Oklay, Nilda; Pajola, Maurizio; Pertile, Marco; Preusker, Frank; Sabau, Lola; Scholten, Frank; Snodgrass, Colin; Thomas, Nicholas; Tubiana, Cecilia; Vincent, Jean-Baptiste; Wenzel, Klaus-Peter; Zaccariotto, Mirco and Pätzold, Martin (2015). On the nucleus structure and activity of comet 67P/Churyumov-Gerasimenko. Science, 347(6220), article no. aaa1044.

Williams, R. M. E.; Grotzinger, J. P.; Dietrich, W. E. ; Gupta, S.; Sumner, D. Y; Wiens, R. C.; Mangold, N.; Malin , M. C.; Edgett, K. S.; Maurice, S.; Forni, O.; Gasnault, O.; Ollila, A.; Newsom, H. E.; Dromart, G.; Palucis, M. C.; Yingst, R. A.; Anderson, R. B.; Herkenhoff, K. E.; Le Mouelic, S. ; Goetz, W.; Madsen, M. B.; Koefoed, A.; Jensen, J. K.; Bridges, J. C.; Schwenzer, S. P.; Lewis , K. W.; Stack, K. M.; Rubin, D.; Kah, L. C.; Bell III, J. F.; Farmer, J. D.; Sullivan, R.; Van Beek, T.; Blaney, D. L.; Pariser, O. and Deen, R. G. (2013). Martian Fluvial Conglomerates at Gale Crater. Science, 340(6136) pp. 1068–1072.

Farley, K. A.; Malespin, C.; Mahaffy, P.; Grotzinger, J. P.; Vasconcelos, P. M.; Milliken, R. E.; Malin, M.; Edgett, K. S.; Pavlov, A. A.; Hurowitz, J. A.; Grant, J. A.; Miller, H. B.; Arvidson, R.; Beegle, L.; Calef, F.; Conrad, P.; Dietrich, W. E.; Eigenbrode, J.; Gellert, R.; Gupta, S.; Hamilton, V.; Hassler, D. M.; Lewis, K. W.; McLennan, S. M.; Ming, D.; Navarro-González, R.; Schwenzer, S. P.; Steele, A.; Stolper, D. M.; Sumner, D. Y.; Vaniman, D.; Vasavada, A.; Williford, K.; Wimmer-Schweingruber, R. F. and Science Team, MSL (2014). In situ radiometric and exposure age dating of the Martian surface. Science, 343(6169), article no. 1247166.

Johnson, Diane; Tyldesley, Joyce; Lowe, Tristan; Withers, Philip. J. and Grady, Monica M. (2013). Analysis of a prehistoric Egyptian iron bead with implications for the use and perception of meteorite iron in ancient Egypt. Meteoritics & Planetary Science, 48(6) pp. 997–1006.

Rotundi, Alessandra; Sierks, Holger; Della Corte, Vincenzo; Fulle, Marco; Gutierrez, Pedro J.; Lara, Luisa; Barbieri, Cesare; Lamy, Philippe L.; Rodrigo, Rafael; Koschny, Detlef; Rickman, Hans; Keller, Horst Uwe; López-Moreno, José J.; Accolla, Mario; Agarwal, Jessica; A’Hearn, Michael F.; Altobelli, Nicolas; Angrilli, Francesco; Barucci, M. Antonietta; Bertaux, Jean-Loup; Bertini, Ivano; Bodewits, Dennis; Bussoletti, Ezio; Colangeli, Luigi; Cosi, Massimo; Cremonese, Gabriele; Crifo, Jean-Francois; Da Deppo, Vania; Davidsson, Björn; Debei, Stefano; De Cecco, Mariolino; Esposito, Francesca; Ferrari, Marco; Fornasier, Sonia; Giovane, Frank; Gustafson, Bo; Green, Simon F.; Groussin, Olivier; Grün, Eberhard; Güttler, Carsten; Herranz, Miguel L.; Hviid, Stubbe F.; Ip, Wing; Ivanovski, Stavro; Jerónimo, José M.; Jorda, Laurent ; Knollenberg, Joerg; Kramm, Rainer; Kührt, Ekkehard; Küppers, Michael; Lazzarin, Monica; Leese, Mark R.; López-Jiménez, Antonio C.; Lucarelli, Francesca; Lowry, Stephen C.; Marzari, Francesco; Epifani, Elena Mazzotta; McDonnell, J. Anthony M.; Mennella, Vito; Michalik, Harald; Molina, Antonio; Morales, Rafael; Moreno, Fernando; Mottola, Stefano; Naletto, Giampiero; Oklay, Nilda; Ortiz, José L.; Palomba, Ernesto; Palumbo, Pasquale; Perrin, Jean-Marie; Rodríguez, Julio; Sabau, Lola; Snodgrass, Colin; Sordini, Roberto; Thomas, Nicolas; Tubiana, Cecilia; Vincent, Jean-Baptiste; Weissman, Paul; Wenzel, Klaus-Peter; Zakharov, Vladimir and Zarnecki, John C. (2015). Dust measurements in the coma of comet 67P/Churyumov-Gerasimenko inbound to the Sun. Science, 347(6220), article no. 905.

Thomas, Nicolas; Sierks, Holger; Barbieri, Cesare; Lamy, Philippe L.; Rodrigo, Rafael; Rickman, Hans; Koschny, Detlef; Keller, Horst Uwe; Agarwal, Jessica; A’Hearn, Michael F.; Angrilli, Francesco; Auger, Anne-Therese; Barucci, M. Antonella; Bertaux, Jean-Loup; Bertini, Ivano; Besse, Sebastien; Bodewits, Dennis; Cremonese, Gabriele; Da Deppo, Vania; Davidsson, Björn; De Cecco, Mariolino; Debei, Stefano; El-Maarry, Mohamed Ramy; Ferri, Francesca; Fornasier, Sonia; Fulle, Marco; Giacomini, Lorenza; Groussin, Olivier; Gutierrez, Pedro J.; Güttler, Carsten; Hviid, Stubbe F.; Ip, Wing-Huen; Jorda, Laurent; Knollenberg, Jörg; Kramm, J.-Rainer; Kührt, Ekkehard; Küppers, Michael; La Forgia, Fiorangela; Lara, Luisa M.; Lazzarin, Monica; Moreno, Josè J. L.; Magrin, Sara; Marchi, Simone; Marzari, Francesco; Massironi, Matteo; Michalik, Harald; Moissl, Richard; Mottola, Stefano; Naletto, Giampiero; Oklay, Nilda; Pajola, Maurizio; Pommerol, Antoine; Preusker, Frank; Sabau, Lola; Scholten, Frank; Snodgrass, Colin; Tubiana, Cecilia; Vincent, Jean-Baptiste and Wenzel, Klaus-Peter (2015). The morphological diversity of comet 67P/Churyumov-Gerasimenko. Science, 347(6220), article no. aaa0440.

Schulte, Peter; Alegret, Laia; Arenillas, Ignacio; Arz, José A.; Barton, Penny J.; Bown, Paul R.; Bralower, Timothy J.; Christeson, Gail L.; Claeys, Philippe; Cockell, Charles S.; Collins, Gareth S.; Deutsch, Alexander; Goldin, Tamara J.; Goto, Kazuhisa; Grajales-Nishimura, José M.; Grieve, Richard A. F.; Gulick, Sean P. S.; Johnson, Kirk R.; Kiessling, Wolfgang; Koeberl, Christian; Kring, David A.; MacLeod, Kenneth G.; Matsui, Takafumi; Melosh, Jay; Montanari, Alessandro; Morgan, Joanna V.; Neal, Clive R.; Nichols, Douglas J.; Norris, Richard D.; Pierazzo, Elisabetta; Ravizza, Greg; Rebolledo-Vieyra, Mario; Reimold, Wolf Uwe; Robin, Eric; Salge, Tobias; Speijer, Robert P.; Sweet, Arthur R.; Urrutia-Fucugauchi, Jaime; Vajda, Vivi; Whalen, Michael T. and Willumsen, Pi S. (2010). The Chicxulub asteroid impact and mass extinction at the Cretaceous-Paleogene boundary. Science, 327(5970) pp. 1214–1218.

Hastings, Gerard (2012). Why corporate power is a public health priority. British Medical Journal, 345 e5124.

Aoudjehane , H. Chennaoui; Avice, G.; Barrat, J.-A.; Boudouma, O.; Chen, G.; Duke, M.J.M; Franchi, I. A.; Gattacceca, J.; Grady, M. M.; Greenwood, R. C.; Herd, C. D. K.; Hewins, R.; Jambon, A.; Marty, B.; Rochette, P.; Smith, C. L.; Sautter, V.; Verchovsky, A.; Weber , P. and Zanda, B. (2012). Tissint martian meteorite: a fresh look at the interior, surface, and atmosphere of Mars. Science, 338(6108) pp. 785–788.

Sutherland, William J.; Bellingan, Laura; Bellingham, Jim R.; Blackstock, Jason J.; Bloomfield, Robert M.; Bravo, Michael; Cadman , Victoria M.; Cleevely, David D.; Clements, Andy; Cohen, Anthony; Cope, David R.; Daemmrich, Arthur A.; Devecchi, Cristina; Diaz Anadaon, Laura; Denegri, Simon; Doubleday, Robert; Dusic, Nicholas R.; Evans, Robert J.; Feng, Wai Y.; Godfray, H. Charles J.; Harris, Paul; Hartley, Sue E.; Hester, Alison J.; Holmes, John; Hughes, Alan; Hulme, Mike; Irwin, Colin; Jennings, Richard C.; Kass, Gary S.; Littlejohns, Peter; Marteau, Theresa M.; McKee, Glenn; Millstone, Erik P.; Nuttall, William J.; Owens, Susan; Parker, Miles M.; Pearson, Sarah; Petts, Judith; Ploszek, Richard; Pullin, Andrew S.; Reid, Graeme; Richards, Keith S.; Robinson, John G.; Shaxson, Louise; Sierra, Leonor; Smith, Beck G.; Spiegelhalter, David J.; Stilgoe, Jack; Stirling, Andy; Tyler, Christopher P.; Winickoff, David E. and Zimmern, Ron L. (2012). A collaboratively-derived science-policy research agenda. PLoS ONE, 7(3)

Kandasamy, Narayanan; Hardy, Ben; Page, Lionel; Schaffner, Markus; Graggaber, Johann; Powlson, Andrew S.; Fletcher, Paul C.; Gurnell, Mark and Coates, John (2014). Cortisol shifts financial risk preferences. Proceedings of the National Academy of Sciences, 111(9) pp. 3608–3613.

Jenniskens, Peter; Fries, Mark D.; Qing-Zhu , Yin; Zoleensky, Michael; Krot , Alexander N.; Sanford, Scott A.; Sears, Derek; Bauford, Robert; Ebel, Denton S.; Friedrich, Jon M.; Nagashima, Kazuhide; Wimpenny, Josh; Yamakawa, Akane; Nishiizumi, Kunihiko; Hamajima , Yasunori; Caffee, Marc; Weten, Kees C.; Laubenstein, Matthias; Davis, Andrew; Simon, Stiven B.; Heck, Philipp R.; Young , Edward D.; Kohl , Issaku E.; Thimens , Mark H.; Nunn, Morgan H.; Mikouchi, Takashi; Hagiya, Kenji; Ohsumi, Kazumasa; Cahill, Thomas A.; Lawton, Jonathan A.; Barnes , David; Steele, Andrew; Rochette, Pierre; Verosub, Kenneth L.; Gattacceca, Jerome; Cooper , George; Glavin, Daniel P.; Burton, Aaron S.; Dworkin, Jason P.; Elsila, Jamie E. ; Pizzerello, Sandra; Ogliore, Ryan; Schitt-Kopplin, Phillipe; Harir, Mourad; Hertkorn, Norbert; Verchovsky, Sasha; Grady, Monica; Nagao, Keisuke; Okazaki, Ryuji; Takechi, Hiroyuki; Hirori, Takahiro; Smith, Ken; Silber, Elizabeth A.; Brown, Peter G; Albers, Jim; Klotz , Doug; Hankey, Mike; Matson, Robert; Fries, Jeffrey A.; Walker , Richard J.; Puchtel, Igor; Lee, Cin-Ty A.; Erdman, Monica E.; Eppich, Gary R.; Roeske, Sarah; Gabelica, Zelimir; Lerche, Michael; Nuevo, Michel; Girten, Beverly and Worden, Simon P. (2012). Radar-enabled recovery of the Sutter’s Mill meteorite, a carbonaceous chondrite regolith breccia. Science, 338(6114) pp. 1583–1587.

Zhang, Fucheng; Kearns, Stuart L.; Orr, Patrick J.; Benton, Michael J.; Zhou, Zhonghe; Johnson, Diane; Xu, Xing and Wang, Xiaolin (2010). Fossilized melanosomes and the colour of Cretaceous dinosaurs and birds. Nature, 463(7284) p. 1075.

Roelfsema, Martine T.; Hoekstra, Rosa A.; Allison, Carrie; Wheelwright, Sally; Brayne, Carol; Matthews, Fiona E. and Baron-Cohen, Simon (2012). Are autism spectrum conditions more prevalent in an information-technology region? A school-based study of three regions in the Netherlands. Journal of Autism and Developmental Disorders, 42(5) pp. 734–739.

Sierks, H.; Lamy, P.; Barbieri, C.; Koschny, D.; Rickman, H.; Rodrigo, R.; A’Hearn, M. F.; Angrilli, F.; Barucci, M. A.; Bertaux, J.- L.; Bertini, I.; Besse, S.; Carry, B.; Cremonese, G.; Da Deppo, V.; Davidsson, B.; Debei, S.; De Cecco, M.; De Leon, J.; Ferri, F.; Fornasier, S.; Fulle, M.; Hviid, S. F.; Gaskell, R. W.; Groussin, O.; Gutierrez, P.; Ip, W.; Jorda, L.; Kaasalainen, M.; Keller, H. U.; Knollenberg, J.; Kramm, R.; Kuhrt, E.; Kuppers, M.; Lara, L.; Lazzarin, M.; Leyrat, C.; Moreno, J. J. L.; Magrin, S.; Marchi, S.; Marzari, F.; Massironi, M.; Michalik, H.; Moissl, R.; Naletto, G.; Preusker, F.; Sabau, L.; Sabolo, W.; Scholten, F.; Snodgrass, C.; Thomas, N.; Tubiana, C.; Vernazza, P.; Vincent, J.- B.; Wenzel, K.- P.; Andert, T.; Patzold, M. and Weiss, B. P. (2011). Images of asteroid 21 Lutetia: a remnant planetesimal from the early solar system. Science, 334(6055) pp. 487–490.

Rose, Gillian (2014). On the relation between ‘visual research methods’ and contemporary visual culture. Sociological Review, 62(1) pp. 24–46.

Spiegelhalter, David; Pearson, Mike and Short, Ian (2011). Visualizing uncertainty about the future. Science, 333(6048) pp. 1393–1400.