Metrics for non-traditional research outputs
Catherine Radbourne
What are non-traditional research outputs (NTROs)?
NTROs comprise a wide variety of outputs that differ in their form and mode of production. They include creative work as research and non-traditional outputs of traditional research that range from highly experimental works of creative art – music or visual art, creative writing, dance or design – through to scholarly editions and translations, website creation, exhibitions of archaeological or scientific objects and commissioned reports. (University of Sydney Library, n.d)
For researchers producing NTROs, they often need to be more proactive from the outset with regard to compiling metrics for their research output because of the different sources they are coming from.
Getting started
(Adapted from the University of Sydney Library, Strategic Publishing Toolkit)
NTROs are disseminated through diverse means that can vary widely according to form and discipline. Pathways to dissemination may be less well-defined than for traditional publications. As a result, it can be difficult to generalise about selecting the best outlet for your work. (University of Sydney Library, n.d)
For researchers needing measures on their NTROs, some examples that could be considered for different types of outputs are below.
- Evaluate the quality of the gallery, venue, event, or outlet in terms of reputation and reach. For example, who are the people or organisation responsible and how are they regarded by their peers and mentors? What contacts and regular clients do they have? Are their events or exhibitions reviewed or covered by the media? What was provided in terms of advertising and promotion?
- For research reports written for external bodies, is the body widely known and well regarded in their field? How was the report disseminated and promoted? Do they have download statistics? Are there a number of websites the report sits on/links to?
- Was support for their outreach activities provided? For example, were promotional materials produced that were shared online? Is a catalogue, recordings or photography available? Is there an accompanying website that can be linked to in social media posts or on their personal website? How many views has it had? How much engagement was there from social media?
- Were they permitted to make a digital copy of their output available via an online repository like QUT eprints? How many downloads has it had and where from?
- Do they have access to data to support the evaluation of the success of the output and their strategy? This might include attendance or ticket sales numbers, website visits or downloads. If this data has not been collected by the organisation or platform, did they have ways that they recorded their own data?
Consider: What other sources might you use to help a researcher demonstrate the reach of their NTRO?
Learn more
The University of Melbourne has an excellent guide on Research Impact for Fine Arts and Music. Pay particular attention to the sections on NTROs (remember, this can be more than just a creative performance).
Exercise: Use a checklist from the University of Melbourne guide for one of the different types of NTROs (e.g. Exhibitions). Search in QUT ePrints Advanced Search (e.g. type ‘women’ into the keywords search box, scroll down to Item Type and check box – Non-traditional Research Output, then click search) to find an NTRO that suits that checklist and see what metrics are available for that record? (click on ‘More statistics’)
If no metrics are available in the research portal, where would you look to find metrics or other evidence of reach for this output? If possible, find some evidence of reach or impact for the output you have selected. How easy or how difficult was it to track down the information?
Challenge me
Attribution
Content in this chapter has been developed by QUT Library, including content derived from:
- “University of Sydney Library, Strategic Publishing Toolkit – Conferences & NTROs” Used with permission.
All information correct at time of publishing 10 January 2022.
ImaGE cREDITS
Royalty-free images used on this page were sourced from unsplash.com.
Icons created by priyanka and Wichai Wi from Noun Project.