Cultural heritage institutions such as libraries, archives, museums and others have been increasingly making content available digitally over the last few decades. Digital heritage can be described as ‘unique resources of human knowledge and expression’ (UNESCO, 2009) in digital form. Making heritage available in this manner creates ‘a new ecosystem of commemorative practices and collective remembering’ (Burkey, 2022, p. 186) and leads to engagement with user communities.

Stellenbosch University Library and Information Service in South Africa is no exception. Its digital heritage repository, SUNDigital Collections,^[1] was established in 2013 and hosts historical documents, images, music collections, indigenous knowledge collections and finding aids for these collections to increase the visibility of special and unique collections held by the library. The repository runs on DSpace open-source software (version 6.3) and makes use of the open-source analytics plugin Matomo for statistics.

Use of the repository is tracked and shows approximately 40,000 to 50,000 visits per annum and approximately 20,000 to 40,000 downloads per annum. These statistics exclude bots, as all bots are blocked from directly crawling the SUNDigital Collections bitstream store (i.e. full text files) and there is no evidence of SQL injection (i.e. item-level access) in the log files to make us suspect that the usage represented by these statistics is in any way misrepresented or inflated due to bot visits. These statistics, however, do not relate anything about the impact of the repository.

Research Impact

Research impact, or societal impact, can be described as the influence work has beyond academia, such as on society, economy, culture, public policy, technology and the environment (University of Sydney Library, 2024).

According to Shaw (2016), much evidence of the value of digitising collections remains anecdotal, but the demonstration of impact is important for informing internal decision making and justifying digitisation activities.

In terms of institutional repositories, where research outputs (e.g. theses, dissertations, articles and conference papers) are hosted, it is fairly easy to establish impact. Views, downloads and mainly citations are analysed, as it is common in academia that a high citation count is an indicator of research impact. Bibliometrics and Altimetric are also employed for the investigation. We link this impact to a potential increase in institutional rankings and our goal, which is knowledge in the service of society.

It is more difficult to define or demonstrate impact in terms of digital heritage collections. According to Terras (2015), it is problematic to fully understand the reuse of digitised content, and various research methods, quantitative, such as bibliometric and log analysis, and qualitative, such as user feedback and surveys, have been employed.

Measuring Impact

Borrego (2020) suggests that analysing citations from scholarly outputs in Google Scholar may be used to measure the impact of digital heritage collections. The methodology is simple, as ‘Google Scholar indexes the full-text [sic] of documents, facilitating the retrieval of citations inserted in the text or in sections that are not the final list of references’ (Borrego, 2020, p. 1). Inserting part of the URL of the digital repository in the search box yields results. In the case of SUNDigital Collections, the search, conducted in May 2024, yielded 52 results. These included masters and doctoral theses and dissertations, research articles, books, book chapters and an online encyclopaedia entry. Some of these research outputs show 10 or more citations, and these in turn may also be investigated to assess impact. As Borrego (2020) proposes, this method does provide evidence of the academic impact heritage collections among researchers and students.

Investigating user statistics in the Matomo analytics plugin is also a method of measuring impact since the system shows links to digital heritage collections and items in the repository from outside websites. Matomo also provides the opportunity to set goals. This could, for example, be triggered when a user visits a certain page title or downloads a certain file, giving automated statistics to specific collections the use of which one would want to track. In the case of SUNDigital Collections, these included links from authoritative sites such as heritage and genealogical portals, websites on South African history and Wikipedia. Links are spread across many records in different collections. Patterns of use also appear in usage statistics, pointing to increased reuse of specific collections or items. These patterns may be explored and often point to specific research projects being conducted by, for example, groups of students.

Bangani et al. (2020) examined acknowledgements in electronic theses and dissertations as a way to determine the value of libraries’ and librarians’ contributions to postgraduate studies. Similarly, the value of specific collections could also be determined. Bangani et al. (2020), however, warn that acknowledgements as a measure of impact should be used with caution, as it is not necessarily a complete reflection of the extent of contribution.

Increasing Impact

Terras (2015) argues that open access and an open approach to the delivery of digitised cultural heritage allows for sharing reuse and fosters development in research across the sector. It seems obvious that making heritage available should be the default, but Terras (2015) suggests that there remain barriers to this, such as works still subject to copyright, orphan works and the idea of what this openness means for institutions in a monetary and reputational sense. In the case of SUNDigital Collections, all records are openly accessible, and no restricted content is considered for upload.

Increasing visibility by means of search engine optimisation is something to consider in order to potentially increase impact. In the case of SUNDigital Collections, sitemaps were created and Google metadata fields were mapped to Dublin Core metadata fields, which we use for description in the repository. A search for a collection in SUNDigital produced a link on the first page of results on various search engines. On Bing and DuckDuckGo, the link appeared second on the list of results, and in the case of Google Chrome, fourth. This ensured that repositories are registered with directories and registries, such as ROARMAP, a registry of open-access repositories and accompanying policies (ROARMAP, 2024), and OpenDOAR, a global directory of Open Access repositories (OpenDOAR, 2024).

Kelly (2019) writes that adding digital heritage collections and links to Wikimedia Commons and Wikipedia is a method for increasing access to digital heritage collections. This is an approach that the library will consider doing in the future. Results show that uploading data to Wikimedia Commons can result in greater reuse of this data outside of Wiki projects (Kelly, 2019), which can in turn lead to increased impact. Enriching metadata by using linked open data may have a similar effect, according to Candela et al. (2019). The library is currently undertaking the conversion of AACR2 bibliographic records on its discovery system to resource description and access (RDA) to improve the description of and access to library and cultural heritage resources.

Marsh et al. (2016) argue that storytelling is also a means of understanding and articulating the impact of digitising ethnographic and heritage materials and view it as a ‘key component of impact assessment’ (Marsh et al, 2016). Linked to this, storytelling and engagement can play a role in enhancing impact, especially when employing citizen science or crowdsourcing, as demonstrated by Adam et al. (2025) and Colla (2025) in their contributions relating to sourcing members of the public and library staff members to engage with collections and enrich descriptions of digital heritage objects. In both cases, this method leads to improved engagement with collections and could increase use and, ultimately, impact.

Conclusion

This study has demonstrated that while assessing the impact of digital heritage collections presents more challenges compared with institutional research outputs, viable methodologies do exist. By leveraging both quantitative and qualitative measures – such as citation analysis, user statistics and engagement patterns – it is possible to gain valuable insights into the influence of these collections within and beyond academia.

Repository managers and cultural heritage institutions should not only focus on measuring impact but also actively explore strategies to enhance the visibility, accessibility and engagement with digital heritage materials. Implementing search engine optimisation, linking collections to platforms such as Wikipedia and Wikimedia Commons, and integrating linked open data can significantly contribute to broader dissemination and reuse. Furthermore, storytelling and participatory approaches, such as citizen science and crowdsourcing, have shown promise in fostering deeper interactions with digital heritage collections.

However, it is crucial to approach impact assessment with a critical lens, acknowledging the limitations of purely numerical metrics. Statistical indicators should be interpreted in context rather than being taken at face value. Additionally, future research could benefit from the use of automated methods to track scholarly and public engagements with digital heritage collections more comprehensively.

By continually refining both measurement techniques and outreach strategies, digital heritage repositories can strengthen their role as vital resources that contribute to scholarship, cultural preservation and public knowledge on a global scale.

References

Abstract

Measuring the impact of digital heritage collections remains a complex challenge compared with traditional institutional research outputs. Although bibliometric and Altimetric indicators provide straightforward measures for academic publications, assessing the influence of digital heritage materials requires a more nuanced approach. This study explores methodologies for evaluating the impact of digital heritage repositories, focusing on both quantitative and qualitative measures. Using Stellenbosch University’s SUNDigital Collections as a case study, the paper examines citation analysis via Google Scholar, web analytics through Matomo, and external referencing patterns to assess scholarly and broader public engagement. The findings highlight the importance of increasing the visibility of digital heritage collections through open-access strategies, search engine optimisation and integration with platforms such as Wikipedia and Wikimedia Commons. Additionally, the role of storytelling, citizen science and metadata enrichment in enhancing the discoverability and reuse of digital heritage materials is discussed. The study concludes that while statistical indicators provide valuable insights, they should be interpreted within a broader contextual framework. Future research should explore automated tracking methods to better understand the evolving impact of digital heritage collections across academic and public domains.

Keywords

Digital heritage; Accessibility of heritage collection; Impact assessment; Bibliometrics; User engagement