Information Studies Research

Generic ICT Usage Issues

Audience Assessment: Evaluation Studies and Market Research

Most heritage organisations undertake the assessment of the commercial viability of projects and the appraisal of economic, cultural, and social markets as an afterthought. Various assessment methods have been applied:

• visitor focus groups;
• communication analysis;
• front-end, formative and summative evaluations;
• visitor and non-visitor surveys; and
• positioning and product surveys.

Heritage projects often waste money pursuing inappropriate objectives or failing to satisfy audience demand. A marked variation exists between those institutions which recognised the importance of carrying out these kinds of studies and their ability to carry them forward. Proposals need to identify the evaluation strategies that will be used by the project at appropriate stages (e.g. before proposal development, formative evaluation during project development, and summative evaluation after it has been completed) and how the results of these studies will be used to benefit the project. The formative evaluations must include the development of prototypes and an examination of them by potential users. How the results of the formative evaluation will be reflected in the development programme is of special importance because these evaluation results might lead to changes which increase costs, or time, or both.

10.2 Evaluations carried out before projects are started need to produce evidence as to the commercial viability of the project, so that it is sustainable after HLF funding ceases, and provide some measure of the cultural, social, and economic benefits it will bring about. One of the major problems with evaluation strategies is the lack of comparative evaluation frameworks. Other difficulties include the fact that they tend to be informally managed, that they produce data of varying quality and depth, and that an insufficient number of evaluations are actually ever made public. For ICT projects, these project evaluations should:

• not be purely paper-based exercises;
• give potential users prototypes to respond to;
• require systematic recording of results;
• be managed by an experienced facilitator able to promote discussion and constructive criticism;
• include end-user and visitor focus groups and schools;
• include both non-visitors and non-users of computer-based resources; and
• be published.

Recommendation
 
45. The HLF should require that applications for ICT projects or for the ICT aspects of larger projects be accompanied by:
 
* results of evaluations carried out in the course of the preparation of the application;
* clearly defined strategies for carrying out formative evaluations and for feeding the results of these evaluations into the development process; and
* clearly defined summative evaluation plans including proposals for the dissemination of results of these so that they can inform other developments in this sector.

Risk Assessment

10.3 IT projects are risky. The risks include those associated with:

• processes of management, design, development, testing, deployment, and ongoing maintenance and support;
• product (or application) development such as system design, suitability of hardware and software, whether assumptions about system resource usage made from a sample data set will scale up, and interface design and user response;
• organisational aspects of the project—for example, organisational structure, support requirements, psychological resistance, lines of communication; and
• business activities such as financial, legal, and regulatory risks.

10.4 To manage risks organisations planning to undertake ICT projects should answer the following questions:

• What can go wrong?
• For each risk, what is the likelihood of it occurring and what is the scale of loss that it will cause whether in time or financial terms?
• How and when will the project team know when something has gone wrong?
• What can be done to prevent (i.e. eliminate or avoid) the risk from occurring?
• What will be the knock-on impact if the risk occurs?
• How will risk aversion procedures be put in place?

10.5 The dangers of ICT projects can be reduced by converting unacceptable risks to manageable ones and, as a consequence, making them acceptable risks. To do this it is essential that the applicant understands the risks that can occur and defines strategies for either avoiding or eliminating them.

Recommendation
 
46. The HLF should only fund ICT projects where the applicant has demonstrated through detailed risk assessment that all risks are acceptable.

Project Management Issues

10.6 IT projects are difficult to manage from inception to delivery. Proposers often underestimate the processes involved and their complexity. One of the major objectives in designing ICT projects is to find ways to reduce risk. Good project design and effective project management play a role in keeping the risks (as examined in ¶s 10.3-10.5) under control.

10.7 English Nature’s approach to project management provides a starting point for HLF guidance:

• a clear, early understanding of project objectives, deliverables, budgets, timescales and resources by those funding the project¼ This information can be detailed up front in a formal Project Initiation Document (PID);
• a clear project management structure with responsibilities and tolerances clearly understood, e.g. a properly appointed Project Board with a recognised Chair and an established Project Team with an identified Project Manager;
• a clear project development methodology in order that all involved (including those funding the project) are clear as to the way the project should progress and to monitor how the project has progressed. The English Nature method in broad terms has seven phases: (1) business analysis; (2) systems analysis; (3) system design; (4) system build; (5) system test; (6) deployment; and (7) review.

10.8 Successful projects are the result of careful planning and good project management. Good project management involves detailed analysis of the risks (¶s 10.3-10.5), the use of an acceptable project management standard, such as PRINCE or PRINCE 2, and the establishment of formal project management roles. Scenarios similar to the one described by English Nature have been shown in a wide range of organisations to help promote successful ICT projects.

10.9 The critical importance of teamwork is especially evident in the preparation of heritage-based education materials. Developing educational products depends on a team of individuals each bringing his or her own specialist skills to the process of developing, managing, supporting, and evaluating educational aspects of a museum or heritage environment. The curator needs to be creator, or else be replaced by a multidisciplinary team. Neither technologists nor curators can communicate successfully alone. Multidisciplinary teams composed of curators, technical staff, educationalists, artists, evaluators, and journalists are essential for IT-based products intended for public use.

10.10 Successful project management often reflects the experience of the project management team. It will be essential, therefore, that HLF not only requires that particular emphasis be placed on an adequate management plan in every application, but also that it satisfies itself that the applicants include individuals capable of managing both the project as a whole and the product, such as a database. This should be adequately maintained, provide the information which it is required to provide in a repeatable, sustainable and user-friendly manner, and be capable of use by others not in, or associated with, the initial project.

Recommendations
 
47. The HLF should require that details of other projects managed by the project team be included with all projects.
 
48. The HLF should require successfully that applicants prepare and submit a Project Initiation Document (PID).
 
49. The HLF should require projects to establish a set of measurable and accessible milestones comprising a clear deliverable or set of deliverables linked to each milestone.
 
50. The HLF should require details of the activities or tasks and resources needed to achieve the deliverable targets.

Rights management

10.11 The rights and heritage value of assets must not be jeopardised by the retroconversion of catalogues and lists describing assets, the creation of digital facsimiles of the assets themselves, or the reuse of these digital resources. Where asset income generation is especially important (e.g. where income makes the resources sustainable), every effort must be made to protect the rights an institution holds in the material. There is a full range of rights which heritage sector organisations can use: copyright, trademarks, patents, and registered designs. Heritage organisations must avoid alienating rights; licensing is emerging as the best approach to digital assets. It is feasible to use licensing to control a range of rights and to restrict quite specifically what rights of a digital asset are being licensed, for how long, for what purposes, and under what constraints. There are other rights which are of issue, some of which are quite restricted. Other Acts or regulations which to be considered include:

• the EU Directive on Data Extraction, which effectively limits the control of databases to 15 years;
• The Environment Protection Act 1990 and the Environmental Information Regulations 1992; and
• the Data Protection Act.

Recommendation
 
51. The HLF should require that applicants seeking support for ICT projects which produce digital resources demonstrate that they have in place suitable strategies to protect their rights in the digital aspects of the materials in their care or that they are creating. A licence agreement might provide a starting point.

10.12 To some extent it is not possible at present to secure rights in digital resources satisfactorily. There will always be a tension between spending money to protect content and acting when an organisation finds that its content has been used without permission. The latter may be too late but the former may prove a constant battle as technological protection strategies are compromised and need continual replacement. Three technologies, currently the subject of much research—watermarks, wrappers, and encryption—offer ways of securing the value of assets. Digital watermarks ‘tag’ a digital object with a digital marker that can be used to identify it as the property of an individual or institution. Digital watermarks are analogous to watermarks still used on good bond paper, although the parallel is closest to 16th and 17th century watermarks which actually include data which make it possible to give date and place to the manufacture of paper. Unlike conventional watermarks which are lost when a page is reproduced (e.g. photocopied), a digital watermark stays with the ‘digital object’ (e.g. image, sound, or text file), no matter how many times it is copied. For example, EMI Central Laboratories have built sound coding watermarks that can be inserted into audio material, but do not appear audibly when that material is played, although their presence can be detected in copied material. Images and text can be watermarked in a similar way, although for images there are both invisible and visible watermarks. The problem is that it is difficult to catch those who infringe the rights in the digital world, because to identify misuse requires finding and analysing the images (or other material) that an organisation suspects are being misused. Because the potential commercial losses from piracy of digital materials are so vast, this is an area of intensive research activity.

Preservation

10.13 The long-term value of heritage data is difficult to predict. Time and time again, however, old records demonstrate their continued value. Natural heritage and environmental data often provide extremely important benchmark records to help heritage specialists understand change over time in species’ distributions, density, and comparative habitats. Recently, we have heard much in the popular press about the value of information collected in the Victorian period about the location of dormouse habitats and how these data will help to guide a new study of dormouse distribution. Historic archaeological records, for instance, continue to inform planning for building development and future archaeological work. In supporting the creation of digital resources the HLF should be aware of its obligation to require grantees to retain:

• records, so that they can be used in the future once the systems on which they were produced are obsolete; and
• different versions of digital records. (This is particularly important where a project depends on time series data.)

10.14 The increasing use of computers to collect, store, and manipulate material raises issues about the need to preserve digital resources themselves beyond the life of the projects generating them. The expense and intellectual labour involved in creating digital materials makes this a priority, but the problems associated with securing their long term retention and availability make such preservation, even for the shortest period, difficult. There are many obstacles:

• the variety of standards of capture, compression, storage, presentation, and display;
• the fact that each of these aspects of technology evolves continually;
• that hardware and software rapidly become obsolete;
• that new types of data storage media replace older types leading to the obsolescence of peripheral devices; and
• that all media seem subject to degradation.

Some materials will become inaccessible as the documentation and state of the system diverge or become separated. As a result of this divergence it will prove difficult to reconnect the materials. Some materials will suffer from contextual divergence and other materials will be difficult to preserve because of legal impediments to their preservation. There is a lack of agreed strategies for managing the long-term preservation of digital materials.

Recommendation
 
52. The HLF should remain appraised of the issues involved in digital preservation with a view to ensuring that resources created with funds it distributes remain viable and usable for coming generations.

10.15 Regularly used data resources, including library catalogues, archival finding aids, and collections management data, are the least likely to be at risk of loss. Irregularly used data, such as those held by archaeological and natural heritage bodies, are at greater risk of loss by virtue of the limited frequency with which they are used. The complexities of curating digital information over the long term make it a less than suitable medium for the preservation of our heritage. Preservation of the archaeological record stored in digital form is likely to be an issue of growing significance during the coming five years. In general, archaeological data provide a unique record and as such, like natural heritage data, they need to be preserved. In both cases data independent of particular hardware and software pose fewer obstacles to their survival than data tied to proprietary standards (see below ¶s 10.18-10.24) or hardware and software. Complex data and multimedia, in particular, will pose major difficulties because not only must the data elements be retained but also the interrelationship between them.

10.16 Data are the raw material from which information and, in turn, knowledge are derived. Many data sets once created, could not be compiled again because:

• their creation destroyed the very material being recorded (e.g. in archaeology);
• the data reflect the distribution of a species at a specific point in time;
• the data reflect the conditions of a specific habitat at a particular time; and
• the costs of re-collecting, re-retroconverting, or re-digitising would be difficult to justify a second time.

Not all organisations supported by the HLF will have the skills to curate their digital resources effectively. As a result the HLF will need to investigate whether or not it should invest in data archives capable of preserving and ensuring reuse of outputs from digital resource creation projects in the heritage sector. Data archives mediate between data-creating, data-serving, and data-using communities. Given the diversity of digital materials and the range of expertise required for their preservation, a distributed archival strategy might be an appropriate model. A relevant example is the Arts and Humanities Data Service (AHDS) which is a centrally managed federation of data archives working with archaeological, historical, visual and performing arts, and textual data. The HLF needs to consider whether it should establish or contract for the management of the digital resources that have been created with its funding if they are to be available as a public good. In order to ensure the long term preservation of the digital information created by its funded projects the HLF should consider funding a data archive or federation of data archives to look after them. While not all information has long term value, nearly all heritage information does.

Recommendations
 
53. Applicants should be required to provide evidence of their approach to data preservation.
 
54. The HLF should investigate the feasibility of supporting a national archaeological data service in conjunction with the statutory bodies, including RCHMs, English Heritage, Historic Scotland, and the HEFCs’ Archaeology Data Service.
 
55. The contract between the HLF and the grantee should be amended to state that the life of digital data is indefinite.
 
56. The HLF should examine whether there is a genuine need for a data archive to preserve the digital resources being created by the projects it funds.

10.17 Documentation is essential if we are to maintain access to and understanding of digital resources. This is especially true of those resources that depend on the integral relationships within discrete data units (e.g. structured data sets such as relational databases, multimedia). In all cases documentation of the data formats and coding used will be crucial for future access, reuse, and the migration of data forward, as will data about special software and hardware (e.g. processor and devices).

Recommendation
 
57. The HLF should require that organisations developing digital data sets produce adequate levels of documentation to make certain that the digital materials they are creating will be understood in the future.

Standards

10.18 Standards are crucial for the meaningful and intelligible exchange and comparison of information. They are also key to the long-term viability of digital information if data and information are to be accessible from a wide variety of hardware platforms and software packages for decades. Standards influence the recording, use, and exchange of information across space and time. They can be used to:

• enhance precision in recording of data;
• enhance the consistency of recording;
• improve the exchange of information;
• assist in the migration of old data to new environments; and,
• aid the discovery, retrieval, and selection of information.

Three types of data standards are central to the use of ICT in the heritage sector: capture, encoding, and quality standards. Other standards, including those related to terminology or software, also need to be given consideration.

10.19 Emphasis must be on ‘portable standards’ because no single standard will provide a vehicle for the medium-term preservation of data—say, 25 to 50 years. Data will need to be migrated from each current standard to the next. Migration will need to take place before the newer data formats have been irrevocably separated from older data format standards. Because such migration will be easiest for standards promulgated by standards bodies, data created using these standards may have ‘preservation-advantage’. Only as a last resort should files be stored in proprietary data formats. There are, however, some currently acceptable proprietary formats—for instance, AutoCad file formats for Computer-Aided Design (CAD) and Kodak’s Photo-CD image format. While some formats are de facto industry standards (e.g. BMP, DXF, PNG, GIF, TIFF), others are propagated by standards organisations (JPEG, MPEG). For example, in the botanical sphere existing standards include the International Transfer Formats (ITF) for Botanic Garden Data. As in most subjects, a specialised group oversees the coordination of evolving standards. For taxonomic information, including ITF, standards management has been undertaken by the Taxonomic Databases Working Group (TDWG), a committee of the International Union for Biological Services. Similar models apply in other areas. A wide range of heritage organisations have become active in the selection and, where necessary, the development of standards that meet their information objectives. For example, the archivists are actively promoting ISAD(G), developed by the International Council for Archives. Many museum professionals argue that it is essential that museums conform to Spectrum, which is essentially a procedural standard for the museum sector and, as such, is an integral part of the MGC Registration Scheme. In the area of technical and encoding standards the European Union maintains a Website that is regularly updated. A list of standards bodies appears in Appendix I.

10.20 A whole host of standards have a role in contemporary IT, from image formats to data structures. As it will be impossible for the HLF to keep up to date with emerging standards, it will need to depend on its expert advisers for guidance in this area. It is worth bearing in mind that the hallmarks of good standards are that they should:

• be applicable across numerous platforms and operating systems;
• be widely used;
• be easily understood and applied by non-specialists;
• require no specialised hardware;
• be in use in consumer areas, as these tend to have large enough market penetration to be cost effective and to have migration paths;
• be well documented; and
• be non-proprietary.

10.21 Wherever possible, it is better to encourage organisations to adopt standards rather than create new ones. Besides ensuring the interoperability of resources created by different institutions in the same heritage sectors, existing standards have generally been validated by extensive field use. This contributes to reducing risks associated with IT-based projects.

10.22 One way of improving quality and aiding retrieval is to use terminology resources such as the Getty Art and Architecture Thesaurus (AAT). Modern approaches to terminology development stress the importance of defining concepts within scope notes and by linking them to other concepts rather than concentrating on restrictive vocabulary. Although classification systems and thesauri are constantly evolving, there is usually a clear migration path, and updates are increasingly being made available via the WWW. The highly structured nature of cultural databases permits high-precision data retrieval. Free text searches using uncontrolled keywords will often produce either no results at all (because the wrong term has been used) or far too many results (because context has not been taken into account). Projects such as SCRAN and AQUARELLE, and organisations such as RCHM(E) have recognised the need for terminology resources to aid this process, providing as they do a structured framework for searching and synonyms for the terms entered. AQUARELLE, in particular, has demonstrated the importance of multilingual thesauri within the cultural heritage sector. There is growing national and international collaboration in the provision of resources to meet this well-established need. An alternative view would argue that such exercises bring few benefits, especially when terminology is continually in a state of flux. In the biological area names change, as do classifications, making collation difficult. There are dictionaries giving lists of species names and habitat names. Boundaries, counties, natural areas, also change, and are not centrally recorded by any government body.

10.23 Digitisation is an excellent example of an area where activity needs to be encouraged and the benefits for access and use are easily quantifiable, but where the standards, guidelines and practices are akin to shifting sands. The crucial factor in the development of a national digitisation strategy will be the agreement and adoption of national and international standards and guidelines. At present there are no widely accepted digitisation standards and it is difficult to identify best practice. For example, while it is possible to demonstrate minimal acceptable resolutions for capturing manuscripts and printed material, suitable metadata guidelines for data which need to be associated with the images themselves are only just emerging. Furthermore, standards for preservation may not be suitable for access and vice versa. For digital moving images there is currently no suitable preservation standard and the same may apply to other image formats such as early photographic collections. In the latter case the current digital formats may offer better preservation prospects than the original medium itself.

10.24 We should ensure that the effort we are making will still be there in the future. HLF can never force people to use particular software. But whatever software is used, projects should make sure that data can be transferred from one computing environment to another. A concentration on use of standard systems for standard applications does seem sensible, for instance for Collections Management. Institutions should really have to justify choosing a new system, given that there are already several good systems in use in a wide range of museums.

Recommendations
 
58. The HLF should point potential applicants to standards resources such as those listed in Appendix I but it should not develop its own standards advisory service.
 
59. The HLF should not fund the development of standards.
 
60. The HLF should require applicants to demonstrate that they have selected standards appropriate to the scale, needs, and objectives of their project and to the heritage sector in which the project originates.
 
61. The HLF should require applicants to demonstrate that they have both the technical skills and trained staff to put the selected standards into use.
 
62. The HLF should require applicants to provide evidence that they are documenting the data to a standard sufficient to support its long-term preservation and reuse.
 
63. The HLF should not be directive about standards, but applicants should document why they have chosen a particular standard and explain their reasoning. 
 
64. In all the cases where data capture, creation, or retroconversion are being carried out, the applicant must provide evidence that the work being done conforms to standards of best practice as currently understood.