Lori Stirland-Legret, Digital Archive Assistant, reflects on the last year of digital activities in Special Collections and Theatre Collection for this year’s World Digital Preservation Day.
Today is the first Thursday of November, meaning it’s World Digital Preservation Day 2024! This is a day to celebrate all things relating to digital preservation, and this year’s theme is ‘Preserving Our Digital Content: Celebrating Communities’. It has also been almost exactly a year since I started working as Digital Archive Assistant for Special Collections and Theatre Collection, so it seemed like a good time to reflect on the work I’ve been doing, as well as the importance of digitally preserving our collections.
One of the main tasks that I’ve undertaken is creating a Digital Asset Register to keep information about all of our digital collections in one place. It contains high level information such as how much storage space each collection takes up, and the types of digital files they contain. With a Digital Asset Register, we can get a better idea of which collections to prioritise to make accessible for users.
Making a Digital Asset Register sounds easy until you consider that Special Collections has over 40 terabytes and Theatre Collection has over 50 terabytes of digital content. To put it into perspective, the average laptop usually has a capacity of around 500 gigabytes of storage – meaning it would take almost 200 normal laptops to hold all of our digital materials! Luckily, we have access to a large amount of network storage as well as Preservica, our digital asset management system.
I’ve also been able to do a lot of work on creating online exhibitions for Special Collections and Theatre Collection. Most recently, in October the Cultural Collections Black History Month online exhibition went live. Working on these has given me a lot of perspective on the amount of work from everyone that goes into making collections accessible: from selecting suitable materials, digitising copies, and doing all the technical coding. However, it feels very satisfying to finally create a finished product which anyone can access from their mobile phone in the palm of their hand! It’s currently still online, so be sure to check it out if you haven’t already.
Looking forward, over the next year one of our main goals is to get more content into Preservica and accessible to users, ensuring our digital collections are as protected and accessible as possible. In the same way that you can put a book at the back of a pile and then forget about it, digital files can be hidden away in folders on old computers only to be lost. Physical archives are vulnerable to a number of real-life disasters, and whilst digital archives have their own vulnerabilities, taking a multifaceted approach gives our records the best chance of surviving for as long as possible.
DV or digital video tape is a format that was launched in 1995 and remained popular until the late 2000s. There are a few varieties of this type of tape but I want to discuss the consumer format known as MiniDV.
DV technology brought a considerable jump in quality from previous camcorder formats such as VHS and Hi8 with more compact and relatively affordable equipment. This made MiniDV a popular choice for arts organisations to document their work so we have a lot of these tapes in our more contemporary theatre and live art collections.
During 2023/24 I worked on the preservation of the audiovisual material in the Rideout collection which includes around 150 MiniDV tapes. Rideout, subtitled Creative Arts for Rehabilitation was established in 1999 by Saul Hewish and Chris Johnston to develop innovative, arts-based approaches to working with prisoners and staff within UK prisons.
The condition of the tapes indicated that they had been stored fairly well with no signs of physical damage or mould which can often be a problem when tapes are stored in conditions with varying temperature and relative humidity.
The process of preserving the audiovisual material on DV tapes involves copying the digital information encoded on the tape to a digital file on a computer. This differs from the process for analogue tape such as VHS which requires a digitisation process i.e. analogue to digital conversion. Copying the data from tape gives the optimal audiovisual quality and also has the benefit of retaining metadata such as the time/date of the recording, camera settings used, and error information. The tape must be played back in a suitable DV playback machine aka “deck” or camcorder in real time and the digital information on the tape can be copied directly to a computer using a Firewire cable connection or IEEE 1394 to give its correct technical name.
The first challenge is that Firewire technology is obsolete. It used to be a fairly standard interface, especially in Macintosh computers but now you need to install a Firewire card to your computer or use some kind of converter cable. We use the first option.
The second challenge is that playback decks or camcorders for DV tapes are no longer manufactured so second-hand units must be purchased. The price for professional decks is rising as the demand for preserving DV based material increases. We currently have four pro decks which all need occasional servicing to keep them running properly. Engineers familiar with this type of equipment are also getting harder to find.
So we have our deck, firewire cable and computer – this is the basic equipment needed to copy DV tapes. I also use a video monitor, amplifier and speakers to monitor the off-tape signal directly during playback.
Another challenge I discovered with this particular collection of tapes was that a fair proportion of them had been recorded in long play (LP) mode. None of my four professional DV decks are long play compatible so it is not possible to make a good quality digital copy of those tapes with them. There are a couple of professional decks that will play long play tapes but they are extremely hard to find and are accordingly very expensive. The solution was straight forward – most Mini DV camcorders will play Long Play tapes back well and are still an affordable option (£100 or less on eBay).
One of our main challenges with the tapes themselves is that they are prone to errors caused by signal dropout of video and audio from dirty or damaged tape. DV uses a technique called error concealment to try to disguise this dropout – this can include replacing the affected area with the same corresponding pixels of the previous frame or next frame (most common). Sometimes this strategy works so well that it is not noticeable in the resulting file, especially with static frame shots that contain a high degree of visual similarity from one frame to the next. However, when there is lots of movement, error concealment can cause the image to appear glitchy or blocky.
Audio errors can result in jarring clicks or loss of signal. Sometimes fast forwarding or rewinding a tape can remove errors, on other occasions manual cleaning of the video heads may be required. Even where the errors cannot be eliminated by these methods, tapes often don’t behave consistently when played back – they can display errors over certain sections and on subsequent plays fail to display the same errors or show errors in different sections of the tape. This behaviour can actually be used to our advantage in creating the best preservation copy possible which I will explain shortly.
In order to copy the information from the tapes to create our digital preservation files we need suitable software. Most Non-Linear Editing (NLE) software packages including Adobe’s popular Premiere Pro no longer support tape-based workflows. There are some older freeware options but these are no longer technically supported and can be unstable or unavailable for certain operating systems. Thankfully, the international community of video archivists and engineers has come to the rescue (literally) with a versatile free open-source solution called DVRescue. Open-Source means that end users and developers have the freedom to study, improve and redistribute the software.
DVrescue supports many of the commonly used professional DV playback decks and its capture interface displays timecodes, time & date of recording as well as a real-time graph of errors detected during the capture. Once your tape capture is completed the software has an analysis tool that pinpoints the position and nature of each error so that you can examine the effect on the video/audio in your file. Finally, there is the option to package the captured raw DV stream into a suitable file container – we use Matroska which is an open standard media container commonly used in video archives.
There are additional features in DVRescue that can be accessed via the command line, most useful of these for me is the merge function. This allows you to take two or more separate passes of the capture and combine them. The process takes advantage of the inconsistency of error display on subsequent plays, mentioned earlier, to return one file containing the best frames from each pass. By capturing short sections of video over known error regions and merging these with your master capture you can create the optimal preservation file.
So far so good, we have a set-up that can optimise our DV captures and produce greatly improved results from previous workflows. However…remember those Long Play recordings I mentioned earlier? The ones we have to play back with a camcorder? Well the camcorder is not supported by DVRescue so we have to use an alternative capture software. I use a discontinued software called Scenalyzer for Windows, this doesn’t have any of the sophisticated error reporting or analysis tools present in DVRescue but it does allow us to capture the tape in AVI format and retain the recording’s metadata. The files can then be analysed using a standalone piece of open-source software called DVAnalyzer. Errors can be identified and located and further corrective tape passes can be made. The resulting files must be converted to a raw DV stream before they can be merged with DVRescue to produce a best quality master copy. This conversion is done using an open-source command line tool – FFMpeg. We can then merge the files as before and re-package the master to a Matroska container.
So we have our workflow for producing preservation copies of these digital tapes. This is all great as long as your tapes are behaving but amongst any collection of DV tapes you will inevitably find problems.
Tapes that are dirty. Playback may be improved by cleaning – I do this by using one of my decks that I can open up to access the tape path, I then hold a special cleaning swab lightly against the tape and fast forward and rewind it. This isn’t always successful but has definitely improved the performance of some tapes.
Tapes causing “head clogs” – within seconds of playback the picture and sound will drop out due to particles shedding from the tape. The deck will not play back any tape until the heads have been manually cleaned. Cleaning and sometimes “baking” i.e. heating the tape to 54°C for a few hours) can improve performance. Where the head clogs are severe, tapes will have to be captured in sections with manual cleaning of the heads in between – a laborious and time-consuming process.
Damaged cassettes – the cassette shell may be damaged and prevent it from playing back or even being accepted by a deck. For these, the tape reels can be removed and transferred into a replacement cassette shell to achieve playback. Due to the size of the tapes this can be a fiddly process.
All that remains after our digital preservation copies have been produced is to create access or viewing copies for each file. These are smaller, more manageable files suitable for easy playback or sharing. I create these using a Non-Linear Editing (NLE) Software e.g. Premiere Pro or Shotcut which allows me to make adjustments for optimal viewing before creating the access copy file. All the digital files are then securely copied to and stored on the university’s Research Data Storage Facility.
I’m happy to report that I’ve managed to produce digital preservation copies of every DV tape in the Rideout collection. Some are perfect with no errors whatsoever, the majority have a few minor errors which may or may not be noticeable and just one has substantial errors throughout, although most of the contents of the tape can still be viewed and listened to. A fair bit of painstaking work was carried out to preserve the collection but I believe the additional effort required to get the optimal results is well worth it. Overcoming the challenge of obsolescent technology to preserve and do justice to these unique cultural works and their creators brings its own rewards.
Nigel Bryant – Audiovisual Digitisation Officer, University of Bristol Theatre Collection
Emma Hancox, Digital Archivist writes about World Digital Preservation Day and our recent steps in digital preservation.
Today (2nd November) is World Digital Preservation Day: an international date to celebrate digital preservation. This year’s theme is ‘Digital Preservation: A Concerted Effort’ and the focus is on the interactions and relationships that make for success in this area. Digital preservation is a key part of the work we do in Special Collections and Theatre Collection to ensure our researchers have access to important digital collections now and in the future.
When you think of archives, old handwritten documents such as letters, diaries, minute books or even a photograph album probably come to mind. Materials such as these created in hard copy in the past (which form a large part of our collections) are almost always created digitally nowadays. Organisations and individuals who approach us to enquire about depositing their archives are increasingly offering us material in digital form and it is imperative that we collect and preserve this material to have a comprehensive archive in the future.
As well as these original or so-called born-digital materials, we hold digital copies or digital surrogates of some of the physical material in our collections. These may be digital photographs of the hard copy material we hold or digital audio or video files from carriers such as cassette or video tapes which have been digitised because of their fragility and the risk of loss.
Although digital archives are still archives, and there are many similarities with our physical collections in terms of how we manage them, they have their own distinct set of challenges. It is very easy to create massive amounts of digital files and to save multiple copies of the same thing. Traditionally archivists appraise collections to select the material suitable for preservation, but this task becomes much more difficult and time consuming in a digital environment. Being able to validate files and check their integrity (whether or not they have changed over time) is vital as we need to be able to demonstrate the authenticity of the materials in our care. File format obsolescence and software dependencies are other potentially problematic areas. A particular file format may be more at risk than another and we cannot always ensure that we will have access to the correct software to be able to make a particular file accessible.
So what are we doing to improve our capacity to deal with digital archives? Towards the end of 2018 I was recruited as the first Digital Archivist and the first member of staff with a completely digital preservation-focused role. Since then, we have joined the Digital Preservation Coalition meaning we are part of an international digital preservation community and network, which is a brilliant opportunity for training, reciprocal support and learning. With the support of the DPC we participated in a booksprint in 2019, which enabled us to write the digital preservation policy that underpins our everyday decision making and an accompanying case study. 2019 was also the year we acquired Preservica as our digital preservation system and since then we have been familiarising ourselves with different types of ingest workflows and ingesting collections with the aim of making them accessible online via Universal Access.
In 2021 we were fortunate to be successful in our application to be one of the partners in the Bridging the Digital Gap scheme run by The National Archives making it possible for us to have a Digital Archives Trainee working with us. Our trainee progressed to the role of Digital Archives Assistant following the traineeship and his work has enabled us to expand our capabilities in terms of the amount of material ingested, compiling a digital asset register, being able to experiment with more complex workflows, using Python to assist with tasks as well as expand into other areas such as web archiving and the consideration of issues around the management of 3D data.
World Digital Preservation Day gives us the opportunity to look back at what we have achieved so far and to think about our future steps. It also reminds us that we are part of an international community facing similar challenges and obstacles and that sharing with each other is the key to overcoming them.