Open-Source Tools for Audio-Visual Archives

At Elías Querejeta Zine Eskola in Donostia (San Sebastián), Spain, from 16 to 19 May 2023.

Table of Contents

• Abstract
• Lectures
• Free and Open-Source Software
• Bibliography

Abstract

The class begins with an introduction on file formats. Building on this the students will practically explore free and open-source software, such as:

• ffmpeg, HandBrake and Audacity for in-depth file transformation
• MediaInfo and ffprobe for technical metadata extraction
• QCTools and astataudit for quality control
• AEO-Light for sound extraction from optical tracks and
• DCP-o-matic for DCP encoding and file extraction

as well as various media players, including VLC, mpv and ffplay.

Resources on infrastructure and workflows for film and video digitisation will be presented and discussed, taking account especially of solutions which can be implemented in difficult environments at little expense and of solutions which can be automatised.

An overview on data preservation and migration, as well as disaster planning and recovery, will round off this class.

The lecturer is Reto Kromer.

Lectures

Tuesday 16 May

• Organisation, PDF, 2.5 MB
• On Audio-Visual File Formats, PDF, 6.3 MB
• A Quiz on Audio-Visual File Formats, PDF, 63 KB
• Used commands #1

Wednesday 17 May

• An Introduction to FFmpeg, PDF, 892 KB
• Used commands #2
• Exercise files #1, ZIP, 1.06 GB, MD5 = c2fdc0fc9290ebaa4757d69fadf3f38c
• Image sequence into ProRes
• Video into H.264 / MPEG-4 AVC
• Image sequence into H.264 / MPEG-4 AVC
• Split screen
• Delta file (difference file)

Thursday 18 May

• Data Storage and Data Migration, PDF, 2.8 MB
• Create DCPs with DCP-o-matic
• Used commands #3
• Exercise files #2, ZIP, 1.38 GB, MD5 = d933ed0d2079fa5cfec0009bb2e6c032
• Video into Matroska/FFV1

Friday 19 May

• A Brief Overview of QCTools, PDF, 448 KB
• Before the Restoration, PDF, 5.9 MB
• Analogue and Digital Preservation in Difficult Times, PDF, 3.3 MB
• Used commands #4
• Answers to the Quiz on Audio-Visual File Formats, PDF, 51 KB
• Sound noise reduction

Free and Open-Source Software

CLI = command-line interface
GUI = graphical user interface

Media Players

• VLC media player (GUI)
• mpv (CLI/GUI)
• ffplay (CLI)

Metadata Extraction

• MediaInfo (GUI and CLI)
• Ashley Blewer: MediaInfo Parameter Definitions, [2023]
• Ashley Blewer: Mediainfo Definitions, 2023-05-09
• ffprobe (CLI)

File Transformations

• ffmpeg (CLI)
• FFmpeg Cookbook for Archivists
• ffmprovisr (can be installed locally on your computers)
• FFCommand Engine (GUI for macOS and Windows)
• HandBrake (GUI)
• Audacity (GUI)
• Tutorial de ffmpeg

Quality Control

• QCTools (GUI) und qcli (CLI)
• QCTools Documentation
• AV Artifact Atlas
• QCTools Cheat Sheet
• Seattle Municipal Archives User Guide, 2015 (download the PDF)

Policy Implementer, Checker, Reporter and Fixer

• MediaConch (GUI and CLI)

Optical Sound Extraction

• AEO-Light (GUI)
• AEO-Light “Manual” (PDF, 5.7 MB)

DCP Encoder (and Player)

• DCP-o-matic (GUI and CLI)
• Carl Hetherington: DCP-o-matic users’ manual (available also as PDF, 2.1 MB)

The Bridge between Still Image and Stream Formats

• RAWcooked (CLI)

“Tool, language and decoders for working with binary data”

• fq (CLI)

Automation

• Reto Kromer: Bash Scripts for Audio-Visual Preservation (a set of Bash scripts)
• BFI Data and Digital Preservation [Joanna White et al.]: Open source transcode scripts (a set of Python scripts)
• IFI Irish Film Archive [Kieran O’Leary et al.]: IFIscripts (a set of Python scripts)

Bibliography

• Nicole Martin: What Is a Digital File?, 2020-09-14
• Agathe Jarczyk, Reto Kromer, Yves Niederhäuser and David Pfluger: Digital Archiving of Film and Video: Principles and Guidance, Version 1.2, Memoriav, Bern 2019
• IMAGO Technical Committee: Photon Path, 2023-05-14
• Nicole Martin: Why Use the Command Line for Digital Archiving and Preservation?, 2020-06-21
• Ashley Blewer: Command Line Interface
• Dave Rice and Annie Schweikert: Microservices in Audiovisual Archives, International Association of Sound and Audiovisual Archives (IASA) Journal, n. 50 (August 2019), p. 53–75
• Joanna White: Open source scripts from the BFI National Archive, 2021-05-25
• Joanna White: Using bash scripts to automate #AVpres workflows, 2020-03-31
• Reto Kromer: “On the Bright Side of Data Migrations”, in IASA Journal, n. 49 (December 2018)
• Reto Kromer: “Matroska and FFV1: One File Format for Film and Video Archiving?”, in Journal of Film Preservation, n. 96 (April 2017)
• Kieran O’Leary: Introduction to FFV1 and Matroska for Film Scans, 2016-10-07
• Shira Peltzman: Unlocking the DCP: Evaluating the Risks, Preservation, and Long-Term Management of Digital Cinema Packages in Audiovisual Archives, MIAP Thesis, New York University, 2013 (PDF, 1.4 MB)
• FFmpeg Artschool: An AMIA Workshop, 2020
• vacingfang: FFmpeg Audio Filter Gallery
• SubRip Text (.srt) format for subtitles

2023-05-26