Open-Source Tools for Audio-Visual Archives

At Elías Querejeta Zine Eskola in Donostia (San Sebastián), Spain, from 4 to 7 June 2024.

Table of Contents

• Abstract
• Lectures
• Free and Open-Source Software
• Bibliography
• Appendix

Abstract

The class begins with an introduction on file formats. Building on this the students will practically explore free and open-source software, such as:

• ffmpeg, HandBrake and Audacity for in-depth file transformation
• MediaInfo and ffprobe for technical metadata extraction
• QCTools and astataudit for quality control
• AEO-Light for sound extraction from optical tracks and
• DCP-o-matic for DCP encoding and file extraction

as well as various media players, including VLC, mpv and ffplay.

Resources on infrastructure and workflows for film and video digitisation will be presented and discussed, taking account especially of solutions which can be implemented in difficult environments at little expense and of solutions which can be automatised.

An overview on data preservation and migration, as well as disaster planning and recovery, will round off this class.

The lecturer is Reto Kromer.

Lectures

Tuesday 4 May

• Organisation, PDF, 2.5 MB
• On Audio-Visual File Formats, PDF, 6.2 MB
• Used commands #1

Wednesday 5 May

• An Introduction to FFmpeg, PDF, 874 KB
• Used commands #2
• A Quiz on Audio-Visual File Formats, PDF, 63 KB
• Exercise files #1, ZIP, 1.06 GB, MD5 = c2fdc0fc9290ebaa4757d69fadf3f38c
• Image sequence into ProRes
• Video into H.264 / MPEG-4 AVC
• Image sequence into H.264 / MPEG-4 AVC

Thursday 6 May

• Data Storage and Data Migration, PDF, 2.8 MB
• Create DCPs with DCP-o-matic
• Used commands #3
• Exercise files #2, ZIP, 1.38 GB, MD5 = d933ed0d2079fa5cfec0009bb2e6c032
• Split screen
• Difference file (delta file)

Friday 7 May

• Before the Restoration, PDF, 4.0 MB
• A Brief Overview of QCTools, PDF, 650 KB
• Open-Source Tools and Resources, PDF, 129 KB
• Analogue and Digital Preservation in Difficult Times, PDF, 3.3 MB
• Used commands #4
• Answers to the Quiz on Audio-Visual File Formats, PDF, 51 KB

Free and Open-Source Software

CLI = command-line interface
GUI = graphical user interface

Media Players

• VLC media player (GUI)
• mpv (CLI/GUI)
• ffplay (CLI)

Metadata Extraction

• MediaInfo (GUI and CLI)
• Ashley Blewer: MediaInfo Parameter Definitions, [2023]
• Ashley Blewer: Mediainfo Definitions, 2023-05-09
• ffprobe (CLI)

File Transformations

• ffmpeg (CLI)
• FFmpeg Cookbook for Archivists
• ffmprovisr (can be installed locally on your computers)
• FFCommand Engine (GUI for macOS and Windows)
• HandBrake (GUI)
• Audacity (GUI)
• Tutorial de ffmpeg

Quality Control

• QCTools (GUI) und qcli (CLI)
• QCTools Documentation
• AV Artifact Atlas
• QCTools Cheat Sheet
• Seattle Municipal Archives User Guide, 2015 (download the PDF)

Policy Implementer, Checker, Reporter and Fixer

• MediaConch (GUI and CLI)

Optical Sound Extraction

• AEO-Light (GUI)
• AEO-Light “Manual” (PDF, 5.7 MB)

DCP Encoder (and Player)

• DCP-o-matic (GUI and CLI)
• Carl Hetherington: DCP-o-matic users’ manual (available also as PDF, 2.1 MB)

The Bridge between Still Image and Stream Formats

• RAWcooked (CLI)

“Tool, language and decoders for working with binary data”

• fq (CLI)

Automation

• Reto Kromer: Bash Scripts for Audio-Visual Preservation (a set of Bash scripts)
• BFI Data and Digital Preservation [Joanna White et al.]: Open source transcode scripts (a set of Python scripts)
• IFI Irish Film Archive [Kieran O’Leary et al.]: IFIscripts (a set of Python scripts)

Bibliography

• Nicole Martin: What Is a Digital File?, 2020-09-14
• Agathe Jarczyk, Reto Kromer, Yves Niederhäuser and David Pfluger: Digital Archiving of Film and Video: Principles and Guidance, Version 1.2, Memoriav, Bern 2019
• IMAGO Technical Committee: Photon Path, 2023-05-14
• Nicole Martin: Why Use the Command Line for Digital Archiving and Preservation?, 2020-06-21
• Ashley Blewer: Command Line Interface
• Dave Rice and Annie Schweikert: Microservices in Audiovisual Archives, International Association of Sound and Audiovisual Archives (IASA) Journal, n. 50 (August 2019), p. 53–75
• Joanna White: Open source scripts from the BFI National Archive, 2021-05-25
• Joanna White: Using bash scripts to automate #AVpres workflows, 2020-03-31
• Reto Kromer: “On the Bright Side of Data Migrations”, in IASA Journal, n. 49 (December 2018)
• Reto Kromer: “Matroska and FFV1: One File Format for Film and Video Archiving?”, in Journal of Film Preservation, n. 96 (April 2017)
• Kieran O’Leary: Introduction to FFV1 and Matroska for Film Scans, 2016-10-07
• FFmpeg Artschool: An AMIA Workshop, 2020
• vacingfang: FFmpeg Audio Filter Gallery
• SubRip Text (.srt) format for subtitles

Appendix

• As discussed with some of the participants, here is more information on Bayer sensors:
  Reto Kromer: Sensor Design and Video Codec Design, 2021-12-09;
  Reto Kromer: The Photon Path: from Sensel Values to Pixel Values, 2022-10-28.
• Open internship positions are always advertised on our website.

2024-07-14