Open-Source Tools and Resources for Audio-Visual Archives

Abstract

The class begins with an introduction on file formats. Building on this the students will practically explore free and open-source software, such as FFmpeg (for in-depth file transformation), QCTools (for quality control), AEO-Light (for sound extraction from optical tracks) and DCP-o-matic (for DCP encoding and file extraction), as well as various media players.

Resources on infrastructure and workflows for film and video digitisation will be presented and discussed, taking account especially of solutions which can be implemented in difficult environments at little expense.

An overview on data preservation and migration, as well as disaster planning and recovery, will round off this class.

Lectures

Tuesday 1^st June, two sessions at 15:00–16:30 and 17:00–18:30

• On Audio-Visual File Formats, PDF, 3.3 MB (MD5 = 59c00994538d58c8b51215c1f8b9d01d)

Wednesday 2^nd June, two sessions at 15:00–16:30 and 17:00–18:30

• An Introduction to FFmpeg, PDF, 623 KB (MD5 = cf09df6cdce54df50dbb5ef8fa61aa90)
• Exercise files #1 (ZIP, 1.06 GB, MD5 = c2fdc0fc9290ebaa4757d69fadf3f38c)
• Used commands #1 (part 1)

Thursday 3^rd June, two sessions at 15:00–16:30 and 17:00–18:30

• Case Study: the Bayer Colour Imaging Array, PDF, 1.3 MB (MD5 = 71579d3d01c575ef8ffb8e191e019b23)
• Used commands #1 (part 2)

Friday 4^th June, two sessions at 15:00–16:30 and 17:00–18:30

• Before the Restoration, PDF, 13.1 MB (MD5 = 776587dc6746bf3511fbbbc2ac9280f6)
• Open-Source Tools and Resources, PDF, 72 KB (MD5 = 67899e67a1960a1f28b8046f3d75861b)
• Exercise files #2 (ZIP, 83 MB, MD5 = 34fbf38f6ecdd0c8b6c72560a6d2c755)
• Used commands #2
• Exercise files #3 (ZIP, 584 MB, MD5 = eec9d797713d80ae42e535dbd18d8b18)
• Used commands #3

Wednesday 9^th June, two sessions at 15:00–16:30 and 17:00–18:30

• Digital Preservation during a Global Emergency, PDF, 3.3 MB (MD5 = 6570ea783185a002f6b681ccc192ded8)
• Data Storage and Data Migration, PDF, 3.3 MB (MD5 = d4037e34cc66afef7a3fd1a4f08a0641)
• Create DCPs with DCP-o-matic
• Exercise files #4 (ZIP, 1.38 GB, MD5 = d933ed0d2079fa5cfec0009bb2e6c032)

Free and Open-Source Software

Media Players

• VLC media player (GUI)
• mpv (CLI and GUI)
• ffplay (CLI)

Metadata Extraction

• MediaInfo (GUI and CLI)
• ffprobe (CLI)

File Transformations

• ffmpeg (CLI)
• FFmpeg Cookbook for Archivists
• ffmprovisr (can be installed locally on your computers)
• FFCommand Engine (GUI for macOS and Windows)

The Missing Piece of Software

• RAWcooked (CLI)

Quality Control

• QCTools (GUI) und qcli (CLI)
• QCTools Documentation
• AV Artifact Atlas
• QCTools Cheat Sheet
• Seattle Municipal Archives User Guide, 2015 (download the PDF)

Policy Implementer, Checker, Reporter and Fixer

• MediaConch (GUI and CLI)

Optical Sound Extraction

• AEO-Light (GUI)
• AEO-Light “Manual” (PDF, 5.7 MB)

DCP Encoder (and Player)

• DCP-o-matic (GUI and CLI)
• Carl Hetherington: DCP-o-matic users’ manual (available also as PDF, 2.1 MB)

Bibliography

• Nicole Martin: What Is a Digital File?, 2020-09-14
• Agathe Jarczyk, Reto Kromer, Yves Niederhäuser and David Pfluger: Digital Archiving of Film and Video: Principles and Guidance, Version 1.2, Memoriav, Bern 2019
• Nicole Martin: Why Use the Command Line for Digital Archiving and Preservation?, 2020-06-21
• Ashley Blewer: Command Line Interface
• Dave Rice and Annie Schweikert: Microservices in Audiovisual Archives, International Association of Sound and Audiovisual Archives (IASA) Journal, n. 50 (August 2019), S. 53–75
• Joanna White: Open source scripts from the BFI National Archive, 2021-05-25
• Joanna White: Using bash scripts to automate #AVpres workflows, 2020-03-31
• IFI Irish Film Archive [Kieran O’Leary]: IFIscripts (using FFmpeg commands in Python scripts)
• Reto Kromer: “On the Bright Side of Data Migrations”, in IASA Journal, n. 49 (December 2018)
• Reto Kromer: “Matroska and FFV1: One File Format for Film and Video Archiving?”, in Journal of Film Preservation, n. 96 (April 2017)
• Kieran O’Leary: Introduction to FFV1 and Matroska for Film Scans, 2016-10-07
• Shira Peltzman: Unlocking the DCP: Evaluating the Risks, Preservation, and Long-Term Management of Digital Cinema Packages in Audiovisual Archives, MIAP Thesis, New York University, 2013 (PDF, 1.4 MB)
• FFmpeg Artschool: An AMIA Workshop, 2020
• vacingfang: FFmpeg Audio Filter Gallery

2023-05-26