Workshop 5 digital audiovisual collections

How to keep file-based content usable !
Includes material from the PRESTO projects and
from Presto Partners ITI at University of
Southampton

1/3 of material has deterioration
1/4 of material cannot be released as it is too easily damaged
Vietnam Film Archive Workshops

At least 2/3 of the material cannot be easily used

So we are digitising !
– or keeping film in cold storage


There is a three-way juggling act, and we
need to understand the trade-offs

 What are the risks?
 What are the mitigation techniques?
 What will it cost?
 What might you expect to lose?

 Is there a systematic approach?


 Technical obsolescence, e.g. formats and players
 Hardware failures, e.g. digital storage systems
 Loss of staff, e.g. skilled transfer operators
 Insufficient budget, e.g. digitisation too expensive
 Accidental loss, e.g. human error during QC
 Stakeholders, e.g. preservation no longer a priority
 Underestimation of resources or effort
 Fire, flood, meteors, aliens…


Doubles every 18 months
100 times every decade
1 million times every 30 years


Medium Storage Density bits/cm² Life, years

Stone 10 10000
Paper 104 1000
Film 107 100
Disc 1010 10

 Each change in „technology‟ is 1000 times denser
 But the media lasts 0.1 times as long

 TCO is a multiplier of unit media cost
 HDD storage is 2-3 times cost of data tape
 Real world costs reduce 50% every 2-3 years !!!
 1 + ½ + ¼ + 1/8 + 1/16 + ... = 2

 Take today‟s raw media cost and x10 for annual rate
 1TB online for 1 year costs $1000
 Multiply again by 4 for lifetime cost

 £1 per hour of audio on data tape forever
 £10k per hour of 4k film on hard disk forever


6 years
2 to 3 years

Ultrium LTO roadmap

Encoding formats
Media formats
Storage hardware
Operating systems
Management software
Networking


 1000 times more HDD capacity over last 15 years
 Only 10 times lower Bit Error Rates (BER)

 HDD Bit Error Rate = 10-14
 1 TB = 1013 bits
 10% chance of an error when reading all of a HDD

 Within a few years, you are more likely than not to
get a read error when copying a HDD


Google study of Annual Failure Rates in HDD servers


Volker Heydegger
study on file format
sensitivity to
corruption

JPEG2000 with one error per 100KB
Compression = Corruption amplifier
Corrupting 0.001% of encoded image results in 30% of pixels
affected in decoded image

 The „industry‟ is
going „tapeless‟
 Networked sharing
of file-based
content


(1) START HERE
Is the format a
(2) problem?
YES
Archive for a
few years NO
(3) (4)

(5c) What cost/quality/risk
option can you afford
Compress
lossy
(5b) (5a)

Compress Uncompress
lossless

END HERE

 Longer lived storage technology
 More reliable storage technology
 More copies
 Encode so content is more resilient
 Use concealment
 Check often and fix quickly


Access
Repair

Both
1 Copy
2 Good Problem Copies
Corrupted
Copies detected Corrupted
or Lost
or Lost
Failure
Failure Detection
Migration
Failure


 Preservation grade DVDs
 Magneto Optical disks
 Digital Film
 Rosetta Discs

 Expensive, Vendor Lock in, Not mainstream

 Risk: the vendor, not the technology


 Relatively „safe‟ technology (compared to HDD)
 Typical „problem rates‟ are 0.1 – 1% of tapes
 New strategies needed for many files on one tape
 Most problems from data tape come from drives

 Field studies show no data lost where multiple copies have
been made and integrity checked


 File checksums common approach
 AV files can be very big (e.g. 1 TB)
 Corruption can be very small (e.g. few bytes)

 Scrubbing takes time and resources
 Moving big files is „expensive‟

 So, make big files into small files!


All on tape (2 copies), hard disk only for staging

Increasing archive
Frequently used on hard disk, two copies tape
All on hard disk (1 copy), safety copy on tape

size
All on hard disk (2 or more copies)
All on flash (2 copies, e.g. USB sticks)


National Centre of Expertise in Film
Could put information online, in Vietnamese
language, to help other institutions


 Needs a Centre of Expertise in Audio
 Needs a Centre of Expertise in Video

 They work with VFI on a common website to
provide information, in Vietnamese language, to
help all other institutions in Vietnam


 SEAPAVAA is the organisation for audiovisual
professionals
 Could have a website with practical information in a
range of languages
 Already runs training programmes and conferences


 FIAF = International Federation of Film Archives
 FIAT-IFTA = International Federation of Television
Archives
 AMIA = Association of Moving Image Archives
◦ Has excellent email list (in English, mainly)
 IASA = International Association of Sound (and
Audiovisual) Archives
◦ Has excellent technical documentation


 A non-profit organisation set up to support
audiovisual preservation and access
 Based in The Netherlands (Holland)
 Supported by INA, BBC and other major institutions
(hosted by Dutch national audiovisual archive)
 Lots of online information (in English, mainly)
 Plans to run training and other services
 www.prestocentre.eu “www.prestocentre.eu”


Workshop 5 digital audiovisual collections

Recommended

Recommended

More Related Content

Similar to Workshop 5 digital audiovisual collections

Similar to Workshop 5 digital audiovisual collections (20)

More from Richard Wright

More from Richard Wright (10)

Recently uploaded

Recently uploaded (20)

Workshop 5 digital audiovisual collections