Proposed Destruction of Archive

[tenchboy]

As per https://www.theguardian.com/society/2023/dec/18/ministry-of-justice-plan-to-destroy-historical-wills-is-insane-say-experts
I remember Oliver Rackham saying, well writing; you want John Romer if you're going to be saying these things*; that the chained library at Hereford Cathedral contains books and manuscripts that are hundreds of years old and which can still be read to day just as they could be on the day that they were written. But your Phd that put on a floppy disc is now unreadable after only twenty years.
Technology moves to fast and past technology becomes obsolescent too soon to be used as an archive with no back-up. It's no so long since it would of have seemed like a good idea to put them on vhs and bin the originals.

They've destroyed the present and the future and now it seems they're intent on destroying the past.

This is just so annoying, it feels as if someone has just taken over the department, with no previous experience, and is just going around saying well, do we really need to keep this sh.t? What about those books? Can't we check them out? Who reads books these days anyway?

Hmm. I'll leave it there before I wander into a ...Rants crossover.

*Or Michael Woods, but he'd say it in Anglo Saxon

[Matatouille]

I'm sure it makes sense from a budget perspective, at least in the very short term. But that's about all you can say for it. Also why does the MoJ have them at all after a handful of years? Isn't this why we have a national archive, to make retaining official documents of potential historical note more efficient and less prone to new manager vandalism? I don't really have anything meaningful to contribute except to say that there was a good Cautionary Tales podcast about the destruction of archives and the march of technology a few weeks ago.
https://www.pushkin.fm/podcasts/caution ... r-the-disc

[jimbob]

I'm sure it makes sense from a budget perspective, at least in the very short term. But that's about all you can say for it. Also why does the MoJ have them at all after a handful of years? Isn't this why we have a national archive, to make retaining official documents of potential historical note more efficient and less prone to new manager vandalism? I don't really have anything meaningful to contribute except to say that there was a good Cautionary Tales podcast about the destruction of archives and the march of technology a few weeks ago.
https://www.pushkin.fm/podcasts/caution ... r-the-disc

And if this government had not been utterly profligate with spaffing our Mone-y at every con artist who donated to the Tory party and spending hundreds of millions of pounds on not flying 100 vulnerable people to Rwanda (whilst agreeing to take refugees from Rwanda) then the "it saves a relatively small amount of money but it's still worth it for the damage" might be a bit more convincing.

[Gfamily]

Burning the originals of the Exchequer records in 1834 didn't go terribly well

[Woodchopper]

I can see both sides of this. I appreciate people’s desire to be able to touch the same piece of paper as a long dead author. They should also be concerned about the prospect of archives being deleted.But there are some reasons to replace a paper archive with a digital one.

I say that as a bibliophile with a large collection of books and papers on an obscure subject that date back to the 1950s, and as someone who has done archival research.

Firstly, paper is a poor medium for long term storage. Some of my books and papers from 60-80 years ago are falling apart. Of course it’s possible to preserve them for longer but that would require climate controlled storage which is expensive.

Paper archives get deliberately destroyed (for example the immigration document needed by the Windrush cases. There are also lots of examples of paper archives being accidentally destroyed by flood, fire or vandalism.

To take the proverbial doctoral candidate, they would be foolish if they kept it all on a floppy disc (or to update a memory stick). But I would also strongly advise someone who kept everything on paper to follow standard practice and keep three independent copies of everything stored separately. In practice that means digitising it.

Secondly, digital archives are vastly easier to use and search. Compare spending hours pouring over piles of paper or reading old ledgers to keyword searching a digital copy.

Of course, digital archives only last as long as someone is willing to manage them and pay for maintenance or server fees etc. People can mention how the BBC dropped the ball with the 1980s doomsday book (which was rescued with a lot of effort). But the same applies to paper archives. They will get destroyed or just crumble away if they aren’t actively preserved.

It’s possible to get the benefits of paper and digital by keeping paper and digital copies of everything. But the vast improvement in efficiency means that for most documents dual paper and digital archiving would entail spending lots of money on a paper archive that would very rarely be used.

So I appreciate the romance of being able to hold actual documents from the past. I’ve spent a lot of time collecting them. But we have to consider how much money it’s worth spending to preserve that emotion. Certainly it’s worth it for somethings. But there is a line to be drawn somewhere.

[tenchboy]

I can see both sides of this. I appreciate people’s desire to be able to touch the same piece of paper as a long dead author. They should also be concerned about the prospect of archives being deleted.But there are some reasons to replace a paper archive with a digital one.

I say that as a bibliophile with a large collection of books and papers on an obscure subject that date back to the 1950s, and as someone who has done archival research.

Firstly, paper is a poor medium for long term storage. Some of my books and papers from 60-80 years ago are falling apart. Of course it’s possible to preserve them for longer but that would require climate controlled storage which is expensive.

Paper archives get deliberately destroyed (for example the immigration document needed by the Windrush cases. There are also lots of examples of paper archives being accidentally destroyed by flood, fire or vandalism.

To take the proverbial doctoral candidate, they would be foolish if they kept it all on a floppy disc (or to update a memory stick). But I would also strongly advise someone who kept everything on paper to follow standard practice and keep three independent copies of everything stored separately. In practice that means digitising it.

Secondly, digital archives are vastly easier to use and search. Compare spending hours pouring over piles of paper or reading old ledgers to keyword searching a digital copy.

Of course, digital archives only last as long as someone is willing to manage them and pay for maintenance or server fees etc. People can mention how the BBC dropped the ball with the 1980s doomsday book (which was rescued with a lot of effort). But the same applies to paper archives. They will get destroyed or just crumble away if they aren’t actively preserved.

It’s possible to get the benefits of paper and digital by keeping paper and digital copies of everything. But the vast improvement in efficiency means that for most documents dual paper and digital archiving would entail spending lots of money on a paper archive that would very rarely be used.

So I appreciate the romance of being able to hold actual documents from the past. I’ve spent a lot of time collecting them. But we have to consider how much money it’s worth spending to preserve that emotion. Certainly it’s worth it for somethings. But there is a line to be drawn somewhere.

Oh yes, I absolutely agree that having records digitized is a must. It has often been said that you can now do a lifetimes work in a couple of months - what wouldn't those researchers of yore have given for what he have now? And to be able to sit at home and click click click have on you desk a copy of the probate of any will you want - or any other document is beyond wonderful (having been part of before and after, old and new, I often feel I want to ring up the National Archive and say "Thank you, do you actual realise how wonderful this is?")(where I live and play was a western outlier of the sixteenth century Wealden iron industry, many of the lakes and ponds were built at that time for pen ponds to drive the hammers and the bellows for the blast furnaces, I have copies of the wills of the ironmasters who leased the lake that I look after and another that is now a field).
But, for all the reasons mentioned, the originals should always be kept as well.

[TopBadger]

new manager vandalism

Good phrase that... 'tis all

[Fishnut]

Digitising archives is invaluable but it also puts them at huge risk. The British Library attack is proof of that. They've basically been out of action for the last couple of months and it could have been so much worse - imagine if it wasn't just the database that had been digitised and then stolen but the archives themselves as well.

There is definitely a balance that is required and we can't turn into a nation of hoarders keeping absolutely everything, but at the same time I don't think that digitising everything is the panacea it's being made out to be. Data degrades over time, even if you find a medium that doesn't become obsolete. I know that to keep paper optimally it needs precise conditions but I can't be the only one who's found paper from decades ago used as packing material or whatever that's still highly readable. Yet computer software from even a few years ago can be impossible to access without specialist tools.

This makes me concerned from a cost perspective as it's not just going to be the storage of the digital archives but the need to ensure that they are accessible as technology progresses - how much will those constant upgrades cost? And how do you ensure that data isn't lost on each upgrade?

There's also the fact that our technological ability to digitise the documents is constantly evolving too. Even within PDFs, older ones are basically photos of documents yet newer ones can extract text from them. Who knows what newer technology will enable?

[bjn]

If digitising, regardless of what you do with the originals, you should do so in a manner that uses long established standards and formats over any new flavour of the month mechanism. These are much more likely to survive tech cycles and still be used at some point in the future. For example, I’m a huge fan of plain text files in my own work. One of the many problems with the BBC doomsday book is that they were inventing completely new tech to get it to work in the first place, using a new recording format. Most of that tech proved to be a dead end, the world moved on and so it was all lost in a sea of bitrot.

The other thing is to dedicate budget to maintaining and preserving the digital version of the archives and the software to access those archives. Don’t just scan stuff and bung it on a hard drive that you then leave in a basement somewhere. Storage costs and compute costs are continually dropping, so in theory your digital assets should get cheaper to own over time. However, software is never “done”, it always needs updating, it always needs debugging, because the environment it lives in is always changing. Ink on paper never needs updating to deal with the latest versions of Paper 3.5 and Ink 4.2’s essential security releases, you digital archive will. There are ways of minimising this, so having your underlying assets held in sane well understood open standard formats that you can easily manage is good place to start. The software to provide access is always going to be the hard bit, as it will be built on ever evolving web technologies, which frankly suck and is very prone to ‘latest shiny’ syndrome.

So yes, having a digital version is good because it’s way easier to search and provides access to many more people, but be prepared to cover the non trivial cost of maintenance as otherwise bitrot will set in and the whole thing will be as dead as the BBC Doomsday book.

[IvanV]

FWIW they are saying they will preserve the physical originals of wills of famous/significant people.

[Gfamily]

FWIW they are saying they will preserve the physical originals of wills of famous/significant people.

As someone on twitter asked, "who's to know who (currently little known, from the late 19th C) will be considered significant in 50/60 years?"
When did we start recognising Mary Seacole as a 'significant' person from history.

[Sciolus]

Wot no XKCD?

Digitising the records is a necessary but not sufficient condition for destroying the originals. We know how to preserve paper and ink pretty well, and have plenty of experience. People have put a lot of effort into understanding the requirements for long-term digital archives, but there is still no actual experience of doing it long term -- and plenty of short-term experience has ended in failure.

There are archival benefits to digitisation -- you can have lots of spatially-distributed copies so are less vulnerable to single points of failure. If you make them easily accessible to the public, you can supplement your archive by effectively crowd-sourcing it, although that should obviously be an added benefit rather than your actual archive strategy. But you still only capture the bits of the original that you think to digitise.

[IvanV]

Digitising the records is a necessary but not sufficient condition for destroying the originals. We know how to preserve paper and ink pretty well, and have plenty of experience. People have put a lot of effort into understanding the requirements for long-term digital archives, but there is still no actual experience of doing it long term -- and plenty of short-term experience has ended in failure.

I have seen it seriously suggested by archivists that digital data be preserved in the long run by storing it in the form of ink dots printed on paper. Because paper is really long lived in comparison to typical digital data storage. You can store rather more data on a piece of paper than you might imagine, about 500kb per A4 sheet. Which is large space reduction relative to traditional printed documents. Here is an open source computer program that implements such an idea. It's a joke, but one with a kernel of truth to it.

[TopBadger]

Microsoft are working on holographic data storage... high density and expected to be much more robust than paper.

https://www.microsoft.com/en-us/research/project/hsd/

[dyqik]

Microsoft are working on holographic data storage... high density and expected to be much more robust than paper.

https://www.microsoft.com/en-us/research/project/hsd/

Which is fine until the devices to read it stop being made five years after it's released, and stop working about 20 years after that.