The digital dark age dilemma

This poses the question of whether a digital image file is really even a photo, or is it just the raw material from which a photo can one day be created?

While hard drives and ‘The Cloud’ (which is really just a sexy name to describe a computer other than your own, accessible only via an internet connection) have their weaknesses, so too do the other alternatives ““ CDs and DVDs, and flash memory (memory cards and USB flash drives).

There is no 100 percent fail-safe image storage system, but then there are few of those kinds of guarantees in life more generally. Nevertheless, there are simple steps to take to put the odds resoundingly in favour of your digital image file collection surviving to a venerable old age…

The statistics outline the challenge: Over 1.5 million new photos are shared every day on Facebook, WhatsApp and Snapchat alone. That equates to about 550 billion a year ““ and the number is growing.

InfoTrends’ 2014 Worldwide Image Capture Forecast estimated consumers snapped 810 billion photos worldwide in 2014. This number will swell to 1 trillion photos in 2015 and 1.3 trillion photos by 2017. And according to Canon Australia, two-thirds of photographers surveyed have lost digital image files due to ‘the damage or failure of physical technology’. Fortunately, only a fraction of the billions of digital image files out there warrant preservation in any form. Film also encouraged photographers to think about the worth of the photograph they were taking before hitting the shutter button. And let’s not even start on the cult of the selfie…

The issue received some high profile attention this year when Dr Vint Cerf, VP and ‘chief internet evangelist’ with Google, and one of the recognised pioneers of the internet, addressed the American Association for the Advancement of Science, on what he saw as a looming ‘Digital Dark Age’.

‘In our zeal to get excited about digitising we digitise photographs thinking it’s going to make them last longer, and we might turn out to be wrong,’ he said. ‘I would say if there are photos you are really concerned about create a physical instance of them. Print them out.’

He noted there were two threats to images which existed only as digital files – the evolution of technology and subsequent obsolescence of various storage media (imagine if your images were stored to floppy disc) and something called ‘bit rot’ ““ the slow deterioration in the performance and integrity of data recorded to storage media.

Cerf called for the development of ‘digital vellum’ to preserve old software and hardware so that out-of-date files could be recovered ““ no matter how old.

Georgia McCabe, CEO of the Photo Marketing Association in the US (herself a key player in the development of Kodak Photo CD technology back in the day) has talked about her concern that we’re creating a ‘digital landfill in the sky’ and how turning that around should be a priority for the photo industry.

The message was further reinforced by a local academic, Dr Chris Chesher, senior lecturer in Digital Cultures at the University of Sydney, who said that computer hard drives had a life of around five years or less, while storing digital images in ‘The Cloud’, Facebook, Flickr, etc ““ offers no guarantee that your precious memories are safe.

‘When you look at the terms of service of a lot of the cloud companies, they say a lot of friendly things [about how] they’ll look after their data,’ he told 702 ABC Sydney.

‘But when it comes down to it ““ terms of service ““ basically, “we’re not going to take any responsibility at all”.

‘The recommendation is to diversify the instances of that data, so printing out your photographs is one of the ways. Even if your hard drive goes down, your cloud service disappears or gets corrupted, you still have the hard copy. Paper is still one of the best archiving forms that we have.’

““ So before we plunge into the digital storage alternatives, the message is that photographs represent an excellent ‘archival medium’ in themselves.

The 28-megapixel NX1 will generate over 400MB of image data in one second in its 15fps continuous burst mode.

Memory cards

In addition to the sheer volume of digital image files, cameras are getting increasingly memory hungry. A camera like the Samsung NX 1 (above) is capable of 15 frames per second at full 28-megapixel resolution, when shooting in the fat, data-rich RAW format. Canon is about to launch 50-megapixel DSLRs. Shooting at 5fps in RAW, they will generate around 250MB of image file data per second. And there are an increasing number of models offering 4K video, which also demands lots of storage space and fast transfer speeds.

So the new generation of high performance cameras need cards which are both high capacity and speedy in transferring data. It’s not something photographers should skimp on. Cheap cards are more liable to be faulty and not likely to have fast read-write speeds. Sandisk, Lexar and Kingston are reputable brands which tend to be first to market with high capacity, fast memory cards. They also offer file recovery software to retrieve images from a card if it fails.

Sandisk claims its Extreme  Pro SDXC range are the fastest  SD cards around, with write  speeds of up to 250MB/s,  which will cope with 4K and  Full HD video recording.

Unfortunately the memory card manufacturers, or at least the SD Association, have made understanding card performance incredibly opaque. Ordinary SD cards are too slow and limited in capacity for modern cameras. They’ve been superseded first by SDHC which offers up to 32GB capacity and more recently by SDXC which can support capacities of up to 2TB (2000GB).

SD and SDHC-compatible cameras can’t use SDXC memory cards, although SD memory cards and SDHC memory cards will work in SDXC-compatible cameras.

For the shrinking number of DSLRs using CompactFlash cards, UDMA 6 and 7 cards deliver the best performance when capturing high resolution images and HD video. These cards are compatible with any camera that uses CompactFlash card storage.

CFast is the emerging version of card format from the CompactFlash Association and provides a higher maximum transfer rate than CompactFlash cards. CFast cards are similar in size to CompactFlash cards, but do not have any pins which can bend in the connector. The format was the capabilities of the next-generation of camcorders and DSLR cameras.

Apart from card capacity, the other key specification is speed – if the data can’t be written to the card sufficiently fast it will slow down a camera’s frame rate or result in glitchy video.

Cards are allocated a Class rating to indicate read/write speed range. The read speed is how long it takes to open a file on the card. More importantly, write speed is how long it takes to save a file to the card. And the higher the Class number, the faster the read/write speed performance.

The SD Association defines Speed Classes as 2, 4, 6 and 10. They then (sort of) change to UHS (Ultra High Speed) 1 and UHS 3.

UHS cards offer minimum write speeds to successfully record video. They are not compatible with all cameras. And there’s a bit of an overlap between Class 10 and USH-1. Confused? This is how the SD Association explains it: ‘It’s important to remember that Speed Class and UHS Speed Class are two different speed indication symbols for different devices. However, a UHS-I memory card may also indicate a Speed Class.’

Now you must be confused!

Camera manuals should recommend the minimum speed class card for your camera. You can move to a higher rated card and should experience some increase in speed, especially shooting RAW, or following a burst of continuous shooting. A lower speed card will drag down the overall responsiveness of your camera.

The high capacity of memory cards can be a hindrance to digital image file storage because by the time the card is full, the task of sorting through the images is massive. According to Sandisk, a 16GB card will store 286 RAW images or 2861 lossless JPEGs from a 16-megapixel camera. Sorting through and organising 200-300 images is a chore; 2000 will go into the too-hard basket for most of us.

So delete early and delete often – don’t make the task harder by carting visual rubbish about in your camera for weeks or months on end. At least then when it comes to the organisational task, you’ve already sorted the potential keepers from the dross. The job is half done.

It’s also worth considering using several smaller capacity cards rather than a big whopper. That forces you to attend to the transfer to a storage medium more frequently. It also means that at any given time before transfer to storage, you haven’t got all your eggs in one basket.

Hard Disk Drives

While your PC is the logical place to organise your digital image file collection, it’s the worst possible place to store it.

Most personal computers are used on a daily basis, so there’s plenty of extra wear and tear on the disk drive. Notebooks can be damaged, lost or stolen. Many people upgrade their computers every two or three years, which makes them a fairly temporary spot to store a lifetime image collection anyway.

A study by US online backup company BackBlaze published last year and based on three years’ data from its installed base of 27,000 HDDs, indicated that in the first 18 months of their lives, consumer grade hard disk drives had an average failure rate of 5.1 percent.

For the next 18 months the failure rate drops to 1.4 percent. After three years, the failure rate explodes to 11.8 percent per year. So after six years you have around a 50-50 chance that the hard drive will have failed. These are not good odds if a computer hard disk failure is all that stands between you and the loss of your precious digital image files.

External hard disk drives will have a longer life because they aren’t worked as hard. On the other hand, being small and portable, they have more chance of coming to a sticky end ““ being dropped, lost or stolen, for instance. With 2TB of storage for around $150, external HDDs represent great value. At those kind of prices, it’s worth considering having one external drive for everyday back-up for your PC, and a separate drive on which you build your image file collection. There will be an extra copy by default on the drive used for regular computer backups.

Just back to that BackBlaze study: there were some surprising differences in failure rates among the three brands being used: Hitachi (now HG ST), Western Digital and Seagate. Overall, HGST was the most reliable, with Western Digital coming in a comfortable second and Seagate having more failures, especially with its 1.5TB drive. An update of the data from January this year showed HGST maintaining a healthy lead in reliability as BackBlaze moved to 4TB drives.

Hard disk drives have three distinct life stages: They are vulnerable in their first 18 months, then comes a period of relative stability, followed by fairly rapid deterioration after Year 3. (Source: BackBlaze)

The new kid on the block slowly replacing hard disc drives are Solid State Drives (SSDs). They are a close relative of the USB drive or thumb drive, which in turn are part of the flash memory family SD and CompactFlash cards belong to.

They are more robust than HDDs, as they have no moving parts. Drop one and it’s still likely to work. HDDs not so much. SSDs are faster at transferring data than hard drives. SSDs also don’t have spinning disks, which draw a lot of power when reading and writing data. But on the other hand, hard drives provide more storage for the dollar.

Samsung’s 250GB external SSD is priced at $180, a 500GB drive is $300, and a 1TB drive is $600. By comparison, a 1TB magnetic drive (HDD) can be acquired for under $100. Maybe next year…

CDs and DVDs

Don’t be put off by stories of CDs deteriorating after two years or so. If you purchase good quality, premium-branded media and take some reasonable precautions, CDs and DVDs should last for a decade or more. Once again, this is not something you can afford to skimp on.

Alarming stories about CD decay are often generated by vendors of alternative technologies – so a ‘storage expert’ from, say, IBM (which is big in magnetic tape storage) will predict CD lifespans of between 2 to 5 years. On the other hand, Verbatim says that, stored properly, its Ultralife Archival Gold CDs and DVDs will last for 100 years.

Here are some estimated lifespans for various optical media from independent website CD-Info.com:
 Unrecorded CD-R and CD-RW: 5-10 years
 Recorded CD-R: 50-200 years
 Recorded CD-RW: 20-100 years
 Recorded DVD-R: 30-100 years
 Recorded DVD-RW: up to 30 years
 Recorded BD-R and BD-RE (Blu ray): 30-200 years

Having purchased premium discs with archival properties, here are some tips for successful optical disc storage:
 . For maximum CD longevity, look for media with a gold reflective layer.

. Treat CDs and DVDs with care ““ hold them by the outer edges or the hole in the centre, don’t touch the surface, avoid scratches, and keep dirt from the disc.

. Keep them in a dry, dark, and cool place since humidity, sunlight, high temperatures, and pollutants can damage the different layers.

. Store them in jewel cases rather than paper slips.

. Use non solvent-based felt-tip permanent markers, suitable for writing on CD or DVD labels.

. Rewrite your rewritable discs as little as possible.

. Choose slow writing speeds to reduce errors and increase quality.

While optical discs have a reasonable lifespan, there are two other issues which are combining to reduce their usefulness: At under IGB and 4.7GB respectively, CDs and DVDs are lacking the capacity for the larger and larger files being generated by high resolution camera sensors; and disc drives and burners are gradually disappearing from PCs.

Beyond archival quality DVDs, there’s another format called M-Disc, developed by Millienniata, for which a 1000 year life is claimed. These are available in 4.7GBG DVD and 25GB Blu ray versions. Verbatim sells DVD and Blu ray M-Discs. The M-Disc media must be burned with a M-Ready drive, but can be read by any DVD or Blueray drive.

Personal cloud systems such as the WD My Cloud consist of external hard disc drives which are also remotely accessible.

‘The Cloud’

Dropbox, Flickr, Bitcasa. Google Drive, Microsoft OneDrive, Smugmug, Canon Irista and Amazon are among the plethora of businesses who want your images stored on their servers.

Amazon put the cat among the pigeons in March by offering an ‘Unlimited Photos Plan’ for just US$11.99 per year. This allows storage of unlimited photos, including some RAW file types (Nikon NE F, Canon CR2 and Sony ARW). Other file types, such as JPEG , BMP, PNG and ‘most’ TIFF files are accepted, but any photo over 2GB is not.

This is just a fraction of what the other providers have been asking. For instance, Dropbox is asking $10.00 per month for up to 1TB or storage. However, for Windows 365 subscribers, Microsoft OneDrive offers 1TB of storage as part of the monthly subscription fee.

For those concerned about privacy and security, NZ-based Mega might be worth a look. It provides encryption in every part of the process, so anything you send to the cloud is encrypted locally, enroute, and on the destination server. What’s more Mega itself doesn’t have any way of accessing your information, as you hold the encryption key so that anything stored on Mega is only able to be opened by you. Mega offers 50GB of storage free, with 500GB for 99 Euros ($140) per year, 2TB for 199 Euros per year, or 4TB for 299 Euros per year.

So is The Cloud The Holy Grail for long-term image storage? This is what Mike Ashenfelder, a digital archiving expert from the US Library of Congress had to say: ‘Cloud services relieve you of the responsibility of tending to storage hardware, but your collection becomes inaccessible if the network connection is disrupted.

‘No online backup service is as reliable as a storage device that you can see and touch. Cloud storage should only be a secondary backup option.’

Finally, there’s what’s being termed ‘personal cloud’ storage, which is a hybrid between an external HDD and a cloud service. Files are stored on your local networked drive so there is full control over your image files, yet they are remotely accessible (so long as you have an internet connection).

Rather than having your data stored at a third-party server farm in the US, it is right next to your router, on a Network-Attached Storage (NA S) device – generally a hard disk drive. The data is password protected and encrypted, and when you delete something you know it’s gone. And once you buy the equipment, there are no annual storage fees.

Unlike desktops and laptops, a NAS device offers a simple OS that focuses on doing a small subset of tasks as efficiently and securely as possible.

Once the Personal Cloud library is set up on the drive, it can be accessed from any networked computer or device, or remotely using Wi-Fi or Internet connections on your phone, tablet or laptop.

So what’s best? Here’s Mike Ashenfelder again: ‘There is no “best” storage medium. CDs, DVDs, flash drives, solid-state drives, spinning-disk hard drives, tape and networked cloud storage all have benefits and drawbacks. CDs and DVDs are light, flat and easy to store but they can be scratched and damaged. Flash drives do not have the moving parts that spinning disks do but they are more affected by extreme temperatures. Spinning-disk drives ““ the kind that “whir” when you turn them on ““ have a large storage capacity but you could easily damage them just by dropping them.

 ‘The best strategy is to backup copies onto at least two different types of media and store a copy in a different geographic location in case some disaster strikes your home or office.

‘In fact, professional photographers ““ who have a financial stake in the accessibility and safekeeping of their digital photographs ““ have a “3-2-1” rule: make three copies, store two on different types of media and one in a different location. ““ But as that keen snapshooter General Patton once said: “A good plan violently executed now is better than a perfect plan executed next week.” In reality, many (most?) of us aren’t going to make three copies on two different types of media and store one in a safety deposit box – it’s just not going not happen.

But at the very least, delete the photos you don’t want, print the photos you love, and store the rest of your collection safely out of harm’s way.

Finally, here’s a comment from a reader responding to a Wall Street Journal article extolling the virtues of cloud storage:

‘Last year on my iMac, I opened iPhoto and found that four years of my photos had disappeared. This included our daughter’s wedding, the first year of our grandson, trips and family gatherings. As my iMac was out of warranty, I paid Apple $19 hoping we could get them back. My WD external hard drive had also failed. I spent hours talking with 5 different Apple Techs over several days. The last guy finally found them and worked with me to restore them.

‘The upshot: there is no perfect storage system. NONE. A week later, I spoke with an LA Times photographer; he and the Apple guy BOTH say, PRINT them to keep in a frame or a book. In iPhoto, they are not really photos (they are similes in essence); NO CD, DVD, Cloud, or computer will keep digital photos safe over time. Digital photos degrade! I was lucky. NOT chancing it again, I make prints and photo books of those I really care about. Others that are a passing fancy, I put on FB or Instagram, then delete them. Be forewarned!”

[By Keith Shipton, publisher of Photo Counter and Pro Counter.]

This article is an excerpt from  Photo Review magazine Issue 64  

Subscribe to  Photo Review Magazine