Above is a screenshot of the current contents of my Lightroom folder – in total Fifty Seven Thousand and Nine files!! Just over half a terabyte of data and the cumulated image files of the last 6-7 years of shooting.
Imagine losing it all due to a hard drive failing!!!
Well in the last 6 years I have had THREE hard drives fail. Yes – Three drives have died.
How much data did I lose?
Wow, I hear you say, you must be really lucky. Actually in some ways I have been, because losing everything was possible – stuff could have been stolen, or house burnt down, or losing everything in the earthquakes. Until this year, I did not have a fully developed backup strategy. But I had 2/3 of one and that is why my data has remained intact.
Lets Talk About Hard Drives
For most people, computers and technology is something that they have to deal with, like a car or DVD player, but they don’t really understand what it is or how it works, or why it matters. Thats OK, not everyone is interested in the same stuff.
But when your precious memories and images are now stored in digital data form, and essentially at the mercy of your technology, it pays to know……something…. about data and backup.
I have worked in the IT industry for nearly 20 years, and for the last 9 have worked selling IT hardware to corporate, enterprise and government customers. So I kinda know a bit about this and applied that knowledge to my own technology requirements at home.
Therefore let us talk about Hard Drives:
Hard drives are the part of the computer where all the files are stored. It doesn’t do any thinking, it merely holds the files there to be accessed when necessary.
Until the last few years, hard drives were based on spinning disk technology – inside are several very shiny platters stacked on top of each other with heads that move across them to read and write data. For those of us who remember the black vinyl records with a needle that moved around the track to make music, it works quite similar to that.
As a result of the design, it has a LOT of moving parts, and any major physical impact on these drives can put the moving parts out of alignment, and your disk fails. Over time they also just wear out. Link to a diagram here.
These are the most commonly available disks as they are cheap and if you want large capacity ie 2TB and over, then this is pretty much the only option. This technology is called SATA and these are usually the drives you get as External Hard Drives.
The newer kind of technology in hard drive design is called SSD or Solid State Drives. This is because they are based on flash technology, which stores data in a different way, and doesn’t rely on moving parts. Link to comparison between the two types here.
However SSD drives have some issues – they are still expensive in comparison to the cheaper SATA option, and also don’t come in big capacity – so if you need to store lots of photos, they may be a more expensive option if more are required.
In the IT industry, we don’t say IF your hard drive will fail, we always say WHEN it will fail, and we always design and plan for that. However the systems available in business are not always viable or affordable for home users.
OK What Else Can I Do Then?
It depends on the device you are using – if you have opted for a laptop then you have limited options to change the hard drive. If you have a tablet, you have even less options, as nearly all of them are sealed.
If you have a standard PC then you have some more options available. Lets explore what some of them might be:
- External Hard Drive (EHD) via USB
Benefit – reasonably cheap, easy to buy and easy to use and manage.
Problems – if you need decent capacity its likely to be SATA, and very likely to fail at some point especially if it gets carried around with a laptop a lot.
2. Several EHD
Same benefits as above, you can have multiple backups but the more you have, the more fiddly the system gets. This means the more likely it is that you DON’T maintain proper full multiple backups – people tend to get lazy and complacent about their technology.
Also still prone to failure, theft or loss if the house is damaged.
3. Several EHD with one or more held in offsite location
If you *are* managing to run a good system of full backups (yay!) and have at least one copy held off site – this protects against theft or overall house damage.
Problem – even more difficult to manage and therefore less likely to really be done. Also need to find a secure site to hold – preferably fireproof, waterproof etc like a safe for ideal situation.
All of the above options work for any device that has a USB plug and allows an EHD to be plugged into it. They are cheap, easily accessible and easy to use, but a manual system invites errors and human nature means its unlikely to be reliably updated.
There are options which can be more automated and more accurate, however more hardware is involved and more initial cost up front. Its also more technical and requires more understanding of technology.
4. RAID 1 Mirror
RAID stands for Redundant Array of Inexpensive Disks. Redundant refers to having more than you actually need, so that if there is a failure, you still have enough to maintain data integrity.
There are different kinds of arrays, but the easiest and most relevant is RAID 1 Mirror.
This is when you have two identical disks and you set one up to instantly copy the other one, anytime a change is made. Once the array is setup, no further intervention is required, and it just magically happens.
Remember how I mentioned losing 3 disks over 5 years – RAID 1 mirror is the reason why no data was lost. My hardware design looks like this:
2 x 180GB SSD in RAID 1 Mirror for main computer – this has my C: and all my programs
2 x 500GB SATA in RAID 1 Mirror for my data files – this is my D:
2 x 2TB SATA drives in RAID 1 Mirror on a network attached storage system that has all my photography and digital art files.
Because I had an instant back up of each drive, if your computer has RAID capability (and if it doesn’t you can add a card into a PC but many have it on the motherboard built it) then the other drive just kicks in, and you carry on, without missing a beat.
Of course, this still can be stolen or lost if the house is permanently damaged. Therefore I also maintain an offsite backup system (currently of one drive, and when I get around to buying a second one, it will go to another site).
RAID still has challenges, you have to buy twice as many drives so it adds cost, it doesn’t work for laptops or tablets internally, you may need to pay someone to set it up for you. However the benefits are significant – once setup you can just forget about it, its real time and instant, and it does work.
Cloud is the new way of storing stuff – its secure, its a service so you just pay a monthly fee (sometimes its even free) but as a photographer there are issues.
- if you have a lot of data it can take MONTHS to upload the current data files
- it relies on good reliable fast internet
- because it uses internet there are added data costs depending on your plan
Right now, I don’t use cloud for several reasons – one is the first line above. Second is I have questions about HOW my data is stored and how that relates to getting it back again. If it takes me 4 months to upload it now, if I need to download it for some reason will THAT also take me 4 months (or longer if there is more data)
What format does it download into – every time it saves, does it take a snapshot or an incremental save – do I have to download every copy and unpack it in order to get all my data correctly?
Also can I pick just some folders to copy or have to copy EVERYTHING on my hard drive (data limits mean that can be a real pain). If the data is encrypted, and I have to unencrypt it during download, how much longer does that take?
I know lots of people happily put stuff in the cloud and think that solves their problems, but if you ever need to get it back again, no one seems to think about that process.
Cloud has a lot of benefits – if you have tablets or other devices with limited built in storage space, it makes a lot of sense to use cloud storage. Its really easy to setup, its secure, offsite, safe from theft or house damage.
But I have many questions too – and if there are any Cloud experts out there, feel free to chime in!
6. Network Attached Storage or NAS
So the other option I have as well as pairs of drives in my PC is a NAS. My internet comes in to a Router that also contains a 4 port switch, and I run cable through the house to my main PC. It also puts out a wireless signal for my iPad and laptop. All my devices run on the same network and therefore can potentially access each other.
For security reasons I don’t allow that but I do have a NAS similar to this device which contains 2 x 2TB drives in a RAID 1 mirror. This gives me all the storage needed, connecting to all the devices I want it to. Because it has the mirrored array, it already has one level of redundancy built in.
However this demands a much higher level of geekery to setup – I had to ask one of my very technical friends to help set mine up, as it was beyond my skillset to do properly and securely. It is a lot more expensive, as you have to buy the device and the larger capacity HD cost more too.
Yet while there is up front cost, my whole system once setup (and paid for) just……goes. Set it up….. and forget about it. Except for the monthly offsite backup copies – one thing to remember to do once a month – easy enough to remember and not too annoying to have to do.
OMG!!! SO MUCH INFORMATION!! SO CONFUSED!!
WHAT DO I DO?
Yeah, sorry about that, but like I said at the beginning, you have to understand a bit about hard drives and data storage so you can make an *educated* choice.
Every week I see a post online – help my hard drive died, what do I do???? I’ve lost all my data!! I thought it was backed up to the cloud but it isn’t!!
My question to you is this:
HOW PRECIOUS IS YOUR DATA TO YOU?
How much do you care about losing your family photos or travel photos or pet photos?
No….really stop and think about that answer……
If you lose all the pictures of your children growing up (cos I bet you don’t have them printed out in albums like the old days) what would that mean to you?
When you stop and think about what it REALLY means, then how hard are you prepared to work to make sure you don’t lose them. How much extra are you prepared to pay now to give yourself several different layers of backup to minimise data loss?
If you really aren’t sure, find a technical expert who you trust and discuss your requirements with them and they can work out what kind of system works best for you.
In my personal opinion, a system as automated as possible and as easy to use as possible will be the most successful one that gets fully integrated and used.
A good backup plan should have 3 layers
IE main device backs up to EHD and the Cloud and a second EHD stored off site somewhere
IE main device has RAID 1 drives, backs up to a NAS with RAID 1 and the Cloud (and or an offsite option too)
One or two levels of protection isn’t enough expecially if you are relying on EHD devices that we know are prone to damage and failure.
Backup is a bit of a complicated and fiddly subject, but in this digital age, it is a necessary evil. Anyone with data they consider precious or necessary should have some form of backup plan in place – one that they have some understanding of why its there, how it works and what the consequences are when it fails.
Otherwise, you will find out the hard way one way or another. Nobody wants that!
(Caveat – I am not an expert – this is my knowledge and opinion, summarised to help educate other less technical people – to invite discussion. Yes I have shortcut a lot of technical details and tried to make it really simple. If you are a geek I apologise in advance – if there are genuine technical errors here, please correct me. If you have useful advice to add, please do so, but keep it simple and easy to understand, please)
Some links for more reading if you are interested:
Backblaze 3-2-1 strategy
Your backup needs a backup
What is the perfect backup plan