The Challenge of File Storage and Backup

(Repost – first posted Sept 2009)

One of the things I hear and read a lot about digital photography is how instant it is and also how limitless it is to take lots and lots of images. Using film you are limited to how many exposures the film has available, and you don’t get to see the final image until it has been processed, and then you find out you chopped off the head of the tallest person!

So shooting in digital allows much more freedom with quantity of images. But the idea that digital is limitless is incorrect. Because all the files, while virtual, take up a certain amount of space, and that space exists in the physical plane as the space allowance on your hard drive.

And the bigger the sensor size, the bigger the file size seems to be. My rule of thumb is “whatever the megapixels of the camera will be the megabyte size of the image”. Its a very loose approximation but does the trick for most people.

So you get yourself a camera and shoot hundreds of images. And over time those hundreds become thousands, and if you are really serious even tens or hundreds of thousands.

These days hard drives are measured in gigabytes (the larger ones are in terabytes) – but what does that really mean.

1 Bit = Binary Digit
8 Bits = 1 Byte
1000 Bytes = 1 Kilobyte (KB)
1000 Kilobytes = 1 Megabyte (MB)
1000 Megabytes = 1 Gigabyte (GB)
1000 Gigabytes = 1 Terabyte (TB)

These days about the smallest size hard drive you will get with a new computer is 80GB (gigabyte). If you have a camera taking 5MB images, theoretically

80 x 1000= 80000 (no of GB times number of megabytes to get total number of megabytes)

80000 divided by 5 = 16000 maximum number of images.

Quite a lot you would think, but you have to remember that a computer hard drive has to have space for all the *other* things – like the operating system, program files, email, mp3 and all the other data it accumulates. And you need to allow about 25% free hard drive space for temporary files each time your computer starts up – so you are already limited to 75% capacity.

And of course if you are shooting in RAW and getting 12MB files like my camera produces, your hard drive fills up even faster. So what are your options?

1. Invest in larger hard drive – either replace the one you have, or if you have space, add an additional one. I refer here to hard drives installed internally in your machine

2. Obtain external hard drive storage. These are now quite cheap and readily available. You have the choice of USB, Firewire or eSATA. USB is the most common interface but not as fast as Firewire or eSATA. eSATA is a newer technology that requires a special port that your computer may not have, but you can get a special card fitted to allow the connection.

Seagate and Western Digital are a couple of well known hard drive brands

3. Store files online. This is essentially what Flickr and Picasa and the other image websites do for you, and why you have to pay for the services if you are uploading lots of images.

4. Burn images onto archival disk – a CD holds 650MB of data, but a DVD disk holds roughly 4GB – so you can fit a lot on a disk.

5. Network Storage – NAS and DAS options. Probably more for the serious enthusiast or professional due to the cost.

But there is also a much bigger question. Not just ‘ where do I fit all these files’ – but ‘ how do I make sure I keep them and never lose them’ ?

Once you download the image off your camera data card, they are at risk. Your computer might die, or get stolen (or if its a laptop – lost). A pipe could burst and flood the house, or even a fire. And once the computer and its hard drive are gone or completely damaged, your precious images are lost to you forever!! So what do you do?

Backup. BACKUP. BACKUP

Always backup your data (not just your photo files but any other precious data) because you never know when something might go wrong.

How do I backup my data?

Excellent question, there are many ways, and the professionals probably use more than one of them in combination. Its an added expense so make your choice as well as you can. Also find someone who knows about computers whose advice you trust to give you some pointers (not just me!)

1. Easiest way to do a backup is to install a RAID 1 array.

RAID 1 is when you install 2 hard drives in your computer using RAID technology. If your computer does not have onboard RAID you will need to install a RAID card (and most laptops can only take one hard drive FYI) The first drive is the drive you see and store your files on, the second drive has absolutely everything copied over to it every time you use the first drive. So if the first drive fails in anyway you have a copy. Once this is correctly installed and configured you don’t need to do anything, it all happens automatically.

I have this on my computer which is now nearly 4 years old, and I have had about 4 hard drive failures, which have not been a disaster because it has been rebuilt from the second drive.

2. Get some external storage, and store copies of everything on it – like a RAID option except manual. You can also get special photography backup hard drives you can take with you that you can insert your data card into, and backup while you are out and about.

Epson P7000 with a screen is an example of one of these

3. Have an external hard drive copy but you STORE THE EXTERNAL COPY SOMEWHERE ELSE!!

So if the house is damaged or broken into, the copy is protected. You might bring it home once a week and do a backup so the most you will lose is a weeks worth of data.

This is my next phase of personal backup philosopy – as soon as I can afford it, I am buying one of these:

Simpletech Duo Pro Drive – it has eSATA for speed and two drives in RAID 1 for extra extra security

Most external harddrives like these come with special software to help you setup your backups for specific folders and schedule for daily backups at the same time every day and other useful things.

4. Online backup services – there are a lot of these available. You pay based on the amount of data you upload. I haven’t used one myself, and I am uncertain about privacy and data storage policies. And I would also want to know in detail what *their* backup and Disaster Recovery procedures were as well.

5. Network or Direct Attached Storage (NAS or DAS) – larger storage solutions for a serious computer user needing a lot of storage.

6. Archival disk storage. I have read some pro photographers do this as well as several other steps above

Then of course there is the trial of once you have LOTS of images, how do you FIND THEM once you have save them? That is the purpose of your image software, and to be honest is a topic I am now beginning to struggle with myself. My personal philosophy is to create lots and lots of folders (within folders) and so far this has worked, but now I am shooting lots more, I am beginning to wonder if there is a cleverer way to do it.

I have had The DAM book recommended to me – DAM is Digital Asset Management. I haven’t read it yet, but I have it on reserve at my library. I’m hoping its going to shed some light on how to better manage lots of photos 🙂

Advertisements

About lensaddiction

Mad keen photographer figuring it out as she goes!
This entry was posted in For Beginners and tagged , , , , , , , . Bookmark the permalink.

5 Responses to The Challenge of File Storage and Backup

  1. Katrina says:

    One thing that I think it is important to remember is that taking lots of pictures doesn’t necessarily mean keeping all of them. With film photography, the tendency is to edit before shooting the pictures unless you’re rich and can afford to use endless amounts of film. With digital photography, the editing process shouldn’t be thrown out the window. It just doesn’t need to happen as the first step any more. It is fine to go ahead and take 100 shots of something if you like, but keeping all 100 of them isn’t necessarily reasonable. If you take a lot of shots of the same thing, you’re probably experimenting with various settings and viewpoints to get the best shots. Afterward you should probably throw away the shots that didn’t work out so well. Maybe you should even throw away most of them and just keep the best 10 or 15. Just because we have the ability to keep them all doesn’t mean we should.

    On the subject of backup media, I’ll point out that dual-layer DVDs hold 8.5 GBs.

    • lensaddiction says:

      Funny, I thought to myself after I published the post “maybe I should have put something in about judicious deleting of images to save space”

      Great minds think alike! And thanks for the tip on the dual layer DVD, I haven’t used them myself and had forgotten they existed 🙂

  2. daroskoding says:

    I hear lots of people say that as you get better, your ‘keeper rate’ goes up so you delete less because (ideally) you’re shooting good shots with less shutter clicks.

    Guess that sorta helps with the space thing.

  3. Pingback: The Essence of File Management In A Terabyte World

  4. Elaine says:

    You’ve put a lot of thought into this Stacy. Thanks for sharing what you are doing with storage. Backup backup backup is the way to go! 🙂

Love to hear your thoughts on my post!

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s