The age of DSLRs and EVILs carrying 16 Mpix or so on board the keeping of a complete photo archive can be a real headache. Just by the size of current graphic files the overall storage capacity required by archives can easily exceed not only a typical laptop drive but the most advanced home NAS as well. And we photo pros, maniacs and even pixel-counters know well that every single shot made is unique and deserve to be kept. Even the technically imperfect shot can be a masterpiece of art some day! And the family archive or travel photos from some exotic places can revive dearest moments of our life must be safeguarded forever, as the memory must live eternally.
So photos are flooded with abundance on our drives, and some of these photos may be lost in time and space just due to similar file names like DSC0001.JPG. As a result, we often have the uniform, unorganized archive with a lot of lost or defect files.
To prevent this sorrow and grim fate of our archives we have a strong need for cataloguing and backing up both our photo collection and the current files in our workflow. Some utilities dedicated for creating photo galleries can provide cataloguing functions as well. But many users are feeling no need in such functionality. I see personally only two loose groups of users satisfied by cataloguing functions of iPhoto, Aperture или Picasa; these are JPEG-shooting amateurs without any affliction to post-processing tons of their shots, or pros with a single camera/lens system verifying every action with any single photo to the last move or click. A vast majority of users have regular problems I try to describe below.
- Many, many shots. One daily session can bring the photographer the abundance of 500 to 2000 shots. Some photos will be dropped away instantly, but the rest of pictures must be processed and kept in safety!
- Similar file names. Somehow you meet the situation when you will copy your “100хххх” directory containing files from “DSCN0001” and up, in the folder already containing the previous directory and files of this name. What to be happened is entirely on your; as a worst scenario, you will lost some results of unique photo session!
- Different methods of processing for different cameras. It is not existed the optimal RAW file converter for all camera types. As an example, the popular Adobe Lightroom is, by the opinion of some users, does not explain adequately the coloristic and sharpness of some modern cameras. So these users recommend for some models alternate RAW processors like RPP, RawTherapee or Capture One. This problem spoils a possibility of using just one photo converter for catalogunin all the files from different cameras.
- Storage media errors and failures. The typical intensity of workflow loads the hardware heavily, increasing the chances of failure. If the hardware error destroys the media, all collections stored and not backed up will be lost.
To solve these problems one at all the two template solutions are existed:
- The right organizing of archive structure for logical keeping, cataloguing and searching the information needed.
- Backing up and automatic cataloguing for minimizing or eliminating the risk of data loss.
Let me share briefly my experience about the viable solution using these templates in practice.
The Structure of Photo Folders
No software solution can save when the workspace seems like a trash can. The most basic action to keep your archive at order is the cataloguing and tagging of the information kept.
Most digital cameras (Pentax SLRs are the lovely exception, for example) wrote shots at folders with uninformative names like “100_FUJI” or “103NIKON”. So when I copy my photo sessions from memory cards to my workflow catalogue, I always spend a half of minute to rename the folder so the name will be self-descriptive in future. I use the form like [CAM_WHAT_WHEN] for these names, so an obscure “330_FUJI” turns into the much clearer “FUJI-X20_Vacation_Nov2013”. This type of name can greatly simplify the searching.
To be mentioned, the folder containing fresh files and directories with photos on every my computer or laptop always named “Workflow” and placed in “My Pictures” parent folder.
This method I imply for photo file names, too. I strictly divide my files by raw (not only RAW but every unprocessed file as well) and photos mastered in Photoshop or other editor. If files of first type can be named such as “1010394.NEF”, then files already processed I always rename according to the template: [WHAT_WIDTHхHEIGHT], where width and height are the resolution of final photo in pixels. Explaining that, the final photo file name can be something like ”HaroldStassenWinsTheElection_2560×1600.jpg”. As a result I always see not only the contents of file but when I worked with it as well. Processed files are kept in directories with raw files in the “Workflow” folder. According to my needs I can move these files to special folders named “Printing” and “Publishing” in the same “Workflow” directory.
Time by time I copy (but not back up!) the processed content from “Workflow” folder manually. All folders not used in current photo edition processes are copying to the different drive, without renaming. I keep these folders in parent archive directories named [Archive_YEAR] — “Archive_2013” e.g.
That I always know where I can find a photographic memory of any single event I shot.
Backing up, Scheduling and Planning
Let me explain the difference between the archive keeping and backing up. Creating an archive means you move your files to the safe place for a long time. Backing up data is moving these data to the safe place with a regular renewal and the possibility of quick restoration if the need emerges.
To create an archive, it is often enough to copy the catalogued data to some safe media. Backing up is a bit more complex action; it requires scheduling updates, synchronizing data, verifying copies and efficiency of restoration as well.
To back up your photo collection manually is the almost impossible task. Backup photo archive requires some specialized software – something with built-in backup functions like iPhoto for Mac OS X or popular 7-Zip archive utility, or specialized backup software like EASEUS ToDo Backup, Handy Backup or Symantec Backup Exec.
This is completely profitable strategy even if you purchase the commercial backup utility. After purchasing the camera and lenses starting around $3000, the cost of $50-$200 for the specialized backup software seems to be negligible. A viable alternative is to use bash scripts for backing up, or use open-source solutions like Bacula or rsync. It is a matter of taste which methods you imply for your photo archive backup.
Overview of Backup and Cataloguing Solutions
As mentioned before, I back up regularly my workflow folders, and sometimes move manually completed photos to my archive. Additionally, sometimes I back up archives as well. I schedule daily copying of “Workflow” folders, and take the backup of archives every three to six months. Once a year I back up all the old archives, as new storage media often appearing at my hands annually.
I have three computers by work: one Windows-based desktop, one “field gun” laptop using Fedora Core and one Apple MacBook Pro w/Retina running under Mac OS X Maverics.
Let me speak about my Mac first. The Mac OS X built-in backup utility, Time Machine, seems pathetic by my taste. It performs full backup of your entire Mac on daily basis, and I can’t change this “scheduling strategy” by any means! New backups are uploading with older versions, just until the empty space on media will be exhausted. Then older versions are deleted, and the process will be continued in endless loop. It is too expensive and waste too much storage space, as I can think about it.
For Mac, I use Get Backup now. It is an adequate and versatile, if not so powerful, backup tool.
Linux (I use Fedora Core distro) has much more backup software than Mac OS X. I want to mention Bacula, the good and strong open-source backup utility. Instead, Linux allows you to use handmade bash scripts for copying automation.
And these are a plentiful bunch of backup utilities for Windows. I uses built-in Windows 7 backup feature for a long time, and now I use Handy Backup Home Standard — the comfortable and feature-packed specialized software for backing up. It is a good investment compared to many other software with similar functions.
I always schedule automatic backup of “Workflow” directories. On a daily basis, my copy Handy Backup performs an incremental backup of every “Workflow” folder to the personal cloud service running on the NAS Western Digital MyBookLive Duo, and a copy of this backup to directly attached Western Digital MyBook Thunderbolt Edition drive. Once a week “Workflow” directories backed up fully. Older versions of full backups are kept for 3 weeks, and after this time these folders will be deleted.
Archives (a main copy) are kept on a different NAS, Apple TimeCapsule 3 Gb. Processed photos are moved to the archive manually once a week. Backing up the archive uses a manual or automatic task of copying all the contents to the Western Digital MyBook Duo drive, plugged through USB. My Handy Backup has a special task to automatize this action.
The processed files kept in my “Printing” and “Publishing” folders on my desktop are backed up weekly by Handy Backup, using USB pen drives as media. These drives, after filling up entirely, are placed in special paper boxes or envelopes with short descriptions of dates and contents written. This action is a completely different than writing photos to the USB drives for customer, and the task of creating that backup is running manually when the need emerges.
Once a year, I take a day of two for viewing and revising my archives, to find and replace corrupt files or to revive a casual forgotten masterpiece undeserving this fate.
The system of cataloguing and backing up described before serves me some years, always providing me the comfortable and reliable access to any single shot taken by me. For the times I use this system, I change some cameras and computers, but I’m lost no one of my photos. As a result, I gladly share my experience in the matters of keeping photo collections, and wish you always to keep your precious artwork in safety and order!