TJOtala

My photo organization strategy has evolved over several years and it has been influenced by work of others like Peter Krogh and Fazal Majid. At this point I use it to manage over 80,000 photos spanning over 3,000 categories (aka tags) across 2,100 folders.

http://tjotala.com/wp/wp-content/uploads/articles/photo-notes/database.jpg;Database Summary

Note that I separate the physical storage of photos from the logical organization of photos. The primary goal of your DAM system should be to let you manage your photos regardless of where they are physically stored. In fact, you may even choose to move some of your photos to off-line media such as CD-ROMs, DVD-ROMs or removable hard drives, yet you still want to be able to manage them.

Glossary

Several years ago I found IMatch, a truly remarkable DAM software that answered all of my needs. In particular, it was the first software solution that finally allowed implement some of the concepts I had desired for so long. Because IMatch uses slightly different terminology than other software packages, here is a short glossary to get synchronized on the key concepts that follow.

Tag / Keyword

Descriptive text label. Flickr in particular has popularized the use of the tag metaphor. Many people think of tags as independent entities and thus one-dimensional. To put it another way there is no relationship, hierarchical or otherwise, between one tag and another. Some DAM systems do not enforce vocabulary or dictionary checks against tags, thereby allowing proliferation of typos, mixed uses of singular/plural (e.g. cat vs. cats), and alternate spellings (e.g. color vs. colour) – several of the oft-mentioned shortcomings of folksonomy.

Album / Category

This could be essentially another synonym for tag. However, it has special meaning in the context of the IMatch software I mentioned above, because categories in IMatch are vastly more powerful than Flickr-style tags. They differ in the following key characteristics:

• Categories may be nested indefinitely in a tree-like hierarchy.
• Category names may be duplicated within the hierarchy, so long as they do not appear under the same parent category.
• Any category, whether a parent or a child, may be assigned to a photo.
• Categories may refer to other categories using Boolean logic expressions. Thus it is possible to define synonyms for categories.
• You choose whether a reference to a parent category should include all of its child categories, or not.
• Categories may refer to filesystem folders, recursively or not.
• Categories may refer to photo metadata.
• Categories may be sealed against accidental assignment.
• You do not type in category name in a text box, you simply click on a checkbox to apply it

Property

In the context of IMatch, you can define arbitrary text fields called properties to store other metadata about your photos. These text fields can be further constrained to only allow selection from a list of items. Most other DAM systems support these types of fields (e.g. Caption) as well, though they may not be user-configurable.

As you have probably guessed by now, I found the Flickr-style tags extremely limited. Nevertheless, as much as it pains me to do so for the remainder of this text I will use the terms tag and tagging to refer to the process of attaching a label of some kind to photos.

Overview

My organization schema is built predominantly as a nested tree of tags. It aims to answer the following basic questions about each photo, where each underlined word is the name of the top-level tag. A key design requirement in my tag hierarchy is to avoid redundancy as much as possible, and to avoid ambiguity. Note that this does create some issues listed later in this posting.

How was this photo taken?

The EXIF metadata captured by digital cameras answers a great deal of these questions. In fact, several of the sub-tags in this tree are generated automatically by IMatch from the EXIF metadata (e.g. camera make and model, and lens). However, there are certain aspects that are not captured, such as the use of filters like CPL, ND, or GND, gels, strobes and so on. This may not be terribly relevant information for most, but it is useful data in case I would like to later reproduce the setup.

What is the Rating of this photo?

I strive to rate my photos on a scale of 1 to 5, with 5 being the highest rating. As I am still learning photography, I have reserved 5 for future expansion as did Peter Krogh in his book. I have also reserved rating of 1 for those technically lousy photos that really ought to be deleted but have some sentimental value that prevents me from doing so (e.g. that low resolution blurry photo of my son’s first steps). In practice I only tag photos with a 3 or 4, and leave the remainder in a large bucket of unrated photos. In the end they probably they merit a 2, but if I have not spent the time justifying a rating then I do not feel like assigning one either. Either way, they are easy to find with the built-in searches of the system.

What inanimate objects are in the photo?

This top-level tag covers structures, buildings, vehicles, and so on. For example, Car, or Bridge. This tag also includes sub-tags for more abstract concepts such as mood (Happy, Goofy, …), ambient conditions (Cold, Cloudy, Foggy, …), and so on. This top-level tag could have been called Keyword for all the mixed content that it identifies, but I liked the symmetry of words beginning with W. Just kidding.

When was this photo taken?

The sub-tags of this top-level tag are automatically generated from the EXIF metadata embedded in the photos, except for the issue with scanned photos. It is a great help in narrowing down a particular search.

Where was this photo taken?

This top-level tag describes the geographical location, and its granularity may vary anywhere from a country/state level down to street address level. For example, USA, California, San Jose, or 345 Park Ave. Generally speaking, the longer I have lived or stayed in a place, the more granular the geographical tagging tends to get, as the number of photos assigned to the top level geographical tag increases beyond manageable.

Who are the subjects in the photo?

This splits into two broad sub-tags: Animals and People. The Animals side splits further into sub-tags around types of animals (mammals, birds, and so on) though I do not aim to recreate the entire organism taxonomy. The People side splits into further sub-tags around relationship to the people in question: Business, Family, Friends, and so on. Note however one of the issue related to representing individual people and events related to people.

Why was this photo taken?

This splits into two broad sub-tags: Activities and Events. Activities are simple descriptions like Dancing, Skiing, Swimming, and so on. Events are similarly simple: Birthday, Holiday, Vacation, and so on.

Where in my Workflow is this photo?

This splits into three sub-tags: New, Needs Processing, and Processed. New is where a photo is automatically assigned to when it is imported into the system. Needs Processing is a transient tag that contains several sub-tags such as Archive, Crop, Publish, Print, and so on. Processed is the final resting place for photos that needed processing of some kind. It contains sub-tags such as Archived, Printed, and so on.

In fact, these are the top-level tags in my schema, and they serve as a handy mnemonic reminder of the goal of their sub-tags.

In addition to tags, I also use a small number of user-defined text fields that let me store more unique data per photo. Good examples of this type of data are Caption (or Title), Description, Copyright, Notes, Author (see below for more on this particular field), and so on.

Implementation

Top-Level Tags

This screenshot shows the tree of all the top-level tags I have defined along the basic schema that I outlined above. There are several additional tags in the list, however:

• The @All tag is a special tag that returns all photos, regardless of any other tags assigned to them.
• The @Interactive tag is a special scratch-pad tag that returns the results of the current tag expression defined by dragging and dropping tags into a grid of AND, OR, and NOT columns.
• The Collections tag is for gathering arbitrary collections of photos that have no other relationship; for example, images that I have used as desktop wallpapers, or as a pocket album in my PDAs.
• The Type tag is used to distinguish between original digital photos, scanned film photos, and derived works. Put another way, if a photo has either the Original tag or Scanned tag, it is the original photo. If it has neither tag, it is a derived work.

Searches

IMatch provides incredibly rich set of expressions to define tags in terms of other tags, filesystem folders, other searches, as well as dynamically from XMP metadata. In fact the tag properties (i.e. category properties) dialog can be pretty daunting for some people:

http://tjotala.com/wp/wp-content/uploads/articles/photo-notes/category-properties.jpg;Category Properties

In the more recent versions of IMatch, the author added an interactive way for defining tag formulas by dragging and dropping tags into a grid of AND, OR, and NOT columns. The result of that formula is then available as the @Interactive special tag which you can copy into another tag.

I take full advantage of this to define a set of permanent and temporary searches into the tag namespace. The screenshot above shows a typical derived tag: it finds all photos that match the formula:

("Ratings.4-Excellent" OR Ratings.5-Stellar") AND "What.Types.Landscape"

Thus, whenever I click on that tag I see a list of landscape photos that I have rated as 4 or 5 on a scale of 1-5.

Workflow Tags

This screenshot shows subset of the workflow tags that I use. It is important to note that while a photo winds its way through the workflow stages, it does not actually move anywhere on the storage device: it simply moves from one tag to another. It is also possible for any given photo to be in multiple stages of the workflow, or have multiple workflow tags assigned to it. I do not limit this in any way even though it would be very easy to do so by configuring these tags as mutually exclusive in IMatch.

While a photo travels through the workflow, I may create a derived work of the photo in which case the original photo exits the workflow and the derived work continues along. For example, let’s say a new photo gets assigned the Needs Processing.Publish tag. When I create a published version of the photo (e.g. with sharpening, framing and captioning), that derived work then gets the Processed.Published tag and the original loses the Needs Processing.Publish tag. If I ever need to find the original photo based on the derived work, I only need to take the base name of the photo and look it up as they are unique across the entire collection of photos.

Automatically Generated Tags

I also use IMatch’s dynamic tags feature to automatically generate sub-trees of tags from the EXIF data of the photos. An example of this is the camera model branch of the tag tree, shown in the screenshot. Unfortunately only some of the equipment data is captured in EXIF metadat, so other equipment still needs to be recorded manually – in particular, the use of filters. Another special category of equipment is teleconverters. I have noticed that Canon’s 2X teleconverter is accurately reflected in the EXIF metadata via expansion of the lens focal length range (e.g. 70-200mm to 140-400mm), while a Tamron 1.4X teleconverter is essentially invisible to the camera.

Known Issues/Problems

A1. Geographical Subject vs. Location

If I take a photo of the Golden Gate Bridge from San Francisco’s Crissy Field, should I tag it as Golden Gate Bridge (the subject) or Crissy Field (the location where the photo was taken)? Most people would say tag it with the subject (Golden Gate Bridge). However, this could mean losing some relevant information – namely, that the photo was taken from Crissy Field as opposed to, say, Baker Beach, Marin Headlands, Coit Tower, or Treasure Island, all of which offer a view of the Golden Gate Bridge. On the other hand, to fully represent this situation means potentially replicating large portions of the Where branch of my tag tree under the What branch as well. The exhaustive way to tag this would be this:

Where: Earth.USA.California.San Francisco.Crissy Field
What: Earth.USA.California.San Francisco.Golden Gate Bridge

Thus far I have chosen to overload the Where location tag to represent both geographical subjects as well as locations. For the sample case above, I would therefore tag the photo with both relevant tags:

Where: Earth.USA.California.San Francisco.Crissy Field
Where: Earth.USA.California.San Francisco.Golden Gate Bridge

This may seem illogical but it saves me the trouble of replicating vast forests of tag trees. Moreover, in most cases the distinction tends to be pretty obvious (e.g. bridge vs. field) so very little harm is done.

A2. Author in a Text Field vs. Tags

Most of the photos in my collection were taken by me. Some were taken by my wife, or one of my kids. This type of data would lend itself well to being captured as tags such as Author.Me, Author.Wife, and so on. Unfortunately, my DAM system can only automatically populate text fields with EXIF metadata (e.g. Canon’s proprietary camera owner name field), but not automatically tag with extracted names. Thus far I have chosen to live with small annoyance, though at some point I may simply auto-tag all new photos as taken by me and deal with the exceptions individually. Alternatively, I may manually use one of the many search features of my DAM software to search for photos with the proprietary EXIF camera owner name and tag them accordingly.

A3. People in a Photo vs. People Related Event

I use the Who.People tag to identify people who are in the picture, and this results in a pretty straightforward tag tree. Now how does one represent a particular event that is tied to a person, for example Events.Birthdays.John Smith without replicating large chunks of the Who.People tags? Fortunately this is a relatively rare occurance in my tag hierarchy, basically limited to personal events like birthdays. Thus far this has not been a problem worth solving.

However, this does raise a related question: how do I find all photos related to John Smith regardless of the context? Put another way, can I easily find the photos that contain John Smith as well as events related to him? Given that such events tend to contain pictures with John in them, one could find a close approximation simply by selecting the appropriate Who.People sub-tag. If I wanted to be particularly diligent, I could always create a collection tag that uses a formula like "Who.People.John Smith" AND "Events.Birthdays.John Smith". In practice this need does not arise often enough to warrant worrying about it too much.

A4. Shifts in People’s Family Relationships

Let’s say that you have the following tag hierarchy:

Who.People.Smith Family.John
Who.People.Smith Family.Jane

Now, sadly John and Jane divorce and subsequently Jane re-marries to Bob Jones and changes her last name. Before any of this, you knew Jane by her maiden name of Doe back in preschool. How do you deal with this mess?

Fortunately, most of my friends seem to be in relatively stable relationships, so this is not a very frequent problem. In the cases that this has happened, I have chosen to utilize IMatch’s ability to refer to other tags in formulas. So here’s the evolution of Jane Doe through the times:

First, Jane as you knew her in preschool:

Who.People.Jane Doe

Now, Jane marries to John. I retire the Jane Doe tag in favor of married Jane by moving the images from that tag to the new tag, and linking the old tag to the current tag:

Who.People.Jane Doe == "Who.People.Smith Family.Jane"
Who.People.Smith Family.John
Who.People.Smith Family.Jane

Then she divorces and marries to Bob. Again, I retire the married Jane Smith tag in favor of the re-married Jane Jones by moving all of Jane photos to the new tag and linking the old tag to the current tag:

Who.People.Smith Family.Jane == "Who.People.Jones Family.Jane"
Who.People.Jones Family.Bob
Who.People.Jones Family.Jane

The general idea here is to keep a single current tag where new photos are assigned. At first this may not seem ideal, as it does not allow you to directly delineate photos of Jane across the relationship changes. However, keep in mind that the primary goal of those tags is to identify all photos of Jane the individual – not three different Janes. The three linked Jane tags may be an overkill, unless you are like and want to be able to easily find photos of the Smith Family or the Jones Family. If I want to find out photos of Jane prior to her marriage(s), I could always combine any of those tags with the date tags to derive a single Jane Doe, married Jane Smith and a re-married Jane Jones.