File Format Migration

Formats are for saving

When an image file is saved, all the bits in the file need to be stacked up in some way so they can be decoded. A file format is a formula for stacking up those bits so various programs can understand them. There quite a few different images formats in use, and many of the variations are slight, such as the differences between CR2 files produced by different Canon cameras. Some of the differences are large and significant, such as that between a TIFF file and a JPEG file. If an image can be opened, it can be saved to a new format. There will be times when you want to change the format, or the variation of the format, that you use to store your images. That's when it's time for a migration.

Know what you've got

Unfortunately, there's no one who can monitor what file types you have in your collection except you. It's up to you to know what you've got, and to deal with obsolescence before it's too late. If you've been making and collecting digital files for a decade or more, you should strongly consider acquiring catalog software that can index lots of file types. You can then look through your files periodically to make sure that nothing is going obsolete.

We strongly suggest that you keep a primary copy of your archive on available storage of some sort, rather than in a drawer that is out of sight. This will help to ensure that you know when a problem arises with one of your storage formats, and that you will be able to accomplish the migration more easily.

Avoiding obsolescence

The most critical reason to do a format migration is because the current format is in danger of becoming unreadable. This is a particular risk for proprietary files from digital cameras because there are so many of them, and because there is a natural cycle of creation. As better cameras come along, software companies have less incentive to support the formats of the older models that have fallen out of use. Images from some of the very expensive digital camera from the late 1990's are now only accessible with very old computers.

While it's very unlikely that your camera files will be totally inaccessible in 10 years, it's not out of the question. Right now there are more than 200 distinct flavors of camera raw files. When software is updated, it needs to be tested with each of these. At some point, companies may decide that it doesn't make sense to spend engineering resources on a camera not been sold in more than a decade. If we get to a world of 2000 proprietary raw files, this could become a real problem.

Taking advantage of new capabilities

Rather than avoiding obsolescence, it's more likely that your motivation to migrate will be the need for increased functionality. For instance, there are a number of features that are available for DNG files that are not possible with proprietary raw. This includes validation capabilities, embedded metadata capabilities, and embedded rendering capabilities. Added security and functionality are likely to be a real draw for many photographers.

Formats in danger

The following list of formats are in at least some danger of obsolescence in the foreseeable future.

• Kodak PhotoCD
• Kodak RAW
• PICT

Migration example - Conversion to DNG

For many readers of this website, the most important format migration will be from proprietary raw to DNG. Between the forward compatibility planning, embedded data capabilities, and the data validation tools, DNG will look more attractive as time goes on. If your archive is made up of proprietary files only, format migration using the DNG converter is pretty simple. Send the whole directory structure to the DNG Converter and let it run. At the end of the process, you'll get a set of DNGs in a folder structure identical to the proprietary raws' folder.

Figure 1 The DNG Converter can create a duplicate directory structure populated with DNG files instead of proprietary raw files, but it will leave all non raw files behind (such as the JPEG and PSD files in this figure).

Is everything coming along?

During the DNG conversions, unfortunately, any non-raw files will be left behind. TIFF files will generate a warning in the DNG converter conversion log, but other non-raws, such as JPEG and PSD files (not to mention non-image files) will all be left behind silently. If you suspect you may have some of these interspersed with your raw files, you'll want to look at another method.

How do I find what's hiding in a directory?

The best way to get a comprehensive view of what's in a directory is to use a cataloging program that can index lots of file types. Expression Media is one that works on Mac and PC, and can show many file types. Here's how you can use it to see what's in a directory.

1. Make sure the folder is up to date in the Expression Media catalog (right-click the folder and choose Update Folder).
2. Click the folder’s dot to show all items.
3. Select all items in the thumbnail area.
4. See which file types light up in blue.

You can use this information to know what's in the collection and to help the migration process.