Data Transfer

What? You mean files may not transfer properly?

We tend to take copying files from drive to drive for granted. Most times it works out fine. Many people will take a quick glance at the original and compare to the copy to get some level of confirmation that files have transferred properly. While this can spot egregious errors, it won't spot the more insidious problems that can silently corrupt images.

While transfer corruption is relatively rare, when it does strike, it is likely to strike repeatedly. There is an unknown cause for the error, and is likely to be hardware related, such as bad RAM or a misbehaving hard drive or enclosure. The transfer validation process will alert you to a problem in the early stages, before a lot of damage has been irreversibly done.

What are the trade-offs?

Transfer validation makes the transfer take significantly longer to accomplish. Not only does the data need to be transferred to the destination, it then needs to be sent back to the original, and compared with the original data. If the program is designed well, this should take roughly twice as long to make a transfer than a simple OS copy.

You'll need to acquire software to perform the validated transfer. There are a number of low-cost, or even free programs that can help you perform a validated transfer.

Do I always have to do it?

Not all transfers need to be validated transfers, but many should be. Here are some examples of kinds of transfers that we strongly recommend be performed with validation:

• Transfer of a fresh ingestion from a laptop to a workstation
• Transfer of working files to an archive drive
• Copy of archive files to backup hard drive
• Migration of any data from old drive to new drive
• Any time you care about the file, and are going to erase it from the source drive after transfer

Files that don't need to be copied with a validated transfer:

• Non-critical files (personal music library being transferred from a workstation to a laptop)
• Files being copied for temporary reasons, such as a slideshow being copied to a laptop for a presentation

Using software to perform a validated transfer

You'll want to find a program to manage your validated transfers. There are a number of good choices on both Mac and PC. Some are free and some have a modest cost. On this page we'll see examples of ChronSync on Mac and SyncBack on PC. Check the Resources area of dpBesflow.org to find other software that can help you accomplish validated transfers.

Setting up a transfer "pipeline"

If you need to transfer from one device to another repeatedly, you should consider setting up a transfer pipeline. Set up a place for outgoing files on the source drive and an incoming folder on the destination drive. (Figure 1)

Figure 1 shows a transfer pipeline. Images on the drive Whitebook are transferred to the drive Archive. The Transfer Me folder is the staging folder for outgoing files, and the Incoming folder is the staging folder for incoming files.

Once you have the staging folders set up, you can create a task in your transfer software, and save the task. Next time you need to transfer files between these two drives, you can open the task and run it. Figure 2 shows how this looks in ChronoSync.

Figure 2 shows a saved task for validated transfer. The staging folders on the source and destination drives are always used for the transfer.

Running the transfer

Let's take a look at how this comes together. Figure 3 shows a validated transfer with SyncBack on PC, and Figure 4 shows a validated transfer with ChronoSync on Mac.

Using a transferred folder

You may have noticed in Figure 1 that there is a folder named 5 Transferred. This is a helpful tool in creating a secure validated transfer workflow. The software transfers images properly, but that's not the end of the process. If you are transferring images from the working folder to the archive, for instance, the transfer is not really done until the files have been backed up in their now permanent home. If that does not happen immediately, you'll want to keep the files in the working folder until they have been safely backed up.

By using a Transferred folder, you create a holding area on the source drive where the image can be preserved. Once they have been backed up in their new location, they can be deleted from the Transferred folder.