Friday 29 January 2010

RECOVERY

Yesterday evening I experienced the worst feeling possible for someone producing music, the loss of files!

Since last summer I've been making lots of percussion noises using the various virtual modular synths I have, but mostly the Arturia Moog Modular and the ARP 2600. This process involves creating sequences of simple percussive sounds and applying changes to the sound over time. You then end up with a sequence with a sound that evolves through time, I then save this as an audio file. I take this audio file into the Propeller Head ReCycle software and create slices of all the various sounds, these can then be saved as separate audio files. The next step involves classifying the sounds which is a manual process but usually I will name a set, noise snares, noise hits, zaps, pings, kicks etc. I then load these various sounds into Logics' EXS-24 sampler and create a drum map which automatically apply's each sample to a different key on the keyboard. These are then saved as kits.

For me these sounds create what I feel is a uniqueness to our 'sound', I've been guilty in the past of using many of the same samples/noises for our songs but with this album I'm really trying to define a new and hopefully different style. As you can imagine these sounds have become the building blocks to this project.

After my Mac upgrade at the end of last year I took the main drive from my old machine and installed this as a spare drive into the new MacPro. For months I've been meaning to clean this drive up and use it as a back-up drive rather than the main place where all the new ideas are kept. Obviously I back this up regularly to a number of other drives, one networked drive and two Lacie FW drives. These are kept in various places to ensure I will never loose anything. At the end of Wednesdays session I decided to clean my drive and make this my main work disc. I copied what I thought were the important Logic files to a temporary space, formatted the old drive and tested that Logic loaded everything correctly, it did so I was happy and went to bed.

Yesterday I opened up Logic to work on a new idea and found that EXS-24 couldn't find any of my samples, the ideas I have worked on so far all worked OK, this was because Logic saves as an option all the instruments to a local project folder. However my 13 odd ideas only used a very small selection of the samples I had created. I'd lost probably 7, 000 of my samples and couldn't figure out why. After trawling through the Logic user forum there were some issues raised to Logic loosing file path references, this could explain why I had lost my samples. I searched my various drives to try and find these, I had thought these were saved into Logics' ESX-24 instrument samples, but I could only find the included library sounds. It then dawned on me that they were obviously somewhere within the drive I had formatted the previous night. I started to feel sick to the stomach at the prospect of loosing my collection.

I had used DiscWarrior before to recover files from a work drive, this worked but I found it was hit and miss plus you have to know what you are looking for as the drive will have millions of files that can possibly be recovered. I found via a very useful link on the Logic user forum a piece of software called ProManager3 by Redmatica (http://www.redmatica.com/Redmatica/ProManager3.html) I purchased this and is successfully found the EXS patches, it then told me what files were missing, the file path and the files name. This was useful and confirmed I had indeed deleted the samples from within the Library folder on my previous drive. So now I have the 7,000 file names that I needed DiscWarrior to search for. I created a simple AppleScript using Automator to export the file names and get Disc Warrior to search for. So far so good.

After about 30 minutes it was looking good and had identified possible matches for file names. It recognised them as audio files and gave me the option to recover them which I started to do. Another 45 minutes later it had recovered all files, but the Mac wasn't recognising them as the correct format and nothing seemed to open them as audio files. Again the sick feeling returned which was made worse by the prospect that I had been feeling optimistic for the past hour.

After more trawling through the web I came across a command line piece of software called Switch (http://www.nch.com.au/switch/plus.html) that allowed batch audio file conversion. My dilemma was what format had I saved these files in, I was pretty sure they were WAVs, so I selected a conversion from .raw to .wav. Another 20 minutes later I had restored 7,000 files. I opened a few of the files and indeed they were back.

However the EXS still wasn't able to find the files :-(, I used the ProManager3 software again to try and restore the links between the patches and samples, it was then I noticed in the file path of the missing files all of my original samples were actually saved as .aiff. So I ran the batch conversion again and then ProManager3. After six hours I had finally restored my precious samples. PHEW.

Moral of this story, don't work on complex things at the end of an evening session when you are tired and make sure you back-up the right things in the first place.

If this ever happens to you, you know who to ask. My rates are reasonable :-)

3 comments:

jsd said...

I got that old familiar nauseous feeling in the pit of my stomach just reading this. Similar things have happened to me although luckily nothing at quite this level of magnitude.

Let this be a lesson to all of us: figure out your backup strategy BEFORE you start working!

Gary said...

Guess what Matt I am doing a full back-up of my cubase stuff, audio files created etc.

I'm glad I read this now and very pleased you have managed to recover your work.

Hopefully you can now have a good weekend!

Gary

omni said...

Really makes one sick if this happens. I still remember it with a bad feeling in the stomache when I lost all of my single tracks and pieces of my first two musical demo-albums many years ago...

Good luck, this never happens again!

PS: Don't forget to backup the new songs twice as we're all waiting for the new album