[X4U] Duplicate files

[Stroller]

On Nov 30, 2005, at 6:14 pm, Richard Gilmore wrote:
>>
>> ... IMHO the best way to remove
>> duplicates already within iPhoto would be to use an Applescript to 
>> call
>> the iPhoto delete functions, so that iPhoto is aware of the file
>> changes;
>
> What sort of Applescript? Would I have to write one? I don't know the 
> first
> thing about Applescript and I have no idea how I would use it to 
> "call" the
> iPhoto delete function and make it compare every file name.

Yeah, I guess you would. I don't know any Applescript, either - I was 
just supposing on the basis of what little I do know - so I guess 
there's not much point in discussing this further.

>> I'd personally MD5 the files to ensure they're duplicates
>> you're removing, not just ones with the same name (and so that you
>> don't get duplicates with different names).
>
> MD5? Mulitple Delete?

 From `man md5`

DESCRIPTION
      Md5 takes as input a message of arbitrary length and produces as 
output a
      128-bit ``fingerprint'' or ``message digest'' of the input.  It is 
con-
      jectured that it is computationally infeasible to produce two 
messages
      having the same message digest, or to produce any message having a 
given
      prespecified target message digest.  The MD5 algorithm is intended 
for
      digital signature applications,

So basically if you run `md5 /path/to/file` it gives a "magic number" 
for that file. If you get two files with the same md5 then they're both 
the same, irrespective of filename or anything else. If you have two 
images with different md5s then they're different, even if they look 
the same (one could be the low-res version you made to send by email, 
for instance) or if they have the same filename.

>> IMHO the best way to handle all your back-ups is to delete all of 
>> those
>> from iPhoto & start again.
>
> This gives me a headache just thinking about it. I've got over 18,000 
> photos

Well, I was just thinking of the ones that you'd imported recently from 
these DVDs. I assumed they'd be easy to find because they all been 
imported in recent weeks.

>> Copy all the photos off the backups into
>> _folders_ then you can trivially run `for foo in `ls -R`; do md5 $foo 
>> ;
>> other stuff ; done`.
>
> I don't anything about Unix either. "foo"?

"foo" is an arbitrary variable, in this case referenced by "$foo".

At a command prompt try typing:
   for foo in `ls` ; do echo "There is a file in this directory called 
$foo" ; done


I'm not necesserily saying that YOU should learn Applescript or Bash 
scripting, or md5 all your files - learning how to do this might take 
longer than removing all the duplicates by hand (but at least you'll 
save time the next time you have a repetitive task) just that it's the 
way I would do it. I'd like to be sure that I had a working automated 
process so that I could just run it & be sure that I hadn't made any 
mistakes removing files by hand.

Stroller.