Copying Large Amounts of data via USB/FW (was Re: [X-Unix] Re: Pokey LAN transfers (Resolved))

[Wing Wong]

Hmmm

Depends on what you want to do better. Moving files around or
selecting the files you want to move.

I have a similar problem at home... Basically, the ever growing list
of drives from which to consolidate data from... And the ever filling
up drives the data is being consolidated to.

The problem, from what I can see in my setup and the one you are
describing, is one of tediousness... Ie, looking for files and mv'ing
them.

Some thoughts and suggestions:

1:

make sure you know which files are which and that there is no name
collisions. Since you are using hfs+, I'm guessing there is no case
sensitivity...

Your can perform a 'find' against the folders and drives to get a
complete list of file names. Likewise, you can get a nice listing
along with file sizes and datestamps/perms:

# to just list recursively
find /path/to/folder

# to list recursively and give file info
find /path/to/drive -ls

So, let's say you have the following paths:

/volumes/400gbdisk/
/volumes/10tbarray/

You have video/pictures/misc in the 10tbarray device and just want to
fire off a command and let it run without manually searching...

#
find /volumes/400gbdisk -name *.mpg -exec mv {} /volumes/10tbarray/videos/mpegs/

Or, for the more adventurous:

#
for ext in mpg avi wmv mp4 flv
do
   find /volumes/400gbdiskl -name *.${ext} -exec mv {}
/volumes/10tbarray/videos/
done

Note: experiment on test folders first... Just in case. The cautious
will replace the mv with a cp. ;)

You can also add some logic and perform a md5 sum hash against the
files first, to ensure they are unique... Or if two files are
similarly named, determine whether they are the same.

Not sure if that addressed your question or not. :/

You can also employ a local filesystem rsync to transfer files of a
particular extension. The benefit of rsync is that should the
operation be interrupted, you can always redo the command again...
Assuming it is formatted properly, lest you end up with
/volumes/10tbarray/videos/videos/videos/videos....


Wing




On 12/27/07, TjL <luomat at gmail.com> wrote:
> Since the previous issue is resolved, I wonder if I can fork the
> discussion to a somewhat related topic:
>
> I have a large amount of data (GBs and GBs) spread across several
> external HDs, some USB2, some Firewire 400.
>
> I am trying to consolidate them and organize them so that similar data
> is on the same drives, i.e. TV shows on one, movies on another, short
> home movies from our digital camera on another, etc.
>
> These range from a folder with dozens and dozens (if not hundreds) of
> smaller files (under 200mb) to folders with a bunch of DVD rips (4-8GB
> but a few files).
>
> All of the drives are HFS+ Journaled, attached to a 2.8Ghz iMac
> running Leopard either by USB (powered hubs) or daily chained Firewire
> drives.
>
> I've pretty much given up on using Finder because it gives such
> useless error messages, and am using 'mv -iv' in Terminal.
>
> Is there a better/faster way to do this?
>
> Thanks!
>
> TjL
> _______________________________________________
> X-Unix mailing list
> X-Unix at listserver.themacintoshguy.com
> http://listserver.themacintoshguy.com/mailman/listinfo/x-unix
>


-- 
Wing Wong
wingedpower at gmail.com