[X-Unix] Re: Pokey LAN transfers (Resolved)

[Richard Hartman]

Thanks to Wing Wong, Nick Scalise, David Ledger and Brian Medley for  
their thoughtful responses. For some reason, Wing Wong's detailed  
response didn't make it to the list and is reproduced (far) below.

Here is a summary of what I learned:

The overhead on my setup of mounting a volume on the desktop (via  
AFP) is evidently enormous:

- Mounting a LAN volume on the local desktop and then using cp on a  
large (2GB) file using a terminal window (copying from mounted LAN  
volume to internal disk drive)  produced transfer rates of only about  
25 megabits/sec.

- Unmounting the LAN volume and instead using scp or rsync (and the  
IP address of the source mac) produced the expected transfer rate of  
100 megabits/sec, which was the speed limit expected by the slowest  
network element in the loop (the 100 mbps nic on one of the macs).

Regards,

Rich

> On Dec 26, 2007 1:54 PM, Richard Hartman < seasoft at west.net> wrote:
> I'm looking for guidance on how best to copy large data sets between
> macs on a network (Finder, terminal cp, other?).
>
> The setup: Copying from mac #1 (a 2006 iMac running 10.4.11 with
> built-in 100bps ethernet) to mac #2 (a recent PBPro running 10.5.1
> with built-in Gigabit ethernet). Macs connected by a Gigabit netgear
> switch.
>
> I had hoped to get sustained transfers between these two macs of
> 50-70 mbps (throttled by the rate-limiting 100 mbps iMac capability).
>
> However, copying a single 2 GByte file, by mounting the (Tiger) imac
> on the (Leopard) MBPro desktop and using terminal "cp" command from
> the MBPro terminal, results in a sustained transfer rate of only
> about 2.5 mB/sec (25 mbps).
>

On Dec 26, 2007, at 4:19 PM, Wing Wong wrote:

> Several factors will limit your transfer speeds...
>
> Based on what you are describing:
>
> imac(100mbps nic) <==> NetGear Gigabit SW <==> PBPro(1gbps nic)
>
> You are hard limited to 100mbps FD, if you are lucky.
>
> The switch will downgrade the port the imac is connected to, to  
> 100mbps FD. The port that the PB is connected to will be detected  
> at 1gbps FD.
>
> When you try to communicate with the imac, your jumbo frames (6000  
> bytes or so, vs the 1500 bytes for a normal 100mbps packet), won't  
> work, so you will also get downgraded to the 1500 byte packet size.  
> Hooray! Since this autodetection needs to be done after the fact,  
> you will suffer a performance penalty for mixing and matching  
> 100mbps and 1gbps gear. For normal web surfing, this isn't an  
> issue, since your performance loss over broadband is so much worse  
> than the performance loss due to the retransmit/conversion.
>
> The situation CAN be helped if you connect the two together  
> directly. They will autodetect at 100mbps FD on both ports and  
> communicate natively.
>
> So... question: How have you mounted the two systems? If you are  
> mounting via AFP/SMB, then you are now also suffering protocol  
> overhead for the filesystem mounting methodology. It's something  
> like 10%-20%.
>
> Basically... your loss looks something like this. Bearing in mind,  
> this is an eyeball figure from the last time I encountered a  
> problem like this:
>
> Typical Packet overhead = ~1-5% (best) , ~5-10% (worse), 15%+  
> (something wrong...)
>
> Typical SMB/AFP/NFS/etc mounted filesystem overhead = ~5-10%  
> (normal) , ~10-15% (tons of small files and atime/mtime records) ,  
> ~15%+ (something wrong...)
>
> So... yeah, given the setup you are describing... losing up to the  
> 30%-40% mark is definitely possible. This loss doesn't factor in  
> the loss due to the local filesystem, memory/cache(since you are  
> using a networked filesystem, there is heavy memory buffering), and  
> issues related to hard disk transfer speeds.
>
> If you want the fastest possible transfer of storage between  
> boxes... or rather, how I normally xfer files quickly between boxes:
>
> 1) setup ssh keys between the two boxes
> 2) use rsync to copy files between systems, scp, or equivelent.
>
> rsync has the added benefit of being able to maintain sync between  
> two machines for a variety of folders.
>
> If you MUST mount a filesystem, and if you can stand to go without  
> certain bits being updated, you can mount a remote filesystem as  
> either read-only, or with noatime, nomtime, so that each read isn't  
> prefixed/postfixed with a write to the source machine to update  
> it's "last accessed" time bits.
>
> The other option would be to attach a USB2.0 Gigabit network  
> adapter to the imac, or use FW networking between the Macs. The  
> adventurous might want to give FW based networking a try, since  
> it's natively supported on the Mac platforms. While the top speed  
> is 400mbps and 800mbps, depending on the cards, you might be able  
> to get some serious performance boost since the processing is done  
> by the firewire chipsets and not by the CPU, thus allowing things  
> like DMA.
>
> Good luck!
>