samba - Oct 2008 - Profile loading/saving on gigabit network only runs at fast ethernet speeds! Help!

Hello everyone,

I could really use some help trying to diagnose a tricky issue within the
domain
I have set up.

I am using samba-3.0.28-1.el5_2.1 and openldap-2.3.27-8.el5_2.4 on a server
running Centos 5.1.

I have everything working properly within the domain, users can log in,
netlogon
scripts are run, their profile is loaded, etc.

I have been extremely happy with this software and can't thank the makers
enough
- great work!

The issue I am having is with the speed of profile loading/saving. This
network
is hosted on a Gigabit Lan - this is a classroom environment where students
must
save large files in their profile (virtual machines actually = ~3-4GB per
profile & will grow...).

I know the best option is to save the work elsewhere, like their mapped home
directory, but it would be highly inconvenient to copy the VM back and forth
every time they wanted to use it (although much faster than the logon/logoff
speeds).

Let me elaborate: I have done speed tests transferring files and directories
directly to writable shares. I have monitored the network speeds with iptraf
on
the server while downloading/uploading a 1 GB or file or large directory on
the
relevant interface (there are 4 local interfaces) and these are my
approximate
results:

DL from Samba Share: 300,000 - 400,000 kb/s
UPL to Samba Share: 300,000 - 400,000 kb/s

These speeds are GREAT! I was blown away! -- But then take a look at the
speeds
at which the network operates when loading/saving a user's profile while
logging
in/out:

Profile Logoff: 50,000 kb/s (a ~3GB profile takes roughly 10 min to load)
Profile Logon: Not Tested - but observed to take about 10 minutes

So my dilemma is that, while the networking is functioning exceptionally
well in
all other areas, it just takes so long to login/logoff! The users need to
have
roaming profiles (they do switch machines sometimes) & I would like them to
be
able to store a lot of data inside their profile without speed becoming too
much
of an issue like it is now.

I have messed around with a few tweaking options in the smb.conf file, but
they
have not helped so far. So, I just went back to using TCP_NODELAY. I have
turned
off oplocks as well.

I am not sure how to proceed now and I am pretty much taking random stabs in
the
dark with this issue, any help would be GREATLY appreciated!

Thank you all,

Jonathan

On 10/14/2008, Jonathan Bougher (jonathanbougher@gmail.com)
wrote:
> Profile Logoff: 50,000 kb/s (a ~3GB profile takes roughly 10 min to load)
Using roaming profiles for profiles so large is - well - insane.

You won't get much better performance...

I'd look for another way (than using huge roaming profiles) to
accomplish your goal.

-- 

Best regards,

Charles

Ouch - insane is not what I was going for, hahaha jk. I just hoped that the
gigabit speeds would better support a larger profile.

If I was getting the full DL/UL speeds for profile transfer it would not be
such an issue because it would only take about a minute and a half.

But I see your wisdom Charles because the profiles can definitely grow much
larger and sooner or later I would be stuck once again...it's just those
dang big VM files!!!

Mike,

The VM's are actually just the VM Files that VMware Server uses to load the
Virtual OS. The students created their own VM's with Vmware Server and they
need to use them periodically (they can screw them up, wehatever they want,
it is a learning environment & then they can rebuild).

The Desktops are running a locally installed WinXP Pro and use the Samba
Server as PDC. The VM files themselves are the big problem here because they
run about 3 GB's - and each student must have his/her own.

I don't fully undertand you solution, but I did pick up on the fact that I
could create a local directory on the client machines in which users could
house their VMs. This would be a great Idea, but the only downside is that
it would limit their mobility.

I do not mind working around this issue in some way, and I really appreciate
all the help so far! In a perfect world however, I would like to diagnose
the issue with the speeds. Why is it that profile loading/saving is SO much
slower than other data transfer? Mabey I am missing something as far as how
profiles are copied/saved. Could someone clue me in and help me work out a
way to speed it up, or else then I will take one of the other routes

Thanks so much!

Jonathan

where are you saving the VM at?  My documents?  you can redirect my 
documents as well as most folders under c:\documents and settings\user

Jonathan Bougher wrote:
> Hello everyone,
>
> I could really use some help trying to diagnose a tricky issue within the
> domain
> I have set up.
>
> I am using samba-3.0.28-1.el5_2.1 and openldap-2.3.27-8.el5_2.4 on a server
> running Centos 5.1.
>
> I have everything working properly within the domain, users can log in,
> netlogon
> scripts are run, their profile is loaded, etc.
>
> I have been extremely happy with this software and can't thank the
makers
> enough
> - great work!
>
> The issue I am having is with the speed of profile loading/saving. This
> network
> is hosted on a Gigabit Lan - this is a classroom environment where students
> must
> save large files in their profile (virtual machines actually = ~3-4GB per
> profile & will grow...).
>
> I know the best option is to save the work elsewhere, like their mapped
home
> directory, but it would be highly inconvenient to copy the VM back and
forth
> every time they wanted to use it (although much faster than the
logon/logoff
> speeds).
>
> Let me elaborate: I have done speed tests transferring files and
directories
> directly to writable shares. I have monitored the network speeds with
iptraf
> on
> the server while downloading/uploading a 1 GB or file or large directory on
> the
> relevant interface (there are 4 local interfaces) and these are my
> approximate
> results:
>
> DL from Samba Share: 300,000 - 400,000 kb/s
> UPL to Samba Share: 300,000 - 400,000 kb/s
>
> These speeds are GREAT! I was blown away! -- But then take a look at the
> speeds
> at which the network operates when loading/saving a user's profile
while
> logging
> in/out:
>
> Profile Logoff: 50,000 kb/s (a ~3GB profile takes roughly 10 min to load)
> Profile Logon: Not Tested - but observed to take about 10 minutes
>
> So my dilemma is that, while the networking is functioning exceptionally
> well in
> all other areas, it just takes so long to login/logoff! The users need to
> have
> roaming profiles (they do switch machines sometimes) & I would like
them to
> be
> able to store a lot of data inside their profile without speed becoming too
> much
> of an issue like it is now.
>
> I have messed around with a few tweaking options in the smb.conf file, but
> they
> have not helped so far. So, I just went back to using TCP_NODELAY. I have
> turned
> off oplocks as well.
>
> I am not sure how to proceed now and I am pretty much taking random stabs
in
> the
> dark with this issue, any help would be GREATLY appreciated!
>
> Thank you all,
>
> Jonathan
>

I Do think the local hard disk speed is cause *some* limitation, but since I
can upload & download the same and other files and directories to the server
at 4x the speed, I do not think that is the main limitation.

I am still perplexed by this issue, why is it that any sort of normal
upload/download data transfer from server to client or vice-versa via
network shares works at speeds of roughly 400,000 kb/s when a profile being
copied back and forth at login/logout runs at a much slower 50,000 kb/s?

Iv'e just been frustrated by this issue and I would really like to get to
the bottom of this, thank you all for your patience and continued support.

Jonathan

On Thu, Oct 16, 2008 at 9:47 AM, Andrew Masterson <
Andrew.Masterson@nuvistaenergy.com> wrote:
> It might be due to the way you are testing - local hard drives usually
> can't make use of gigabit networks for downloading because their write
> speeds are too slow.
>
> -Andrew
>
>
> -------------------------------------
> > On Behalf Of Jonathan Bougher
> > Sent: Wednesday, October 15, 2008 4:08 PM
> > To: Adam Williams
> > Cc: samba@lists.samba.org
> > Subject: Re: [Samba] Profile loading/saving on gigabit network only
> runs
> > atfast ethernet speeds! Help!
> >
> > Ok that idea sounds golden, I will try it out the first chance I get.
> This
> > is a big improvement over the way it is now, it's a little harder
on
> > users,
> > but not much. I thank you very much!
> >
> > However, just because I am a curious kind of guy, I do not yet know
> > *why*such a slowdown occurs when saving a profile vs. just
> > uploading/downloading
> > to a share. Is this a result of the old profile having to be
> overwritten?
> > I
> > know I need to read up on this a little more but has anyone had direct
> > experience with this type of issue, or any ways to speed up that
> service
> > in
> > particular (or what could be slowing it down).
> >
> > Thank you all so much, you have already helped tremendously - please
> help
> > me
> > out just a little more to understand this better. :) Thanks!
> >
> > On Tue, Oct 14, 2008 at 2:39 PM, Adam Williams
> > <awilliam@mdah.state.ms.us>wrote:
> >
> > > page 211 of samba 3 by example.pdf has instructions on how to
> redirect
> > > folders using registry changes.  you can easily redirect my
> documents by
> > > right clicking on it and changing the target.  i don't save
files to
> my
> > > desktop because that just results in a cluttered desktop and
large
> > roaming
> > > profiles.  No I don't think VMWare will run much slower than
it is
> now.
> > In
> > > testing on my Poweredge servers, desktop PCs, etc, it seems in my
> > experience
> > > that hard drives max out at reads/writes of 25 megabytes a
second,
> and
> > even
> > > 100megabit transfers about 12 megabytes a second on a switched
> network.
> > I'd
> > > probably try a test, take a user, redirect my documents to a
folder
> on
> > the
> > > server, put the VM in it, and run it, and see how it affects your
> > > performance, you may not notice anything different.  I ran
Windows
> XP
> > and
> > > Fedora 8 on Microsoft Virtual PC in a test environment on my 100
> megabit
> > > notebook PC with the VMs stored on a shared drive on a file
server
> and I
> > > didn't notice much of a difference as running an operating
system
> > locally.
> > >
> > >
> > > Jonathan Bougher wrote:
> > >
> > >> The VM is being saved to the Desktop, within a folder (or
multiple
> > folders
> > >> if the user puts it there)
> > >>
> > >> If I redirect My Docs, and the VM files are there - Then
won't
> VMware
> > run
> > >> much more slowly trying to access this stuff across the
network
> when
> > the
> > >> user wants to run the VM? Profile loading/saving would speed
up,
> but
> > overall
> > >> performance would be impacted negatively I think
> > >>
> > >> Thanks for the reply, I will look into it further - do you
have any
> > >>  thoughts about what I stated above?
> > >>
> > >>
> > --
> > To unsubscribe from this list go to the following URL and read the
> > instructions:  https://lists.samba.org/mailman/listinfo/samba
>

On 10/16/2008 11:25 AM, Jonathan Bougher wrote:
> I Do think the local hard disk speed is cause *some* limitation, but since
I
> can upload & download the same and other files and directories to the
server
> at 4x the speed, I do not think that is the main limitation.
> 
> I am still perplexed by this issue, why is it that any sort of normal
> upload/download data transfer from server to client or vice-versa via
> network shares works at speeds of roughly 400,000 kb/s when a profile being
> copied back and forth at login/logout runs at a much slower 50,000 kb/s?
Maybe because it is not a simple copy... it has to compare all of the
files in the local copy of the profile to what is contained in the
remote stored profile... so it would be the comparison process that
causes it to be slower.

Now, if it is only copying one huge file, then maybe it could be
considered a bug, because I would think that the actual copy process
should be the same...

-- 

Best regards,

Charles