Damir Dezeljin
2002-Mar-19 06:44 UTC
[Samba] Problems with offline files and Explorer 'hangs'
I'm working on a HSM product for Linux. The product has to handle file writing and reading trough samba. When there remains only for ex. 10% of free disk space, the product convert files that aren't accessed for a while to offline files - product marks files offline and removes files contents from disk and make it sparse file. When some client from samba tries to access one of offline files, the product put the samba process to sleep (kernel sleep) until the file is recalled. I encountered two problems: - When someone with windows explorer goes into an exported share, read events are generated for .exe files and this cause the recall (explorer try to get an icon data from the file). - When someone tries to copy a directory with offline files from samba export to local HDD, the samba process on the server is put to sleep and explorer "hangs" until the process is released from the sleep And now questions: - Starting with NTFS 5, M$ support an "offline" file attribute. If the server is located on Win2k or greater, clients "detect" this attribute and don't try to read an icon data from the file (don't trigger a read event). Is there a way to mark a file on samba share with this attribute? If not, where I have to hack sources (I'm not familiar with samba sources)? - Is there a way to configure samba so that a client accessing one SAMBA export produce more processes or threads to avoid Explorer hangs? Regards, Dezo
David Collier-Brown
2002-Mar-19 12:04 UTC
[Samba] Problems with offline files and Explorer 'hangs'
Damir Dezeljin wrote:> - Starting with NTFS 5, M$ support an "offline" file attribute. If the > server is located on Win2k or greater, clients "detect" this attribute and > don't try to read an icon data from the file (don't trigger a read event). > Is there a way to mark a file on samba share with this attribute? If not, > where I have to hack sources (I'm not familiar with samba sources)?Hmmn, this could also be set to mimimize the time taken to scan large directories. reading files to see which pretty icon to use. --dave -- David Collier-Brown, | Always do right. This will gratify Performance & Engineering | some people and astonish the rest. Americas Customer Engineering, | -- Mark Twain (905) 415-2849 | davecb@canada.sun.com
Ulf Bertilsson
2002-Mar-20 01:18 UTC
[Samba] Problems with offline files and Explorer 'hangs'
> > - Starting with NTFS 5, M$ support an "offline" file > attribute. If the > > server is located on Win2k or greater, clients "detect" > this attribute > > and don't try to read an icon data from the file (don't > trigger a read > > event). Is there a way to mark a file on samba share with this > > attribute? If not, where I have to hack sources (I'm not > familiar with > > samba sources)? > > > Hmmn, this could also be set to mimimize the > time taken to scan large directories. reading > files to see which pretty icon to use.Indeed. On my OS this is TERRIBLE when using "explorer" with "Web" view enabled. If you look with an file IO utility on the host beeing browsed this way the preformance is terible as well as every file is acces several times. By configuring the config for "explorer" to not use Web as well as other grapical stuff will prove 2-3 times increase in practical browsing preformance. I did have some logs and captures to illustrate this. I was thinking about pacthing the IO function to give generic file data to explorer at firxst read with some dummy cache to realy spice up the preformance. I can live without icons. Interesting stuff this, I use "Dopus" explorer replacement so I don't have this problem right now :) Is this attribute on file system level or in the SMB packet ? -- Ulf
MCCALL,DON (HP-USA,ex1)
2002-Mar-20 06:44 UTC
[Samba] Problems with offline files and Explorer 'hangs'
Hi Jeremy, I think this whole 'offline' file attribute thing is being misread (maybe by me...). But I THINK what this is really doing is when you are connected to a network share, you have the ability to mark files or folders to be "available offline". And what this REALLY does is make local copies of the files/folders, so that if you break your network connection, they are still available locally, and then get synced up later when the network connection is restored. So the idea of using this for minimizing scan times, etc, really isn't pertinent; you're not trying to read icon data (or any other data) off the SERVER version of the file because you are actually accessing the LOCAL copy of the file... You can do this on a directory or file on a samba server as well - really isn't a 'server-side' implementation. Just use a win2k client, bring up your samba share, and right click on a directory in it, and choose "make available offline". It will 'syncronize' the directory (for which read copy every damn file in the directory onto your local hard disk), and now the icon for the folder will change to have a little double vertical arrow icon superimopsed over it (indicating an offline capable directory).... Hope this helps, Don -----Original Message----- From: jra@samba.org [mailto:jra@samba.org] Sent: Tuesday, March 19, 2002 3:44 PM To: David.Collier-Brown@Sun.COM Cc: Damir Dezeljin; samba@lists.samba.org; samba-technical@lists.samba.org Subject: Re: [Samba] Problems with offline files and Explorer 'hangs' On Tue, Mar 19, 2002 at 02:56:29PM -0500, David Collier-Brown wrote:> Damir Dezeljin wrote: > > > - Starting with NTFS 5, M$ support an "offline" file attribute. If the > > server is located on Win2k or greater, clients "detect" this attributeand> > don't try to read an icon data from the file (don't trigger a readevent).> > Is there a way to mark a file on samba share with this attribute? Ifnot,> > where I have to hack sources (I'm not familiar with samba sources)? > > > Hmmn, this could also be set to mimimize the > time taken to scan large directories. reading > files to see which pretty icon to use.What POSIX call do we use to detect an offline file ? Jeremy.
MCCALL,DON (HP-USA,ex1)
2002-Mar-20 11:37 UTC
[Samba] Problems with offline files and Explorer 'hangs'
Ahhh, Thanks for the clarification! Don -----Original Message----- From: Damir Dezeljin [mailto:programing@nib.si] Sent: Wednesday, March 20, 2002 10:21 AM To: MCCALL,DON (HP-USA,ex1) Cc: 'jra@samba.org'; David.Collier-Brown@Sun.COM; samba@lists.samba.org; samba-technical@lists.samba.org Subject: RE: [Samba] Problems with offline files and Explorer 'hangs' Hi Don.> I think this whole 'offline' file attribute thing is being misread (maybeby> me...). But I THINK what this is really doing is when you are connectedto> a network share, you have the ability to mark files or folders to be > "available offline". And what this REALLY does is make local copies ofthe> files/folders, so that if you ...No the problem isn't connected with Windows capabilities to make offline copies of shared resources. To ilustrate what I meant, I put an picture of my Win2k Explorer accessing a shared resource on Win2k server which has some offline files (that with 0 bytes filesize - I intentionally create those files with size == 0 bytes) - http://www.dezo.org/offline.jpg. Note the litle gear on the file icon. This means that the file is offline and Windows explorer shows an default icon associated with this file type. (this works with Win2k, WinXP and WinNT sp6 Windows Explrer) This feture is very usefullin combination with HSM products and with directory with a lot of files with icons. On NTFS this offline flag is representet with a bit set to 1 in 32bit metadata asociated with the file (lihe hidden or archive atribute) is than applications thing to do anything usefull with it. Regards, Dezo