Hello List, I've searched the internet and I haven't been able to figure this out. I hope someone here can point me in the right direction. I have a MacOSX 10.3 machine and a WinXP machine that are being used to produce large video clips (@1gig). After they are rendered and ready for playback, they are copied to the playout machines using a system that uses watch folders shared out over samba. The watch folder works based on file size. For example, 40 seconds after a file's filesize stops growing, the system assumes the file is complete and starts moving the file onto the playout device. The problem I've found is that windows xp and mac osx (10.3 at least) both cause samba to allocate the entire file size at the start of the transfer. For large files this causes a problem since the file is not finished writing to the smb share when the playout transfer system takes the file and tries to move it and then delete it. Any help on how to change OSX and WinXP's behavior or how to keep samba from allocating disk space at the start of a write would be very appreciated. Thanks, Victor.
On Tue, Dec 21, 2004 at 01:09:59PM -0500, Victor Moran wrote:> Hello List, > > I've searched the internet and I haven't been able to figure this out. > I hope someone here can point me in the right direction. > > I have a MacOSX 10.3 machine and a WinXP machine that are being used to > produce large video clips (@1gig). After they are rendered and ready > for playback, they are copied to the playout machines using a system > that uses watch folders shared out over samba. > > The watch folder works based on file size. For example, 40 seconds > after a file's filesize stops growing, the system assumes the file is > complete and starts moving the file onto the playout device. > > The problem I've found is that windows xp and mac osx (10.3 at least) > both cause samba to allocate the entire file size at the start of the > transfer. For large files this causes a problem since the file is not > finished writing to the smb share when the playout transfer system > takes the file and tries to move it and then delete it. > > Any help on how to change OSX and WinXP's behavior or how to keep samba > from allocating disk space at the start of a write would be very > appreciated.If the client writes at the end of the file, and then fills in the gaps then there is nothing we can do to fix this in Samba. Who wrote the watch system ? Pre-allocating a known filesize by writing at the end then filling in the blank is a common idiom in file transfers, I would expect a watch system to deal with this. Jeremy.
Victor Moran wrote:>I have a MacOSX 10.3 machine and a WinXP machine that are being used to >produce large video clips (@1gig). After they are rendered and ready >for playback, they are copied to the playout machines using a system >that uses watch folders shared out over samba. > >The watch folder works based on file size. For example, 40 seconds >after a file's filesize stops growing, the system assumes the file is >complete and starts moving the file onto the playout device.A better way is to use temporary file names during the copy, and then rename it when complete (or copy to a folder and then move). Eg : copy <video_clip> to //server/share/<video_clip>_ move <video_clip>_ to <video_clip> Watching program ignored files that end with "_" Or : copy <video_clip> to //server/share/incoming/<video_clip> move //server/share/incoming/<video_clip> to //server/share/<video_clip> Watching program ignores contents of 'incoming'. There are other ways of course, such as adding a semaphore file after the file copy is complete and so on - but I prefer the above method as it is fairly simple to do at both ends, and the rename/move is near enough an atomic process as far as any watching process is concerned (the file is simply not there at all, then it appears and is known to be complete). Simon -- Simon Hobson MA MIEE, Technology Specialist Colony Gift Corporation Limited Lindal in Furness, Ulverston, Cumbria, LA12 0LD Tel 01229 461100, Fax 01229 461101 Registered in England No. 1499611 Regd. Office : 100 New Bridge Street, London, EC4V 6JA.