Btrfs devel - Oct 2013 - swapfile on btrfs, temporary solution for wiki

Hello list, i know what btrfs  don''t support swap files.
I read arch wiki and when i reading about systemd addon for auto
create swapfile on btrfs, i invent the way, how create and using swap
file, just see following sh code:

swapfile=$(losetup -f) #free loop device
truncate -s 8G /swap   #create 8G sparse swap file
losetup $swapfile /swap #mount file to loop
mkswap  $swapfile
swapon  $swapfile

i just adding this to rc.local and this just work.
May be, add it to Wiki as temporary solution to using swap file?
(sorry for my bad english)
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Hello, i suggest temporary solution to use swap file under btrfs.
I test it, and it work good.

I invent simple the way, how create and using swap file, just see
following sh code:

swapfile=$(losetup -f) #free loop device
truncate -s 8G /swap   #create 8G sparse swap file
losetup $swapfile /swap #mount file to loop
mkswap  $swapfile
swapon  $swapfile

i just adding this to rc.local and this work good.
May be, add it to btrfs Wiki as temporary solution to using swap file?

Timofey

2013/10/21 Тимофей Титовец <nefelim4ag@gmail.com>:
> Hello list, i know what btrfs  don''t support swap files.
> I read arch wiki and when i reading about systemd addon for auto
> create swapfile on btrfs, i invent the way, how create and using swap
> file, just see following sh code:
>
> swapfile=$(losetup -f) #free loop device
> truncate -s 8G /swap   #create 8G sparse swap file
> losetup $swapfile /swap #mount file to loop
> mkswap  $swapfile
> swapon  $swapfile
>
> i just adding this to rc.local and this just work.
> May be, add it to Wiki as temporary solution to using swap file?
> (sorry for my bad english)
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On 10/25/2013 05:52 AM, Timofey Titovets wrote:
> Hello, i suggest temporary solution to use swap file under btrfs.
> I test it, and it work good.
>
> I invent simple the way, how create and using swap file, just see
> following sh code:
>
> swapfile=$(losetup -f) #free loop device
> truncate -s 8G /swap   #create 8G sparse swap file
> losetup $swapfile /swap #mount file to loop
> mkswap  $swapfile
> swapon  $swapfile
>
> i just adding this to rc.local and this work good.
> May be, add it to btrfs Wiki as temporary solution to using swap file?
>
> Timofey
Hi,
Yes, Timofey, it does work. I tested the same over a year ago. And it 
does say briefly about this option in wiki

https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F
"A workaround, albeit with poor performance, is to mount a swap file via 
a loop device."

best
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Thu, 24 Oct 2013 23:52:01 +0300
Timofey Titovets <nefelim4ag@gmail.com> wrote:
> Hello, i suggest temporary solution to use swap file under btrfs.
> I test it, and it work good.
> 
> I invent simple the way, how create and using swap file, just see
> following sh code:
> 
> swapfile=$(losetup -f) #free loop device
> truncate -s 8G /swap   #create 8G sparse swap file
> losetup $swapfile /swap #mount file to loop
> mkswap  $swapfile
> swapon  $swapfile
> 
> i just adding this to rc.local and this work good.
> May be, add it to btrfs Wiki as temporary solution to using swap file?
I always thought Btrfs does not allow swap files on purpose, because it is not
deadlock-proof when used in the swapping context.

Imagine you try swapping out pages to free up some memory, and in the process
Btrfs needs to allocate some memory to actually perform the write, the kernel
says "Sure, but for that we need to swap out some more pages..." You
see where
that goes.

Same issue is possible with swap on other complex filesystems an example
being networked ones like NFS and SMB/CIFS.

It might work for some time (or even work 99% of time), but you still may be
endangering your system to a possibility of a lock-up, and certainly adding
that to any Wiki/FAQ/website as "the solution" might not be the best
choice.

-- 
With respect,
Roman

On Fri, Oct 25, 2013 at 04:35:46PM +0600, Roman Mamedov
wrote:
> On Thu, 24 Oct 2013 23:52:01 +0300
> Timofey Titovets <nefelim4ag@gmail.com> wrote:
> 
> > Hello, i suggest temporary solution to use swap file under btrfs.
> > I test it, and it work good.
> > 
> > I invent simple the way, how create and using swap file, just see
> > following sh code:
> > 
> > swapfile=$(losetup -f) #free loop device
> > truncate -s 8G /swap   #create 8G sparse swap file
> > losetup $swapfile /swap #mount file to loop
> > mkswap  $swapfile
> > swapon  $swapfile
> > 
> > i just adding this to rc.local and this work good.
> > May be, add it to btrfs Wiki as temporary solution to using swap file?
> 
> I always thought Btrfs does not allow swap files on purpose, because it is
not
> deadlock-proof when used in the swapping context.
   It''s more that the current swap interface is based on device+block
list, and if you balance a filesystem, the blocks for a file move --
but there''s no way of telling the swap code to cope with that.
> Imagine you try swapping out pages to free up some memory, and in the
process
> Btrfs needs to allocate some memory to actually perform the write, the
kernel
> says "Sure, but for that we need to swap out some more pages..."
You see where
> that goes.
> 
> Same issue is possible with swap on other complex filesystems an example
> being networked ones like NFS and SMB/CIFS.
   The network filesystems have a similar problem as btrfs -- they
don''t export devices, and you don''t get direct access to the
low-level
blocks under the FS, so the swap code can''t deal with it.

   That said, there''s been a lot of work recently on getting
swap-over-NFS to work properly -- effectively giving a new interface
for the swap code that doesn''t rely on direct mapping to device
blocks. That new interface gives us the minimal external
infrastructure necessary to consider doing swapfiles on btrfs.
> It might work for some time (or even work 99% of time), but you still may
be
> endangering your system to a possibility of a lock-up, and certainly adding
> that to any Wiki/FAQ/website as "the solution" might not be the
best choice.
   The deadlock situation is dealt with by adding a flag to the memory
allocator in the swap-critical paths, which says you''re not allowed to
swap anything when you make the allocation. That at least allows the
memory allocation to fail (hopefully gracefully) without deadlocking.
I suspect that this is also part of the swap-on-NFS work -- adding
that flag everywhere necessary.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
       --- He''s playing Schubert.  I think Schubert is losing. ---

Is the NFS over network code upstream ?

On 10/25/13, Hugo Mills <hugo@carfax.org.uk>
wrote:
> On Fri, Oct 25, 2013 at 04:35:46PM +0600, Roman Mamedov wrote:
>> On Thu, 24 Oct 2013 23:52:01 +0300
>> Timofey Titovets <nefelim4ag@gmail.com> wrote:
>>
>> > Hello, i suggest temporary solution to use swap file under btrfs.
>> > I test it, and it work good.
>> >
>> > I invent simple the way, how create and using swap file, just see
>> > following sh code:
>> >
>> > swapfile=$(losetup -f) #free loop device
>> > truncate -s 8G /swap   #create 8G sparse swap file
>> > losetup $swapfile /swap #mount file to loop
>> > mkswap  $swapfile
>> > swapon  $swapfile
>> >
>> > i just adding this to rc.local and this work good.
>> > May be, add it to btrfs Wiki as temporary solution to using swap
file?
>>
>> I always thought Btrfs does not allow swap files on purpose, because it
is
>> not
>> deadlock-proof when used in the swapping context.
>
>    It''s more that the current swap interface is based on
device+block
> list, and if you balance a filesystem, the blocks for a file move --
> but there''s no way of telling the swap code to cope with that.
>
>> Imagine you try swapping out pages to free up some memory, and in the
>> process
>> Btrfs needs to allocate some memory to actually perform the write, the
>> kernel
>> says "Sure, but for that we need to swap out some more
pages..." You see
>> where
>> that goes.
>>
>> Same issue is possible with swap on other complex filesystems an
example
>> being networked ones like NFS and SMB/CIFS.
>
>    The network filesystems have a similar problem as btrfs -- they
> don''t export devices, and you don''t get direct access to
the low-level
> blocks under the FS, so the swap code can''t deal with it.
>
>    That said, there''s been a lot of work recently on getting
> swap-over-NFS to work properly -- effectively giving a new interface
> for the swap code that doesn''t rely on direct mapping to device
> blocks. That new interface gives us the minimal external
> infrastructure necessary to consider doing swapfiles on btrfs.
>
>> It might work for some time (or even work 99% of time), but you still
may
>> be
>> endangering your system to a possibility of a lock-up, and certainly
>> adding
>> that to any Wiki/FAQ/website as "the solution" might not be
the best
>> choice.
>
>    The deadlock situation is dealt with by adding a flag to the memory
> allocator in the swap-critical paths, which says you''re not
allowed to
> swap anything when you make the allocation. That at least allows the
> memory allocation to fail (hopefully gracefully) without deadlocking.
> I suspect that this is also part of the swap-on-NFS work -- adding
> that flag everywhere necessary.
>
>    Hugo.
>
> --
> === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk
==>   PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
>        --- He''s playing Schubert.  I think Schubert is losing. ---
>

-- 
Karim Allah Ahmed.
LinkedIn <http://eg.linkedin.com/pub/karim-allah-ahmed/13/829/550/>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Is the swap over NFS code upstream ?

On 10/25/13, Hugo Mills <hugo@carfax.org.uk>
wrote:
> On Fri, Oct 25, 2013 at 04:35:46PM +0600, Roman Mamedov wrote:
>> On Thu, 24 Oct 2013 23:52:01 +0300
>> Timofey Titovets <nefelim4ag@gmail.com> wrote:
>>
>> > Hello, i suggest temporary solution to use swap file under btrfs.
>> > I test it, and it work good.
>> >
>> > I invent simple the way, how create and using swap file, just see
>> > following sh code:
>> >
>> > swapfile=$(losetup -f) #free loop device
>> > truncate -s 8G /swap   #create 8G sparse swap file
>> > losetup $swapfile /swap #mount file to loop
>> > mkswap  $swapfile
>> > swapon  $swapfile
>> >
>> > i just adding this to rc.local and this work good.
>> > May be, add it to btrfs Wiki as temporary solution to using swap
file?
>>
>> I always thought Btrfs does not allow swap files on purpose, because it
is
>> not
>> deadlock-proof when used in the swapping context.
>
>    It''s more that the current swap interface is based on
device+block
> list, and if you balance a filesystem, the blocks for a file move --
> but there''s no way of telling the swap code to cope with that.
>
>> Imagine you try swapping out pages to free up some memory, and in the
>> process
>> Btrfs needs to allocate some memory to actually perform the write, the
>> kernel
>> says "Sure, but for that we need to swap out some more
pages..." You see
>> where
>> that goes.
>>
>> Same issue is possible with swap on other complex filesystems an
example
>> being networked ones like NFS and SMB/CIFS.
>
>    The network filesystems have a similar problem as btrfs -- they
> don''t export devices, and you don''t get direct access to
the low-level
> blocks under the FS, so the swap code can''t deal with it.
>
>    That said, there''s been a lot of work recently on getting
> swap-over-NFS to work properly -- effectively giving a new interface
> for the swap code that doesn''t rely on direct mapping to device
> blocks. That new interface gives us the minimal external
> infrastructure necessary to consider doing swapfiles on btrfs.
>
>> It might work for some time (or even work 99% of time), but you still
may
>> be
>> endangering your system to a possibility of a lock-up, and certainly
>> adding
>> that to any Wiki/FAQ/website as "the solution" might not be
the best
>> choice.
>
>    The deadlock situation is dealt with by adding a flag to the memory
> allocator in the swap-critical paths, which says you''re not
allowed to
> swap anything when you make the allocation. That at least allows the
> memory allocation to fail (hopefully gracefully) without deadlocking.
> I suspect that this is also part of the swap-on-NFS work -- adding
> that flag everywhere necessary.
>
>    Hugo.
>
> --
> === Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk
==>   PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
>        --- He''s playing Schubert.  I think Schubert is losing. ---
>

-- 
Karim Allah Ahmed.
LinkedIn <http://eg.linkedin.com/pub/karim-allah-ahmed/13/829/550/>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Sat, Oct 26, 2013 at 10:30:18AM +0100, karim.allah.ahmed@gmail.com
wrote:
> Is the swap over NFS code upstream ?
   Yes:

Symbol: NFS_SWAP [=y]
Type  : boolean
Prompt: Provide swap over NFS support
  Location:
    -> File systems
      -> Network File Systems (NETWORK_FILESYSTEMS [=y])
(2)     -> NFS client support (NFS_FS [=m])
  Defined at fs/nfs/Kconfig:89
  Depends on: NETWORK_FILESYSTEMS [=y] && NFS_FS [=m]
  Selects: SUNRPC_SWAP [=y]

   Hugo.
> On 10/25/13, Hugo Mills <hugo@carfax.org.uk> wrote:
> > On Fri, Oct 25, 2013 at 04:35:46PM +0600, Roman Mamedov wrote:
> >> On Thu, 24 Oct 2013 23:52:01 +0300
> >> Timofey Titovets <nefelim4ag@gmail.com> wrote:
> >>
> >> > Hello, i suggest temporary solution to use swap file under
btrfs.
> >> > I test it, and it work good.
> >> >
> >> > I invent simple the way, how create and using swap file, just
see
> >> > following sh code:
> >> >
> >> > swapfile=$(losetup -f) #free loop device
> >> > truncate -s 8G /swap   #create 8G sparse swap file
> >> > losetup $swapfile /swap #mount file to loop
> >> > mkswap  $swapfile
> >> > swapon  $swapfile
> >> >
> >> > i just adding this to rc.local and this work good.
> >> > May be, add it to btrfs Wiki as temporary solution to using
swap file?
> >>
> >> I always thought Btrfs does not allow swap files on purpose,
because it is
> >> not
> >> deadlock-proof when used in the swapping context.
> >
> >    It''s more that the current swap interface is based on
device+block
> > list, and if you balance a filesystem, the blocks for a file move --
> > but there''s no way of telling the swap code to cope with
that.
> >
> >> Imagine you try swapping out pages to free up some memory, and in
the
> >> process
> >> Btrfs needs to allocate some memory to actually perform the write,
the
> >> kernel
> >> says "Sure, but for that we need to swap out some more
pages..." You see
> >> where
> >> that goes.
> >>
> >> Same issue is possible with swap on other complex filesystems an
example
> >> being networked ones like NFS and SMB/CIFS.
> >
> >    The network filesystems have a similar problem as btrfs -- they
> > don''t export devices, and you don''t get direct
access to the low-level
> > blocks under the FS, so the swap code can''t deal with it.
> >
> >    That said, there''s been a lot of work recently on getting
> > swap-over-NFS to work properly -- effectively giving a new interface
> > for the swap code that doesn''t rely on direct mapping to
device
> > blocks. That new interface gives us the minimal external
> > infrastructure necessary to consider doing swapfiles on btrfs.
> >
> >> It might work for some time (or even work 99% of time), but you
still may
> >> be
> >> endangering your system to a possibility of a lock-up, and
certainly
> >> adding
> >> that to any Wiki/FAQ/website as "the solution" might not
be the best
> >> choice.
> >
> >    The deadlock situation is dealt with by adding a flag to the memory
> > allocator in the swap-critical paths, which says you''re not
allowed to
> > swap anything when you make the allocation. That at least allows the
> > memory allocation to fail (hopefully gracefully) without deadlocking.
> > I suspect that this is also part of the swap-on-NFS work -- adding
> > that flag everywhere necessary.
> >
> >    Hugo.
> >
-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ==  PGP
key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
                   --- ©1973 Unclear Research Ltd ---

karim.allah.ahmed@gmail.com posted on Sat, 26 Oct 2013 10:30:18 +0100 as
excerpted:
> Is the swap over NFS code upstream ?
?esrever ni txet referp uoy od ;tsop-pot t''nod esaelP

(Please don''t top-post; do you prefer text in reverse?)

Quote the context you need so your question makes sense and reply below 
it, and you''re more likely to get useful answers as it''ll be
less work to
reply properly.  Here''s how it should have looked, now quoted another 
level for my reply:
>> That said, there''s been a lot of work recently on getting
>> swap-over-NFS to work properly -- effectively giving a new interface
>> for the swap code that doesn''t rely on direct mapping to
device
>> blocks. That new interface gives us the minimal external
>> infrastructure necessary to consider doing swapfiles on btrfs.
>>
> Is the swap over NFS code upstream ?
The answer is yes, I''ve definitely seen commit comments about swap over
NFS on recent commits, so the work is indeed going in, tho it may well 
still be a WIP.

(I don''t use NFS here and in fact with 8-16 gigs RAM common these days,
I
don''t use swap so much any more either and often don''t even
have the
kernel swap option on for my current configs unless I''m using it for 
suspend-to-disk, so I''ve not followed the issue closely enough to know 
current in-kernel status, but based on the commits I''ve happened
across,
yes, the work is certainly going in, if the feature isn''t already there
and working in general and they''re simply tweaking it now.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

On Thu, Oct 24, 2013 at 11:52:01PM +0300, Timofey Titovets
wrote:
> Hello, i suggest temporary solution to use swap file under btrfs.
> I test it, and it work good.
> 
> I invent simple the way, how create and using swap file, just see
> following sh code:
> 
> swapfile=$(losetup -f) #free loop device
> truncate -s 8G /swap   #create 8G sparse swap file
I''d suggest to do full dd on that file and make it NOCOW first. That
way
it should minimize attempts to actually allocate the file blocks at the
time they''re needed for swap out.

Also a bit of stress testing that this actually works is needed.
> losetup $swapfile /swap #mount file to loop
> mkswap  $swapfile
> swapon  $swapfile

david
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

> I''d suggest to do full dd on that file and make it NOCOW first.
That way
> it should minimize attempts to actually allocate the file blocks at the
> time they''re needed for swap out.
I think this is good idea for HDD, on SSD this no make sence.
>
https://btrfs.wiki.kernel.org/index.php/FAQ#Does_btrfs_support_swap_files.3F
> "A workaround, albeit with poor performance, is to mount a swap file
via a loop device."
I did not pay attention to it, i think this is enough.
If nobody no want say addition comments, this thread can be closed.
Thanks to all, how answer.

With best regards,
Timofey
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Thanks to all, who* answer.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html