netfilter buglog - Oct 2023 - [Bug 1716] New: The -s option does not always exclude stateful information for sets

https://bugzilla.netfilter.org/show_bug.cgi?id=1716

            Bug ID: 1716
           Summary: The -s option does not always exclude stateful
                    information for sets
           Product: nftables
           Version: 1.0.x
          Hardware: x86_64
                OS: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: nft
          Assignee: pablo at netfilter.org
          Reporter: kfm at plushkava.net

All of the following examples concern the exact same loaded ruleset.

Here is an example of stateful information being properly omitted.

# nft -s list set netdev filter block_bogons | head -n6
table netdev filter {
        set block_bogons {
                type ipv4_addr
                flags interval,timeout
                timeout 1d
                elements = { 0.0.0.0/8, 10.0.0.0/8,

Here is an example of stateful information not being omitted, despite the use
of the -s option. The second address has been obfuscated for reasons of
privacy.

# nft -s list set netdev filter dropped | head -n7
table netdev filter {
        set dropped {
                type ipv4_addr . ipv4_addr
                size 65535
                flags dynamic,timeout
                timeout 1h
                elements = { 146.88.240.17 . ___________ timeout 1h,

Here is another example of stateful information not being omitted. The
addresses have been obfuscated for reasons of privacy.

# nft -s list set ip filter meter_client | head -n7
table ip filter {
        set meter_client {
                type ipv4_addr
                size 131070
                flags dynamic,timeout
                timeout 2m30s
                elements = { ______________ limit rate 12/minute burst 30
packets timeout 2m30s, _____________ limit rate 12/minute burst 30 packets
timeout 2m30s,

The expected outcome is that specifying the -s option consistently excludes the
stateful information associated with the set elements.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20231025/11dacd8a/attachment-0001.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1716

kfm at plushkava.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |1461

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20231025/a6927d1c/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1716

--- Comment #1 from Pablo Neira Ayuso <pablo at netfilter.org> ---
(In reply to kfm from comment #0)
> All of the following examples concern the exact same loaded ruleset.
> 
> Here is an example of stateful information being properly omitted.
> 
> # nft -s list set netdev filter block_bogons | head -n6
> table netdev filter {
>         set block_bogons {
>                 type ipv4_addr
>                 flags interval,timeout
>                 timeout 1d
>                 elements = { 0.0.0.0/8, 10.0.0.0/8,
> 
> Here is an example of stateful information not being omitted, despite the
> use of the -s option. The second address has been obfuscated for reasons of
> privacy.
> 
> # nft -s list set netdev filter dropped | head -n7
> table netdev filter {
>         set dropped {
>                 type ipv4_addr . ipv4_addr
>                 size 65535
>                 flags dynamic,timeout
>                 timeout 1h
>                 elements = { 146.88.240.17 . ___________ timeout 1h,
-s/--stateful strips off 'expires' from the listing which tells you how
much
time is left for this element to expire. The 'timeout' specify the
timeout for
this element which might different from the default timeout.
> Here is another example of stateful information not being omitted. The
> addresses have been obfuscated for reasons of privacy.
> 
> # nft -s list set ip filter meter_client | head -n7
> table ip filter {
>         set meter_client {
>                 type ipv4_addr
>                 size 131070
>                 flags dynamic,timeout
>                 timeout 2m30s
>                 elements = { ______________ limit rate 12/minute burst 30
> packets timeout 2m30s, _____________ limit rate 12/minute burst 30 packets
> timeout 2m30s,
>
> The expected outcome is that specifying the -s option consistently excludes
> the stateful information associated with the set elements.
This shows 'limit rate' policy which does not exposed any internal
stateful
information such as current tokens in the bucket compared to counter and quota.

Not related to this report but it should be possible to strip off
'timeout'
when it is the same as the default set timeout, to avoid repetitive printing of
the same timeout for every element, ie. only expires would be shown. In such
case, if -s/--stateful is specified, then expires would not be not shown.

For the record, 'expires' is stripped off from -s/--stateful since:

commit 37988cf255e51efba0d81dbc43eb4f0a41e99813
Author: Varsha Rao <rvarsha016 at gmail.com>
Date:   Tue Jun 6 11:55:40 2017 +0530

    src: Remove expire information from list stateless ruleset.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20231025/1d4de092/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1716

--- Comment #2 from kfm at plushkava.net ---
(In reply to Pablo Neira Ayuso from comment #1)
> (In reply to kfm from comment #0)
> > All of the following examples concern the exact same loaded ruleset.
> > 
> > Here is an example of stateful information being properly omitted.
> > 
> > # nft -s list set netdev filter block_bogons | head -n6
> > table netdev filter {
> >         set block_bogons {
> >                 type ipv4_addr
> >                 flags interval,timeout
> >                 timeout 1d
> >                 elements = { 0.0.0.0/8, 10.0.0.0/8,
> > 
> > Here is an example of stateful information not being omitted, despite
the
> > use of the -s option. The second address has been obfuscated for
reasons of
> > privacy.
> > 
> > # nft -s list set netdev filter dropped | head -n7
> > table netdev filter {
> >         set dropped {
> >                 type ipv4_addr . ipv4_addr
> >                 size 65535
> >                 flags dynamic,timeout
> >                 timeout 1h
> >                 elements = { 146.88.240.17 . ___________ timeout 1h,
> 
> -s/--stateful strips off 'expires' from the listing which tells you
how much
> time is left for this element to expire. The 'timeout' specify the
timeout
> for this element which might different from the default timeout.
Ah, I see.
> 
> > Here is another example of stateful information not being omitted. The
> > addresses have been obfuscated for reasons of privacy.
> > 
> > # nft -s list set ip filter meter_client | head -n7
> > table ip filter {
> >         set meter_client {
> >                 type ipv4_addr
> >                 size 131070
> >                 flags dynamic,timeout
> >                 timeout 2m30s
> >                 elements = { ______________ limit rate 12/minute burst
30
> > packets timeout 2m30s, _____________ limit rate 12/minute burst 30
packets
> > timeout 2m30s,
> >
> > The expected outcome is that specifying the -s option consistently
excludes
> > the stateful information associated with the set elements.
> 
> This shows 'limit rate' policy which does not exposed any internal
stateful
> information such as current tokens in the bucket compared to counter and
> quota.
Ditto. Having been explained, it makes perfect sense. I think this issue can be
resolved as INVALID then. Pardon me for wasting your time with it.
> 
> Not related to this report but it should be possible to strip off
'timeout'
> when it is the same as the default set timeout, to avoid repetitive
printing
> of the same timeout for every element, ie. only expires would be shown. In
> such case, if -s/--stateful is specified, then expires would not be not
> shown.
Now that I understand, I am not sure that such is necessary. What would be
logical - at least to my mind - is for the element's timeout to be shown in
the
case that it was explicitly specified, including by the packet path,
irrespective of whether it matches the default timeout or not. That way, the
output would accurately reflect the present state of affairs. I'm not sure
whether it already works that way but if it does, I don't think anything
needs
to be changed.

Tangientally: another possibility could be to extend the purpose of the -o
option, so as to perform that kind of optimisation for people that truly want
it. It could thus appeal to different use-cases. For example, one might not
wish for this sort of optimisation to be applied in the course of inspecting a
set, but might do when if writing out a dynamically maintained set for later
inclusion by a reloaded ruleset. Then again, it could have the effect of making
it more difficult to reason with nft and/or increase the maintenance burden.

To summarise, wherever there is the option of either being accurate or
optimising towards terseness, I would tend towards accuracy on the basis that a
higher level of consistency is maintained (even in the case the optimisation
does not alter the overall meaning of the presented commands/declarations).

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20231025/180ea50e/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1716

--- Comment #3 from kfm at plushkava.net ---
s/when if/if/

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20231025/4f7b9dff/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1716

Pablo Neira Ayuso <pablo at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |pablo at netfilter.org,
                   |                            |phil at nwl.cc

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20231025/8721df66/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1716

--- Comment #4 from Pablo Neira Ayuso <pablo at netfilter.org> ---
Added Phil to Cc.

(In reply to Pablo Neira Ayuso from comment #1)
> Not related to this report but it should be possible to strip off
'timeout'
> when it is the same as the default set timeout, to avoid repetitive
printing
> of the same timeout for every element, ie. only expires would be shown. In
> such case, if -s/--stateful is specified, then expires would not be not
> shown.
> 
> For the record, 'expires' is stripped off from -s/--stateful since:
> 
> commit 37988cf255e51efba0d81dbc43eb4f0a41e99813
> Author: Varsha Rao <rvarsha016 at gmail.com>
> Date:   Tue Jun 6 11:55:40 2017 +0530
> 
>     src: Remove expire information from list stateless ruleset.
Hm, -s/--stateful skips 'expires' in the listing.

I had a recent discussion with Phil on what 'nft reset' should be
resetting.
Now 'nft reset' only resets stateful objects that are declared in the
ruleset,
not timeouts.

Probably this needs to be changed to get things aligned and provide an option
not to print the 'expires' part of the timeout to keep things
consistent.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20231025/546b21b5/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1716

--- Comment #5 from Phil Sutter <phil at nwl.cc> ---
(In reply to Pablo Neira Ayuso from comment #4)
> Added Phil to Cc.
> 
> (In reply to Pablo Neira Ayuso from comment #1)
> > Not related to this report but it should be possible to strip off
'timeout'
> > when it is the same as the default set timeout, to avoid repetitive
printing
> > of the same timeout for every element, ie. only expires would be
shown. In
> > such case, if -s/--stateful is specified, then expires would not be
not
> > shown.
> > 
> > For the record, 'expires' is stripped off from -s/--stateful
since:
> > 
> > commit 37988cf255e51efba0d81dbc43eb4f0a41e99813
> > Author: Varsha Rao <rvarsha016 at gmail.com>
> > Date:   Tue Jun 6 11:55:40 2017 +0530
> > 
> >     src: Remove expire information from list stateless ruleset.
> 
> Hm, -s/--stateful skips 'expires' in the listing.
The long-option form of '-s' is '--stateless'. I guess you are
aware, but the
typo might confuse readers.

AIUI, a set element's expires value is (part of) it's state, while its
timeout
value is not. The latter is similar to the set's timeout value, just a
"configuration detail". Therefore, --stateless option should strip
expires
value but keep timeout value.
> I had a recent discussion with Phil on what 'nft reset' should be
resetting.
> Now 'nft reset' only resets stateful objects that are declared in
the
> ruleset, not timeouts.
> 
> Probably this needs to be changed to get things aligned and provide an
> option not to print the 'expires' part of the timeout to keep
things
> consistent.
It is your decision to make, I deliberately included the expires value into set
element reset.

Cheers, Phil

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20231026/f155450b/attachment.html>