netfilter buglog - Nov 2018 - [Bug 1299] New: add set - syntax has changed - update documentation

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

            Bug ID: 1299
           Summary: add set - syntax has changed - update documentation
           Product: nftables
           Version: unspecified
          Hardware: x86_64
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: nft
          Assignee: pablo at netfilter.org
          Reporter: james at nurealm.net

Arch Linux
nftables 1:0.9.0-1

On the wiki:
https://wiki.nftables.org/wiki-nftables/index.php/Sets

the example is shown:

% nft add set filter blackhole { type ipv4_addr\;}

where man 8 nft shows:

add set [family] table set { type type ; ... }

the "family" is shown as optional.

"family" no longer appears to be optional.  Is this intentional?  Or a
bug?

$ sudo nft list tables
table inet filter
table ip nat
table ip private

$ sudo nft add set filter foreign \{ type ipv4_addr \; \}
Error: Could not process rule: No such file or directory
add set filter foreign { type ipv4_addr ; }
        ^^^^^^

But instead,

$ sudo nft add set inet filter foreign \{ type ipv4_addr \; \}

will successfully create the set:

$ sudo nft list ruleset
table inet filter {
        set foreign {
                type ipv4_addr
        }
...

If not a bug, then erroneous documentation is very much a problem.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20181116/315f3995/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

Florian Westphal <fw at strlen.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |fw at strlen.de

--- Comment #1 from Florian Westphal <fw at strlen.de> ---
(In reply to James Feeney from comment #0)
> $ sudo nft list tables
> table inet filter
> table ip nat
> table ip private
> 
> $ sudo nft add set filter foreign \{ type ipv4_addr \; \}
> Error: Could not process rule: No such file or directory
> add set filter foreign { type ipv4_addr ; }
>         ^^^^^^
This is short for 'nft add set ip filter', but there is no
table named 'filter' in the ip family.

I would very much like to enforce family, but it might break
existing scripts.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20181116/125ad69e/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

--- Comment #2 from James Feeney <james at nurealm.net> ---
I'm confused.  "filter" is just an arbitrary, but commonly used,
table name,
where the nft man page shows, in effect:

add set <address_family> <arbitrary_table_name>
<arbitrary_set_name> { ... }

The table named "filter" can be seen to already exist from the result
of "nft
list tables".

So - I do not understand your statement that "there is no table named
'filter'
in the ip family."

Or, do you mean to suggest that the table names are not arbitrary, and that nft
has defined, but not documented, some limited set of allowed table names?

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20181116/d57494f8/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

--- Comment #3 from Florian Westphal <fw at strlen.de> ---
(In reply to James Feeney from comment #2)
> I'm confused.  "filter" is just an arbitrary, but commonly
used, table name,
> where the nft man page shows, in effect:
> 
> add set <address_family> <arbitrary_table_name>
<arbitrary_set_name> { ... }
> 
> The table named "filter" can be seen to already exist from the
result of
> "nft list tables".
No.
table inet filter  // inet family
table ip nat       // ip family
table ip private   // ip family

so, there is no table named "filter" in the ip family, only in the
inet family.

$ nft add set filter foreign \{ type ipv4_addr \; \}

... tells nft to add a new set named foreign to a table named "filter"
in the
ip family, but there is no such table.

'table ip filter' would make this work, albeit its probably not what was
intended. Using 'nft add set inet filter ..' tells nft do add a set to a
table
named "filter" in the "inet" family, which does exist.
> Or, do you mean to suggest that the table names are not arbitrary, and that
> nft has defined, but not documented, some limited set of allowed table
names?
No undocumented magic, they are just names/identifiers.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20181116/c552e702/attachment-0001.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

--- Comment #4 from James Feeney <james at nurealm.net> ---
Hmm - I suspect that you are presuming a distinction that is not specifically
expressed in the documentation, when you distinguish between "a name in the
ip
family" and "a name in the inet family".

You seem to suggest that any arbitrary table name exists within the distinct
context of an address family.  This would imply that I could create two
distinct tables having identical table names, but defined with respect to two
distinct address families.  For instance, there could exist both an "inet
filter" table and an "ip filter" table.

Now, if that understanding is correct, this point really really needs to be
expanded in the documentation, which seems to imply that individual table names
are distinct, regardless of the address family, both by showing
"family" as
optional, and where the man page reads "The IPv4/IPv6/Inet address families
handle IPv4, IPv6 or both types of packets."  Here, it seems to suggest
that
"Inet" is simply an equivalent to both IPv4 and IPv6.

This man page conspicuously does *not* say that "ip", "ip6",
and "inet" are
distinct and that "inet" is *not* equivalent to "ip" and
"ip6".

So please, if that is correct, add clarification to the man page and to the
wiki.

And, back to the problem at hand, should I then dispense with using address
family "inet", and only create distinct "ip" and
"ip6" tables for "set" rules?

What is the point of the "inet" address family, if it is not
equivalent to "ip"
and "ip6", with respect to table names?  The "inet" address
family seems to
confuse things.  Or, maybe its use is only "half way" supported?

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20181116/133a3778/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

--- Comment #5 from James Feeney <james at nurealm.net> ---
BTW, from the man page, "If an identifier is specified without an  address
family, the ip family is used by default."

It seems that I did not appreciate the importance of this, where, then, every
reference to "<address_family> <arbitrary_table_name>
<arbitrary_set_name>" is,
implicitly "ip <arbitrary_table_name>
<arbitrary_set_name>", when the address
family is not stated explicitly.

The implicit address family "ip" doesn't seem important without
first
understanding and appreciating that the table names are unique to each address
family.  Again, please please give additional emphasis to this point in the
documentation.
> I would very much like to enforce family, but it might break existing
scripts.
You understand that I *literally* did not understand what you were talking
about when you first said "This is short for 'nft add set ip
filter'", even
though the meaning is obvious in retrospect, yes?

I, then, want to encourage everyone to move to enforcing the use of address
family in the nft commands, and, yes, break those scripts.  Now is a good time
to fix nft scripts, while so many people are still using iptables.

Thanks for helping me to clarify these issues.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20181117/8aa7ca20/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

--- Comment #6 from James Feeney <james at nurealm.net>
---
>From the man page: "All nftables objects exist in address family
specific
namespaces, therefore all identifiers include an address family."

I'm thinking that a few obvious-in-retrospect examples of "address
family
specific namespace objects" would be very helpful, at this point in the man
page, to emphasize the significance of this point.

In particular, again in retrospect, the man page says quite plainly
"nftables
objects".  Now, apparently, "nftables objects" is referring to
"table", and
then, "identifiers" is referring to "table name".  But that
is not
"spelled-out" explicitly, and it should be made explicit.

"All nftables objects exist in address family specific namespaces,
therefore
all identifiers include an address family. If an identifier is specified
without an  address family, the ip family is used by default."

The two sentences in that paragraph are so abstract and succinct that they seem
almost "throwaway".  Instead, they are very very important to
understand and
appreciate.

Yeah, I'm being a bit lame here, simply "RTFM", but maybe I'm
not the only one.
 A bold subsection header would also be appropriate there, maybe "TABLES
AND
TABLE NAMES" or some such.

And then also, include the bit about that the "inet" address family is
*not*
simply "the sum" of the "ip" address family and the
"ip6" address family, which
is what seems to be otherwise implied by the descriptions, in context:

  ip     IPv4 address family.
  ip6    IPv6 address family.
  inet   Internet (IPv4/IPv6) address family.

In other words, the "inet" address family requires some different sort
of
definition/description.  "IPv4/IPv6 address family" is misleading.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20181117/d262bd50/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

--- Comment #7 from James Feeney <james at nurealm.net> ---
To be more clear, the misleading interpretation would be that creating a table
"inet filter" would then effectively create two tables, "ip
filter" and "ip6
filter", such that any "nftable object identifier",
"specified without an
address family", would, at least, always match the "implied"
table "ip filter",
in this case.  But, now it is apparent that that is not the way things work,
and that that would be a misinterpretation.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20181117/75c06dfe/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

--- Comment #8 from James Feeney <james at nurealm.net> ---
Reflecting on this further, I'd say that the concepts of "address
family
specific namespaces" and "table names" is the *very foundation*,
the "top
level", of the nftables User Interface.

So, the man page should not just "toss off" discussion of these two
foundational concepts with one or two sentences buried in the middle of the man
page with no header.

They really ought to be the first thing that the reader sees, after the syntax
formalities:

NAMESPACES AND TABLE NAMES
...
        ADDRESS FAMILIES
        ...
                IPV4/IPV6/INET ADDRESS FAMILIES
                ...
        ...

If I understand, the set of all "nftables objects" includes, in
addition to
"tables", also, basically, all of the "header" items in the
man page, "chains",
"rules", "sets", "maps", etc.  And each of these
exists within a distinct and
separate namespace.  So "address_family" is at the top of a hierarchy,
"table_name" is a second level object, "sets, chains, maps,
etc." are third
level, "types, hooks, flags, etc." are fourth level, and
"rules" are last.

It is a bit unfortunate that the configuration syntax is "TABLE
<address_family> <table_name>" instead of "FAMILY
<address_family>
<table_name>", since "address family" is at the *top* of
the "rule" hierarchy,
the most fundamental division.  

It's never too late to change that, or, at least, accept both key-words
equivalently.  That would smooth-out the nftables learning curve.  A mrore
consistent format would have "FAMILY <address_family> { TABLE
<table_name>
{...} TABLE <table_name> {...} }".

It would be helpful to include a full list of all "nftables objects"
in an
opening NAMESPACES section of the man page, categorized by their place in a
hierarchy, completely enumerating the set members, to orient the reader.

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20181117/66fae0d7/attachment.html>

https://bugzilla.netfilter.org/show_bug.cgi?id=1299

kfm at plushkava.net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kfm at plushkava.net

-- 
You are receiving this mail because:
You are watching all bug changes.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.netfilter.org/pipermail/netfilter-buglog/attachments/20200128/1090b9e9/attachment.html>