netfilter buglog - Jun 2011 - [Bug 719] New: ipset restore fails randomly

http://bugzilla.netfilter.org/show_bug.cgi?id=719

           Summary: ipset restore fails randomly
           Product: ipset
           Version: unspecified
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: critical
          Priority: P3
         Component: default
        AssignedTo: netfilter-buglog at lists.netfilter.org
        ReportedBy: martinbarrowcliff at gmail.com
   Estimated Hours: 0.0


For some time (1 year) I have observed erratic behavior for ipset restore.

I have 20 sets, of all types. None are heavily populated today.
I am saving all sets to a single file, aka ipset.save.

On reboot my sys5 init script does:
ipset -R < /etc/sysconfig/ipset.save.

Sometimes it loads perfectly, and sometimes it hangs my system.
I get no error codes back from ipset in the latter case so my
attempts to script a retry are of no avail.
I inspected the saved files closely and the saved file is clean.
I load all the kern modules before I do a restore.
I added delays between each sys5 operation.
However, when the problem occurs, only the first 2 sets are loaded.
This happens more often on a reboot, than a firewall reload,
but it does happen on both. 

I am using a very stable homespun server (not a dist) with grsec patched kernel
at 2.6.36, however; I have seen this issue for my last 5 kernels and several
versions of ipset. I have NEVER seen anything in my logs to explain this issue. 

I have resorted to loading each ipset individually in an attempt to 
isolate the problem, and it seems to work fine as of now; so I believe this 
may narrow down the problem. 
I really love my ipsets. Can I get some feedback on this please?

Marty B.
martinbarrowcliff at gmail.com


-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
You are watching all bug changes.

http://bugzilla.netfilter.org/show_bug.cgi?id=719


Jozsef Kadlecsik <kadlec at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |kadlec at netfilter.org
             Status|NEW                         |ASSIGNED




--- Comment #1 from Jozsef Kadlecsik <kadlec at netfilter.org>  2011-06-02
13:30:41 ---
Could you provide the ipset version number? At the minimum that's required
to
locate the problem.

Best regards,
Jozsef


-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
You are watching all bug changes.

http://bugzilla.netfilter.org/show_bug.cgi?id=719





--- Comment #2 from martin barrowcliff <martinbarrowcliff at gmail.com> 
2011-06-02 15:10:58 ---
The version of ipset I am using is:

ipset v4.5, protocol version 4.
Kernel module protocol version 4.


-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
You are watching all bug changes.

http://bugzilla.netfilter.org/show_bug.cgi?id=719


Jan Engelhardt <jengelh at medozas.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |jengelh at medozas.de
         AssignedTo|netfilter-                  |kadlec at netfilter.org
                   |buglog at lists.netfilter.org  |
             Status|ASSIGNED                    |NEW




--- Comment #3 from Jan Engelhardt <jengelh at medozas.de>  2011-06-24
20:05:10 ---
This is probably fixed in ipset 6 one way or another :)


-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
You are watching all bug changes.

http://bugzilla.netfilter.org/show_bug.cgi?id=719





--- Comment #4 from Jozsef Kadlecsik <kadlec at netfilter.org>  2011-08-10
14:35:43 ---
What kind of architecture do you run? Maybe it's an endianness problem.

But the quickest path would be to upgrade to the newest ipset version. With
2.6.36 an absolutely patch is required only and the kernel part of ipset can be
compiled as an external module. Please have a look at the README file in
the newest ipset package.


-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

http://bugzilla.netfilter.org/show_bug.cgi?id=719


Jozsef Kadlecsik <kadlec at netfilter.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED




-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

http://bugzilla.netfilter.org/show_bug.cgi?id=719





--- Comment #5 from martin barrowcliff <martinbarrowcliff at gmail.com> 
2011-08-10 19:34:11 ---
I have looked at version 6 however I have not yet attemped to convert all my
sets
to the required, more technical v6 format. 
It took me a long time to become memory familiar and proficient with v4 and now 
you suggest I learn a new format, (and forget the old) just because there is
some 
kind of a bug in the v4 code? 
Maybe that's the only option, but after a couple years with ipset I am
expected to be happy with those changes?
As far as I can see they were not even necessary; just set name-type changes.

As I have previously stated, v4 has issues that I was unable to resolve. see
719.
If I must move to the new code I can do that, but I am not very convinced v6 is
actually a marked improvement. By the time I learn to use it, you may change
the format again...

Great module but becomming less user friendly at every change. 
So if v4 is history, please let us know we 'must' upgrade for
reliability.


-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

http://bugzilla.netfilter.org/show_bug.cgi?id=719





--- Comment #6 from martin barrowcliff <martinbarrowcliff at gmail.com> 
2011-08-10 19:46:42 ---
No endian issues. The system in question is a Intel Atom-330.
I could forward logs and configs if requested, but they show no 
problems until iptables starts. Then I get hung...


-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.

http://bugzilla.netfilter.org/show_bug.cgi?id=719


martin barrowcliff <martinbarrowcliff at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|                            |FIXED




-- 
Configure bugmail: http://bugzilla.netfilter.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching all bug changes.