With the impending release of Shorewall 3.4, I have begun thinking about where we go from here. The following are some thoughts about the current state of Shorewall and possibilities for the future. Activity on the mailing lists and IRC channel has been steadily declining for the last couple of years. This signals to me that the rate at which people are adopting Shorewall is waning (I grant that the documentation has gotten better over the years which helps lower the noise level somewhat). While I''ve never had any ambitions toward dominating the OSS firewall market, Shorewall takes a lot of work so I would prefer to spend my effort on something that people want to use. Maybe it is still Shorewall -- maybe it is something else. My list of the major Shorewall shortcomings: 1) No GUI (other than Webmin). That is something that I personally am unlikely to change because I have no interest in spending my time developing one. Besides, first attempts at GUI creation tend to have disappointing results at best. 2) Performance and scaling issues. Shorewall is being used in larger and more complex configurations and the time required to run the generated script increases non-linearly with configuration complexity. This has been a common theme on the mailing list recently. Performance of the shell-based compiler is also awful when dealing with large complex configurations. 3) No IPv6 Support. We''ve talked about this before. 4) Lack of an ability to make quick "on the fly" configuration changes (this is closely related to the performance problem -- if "shorewall restart" always took less that a second, I don''t believe that this would be an issue). Resolving these issues will require considerable change to the current shell-based product and I suspect that just adding IPv6 support would make the performance problem even worse. My feeling is that the current shell-based implementation has gone about as far as it can go and that adding additional major function to it while trying to maintain even the current mediocre level of performance will be difficult. I have begun some experimentation with rewriting the compiler in Perl and that is looking promising. Converting to Perl will unfortunately present migration/compatibility issues with compile-time extension scripts although I''ve been able to retain shorewall.conf and /etc/shorewall/params functionality for the most part. A Perl-based compiler would of course mean the end of the road for Shorewall Embedded System support (at least for the full Shorewall product) since Perl is typically not available on those systems. Embedded Systems could still support Shorewall Lite. Here are a couple of possible approaches to the run-time performance problem. a) Keep the current zone-policy-rule paradigm but have the compiler generate ''iptables-restore'' input. This is clearly preferable to the current approach of running iptables repeatedly in the compiled script. We would still be stuck with the order (n**2) scaling issues inherent in that paradigm (see http://www.shorewall.net/ScalabilityAndPerformance.html). b) Consider other paradigms which do not suffer from the scaling issues implicit in zone-policy-rule. For example, I''m familiar with one organization that has created a firewall configuration tool based on a service-policy-rule paradigm which scales much better. Both approaches present migration/compatibility issues. Approach a) maintains compatibility with the tabular config files from earlier Shorewall releases but would likely invalidate most runtime extension scripts. We could probably continue to call the product "Shorewall" though as the changes would be more evolutionary than revolutionary. Approach b) is clearly a new product -- "Shorewall-ng" or something. Approach a) is not as straight-forward as it might first appear. To accommodate Shorewall Lite, the compiler cannot simply create a single monolithic iptables-restore file because part of the contents of that file cannot be determined until run-time (all uses of ''detect'' for example). So at run-time, the script still has to perform some configuration analysis then glue together an iptables-restore file that is appropriate for the current configuration. So long as we are discussing what to add to Shorewall, it is also worth asking what the next version of the product can leave behind. Given that BRIDGING=Yes won''t work with kernel version 2.6.20 and later, that feature is a rather obvious choice for elimination. ECN is now widely deployed on the Internet so /etc/shorewall/ecn might also be a candidate for extinction. Are there others? I welcome your input and look forward to further discussion. Thanks, -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> 2) Performance and scaling issues. Shorewall is being used in larger and > more complex configurations and the time required to run the generated > script increases non-linearly with configuration complexity. This has > been a common theme on the mailing list recently. Performance of the > shell-based compiler is also awful when dealing with large complex > configurations.Hi Tom and list, is performance really such a PITA? I have maybe 50 shorewall installations in the wild. From simple setups up to quite complex ones. Using almost every feature and (im)possibilities. But restart performance was never an issue, neither for me nor for my customers. Let it compile 30 or even more seconds, the activating is still fast enough and traffic is not affected during compile time. I know some commercial products that perform worse... It works for me as it is. And this very well. Thanks Tom for this nice project. Alex ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Hi Alex, Alexander Wilms wrote:> But restart performance was never an issue, neither for me nor for my > customers. > Let it compile 30 or even more seconds, the activating is still fast enough > and traffic is not affected during compile time. I know some commercial > products that perform worse...It is not a problem for me either but on IRC recently, Tuomo Soini was telling me that he just upgraded his largest firewall to a Dual-core Xeon from a P4 2Ghz and start dropped from two minutes to 45 seconds (that''s just the script time). And other people complain about 4-5 minute startup times on 2Ghz CPUs.> It works for me as it is. And this very well. Thanks Tom for this nice > project.You''re welcome Alex. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
On Fri, 23 Feb 2007 16:02:06 -0800 Tom Eastep <teastep@shorewall.net> wrote:> I have begun some experimentation with rewriting the compiler in Perl and > that is looking promising. Converting to Perl will unfortunately present > migration/compatibility issues with compile-time extension scripts although > I''ve been able to retain shorewall.conf and /etc/shorewall/params > functionality for the most part. A Perl-based compiler would of course mean > the end of the road for Shorewall Embedded System support (at least for the > full Shorewall product) since Perl is typically not available on those > systems. Embedded Systems could still support Shorewall Lite.Have you thought of lua? should give you better performance than perl and would still be small enough for embedded. I can''t say I have been looking at the shorewall code, but lua is very table oriented, which might be good for your table based config files. btw. The LEAF webconf guy, Nathan, is working on a lua based webconf framework. Natanael Copa ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> With the impending release of Shorewall 3.4, I have begun thinking about > where we go from here. The following are some thoughts about the current > state of Shorewall and possibilities for the future. > > Activity on the mailing lists and IRC channel has been steadily declining > for the last couple of years.Hi Tom, I''m not sure Shorewall usage has declined over the years. That may be true for those trying to protect their personal systems, maybe they are now simply using their distributions "firewall" functionality. For those building real firewalls, I don''t know which tool could be used instead of Shorewall.> 1) No GUI (other than Webmin). That is something that I personally amNot having a GUI is perfectly fine for me.> 2) Performance and scaling issues. Shorewall is being used in larger and > more complex configurations and the time required to run the generated > script increases non-linearly with configuration complexity. This has > been a common theme on the mailing list recently. Performance of the > shell-based compiler is also awful when dealing with large complex > configurations.I''m in the good situation that performance has never been a problem for me. Maybe that''s because I try to keep too much complexity away from my firewalls and because I have some nice hardware for the larger setups. My question is whether it''s possible to use perl for some kind of Shorewall-accelerator. I mean if it would be possible to create a simple shell to perl converter which then runs perl instead of the shell and does exactly the same, then it could be used whenever perl is available, otherwise a shell would be used. As always, I want to thank Tom for a very nice and well maintained product. Simon ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
From: "Simon Matter" <simon.matter@invoca.ch>> I''m not sure Shorewall usage has declined over the years. That may be > true for those trying to protect their personal systems, maybe they are > now simply using their distributions "firewall" functionality. For those > building real firewalls, I don''t know which tool could be used instead of > Shorewall.I agree.> Not having a GUI is perfectly fine for me.The same for me. But, if we want to evaluate possible solutions, I am wondering if Firewall Builder: http://www.fwbuilder.org/archives/cat_about.html could be adapted to generate Shorewall configuration files. Yes, I do know that it can direclty generate iptables commands, but I think that it could be useful for novice users to have the capability to create a configuration with a GUI, and possibly fine tune it later with the config files.> I''m in the good situation that performance has never been a problem for > me. Maybe that''s because I try to keep too much complexity away from > my firewallsThe same for me.> As always, I want to thank Tom for a very nice and well maintained product.Thank you, Tom, also from another very satisfied Shorewall user. Elio ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
On Fri, 2007-02-23 at 16:02, Tom Eastep wrote: <snip>> Activity on the mailing lists and IRC channel has been steadily declining > for the last couple of years. This signals to me that the rate at which > people are adopting Shorewall is waning (I grant that the documentation has > gotten better over the years which helps lower the noise level somewhat). > While I''ve never had any ambitions toward dominating the OSS firewall > market, Shorewall takes a lot of work so I would prefer to spend my effort > on something that people want to use. Maybe it is still Shorewall -- maybe > it is something else.Tom, I just did a few Google searches, and I now think it''s something else. Shorewall is included in most distributions (Debian, Ubuntu, OpenSUSE, Gentoo, etc.), and the website is mirrored in multiple locations. It looks like people are asking for assistance in a variety of places. Shorewall support looks distributed and successful. Example: http://gentoo-wiki.com/HOWTO_Shorewall_Firewall_IPsec_VPN_and_2.6_kernel http://gentoo-wiki.com/SECURITY_Howto_setup_a_firewall_with_shorewall I''d worry when distributions start dropping Shorewall. That''s an indication of decline. -- Mike Noyes <mhnoyes at users.sourceforge.net> http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, sitedocs ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Natanael Copa wrote:> > Have you thought of lua? should give you better performance than perl > and would still be small enough for embedded. I can''t say I have been > looking at the shorewall code, but lua is very table oriented, which > might be good for your table based config files.I suspect that the parser part of Shorewall might be difficult to write entirely in LUA (although I''ve only browsed the reference manual). Also, LUA appears to be a language that may only be embedded in C -- it does not stand on its own. So what you end up with is a C program that uses a "smart library". The change in Shorewall 3.0 which produced the compiler and Shorewall Lite was intended to pave the way toward an environment where the heavy lifting could be done on a central system and the generated firewall script exported to smaller less-powerful devices. In that model, the language in which the compiler is written is not so important for portability. The current compiler can even run under Cygwin on Windows systems (or it could the last time I tried it). I (or someone else -- any volunteers?) will probably continue to support Shorewall 3.4 for the forseeable future (although enhancements will be minimal). So those who are happy with the current state of the package can continue to use it. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Mike Noyes wrote:> > > I''d worry when distributions start dropping Shorewall. That''s an > indication of decline. >Good point. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Elio Tondo wrote:> >> Not having a GUI is perfectly fine for me. > > The same for me. But, if we want to evaluate possible solutions, I am > wondering if Firewall Builder: > > http://www.fwbuilder.org/archives/cat_about.html > > could be adapted to generate Shorewall configuration files. Yes, I do know > that it can direclty generate iptables commands, but I think that it could be > useful for novice users to have the capability to create a configuration with > a GUI, and possibly fine tune it later with the config files. >I have lunch with Steve Herber[1] periodically and he often makes this same suggestion. I''ll take a serious look at this possibility. Thanks, -Tom [1] Steve also subscribes to this list -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Simon Matter wrote:> My question is whether it''s possible to use perl for some kind of > Shorewall-accelerator. I mean if it would be possible to create a simple > shell to perl converter which then runs perl instead of the shell and does > exactly the same, then it could be used whenever perl is available, > otherwise a shell would be used.If we look at overall performance, the compiler and the generated script need to be considered separately. I don''t believe that the fact that the generated script is interpreted using a Bourne shell is a significant contributing factor to its performance. The bulk of the time is spent in fork() and exec() while launching iptables 1000s of times. So I think that we''re safe in leaving that part alone. The compiler is a different issue. One problem is that some of the things needed to make Shorewall generate iptables-restore input would be very slow if implemented in the shell. Even though Perl is a somewhat quirky language, it works really well for writing the compiler part of Shorewall. But the Perl version of the compiler employs quite different algorithms from those used in the shell. My belief is that a shell->Perl translator capable of translating the current Shorewall shell code would comprise a much larger project then Shorewall itself. One example of what it would need to do: Because the shell provides exactly one hash table (it''s symbol table), the current Shorewall code manufactures symbols on the fly (using ''eval'') in an attempt to provide some of the same function that Perl hashes do. It would take a very smart program to see through what the current code is doing and create Perl hashes instead. Another factor concerning the current code is that it is beginning to get somewhat fragile. Today''s Shorewall is doing many things that were not envisioned in the original design. The shell also tends to promote some bad programming practices, owing to the fact that all symbols are global (and I didn''t start from the beginning setting the -u option). As a consequence, it is getting more and more difficult to extend the code to do new things without breaking old things. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Tom Eastep wrote:> Natanael Copa wrote: > >> Have you thought of lua? should give you better performance than perl >> and would still be small enough for embedded. I can''t say I have been >> looking at the shorewall code, but lua is very table oriented, which >> might be good for your table based config files. > > I suspect that the parser part of Shorewall might be difficult to write > entirely in LUA (although I''ve only browsed the reference manual). Also, LUA > appears to be a language that may only be embedded in C -- it does not stand > on its own. So what you end up with is a C program that uses a "smart library". >Tom, Since I was mentioned in this thread already, I''ll give my $0.02. Feel free to send to /dev/null A) The Lua manual is really bad. LuaForge is a mess. I really needed Programming in Lua to make sense of the language (Ver 1 is online http://www.lua.org/pil/ Version 2 is only available in dead-tree format and was worth the cost. B) The Lua parser is surprisingly good for regex type stuff. Its just /different/ from the other "standard" regexes. C) You are right about Lua wanting to be embedded in C (or some other language) - imagine however, a tiny "C" wrapper that forked iptables-restore[1] once and then used a lua parser to send all the data to that one process instead of forking iptables 1000''s of times. The C code would be minimal, just enough to instantiate a Lua VM. D) Shorewall is YOUR fun project... so none of the above need apply. ;-) [1] - Believe it or not, http://www.netfilter.org/documentation/FAQ/netfilter-faq-4.html#ss4.5 says this is the "official" way to program iptables without using system() calls. ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Tom Eastep wrote:> ... > Activity on the mailing lists and IRC channel has been steadily declining > for the last couple of years. This signals to me that the rate at which > people are adopting Shorewall is waning (I grant that the documentation has > gotten better over the years which helps lower the noise level somewhat). > While I''ve never had any ambitions toward dominating the OSS firewall > market, Shorewall takes a lot of work so I would prefer to spend my effort > on something that people want to use. Maybe it is still Shorewall -- maybe > it is something else.Tom, sorry i haven''t had a chance to comment on this yet. I think it''s important that we target Shorewall at the right features for our audience. (Perhaps we''re due for another survey? :-)> ... > My list of the major Shorewall shortcomings: > > 1) No GUI (other than Webmin). That is something that I personally am > unlikely to change because I have no interest in spending my time > developing one. Besides, first attempts at GUI creation tend to have > disappointing results at best.I really don''t get this. I''ve never understood why people think it''s an issue. Shorewall''s configuration files don''t lend themselves to GUIs. I mean, presenting things in a spreadsheet-like table isn''t *that* critical, is it? Any focus on GUIs should be directed towards Webmin, IMO.> ... > 2) Performance and scaling issues. Shorewall is being used in larger and > more complex configurations and the time required to run the generated > script increases non-linearly with configuration complexity. This has > been a common theme on the mailing list recently. Performance of the > shell-based compiler is also awful when dealing with large complex > configurations.I think this is the biggie. It''s one of the most common complaints (along with 4 below), and experienced IT professionals are our biggest user base.> 3) No IPv6 Support. We''ve talked about this before.I would rate this as important, but not as high a priority as 2 & 4.> 4) Lack of an ability to make quick "on the fly" configuration changes (this > is closely related to the performance problem -- if "shorewall restart" > always took less that a second, I don''t believe that this would be an > issue).I agree.> Resolving these issues will require considerable change to the current > shell-based product and I suspect that just adding IPv6 support would make > the performance problem even worse. My feeling is that the current > shell-based implementation has gone about as far as it can go and that > adding additional major function to it while trying to maintain even the > current mediocre level of performance will be difficult.My gut feel is to leave off IPv6 until after the rewrite.> I have begun some experimentation with rewriting the compiler in Perl and > that is looking promising. Converting to Perl will unfortunately present > migration/compatibility issues with compile-time extension scripts although > I''ve been able to retain shorewall.conf and /etc/shorewall/params > functionality for the most part.You know my feelings about perl - it is so much better a language for string processing than shell. I think it would cut your code size substantially, and thus result in much more maintainable and bug-free code, as long as you don''t write it like a typical perl programmer (which i already know you aren''t ;-).> ... > Here are a couple of possible approaches to the run-time performance problem. > > a) Keep the current zone-policy-rule paradigm but have the > compiler generate ''iptables-restore'' input. > > This is clearly preferable to the current approach of running > iptables repeatedly in the compiled script. We would still be stuck > with the order (n**2) scaling issues inherent in that paradigm > (see http://www.shorewall.net/ScalabilityAndPerformance.html).I suspect iptables-restore would give us enough of a performance boost that the O(n**2) issues could be ignored for a bit longer. :-)> b) Consider other paradigms which do not suffer from the scaling issues > implicit in zone-policy-rule. > > For example, I''m familiar with one organization that has created a > firewall configuration tool based on a service-policy-rule > paradigm which scales much better.Can you explain the service-policy-rule paradigm a little more (pointing us to some documentation would be fine)? The zone-policy-rule paradigm is the main reason i was first attracted to Shorewall, and i wouldn''t depart from it without being thoroughly convinced of the merits of a different approach.> Both approaches present migration/compatibility issues. Approach a) > maintains compatibility with the tabular config files from earlier Shorewall > releases but would likely invalidate most runtime extension scripts. We > could probably continue to call the product "Shorewall" though as the > changes would be more evolutionary than revolutionary. Approach b) is > clearly a new product -- "Shorewall-ng" or something.It may eventually come to that, but i''m not sure it''s time for a new Shorewall product just yet. You''ve managed mostly to maintain backwards compatibility for much longer than most Free Software projects exist. That is a good thing, and has resulted in a lot of people saving a lot of time.> Approach a) is not as straight-forward as it might first appear. To > accommodate Shorewall Lite, the compiler cannot simply create a single > monolithic iptables-restore file because part of the contents of that file > cannot be determined until run-time (all uses of ''detect'' for example). So > at run-time, the script still has to perform some configuration analysis > then glue together an iptables-restore file that is appropriate for the > current configuration.Obviously, if the complexity of this approach makes it infeasible, another approach may be needed, but it seems like the next logical step.> So long as we are discussing what to add to Shorewall, it is also worth > asking what the next version of the product can leave behind. Given that > BRIDGING=Yes won''t work with kernel version 2.6.20 and later, that feature > is a rather obvious choice for elimination. ECN is now widely deployed on > the Internet so /etc/shorewall/ecn might also be a candidate for extinction. > Are there others?I''ve never used either of them, so i don''t mind. One thing that i think would be good to add, however, would be some more idiot proofing of the multi-ISP support (initially, an option to set up all of the masqing automatically - there may be other improvements that could be made as well). This isn''t a high priority, but i think it would reduce the support load. -- Paul <http://paulgear.webhop.net> -- A: Because it breaks the logical sequence of discussion. Q: Why shouldn''t i write my replies at the top of emails? ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Hi, I was wondering if someone else thought about going deeper into the dynamic approach. For example, it whould be nice to have some statement/configuration control embebbed on config files: Today: #----------------------------------- #Rules to my not-always-up device: INCLUDE rules-eth6.inc #----------------------------------- Tomorrow :) ----------------------------------- #Rules to my not-always-up device: INCLUDE <maybe-a-shell-based-test> rules-eth6.inc ----------------------------------- I know this kind of thing would make shorewall much more complex but I think It whould bring some nice new possibilities. I think ''doing things on-the-fly'' helps a lot on other problems like ''improving shorewall suppport for automated (non-human) dynamic reconfiguration'', and suddenly, I´m not so sure it´s only about speed. Tanks, Longarai Thursday, March 1, 2007, 7:46:55 PM, you wrote:> Tom Eastep wrote: >> ... >> Activity on the mailing lists and IRC channel has been steadily declining >> for the last couple of years. This signals to me that the rate at which >> people are adopting Shorewall is waning (I grant that the documentation has >> gotten better over the years which helps lower the noise level somewhat). >> While I''ve never had any ambitions toward dominating the OSS firewall >> market, Shorewall takes a lot of work so I would prefer to spend my effort >> on something that people want to use. Maybe it is still Shorewall -- maybe >> it is something else.> Tom, sorry i haven''t had a chance to comment on this yet. I think it''s > important that we target Shorewall at the right features for our > audience. (Perhaps we''re due for another survey? :-)>> ... >> My list of the major Shorewall shortcomings: >> >> 1) No GUI (other than Webmin). That is something that I personally am >> unlikely to change because I have no interest in spending my time >> developing one. Besides, first attempts at GUI creation tend to have >> disappointing results at best.> I really don''t get this. I''ve never understood why people think it''s an > issue. Shorewall''s configuration files don''t lend themselves to GUIs. > I mean, presenting things in a spreadsheet-like table isn''t *that* > critical, is it? Any focus on GUIs should be directed towards Webmin, IMO.>> ... >> 2) Performance and scaling issues. Shorewall is being used in larger and >> more complex configurations and the time required to run the generated >> script increases non-linearly with configuration complexity. This has >> been a common theme on the mailing list recently. Performance of the >> shell-based compiler is also awful when dealing with large complex >> configurations.> I think this is the biggie. It''s one of the most common complaints > (along with 4 below), and experienced IT professionals are our biggest > user base.>> 3) No IPv6 Support. We''ve talked about this before.> I would rate this as important, but not as high a priority as 2 & 4.>> 4) Lack of an ability to make quick "on the fly" configuration changes (this >> is closely related to the performance problem -- if "shorewall restart" >> always took less that a second, I don''t believe that this would be an >> issue).> I agree.>> Resolving these issues will require considerable change to the current >> shell-based product and I suspect that just adding IPv6 support would make >> the performance problem even worse. My feeling is that the current >> shell-based implementation has gone about as far as it can go and that >> adding additional major function to it while trying to maintain even the >> current mediocre level of performance will be difficult.> My gut feel is to leave off IPv6 until after the rewrite.>> I have begun some experimentation with rewriting the compiler in Perl and >> that is looking promising. Converting to Perl will unfortunately present >> migration/compatibility issues with compile-time extension scripts although >> I''ve been able to retain shorewall.conf and /etc/shorewall/params >> functionality for the most part.> You know my feelings about perl - it is so much better a language for > string processing than shell. I think it would cut your code size > substantially, and thus result in much more maintainable and bug-free > code, as long as you don''t write it like a typical perl programmer > (which i already know you aren''t ;-).>> ... >> Here are a couple of possible approaches to the run-time performance problem. >> >> a) Keep the current zone-policy-rule paradigm but have the >> compiler generate ''iptables-restore'' input. >> >> This is clearly preferable to the current approach of running >> iptables repeatedly in the compiled script. We would still be stuck >> with the order (n**2) scaling issues inherent in that paradigm >> (see >> http://www.shorewall.net/ScalabilityAndPerformance.html).> I suspect iptables-restore would give us enough of a performance boost > that the O(n**2) issues could be ignored for a bit longer. :-)>> b) Consider other paradigms which do not suffer from the scaling issues >> implicit in zone-policy-rule. >> >> For example, I''m familiar with one organization that has created a >> firewall configuration tool based on a service-policy-rule >> paradigm which scales much better.> Can you explain the service-policy-rule paradigm a little more (pointing > us to some documentation would be fine)? The zone-policy-rule paradigm > is the main reason i was first attracted to Shorewall, and i wouldn''t > depart from it without being thoroughly convinced of the merits of a > different approach.>> Both approaches present migration/compatibility issues. Approach a) >> maintains compatibility with the tabular config files from earlier Shorewall >> releases but would likely invalidate most runtime extension scripts. We >> could probably continue to call the product "Shorewall" though as the >> changes would be more evolutionary than revolutionary. Approach b) is >> clearly a new product -- "Shorewall-ng" or something.> It may eventually come to that, but i''m not sure it''s time for a new > Shorewall product just yet. You''ve managed mostly to maintain backwards > compatibility for much longer than most Free Software projects exist. > That is a good thing, and has resulted in a lot of people saving a lot > of time.>> Approach a) is not as straight-forward as it might first appear. To >> accommodate Shorewall Lite, the compiler cannot simply create a single >> monolithic iptables-restore file because part of the contents of that file >> cannot be determined until run-time (all uses of ''detect'' for example). So >> at run-time, the script still has to perform some configuration analysis >> then glue together an iptables-restore file that is appropriate for the >> current configuration.> Obviously, if the complexity of this approach makes it infeasible, > another approach may be needed, but it seems like the next logical step.>> So long as we are discussing what to add to Shorewall, it is also worth >> asking what the next version of the product can leave behind. Given that >> BRIDGING=Yes won''t work with kernel version 2.6.20 and later, that feature >> is a rather obvious choice for elimination. ECN is now widely deployed on >> the Internet so /etc/shorewall/ecn might also be a candidate for extinction. >> Are there others?> I''ve never used either of them, so i don''t mind. One thing that i think > would be good to add, however, would be some more idiot proofing of the > multi-ISP support (initially, an option to set up all of the masqing > automatically - there may be other improvements that could be made as > well). This isn''t a high priority, but i think it would reduce the > support load.------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
On Thu, 2007-03-01 at 14:46, Paul Gear wrote:> Tom Eastep wrote:<snip>> > I have begun some experimentation with rewriting the compiler in Perl and > > that is looking promising. Converting to Perl will unfortunately present > > migration/compatibility issues with compile-time extension scripts although > > I''ve been able to retain shorewall.conf and /etc/shorewall/params > > functionality for the most part. > > You know my feelings about perl - it is so much better a language for > string processing than shell. I think it would cut your code size > substantially, and thus result in much more maintainable and bug-free > code, as long as you don''t write it like a typical perl programmer > (which i already know you aren''t ;-).Paul, What is your opinion of microperl? Would in suffice for the proposed rewrite? -- Mike Noyes <mhnoyes at users.sourceforge.net> http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, sitedocs ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
On Fri, 2007-03-02 at 07:07 -0800, Mike Noyes wrote:> On Thu, 2007-03-01 at 14:46, Paul Gear wrote: > > Tom Eastep wrote: > <snip> > > > I have begun some experimentation with rewriting the compiler in Perl and > > > that is looking promising. Converting to Perl will unfortunately present > > > migration/compatibility issues with compile-time extension scripts although > > > I''ve been able to retain shorewall.conf and /etc/shorewall/params > > > functionality for the most part. > > > > You know my feelings about perl - it is so much better a language for > > string processing than shell. I think it would cut your code size > > substantially, and thus result in much more maintainable and bug-free > > code, as long as you don''t write it like a typical perl programmer > > (which i already know you aren''t ;-). > > Paul, > What is your opinion of microperl? Would in suffice for the proposed > rewrite?My vote probably dont count since i havent contributed with any code, anyway... I just compiled microperl on my gento/hardened/uclibc chroot. It compiled on first try, but the file was 1155980 (1254803 before strip). I have also the full perl but its huge so its never used. If you intend to continue be The firewall for embedded I''d suggest avoiding perl/python/ruby/php. We had a long debate some time ago on alpinelinux list what to do with webconf since we wanted to replace shell with some kind of high level language. We ended with lua, which definitively gives the most for the bucks. As a comparation to microperl, lua is 160kb. http://shootout.alioth.debian.org/debian/benchmark.php?test=all&lang=lua&lang2=perl my $0.02 Natanael Copa ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
On Fri, 2007-03-02 at 09:08, Natanael Copa wrote:> On Fri, 2007-03-02 at 07:07 -0800, Mike Noyes wrote:<snip>> > What is your opinion of microperl? Would in suffice for the proposed > > rewrite? > > My vote probably dont count since i havent contributed with any code, > anyway... > > I just compiled microperl on my gento/hardened/uclibc chroot. It > compiled on first try, but the file was 1155980 (1254803 before strip). > > I have also the full perl but its huge so its never used. If you intend > to continue be The firewall for embedded I'd suggest avoiding > perl/python/ruby/php. > > We had a long debate some time ago on alpinelinux list what to do with > webconf since we wanted to replace shell with some kind of high level > language. We ended with lua, which definitively gives the most for the > bucks. > > As a comparation to microperl, lua is 160kb. > http://shootout.alioth.debian.org/debian/benchmark.php?test=all〈=lua&lang2=perlNatanael, Lua looks like a definite win in the embedded space. However, shorewall has a wider user base than that. Ubiquitous linux language support is only available for C, C++, Perl, and Python (ref. LSB 3.2). http://www.linux-foundation.org/en/LSB_Roadmap http://www.linux-foundation.org/en/LsbPerl In addition, it looks like quite a few embedded distributions are making microperl packages available. -- Mike Noyes <mhnoyes at users.sourceforge.net> http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, sitedocs ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ Shorewall-devel mailing list Shorewall-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-devel
Natanael Copa wrote:> On Fri, 2007-03-02 at 07:07 -0800, Mike Noyes wrote:>> Paul, >> What is your opinion of microperl? Would in suffice for the proposed >> rewrite? > > My vote probably dont count since i havent contributed with any code, > anyway... > > I just compiled microperl on my gento/hardened/uclibc chroot. It > compiled on first try, but the file was 1155980 (1254803 before strip). > > I have also the full perl but its huge so its never used. If you intend > to continue be The firewall for embedded I''d suggest avoiding > perl/python/ruby/php. > > We had a long debate some time ago on alpinelinux list what to do with > webconf since we wanted to replace shell with some kind of high level > language. We ended with lua, which definitively gives the most for the > bucks. > > As a comparation to microperl, lua is 160kb. > http://shootout.alioth.debian.org/debian/benchmark.php?test=all&lang=lua&lang2=perlOne thing that we should keep in mind is that Shorewall has never been intended as ''The firewall for embedded''. The Leaf Bering team happened to adopt Shorewall as their firewall tool and I''ve tried to support their effort. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>The Leaf Bering team happened to adopt Shorewall as their firewall tool andI''ve tried to support their effort. Quite well you have. Thanks! Bob Coffman Shorewall on Leaf end user ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
Natanael Copa wrote:> > > My vote probably dont count since i havent contributed with any code, > anyway... > > I just compiled microperl on my gento/hardened/uclibc chroot. It > compiled on first try, but the file was 1155980 (1254803 before strip). > > I have also the full perl but its huge so its never used. If you intend > to continue be The firewall for embedded I''d suggest avoiding > perl/python/ruby/php. > > We had a long debate some time ago on alpinelinux list what to do with > webconf since we wanted to replace shell with some kind of high level > language. We ended with lua, which definitively gives the most for the > bucks.Do these embedded distros generally include some version of libstdc++ ? -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
On Sun, 2007-03-04 at 11:53, Natanael Copa wrote:> On Sat, 2007-03-03 at 21:01 -0800, Tom Eastep wrote: > > Natanael Copa wrote: > > > > > > I have also the full perl but its huge so its never used. If you intend > > > to continue be The firewall for embedded I''d suggest avoiding > > > perl/python/ruby/php. > ... > > > > Do these embedded distros generally include some version of libstdc++ ? > > Yes. uClibc++.Natanael, Do you have build sizes for libstdc++ available? I don''t see a leaf package for it available at this time. -- Mike Noyes <mhnoyes at users.sourceforge.net> http://sourceforge.net/users/mhnoyes/ SF.net Projects: leaf, sitedocs ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
On Sat, 2007-03-03 at 21:01 -0800, Tom Eastep wrote:> Natanael Copa wrote: > > > > I have also the full perl but its huge so its never used. If you intend > > to continue be The firewall for embedded I''d suggest avoiding > > perl/python/ruby/php....> > Do these embedded distros generally include some version of libstdc++ ?Yes. uClibc++. Natanael ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
On Sun, 2007-03-04 at 11:04 -0800, Mike Noyes wrote:> On Sun, 2007-03-04 at 11:53, Natanael Copa wrote: > > On Sat, 2007-03-03 at 21:01 -0800, Tom Eastep wrote: > > > Natanael Copa wrote: > > > > > > > > I have also the full perl but its huge so its never used. If you intend > > > > to continue be The firewall for embedded I''d suggest avoiding > > > > perl/python/ruby/php. > > ... > > > > > > Do these embedded distros generally include some version of libstdc++ ? > > > > Yes. uClibc++. > > Natanael, > Do you have build sizes for libstdc++ available? I don''t see a leaf > package for it available at this time.I replied to the leaf-devel list before I saw your posting here. In alpinelinux the runtime package (tar.gz) is 64kb. Disk usage uncompressed is around 200kb. This is with -fstack-protector turned on. It would become smaller for uclibc-bering. Natanael Copa ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net''s Techsay panel and you''ll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV