LARTC - Feb 2004 - ESFQ Modification

Hi!

Some time ago I faced a problem in limiting traffic on host with
multiple uplinks. Since all the stuff worked nice seemed that there
will be no problems. But then I realized that P2P users are smart
enough to bypass limits as sfq doesn''t give fair sharing in this case
(thousands of connections from one user versus few from the other).
I tried IMQ but it''s instability in my configuration was painfull.
So I made something like that:

1. i use IPMARK patch for the iptables to mark all the connections
   in P2P related class depending on source IP (i use SNAT),
2. modified ESFQ to create hash depending on FWMARK instead of src ip
3. and it worked. So I have uplink policy based on source ip in
   snat-ed environment without using IMQ.

I''m looking for the opinions, cause I may be wrong in this.
Patch for the files below, cause it''s short


diff -urN ./orig/pkt_sched.h ./patched/pkt_sched.h
--- ./orig/pkt_sched.h  2004-02-26 09:27:54.000000000 +0100
+++ ./patched/pkt_sched.h       2004-01-07 21:23:58.000000000 +0100
@@ -162,6 +162,7 @@
        TCA_SFQ_HASH_CLASSIC,
        TCA_SFQ_HASH_DST,
        TCA_SFQ_HASH_SRC,
+       TCA_SFQ_HASH_FWMARK,
 };
 
diff -urN ./orig/q_esfq.c ./patched/q_esfq.c
--- ./orig/q_esfq.c     2004-02-26 09:28:10.000000000 +0100
+++ ./patched/q_esfq.c  2004-01-07 22:08:04.000000000 +0100
@@ -30,7 +30,7 @@
 {
        fprintf(stderr, "Usage: ... esfq [ perturb SECS ] [ quantum BYTES ]
[ depth FLOWS ]\n\t[ divisor HASHBITS ] [ limit PKTS ] [ hash
HASHTYPE]\n");
        fprintf(stderr,"Where: \n");
-       fprintf(stderr,"HASHTYPE := { classic | src | dst }\n");
+       fprintf(stderr,"HASHTYPE := { classic | src | dst | fwmark
}\n");
 }
 
 #define usage() return(-1)
@@ -95,6 +95,9 @@
                        } else
                        if(strcmp(*argv,"src") == 0) {
                                opt.hash_kind= TCA_SFQ_HASH_SRC;
+                       } else 
+                       if(strcmp(*argv,"fwmark") == 0) {
+                               opt.hash_kind= TCA_SFQ_HASH_FWMARK;
                        } else {
                                fprintf(stderr, "Illegal
\"hash\"\n");
                                explain();
@@ -148,6 +151,9 @@
        case TCA_SFQ_HASH_SRC:
                fprintf(f,"src");
                break;
+       case TCA_SFQ_HASH_FWMARK:
+               fprintf(f,"fw");
+               break;
        default:
                fprintf(f,"Unknown");
        }
diff -urN ./orig/sch_esfq.c ./patched/sch_esfq.c
--- ./orig/sch_esfq.c   2004-02-26 09:27:54.000000000 +0100
+++ ./patched/sch_esfq.c        2004-01-07 21:39:24.000000000 +0100
@@ -117,6 +117,7 @@
 {
        u32 h, h2;
        u32 hs;
+       u32 nfm;
 
        switch (skb->protocol) {
        case __constant_htons(ETH_P_IP):
@@ -124,6 +125,7 @@
                struct iphdr *iph = skb->nh.iph;
                h = iph->daddr;
                hs = iph->saddr;
+               nfm = skb -> nfmark;
                h2 = hs^iph->protocol;
                if (!(iph->frag_off&htons(IP_MF|IP_OFFSET)) &&
                    (iph->protocol == IPPROTO_TCP ||
@@ -137,6 +139,7 @@
                struct ipv6hdr *iph = skb->nh.ipv6h;
                h = iph->daddr.s6_addr32[3];
                hs = iph->saddr.s6_addr32[3];
+               nfm = skb->nfmark;
                h2 = hs^iph->nexthdr;
                if (iph->nexthdr == IPPROTO_TCP ||
                    iph->nexthdr == IPPROTO_UDP ||
@@ -148,6 +151,7 @@
                h = (u32)(unsigned long)skb->dst;
                hs = (u32)(unsigned long)skb->sk;
                h2 = hs^skb->protocol;
+               nfm = skb->nfmark;
        }
        switch(q->hash_kind)
        {
@@ -157,6 +161,8 @@
                return esfq_hash_u32(q,h);
        case TCA_SFQ_HASH_SRC:
                return esfq_hash_u32(q,hs);
+       case TCA_SFQ_HASH_FWMARK:
+               return esfq_hash_u32(q,nfm);
        default:
                if (net_ratelimit())
                        printk(KERN_DEBUG "esfq unknown hash method,
fallback to classic\n");


-- 
Greetings,
 Robert                          mailto:rkurjata@ire.pw.edu.pl

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Robert Kurjata wrote:
> Some time ago I faced a problem in limiting traffic on host with
> multiple uplinks. Since all the stuff worked nice seemed that there
> will be no problems. But then I realized that P2P users are smart
> enough to bypass limits as sfq doesn''t give fair sharing in this
> case (thousands of connections from one user versus few from the
> other).  I tried IMQ but it''s instability in my configuration was
> painfull.  So I made something like that:
>
> 1. i use IPMARK patch for the iptables to mark all the connections
> in P2P related class depending on source IP (i use SNAT),
> 2. modified ESFQ to create hash depending on FWMARK instead of src
> ip 3. and it worked. So I have uplink policy based on source ip in
> snat-ed environment without using IMQ.
>
> I''m looking for the opinions, cause I may be wrong in this.
> Patch for the files below, cause it''s short
Quite an unorthodox solution, I must say. But I guess it''s as valid as
anything is. SFQ and ESFQ are usually for situations where you have a
large amount of connections (hashes) that you just cannot track
invidually. Hence stochastic - and hence the options for perturbation
and so. If there''s only a few hashes, as most likely is in your NFMARK
case, most of the time they will hit separate hash buckets - but on
some perturbation, they might hit the same hash buckets again and
fairness is not achieved.

The patch as it was by my brief peek, looked rather okay, though.

Have you looked in to the WRR scheduler? It is meant to give an equal
share of bandwidth to all ''local'' machines with weighted round
robin
scheduling and sounds like exactly what you are looking for.

-- Naked

_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/