Hi I believe that whole question is in topic. Is there any way to recognize ( and then shape ) p2p traffic which is encrypted? Modern p2p clients have this ability moreover some of them have this enabled by default. Now I''m using ipp2p for iptables but as I know this doesn''t recognize encrypted traffic. Thanks in advance. Pozdrawiam Szymon Turkiewicz
sAwAr wrote:> > Hi > > I believe that whole question is in topic. > Is there any way to recognize ( and then shape ) p2p traffic which is encrypted? > Modern p2p clients have this ability moreover some of them have this enabled by default. > Now I''m using ipp2p for iptables but as I know this doesn''t recognize encrypted traffic. > > Thanks in advance. > > Pozdrawiam > Szymon TurkiewiczHave not tried this. An idea. P2P initiations are not encrypted AFAIK. Thus connections can be marked and related traffic shaped. If initiation is also encrypted, then I think we have a serious problem. Mohan
Some clients P2P clients are nice about there encryption and negotiate encryption ahead of time using plain communication. I.E. Limewire, Azureus. However, some just start TLS and that is all you can see. Looking at ipp2ps signatures, I don''t see anything that leads me to believe they track that kind of info. David Bierce On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote:> sAwAr wrote: >> Hi >> I believe that whole question is in topic. Is there any way to >> recognize ( and then shape ) p2p traffic which is encrypted? >> Modern p2p clients have this ability moreover some of them have >> this enabled by default. Now I''m using ipp2p for iptables but as I >> know this doesn''t recognize encrypted traffic. >> Thanks in advance. >> Pozdrawiam >> Szymon Turkiewicz > Have not tried this. An idea. P2P initiations are not encrypted > AFAIK. Thus connections can be marked and related traffic shaped. If > initiation is also encrypted, then I think we have a serious problem. > > Mohan > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Rtorrent which I use sometimes have ability to completely disable plain text
communication :
man rtorrent
allow_incoming (allow incoming encrypted connections),
try_outgoing (use encryption for outgoing connections), require (disable
unencrypted handshakes), require_RC4 (also disable plaintext
transmission after the initial encrypted handshake), enable_retry (if the
initial outgoing connection fails, retry with encryption turned on if it was
off or off if it was on), prefer_plain text (choose plaintext when peer
offers a choice between plaintext transmission and RC4 encryption, otherwise
RC4 will be used).
and many other clients have similar abilities.
I''m afraid that full encrypted and enabled by default communication is
only a
matter of time and we will lose this "fight" very soon.
> Some clients P2P clients are nice about there encryption and negotiate
> encryption ahead of time using plain communication. I.E. Limewire,
> Azureus. However, some just start TLS and that is all you can see.
>
> Looking at ipp2ps signatures, I don''t see anything that leads me
to
> believe they track that kind of info.
>
>
>
> David Bierce
>
> On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote:
> > sAwAr wrote:
> >> Hi
> >> I believe that whole question is in topic. Is there any way to
> >> recognize ( and then shape ) p2p traffic which is encrypted?
> >> Modern p2p clients have this ability moreover some of them have
> >> this enabled by default. Now I''m using ipp2p for iptables
but as I
> >> know this doesn''t recognize encrypted traffic.
> >> Thanks in advance.
> >> Pozdrawiam
> >> Szymon Turkiewicz
> >
> > Have not tried this. An idea. P2P initiations are not encrypted
> > AFAIK. Thus connections can be marked and related traffic shaped. If
> > initiation is also encrypted, then I think we have a serious problem.
> >
> > Mohan
> > _______________________________________________
> > LARTC mailing list
> > LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1> Hi > > I believe that whole question is in topic. > Is there any way to recognize ( and then shape ) p2p traffic which is encrypted? > Modern p2p clients have this ability moreover some of them have this enabled by default. > Now I''m using ipp2p for iptables but as I know this doesn''t recognize encrypted traffic. > > Thanks in advance. >As far as I am concerned there is no way to recognize encrypted p2p. I`ve tested ipp2p, and layer7, and also Mikrotik`s 3.0 p2p mark systems [ which in fact uses modified layer7 ], and none of them cant recognize this traffic simply because it is enctypted. - -- [ ... and the Slackware for All... ] gpg_key: http://szuwar.mayhem.net.pl/keys/szuwar_mayhem-public.gpg -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (MingW32) iD8DBQFHOZF96pLYJHrOtfoRAhxNAKCoOEqujQ8ibpeKKLcy4ODHXQmgAgCgh8Ur k+pLmHu2JzhZmM1mtxD9pL4=DKs2 -----END PGP SIGNATURE-----
On 11/11/07 19:51, sAwAr wrote:> Is there any way to recognize ( and then shape ) p2p traffic which is > encrypted? Modern p2p clients have this ability moreover some of > them have this enabled by default. Now I''m using ipp2p for iptables > but as I know this doesn''t recognize encrypted traffic.Does this mean that we are down to handling traffic based on the sustained stream(s)? I.e. how long the streams have been active, how many packets per second, how many streams a given end point has, speed of traffic, average size of packets? Encrypted or not, I believe all traffic can be somewhat recognized by its usage pattern(s). However there may be more false positives. We may end up recognizing what we know as good and putting the rest at a lower class of service. Thought''s / comments / objections / flame wars? Grant. . . .
On 13.11.2007 16:09, Grant Taylor wrote:> On 11/11/07 19:51, sAwAr wrote: >> Is there any way to recognize ( and then shape ) p2p traffic which is >> encrypted? Modern p2p clients have this ability moreover some of >> them have this enabled by default. Now I''m using ipp2p for iptables >> but as I know this doesn''t recognize encrypted traffic. > > Does this mean that we are down to handling traffic based on the > sustained stream(s)? I.e. how long the streams have been active, how > many packets per second, how many streams a given end point has, speed > of traffic, average size of packets? > > Encrypted or not, I believe all traffic can be somewhat recognized by > its usage pattern(s). However there may be more false positives. We > may end up recognizing what we know as good and putting the rest at a > lower class of service.Well, you can surely try. But then again, I have been doing research (publication pending) in traffic-pattern-based detection of VoIP flows and peer-to-peer connections. While it usually is easy to find a pattern matching your particular traffic class very well, part of this research has been dedicated to automatically circumvent these systems. Why that? Simple. Application evolve to circumvent detection. If you can simulate that evolution in the lab, you can find out where your detection algorithms will fail. Of course, that enumeration of possible failure modes is non-exhaustive. Bottom line: This is an arms race. Unless you do lots of research and testing, detection will always be trying to catch up. If detection manages to catch up, circumvention will advance, but you may have a small time window where you can enjoy the "win". However, winning becomes more and more expensive. Clients can expend considerable amount of CPU power to avoid detection. You don''t have that luxury in filter systems unless you have one filter per client. Regards, Carl-Daniel
On 11/13/07 09:37, Carl-Daniel Hailfinger wrote:> Well, you can surely try. But then again, I have been doing research > (publication pending) in traffic-pattern-based detection of VoIP > flows and peer-to-peer connections. While it usually is easy to find > a pattern matching your particular traffic class very well, part of > this research has been dedicated to automatically circumvent these > systems. Why that? Simple. Application evolve to circumvent > detection. If you can simulate that evolution in the lab, you can > find out where your detection algorithms will fail. Of course, that > enumeration of possible failure modes is non-exhaustive. > > Bottom line: This is an arms race. Unless you do lots of research and > testing, detection will always be trying to catch up. If detection > manages to catch up, circumvention will advance, but you may have a > small time window where you can enjoy the "win". However, winning > becomes more and more expensive. Clients can expend considerable > amount of CPU power to avoid detection. You don''t have that luxury in > filter systems unless you have one filter per client.All very good points with regard to pattern based detecting P2P (and the likes) traffic. What do you think about recognizing the traffic you do want and treating all else as a second or third class citizen. Or is this just a form of net neutrality? Or really is this entire discussion such. Further does the net neutrality issue apply to companies (read: non ISPs) wanting to filter their own internal traffic. Additionally as an aside will you please provide more information on your pending publication? I''d likely be curious to read (what ever) when ever it is published. Thanks in advance. Grant. . . .
Hello , Tuesday, November 13, 2007, 5:09:32 PM, you wrote:> Encrypted or not, I believe all traffic can be somewhat recognized by > its usage pattern(s). However there may be more false positives. We > may end up recognizing what we know as good and putting the rest at a > lower class of service.> Thought''s / comments / objections / flame wars?It''s true that encrypted/unencrypted ratio of p2p traffic increased about 10 times for past 2 years. There were a lot of predictions before about such situation will happen. I guess that we have revise shaping policing by some usage patterns as Grant said before. I guess I''ll start to reread about all netfilter features and it helps me to figure out how to solve this problem. -- Konstantin
As you might have seen, these are words from ipp2p author: """ I have seen some pieces of code from ipoque which can detect encypted bittorrent and edonkey traffic. Unforunately, this code will not work with iptables, because it needs more information about the flow history and the history of an ip address. Right now, I do not have the time and the money to develop a filter like this, but if you are interested in a developement in this direction, please contact me. """ I *think* that we need something like a "bittorrent helper" in the kernel to keep this extra information about the flow history and then an iptables plugin to match. What do you think? Maybe we could contact him to know what kind of information is it? On Nov 12, 2007 9:17 AM, sawar <sawar@interia.pl> wrote:> Rtorrent which I use sometimes have ability to completely disable plain text > communication : > > man rtorrent > allow_incoming (allow incoming encrypted connections), > try_outgoing (use encryption for outgoing connections), require (disable > unencrypted handshakes), require_RC4 (also disable plaintext > transmission after the initial encrypted handshake), enable_retry (if the > initial outgoing connection fails, retry with encryption turned on if it was > off or off if it was on), prefer_plain text (choose plaintext when peer > offers a choice between plaintext transmission and RC4 encryption, otherwise > RC4 will be used). > > and many other clients have similar abilities. > I''m afraid that full encrypted and enabled by default communication is only a > matter of time and we will lose this "fight" very soon. > > > > Some clients P2P clients are nice about there encryption and negotiate > > encryption ahead of time using plain communication. I.E. Limewire, > > Azureus. However, some just start TLS and that is all you can see. > > > > Looking at ipp2ps signatures, I don''t see anything that leads me to > > believe they track that kind of info. > > > > > > > > David Bierce > > > > On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote: > > > sAwAr wrote: > > >> Hi > > >> I believe that whole question is in topic. Is there any way to > > >> recognize ( and then shape ) p2p traffic which is encrypted? > > >> Modern p2p clients have this ability moreover some of them have > > >> this enabled by default. Now I''m using ipp2p for iptables but as I > > >> know this doesn''t recognize encrypted traffic. > > >> Thanks in advance. > > >> Pozdrawiam > > >> Szymon Turkiewicz > > > > > > Have not tried this. An idea. P2P initiations are not encrypted > > > AFAIK. Thus connections can be marked and related traffic shaped. If > > > initiation is also encrypted, then I think we have a serious problem. > > > > > > Mohan > > > _______________________________________________ > > > LARTC mailing list > > > LARTC@mailman.ds9a.nl > > > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > > > > _______________________________________________ > > LARTC mailing list > > LARTC@mailman.ds9a.nl > > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >-- Marco Casaroli SapucaiNet Telecom +55 35 34712377 ext. 5
About ipp2p, Right now, the battle against p2p is lost with l7 detection from ipp2p, l7 filter and others. Why ?? It is a known fact that pattern matching does not work with full encrypted P2P handshakes based on DHT key exchange algorithms with byte padding. You have absolutely no byte pattern and no fixed packet lengths in the stream. So something like a flow history will fail or might have a very high false +ve rate. The thing is that there are proprietary solutions which can detect fully encrypted p2p streams based on a heuristic approach. (AFAIK ipoque is selling a proprietary library for this which is integrated in some firewall vendors). I have not seen any open source development into this direction. Klaus, (former) maintainer of ipp2p Marco Aurelio wrote:> As you might have seen, these are words from ipp2p author: > > """ > > I have seen some pieces of code from ipoque which can detect encypted bittorrent > and edonkey traffic. Unforunately, this code will not work with > iptables, because it needs > more information about the flow history and the history of an ip address. > > Right now, I do not have the time and the money to develop a filter > like this, but > if you are interested in a developement in this direction, please contact me. > > """ > > I *think* that we need something like a "bittorrent helper" in the > kernel to keep this extra information about the flow history and then > an iptables plugin to match. What do you think? Maybe we could contact > him to know what kind of information is it? > > > On Nov 12, 2007 9:17 AM, sawar <sawar@interia.pl> wrote: >> Rtorrent which I use sometimes have ability to completely disable plain text >> communication : >> >> man rtorrent >> allow_incoming (allow incoming encrypted connections), >> try_outgoing (use encryption for outgoing connections), require (disable >> unencrypted handshakes), require_RC4 (also disable plaintext >> transmission after the initial encrypted handshake), enable_retry (if the >> initial outgoing connection fails, retry with encryption turned on if it was >> off or off if it was on), prefer_plain text (choose plaintext when peer >> offers a choice between plaintext transmission and RC4 encryption, otherwise >> RC4 will be used). >> >> and many other clients have similar abilities. >> I''m afraid that full encrypted and enabled by default communication is only a >> matter of time and we will lose this "fight" very soon. >> >> >>> Some clients P2P clients are nice about there encryption and negotiate >>> encryption ahead of time using plain communication. I.E. Limewire, >>> Azureus. However, some just start TLS and that is all you can see. >>> >>> Looking at ipp2ps signatures, I don''t see anything that leads me to >>> believe they track that kind of info. >>> >>> >>> >>> David Bierce >>> >>> On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote: >>>> sAwAr wrote: >>>>> Hi >>>>> I believe that whole question is in topic. Is there any way to >>>>> recognize ( and then shape ) p2p traffic which is encrypted? >>>>> Modern p2p clients have this ability moreover some of them have >>>>> this enabled by default. Now I''m using ipp2p for iptables but as I >>>>> know this doesn''t recognize encrypted traffic. >>>>> Thanks in advance. >>>>> Pozdrawiam >>>>> Szymon Turkiewicz >>>> Have not tried this. An idea. P2P initiations are not encrypted >>>> AFAIK. Thus connections can be marked and related traffic shaped. If >>>> initiation is also encrypted, then I think we have a serious problem. >>>> >>>> Mohan >>>> _______________________________________________ >>>> LARTC mailing list >>>> LARTC@mailman.ds9a.nl >>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >>> _______________________________________________ >>> LARTC mailing list >>> LARTC@mailman.ds9a.nl >>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >> _______________________________________________ >> LARTC mailing list >> LARTC@mailman.ds9a.nl >> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >> > > >
Klauss, Could you Might be you can sponsor the development ... Regards Sébastien Klaus escribió:> About ipp2p, > > Right now, the battle against p2p is lost with l7 detection from ipp2p, > l7 filter and others. > > Why ?? It is a known fact that pattern matching does not work with full > encrypted P2P handshakes based on DHT key exchange algorithms with byte > padding. You have absolutely no byte pattern and no fixed packet lengths > in the stream. So something like a flow history will fail or might have > a very high false +ve rate. > > The thing is that there are proprietary solutions which can detect fully > encrypted p2p streams based on a heuristic approach. (AFAIK ipoque is > selling a proprietary library for this which is integrated in some > firewall vendors). I have not seen any open source development into this > direction. > > Klaus, (former) maintainer of ipp2p > > > Marco Aurelio wrote: > >> As you might have seen, these are words from ipp2p author: >> >> """ >> >> I have seen some pieces of code from ipoque which can detect encypted bittorrent >> and edonkey traffic. Unforunately, this code will not work with >> iptables, because it needs >> more information about the flow history and the history of an ip address. >> >> Right now, I do not have the time and the money to develop a filter >> like this, but >> if you are interested in a developement in this direction, please contact me. >> >> """ >> >> I *think* that we need something like a "bittorrent helper" in the >> kernel to keep this extra information about the flow history and then >> an iptables plugin to match. What do you think? Maybe we could contact >> him to know what kind of information is it? >> >> >> On Nov 12, 2007 9:17 AM, sawar <sawar@interia.pl> wrote: >> >>> Rtorrent which I use sometimes have ability to completely disable plain text >>> communication : >>> >>> man rtorrent >>> allow_incoming (allow incoming encrypted connections), >>> try_outgoing (use encryption for outgoing connections), require (disable >>> unencrypted handshakes), require_RC4 (also disable plaintext >>> transmission after the initial encrypted handshake), enable_retry (if the >>> initial outgoing connection fails, retry with encryption turned on if it was >>> off or off if it was on), prefer_plain text (choose plaintext when peer >>> offers a choice between plaintext transmission and RC4 encryption, otherwise >>> RC4 will be used). >>> >>> and many other clients have similar abilities. >>> I''m afraid that full encrypted and enabled by default communication is only a >>> matter of time and we will lose this "fight" very soon. >>> >>> >>> >>>> Some clients P2P clients are nice about there encryption and negotiate >>>> encryption ahead of time using plain communication. I.E. Limewire, >>>> Azureus. However, some just start TLS and that is all you can see. >>>> >>>> Looking at ipp2ps signatures, I don''t see anything that leads me to >>>> believe they track that kind of info. >>>> >>>> >>>> >>>> David Bierce >>>> >>>> On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote: >>>> >>>>> sAwAr wrote: >>>>> >>>>>> Hi >>>>>> I believe that whole question is in topic. Is there any way to >>>>>> recognize ( and then shape ) p2p traffic which is encrypted? >>>>>> Modern p2p clients have this ability moreover some of them have >>>>>> this enabled by default. Now I''m using ipp2p for iptables but as I >>>>>> know this doesn''t recognize encrypted traffic. >>>>>> Thanks in advance. >>>>>> Pozdrawiam >>>>>> Szymon Turkiewicz >>>>>> >>>>> Have not tried this. An idea. P2P initiations are not encrypted >>>>> AFAIK. Thus connections can be marked and related traffic shaped. If >>>>> initiation is also encrypted, then I think we have a serious problem. >>>>> >>>>> Mohan >>>>> _______________________________________________ >>>>> LARTC mailing list >>>>> LARTC@mailman.ds9a.nl >>>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >>>>> >>>> _______________________________________________ >>>> LARTC mailing list >>>> LARTC@mailman.ds9a.nl >>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >>>> >>> _______________________________________________ >>> LARTC mailing list >>> LARTC@mailman.ds9a.nl >>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >>> >>> >> >> > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > > >
Sorry ... I''m little bite tired ... I mean that we might sponsor Klauss and L7 team to develop this ... Regards Sébastien CRAMATTE escribió:> Klauss, > > Could you > Might be you can sponsor the development ... > > Regards > > Sébastien > > > Klaus escribió: > >> About ipp2p, >> >> Right now, the battle against p2p is lost with l7 detection from ipp2p, >> l7 filter and others. >> >> Why ?? It is a known fact that pattern matching does not work with full >> encrypted P2P handshakes based on DHT key exchange algorithms with byte >> padding. You have absolutely no byte pattern and no fixed packet lengths >> in the stream. So something like a flow history will fail or might have >> a very high false +ve rate. >> >> The thing is that there are proprietary solutions which can detect fully >> encrypted p2p streams based on a heuristic approach. (AFAIK ipoque is >> selling a proprietary library for this which is integrated in some >> firewall vendors). I have not seen any open source development into this >> direction. >> >> Klaus, (former) maintainer of ipp2p >> >> >> Marco Aurelio wrote: >> >> >>> As you might have seen, these are words from ipp2p author: >>> >>> """ >>> >>> I have seen some pieces of code from ipoque which can detect encypted bittorrent >>> and edonkey traffic. Unforunately, this code will not work with >>> iptables, because it needs >>> more information about the flow history and the history of an ip address. >>> >>> Right now, I do not have the time and the money to develop a filter >>> like this, but >>> if you are interested in a developement in this direction, please contact me. >>> >>> """ >>> >>> I *think* that we need something like a "bittorrent helper" in the >>> kernel to keep this extra information about the flow history and then >>> an iptables plugin to match. What do you think? Maybe we could contact >>> him to know what kind of information is it? >>> >>> >>> On Nov 12, 2007 9:17 AM, sawar <sawar@interia.pl> wrote: >>> >>> >>>> Rtorrent which I use sometimes have ability to completely disable plain text >>>> communication : >>>> >>>> man rtorrent >>>> allow_incoming (allow incoming encrypted connections), >>>> try_outgoing (use encryption for outgoing connections), require (disable >>>> unencrypted handshakes), require_RC4 (also disable plaintext >>>> transmission after the initial encrypted handshake), enable_retry (if the >>>> initial outgoing connection fails, retry with encryption turned on if it was >>>> off or off if it was on), prefer_plain text (choose plaintext when peer >>>> offers a choice between plaintext transmission and RC4 encryption, otherwise >>>> RC4 will be used). >>>> >>>> and many other clients have similar abilities. >>>> I''m afraid that full encrypted and enabled by default communication is only a >>>> matter of time and we will lose this "fight" very soon. >>>> >>>> >>>> >>>> >>>>> Some clients P2P clients are nice about there encryption and negotiate >>>>> encryption ahead of time using plain communication. I.E. Limewire, >>>>> Azureus. However, some just start TLS and that is all you can see. >>>>> >>>>> Looking at ipp2ps signatures, I don''t see anything that leads me to >>>>> believe they track that kind of info. >>>>> >>>>> >>>>> >>>>> David Bierce >>>>> >>>>> On Nov 11, 2007, at 9:48 PM, Mohan Sundaram wrote: >>>>> >>>>> >>>>>> sAwAr wrote: >>>>>> >>>>>> >>>>>>> Hi >>>>>>> I believe that whole question is in topic. Is there any way to >>>>>>> recognize ( and then shape ) p2p traffic which is encrypted? >>>>>>> Modern p2p clients have this ability moreover some of them have >>>>>>> this enabled by default. Now I''m using ipp2p for iptables but as I >>>>>>> know this doesn''t recognize encrypted traffic. >>>>>>> Thanks in advance. >>>>>>> Pozdrawiam >>>>>>> Szymon Turkiewicz >>>>>>> >>>>>>> >>>>>> Have not tried this. An idea. P2P initiations are not encrypted >>>>>> AFAIK. Thus connections can be marked and related traffic shaped. If >>>>>> initiation is also encrypted, then I think we have a serious problem. >>>>>> >>>>>> Mohan >>>>>> _______________________________________________ >>>>>> LARTC mailing list >>>>>> LARTC@mailman.ds9a.nl >>>>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> LARTC mailing list >>>>> LARTC@mailman.ds9a.nl >>>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >>>>> >>>>> >>>> _______________________________________________ >>>> LARTC mailing list >>>> LARTC@mailman.ds9a.nl >>>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >>>> >>>> >>>> >>> >>> >> _______________________________________________ >> LARTC mailing list >> LARTC@mailman.ds9a.nl >> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >> >> >> >> > > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > > >
> I believe that whole question is in topic. > Is there any way to recognize ( and then shape ) p2p traffic which is encrypted? > Modern p2p clients have this ability moreover some of them have this enabled by default. > Now I''m using ipp2p for iptables but as I know this doesn''t recognize encrypted traffic.One way to do this is to look for the style of traffic. For example, I look for lots of connections from one PC to port numbers above 1024. This will also incorrectly recognise some other traffic, but on the whole it works well for me. The following are some examples using connlimit (now included in vanilla kernel) and ipset (see http://ipset.netfilter.org/) # first look for style of traffic and log that client to an ipset iptables -t mangle -A FORWARD -o ppp0 -p tcp --dport 1024: \ -m connlimit --connlimit-above 10 -j SET --add-set p2p src iptables -t mangle -A FORWARD -o ppp0 -p udp --dport 1024: \ -m connlimit --connlimit-above 10 -j SET --add-set p2p src iptables -t mangle -A FORWARD -i ppp0 -p tcp --sport 1024: \ -m connlimit --connlimit-above 10 -j SET --add-set p2p dst iptables -t mangle -A FORWARD -i ppp0 -p udp --sport 1024: \ -m connlimit --connlimit-above 10 -j SET --add-set p2p dst # then shape traffic above port 1024 for those detected clients iptables -t mangle -A FORWARD -o ppp0 -p tcp --dport 1024: \ -m set --set p2p dst -j MARK --set-mark 60 iptables -t mangle -A FORWARD -i ppp0 -p tcp --sport 1024: \ -m set --set p2p dst -j MARK --set-mark 60 iptables -t mangle -A FORWARD -o ppp0 -p udp --dport 1024: \ -m set --set p2p dst -j MARK --set-mark 60 iptables -t mangle -A FORWARD -i ppp0 -p udp --sport 1024: \ -m set --set p2p dst -j MARK --set-mark 60 Regards, Andy Beverley
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I believe "fighting" is the wrong approach. Badly shaping the wrong traffic is just as bad, if not worse IMO. An ISP in my neck of the woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a result of their P2P fight. Needless to say we no longer use them, and we encourage clients, friends, and colleagues not to as well. I don''t use P2P but I do use ssh, imaps, sftp, and https daily. Screwing with these services is not useful. Limiting your rules to specific ports is pretty useless. This has been done before, and it failed miserably. For me, if P2P does not belong at all, for instance on a corporate network, then a default deny on the outbound works much better. We then only allow specific connections on a case by case basis. For instances where I am not able to block p2p, I define specific rules for high and low priority, and leave everything else in the default. If the end user wants to use the bulk of his or her bandwidth for P2P, so be it. Of course in this case bandwidth accounting is far more useful. I would also encourage your users to use software that is or can be well behaved. Software that allows you set a proper TOS for instance. If possible work with the end users. I have personally found that the best solutions are not tech solutions. Having a well defined Acceptable Use Policy, plus a constructive dialogue with my users has been far more effective than any shaping routine I/we could come up with. Ask yourself, what is the problem you are really trying to solve. Andrew Beverley wrote:>> I believe that whole question is in topic. >> Is there any way to recognize ( and then shape ) p2p traffic which is encrypted? >> Modern p2p clients have this ability moreover some of them have this enabled by default. >> Now I''m using ipp2p for iptables but as I know this doesn''t recognize encrypted traffic. > > One way to do this is to look for the style of traffic. For example, I > look for lots of connections from one PC to port numbers above 1024. > This will also incorrectly recognise some other traffic, but on the > whole it works well for me. > > The following are some examples using connlimit (now included in vanilla > kernel) and ipset (see http://ipset.netfilter.org/) > > # first look for style of traffic and log that client to an ipset > iptables -t mangle -A FORWARD -o ppp0 -p tcp --dport 1024: \ > -m connlimit --connlimit-above 10 -j SET --add-set p2p src > iptables -t mangle -A FORWARD -o ppp0 -p udp --dport 1024: \ > -m connlimit --connlimit-above 10 -j SET --add-set p2p src > iptables -t mangle -A FORWARD -i ppp0 -p tcp --sport 1024: \ > -m connlimit --connlimit-above 10 -j SET --add-set p2p dst > iptables -t mangle -A FORWARD -i ppp0 -p udp --sport 1024: \ > -m connlimit --connlimit-above 10 -j SET --add-set p2p dst > > # then shape traffic above port 1024 for those detected clients > iptables -t mangle -A FORWARD -o ppp0 -p tcp --dport 1024: \ > -m set --set p2p dst -j MARK --set-mark 60 > iptables -t mangle -A FORWARD -i ppp0 -p tcp --sport 1024: \ > -m set --set p2p dst -j MARK --set-mark 60 > iptables -t mangle -A FORWARD -o ppp0 -p udp --dport 1024: \ > -m set --set p2p dst -j MARK --set-mark 60 > iptables -t mangle -A FORWARD -i ppp0 -p udp --sport 1024: \ > -m set --set p2p dst -j MARK --set-mark 60 > > > Regards, > > Andy Beverley > > > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.5 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFHU98owRXgH3rKGfMRAmszAKCIhVoUnfuWDIaWQqwE1WfuSz9sNwCgipFZ wqrptNaNg3HMFE79AvbQ+fI=gb3i -----END PGP SIGNATURE-----
> I believe "fighting" is the wrong approach. Badly shaping the wrong > traffic is just as bad, if not worse IMO. An ISP in my neck of the > woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a > result of their P2P fight. Needless to say we no longer use them, and > we encourage clients, friends, and colleagues not to as well. I don''t > use P2P but I do use ssh, imaps, sftp, and https daily. Screwing with > these services is not useful.Using the rules in the example previously given specifically steers well clear of these services.> Limiting your rules to specific ports is > pretty useless. This has been done before, and it failed miserably.Agreed.> For me, if P2P does not belong at all, for instance on a corporate > network, then a default deny on the outbound works much better. We then > only allow specific connections on a case by case basis.I have seen this work very well on corporate networks, and would recommend this approach where possible. Unfortunately though, on a normal home user network, there are so many different possibilities that this isn''t very practical.> For instances > where I am not able to block p2p, I define specific rules for high and > low priority, and leave everything else in the default. If the end user > wants to use the bulk of his or her bandwidth for P2P, so be it. Of > course in this case bandwidth accounting is far more useful.Again, this depends on the circumstances. If you only have 2Mbit/s to share between 100 users then each user cannot have their own ''share'' of the connection. Equally, people downloading in a responsible way are lumped into the same category as p2p users, which is not fair. Bandwidth accounting is a possibility, and something I haven''t investigated. For those who want to fairly share bandwidth beween users, I would recommend the ESFQ patches. These allow bandwidth sharing to be done on an IP address basis, rather than per connection. This prevents the hundreds of p2p connections from drowning out single downloads.> I would also encourage your users to use software that is or can be well > behaved. Software that allows you set a proper TOS for instance. If > possible work with the end users. > I have personally found that the best solutions are not tech solutions. > Having a well defined Acceptable Use Policy, plus a constructive > dialogue with my users has been far more effective than any shaping > routine I/we could come up with.Agreed. However, in a situation where you have a lot of users coming and going, it is not easy to educate the many hundreds of users. I guess it all boils down to your own situation. Traffic shaping on a corporate network or on a network where your users are static can be done using the above techniques. However, sharing a small connection between hundreds of regularly changing users is difficult, and I have found the ''blunt'' rules previously described to work very well with no complaints. Regards, Andy Beverley
Hi, We had similiar problem with p2p, used ipp2p and L7filter together before and worked well until clients( mostly clever ones) started getting around it with encryption. We have about 700 wireless clients hitting our network and our network was taking big knocks with guys using couple of gigs day on entry level packages. Was going to use Ipoque, but was quite pricy for us, Only solutions for us to use a daily limit of eg 500MB, then they get slowed down to slower speeds, This worked like a charm Out of interest we used freeradius / pptpd|pppd with some custom perl scripts and tc rules Sew On Dec 3, 2007 9:33 PM, Andrew Beverley <andy@andybev.com> wrote:> > I believe "fighting" is the wrong approach. Badly shaping the wrong > > traffic is just as bad, if not worse IMO. An ISP in my neck of the > > woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a > > result of their P2P fight. Needless to say we no longer use them, and > > we encourage clients, friends, and colleagues not to as well. I don''t > > use P2P but I do use ssh, imaps, sftp, and https daily. Screwing with > > these services is not useful. > > Using the rules in the example previously given specifically steers well clear > of these services. > > > Limiting your rules to specific ports is > > pretty useless. This has been done before, and it failed miserably. > > Agreed. > > > For me, if P2P does not belong at all, for instance on a corporate > > network, then a default deny on the outbound works much better. We then > > only allow specific connections on a case by case basis. > > I have seen this work very well on corporate networks, and would > recommend this > approach where possible. Unfortunately though, on a normal home user network, > there are so many different possibilities that this isn''t very practical. > > > For instances > > where I am not able to block p2p, I define specific rules for high and > > low priority, and leave everything else in the default. If the end user > > wants to use the bulk of his or her bandwidth for P2P, so be it. Of > > course in this case bandwidth accounting is far more useful. > > Again, this depends on the circumstances. If you only have 2Mbit/s to share > between 100 users then each user cannot have their own ''share'' of the > connection. Equally, people downloading in a responsible way are lumped > into the > same category as p2p users, which is not fair. Bandwidth accounting is a > possibility, and something I haven''t investigated. > > For those who want to fairly share bandwidth beween users, I would > recommend the > ESFQ patches. These allow bandwidth sharing to be done on an IP address basis, > rather than per connection. This prevents the hundreds of p2p connections from > drowning out single downloads. > > > I would also encourage your users to use software that is or can be well > > behaved. Software that allows you set a proper TOS for instance. If > > possible work with the end users. > > I have personally found that the best solutions are not tech solutions. > > Having a well defined Acceptable Use Policy, plus a constructive > > dialogue with my users has been far more effective than any shaping > > routine I/we could come up with. > > Agreed. However, in a situation where you have a lot of users coming > and going, > it is not easy to educate the many hundreds of users. > > I guess it all boils down to your own situation. Traffic shaping on a > corporate > network or on a network where your users are static can be done using > the above > techniques. However, sharing a small connection between hundreds of regularly > changing users is difficult, and I have found the ''blunt'' rules previously > described to work very well with no complaints. > > > Regards, > > Andy Beverley > > > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >
Thanks for sharing. Just one question, how are you implementing the daily limit? Regards, Mario Antonio ----- Original Message ----- From: "the sew" <sewlist@gmail.com> To: "Andrew Beverley" <andy@andybev.com> Cc: lartc@mailman.ds9a.nl Sent: Monday, December 10, 2007 8:37:07 AM (GMT-0500) America/New_York Subject: Re: [LARTC] How to fight with encrypted p2p Hi, We had similiar problem with p2p, used ipp2p and L7filter together before and worked well until clients( mostly clever ones) started getting around it with encryption. We have about 700 wireless clients hitting our network and our network was taking big knocks with guys using couple of gigs day on entry level packages. Was going to use Ipoque, but was quite pricy for us, Only solutions for us to use a daily limit of eg 500MB, then they get slowed down to slower speeds, This worked like a charm Out of interest we used freeradius / pptpd|pppd with some custom perl scripts and tc rules Sew On Dec 3, 2007 9:33 PM, Andrew Beverley <andy@andybev.com> wrote:> > I believe "fighting" is the wrong approach. Badly shaping the wrong > > traffic is just as bad, if not worse IMO. An ISP in my neck of the > > woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a > > result of their P2P fight. Needless to say we no longer use them, and > > we encourage clients, friends, and colleagues not to as well. I don''t > > use P2P but I do use ssh, imaps, sftp, and https daily. Screwing with > > these services is not useful. > > Using the rules in the example previously given specifically steers well clear > of these services. > > > Limiting your rules to specific ports is > > pretty useless. This has been done before, and it failed miserably. > > Agreed. > > > For me, if P2P does not belong at all, for instance on a corporate > > network, then a default deny on the outbound works much better. We then > > only allow specific connections on a case by case basis. > > I have seen this work very well on corporate networks, and would > recommend this > approach where possible. Unfortunately though, on a normal home user network, > there are so many different possibilities that this isn''t very practical. > > > For instances > > where I am not able to block p2p, I define specific rules for high and > > low priority, and leave everything else in the default. If the end user > > wants to use the bulk of his or her bandwidth for P2P, so be it. Of > > course in this case bandwidth accounting is far more useful. > > Again, this depends on the circumstances. If you only have 2Mbit/s to share > between 100 users then each user cannot have their own ''share'' of the > connection. Equally, people downloading in a responsible way are lumped > into the > same category as p2p users, which is not fair. Bandwidth accounting is a > possibility, and something I haven''t investigated. > > For those who want to fairly share bandwidth beween users, I would > recommend the > ESFQ patches. These allow bandwidth sharing to be done on an IP address basis, > rather than per connection. This prevents the hundreds of p2p connections from > drowning out single downloads. > > > I would also encourage your users to use software that is or can be well > > behaved. Software that allows you set a proper TOS for instance. If > > possible work with the end users. > > I have personally found that the best solutions are not tech solutions. > > Having a well defined Acceptable Use Policy, plus a constructive > > dialogue with my users has been far more effective than any shaping > > routine I/we could come up with. > > Agreed. However, in a situation where you have a lot of users coming > and going, > it is not easy to educate the many hundreds of users. > > I guess it all boils down to your own situation. Traffic shaping on a > corporate > network or on a network where your users are static can be done using > the above > techniques. However, sharing a small connection between hundreds of regularly > changing users is difficult, and I have found the ''blunt'' rules previously > described to work very well with no complaints. > > > Regards, > > Andy Beverley > > > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >_______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
On Dec 10, 2007 4:09 PM, Mario Antonio Garcia <dino@webjogger.net> wrote:> Thanks for sharing.No problem> > Just one question, how are you implementing the daily limit?two ways of detecting them: 1st is the /etc/ppp/ip-up.local which executes the script to check usage against the radius db and shape them on authentication, Once they logged in , I dont want to kick everone off every few hours to check usage, I have a "nice" cron job running every 3 hours, to check every single user against the db and if they reached their quotas ,they get shaped while being online., Radius stores all info about the nas in the db, so makes it quite strait forward. Also nicely added is our reselling guys who maintain the clients get a report everytime of users who exceeded the limit and can be aware of which clients is the problem if they phone to complain, Also nice is that usually the infected pc''s gets knocked off first to save alot of bandwidth Sew> > Regards, > > Mario Antonio > > > ----- Original Message ----- > From: "the sew" <sewlist@gmail.com> > To: "Andrew Beverley" <andy@andybev.com> > Cc: lartc@mailman.ds9a.nl > Sent: Monday, December 10, 2007 8:37:07 AM (GMT-0500) America/New_York > Subject: Re: [LARTC] How to fight with encrypted p2p > > Hi, > > We had similiar problem with p2p, used ipp2p and L7filter together > before and worked well until clients( mostly clever ones) started > getting around it with encryption. We have about 700 wireless clients > hitting our network and our network was taking big knocks with guys > using couple of gigs day on entry level packages. > > Was going to use Ipoque, but was quite pricy for us, Only solutions > for us to use a daily limit of eg 500MB, then they get slowed down to > slower speeds, This worked like a charm > > Out of interest we used freeradius / pptpd|pppd with some custom perl > scripts and tc rules > > Sew > > On Dec 3, 2007 9:33 PM, Andrew Beverley <andy@andybev.com> wrote: > > > I believe "fighting" is the wrong approach. Badly shaping the wrong > > > traffic is just as bad, if not worse IMO. An ISP in my neck of the > > > woods plays havoc with encrypted mail (SMTP + TLS as well as IMAPS) as a > > > result of their P2P fight. Needless to say we no longer use them, and > > > we encourage clients, friends, and colleagues not to as well. I don''t > > > use P2P but I do use ssh, imaps, sftp, and https daily. Screwing with > > > these services is not useful. > > > > Using the rules in the example previously given specifically steers well clear > > of these services. > > > > > Limiting your rules to specific ports is > > > pretty useless. This has been done before, and it failed miserably. > > > > Agreed. > > > > > For me, if P2P does not belong at all, for instance on a corporate > > > network, then a default deny on the outbound works much better. We then > > > only allow specific connections on a case by case basis. > > > > I have seen this work very well on corporate networks, and would > > recommend this > > approach where possible. Unfortunately though, on a normal home user network, > > there are so many different possibilities that this isn''t very practical. > > > > > For instances > > > where I am not able to block p2p, I define specific rules for high and > > > low priority, and leave everything else in the default. If the end user > > > wants to use the bulk of his or her bandwidth for P2P, so be it. Of > > > course in this case bandwidth accounting is far more useful. > > > > Again, this depends on the circumstances. If you only have 2Mbit/s to share > > between 100 users then each user cannot have their own ''share'' of the > > connection. Equally, people downloading in a responsible way are lumped > > into the > > same category as p2p users, which is not fair. Bandwidth accounting is a > > possibility, and something I haven''t investigated. > > > > For those who want to fairly share bandwidth beween users, I would > > recommend the > > ESFQ patches. These allow bandwidth sharing to be done on an IP address basis, > > rather than per connection. This prevents the hundreds of p2p connections from > > drowning out single downloads. > > > > > I would also encourage your users to use software that is or can be well > > > behaved. Software that allows you set a proper TOS for instance. If > > > possible work with the end users. > > > I have personally found that the best solutions are not tech solutions. > > > Having a well defined Acceptable Use Policy, plus a constructive > > > dialogue with my users has been far more effective than any shaping > > > routine I/we could come up with. > > > > Agreed. However, in a situation where you have a lot of users coming > > and going, > > it is not easy to educate the many hundreds of users. > > > > I guess it all boils down to your own situation. Traffic shaping on a > > corporate > > network or on a network where your users are static can be done using > > the above > > techniques. However, sharing a small connection between hundreds of regularly > > changing users is difficult, and I have found the ''blunt'' rules previously > > described to work very well with no complaints. > > > > > > Regards, > > > > Andy Beverley > > > > > > _______________________________________________ > > LARTC mailing list > > LARTC@mailman.ds9a.nl > > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > > > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > > > > >