Hi everyone, today I got an e-mail from a company claiming that my server is doing port scans on their firewall machine. I found that hard to believe so I started checking the box. The company rep told me that the scan was originating at port 80 with destination port 8254 on their machine. I couldn't find any hints as to why that computer was subject to the alleged port scans. Searching in logs and crontab entries did not reveal the domain name or IP address of the machine except for my web mailer. It seems that someone from the company's network is accessing the web mailer in 10-15 minute intervals which is absolutely believable since one of my users works for the company and checks his mail via the web mailer. The strange part is that the company rep said these scans started some time on Sunday, while my user definitely was not using the company's hardware. Apparently, the company uses NetScreen hardware and/or software for such intrusion detection / prevention mechanisms and the log he provided read: [Root]system-alert-00016: Port scan! From $my-server-ip:80 to $their-server-ip:8254, proto TCP (zone Untrust, int ethernet1). Occurred 1 times. My questions are: 1. Can this be malicious code on my side? Both port 80 and 443 are bound to Apache's httpd so they shouldn't be available to other processes, right? 2. I'm using ipfw as a firewall where everything is denied except for a rather tight permitting ruleset that (of course) allows communication to/from port 80/443 on my machine but not to the destination port 8254. If the firewall prohibits access to a remote port 8254, processes on my side shouldn't be able to initiate a connection to that port. If there is a connection to that port, it had to be established earlier by the remote machine. Am I correct? 3. Does anyone know when the NetScreen hardware / software labels something "port scan"? As far as I can tell, the server is free of malicious code, I especially looked for PHP (and similar) files belonging to freely available port scanners etc.; everything seems to be alright. While I was investigating, no one but me was logged in. Any help is greatly appreciated! Clemens
On 0, Clemens Renner <claim@rinux.net> wrote:> Hi everyone, > > today I got an e-mail from a company claiming that my server is doing > port scans on their firewall machine. I found that hard to believe so I > started checking the box. > > The company rep told me that the scan was originating at port 80 with > destination port 8254 on their machine. I couldn't find any hints as to > why that computer was subject to the alleged port scans. Searching in > logs and crontab entries did not reveal the domain name or IP address of > the machine except for my web mailer. It seems that someone from the > company's network is accessing the web mailer in 10-15 minute intervals > which is absolutely believable since one of my users works for the > company and checks his mail via the web mailer. The strange part is that > the company rep said these scans started some time on Sunday, while my > user definitely was not using the company's hardware. > > Apparently, the company uses NetScreen hardware and/or software for such > intrusion detection / prevention mechanisms and the log he provided read: > > [Root]system-alert-00016: Port scan! From $my-server-ip:80 to > $their-server-ip:8254, proto TCP (zone Untrust, int ethernet1). Occurred > 1 times. > > My questions are: > 1. Can this be malicious code on my side? Both port 80 and 443 are bound > to Apache's httpd so they shouldn't be available to other processes, right? > > 2. I'm using ipfw as a firewall where everything is denied except for a > rather tight permitting ruleset that (of course) allows communication > to/from port 80/443 on my machine but not to the destination port 8254. > If the firewall prohibits access to a remote port 8254, processes on my > side shouldn't be able to initiate a connection to that port. If there > is a connection to that port, it had to be established earlier by the > remote machine. Am I correct? > > 3. Does anyone know when the NetScreen hardware / software labels > something "port scan"? > > As far as I can tell, the server is free of malicious code, I especially > looked for PHP (and similar) files belonging to freely available port > scanners etc.; everything seems to be alright. While I was > investigating, no one but me was logged in. > > Any help is greatly appreciated! > ClemensAsk them for a packet capture of the incident(s). It may well be that they have a false positive case on their hands. Portscan detection is very much prone to false positives, many things can appear to be portscans when they really aren't. A log message like the one they gave you is nowhere near enough information to determine if the attempt was a real portscan or not. +--------------------------------------------------------------------+ Nigel Houghton Research Engineer Sourcefire Inc. Vulnerability Research Team There is no theory of evolution, just a list of creatures Vin Diesel allows to live.
Clemens Renner wrote:> Hi everyone, > > today I got an e-mail from a company claiming that my server is doing > port scans on their firewall machine. I found that hard to believe so > I started checking the box. > > The company rep told me that the scan was originating at port 80 with > destination port 8254 on their machine. I couldn't find any hints as > to why that computer was subject to the alleged port scans. Searching > in logs and crontab entries did not reveal the domain name or IP > address of the machine except for my web mailer. It seems that someone > from the company's network is accessing the web mailer in 10-15 minute > intervals which is absolutely believable since one of my users works > for the company and checks his mail via the web mailer. The strange > part is that the company rep said these scans started some time on > Sunday, while my user definitely was not using the company's hardware. > > Apparently, the company uses NetScreen hardware and/or software for > such intrusion detection / prevention mechanisms and the log he > provided read: > > [Root]system-alert-00016: Port scan! From $my-server-ip:80 to > $their-server-ip:8254, proto TCP (zone Untrust, int ethernet1). > Occurred 1 times.some of their clients accessed your machine a few times and had sequential port numbers on their side.. then netscreen got confused. (probably) on the safe side, run snort on your outside interface for a while.> > My questions are: > 1. Can this be malicious code on my side? Both port 80 and 443 are > bound to Apache's httpd so they shouldn't be available to other > processes, right? > > 2. I'm using ipfw as a firewall where everything is denied except for > a rather tight permitting ruleset that (of course) allows > communication to/from port 80/443 on my machine but not to the > destination port 8254. If the firewall prohibits access to a remote > port 8254, processes on my side shouldn't be able to initiate a > connection to that port. If there is a connection to that port, it had > to be established earlier by the remote machine. Am I correct? > > 3. Does anyone know when the NetScreen hardware / software labels > something "port scan"? > > As far as I can tell, the server is free of malicious code, I > especially looked for PHP (and similar) files belonging to freely > available port scanners etc.; everything seems to be alright. While I > was investigating, no one but me was logged in. > > Any help is greatly appreciated! > Clemens > _______________________________________________ > freebsd-security@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-security > To unsubscribe, send any mail to > "freebsd-security-unsubscribe@freebsd.org"
On Tuesday, 2006-07-18 at 18:11:50 +0200, Clemens Renner wrote:> [Root]system-alert-00016: Port scan! From $my-server-ip:80 to > $their-server-ip:8254, proto TCP (zone Untrust, int ethernet1). Occurred > 1 times.With IPFilter, I often see "dangling FINs" in the log. These occur when the TCP connection has been shut down but an additional FIN is still travelling. IPFilter will have abandoned the state for the connection, so for it these FIN are not associated to a connection. Since the message they gave you is of the "Danger, Will Robinson" kind, this could be the case. They can't prove it wrong. To me, this is a case of stupid until proven intelligent. HTH, Lupe Christoph PS: I thought a port scan means somebody is probing many ports. How can one packet be considered a port scan?!? -- | You know we're sitting on four million pounds of fuel, one nuclear | | weapon and a thing that has 270,000 moving parts built by the lowest | | bidder. Makes you feel good, doesn't it? | | Rockhound in "Armageddon", 1998, about the Space Shuttle |
On Tue, Jul 18, 2006, Clemens Renner wrote:> today I got an e-mail from a company claiming that my server is doing > port scans on their firewall machine. I found that hard to believe so I > started checking the box.Do you have mod_proxy or other modules with proxy functionality in your web server? -cs
Clemens Renner wrote:> Hi everyone, > > today I got an e-mail from a company claiming that my server is doing > port scans on their firewall machine. I found that hard to believe so I > started checking the box. > > The company rep told me that the scan was originating at port 80 with > destination port 8254 on their machine. I couldn't find any hints as to > why that computer was subject to the alleged port scans. Searching in > logs and crontab entries did not reveal the domain name or IP address of > the machine except for my web mailer. It seems that someone from the > company's network is accessing the web mailer in 10-15 minute intervals > which is absolutely believable since one of my users works for the > company and checks his mail via the web mailer. The strange part is that > the company rep said these scans started some time on Sunday, while my > user definitely was not using the company's hardware. > > Apparently, the company uses NetScreen hardware and/or software for such > intrusion detection / prevention mechanisms and the log he provided read:Almost definitely a false alarm. Firewalls (not just Netscreen) keep track of active TCP connections passing through them. If they stay idle for too long, the firewall assumes the other end died and drops it from its tracking table. Someone behind their firewall viewed your website. If you have, say, 6 images on it, then 7 connections get maintained in the firewall's state table, probably from sequential source port numbers. If you have Apache's keepalives on, then those 7 HTTP connections get held open for a while in case they request more pages/images from you. The problem is when Apache's keepalive interval is longer than the firewall's idle connection retention interval. If the firewall is configured to forget about idle connections after 5 minutes and Apache's keeping connections alive for 8 minutes, then two minutes after the firewall forgets about it, it will log Apache's attempt to close the connection as a FIN scan from 7 different ports. Find out what that TCP interval is on their Netscreen and adjust your Apache keepalive to be less than that. I think we went all the way down to 2 minutes before the dumber firewall admins stopped emailing us. This isn't limited to Netscreen either... Sonicwalls were overly sensitive to this a while back but I think they put out a firmware update to shut up some of the false alarms. PIX firewalls tend to have longer defaults so you don't run into that as much. If you're an ISP, every now and then you'll get similar complaints from your customers complaining that your nameserver is attacking them. Same story -- a slow DNS lookup that takes longer than their firewall is willing to wait on a UDP response, and they assume that every single thing a firewall logs is from an OMG WTF DDOS script kiddie... :) -- Mike Andrews * mandrews@bit0.com * http://www.bit0.com It's not news, it's Fark.com. Carpe cavy!
Clemens Renner wrote:> Hi everyone, > > today I got an e-mail from a company claiming that my server is doing > port scans on their firewall machine. I found that hard to believe so > I started checking the box. > > The company rep told me that the scan was originating at port 80 with > destination port 8254 on their machine. I couldn't find any hints as > to why that computer was subject to the alleged port scans. Searching > in logs and crontab entries did not reveal the domain name or IP > address of the machine except for my web mailer. It seems that someone > from the company's network is accessing the web mailer in 10-15 minute > intervals which is absolutely believable since one of my users works > for the company and checks his mail via the web mailer. The strange > part is that the company rep said these scans started some time on > Sunday, while my user definitely was not using the company's hardware. > > Apparently, the company uses NetScreen hardware and/or software for > such intrusion detection / prevention mechanisms and the log he > provided read: > > [Root]system-alert-00016: Port scan! From $my-server-ip:80 to > $their-server-ip:8254, proto TCP (zone Untrust, int ethernet1). > Occurred 1 times. > > My questions are: > 1. Can this be malicious code on my side? Both port 80 and 443 are > bound to Apache's httpd so they shouldn't be available to other > processes, right? > > 2. I'm using ipfw as a firewall where everything is denied except for > a rather tight permitting ruleset that (of course) allows > communication to/from port 80/443 on my machine but not to the > destination port 8254. If the firewall prohibits access to a remote > port 8254, processes on my side shouldn't be able to initiate a > connection to that port. If there is a connection to that port, it had > to be established earlier by the remote machine. Am I correct? > > 3. Does anyone know when the NetScreen hardware / software labels > something "port scan"? > > As far as I can tell, the server is free of malicious code, I > especially looked for PHP (and similar) files belonging to freely > available port scanners etc.; everything seems to be alright. While I > was investigating, no one but me was logged in. > > Any help is greatly appreciated! > Clemens > _______________________________________________ > freebsd-security@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-security > To unsubscribe, send any mail to > "freebsd-security-unsubscribe@freebsd.org" >We had a client that was being bombarded with a SYN flood on port 80, and of course enabling syn cookies helped. However all the IP's that were sending the SYN flood were spoofed, and we were getting complains left right and center of this customer DoSing or port scanning other customers. In the end, we just asked the complainant to provide move verbose logging of the incident. -jt
> -----Original Message----- > From: owner-freebsd-security@freebsd.org > [mailto:owner-freebsd-security@freebsd.org] On Behalf Of comm@rwx.ca > Sent: Friday, July 21, 2006 12:43 AM > To: Clemens Renner > Cc: freebsd-security@freebsd.org > Subject: Re: Port scan from Apache? > > > Clemens Renner wrote: > > Hi everyone, > > > > today I got an e-mail from a company claiming that my > server is doing > > port scans on their firewall machine. I found that hard to > believe so > > I started checking the box.Let me put my 2/c (CAD) into this, as a user of netscreens, the CTO of a Managed network security service. The person who sent you the 'alert' might be wrong. We see "port scans" from web servers (incrementing source ports > 1024, destination port 80) and it is usually just noise, internet traffic, and the failure of his netscreen to properly close the connection. Can you correlate the netscreen logs with times his users have accessed your web site? Do you have complaints from just this one person? Send him a note telling him this is just normal internet traffic and that he should try to understand the three way TCP handshake, and what stateful firewalls do when they close their side of the TCP connection before you do. If it happens A LOT, to lots of different networks, then, well, it is possible you have a worm, do a tcpdump on the traffic and look for it. Another possibility, is that your web site spawns many different http threads for each user connection (do you have a zillion thumbnail gifs? Each one could spawn a different tcp connection) Do you have an unusually high keep-alive? It YOUR firewall closing (timing out) the tcp connection? Mostly, if this was just one complaint, grep your web server logs for his user connecting, tell him this is just normal tcp traffic and go about your business from then on. If he gets rude, blacklist him and/or send him a $50 lawyer letter and tell him to either drop dead or call his local FBI (or RCMP) office.