CentOS - Sep 2011 - Sort logfiles on common lines?

I have a huge mysql.log file full of errors. I'd like to sort it by
the most common line, and work from there. I did go through the
manpage for sort, and googled a bit, but I found nothing relevant.

Here is an example of the output:
[root@ log]# tail mysqld.log
110925 11:05:35 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_summary_ad_hourly.MYI'; try to repair it
110925 11:05:35 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_summary_ad_hourly.MYI'; try to repair it
110925 12:05:28 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_intermediate_ad.MYI'; try to repair it
110925 12:05:28 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_intermediate_ad.MYI'; try to repair it
110925 12:05:28 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_intermediate_ad.MYI'; try to repair it
110925 12:05:28 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_summary_ad_hourly.MYI'; try to repair it
110925 13:09:43 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_intermediate_ad.MYI'; try to repair it
110925 13:09:43 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_intermediate_ad.MYI'; try to repair it
110925 13:09:43 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_intermediate_ad.MYI'; try to repair it
110925 13:09:43 [ERROR] /usr/libexec/mysqld: Incorrect key file for
table './ox_data_summary_ad_hourly.MYI'; try to repair it
[root@ log]# wc -l mysqld.log
20686 mysqld.log
[root@ log]# cat mysqld.log | grep ERROR | wc -l
20332
[root@ log]#


Is there a way to get the most common (unique) lines of the file?


By the way, I'm not sure if this is RHEL or CentOS, or which version:
[root@ log]# uname -a
Linux example.com 2.6.18-194.32.1.el5xen #1 SMP Wed Jan 5 18:44:24 EST
2011 x86_64 x86_64 x86_64 GNU/Linux
[root@ log]# uname -o
GNU/Linux
[root@ log]#

I assume that it is one of these, as Yum is installed. How would I find out?

Thanks!

-- 
Dotan Cohen

http://gibberish.co.il
http://what-is-what.com

On 09/25/11 11:51 AM, Dotan Cohen wrote:
> ...
> 110925 13:09:43 [ERROR] /usr/libexec/mysqld: Incorrect key file for
> table './ox_data_summary_ad_hourly.MYI'; try to repair it
> [root@ log]# wc -l mysqld.log
> 20686 mysqld.log
> [root@ log]# cat mysqld.log | grep ERROR | wc -l
> 20332
> [root@ log]#
>
>
> Is there a way to get the most common (unique) lines of the file?
sort -k 3 | uniq -f 2


which will sort starting at field 3, and then print lines that are 
unique, skipping the first 2 fields, where fields by default are blank 
separated.



-- 
john r pierce                            N 37, W 122
santa cruz ca                         mid-left coast

On Sun, 25 Sep 2011 21:51:51 +0300
Dotan Cohen wrote:
> Is there a way to get the most common (unique) lines of the file?
If you want what I think you want, a combination of cut and sort will do it.
>By the way, I'm not sure if this is RHEL or CentOS, or which version:
> I assume that it is one of these, as Yum is installed. How would I find
out?
cat /etc/redhat-release

-- 
MELVILLE THEATRE ~ Real D 3D Digital Cinema ~ www.melvilletheatre.com
www.creekfm.com - FIFTY THOUSAND WATTS of POW WOW POWER!