T
tomfmason
Guest
These assume you use a standard log format. If not you may have to adjust them to suit your format
Most viewed pages(top ten).
Top ten referrers:
Search logs:
And here is a script I wrote a long time ago that incorparates these and few others
I wrote that a long time ago and now I would not suggest using hosts.deny to ban an ip. Iptables would be a much better choice. I am just to lazy to fix it
Most viewed pages(top ten).
Code:
awk '{print $7}' /path/to/log |sort |uniq -c |sort -rn |head -10
Top ten referrers:
Code:
awk '{print $11}' /path/to/log |sort |uniq -c |sort -rn |head -10
Search logs:
Code:
grep /path/to/log query |awk '{print $8}'|tail -n+5|sort|uniq|tr -d [1]
And here is a script I wrote a long time ago that incorparates these and few others
Code:
#!/bin/bash
# usage
# ./this_script search pattern log_file
# all hosts with 5 or more matches of the given pattern will be banned
# ./this_script ban_from_log log_file
# all hosts that appear more than 5 times in the given log file will be banned
# ./this_script ban_rfi log_file
# bans all hosts that match the rfi pattern(rfi's and proxy requests)
# ./this_script most_viewed log_file
# shows the top ten viewed pages with the number of views
# ./this_script statuses response log_file
# shows the top ten viewed pages for the given response header e.g. 404, 200 ect
# ./this_script referrers log_file
# shows the top ten referrers and page views for each
#
# author tomfmason
ban_file=/etc/hosts.deny
function ban_ip() {
exists=`grep ${1} $ban_file`
if [ ! "$exists" ]; then
echo "ALL: ${1}" >> $ban_file
fi
}
function search() {
ret=`grep ${1} ${2} |awk '{print $8}'|tail -n+5|sort|uniq|tr -d [1]`
for r in $ret; do
ban_ip $r
done
}
function ban_from_log() {
ret=`awk '{print $8}' ${1}|tail -n+5|sort|uniq|tr -d [1]`
for r in $ret; do
ban_ip $r
done
}
function ban_rfi() {
ret=`awk '{print $1 " " $7}' ${1} |grep -iE '(http|https|ftp)'|awk '{print $1}'|tail -n+5|sort|uniq`
for r in $ret; do
ban_ip $r
done
}
function most_viewed() {
awk '{print $7}' ${1} |sort |uniq -c |sort -rn |head -10
}
function statuses() {
awk '{print $7 " " $9}' ${2} |grep -iE '${1}' |sort | uniq -c | sort -rn |awk '{print $1 " " $2 }' |head -10
}
function referrers() {
awk '{print $11}' ${1} |sort |uniq -c |sort -rn |head -10
}
if type "$1" | grep -qF "$1 is a function"; then "$@"; fi
I wrote that a long time ago and now I would not suggest using hosts.deny to ban an ip. Iptables would be a much better choice. I am just to lazy to fix it
