Find -exec grep usage

We use “find” command on a daily basis. But, most of the time, it’s just for simple search like
find / -name

There are more useful stuffs that you can do with “find” command if you combine with the regular Linux ‘grep’ command.
This lets you search for text strings and regular expressions in multiple directories in a single shot.

Grep command in it’s simple usage is as follows:

grep ‘apache’ * – will search for the pattern ‘apache’ in all the files in the current directory. But, if you combine the find with grep command, you can do lot more things, like searching for a pattern in multiple directories. For eg:

find . -type f -exec grep -il "^sample$" {} \;

Here “.” means in the current directory and all it’s subdirectories
“-type f” means to search in files
“-exec” lets you execute a command, in this case the “grep” command
“-i” means case insensitive search
“-l” lists the filenames containing the pattern “sample”
“^” Begins with the character following.
“$” Ends with the character preceding.
“{} \;” means that you’re about to feed the grep command a lot of files.

Another eg: find htdocs cgi-bin -name “*.cgi” -type f -exec chmod 755 {} \;

The above command searches through the “htdocs” and “cgi-bin” directories for files that end with the extension “.cgi”. When these files are found, their permission is changed to mode 755 (rwxr-xr-x).

find . -type f \( -name “*.c” -o -name “*.sh” \)

The above command searches for multiple files with extension .c or .sh. Keep adding -o for more specific searches.

find . -mtime -5 -type f

The above command finds all files that has been modified in the last 5 days. You can search for directories by using the flag “-type d”. Omitting “-type f/d” will search for both files and directories modified in last 5 days.

find . -size +100k -a -size -500k

The above command searches files with size between 100 kilobytes and 500 kilobytes

find /home/soj/ -mtime -2 -exec ls {} \;

The above command lists all the files under /home/soj/ that has been modified within last 2 days (note the option -2)

find /home/soj/ -mtime +200 -exec ls {} \;

The above command lists all the files under /home/soj/ that are older than 200 days (note the option +200)

find /media -name ‘*.mp3’ -size -5000k

The above command finds files with extension ‘mp3’ that are less than 5MB (5000 kilobytes) under the directory ‘media’. If you want to search for files greater than 5MB, use -size +5000k in the above command

find . -name “*.txt” -exec -ok cp {} test \;

The above ‘find’ command finds files with ‘txt’ extension by substituting a file name for the brackets, and then asked for confirmation before copying the file to the ‘test’ directory

find ~soj -perm -644

The above command will match all files that have, at a minimum, the rw permission set for user AND r permission for group AND r permission set for others.

find ~soj -perm 644

The above command will match all files that exactly have the rw permission set for user AND r permission for group AND r permission set for others.

find * -mtime +100 exec rm {} \; (Try this command on YOUR OWN RISK)

The above command is DANGEROUS. Here the find command will search for the files that are older than 100 days, as mentioned in the argument for ‘mtime’. This way you can set the time to any number of days and delete files older than the time frame. But, you don’t want to do this as you might NOT KNOW what all it might delete..

What if you want to search for a particular file, say config.cfg, under your current directory and sub directories and then replace a string ‘old’ with another one ‘new’ on all the files with file name config.cfg

find . -type f -name config.cfg -exec sed -i "s/old/new/ig" {} +;
Advertisements

List Open Files and Sockets (lsof)

To check on the open ports on your local system: (You can change localhost to server_name in case you want to check remote servers for open ports)

[soj@centos perl]$ sudo nmap -sTU localhost

PORT STATE SERVICE
22/tcp open ssh
25/tcp open smtp
53/tcp open domain
80/tcp open http

To list open files and sockets, you can use “lsof” command.

sudo “lsof -i” lists all open Internet files/sockets

From the above nmap output, you can check the command for each services as follows:

[soj@centos perl]$ sudo lsof -i :ssh
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sshd 1579 root 3u IPv4 10986 0t0 TCP *:ssh (LISTEN)
sshd 1579 root 4u IPv6 10988 0t0 TCP *:ssh (LISTEN)

[soj@centos perl]$ sudo lsof -i :smtp
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
sendmail 26450 root 4u IPv4 38120 0t0 TCP localhost.localdomain:smtp (LISTEN)

[soj@centos perl]$ sudo lsof -i :domain
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
named 1310 named 20u IPv4 9857 0t0 TCP localhost.localdomain:domain (LISTEN)
named 1310 named 24u IPv4 38186 0t0 TCP centos.sandbox:domain (LISTEN)
named 1310 named 512u IPv4 9856 0t0 UDP localhost.localdomain:domain
named 1310 named 513u IPv4 38185 0t0 UDP centos.sandbox:domain

[soj@centos perl]$ sudo lsof -i :http
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
httpd 1627 root 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20083 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20084 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)

You can use lsof command with port number as well-

[soj@centos ~]$ sudo lsof -i :80
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
httpd 1627 root 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20083 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20084 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)

———————-
For finding files that have a link count less than 1 (ie. the file was removed, but the process keeps on writing)

[soj@centos ~]$ sudo lsof +L 1

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NLINK NODE NAME
nautilus 1989 soj 19r REG 253,0 5752 0 393090 /home/soj/.local/share/gvfs-metadata/home (deleted)
nautilus 1989 soj 20r REG 253,0 32768 0 399003 /home/soj/.local/share/gvfs-metadata/home-703674be.log (deleted)
gnome-ter 2072 soj 22u REG 253,0 4721 0 261785 /tmp/vteIRXN5V (deleted)
gnome-ter 2072 soj 23u REG 253,0 4304 0 261786 /tmp/vteLJXN5V (deleted)
gnome-ter 2072 soj 24u REG 253,0 0 0 261810 /tmp/vteEGXN5V (deleted)
gnome-ter 2072 soj 25u REG 253,0 20310 0 261823 /tmp/vteLO8R5V (deleted)
gnome-ter 2072 soj 26u REG 253,0 8192 0 261824 /tmp/vteQG8R5V (deleted)
———————–
To list all open files in /home/soj

[soj@centos ~]$ sudo lsof +d /home/soj/

(with option +D, you get to see all the open files within the directory recursively)

su 26894 root cwd DIR 253,0 4096 392814 /home/soj
bash 26899 root cwd DIR 253,0 4096 392814 /home/soj
gedit 27244 soj cwd DIR 253,0 4096 392814 /home/soj
vi 27507 root cwd DIR 253,0 4096 392814 /home/soj
lsof 27539 root cwd DIR 253,0 4096 392814 /home/soj
lsof 27540 root cwd DIR 253,0 4096 392814 /home/soj

The above command is similar to

[soj@centos ~]$ sudo fuser -v /home/soj/
USER PID ACCESS COMMAND
/home/soj/: soj 1084 ..c.. bash
root 26894 ..c.. su
root 26899 ..c.. bash
soj 27244 ..c.. gedit
root 27507 ..c.. vi

{Try a better command: fuser -m -v /home/soj/}

To check on the PID (27507)
[soj@centos ~]$ sudo lsof -p 27507
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
vi 27507 root cwd DIR 253,0 4096 392814 /home/soj
vi 27507 root rtd DIR 253,0 4096 2 /
vi 27507 root txt REG 253,0 765880 261727 /bin/vi
vi 27507 root mem REG 253,0 61624 392478 /lib64/libnss_files-2.12.so
vi 27507 root mem REG 253,0 99158752 670862 /usr/lib/locale/locale-archive
vi 27507 root mem REG 253,0 17896 392537 /lib64/libattr.so.1.1.0
vi 27507 root mem REG 253,0 19536 392468 /lib64/libdl-2.12.so
vi 27507 root mem REG 253,0 135896 392505 /lib64/libtinfo.so.5.7
vi 27507 root mem REG 253,0 1832712 392462 /lib64/libc-2.12.so
vi 27507 root mem REG 253,0 31856 392539 /lib64/libacl.so.1.1.0
vi 27507 root mem REG 253,0 140096 392501 /lib64/libncurses.so.5.7
vi 27507 root mem REG 253,0 122008 392523 /lib64/libselinux.so.1
vi 27507 root mem REG 253,0 595816 392470 /lib64/libm-2.12.so
vi 27507 root mem REG 253,0 148504 392980 /lib64/ld-2.12.so
vi 27507 root 0u CHR 136,1 0t0 4 /dev/pts/1
vi 27507 root 1u CHR 136,1 0t0 4 /dev/pts/1
vi 27507 root 2u CHR 136,1 0t0 4 /dev/pts/1
vi 27507 root 4u REG 253,0 12288 399277 /home/soj/scripts/bash/.correctfile.txt.swp
————————
Run the following command to list all the open ports, even those opened by backdoors/trojans/rootkits that are hidden to netstat and ps commands

[soj@centos ~]$ sudo lsof | grep -i “listen”
rpcbind 1215 rpc 8u IPv4 9378 0t0 TCP *:sunrpc (LISTEN)
rpcbind 1215 rpc 11u IPv6 9383 0t0 TCP *:sunrpc (LISTEN)
named 1310 named 20u IPv4 9857 0t0 TCP localhost.localdomain:domain (LISTEN)
named 1310 named 21u IPv4 9860 0t0 TCP localhost.localdomain:rndc (LISTEN)
named 1310 named 22u IPv6 9861 0t0 TCP centos.sandbox:rndc (LISTEN)
named 1310 named 24u IPv4 38186 0t0 TCP centos.sandbox:domain (LISTEN)
rpc.statd 1333 rpcuser 9u IPv4 10041 0t0 TCP *:52970 (LISTEN)
rpc.statd 1333 rpcuser 11u IPv6 10049 0t0 TCP *:40908 (LISTEN)
sshd 1579 root 3u IPv4 10986 0t0 TCP *:ssh (LISTEN)
sshd 1579 root 4u IPv6 10988 0t0 TCP *:ssh (LISTEN)
httpd 1627 root 4u IPv6 11190 0t0 TCP *:http (LISTEN)
smbd 1653 root 24u IPv6 11408 0t0 TCP *:microsoft-ds (LISTEN)
smbd 1653 root 25u IPv6 11410 0t0 TCP *:netbios-ssn (LISTEN)
miniserv. 1685 root 6u IPv4 11470 0t0 TCP *:ndmp (LISTEN)
httpd 20083 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20084 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20085 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20086 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20088 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20089 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20090 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
httpd 20091 apache 4u IPv6 11190 0t0 TCP *:http (LISTEN)
sendmail 26450 root 4u IPv4 38120 0t0 TCP localhost.localdomain:smtp (LISTEN)
mysqld 27150 mysql 10u IPv4 43916 0t0 TCP *:mysql (LISTEN)

Mount USB drive on Linux

Optical drives are soon going to be history just like our good old floppy disk drives. Laptops in future won’t have DVD or CD drives. The advantage is you will have a light weight laptop; and with all the multimedia that you download, you need a higher capacity storage. The solution is high capacity USB drive or Tera byte external hard disk drives that are all connected to your USB drives. In Windows, the USB gets detected automatically and even some *nix distros provide this kinda feature. In case your USB drive is not auto detected by your *nix distro, just like the Centos 6 that I installed couple of weeks back, then you will have to scan and mount the USB drive manually as follows:

Plug in your USB drive into your USB port. Ubuntu automatically detects, scans and mounts just like Windows. But, in Centos, nothing happened even after plugging in my USB drive. So, you have to now check if the USB is detected by your Linux OS. Issue the following command to check if USB drive is detected:

[soj@linuxgenius ~]$ cat /proc/scsi/scsi 
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
Vendor: ATA Model: ST9100822A Rev: 3.02
Type: Direct-Access ANSI SCSI revision: 05
Host: scsi0 Channel: 00 Id: 01 Lun: 00
Vendor: TSSTcorp Model: CD/DVDW TS-L532M Rev: HR04
Type: CD-ROM ANSI SCSI revision: 05
Host: scsi2 Channel: 00 Id: 00 Lun: 00
Vendor: Seagate Model: FreeAgent Rev: 0138
Type: Direct-Access ANSI SCSI revision: 02

From the above info, you can see the one I marked in bold. My external hard disk is detected by the Kernel.

In case, the USB drive is not detected, then scan manually and the USB should get detected:
Issue the following command to manually scan the disk drive:

rescan-scsi-bus -l

Once the USB drive is detected, then you have to check what drive your thumb drive or external hard drive is. From the command line, type the following:

[soj@linuxgenius ~]$ dmesg | grep sd
sd 0:0:0:0: [sda] 195371568 512-byte logical blocks: (100 GB/93.1 GiB)
sd 0:0:0:0: [sda] Write Protect is off
sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn’t support DPO or FUA
sda: sda1 sda2 sda3
sd 0:0:0:0: [sda] Attached SCSI disk
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
sdhci-pci 0000:06:06.4: SDHCI controller found [104c:8034] (rev 0)
sdhci-pci 0000:06:06.4: PCI INT A -> GSI 22 (level, low) -> IRQ 22
dracut: Scanning devices sda3 for LVM logical volumes VolGroup/lv_root VolGroup/lv_swap
sd 0:0:0:0: Attached scsi generic sg0 type 0
EXT4-fs (sda2): mounted filesystem with ordered data mode
sd 2:0:0:0: Attached scsi generic sg2 type 0
sd 2:0:0:0: [sdb] 1953525166 512-byte logical blocks: (1.00 TB/931 GiB)
sd 2:0:0:0: [sdb] Write Protect is off
sd 2:0:0:0: [sdb] Mode Sense: 1c 00 00 00
sd 2:0:0:0: [sdb] Assuming drive cache: write through
sd 2:0:0:0: [sdb] Assuming drive cache: write through
sdb: sdb1

sd 2:0:0:0: [sdb] Assuming drive cache: write through
sd 2:0:0:0: [sdb] Attached SCSI disk

From the above, you can identify that sdb1 is the drive where the thumb drive is. So, now you just need to mount it as follows:

[root@linuxgenius /]# mkdir /usb
[root@linuxgenius /]# mount /dev/sdb1 /usb
[root@linuxgenius /]# mount (check the mount result)

That’s it.

Just an additional info on mounting a disk image:

mount -o loop disk1.iso /mnt/disk

Notify-send and Cron jobs

In one of my previous post, I have mentioned that you can use notify-send command to trigger popup messages on your GUI terminal. This will work fine when you issue the following command at the terminal window as follows:
notify-send "`uptime`"

But, this command will only be useful if you use it in a shell script to alert you on your Gnome desktop whenever an event triggers. If you just schedule it the normal way in crontab, the message won’t pop up on your screen.. So, you need to check your display using the command as follows:

echo $DISPLAY
:0.0 (This is my Display)

Then, you have to use this DISPLAY information in your crontab as follows:
*/3 * * * * DISPLAY=:0.0 /home/soj/scripts/bash/url_monitor.sh 2>&1

I have tried this on Centos and other *nix distributions and it works like a charm.

Netstat to check on DOS attack

I use the following command to check on the number of connections from IP’s at port :80

netstat -ant | awk ‘$4 ~ /:80$/’ | awk ‘{print $5}’ | awk -F: ‘{print $1}’ | sort | uniq -c

Also, the following code is used to check the various state of tcp connections and the number of connections associated with each state:

netstat -ant | awk ‘{print $6}’| sort | uniq -c | sort -n

The following iptables rule prevents the DOS attack on your webserver by limiting maximum 30 connection per minute. This limit/minute will be enforced only after the total number of connection have reached the limit-burst level

iptables -A INPUT -p tcp –dport 80 -m limit –limit 30/minute –limit-burst 100 -j ACCEPT

Some examples on egrep and awk

In case you want to know the service listening to any port#, say port 139

egrep ‘\<139/tcp\>’ /etc/services

You get the same output as above using awk as follows:

awk ‘$2 ~ /^139\/tcp/ {print $1,$2}’ /etc/services

Though egrep is easier one, I somehow love using awk for text manupulations..

Boot windows from grub prompt

Recently, I screwed up my Linux partition on my old laptop and when rebooted, I was stuck at grub prompt. Since it’s a dual boot, I knew I could somehow boot into Windows and save atleast my data on my Windows partition.

This is how I did… In my case, I had my Windows partition on /dev/hda (hd0)

grub> rootnoverify (hd0,0)
grub> makeactive
grub> chainloader +1
grub> boot

Once I logged in my Windows OS, I fixed my Master Boot Record, so I don’t have to issue the above commands to boot Windows OS. I used fixmbr tool to fix my MBR since my DVD drive wasn’t working and it wasn’t possible for me to go into restore mode by using Windows DVD and fix my MBR manually.

Finally, I wanted to have my linux back. Since my DVD drive wasn’t working, I setup a TFTPD server on my a linux OS which was on my vmware workstation on a remote laptop and configured DHCP so the screwed up laptop gets IP dynamically when booting. I installed Centos 6 on it and works like a charm.. May be I’ll blog later how TFTPD and DHCP was configured to remotely install Centos 6 on my laptop..