Home / Learn Linux / Lesson 9

Finding files and text: find, locate and grep

Find any file by name, size, age or owner, act on what you find, and search inside files.

Lesson

9 of 30

Time

About 50 minutes

On this page

1. Set up something to search
2. Where's that command? which and whereis
3. locate: fast, from an index
4. find: search the live filesystem
5. Doing something with what you find
6. grep: search inside files
7. Text inside binary files: strings
8. Common mistakes
9. Practice
10. Cheat sheet

Set up something to search

Make a small project with files of different types, sizes and ages. truncate -s makes a file of any size instantly, and touch -d (lesson 5) backdates it:

$ mkdir -p ~/find-lab/{src,docs,logs,tmp}
$ cd ~/find-lab
$ printf "def main():\n    print('hello')\n# TODO: add args\n" > src/app.py
$ printf "import os\n# todo: clean up\n" > src/util.py
$ printf "# Notes\nDeploy on Friday.\nTODO: write tests\n" > docs/README.md
$ printf "ERROR disk full\nINFO started\nerror: timeout\nINFO done\n" > logs/app.log
$ truncate -s 20M logs/old.log
$ truncate -s 5M tmp/cache.bin
$ touch -d "40 days ago" logs/old.log docs/README.md
$ mkdir src/Tests && touch src/Tests/test_app.py
$ tree
.
├── docs
│   └── README.md
├── logs
│   ├── app.log
│   └── old.log
├── src
│   ├── Tests
│   │   └── test_app.py
│   ├── app.py
│   └── util.py
└── tmp
    └── cache.bin

6 directories, 7 files

Run the examples from ~/find-lab unless they say otherwise.

Where's that command? which and whereis

You met which in lesson 2. whereis also finds a command's man pages:

$ whereis -b grep
grep: /usr/bin/grep
$ whereis -m grep
grep: /usr/share/man/man1/grep.1.gz /usr/share/info/grep.info.gz

-b shows only the program and -m only the manual pages. Both commands only look in the standard places for programs, so for anything else you need locate or find.

locate: fast, from an index

locate searches a database of every file name on the system, so it answers in milliseconds even on a huge disk. Ubuntu's modern version is called plocate, and it isn't installed by default:

$ sudo apt install -y plocate
Installing:
  plocate

...
$ locate sources.list
/etc/apt/sources.list
/etc/apt/sources.list.d
/etc/apt/sources.list.d/ubuntu.sources
/etc/cloud/templates/sources.list.ubuntu.deb822.tmpl
/usr/share/man/de/man5/sources.list.5.gz
/usr/share/man/fr/man5/sources.list.5.gz
/usr/share/man/man5/sources.list.5.gz
/usr/share/man/nl/man5/sources.list.5.gz
/usr/share/man/pt/man5/sources.list.5.gz
/var/lib/dpkg/info/python3-pkg-resources.list
/var/lib/ubuntu-advantage/apt-esm/etc/apt/sources.list
/var/lib/ubuntu-advantage/apt-esm/etc/apt/sources.list.d
/var/lib/ubuntu-advantage/apt-esm/etc/apt/sources.list.d/ubuntu-esm-apps.sources

Installing it builds the database for the first time. After that, a timer rebuilds it once a day. That's the catch: locate only knows what was there at the last rebuild. A file you created five minutes ago isn't in it until you run sudo updatedb:

$ touch brand-new.txt
$ locate brand-new.txt
$ echo $?
1
$ sudo updatedb
$ locate brand-new.txt
/home/ubuntu/find-lab/brand-new.txt

Useful options:

$ locate -c .conf
619
$ locate -i -l 3 readme
/etc/alternatives/README
/etc/apparmor.d/local/README
/etc/apparmor.d/rsyslog.d/README

Try it

1. How many paths on the system contain python3?
2. Find files named exactly os-release, matching only the name.

$ locate -c python3
15228
$ locate -b '\os-release'
/etc/os-release
/usr/lib/os-release
/usr/share/man/man5/os-release.5.gz

find: search the live filesystem

find walks the directories themselves, so it's slower than locate but always current, and it can search by much more than the name. Its shape is:

find  where  what-to-match  what-to-do

By name

$ find . -name "*.py"
./src/util.py
./src/Tests/test_app.py
./src/app.py
$ find . -iname "test*"
./src/Tests
./src/Tests/test_app.py

. means start here. -name matches the file name, and -iname ignores case, which is why Tests and test_app.py both matched. Always put quotes around patterns with *. Without them, Bash expands the * itself before find sees it, and you get strange errors or wrong results.

By type

-type f is regular files, -type d directories and -type l links:

$ find . -type d
.
./tmp
./src
./src/Tests
./docs
./logs

By size

-size takes + for bigger than and - for smaller than, with k, M or G:

$ find . -size +1M
./tmp/cache.bin
./logs/old.log
$ find . -type f -size -1k
./brand-new.txt
./src/Tests/test_app.py

By time

-mtime -7 means modified in the last 7 days, -mtime +30 more than 30 days ago. -mmin does the same in minutes. -newer file finds anything modified after that file:

$ find . -type f -mtime +30
./docs/README.md
./logs/old.log
$ find . -type f -mmin -60
./brand-new.txt
./tmp/cache.bin
./src/util.py
./src/Tests/test_app.py
./src/app.py
./logs/app.log

By owner and permissions

-user, -group and -perm match on who owns a file and its permissions (lesson 14). Searching the whole system means lots of directories you can't read, so throw away the errors with 2> /dev/null (lesson 8):

$ find /etc -user ubuntu 2> /dev/null
$ find /usr/bin -perm -4000
/usr/bin/chsh
/usr/bin/mount
/usr/bin/passwd
/usr/bin/su
...
$ find /var/log -maxdepth 1 -group adm -type f -name "*.log"
/var/log/kern.log
/var/log/cloud-init.log
/var/log/cloud-init-output.log
/var/log/apport.log
/var/log/auth.log

The first finds nothing, because ubuntu doesn't own anything in /etc. The second lists programs with the SUID bit set (lesson 15 explains why passwd needs it). -maxdepth 1 stops find going into subdirectories.

Combining tests

Several tests in a row must all match. -o means or, ! means not, and escaped parentheses \( \) group them:

$ find . -type f -name "*.py" ! -path "*Tests*"
./src/util.py
./src/app.py
$ find . -type f \( -name "*.md" -o -name "*.log" \)
./docs/README.md
./logs/old.log
./logs/app.log

Try it

1. Find every directory under /etc named ssh.
2. Find files in /var/log bigger than 100 KB, without any error messages.
3. Find files in ~/find-lab that are not .py files.

$ find /etc -type d -name ssh 2> /dev/null
/etc/ssh
$ find /var/log -type f -size +100k 2> /dev/null
/var/log/kern.log
/var/log/syslog
/var/log/lastlog
/var/log/cloud-init.log
/var/log/journal/8a186bd3d15b4c01a715344980aa3926/system.journal
/var/log/journal/8a186bd3d15b4c01a715344980aa3926/user-1000.journal
/var/log/dpkg.log
/var/log/auth.log
$ find . -type f ! -name "*.py"
./brand-new.txt
./tmp/cache.bin
./docs/README.md
./logs/old.log
./logs/app.log

Doing something with what you find

By default find prints each match. It can also act on them.

-ls shows details like ls -l:

$ find . -size +1M -ls
   534133      0 -rw-rw-r--   1 ubuntu   ubuntu    5242880 Oct 21 11:36 ./tmp/cache.bin
   534132      0 -rw-rw-r--   1 ubuntu   ubuntu   20971520 Sep 11 11:36 ./logs/old.log

-exec runs any command on each match. {} stands for the file found, and the command ends with \;. With + instead of \;, find passes many files to one run of the command, which is much faster:

$ find . -name "*.py" -exec wc -l {} \;
2 ./src/util.py
0 ./src/Tests/test_app.py
3 ./src/app.py
$ find . -name "*.py" -exec wc -l {} +
 2 ./src/util.py
 0 ./src/Tests/test_app.py
 3 ./src/app.py
 5 total

-ok is -exec that asks first, and -delete deletes every match. Run any find without -delete first and read the list, because -delete doesn't ask:

$ find tmp -name "*.bin"
tmp/cache.bin
$ find tmp -name "*.bin" -delete
$ ls tmp

That pattern, "find files older than N days and delete them", is how log and backup cleanup scripts work:

$ find /var/backups/myapp -name "*.tar.gz" -mtime +30 -delete

Try it

1. Show ls -l style details for every .log file in ~/find-lab.
2. Count the lines of every .md and .py file in one wc run.
3. Delete every empty file in ~/find-lab (-empty matches empty files), but look at the list first.

$ find . -name "*.log" -ls
   534132      0 -rw-rw-r--   1 ubuntu   ubuntu   20971520 Sep 11 11:36 ./logs/old.log
   534131      4 -rw-rw-r--   1 ubuntu   ubuntu         54 Oct 21 11:36 ./logs/app.log
$ find . -type f \( -name "*.md" -o -name "*.py" \) -exec wc -l {} +
  2 ./src/util.py
  0 ./src/Tests/test_app.py
  3 ./src/app.py
  3 ./docs/README.md
  8 total
$ find . -type f -empty
./brand-new.txt
./src/Tests/test_app.py
$ find . -type f -empty -delete

grep: search inside files

find looks at names and details. grep looks at what's inside, and prints every line that contains a pattern:

$ grep ERROR logs/app.log
ERROR disk full
$ grep -i error logs/app.log
ERROR disk full
error: timeout

These options cover almost everything:

$ grep -n -i todo src/*.py docs/README.md
src/app.py:3:# TODO: add args
src/util.py:2:# todo: clean up
docs/README.md:3:TODO: write tests
$ grep -c INFO logs/app.log
2
$ grep -v INFO logs/app.log
ERROR disk full
error: timeout
$ grep -rl -i todo .
./src/util.py
./src/app.py
./docs/README.md
$ grep -E "ERROR|timeout" logs/app.log
ERROR disk full
error: timeout

-r with -n is how you find where something is defined in a big codebase or config tree, and -l gives you just the file names to open:

$ grep -rn "PermitRootLogin" /etc/ssh 2> /dev/null
/etc/ssh/sshd_config:54:#PermitRootLogin prohibit-password
/etc/ssh/sshd_config:104:# the setting of "PermitRootLogin prohibit-password".

grep in pipes

grep reads from a pipe just as well as from a file, so it's the filter you'll put in the middle of everything:

$ ls /usr/bin | grep -c zip
13
$ cat /etc/passwd | grep -w ubuntu
ubuntu:x:1000:1000:Ubuntu:/home/ubuntu:/bin/bash
$ apt list --installed 2> /dev/null | grep -i "^openssh"
openssh-client/resolute-updates,resolute-security,now 1:10.2p1-2ubuntu3.8 arm64 [installed,automatic]
openssh-server/resolute-updates,resolute-security,now 1:10.2p1-2ubuntu3.8 arm64 [installed]
openssh-sftp-server/resolute-updates,resolute-security,now 1:10.2p1-2ubuntu3.8 arm64 [installed]

A first taste of patterns

grep patterns are regular expressions. Lesson 12 covers them properly, but three characters will already save you time: ^ means the start of a line, $ the end, and . any one character:

$ grep "^INFO" logs/app.log
INFO started
INFO done
$ grep "done$" logs/app.log
INFO done
$ grep -v "^#" /etc/hosts
127.0.0.1 localhost

::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
127.0.1.1 linux

grep -v "^#" hides comment lines, which is the quickest way to see what a long config file actually sets. Add | grep -v "^$" to hide the blank lines too.

Try it

1. Count the lines in /etc/passwd that end in nologin.
2. Show the ERROR line in logs/app.log with one line of context after it.
3. List every file under /etc/apt that mentions ubuntu.com.

$ grep -c "nologin$" /etc/passwd
27
$ grep -A 1 ERROR logs/app.log
ERROR disk full
INFO started
$ grep -rl ubuntu.com /etc/apt
/etc/apt/trusted.gpg.d/ubuntu-keyring-2018-archive.gpg
/etc/apt/trusted.gpg.d/ubuntu-keyring-2012-cdimage.gpg
/etc/apt/sources.list.d/ubuntu.sources

Text inside binary files: strings

Programs and data files are binary, so cat turns them into garbage. strings pulls out every run of readable text, which is a quick way to see what's in a file you don't recognise, or to find a version number or a hidden path inside a program:

$ strings /usr/bin/hostname
ha/lib/ld-linux-aarch64.so.1
__libc_start_main
__cxa_finalize
abort
__errno_location
...
$ strings /usr/bin/hostname | grep -i -m 3 usage
Usage: hostname [-b] {hostname|-F file}         set host name (from file)

grep -m 3 stops after three matches.

Common mistakes

• **Unquoted wildcards in find.** find . -name *.py lets Bash expand *.py first. Write "*.py".
• **-delete before checking.** Run the same find without it, read the list, then add it.
• **Thinking locate is broken.** It doesn't see new files until sudo updatedb.
• **Drowning in Permission denied.** Add 2> /dev/null, or run it with sudo if you need those directories too.
• **Forgetting -r.** grep word /etc fails with Is a directory. Use grep -r word /etc.
• **grep matching too much.** grep ubuntu also matches ubuntu-advantage. Use -w for whole words, or ^ and $ to anchor.

Practice

1. Find every .conf file directly inside /etc (not in subdirectories) that's bigger than 2 KB.

$ find /etc -maxdepth 1 -name "*.conf" -size +2k
/etc/debconf.conf
/etc/gai.conf
/etc/overlayroot.conf
/etc/adduser.conf
/etc/sensors3.conf
/etc/hdparm.conf
/etc/sudo.conf
/etc/ca-certificates.conf
/etc/sudo_logsrvd.conf

1. Find the files in /var/log modified in the last 10 minutes.

$ find /var/log -type f -mmin -10 2> /dev/null
/var/log/wtmp.db
/var/log/kern.log
/var/log/syslog
/var/log/lastlog
/var/log/sysstat/sa21
/var/log/journal/8a186bd3d15b4c01a715344980aa3926/system.journal
/var/log/journal/8a186bd3d15b4c01a715344980aa3926/user-1000.journal
/var/log/apt/history.log
/var/log/apt/eipp.log.xz
/var/log/apt/term.log
/var/log/alternatives.log
/var/log/dpkg.log
/var/log/wtmp
/var/log/auth.log

1. In ~/find-lab, list the files containing todo in any case, with how many matches each has.

$ grep -ric todo .
./src/util.py:1
./src/app.py:1
./docs/README.md:1
./logs/old.log:0
./logs/app.log:0

1. Show /etc/ssh/sshd_config without comments and blank lines.

$ grep -v "^#" /etc/ssh/sshd_config | grep -v "^$"
Include /etc/ssh/sshd_config.d/*.conf
KbdInteractiveAuthentication no
UsePAM yes
X11Forwarding yes
PrintMotd no
AcceptEnv LANG LC_* COLORTERM NO_COLOR
Subsystem       sftp    /usr/lib/openssh/sftp-server

1. Find every .py file in ~/find-lab and count the lines that contain import, in one command.

$ find . -name "*.py" -exec grep -c import {} +
./src/util.py:1
./src/app.py:0

1. Use locate to count how many .png files there are on the system, ignoring case.

$ locate -i -c "*.png"
51

1. Create old-backup.tar.gz in ~/find-lab dated 60 days ago, then find and delete every .tar.gz file there older than 30 days, checking the list first.

$ touch -d "60 days ago" old-backup.tar.gz
$ find . -name "*.tar.gz" -mtime +30
./old-backup.tar.gz
$ find . -name "*.tar.gz" -mtime +30 -delete
$ ls *.tar.gz
ls: cannot access '*.tar.gz': No such file or directory

Cheat sheet

← Pipes and redirectionEditing files with Vim →