Post

Linux pt 2: Commands (90 Days of DevOps)

Posted
20 min read
Linux pt 2: Commands (90 Days of DevOps)

Linux Commands

We use our new Vagrant SSH session, established in the previous post, to work with some common Linux commands:

man

man is the command to use regularly if you want to become proficient with the Linux command line. It is often used by experienced Linux users to quickly reference a particular command’s many available options.

The most basic syntax is simply man <command_name>, where <command_name> is the actual name of a Linux command. That command can even be man itself, so we can use man to bring up its own manual:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40

$ man man
MAN(1)                        Manual pager utils                        MAN(1)

NAME
       man - an interface to the system reference manuals

SYNOPSIS
       man [man options] [[section] page ...] ...
       man -k [apropos options] regexp ...
       man -K [man options] [section] term ...
       man -f [whatis options] page ...
       man -l [man options] file ...
       man -w|-W [man options] page ...

DESCRIPTION
       man  is  the system's manual pager.  Each page argument given to man is
       normally the name of a program, utility or function.  The  manual  page
       associated with each of these arguments is then found and displayed.  A
       section, if provided, will direct man to look only in that  section  of
       the  manual.   The  default action is to search in all of the available
       sections following a pre-defined order (see DEFAULTS), and to show only
       the first page found, even if page exists in several sections.


       The table below shows the section numbers of the manual followed  by  the  types  of
       pages they contain.

       1   Executable programs or shell commands
       2   System calls (functions provided by the kernel)
       3   Library calls (functions within program libraries)
       4   Special files (usually found in /dev)
       5   File formats and conventions, e.g. /etc/passwd
       6   Games
       7   Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7),
           man-pages(7)
       8   System administration commands (usually only for root)
       9   Kernel routines [Non standard]

...

As we can see, the man command outputs a lot of detail. As mentioned in the output above, the system reference manual is divided into many categorized, numbered sections.

whatis

For times when we simply want a one-line description of what a specific Linux command is for, there is the whatis command. Its most basic syntax is whatis <command_name>, where <command_name> is the name of the command we want info about.

Below are some examples:

  • Example 1:

    1
2

    $ whatis man
man (1)              - an interface to the system reference manuals

    Notice the number (1) in this whatis command’s output. It tells us that the manual page being summarized is part of section 1 of the system manual, which is where the manual pages for general-purpose executables and shell commands are grouped.

    Often when we look up man pages, they will either be part of this section, or section 8, which is where the pages for administrative commands are grouped.

Similarly to man, we can use the whatis command to get information about itself:

  • Example 2:

    1
2

    $ whatis whatis
whatis (1)           - display one-line manual page descriptions

Below, we look at a number of common Linux commands.

sudo

What is it?

1
2

$ whatis sudo
sudo (8)             - execute a command as another user

The most common use of sudo is to allow a trusted user to run a Linux command that requires administrative access to the system.

  • Example: 
    1

    $ sudo poweroff

To get a general idea of each of the remaining commands on our list, we start by running whatis for each command as we’ve just done above.

pwd

1
2

$ whatis pwd
pwd (1)              - print name of current/working directory

This command is straightforward. It outputs the full path of the current working directory.

  • Example: 
    1
2

    $ pwd
/home/vagrant

    As shown, we are currently working in the home directory of a user named vagrant.

mkdir

1
2

$ whatis mkdir
mkdir (1)            - make directories
  • Example: 
    1
2
3

    $ mkdir directory_1
$ mkdir directory_2
$ mkdir directory_3

    These three commands create new directories named directory_1, directory_2, and directory_3 inside the current working directory.

touch

1
2

$ whatis touch
touch (1)            - change file timestamps

The official whatis summary for touch says that it’s for changing file timestamps.

However touch is very often used at the command line to simply create a new, blank file from scratch:

  • Example: 
    1
2
3

    $ touch file1
$ touch file2
$ touch file3

    These three commands create new files named file1, file2, and file3 inside the current working directory. Note that Linux does not specifically require a file’s name to end with any kind of file extension (e.g. “file1.txt”).

ls

1
2

$ whatis ls
ls (1)               - list directory contents
  • Example: 
    1
2

    $ ls
directory_1  directory_2  directory_3  file1  file2  file3

    The command lists for us the contents of the current working directory, which include all of the new (sub)directories and files we’ve created.

cd

1
2

$ whatis cd
cd: nothing appropriate.

What happened with this whatis query? On our Linux VM, there is no manual page for cd .

That’s because cd is a “builtin” shell command, so it does not technically count as a Linux command. Luckily, for builtin commands we can still get info by running the help command, e.g. help cd.

That command provides the equivalent of a man page for the built-in command cd (which is a lot of text, so it won’t be included here in this blog post).

  • Example 1: 
    1

    $ cd directory_1

    This changes the current working directory from /home/vagrant to /home/vagrant/directory_1. After changing directories, we could confirm the change by running pwd again, if we wanted.

    1
2

    $ pwd
/home/vagrant/directory_1
  • Example 2: 
    1

    $ cd ..

    As in other OSes, “..” is shorthand for the directory containing the current working directory. Running the example command above brings us back from a working directory of /home/vagrant/directory_1 to the original directory of /home/vagrant.

rmdir

1
2

$ whatis rmdir
rmdir (1)            - remove empty directories

Note that this description says “remove empty directories.” The rmdir command is specifically designed to delete directories that are empty, and will fail if run against a directory containing files or subdirectories.

  • Example: 
    1

    $ rmdir directory_1

    This command removes the empty directory named directory_1 that we created earlier. The command completes successfully without any kind of confirmation.

    Optionally, we could choose to run ls again, noticing that the deleted directory’s name no longer appears in the command’s output.

    1
2

    $ ls
directory_2  directory_3  file1  file2  file3

mv

1
2

$ whatis mv
mv (1)               - move (rename) files

It might be a little counter-intuitive that mv can also be used for renaming things, but that is how it works.

  • Example 1: 
    1

    $ mv file2 directory_2

    This moves the file named file2 into the directory named directory_2.

  • Example 2: 
    1

    mv file1 renamed_file1

    This renames the file named file1, giving it a new name of renamed_file1.

cp

1
2

$ whatis cp
cp (1)               - copy files and directories
  • Example 1: 
    1

    $ cp file3 copy_of_file3

    This makes a copy of the file named file3. That copy is named copy_of_file3.

  • Example 2: 
    1

    $ cp -r directory_2 directory_3

    This makes a copy of directory_2 and all of its contents inside directory_3. The -r option (for “recursive”) is what ensures that all of directory_2 ‘s contents are included in the new copy.

rm

1
2

$ whatis rm
rm (1)               - remove files or directories
  • Example 1: 
    1

    $ rm copy_of_file3

    This removes the file named copy_of_file3.

  • Example 2: 
    1

    $ rm -r directory_3

    This removes removes the directory named directory_3 and all of its contents. As with the cp command, the -r “recursive” option specifies that all of a directory’s contents are handled by the command.

    If we were to run ls again now, the file and directory we removed would no longer be listed. They have been “deleted” (not technically, but that’s a topic for another day).

echo

1
2

$ whatis echo
echo (1)             - display a line of text

By itself, echo simply outputs some specified text to the terminal.

  • Example 1: 
    1
2

    $ echo "Hello world!"
Hello world!

In a more useful example, we write the echo command’s output to a file using the > special character:

  • Example 2: 
    1

    $ echo "This is the text for file3." > file3

    This time the echo command’s output text is redirected into a file. As a result, the command completes silently with no visible output.

  • Example 3: 
    1

    $ echo "This is another line of text for file3." >> file3

    Using >> instead of > is important when we want to append a command’s output to a file that already contains data. If we were to accidentally use > when we meant to use >>, Linux would happily overwrite any existing data in the destination file with the new output!

cat

1
2

$ whatis cat
cat (1)              - concatenate files and print on the standard output

This description can be somewhat misleading. Rather than being used to concatenate (join) output from multiple files, cat is probably most often used to quickly access a single text file’s contents.

  • Example 1: 
    1
2
3

    $ cat file3
This is the text for file3.
This is another line of text for file3.

    We see that cat outputs the text contents of file3 to the terminal.

  • Example 2: 
    1

    $ cat file3 > file4

    This time, instead of displaying the contents of file3, we redirect that text into a new file named file4. If the target file does not exist, the command takes care of creating the new file automatically so that the text can be written.

Again, > will overwrite the target file’s existing file contents, so it is important to use >> whenever the intention is to append to the existing file’s contents instead.

clear

1
2

$ whatis clear
clear (1)            - clear the terminal screen

This command is especially useful after running a command whose detailed output clutters our terminal with text. Without running clear in between commands, it can sometimes become hard to tell where one command’s output ends and the next command’s output begins.

locate

1
2

$ whatis locate
locate (1)           - find files by name, quickly

Note: locate isn’t installed by default on the Debian VM used in this post, but the output above is what appears once it is installed.

If installed, the locate command can be used when trying to find files on a Linux system.

Instead of searching through files and directories one by one, locate finds things indirectly by using an index which must be kept up to date.

  • Example: 
    1

    $ sudo updatedb

    First, we run updatedb, which builds/updates the index that the locate command will use. Notice that the command is run using sudo, since the updatedb command requires administrative permissions to this Linux machine.

    The command completes silently. Next, we run a search.

    1

    $ locate hosts

    The command returns all of the previously-indexed files containing “hosts” in their name.

    1
2
3
4
5

    /etc/hosts
/etc/hosts.allow
/etc/hosts.deny
...

When in doubt, the find command is a reliable tool for finding files, and is more often available on any given Linux system than locate . However, find can take a while to run if a lot of files/directories are being searched, since the OS needs to look at every file in the search path, one by one.

history

1
2

$ whatis history
history (3readline)  - GNU History Library

The above is maybe one of the less helpful whatis entries for our purposes here.

The history command keeps a log of the commands the current user has entered at the terminal.

  • Example: 
    1
2
3
4
5
6
7
8
9
10
11
12

    $ history
...
20  echo "This is the text for file3." > file3
21  echo "This is another line of text for file3." >> file3
22  cat file3
23  cat file3 > file4
24  clear
25  whatis locate
26  sudo updatedb
27  locate hosts
28  whatis history
29  history

As shown above, the commands that we’ve entered were recorded one by one in a numbered format. These line numbers can be used to quickly re-run a long command from the history without having to retype it, which can really come in handy with more complex commands.

This is done by entering the ! special character, followed immediately by the history line number of the command you wish to re-run (and hitting enter).

For example, assuming the command history shown above, entering the command !21 would be the same thing as retyping the entire command echo "This is another line of text for file3." >> file3 , and running that.

There are also options available for adding time/date stamps to each entry in the history, changing how much history is recorded, and resetting the history list (history -c).

User / Group management

useradd

1
2

$ whatis useradd
useradd (8)          - create a new user or update default new user information

We will run useradd to create a new user on our Linux system. As we can tell from the (8) above, the command is an administrative command, so we will use sudo when running it to ensure the command is run with administrative permissions.

  • Example: 
    1

    $ sudo useradd NewUser

    The command completes silently, creating NewUser.

passwd

1
2

$ whatis passwd
passwd (1)           - change user password

When run with no arguments, passwd is used to change the password of the user account that was used to run the command.

Note that this will not work as our current user (vagrant). That’s because password authentication for our user is disabled for security reasons (earlier, we logged in automatically using an SSH key rather than a password).

We can change the password just fine for NewUser, though. Since we are changing another user’s password, we need to run the command with administrative permissions or it won’t work:

  • Example: 
    1
2

    $ passwd NewUser
passwd: You may not view or modify password information for NewUser.

    1
2
3
4

    $ sudo passwd NewUser
New password: 
Retype new password: 
passwd: password updated successfully

groupadd

1
2

$ whatis groupadd
groupadd (8)         - create a new group

User groups in Linux are useful for organizing them logically, as well as for managing system permissions for sets of users.

Just like useradd, groupadd is an administrative command. We run the command with sudo so that our user temporarily gets the required administrative permissions.

  • Example: 
    1

    $ sudo groupadd DevOps

    The command completes silently, creating a new group named DevOps on our system.

usermod

1
2

$ whatis usermod
usermod (8)          - modify a user account

usermod is commonly used to manage a user account’s group membership. It is an administrative command, so again we use sudo .

  • Example: 
    1

    $ sudo usermod NewUser -a -G DevOps

    If we check the system manual (man usermod), we see that the -a option is for “append” and the -G option is for “group.”

    That means the example command shown above appends the DevOps group to the current list of groups that NewUser is a member of.

    The command completes silently, but we can always confirm all of the groups a user is a member of with a groups command like the one below:

    1
2

    $ groups NewUser
NewUser : NewUser DevOps

    We see that the NewUser account is now a member of two groups: its default group (NewUser) and the DevOps group we just created.

Permissions / file ownership

First of all, when we are interested in viewing file/directory permissions, we add the -l option when running ls . As the man page for ls will tell us, -l specifies that we want to list directory contents in “long” format.

Example:

1
2
3
4
5

$ ls -l
drwxr-xr-x 2 vagrant vagrant 4096 Aug 21 18:13 directory_2
-rw-r--r-- 1 vagrant vagrant   68 Aug 21 18:15 file3
-rw-r--r-- 1 vagrant vagrant   68 Aug 21 18:15 file4
-rw-r--r-- 1 vagrant vagrant    0 Aug 21 18:12 renamed_file1

This longer format prints a dedicated line for each file or directory being listed, allowing for much more detail. Instead of just file/directory names, we also see a set of permissions on the left of each entry. There’s also other useful info for each entry, including the file/directory’s owner name, group name, size, and timestamp.

The permissions are shown as a sequence of letters and/or dashes, representing the following information:

  • 1st character: File type (d for directory or - for file)

  • Remaining nine characters: Basic permissions assigned to the file/directory:
    • r for read permission
    • w for write permission
    • x for execute permission
    • - for no permission
  • These nine characters are used to describe three sets of permissions:
    • 1st - 3rd characters: Read, Write, Execute permissions granted to the User who owns this file/directory
    • 4th - 6th characters: Read, Write, Execute permissions granted to members of the Group that owns this file/directory
    • 7th - 9th characters: Read, Write, Execute permissions granted on this file/directory to any Other users not covered by the previous two permission sets
For example, in the ls -l command output from earlier, we can see from the first line that:
  • directory_2 is a directory rather than a file (d)
  • The vagrant User has full permissions to it (rwx)
  • Members of a Group named vagrant have read and execute permissions on it but no write permissions (r-x)
  • Other users not covered by the previous two permission sets are allowed read and execute permissions but not write permissions (r-x) on this directory.

Read, write, and execute permissions mean slightly different things when applied to files vs directories. Most notably, execute permission on a file allows for executing/running that file as a program/script. On the other hand, execute permission on a directory allows for accessing that directory’s contents. This article covers the differences in detail.

The File Mode

In order to understand the commands used to modify file permissions, it is helpful to understand the idea of a file mode, which involves octal values. Octal values are a numeric notation for the permissions we are shown when we run ls -l . For example, instead of rwxr-xr-x , the file mode for directory_2 is 755 .

The relationship between Linux file permissions and the corresponding file modes is:

  • A basic file mode is made up of three digits. The first represents User owner permissions, the second represents Group owner permissions, and the third represents permission for Other users.

  • For each digit of the file mode:

    • r permissions add 4 to the digit
    • w permissions add 2 to the digit
    • x permissions add 1 to the digit
    • - permissions (no permission) adds 0 to the digit
So in the earlier example of directory_2 :
  • The User owner permissions of rwx becomes 7 (4 + 2 + 1)
  • The Group owner permissions of r-x becomes 5 (4 + 0 + 1)
  • The Other permissions of r-x becomes 5 (4 + 0 + 1)
The result is a file mode of 755 for directory_2 .
With an understanding of file modes, we can start editing file permissions using the chmod command.

chmod

1
2

$ whatis chmod
chmod (1)            - change file mode bits

We can use chmod to update the permissions set on directory_2, by specifying the new file mode we want.

For example, if only the directory’s User and Group owners should have access to it (and nobody else should have any access), then we can run:

  • Example: 
    1

    $ chmod 750 directory_2

    The command completes silently. We can confirm the new permissions using ls :

    1
2
3
4

    $ ls -l
drwxr-x--- 2 vagrant vagrant 4096 Aug 21 18:38 directory_2
...

    Because we’ve changed the “Others” digit of directory_2 ‘s file mode to 0, “Other” users no longer have any access to directory_2 . Its file mode went from 755 (before our chmod command) to 750 (after our command).

chown

1
2

$ whatis chown
chown (1)            - change file owner and group

We’ll use this command to make NewUser the owner of one of our files.

Even though chown isn’t categorized as an administrative command, we need to run the command with sudo to perform this change.

  • Example: 
    1

    $ sudo chown NewUser renamed_file1

    The command completes silently. We can confirm the new ownership using ls :

    1
2
3
4
5

    $ ls -l
drwxr-x--- 2 vagrant vagrant 4096 Aug 21 18:38 directory_2
-rw-r--r-- 1 vagrant vagrant   68 Aug 21 18:15 file3
-rw-r--r-- 1 vagrant vagrant   68 Aug 21 18:15 file4
-rw-r--r-- 1 NewUser vagrant    0 Aug 21 18:42 renamed_file1

    As we can see, NewUser is now the User owner of renamed_file1, and that user has assumed the rw- permission set that used to be assigned to the file’s previous owner.

We could also use a similar command, chgrp , to change which group is the Group owner of the file:
  • Example: 
    1

    $ sudo chgrp DevOps renamed_file1

    Again, the command completes silently, and we can confirm the new ownership using ls :

    1
2
3
4
5

    $ ls -l
drwxr-x--- 2 vagrant vagrant 4096 Aug 21 18:38 directory_2
-rw-r--r-- 1 vagrant vagrant   68 Aug 21 18:15 file3
-rw-r--r-- 1 vagrant vagrant   68 Aug 21 18:15 file4
-rw-r--r-- 1 NewUser DevOps    0 Aug 21 18:42 renamed_file1

Text search / manipulation

Below is a list of powerful command-line tools for dealing with text files/output. Each of these can be worth an in-depth study on their own.

grep - print lines that match patterns

cut - remove sections from each line of files

awk - pattern scanning and text processing language

sed - stream editor for filtering and transforming text

This blog post covered a lot of core Linux commands. It is not at all intended to be a comprehensive resource, so I will conclude this post here. Those wanting to dive deep into the Linux command line can try having a look here (I have no affiliation with the linked site or its author).

See you in the next post!

<< Back to 90 Days of DevOps posts

<<< Back to all posts

90DaysOfDevOps, Linux
devops linux linux-commands
This post is licensed under CC BY-NC-SA 4.0 by the author.