My preparation notes for the RHCSA 8 exam.

Essential Tools

Documentation

The ability to understand a program using local documentation resources; man, info, /usr/share/doc, within the RPM package.

man

To browse man pages for a keyword use -k, e.g. scan documentation for all things relating to password:

man -k password

Alternatively:

mandb
apropos passwd

Specific sections with man, refer to different topics, e.g. section 5 is about config files, so man 5 passwd would bring up the documentation on /etc/passwd.

1 = user commands 5 = configuration files 7 = broad topics such as background 8 = sys admin

man -k user | grep 8 | grep create

/usr/share/doc

A gold mine of documents and sample configuration files. Usually for distributions that are not considered core, and don’t offer man or info pages.

RPM bundled documentation

$ rpm -qd tmux
/usr/share/doc/tmux/CHANGES
/usr/share/doc/tmux/FAQ
/usr/share/doc/tmux/TODO
/usr/share/man/man1/tmux.1.gz

General Searching Techniques

General search engine:

$ updatedb
$ locate passwd

Search path for passwd:

$ which passwd
/usr/bin/passwd

Search one-line man page descriptions:

$ whatis passwd
passwd (1)           - update user's authentication tokens
sslpasswd (1ssl)     - compute password hashes
passwd (5)           - password file

Find binaries and man pages for ls:

$ whereis -bm ls
ls: /usr/bin/ls /usr/share/man/man1/ls.1.gz /usr/share/man/man1/ls.1p.gz

Shell history

  • history dump history, by default the last 1000 commands
  • ctrl+r to search backwards through history for pattern
  • history -c clear history (in-memory only)
  • history -w write history
  • !32 run history event 32 (again)

Globbing

aka using wildcards see man 7 glob

  • ls host* zero or more chars
  • ls ?ost any single char
  • ls [hm]ost groups of chars
  • ls [!hm]ost negated groups of chars
  • ls [0-9][0-9]script multiple groups of restricted chars

I/O Redirection and Pipes

  • < stdin from a file or another programs stdout
  • > stdout to new file (overwrite if exists)
  • >> stdout to file (appending if exists)
  • 2> stderr redirection
  • 2>&1 stderr to stdout (useful for piping stderr, as pipes only work with stdout)
  • | pipe stdout from one program to stdin of another (pipes only support stdout to stdin communication, i.e. not stderr)

Essential File Management

Linux file system layout

See man hier and man file-hierarchy

Big hitters:

  • Boot partition: /boot/ and /efi/
  • System configuration: /etc/
  • Scripts and binaries: /bin/, /sbin/, /usr/sbin/ now all link back to /usr/bin/
  • Shared libraries: /lib/, /lib64/ link to /usr/lib/ and /usr/lib64/ respectively
  • Virtual kernel file system: /proc/ such as /proc/meminfo
  • Persistent variable data: /var/ such as /var/cache/, /var/log/, /var/tmp

Finding Files

locate

$ updatedb
$ locate passwd

find

Basic examples:

$ find / -size +100M -exec ls -l {} \;
$ find /etc -name motd  #named motd
$ find /etc -user schnerg  #owned by user shnerg
$ find / -mtime 3  #modified in last 3 days
$ find / -mtime +3  #not within the last 3 days

$ id ben
uid=1000(ben) gid=1000(ben) groups=1000(ben),1004(finance)
$ find / -uid 1000

$ find / -user ben -type f  #filter by files
$ find / -user ben -type f -exec cp {} /home/mary \; #execute a shell command against each result file `{}`

Archiving and compression with tar

Creating archives:

tar cvf foo.tar directory1 file1 file2
tar czvf foo.tar.gz directory1 file1 file2 #with gzip
tar cjvf foo.tar.bz directory1 file1 file2 #with bzip

List contents (without extraction):

tar tvf foo.tar

Extract them:

tar xvf foo.tar
tar xzvf foo.tar.gz
tar xjvf foo.tar.bz

Extract from a base directory:

tar xvf foo.tar -C /

Diffencing an archives contents with an existing exploded structure:

$ tar -dzvf dir1-v2.tar.gz
directory1/
directory1/file4
directory1/wookie4
tar: directory1/wookie4: Warning: Cannot stat: No such file or directory
directory1/wookie3
tar: directory1/wookie3: Warning: Cannot stat: No such file or directory
directory1/file3
directory1/imp1
directory1/imp1: Mod time differs
directory1/imp1: Size differs
directory1/imp2
directory1/file1
directory1/file2

Compression:

gzip file1
gzip -d file1
bzip2 file1

Listing compression stats on a compressed file:

$ gzip -l hello1.gz
         compressed        uncompressed  ratio uncompressed_name
                 83                  62   6.5% hello1

Archiving with star.

star -c -f=foo.tar directory1 hello1 hello2
star -cz -f=foo.tar.gz directory1 hello1 hello2 #with compression

Listing

$ star -t -f=foo.tar
directory1/
directory1/file4
directory1/file3
directory1/imp1
directory1/imp2
directory1/file1
directory1/file2
hello1
hello2

Extract a specific (hello1) file from the archive:

star -x -f=foo.tar hello1

Soft links, or symbolic links (symlinks), are simply pointers to other files. Symlinks can span multiple file systems. Permissions on symlinks aren’t real. The underlying permissions of the target file is what gets applied. They can easily be created with ln like so:

ln -s /etc/motd ~/motd

Hard links are links to a specific inode (shown with ls -i) on the file system. Due to this coupling, cannot span different file systems or devices.

$ ls -l
drwxrwxr-x. 2 ben ben 4096 May 13 20:42 directory1
-rw-rw-r--. 1 ben ben    0 May 13 20:53 hello1
lrwxrwxrwx. 1 ben ben    9 May 14 17:31 motd -> /etc/motd

$ ln hello1 hello1-hardlink
$ ls -l
drwxrwxr-x. 2 ben ben 4096 May 13 20:42 directory1
-rw-rw-r--. 2 ben ben    0 May 13 20:53 hello1
-rw-rw-r--. 2 ben ben    0 May 13 20:53 hello1-hardlink
lrwxrwxrwx. 1 ben ben    9 May 14 17:31 motd -> /etc/motd

In the ls long listing output, take note of the 2nd column, which represents the count of references to the same inode, which increases after creating a hard link. Some properties of hard links:

  • Hard links will always report the same metadata such as permission bits, modification timestamps, etc
  • inode reference counts will increase for each hard link.
  • Removal of the target file or hard link will not result in broken links, as they both physically reference the same inode.

Working with text

Regular expressions

See man 7 regex

  • . any single character

  • ? one or more

  • * zero or more

  • cat for concatenation, commonly used to dump contents to stdout

  • tac concatenation in reverse order

  • cut parses fields based on simple delimiter cut -d : -f 1 /etc/passwd cuts the first field in /etc/passwd based on a colon delimiter

  • sort can sort alphabetically or numerically e.g. cut -d : -f 3 /etc/passwd | sort -n

  • head first n lines

  • tail last n lines

  • tr translator e.g. lower to upper casing cut -d : -f 1 /etc/passwd | tr [a-z] [A-Z]

grep

The pinnacle of text processing, handed down by god himself.

grep '^#' /etc/sysconfig/sshd

Noteworthy:

  • always place regex between single quotes to avoid ambiguity of globbing
  • use -e to specify multiple expressions e.g. man -k password | grep -e '1' -e '8'
  • -B will provide n lines of before context e.g. -B 5 shows preceding 5 lines of each match
  • -v to inverse (e.g. things not comments grep -v '^#')
  • -i case insensitive
  • [^linux] negate characters, this will match against any characters that are not ‘l’, ‘i’, ‘n’, ‘u’ or ‘x’.
  • -E extended regular expression support

sed and awk

Powerful, line oriented text editors and full blown text based languages in their own right.

awk -F : '/anna/ { print $4 }' /etc/passwd

sed is a stream based (i.e. non-interactive) editor.

Print line 5 (-n will suppress auto printing of pattern space):

sed -n 5p /etc/passwd

Change user bill to william (-i is in-place mode and will mutate the target file, use without -i to test and write out to stdout first):

sed -i s/bill/william/g /etc/passwd

Delete line 4 (using -e editor mode):

sed -i -e '4d' /etc/passwd

Connecting a Linux host

Consoles Terminals and TTYs

A console is the environment which a user is presented with (e.g. graphical or textual)

A terminal an envionment opened on a console that provides access to a shell.

Graphical environments are optional in Linux. To make multiple consoles possible, has the concept of a virtual terminal aka a TTY (short for TeleTYpewriter).

Every terminal is associated with a device /dev/tty1 to /dev/tty6.

This interestly also applies to terminal emulators that are launched a graphical environment such as GNOME, /dev/pts/1, /dev/pts/2 and so on. Use the tty program to output the connected TTY.

The shortcut Alt+F1-6 (or the chvt program) will jump you between TTY1 through to TTY6:

  • TTY1 graphical login
  • TTY2 graphical console
  • TTY3 graphical session
  • TTY4-6 non-graphical consoles

Switch Users (su)

When creating a shell, its environment dictates much of its behavior.

su by default will create a sub shell, that will simply use the existing environment. The bashrc file is used to bootstrap a sub shell.

This often is not wanted. More useful is to create fresh the environment of the target user.

This is known as a login shell, and can be obtained by passing a bare - (dash), -l or --login to the su command. The profile file is used to bootstrap a login shell.

su - shnerg
su -l shnerg
su --login shnerg

/etc/profile is the global shell configuration, and applies to all users login shells.

A login shell (.bash_profile) vs interactive shell (.bashrc).

sudo

sudo executes a command as another user, without requiring use of a login shell.

sudo uses a pluggable based policy, /etc/sudoers by default, to determine what users can do.

/etc/sudoers should never be edited directly, but instead using the visudo command.

The %wheel rule is commonly (lazily?) used to grant users sudo access, by putting them in the wheel group.

SSH

Remote encrypted access, using OpenSSH server daemon.

systemctl status sshd

SSH supports authentication via a simple username and password, but also using an asymetric keypair.

ssh-keygen -t dsa

Managing users and groups

Broadly, there are users for services, humans and root.

Conventions for UID (see /etc/login.defs):

  • < 201: privilaged users
  • 201 - 999: system accounts
  • 1000 - 60000: average joe users

Humans don’t always need to interact directly with a Linux host, for example a web or email server. If this is the case, their default shell should be changed from /bin/bash to /sbin/nologin

Creating users

useradd shnerg will register a new local user account on the system. This involves:

  • create entry in /etc/passwd
  • create entry in /etc/shadow
  • create home directory /home/shnerg
  • create user specific bash initialisation scripts .bash_profile, .bashrc and .bash_logout

The /etc/skel/ directory provides the skeleton scripts and files to be copied into new users home directories.

To remove a user and their home directory, use the -r option, and -f even if the user is logged in.

userdel -rf shnerg

User properties

User objects are made up of many attributes, shown by usermod --help

  • -c an arbitrary annotation such as a role (GECOS field)
  • -d home dir path
  • -e point in time to disable the user
  • -g -u gid uid
  • -G groups
  • -s default shell such as /bin/bash, /sbin/nologin
  • -R location to chroot the user into, interesting!
  • -L -U lock unlock

User configuration files

  • /etc/default/useradd default new user properties
  • /etc/login.defs more default new user properties (if conflicts, takes precedence)
  • /etc/skel/ cloned to new user home directories
  • /etc/passwd user database, all properties of users are encoded here
  • /etc/shadow user password storage and properties, the format of an entry: login:encrypted-password:password-changed-date:min-age:max-age:warning-days:inactive-days:user-expiry-date. Use passwd -S shnerg to display password props for a user.
  • /etc/group all groups

Creating and managing groups

groupadd, groupdel and groupmod

The most common property is the gid

Ways to add users to a group:

  • vi /etc/group
  • vigr for vi with group validation
  • usermod -aG shnerg people

Use getent group finance to validate a group exists, and id <user> to validate the group memberships the user has.

Some facinating (to me anyway) group management programs include newgrp to switch the primary group for the current session, and sg to execute a command as a different group.

Managing password properties

Programs to be across: passwd, chage

  • passwd -S mike displays all password related props
  • echo password | passwd --stdin to set password programmatically (by default will interactively prompt)
  • default password attributes are controlled by /etc/login.defs

Managing Permissions

File permissions are applied at 3 levels; the user, the group and others. Each can read, write and/or execute.

A sample file permission bitmap could be -rwxrw-rw-. The first bit - a dash indicates its a plain old file (there are several types, such as l for symlink, d a directory, …).

Then follows the user, group and others bits.

note: Linux uses a simplistic exit on match algorithm. If the user matches and has no permissions, Linux will not bother evaluating the group or others permission bits (even if they would grant access!).

Changing file ownership

  • chown change owner, can take the names of the login and group like so chown anna:sales sales. Either the user or group can be omitted to not change its existing value.
  • chgrp will change only the group ownership. Its redundant these days with the powers that chown has.

Managing basic permissions

Linux supports three levels of permissions:, known affectionately as UGO (user/group/others).

Permission Octal File Dir
read 4 open list
write 2 modify create/delete
execute 1 run cd

chmod supports symbolic and octal variations of permissions. Some symbolic examples:

In octal notation, set read/write/execute for the user, read/write for the group and just read for others:

chmod 764 afile

In symbolic notation, set user bits to read/execute, remove the write permission for the group (leaving other permission in tact), and add execute permission for others:

chmod u=rx,g-w,o+x afile

More examples:

chmod u+x file1
chmod g-rw file1
chmod o+wx file1

Perhaps the most useful form, apply execute permission to user, group and others:

chmod +x file1

To navigate directory structure, requires execute permission on the directory. Execute bits could be set on directories, but not files, to allow a browsable tree, using chmod with the X (big x) modifier.

When creating new files, the default owner and group will be that of the user (e.g. ben). newgrp finance will default the group to finance.

Default permissions are applied with umask.

groupadd finance #add group
getent group #verify
usermod -G finance amy #add user to group
mkdir /home/finance #create a dir
chown :finance /home/finance #change its group
chmod -R o-rwx g+rw /home/finance #remove other perms and +rw group perms
exit #logout user to reload groups

Recursively setting execute on directories only:

chmod ugo-x -R finance #strip execute on everything
chmod ug+X -R finance #user and group directory exec bit only

To apply the permission bits to all specify a (as opposed to the usual u, g and o):

chmod a+r file1

Understanding umask (user mask)

Simply put is a bit mask.

This mask is applied to the system wide defaults 666 for files, and 777 for directories.

$ umask
0022

Breaking down each bit:

  • The first 0 will not apply any mask to the special bits (suid/guid/sticky bit)
  • The second 0 will apply no mask to the owner
  • The third 2 will mask/strip out (think subtract) write permission (2 in octal) for the group
  • The forth bit 2 will mask/strip out write for others

In practice umask values of 0, 2 and 7 are used:

  • 0 means 6 for files, and 7 for directories
  • 2 means 4 for files and 5 for directories
  • 7 means 0 for files and 0 for directories

The base /etc/bashrc and /etc/profile bash environment bootstrapping files contain entries for setting up default umask values.

Special permissions

Permission Octal File Dir
suid 4 run as owner -
sgid 2 run as group inherit group owner
sticky 1 - only delete if owner

suid

The running of processes as their original owner. Impersonation if you will. Known as suid. Take for example the /usr/bin/passwd program:

-rwsr-xr-x.   1 root root       27872 Feb  5  2016 passwd
   ^

Note the s (suid) bit. While passwd is owned and grouped by root, its runnable by average joe users under roots context, as if being run by the real root user.

Can be set with chmod:

chmod u+s file1
chmod 4500 file1
chmod 2500 file1
chmod 6444 file

sgid

Very useful for defining a group owner that gets inherited within a directory tree.

  • Imagine a /data/sales/ dir.
  • If a user mike creates a file (or dir) within /data/sales/ the user will be set to mike, and the group also set to mike.
  • In a group environment, such as /data/sales/, it would be more useful if the group was set to the sales group
  • The sgid special bit will propagate the group owner to new files or directories and is set with chmod g+s /data/sales/ or chmod 2770 /data/sales/

Sticky bit

Prevents the removal of files and/or directories unless that user is the owner. To set:

chmod +t mydir
chmod 1777 mydir

+t sets the sticky bit:

drw-rw---T. 1 ben  finance    0 May 14 19:06 mydir
         ^

Understanding ACLs

Several years later, in addition to the special bits, ACL (access control list) support was introduced to the kernel.

ACLs offer a few benefits over the simple UGO system:

  • more granular inheritable permission chains on specific directories
  • multiple owners

Scenario, under /data/ exists accounting/ (owned by root:accounting) and sales/ (owned by root:sales).

We want to grant the sales group rx permission to /data/accounting/.

This is not possible with the simple UGO model.

For directories:

  • setfacl -R -m g:sales:rx accounting to set ACLs on existing files and directories
  • setfacl -m d:g:sales:rx accounting to set the default ACL on new objects that are created
  • getfacl accounting to view ACLs

For files:

  • setfacl -m u:george:r myfile

To remove ACLs involves using the dash -

For example setfacl -m d:o::- secret-dir will strip all ACLs for others. Interestingly this (i.e. no permissions for others) will propagate down the tree to any new objects created within secret-dir, awesome!

ls will tack a + symbol to the end of the permission breakdown (e.g. drwxrwxr-x+), to indicate an ACL exists.

Configuring Networking

Network device naming

  • BIOS naming based on hardware properties such as em[1-N] for embedded NICs, p[slot_number]p[port_number]
  • udev naming ethX
  • Physical naming similar to BIOS naming with more variations
  • Logical naming such as vlan or alias
  • To get classical ethX naming, use biosdevname=0 and net.ifnames=0 GRUB boot options

Managing runtime network configuration with ip

ip addr help ip addr add 10.0.0.10/24 dev enpls0 ip link show

Storing network configuration persistently

Persistent network configuration is stored in /etc/sysconfig/network-scripts/, each NIC device is represented e.g. ifcfg-enp1s0

The Network Manager service is responsible for managing these configurations. An active NM configuration for a NIC device is termed a connection.

Frontends include nmcli and nmtui

nmcli

man nmcli-examples

Bash tab completion rocks for CLI’s like nmcli, check its installed with rpm -qa bash-completion

To add a new connection:

nmcli connection add ifname enp1s0 type ethernet ip4 192.168.4.210/24 gw4 192.168.4.2

To activate other connection profiles for the same device:

nmcli connection up enp1s0-profile

Verify connection status:

nmcli connection show

Modify an existing connection profile to define the DNS:

nmcli connection modify enp1s0-profile ipv4.dns 8.8.8.8

nmcli in the above will update /etc/resolv.conf

nmcli also features an interacive edit mode nmcli connection edit simoid-enp1s0, which will display a shell nmcli>

Routing and DNS

ip route show ip route del default via 192.168.4.1 ip route add default via 192.168.4.2

Using nmcli to set persistent routes (default gateway):

nmcli connection edit simoid-enp1s0
nmcli> set ipv4.gatway 192.168.122.1
nmcli> save
nmcli> quit
nmcli connection up simoid-enp1s0

Setting the hostname on RHEL is done with hostnamectl:

  • hostnamectl status
  • hostnamectl set-hostname host14.bencode.net

Network Analysis Tools

Useful troubleshooting tools:

  • hostname
  • ping send ICMP packets to other hosts and subnets
  • traceroute shows routers involves
  • dig DNS hostname resolution
  • netstat now ss list open ports
  • nmap firewall validation of remote hosts
  • ip

Rando cool

  • ctrl+l = clear terminal
  • ls -d don’t show contents of directories
  • \ls un-alias a command, by preceding it with a backslash \
  • alias to display evaluated bash aliases
  • tar command options are not prefixed with a hypen -
  • tac is the inverse program of cat
  • chvt jumps between TTY e.g. chvt 3
  • ssh-keygen supports a number of ciphers, set using [-t dsa | ecdsa | ed25519 | rsa], RSA by default