Introduction:

In our previous blog post, we explored the fundamentals of Linux commands that are essential for navigating and managing files and directories. Mastering these basics is crucial, but to truly harness the power of Linux, it’s time to dive deeper.

Welcome to the next step in your Linux journey—intermediate commands. In this post, we’ll build on the foundation you’ve already established and introduce more advanced tools and techniques. These commands will empower you to perform more complex tasks, automate processes, and manage your system more efficiently.

What to Expect:

Enhanced Text Processing

Commands like awk and sed will help you manipulate and extract information from files with precision.

Advanced File Management

Learn to archive, synchronize, and manage files and directories using tar, rsync, and other powerful utilities.

System Monitoring and Management

Tools such as htop, ps, and systemctl will provide insights into system performance and process management.

Network Tools

Commands like netstat, ss, and ip will help you monitor and configure network connections.

Automation and Scheduling

Get acquainted with scheduling tasks using cron and managing environment configurations with source.

Containerization

Dive into container management with docker, a critical tool for modern software development and deployment.

By mastering these intermediate commands, you’ll enhance your ability to troubleshoot, automate tasks, and gain a deeper understanding of how your Linux system operates. Let’s get started and take your Linux skills to the next level!

List of 20 intermediate-level Linux commands. Each command comes with a brief description of its purpose:

1.awk
Pattern scanning and processing language. For example I am having a file name players.txt

Example1: awk '{print}' filename.txt
It prints all the data in the file as shown below.

$ awk '{print}' players.txt
Sachin  IND  Batsman   
Lara    WI   Batsman
Akram   PAK  Bowler
Bond    NZ   Bowler
Kallis  SA   Allrounder
Bevan   AUS  llrounder
Stewart ENG  Wicketkeeper

Example2: awk '/pattern/ {print}' filename.txt
Print the lines which match the given pattern.

$ awk '/Batsman/ {print}' players.txt
Sachin  IND  Batsman   
Lara    WI   Batsman

In the above example, the awk command prints all the line which matches with the pattern ‘Batsman’.

Example3: awk '{print $1,$3}' filename.txt
Splitting a Line Into Fields: 
For each line, the awk command splits the line delimited by whitespace character by default and stores it in the $n variables. If the line has 3 words, it will be stored in $1, $2 and $3 respectively. Also, $0 represents the whole line. 

$ awk '{print $1,$3}' players.txt
Output: 
Sachin   Batsman
Lara     Batsman
Akram    Bowler
Bond     Bowler
Kallis   Allrounder
Bevan    Allrounder
Stewart  Wicketkeeper

2.sed
Stream editor for filtering and transforming text.
Example 1: 
sed 's/old/new/g' file.txt

Suppose you have a file called kibana.txt containing the following text:
Kibana is used to simplify log analysis – it is open-source software designed for data visualization. Kibana is a visual interface tool that allows you to explore, visualize, and create dashboards over log data collected in Elasticsearch clusters.

To replace the word "Kibana" with "ELK" in the file, you would use the sed command like this:
$ sed 's/Kibana/ELK/' kibana.txt

Output:
ELK is used to simplify log analysis – it is open-source software designed for data visualization. ELK is a visual interface tool that allows you to explore, visualize, and create dashboards over log data collected in Elasticsearch clusters.

Parenthesize first character of each word:
This sed example prints the first character of every word in parenthesis.
Example 2:
$ echo "Welcome To GenexDB Solutions" | sed 's/\(\b[A-Z]\)/\(\1\)/g'

Output:
(W)elcome (T)o (G)enexDB (S)olutions

Replacing string on a specific line number:
You can restrict the sed command to replace the string on a specific line number.
Example 3:
$ sed '2 s/Kibana/ELK/' kibana.txt

Output:
Kibana is used to simplify log analysis – it is open-source software designed for data visualization. ELK is a visual interface tool that allows you to explore, visualize, and create dashboards over log data collected in Elasticsearch clusters.

3.tar
The tar command is used in Linux to create compressed archives of files and directories.
Example: tar -czvf archive.tar.gz /path/to/directory

Creating a Compressed Archive.
Let's say you have a directory called genexdbs that you want to compress into a .tar.gz archive.

$ tar -czvf archive.tar.gz /root/genexdbs

-c: Creates a new archive.
-z: Compresses the archive using gzip.
-v: Displays the progress in the terminal (verbose mode).
-f: Specifies the name of the archive file (archive.tar.gz).

This command will take the contents of /root/genexdbs and compress them into a file named archive.tar.gz in the current directory. The -v option will list all files being added to the archive, so you can see the progress.

Output:
/root/genexdbs/
/root/genexdbs/players.txt
/root/genexdbs/Kibana

4.rsync
Sync files and directories between two locations.
Example: rsync -avh source/ destination/

a: Archive mode, which preserves permissions, symlinks, and other attributes.
v: Verbose mode, which provides detailed output.
h: Human-readable format, which makes the output easier to understand.

The rsync command in Linux is used for synchronizing files and directories between two locations. It’s commonly used for backups and mirroring, and it can efficiently copy only the differences between source and destination.

Example: Synchronizing a Directory
Suppose you have a directory named genexdbs_dir and you want to synchronize it with another directory named mysqldata_dir on the same machine.

$ rsync -avh /home/genexdbs/scripts/ /home/genexdbs/mypythonproject/

Output:
sending incremental file list
./
domain.sh
script.sh
test30days.sh

sent 28.82K bytes  received 133 bytes  57.91K bytes/sec
total size is 28.30K  speedup is 0.98

To confirm check,
/home/genexdbs/mypythonproject/$ ls
domain.sh   script.sh   test30days.sh    venv36

5.htop
Interactive process viewer (improved version of top).

Example: 
$ htop

Output:
  1  [|||||||||||||||                                23.8%]   Tasks: 80, 184 thr; 1 running
  2  [|||||||||||||||||||||||||||                    50.0%]   Load average: 0.58 0.45 0.38 
  3  [|||||||||||||                                  17.5%]   Uptime: 2 days, 4:17:13
  Mem[|||||||||||||||||||||                          2.5G/8G]
  Swp[|||                                            0.5G/2G]

  PID USER      PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
  1197 root       20   0  162M  9424  4708 S  1.9  0.1  0:10.17 systemd
  1187 user1      20   0  304M  7252  5560 S  0.6  0.1  0:02.53 bash
  1204 user2      20   0  305M  7564  5884 S  0.6  0.1  0:02.56 sshd

6.ps
Report a snapshot of current processes.

Example: 
$ ps aux

Output:
$ ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  0.2  0.0 170844 14356 ?        Ss   09:19   0:55 /sbin/init splash
root           2  0.0  0.0      0     0 ?        S    09:19   0:00 [kthreadd]
root           3  0.0  0.0      0     0 ?        I<   09:19   0:00 [rcu_gp]

7.pkill
Send signals to processes by name.

Example: 
$ pkill -f process_name
Kill all processes with the name 'process_name'.

Difference between SIGTERM and SIGKILL:
SIGTERM, it sends a termination signal to the process which helps in exit gracefully. Whereas SIGKILL sends a kill signal to the process, which terminate the processes forcefully and immediately.

For Example:
To send signal `SIGTERM` and PID `4321`, we can use the following command.
kill -15 4321

To send signal `SIGKILL` and PID `4321`, we can use the following command.
kill -9 4321

To find the PID of a specific program, you can use the pgrep command followed by the program’s name. For example, to find the PID of the “chrome” process, you would enter:

$ pgrep chrome

Output:
8614
8622
8624
8630
8631
8633
8659
8661

8.netstat
Displays network status and protocol statistics.

Example:
$ netstat

Output:
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 genexdbs-ThinkPad:35296 ec2-3-108-107-38.:https ESTABLISHED
tcp        0      0 localhost:39892         localhost:9200          ESTABLISHED
tcp        0      0 localhost:42268         localhost:9200          ESTABLISHED
tcp        0      0 localhost:46430         localhost:9200          ESTABLISHED
tcp        0      0 localhost:38782         localhost:9200          ESTABLISHED

$ netstat -tuln
Output:
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 127.0.0.1:25            0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:3000            0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:587           0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.1:631           0.0.0.0:*               LISTEN

9.ss
Utility to investigate sockets. To obtain a comprehensive list of all active sockets on your system.

Example: 
$ ss -tuln
Output:
Netid  State  Recv-Q  Send-Q   Local Address:Port                     Peer Address:Port        Process     
u_dgr  ESTAB  0       0        /run/systemd/notify 22031                        *   0                       
u_dgr  ESTAB  0       0        /run/systemd/journal/dev-log 22057               *   0                       
u_dgr  ESTAB  0       0        /run/systemd/journal/socket 22061                *   0                       
u_seq  ESTAB  0       0        @0001c 110899                                    *   110900

10.ifconfig
Configure network interfaces (deprecated in favor of ip).

Example: 
$ ifconfig eth0
This command will provide a comprehensive list of all network interfaces along with their respective IP addresses, MAC addresses, and other relevant details.

11.ip
Show/manipulate routing, devices, policy routing, and tunnels.

Example: 
$ ip addr show
This command displays details such as interface names (`eth0`, `wlan0`), MAC addresses, IPv4 and IPv6 addresses, subnet masks, and more.

12.man
The man command is used to display the manual pages for other commands and utilities.

Example: 
$ man ls

13.history
Display the command history.

Example: 
$ history

$ history 7
Output:
 1025  pip show pymysql
 1026  python --version
 1027  python -m site --user-site
 1028  pip show pymysql
 1029  pip install mysql-connector-python
 1030  cd scripts/
 1031  ls
 
This command displays the last 7 commands from the command history.

14.alias
Create shortcuts for commands.

Example: 
$ alias ll='ls -la'
This creates a shortcut ll that lists files in long format with hidden files.

To create an alias with a parameter
$ alias grep='grep --color=auto'
This alias makes grep always use color highlighting for matches.

To create an alias to navigate to a frequently used directory:
$ alias docs='cd ~/Documents'
The docs alias will take you directly to the Documents directory.

15.crontab
Schedule periodic tasks.

Example: 
To view or edit your cron jobs
$ crontab -e
This opens the current user's crontab file in the default text editor (usually vi or nano). You can add or modify cron jobs here.

To list the current user's cron jobs, use:
$ crontab -l
This displays all the cron jobs set for the current user.

To remove all cron jobs for the current user, use:
$ crontab -r
You will be prompted to confirm if you want to remove the existing cron jobs.

Crontab File Format:
* * * * * /path/to/command
- - - - -
| | | | |
| | | | +---- Day of the week (0 - 7) (Sunday is both 0 and 7)
| | | +------ Month (1 - 12)
| | +-------- Day of the month (1 - 31)
| +---------- Hour (0 - 23)
+------------ Minute (0 - 59)

Example Cron Jobs
To Run a Command Every Day at Midnight
0 0 * * * /home/genexdbs/script.sh

To Run a Command Every Hour
0 * * * * /home/genexdbs/script.sh

To Run a Command Every Monday at 5 PM
0 17 * * 1 /home/genexdbs/script.sh

To Run a Command Every 15 Minutes
*/15 * * * * /home/genexdbs/script.sh

16.systemctl
Control and manage the systemd system and service manager.

Example: 
$ systemctl status

To check the status of a specific service,
$ sudo systemctl status <service-name>

To start a service
$ sudo systemctl start <service-name>

To stop a running service
$ sudo systemctl stop <service-name>

To restart a service
$ sudo systemctl restart <service-name>

To reload the configuration of a service without restarting it
$ sudo systemctl reload <service-name>

To enable a service to start automatically at boot time
$ sudo systemctl enable <service-name>

To prevent a service from starting automatically at boot time
$ sudo systemctl disable <service-name>

To check the overall system status, including boot time and uptime
$ sudo systemctl status

17.journalctl
Query and display messages from the journal.

Example: 
$ sudo journalctl -xe
To view the most recent logs with additional explanations for errors.

Output:
Aug 16 18:10:19 genexdbs-ThinkPad-E14-Gen-2 acvpnagent[1059]: Function: getDnsConfiguration File: ../../vpn/Common/Utility/NetInterface_unix.cpp Line: 1147 Invoked Function: CDBusNMHelper::GetDNSConfig >
Aug 16 18:10:19 genexdbs-ThinkPad-E14-Gen-2 acvpnagent[1059]: Function: GetDNSConfig File: ../../vpn/Common/Utility/linux/DBusNMHelper.cpp Line: 295 Unable to get any DNS server for interface br-9393130>
Aug 16 18:10:19 genexdbs-ThinkPad-E14-Gen-2 acvpnagent[1059]: Function: getDnsConfiguration File: ../../vpn/Common/Utility/NetInterface_unix.cpp Line: 1147 Invoked Function: CDBusNMHelper::GetDNSConfig >
Aug 16 18:10:19 genexdbs-ThinkPad-E14-Gen-2 acvpnagent[1059]: Function: GetDNSConfig File: ../../vpn/Common/Utility/linux/DBusNMHelper.cpp Line: 264 Invoked Function: DBusNMHelper::getDNSConfigInternal >
Aug 16 18:10:19 genexdbs-ThinkPad-E14-Gen-2 acvpnagent[1059]: Function: GetDNSConfig File: ../../vpn/Common/Utility/linux/DBusNMHelper.cpp Line: 283 Invoked Function: DBusNMHelper::getDNSConfigInternal >

18.docker
Manage Docker containers.

Example:
To start docker service
$ sudo systemctl start docker

To enable docker to start on boot
$ sudo systemctl enable docker

To run a container
$ sudo docker run -d -p 80:80 --name mynginx nginx

To list running containers
$ sudo docker ps

To list all containers including stopped
$ sudo docker ps -a

To stop a running container
$ sudo docker stop CONTAINER_ID

To start a stopped container
$ sudo docker start CONTAINER_ID

To remove a container
$ sudo docker rm CONTAINER_ID

To remove a stopped container
$ sudo docker container prune

To list docker networs
$ sudo docker network ls

To list docker volumes
$ sudo docker volume ls

To show docker system information
$ sudo docker info

To view logs of a container
$ sudo docker logs CONTAINER_ID

19. whereis ls
The whereis command in Linux is used to locate the binary, source, and manual page files for a command.

Example:
To find the locations of the ls command

$ whereis ls

Output:
ls: /bin/ls /usr/share/man/man1/ls.1.gz

To find only the binaries for the grep command

$ whereis -b grep

Output:
grep: /bin/grep /usr/bin/grep

To find only the source files for the gcc command

$ whereis -s gcc

Output:
gcc:

To find locations for multiple commands at once

whereis ls grep

Output:

ls: /bin/ls /usr/share/man/man1/ls.1.gz
grep: /bin/grep /usr/bin/grep /usr/share/man/man1/grep.1.gz

20.lpr 
To print files.

Example:
Print a text file named document.txt:
$ lpr document.txt

Print two copies of document.txt:
$ lpr -#2 document.txt

Print in duplex mode (both sides of the paper):
$ lpr -o sides=two-sided-long-edge document.txt

To check the print queue and see the status of print jobs:
$ lpq

To cancel a print job with job ID 123:
$ cancel 123

Congratulations on making it through the intermediate level of Linux commands! By now, you’ve significantly expanded your toolkit, and you’re ready to tackle more advanced challenges. But the journey doesn’t stop here.

In our next blog post, we’ll take it up a notch and dive into professional-level Linux commands. These are the tools and techniques used by seasoned system administrators and developers to automate, secure, and optimize their environments. Whether you’re aiming to become a Linux expert or simply want to refine your skills, our next post will be your guide.

Stay tuned for more!