Checking Server Load and Memory Usage on an Ubuntu Server via SSH

When managing a remote Ubuntu server, it's important to monitor server load and memory usage to ensure that the server is running optimally. There are several tools available for checking server load and memory usage, including top,free, and sar. In this tutorial, we will walk you through how to use these tools to monitor server load and memory usage on an Ubuntu server.

Prerequisites

Before you get started, you will need:

A remote Ubuntu server accessible via SSH
SSH client software installed on your local machine
Administrative access to the remote server

Step 1: Connect to the Remote Server via SSH

To connect to your remote Ubuntu server via SSH, open a terminal window on your local machine and type the following command:

ssh username@remote_server_ip_address

Replace username with your username on the remote server, and remote_server_ip_address with the IP address of your remote server.

If this is your first time connecting to the remote server, you will be prompted to accept the server's fingerprint. Type "yes" to continue.

You will then be prompted to enter your password for the remote server. Enter your password and press enter.

You are now connected to your remote server via SSH.

Step 2: Check Server Load with top and find the process causing it

The top command provides a real-time view of the processes running on the server, including their CPU and memory usage. To check the server load with top, type the following command in the terminal window:

top -c

This will display a list of processes currently running on the server, along with their resource usage. The -c option will display the full command line of each process.

We'll need to figure out which processes are hogging CPU and memory resources. To receive a sorted list of processes based on CPU consumption, press Shift+P, and to get a list of processes based on memory usage, press Shift+M. This will assist you in determining which apps or services are generating the server load issues. Once you've identified the process or service, you may examine its error log or access logs and other information to determine what's causing the resource usage surge.

For example, here is an example of top -c command result which is sorted in CPU usage base.

Because this is a RunCloud-based server, you will notice various RunCloud-related items. The preceding figure shows that a PHP-FPM process is now taking the most CPU, yet it is most likely within typical utilisation. However, if the value is higher than average usage, we can review the access log to see if it's a bruteforce attack, etc. To exit the top command, press ctrl+C.

We can check the /home/runcloud/logs/nginx folder to find your web application access log. If it's a OpenLiteSpeed server then it will be in /home/runcloud/logs. For this case the web application owner is 'runcloud' so the path depends on your web application owner.

The access log will looks as below

34.218.238.116 - - [10/Apr/2023:23:16:39 +0530] "POST /wp-cron.php HTTP/1.1" 200 0 "-" "WordPress/6.2; http://test.link" "-"

34.218.238.116: This is the IP address of the client who made the request. In this case, the IP address is "34.218.238.116".

-: This field represents the remote user that made the request. In this case, it's empty, which means that the user is unknown.
-: This field represents the authenticated user that made the request. In this case, it's also empty, which means that the user is not authenticated.

[10/Apr/2023:23:16:39 +0530]: This field represents the date and time when the request was made. In this case, it was on April 10th, 2023 at 23:16:39, with a timezone offset of +0530.

"POST /wp-cron.php HTTP/1.1": This field represents the HTTP method, the requested URL, and the HTTP protocol version. In this case, the client made a POST request to "/wp-cron.php" using HTTP/1.1.
200: This field represents the HTTP status code returned by the server. In this case, the server returned a status code of 200, which means that the request was successful.
0: This field represents the size of the response sent by the server, in bytes. In this case, the server sent a response with a size of 0 bytes, which means that no content was returned.
"-": This field represents the referer URL, which is the page that referred the client to the current page. In this case, the referer URL is empty, which means that there was no referrer.

"WordPress/6.2; http://test.link": This field represents the user agent that made the request, which is the software used by the client to make the request. In this case, the user agent is "WordPress/6.2" and the website URL is "http://test.link".

Now the task is to find what causing the spike in server load for that we need to know which IP is accessing this web application and requested field. We can use the grep command as shown below for that.

grep "16/Apr/2023" /home/runcloud/logs/nginx/app-greenfelder_access.log | awk '{print $1}' | sort -n | uniq -c | sort -n

Here is the explanation for each of these commands.

grep "16/Apr/2023" /home/runcloud/logs/nginx/app-greenfelder_access.log: This command searches for all lines in the /home/runcloud/logs/nginx/app-greenfelder_access.log file that contain the date string "16/Apr/2023". The output of this command is a list of all the matching log entries
awk '{print $1}': This command uses awk to extract the first field of each log entry, which is the IP address of the client. This effectively filters out all the other information in the log entry and leaves only the IP address.
sort -n: This command sorts the list of IP addresses numerically (i.e., by IP address) in ascending order. This makes it easier to count the number of occurrences of each IP address in the next step.
sort -n: This command sorts the list of IP addresses and their counts numerically in ascending order of count. This puts the IP addresses with the fewest hits at the top of the list and the IP addresses with the most hits at the bottom of the list.

The above command will list out all IP address access and it can be difficult to go through all of it. You can short the result by appending below command to above command "tail -n 10" (for listing top 10 results).

grep "16/Apr/2023" /home/runcloud/logs/nginx/app-greenfelder_access.log | awk '{print $1}' | sort -n | uniq -c | sort -n | tail -n 10

To find the most often accessed URL, simply alter the awk command to awk 'print $7' because the URL position is on the 7 field. Please keep in mind that the OpenLiteSpeed server's IP value is on $2 and the URL value is $8.

For example to find the most frequently accessed URL for OpenLiteSpeed server run the below command.

grep "16/Apr/2023" /home/runcloud/logs/app-greenfelder_access.log | awk '{print $8}' | sort -n | uniq -c | sort -n | tail -n 10

Based on the result, you can either block unknown IP's or restrict access to some URL like xmlrpc.php file etc.

Step 3: Check Memory Usage with `free`

The free command displays the amount of free, used, and total memory on the server, as well as the amount of memory being used for buffers and cache. To check the memory usage with free, type the following command in the terminal window:

free -m

This will display the memory usage in megabytes. To exit the freeCtrl+C.

Step 4: Check Historical Performance Data with `sar`

The sar command provides historical data on server performance, including CPU usage, memory usage, disk activity, and network activity.

First we need to install sysstat in your server.

First, make sure your Ubuntu server is up-to-date. You can update the package list and upgrade the existing packages using the following commands:

sudo apt update && apt upgrade

Once the system is updated, you can install the sysstat package using the apt package manager. Type the following command to install sysstat:

sudo apt install sysstat

During the installation process, you may be prompted to confirm the installation of the package and its dependencies. Type "Y" and hit enter to proceed.
Open the sysstat configuration file located at /etc/default/sysstat using your preferred text editor. For example, you can use the nano editor by running the following command:

sudo nano /etc/default/sysstat

In the configuration file, locate the line that starts with ENABLED= and change the value to "true" to enable sysstat. The line should look like this:

ENABLED="true"

By default, sysstat collects system performance data every 10 minutes. If you want to change the collection interval, you can modify the INTERVAL= value in the configuration file. For example, to collect data every 5 minutes, you can set the value to

INTERVAL="5"

Save the changes to the configuration file and exit your text editor.

sar

sar -u

This will display CPU usage data. To display memory usage data, use the -r option instead:

sar -r

By default, sar displays performance data for the current day. If you want to display data for a different day, you can use the -f option followed by the path to the file that contains the data. For example, to display data for April 10th, 2023, you would run the following command.

sar -u -f /var/log/sysstat/sa10

To exit the sar command, press Ctrl+C.

Conclusion

By using these tools to monitor server load and memory usage on your remote Ubuntu server via SSH, you can gain a better understanding of how your server is performing and identify any potential issues. With regular monitoring and analysis, you can optimize your server's performance and ensure that it's running smoothly.

Search This Blog