NGINX Access and Error Logs

When working with various applications, it is often necessary to refer to logs that record and store various data, ranging from debugging information to problem occurrences. With Nginx, users can also work with event logs that help resolve different issues. 

Nginx uses two types of logs:

1. Access Log records all requests that are received and processed through Nginx.
2. Error Log records various errors, including configuration errors for both Nginx and third-party services (such as php-fpm).

Today, we will explore the structure of the access logs and error logs in Nginx.

Prerequisites

To work with Nginx logs, we will need the following:

• One server or virtual machine with any pre-installed Linux distribution: Ubuntu, Debian, RHEL, CentOS, and many others. This article will use Ubuntu 22.04.

• An installed Nginx web server. You can install Nginx on Ubuntu by following this guide. In this article, we will use Nginx 1.18.0 installed from the Ubuntu distribution repositories.

Access Log

As mentioned earlier, the access_log is used to record all requests from clients. Each time a request from a client is received, Nginx writes this request into the access log. The saved entry contains a timestamp and client information, including the requested resource’s address, client address, and more. By default, the access log is written to the file access.log located at /var/log/nginx.

A typical log entry from the access log has the following format:

166.1.227.189 - - [20/Nov/2024:13:52:04 +0300] "GET /favicon.ico HTTP/1.1" 404 197 "http://91.206.179.176/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36"

Where:

• 166.1.227.189: Client's IP address that made the request.

• -: Client identifier. If a dash is returned, it means identd authentication is not used.

• -: Username. If a dash is returned, it means that authentication on the requested resource is either not required or was not successful.

• [20/Nov/2024:13:52:04 +0300]: Timestamp of when the request was made: date, time, and timezone.

• GET /favicon.ico HTTP/1.1: HTTP request string. It consists of:

• GET: The method used to access the requested resource.

• /favicon.ico: The requested resource.

• HTTP/1.1: The version of the HTTP protocol used.

• 404: HTTP response code. In this case, the 404 code means the requested resource was not found.

• 197: The size of the transmitted response in bytes (including headers and body).

• http://91.206.179.176/: HTTP referer (contains the URL of the request's source).

• Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.0.0 Safari/537.36: Client's User-Agent (contains information about the browser and operating system).

You can configure the access log at the web server level by setting the necessary options in the nginx.conf file or separately for each website by using its configuration file in the /etc/nginx/sites-available directory (if Nginx was installed from the operating system's repositories) or in the /etc/nginx/conf.d directory.

Two parameters are used for configuring the access log:

• log_format: Specifies the log format and the data to be logged. It uses many built-in variables (which will be covered later).

• access_log: Specifies the location of the access log file. By default, it uses the path /var/log/nginx/access.log.

Here’s an example of configuring the access.log in the main configuration file /etc/nginx/nginx.conf:

log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent"';

access_log /var/log/nginx/access.log main;

access_log /var/log/nginx/access.log main;

In this example, we use the following variables:

• $remote_addr: Records the client's IP address.
• $remote_user: Records the username passed during authentication (if authentication is enabled on the site).
• $time_local: Records the local time when the request was made.
• $request: Records the HTTP request string (including method, URI, and protocol version).
• $status: Records the HTTP response status.
• $body_bytes_sent: Records the number of bytes in the response body (excluding headers).
• $http_referer: Records the HTTP Referer header (the page address from which the request was made).
• $http_user_agent: Records the User-Agent header (information about the browser/client).

Refer to the Nginx documentation for more detailed information on all built-in variables.

Here’s the full content of the nginx.conf file:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
	# multi_accept on;
}

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##
           log_format main '$remote_addr - $remote_user [$time_local] "$request" '
                '$status $body_bytes_sent "$http_referer" '
                '"$http_user_agent"';
	access_log /var/log/nginx/access.log;
	##
	# Gzip Settings
	##

	gzip on;

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}

Error Log

For recording various types of errors, there is a second log in Nginx: error_log. This log records errors such as:

• Configuration errors
• Problems with connecting to backend servers
• Errors related to file access/permissions
• Internal server errors (5xx)
• SSL-related issues

By default, the error log is written to the error.log file located at /var/log/nginx.

A typical log entry from the error.log file looks like this:

20/Nov/2024:14:00:00 [error] 1234#5678: *10 open() "/var/www/html/test-page.html" failed (2: No such file or directory), client: 127.0.0.1, server: my-local-server.com, request: "GET /test-page.html HTTP/1.1", host: "example.com"

Where:

• 20/Nov/2024:14:00:00: The date and time when the error occurred.

• [error]: The error level. It can have values like [error], [info], [warn], [crit], and others. We’ll cover the logging levels in the next chapter.

• 1234#5678: The process and thread identifier.

• open() "/var/www/html/test-page.html" failed (2: No such file or directory): Error description. In this case, the request was made for a non-existent file (page) named test-page.html.

• client: 127.0.0.1: The IP address of the client who made the request. In this example, the request was sent from the local machine (localhost).

• server: my-local-server.com: The server name to which the request was sent.

• request: "GET /test-page.html HTTP/1.1": The HTTP request string. It consists of:

• GET: The method used to access the requested resource.

• /test-page.html: The requested resource.

• HTTP/1.1: The version of the HTTP protocol used.

• host: "example.com": The host name to which the request was sent.

You can configure the error log in the same way as the access log by setting the necessary options in the nginx.conf file at the web server level or separately for each website.

Error log parameters:

• log_format: Specifies the log format and the data to be logged. 

• error_log: Specifies the path to the error log.

Here’s an example of configuring the error.log, which should be included in the main configuration file for the web server nginx.conf:

log_format error '$remote_addr - $remote_user [$time_local] '
                           '"$request" $status $body_bytes_sent '
                           '"$http_referer" "$http_user_agent" "$gzip_ratio"';
error_log /var/log/nginx/access.log main;

Let's look at the variables in more detail:

• $remote_addr: Records the client's IP address.
• $remote_user: Records the username passed during authentication (if authentication is enabled on the site).
• $time_local: Records the local time, i.e., the time when the request was made.
• $request: Records the HTTP request string (including method, URI, and protocol version).
• $status: Records the HTTP response status.
• $body_bytes_sent: Records the number of bytes in the response body (excluding headers).
• $http_referer: Records the HTTP Referer header (the address of the page from which the request was made).
• $http_user_agent: Records the User-Agent header (information about the browser/client).
• $gzip_ratio: Records the Gzip compression ratio (if Gzip compression is enabled in Nginx settings).

Here’s the full content of the nginx.conf file:

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
	worker_connections 768;
	# multi_accept on;
}

http {

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##
  log_format error '$remote_addr - $remote_user [$time_local] '
                           '"$request" $status $body_bytes_sent '
                           '"$http_referer" "$http_user_agent" "$gzip_ratio"';
error_log /var/log/nginx/access.log error;
	##
	# Gzip Settings
	##

	gzip on;

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}

Logging Levels in Nginx Logs

Nginx has eight log levels, which apply to both the access log and the error log:

• debug: Intended for recording the most detailed information, including the internal mechanisms of Nginx. It logs information about each request and module. This level is used for diagnosing complex issues, especially during development or configuration.

• info: Used for logging informational messages about the server’s operation. These messages might include, for example, information about starting or stopping server processes.

• notice: Indicates important but non-critical events. These messages help understand how the system is operating but do not require intervention from the web server administrator. An example would be a configuration update.

• warn: Logs warnings that are not critical but could potentially lead to issues in the future.

• error: Reports problems that caused the failure of specific requests but do not affect the overall operation of the server. For instance, this could include being unable to open a file or connect to a backend.

• crit: Indicates critical errors that may disrupt the normal server operation. For example, this could include a module failure or resource shortage.

• alert: Reports serious issues that require immediate attention.

• emerg: The highest logging level. Indicates fatal errors that prevent the server from continuing to function. These messages require immediate correction.

Conclusion

Using Nginx access and error logs can significantly simplify troubleshooting and debugging issues. Nginx logging can be flexibly configured to record only the necessary data. You can also configure logging for each website individually or enable logging at the web server level for all sites.