System Logs
Logs are the first thing to check when something doesn't work. On modern Linux there are two systems: journald (journalctl) for service logs, and text files in /var/log/ for application logs.
02
journalctl: system and service logs
Most used commands
bash
# All logs (from oldest)
journalctl
# Logs in real time (like tail -f)
journalctl -f
# Last N messages
journalctl -n 50
# Log of a specific service
journalctl -u nginx
journalctl -u mariadb
journalctl -u sshd
journalctl -u php8.2-fpm
# Log of a service in real time
journalctl -u nginx -f
# Log of current boot
journalctl -b
# Log of previous boot (useful to understand why the server restarted)
journalctl -b -1
# Only errors and critical messages
journalctl -p err
journalctl -p err -b # Only errors from current boot
# Logs in a time interval
journalctl --since "2024-03-01" --until "2024-03-28"
journalctl --since "1 hour ago"
journalctl --since "today"
Filter by keyword
bash
journalctl -u nginx | grep "error"
journalctl -u sshd | grep "Failed"
journalctl | grep -i "oom\|killed"
03
Traditional log files in /var/log/
| File | Content |
|---|---|
/var/log/syslog | General system log (Debian/Ubuntu) |
/var/log/messages | General log (CentOS/AlmaLinux) |
/var/log/auth.log | Login, sudo, SSH (Debian/Ubuntu) |
/var/log/secure | Login, SSH (CentOS) |
/var/log/nginx/access.log | HTTP requests received by Nginx |
/var/log/nginx/error.log | Nginx errors |
/var/log/apache2/error.log | Apache errors |
/var/log/mysql/error.log | MySQL/MariaDB errors |
/var/log/fail2ban.log | Fail2ban blocks |
/var/log/kern.log | Kernel log |
/var/log/dpkg.log | Package installations (Debian/Ubuntu) |
Useful commands for log files
bash
# Read a file in real time
tail -f /var/log/nginx/error.log
# Last 100 lines
tail -100 /var/log/nginx/access.log
# Search for a word in logs
grep "error" /var/log/nginx/error.log
grep -i "refused\|denied" /var/log/auth.log
# How many times an IP appears in the access log
grep "1.2.3.4" /var/log/nginx/access.log | wc -l
# IPs making the most requests (possible attack)
awk '{print $1}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20
04
Understand why the server restarted
bash
# Last reboot
last reboot | head -5
# OOM killer (process terminated due to lack of RAM)
journalctl -b -1 | grep -i "oom\|killed process\|out of memory"
dmesg | grep -i "oom"
# Kernel panic or crash
journalctl -b -1 -p 0..3
# Current uptime
uptime
05
Find suspicious SSH access
bash
# Failed login attempts
grep "Failed password" /var/log/auth.log | tail -20
# IPs attempting brute force
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn | head -10
# Successful logins
grep "Accepted" /var/log/auth.log | tail -20
# Last accesses
last | head -20
06
Manage log size
bash
# How much space journald logs use
journalctl --disk-usage
# Delete logs older than 30 days
journalctl --vacuum-time=30d
# Limit logs to 500MB total
journalctl --vacuum-size=500M
# Configure limits permanently
nano /etc/systemd/journald.conf
ini
[Journal]
SystemMaxUse=500M
MaxRetentionSec=1month
bash
systemctl restart systemd-journald
07
Automatic log rotation (logrotate)
Log rotation prevents files from becoming enormous. Verify it is configured:
bash
# Test the configuration
logrotate --debug /etc/logrotate.conf
# Force manual rotation
logrotate -f /etc/logrotate.conf
# Nginx configuration (example)
cat /etc/logrotate.d/nginx
Verwandte Artikel
Server Management
Server Reboot
How to properly restart your VPS or VDS, both via SSH and from the control panel
2 Min. Lesezeit
Server Management
Resource Monitoring
How to check CPU, RAM, disk and network on your server in real time
2 Min. Lesezeit
Server Management
Disk Management
How to check disk space, find large files and free up disk space
2 Min. Lesezeit
