/munetakaJupyterHub/Computer/2024/HAProxy/ubuntuLinux/02-setUpLogHAProxy.md

§2024-06-05

Step 1: Configure HAProxy

Ensure your /etc/haproxy/haproxy.cfg has the correct log directives. It looks like you already have the following configuration in place:

global
	log /dev/log local0
	log /dev/log local1 notice

local0 to local7: user-defined facilities for custom purposes local0: Could be used for general operational messages, such as informational messages, debug information, and standard traffic logs. local1: Could be used for more critical events, such as alerts, errors, or other significant conditions that need immediate attention.

¶ Step 2 install syslog-ng

sudo apt-get update
sudo apt-get install syslog-ng

 cp -v /etc/syslog-ng/syslog-ng.conf /etc/syslog-ng/syslog-ng.conf.backup
'/etc/syslog-ng/syslog-ng.conf' -> '/etc/syslog-ng/syslog-ng.conf.backup'

¶Step 3 edit /etc/syslog-ng/syslog-ng.conf

@version: 3.38
@include "scl.conf"

source s_local {
    unix-stream("/dev/log");
};

destination d_haproxy_local0 {
    file("/var/log/haproxy-local0.log");
};

destination d_haproxy_local1 {
    file("/var/log/haproxy-local1.log");
};

filter f_local0 {
    facility(local0);
};

filter f_local1 {
    facility(local1);
};

log {
    source(s_local);
    filter(f_local0);
    destination(d_haproxy_local0);
};

log {
    source(s_local);
    filter(f_local1);
    destination(d_haproxy_local1);
};

¶ Step 4. test and restart

# syslog-ng -s
[2024-06-05T08:45:19.659547] smart-multi-line: error opening smart-multi-line.fsm file; filename='/usr/share/syslog-ng/smart-multi-line.fsm', error='No such file or directory (2)'
[2024-06-05T08:45:19.659690] smart-multi-line: your smart-multi-line.fsm seems to be empty or non-existent, automatic multi-line log extraction will probably not work; filename='/usr/share/syslog-ng/smart-multi-line.fsm'

this error 'smart-multi-line.fsm` presists even I do not have any multi-line in syslog-ng.conf file

$ sudo systemctl restart syslog-ng
$ sudo systemctl status syslog-ng

$ sudo systemctl restart haproxy
$ sudo systemctl status haproxy

¶ Step 5. verify

root@pi3HAProxy:~# tail -f /var/log/haproxy.log 
2024-06-05T07:22:41.811253+08:00 pi3HAProxy haproxy[4242]: 210.242.152.235:47980 [05/Jun/2024:07:22:41.806] front__nextCloud_45101/1: SSL handshake failure (error:0A00009C:SSL routines::http request)

root@pi3HAProxy:~# tail -f /var/log/haproxy-local0.log 
Jun  5 08:19:30 pi3HAProxy haproxy[4242]: 193.34.75.12:48560 [05/Jun/2024:08:19:30.917] http_80_front~ http_8088_back/nginx_server 0/0/20/1/21 200 198 - - ---- 1/1/0/0/0 0/0 "HEAD / HTTP/1.1"
Jun  5 08:19:33 pi3HAProxy haproxy[4242]: 45.139.67.241:24652 [05/Jun/2024:08:19:33.107] http_80_front~ http_8088_back/nginx_server 0/0/26/4/30 200 2711 - - ---- 1/1/0/0/0 0/0 "GET / HTTP/1.1"

¶ Step 6 rotate log file

Edit sudo nano /etc/logrotate.d/haproxy as

# cp -v /etc/logrotate.d/haproxy /etc/logrotate.d/haproxy.backup

/var/log/haproxy-local0.log /var/log/haproxy-local1.log {
    daily
    rotate 7
    compress
    missingok
    notifempty
    create 0640 root root
    sharedscripts
    postrotate
        systemctl reload syslog-ng > /dev/null 2>&1 || true
    endscript
}

daily: Rotate the log files daily.
rotate 7: Keep 7 days of logs before deleting the oldest ones.
compress: Compress the rotated log files to save space.
missingok: Ignore missing log files (don’t error out if a log file is missing).
notifempty: Do not rotate the log if it is empty.
create 0640 root root: Create new log files with specified permissions and ownership after rotating.
sharedscripts: Ensure the postrotate script is run only once per logrotate invocation.
postrotate ... endscript: Reload syslog-ng after the logs are rotated to ensure it continues to write to the new log files.

¶ test rotate

# logrotate -d /etc/logrotate.d/haproxy
warning: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.

reading config file /etc/logrotate.d/haproxy
Reading state from file: /var/lib/logrotate/status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state

Handling 1 logs

rotating pattern: /var/log/haproxy-local0.log /var/log/haproxy-local1.log  after 1 days (7 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/haproxy-local0.log
error: skipping "/var/log/haproxy-local0.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
Creating new state
considering log /var/log/haproxy-local1.log
error: skipping "/var/log/haproxy-local1.log" because parent directory has insecure permissions (It's world writable or writable by group which is not "root") Set "su" directive in config file to tell logrotate which user/group should be used for rotation.
Creating new state

fixing /var/log permission

# chmod 755 /var/log/
# chown root:root /var/log/

retry

root@pi3HAProxy:~# logrotate -d /etc/logrotate.d/haproxy
warning: logrotate in debug mode does nothing except printing debug messages!  Consider using verbose mode (-v) instead if this is not what you want.

reading config file /etc/logrotate.d/haproxy
Reading state from file: /var/lib/logrotate/status
Allocating hash table for state file, size 64 entries
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state
Creating new state

Handling 1 logs

rotating pattern: /var/log/haproxy-local0.log /var/log/haproxy-local1.log  after 1 days (7 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/haproxy-local0.log
Creating new state
  Now: 2024-06-05 09:05
  Last rotated at 2024-06-05 09:00
  log does not need rotating (log has already been rotated)
considering log /var/log/haproxy-local1.log
  log /var/log/haproxy-local1.log does not exist -- skipping
Creating new state
not running postrotate script, since no logs were rotated

make it auto

sudo logrotate -f /etc/logrotate.d/haproxy