/munetakaJupyterHub/Computer/2024/HAProxy/archLinux/05-munetaka.me.md

§2024-06-04

Use HAPrpxy as load balancer.

USGFLEX200 setup
- WAN/LAN1/any --> pi2Nginx(192.168.48.249). all pi2Nginx_secrvice
- HighPorts is TCP/43000-48500

http://munetaka.me:80 ---> http://munetaka:8080 server by nginx

1.01. /etc/nginx/nginx.conf

backup

[root@pi2NginxMuNeTaka nginx]# mv -v /etc/nginx/nginx.conf /etc/nginx/nginx.conf.2024-06-04
renamed '/etc/nginx/nginx.conf' -> '/etc/nginx/nginx.conf.2024-06-04'

new nginx.conf as,

# ubuntu use user www-data which arch use http
# user www-data;
user http;
worker_processes auto;
pid /run/nginx.pid;

events {
    worker_connections 768;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    # Logging settings
    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    # Gzip settings
    gzip on;
    gzip_disable "msie6";

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

in /etc/nginx/site-avail/updtream-8080 as

server {
    listen 8080;
    server_name localhost;

    location / {
        root /usr/share/nginx/html/;
        index index.html;
    }
}

create a link

[root@pi2NginxMuNeTaka nginx]# cd sites-enabled/

[root@pi2NginxMuNeTaka sites-enabled]# ln -sf /etc/nginx/sites-avaliable/upstream-80 upstream-80

systemctl restart nginx and

[root@pi2NginxMuNeTaka sites-enabled]# systemctl restart nginx
[root@pi2NginxMuNeTaka sites-enabled]# systemctl status  nginx
● nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: disabled)
     Active: active (running) since Tue 2024-06-04 05:59:16 CST; 10s ago
    Process: 4034 ExecStart=/usr/bin/nginx (code=exited, status=0/SUCCESS)
   Main PID: 4035 (nginx)
      Tasks: 5 (limit: 1569)
        CPU: 202ms
     CGroup: /system.slice/nginx.service
             ├─4035 "nginx: master process /usr/bin/nginx"
             ├─4036 "nginx: worker process"
             ├─4037 "nginx: worker process"
             ├─4038 "nginx: worker process"
             └─4039 "nginx: worker process"

1.02 test

http://munetaka.me, old not modified yet,

munetaka.me_8080.png

Set up HAProxy to do load balance,

http://munekata.me:80 --> http://munetaka.me:8080

2.1. backup

sudo cp -v /etc/haproxy/haproxy /etc/haproxy/haproxy.backup

2.2. /etc/haproxy/haproxy.conf as,

global
    log /dev/log    local0
    log /dev/log    local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    timeout server  50000
    errorfile 400 /etc/haproxy/errors/400.http
    errorfile 403 /etc/haproxy/errors/403.http
    errorfile 404 /etc/haproxy/errors/404.http
    errorfile 408 /etc/haproxy/errors/408.http
    errorfile 500 /etc/haproxy/errors/500.http
    errorfile 502 /etc/haproxy/errors/502.http
    errorfile 503 /etc/haproxy/errors/503.http
    errorfile 504 /etc/haproxy/errors/504.http

frontend http_front
    bind *:80
    default_backend http_back

backend http_back
    server nginx_server 127.0.0.1:8080 check

2.3. restart and test

for archinux operation

& sudo mkdir /run/haproxy
$ sudo -R haproxy:haproxy /run/haproxy

[root@pi2NginxMuNeTaka alexlai]# systemctl restart haproxy
[root@pi2NginxMuNeTaka alexlai]# systemctl status  haproxy
● haproxy.service - HAProxy Load Balancer
     Loaded: loaded (/usr/lib/systemd/system/haproxy.service; disabled; preset: disabled)
     Active: active (running) since Tue 2024-06-04 10:30:28 CST; 10s ago
   Main PID: 7853 (haproxy)
     Status: "Ready."
      Tasks: 5 (limit: 1569)
        CPU: 604ms
     CGroup: /system.slice/haproxy.service
             ├─7853 /usr/bin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock
             └─7855 /usr/bin/haproxy -Ws -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid -S /run/haproxy-master.sock

Jun 04 10:30:27 pi2NginxMuNeTaka systemd[1]: Starting HAProxy Load Balancer...
Jun 04 10:30:28 pi2NginxMuNeTaka haproxy[7853]: [NOTICE]   (7853) : New worker (7855) forked
Jun 04 10:30:28 pi2NginxMuNeTaka haproxy[7853]: [NOTICE]   (7853) : Loading success.
Jun 04 10:30:28 pi2NginxMuNeTaka systemd[1]: Started HAProxy Load Balancer.

if [/usr/bin/haproxy.main()] Cannot chroot(/var/lib/haproxy). when start haproxy then

sudo mkdir -p /var/lib/haproxy
sudo chown haproxy:haproxy /var/lib/haproxy
sudo chmod 755 /var/lib/haproxy
sudo haproxy -c -f /etc/haproxy/haproxy.cfg

2.4. test with http://munetaka.me:80

munetaka.me_8080.png

3.0. Let's encryupt

using CertBolt

Yes, my web server is not currently running on this machine. Stop your webserver, then run this command to get a certificate. Certbot will temporarily spin up a webserver on your machine. sudo certbot certonly --standalone

$ sudo pacman -Rns cerbot certbot-nginx
# pacman -S certbot

on munetaka.me the certificate is specifically for nginx, delete and restart

[root@pi2NginxMuNeTaka letsencrypt]# sudo certbot revoke --cert-name munetaka.me --reason keycompromise <-- remove old munetaka.me

[root@pi2NginxMuNeTaka letsencrypt]# certbot delete --cert-name munetaka.me
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
The following certificate(s) are selected for deletion:

  * munetaka.me

WARNING: Before continuing, ensure that the listed certificates are not being
used by any installed server software (e.g. Apache, nginx, mail servers).
Deleting a certificate that is still being used will cause the server software
to stop working. See https://certbot.org/deleting-certs for information on
deleting certificates safely.

Are you sure you want to delete the above certificate(s)?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
(Y)es/(N)o: y
Deleted all files relating to certificate munetaka.me.

[root@pi2NginxMuNeTaka letsencrypt]# rm -rf  /etc/letsencrypt/live/*

[root@pi2NginxMuNeTaka letsencrypt]# certbot certonly --standalone -d munetaka.me
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Requesting a certificate for munetaka.me

Successfully received certificate.
Certificate is saved at: /etc/letsencrypt/live/munetaka.me/fullchain.pem
Key is saved at:         /etc/letsencrypt/live/munetaka.me/privkey.pem
This certificate expires on 2024-09-02.
These files will be updated when the certificate renews.

NEXT STEPS:
- The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
If you like Certbot, please consider supporting our work by:
 * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
 * Donating to EFF:                    https://eff.org/donate-le
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

[root@pi2NginxMuNeTaka letsencrypt]# ls -l /etc/letsencrypt/live/munetaka.me/
total 4
lrwxrwxrwx 1 root root  35 Jun  4 13:39 cert.pem -> ../../archive/munetaka.me/cert1.pem
lrwxrwxrwx 1 root root  36 Jun  4 13:39 chain.pem -> ../../archive/munetaka.me/chain1.pem
lrwxrwxrwx 1 root root  40 Jun  4 13:39 fullchain.pem -> ../../archive/munetaka.me/fullchain1.pem
lrwxrwxrwx 1 root root  38 Jun  4 13:39 privkey.pem -> ../../archive/munetaka.me/privkey1.pem
-rw-r--r-- 1 root root 692 Jun  4 13:39 README
[root@pi2NginxMuNeTaka letsencrypt]# ls -l /etc/letsencrypt/archive/munetaka.me
total 16
-rw-r--r-- 1 root root 1480 Jun  4 13:39 cert1.pem
-rw-r--r-- 1 root root 1826 Jun  4 13:39 chain1.pem
-rw-r--r-- 1 root root 3306 Jun  4 13:39 fullchain1.pem
-rw------- 1 root root  241 Jun  4 13:39 privkey1.pem

3.1. combine keys to be used for haproxy

[root@pi2NginxMuNeTaka letsencrypt]# sudo cat /etc/letsencrypt/live/munetaka.me/fullchain.pem /etc/letsencrypt/live/munetaka.me/privkey.pem > /etc/letsencrypt/live/munetaka.me/haproxy.pem
[root@pi2NginxMuNeTaka letsencrypt]# sudo chmod 600 /etc/letsencrypt/live/munetaka.me/haproxy.pem

And /etc/haproxy/haproxy.conf as,

global
    log /dev/log    local0
    log /dev/log    local1 notice
    chroot /var/lib/haproxy
    stats socket /run/haproxy/admin.sock mode 660 level admin
    stats timeout 30s
    user haproxy
    group haproxy
    daemon

defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    timeout connect 5000
    timeout client  50000
    timeout server  50000
    # ubuntu uses /etc/haproxy/errors/ directories
    # the followings are for ubuntu
    errorfile 400 /usr/share/haproxy/examples/errorfiles/400.http
    errorfile 403 /usr/share/haproxy/examples/errorfiles/403.http
    # HAProxy does not inherently generate 404 errors but can relay them from backend servers to clients.
    # errorfile 404 /usr/share/haproxy/examples/errorfiles/404.http
    errorfile 408 /usr/share/haproxy/examples/errorfiles/408.http
    errorfile 500 /usr/share/haproxy/examples/errorfiles/500.http
    errorfile 502 /usr/share/haproxy/examples/errorfiles/502.http
    errorfile 503 /usr/share/haproxy/examples/errorfiles/503.http
    errorfile 504 /usr/share/haproxy/examples/errorfiles/504.http

frontend https_front
    # bind *:443 ssl crt /etc/letsencrypt/live/munetaka.me/fullchain.pem key /etc/letsencrypt/live/munetaka.me/privkey.pem
    bind *:443 ssl crt /etc/letsencrypt/live/munetaka.me/haproxy.pem
    default_backend local_nginx


backend local_nginx
    option ssl-hello-chk
    server nginx_server 127.0.0.1:8080 ssl verify none

3.2. check haproxy.conf

[root@pi2NginxMuNeTaka letsencrypt]# haproxy -c -V -f /etc/haproxy/haproxy.cfg
Configuration file is valid

[root@pi2NginxMuNeTaka alexlai]# cp -v /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.http '/etc/haproxy/haproxy.cfg' -> '/etc/haproxy/haproxy.cfg.http'

[root@pi2NginxMuNeTaka alexlai]# nano /etc/haproxy/haproxy.cfg [root@pi2NginxMuNeTaka alexlai]# diff /etc/haproxy/haproxy.cfg /etc/haproxy/haproxy.cfg.http 31,33c31,33 < frontend https_front < bind *:443 ssl crt /etc/letsencrypt/live/munetaka.me/fullchain.pem key /etc/letsencrypt/live/munetaka.me/privkey.pem < default_backend local_nginx

frontend http_front bind *:80 default_backend http_back 35,37c35,36 < backend local_nginx < option ssl-hello-chk < server nginx_server 127.0.0.1:8080 ssl verify none

backend http_back server nginx_server 127.0.0.1:8080 check


3.2. check

$ sudo systemctl restart haproxy



3.5. make backend server https://munetaka.me:443 ---> localhost:8080

update /etc/nginx/sites-avaliable/upstream-8080 as

[root@pi2NginxMuNeTaka alexlai]# cp -v /etc/nginx/sites-avaliable/upstream-8080 /etc/nginx/sites-avaliable/upstream-8080.backup '/etc/nginx/sites-avaliable/upstream-8080' -> '/etc/nginx/sites-avaliable/upstream-8080.backup' [root@pi2NginxMuNeTaka alexlai]# nano /etc/nginx/sites-avaliable/upstream-8080 [root@pi2NginxMuNeTaka alexlai]# diff /etc/nginx/sites-avaliable/upstream-8080 /etc/nginx/sites-avaliable/upstream-8080.backup 2c2 < listen 8080 ssl;

listen 8080;

4,7d3 < < # generated by CertBolt, https://certbot.eff.org/ < ssl_certificate /etc/letsencrypt/live/munetaka.me/fullchain.pem; < ssl_certificate_key /etc/letsencrypt/live/munetaka.me/privkey.pem;


> verify haproxy -c -V -f /etc/haproxy/haproxy.cfg

[root@pi2NginxMuNeTaka alexlai]# haproxy -c -V -f /etc/haproxy/haproxy.cfg [NOTICE] (7920) : haproxy version is 2.9.7-5742051 [NOTICE] (7920) : path to executable is /usr/bin/haproxy [ALERT] (7920) : config : parsing [/etc/haproxy/haproxy.cfg:32] : 'bind *:443' in section 'frontend' : No Private Key found in '/etc/letsencrypt/live/munetaka.me/fullchain.pem.key'. [ALERT] (7920) : config : Error(s) found in configuration file : /etc/haproxy/haproxy.cfg [ALERT] (7920) : config : Fatal errors found in configuration. 3.3. restart nginx

[root@pi2NginxMuNeTaka alexlai]# systemctl restart nginx [root@pi2NginxMuNeTaka alexlai]# systemctl status nginx ● nginx.service - A high performance web server and a reverse proxy server Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; preset: disabled) Active: active (running) since Tue 2024-06-04 10:47:43 CST; 6s ago Process: 7881 ExecStart=/usr/bin/nginx (code=exited, status=0/SUCCESS) Main PID: 7883 (nginx) Tasks: 5 (limit: 1569) CPU: 215ms CGroup: /system.slice/nginx.service ├─7883 "nginx: master process /usr/bin/nginx" ├─7884 "nginx: worker process" ├─7885 "nginx: worker process" ├─7886 "nginx: worker process" └─7887 "nginx: worker process"

Jun 04 10:47:43 pi2NginxMuNeTaka systemd[1]: Starting A high performance web server and a reverse proxy server... Jun 04 10:47:43 pi2NginxMuNeTaka systemd[1]: Started A high performance web server and a reverse proxy server.


3.4. restart haproxy.service 

check https://munetaka.me:8080 amd https://munetaka.me:443