From 42b52daaee361fc77c16ccb95c83cd0ba475a298 Mon Sep 17 00:00:00 2001 From: dogeystamp Date: Mon, 6 Jun 2022 22:05:56 -0400 Subject: [PATCH] Add post nginx.md --- src/posts/nginx.md | 213 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 213 insertions(+) create mode 100644 src/posts/nginx.md diff --git a/src/posts/nginx.md b/src/posts/nginx.md new file mode 100644 index 0000000..1f1b9f5 --- /dev/null +++ b/src/posts/nginx.md @@ -0,0 +1,213 @@ + + + + +# flexible web service configuration in ansible + +At the time of writing this article, I just finished transferring my website and services to my new domain, dogeystamp.com. +Normally, this would be as simple as rewriting a few configuration files. +However, since I use Ansible to automate configuration tasks, it was more complicated. + +This post will be documenting how I rewrote [my playbook](https://git.dogeystamp.com/dogeystamp/homeserver-ansible) in order to make my webserver configuration more flexible and modular, making future changes easier to perform. + +## previous issues + +Before switching domains (commit `495216318`), my nginx configuration file hard-coded the locations of each service. +It was also monolithic, which means that all of these locations were stuffed in a single file at `/etc/nginx/nginx.conf`. + +Excerpt from nginx.conf: + +``` +server_name {{ domain }}; + +location ~* ^(\/_matrix|\/_synapse\/client) { + proxy_pass http://localhost:8008; + proxy_set_header X-Forwarded-For $remote_addr; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header Host $host; + + client_max_body_size 50M; +} + +location = / { + return 301 https://{{ domain }}/site/index.html; +} + +location /site { + index index.html; +} + +location /wiki { + index index.php; +} + +location /rw { + index index.php; +} + +location /git/ { + proxy_pass http://localhost:3000/ ; +} + +location /mus/ { + proxy_pass http://localhost:4533/mus/ ; +} +``` + +During Ansible playbook execution, a single template task was used to deploy this file to the webserver: + +``` +- name: Configure nginx + template: + src: nginx.conf.j2 + dest: /etc/nginx/nginx.conf + notify: Restart webserver + +``` + +This system is slightly clunky. + +First, reading the configuration might be confusing, as multiple services have location blocks in it. +If someone wants to find the configuration for, say, Gitea, it might not be immediately clear where to find it. + +Second, the template file doesn't provide an easy method to switch the location of each service. +For example, switching to this new domain required placing each service in its own subdomain, rather than a subdirectory like here. +Enacting this new policy would mean rewriting each `location` block as a `server` block instead, which takes time. + +## flexible configuration + +Because of these issues, I decided to make some changes. + +First, I split the configuration file into multiple smaller ones. +This essentially means moving the parts relevant to specific services into separate files. + +For the automatic configuration in Ansible, I made a higher-level abstraction of the configuration. + +There are server blocks which correspond to each subdomain or domain, +and each server block can host a variable amount of services. +Services each have their own configuration file template, and they can be assigned to specific paths within their domain. + +Doing this allows a multitude of configurations, like assigning a single service to each subdomain, +or assigning all the services to the main domain as subdirectories. + +### implementation + +#### settings + +The abstract configuration is encoded as a YAML dictionary within Ansible. + + +Individual service settings: + +``` +nginx_services: + wiki: + path: "/" + navidrome: + path: "/" + gitea: + path: "/" +``` + +Each service has a specific path relative to the subdomain. + +Individual server block settings: + +``` +server_blocks: + wiki: + domain: "wiki.{{ domain }}" + ssl_cert: "{{ domain }}" + listens: "{{ default_listens }}" + services: + - wiki + + navidrome: + domain: "mus.{{ domain }}" + ssl_cert: "{{ domain }}" + listens: "{{ default_listens }}" + services: + - navidrome +``` + +Each server block has: + +* Associated domain +* SSL certificate to use +* List of ports to listen to +* List of services. + +#### templates + +Each service has an associated template with its specific nginx configuration. +These are stored within the `templates/srv_conf` folder of the webserver role, and have corresponding +names to the items in nginx\_services. + +For example, this is the template for the website: + +``` +location {{ nginx_services[srv].path }} { + root {{ webroot }}/{{ server_blocks[item].domain }}{{ nginx_services[srv].path }}; + index index.html; +} +``` + +This is a location block that could fit under any of the server blocks. +Multiple variables are being used in order to fit more use cases. +This allows the service to be placed in different URIs and domains, +or to switch the path where website files are placed. + +For another service, say the Gitea instance, this could be replaced with a location block that +does a proxy pass to the backend server. + +#### execution + +Now that Ansible knows the desired configuration, it has to be able to deploy it properly. + +First, it deploys the main nginx configuration: + +``` +- name: Create nginx SSL configuration file + template: + src: ssl.conf.j2 + dest: /etc/nginx/ssl.conf + notify: Restart webserver +``` + +Now, it only contains basic, general configuration settings. + +After that, Ansible deploys each individual server block as a separate file in conf.d. + +``` +- name: Create nginx server blocks + template: + src: server_block.j2 + dest: "/etc/nginx/conf.d/{{ item }}.conf" + with_items: "{{ server_blocks }}" + notify: Restart webserver +``` + +Most of the logic is actually performed within the Jinja template `server_block.j2`. + +``` +server { + ssl_certificate /etc/ssl-acme/certs/fullchain_{{ server_blocks[item].ssl_cert }}.crt; + ssl_certificate_key /etc/ssl-acme/keys/{{ server_blocks[item].ssl_cert }}.key; + + include ssl.conf; + + server_name {{ server_blocks[item].domain }}; + + {% for listener in server_blocks[item].listens %} + listen {{ listener }}; + {% endfor %} + + {% for srv in server_blocks[item].services %} + {% include "srv_conf/" + srv + ".conf.j2" %} + {% endfor %} +} +``` + +This individual `server` block listens to requests for a single domain, indicated by the `server_name` directive. +Then, Jinja goes through all the ports to listen to and add the appropriate `listen` directives. +Finally, all the relevant service templates are imported from the `srv_conf` folder based on the previous configuration.