Why use a proxy (nginx)

Jun 27 2023

Someone I know built his first service that allows clients to connect with a WebSocket. It is a small game server and people can play tic-tac-toe together.

When setting up the game server the question came up about why we need to configure nginx as a proxy.

Here I want to answer why and show a few use cases of such a proxy.

It is important to note that deploying applications has and still is drastically changing. We are going to rent a VPS (virtual private server), which is a virtual machine that runs some operating system, in our case Fedora on it and we will configure it ourselves. This is a bit of an old-school approach or at least there are new platforms and managed services coming up that directly allow you to deploy applications with less configuration needed on your part. With the approach used here you will need to do some sysadmin work such as installing a firewall, configuring a proxy, maybe later setting up docker, creating an SSL certificate, and so on. This is quite a lot of work and some knowledge is required to do that. In case you are starting out you might first want to check out more ‘modern’ approaches that do most of these things for you. The advantage of the approach described here is that everything is under your control, it might be cheaper than some platforms regarding hosting costs and you can reuse your server for multiple things at the same time which can come in quite handy. It also forces you to learn a lot of parts that make up a server and some sysadmin tasks.

If you want to deploy a static site maybe check out Netlify, and for hosting apps directly there exists Digitalocean. If you know some good alternative services please let me know.

Also, I will provide some snippets of commands that are needed for configuring Nginx on Fedora this will not be a complete set up but will provide an overview of the usage of Nginx.

The images displayed were created using tldraw.com, see the snapshot link here.

Initial Set up

We have a tic tac toe server application built in Java using the javalin web framework that we want to deploy to our machine. The VPS has been rented on hetzner (many other providers exist with a similarly competitive pricing scheme). This rented machine will be called Gandalf which is its hostname that runs Fedora as its operating system, as an example, it will have IP 203.0.113.0. A domain name has been purchased which we will call here lamapalusa.com (still free to buy), and DNS records are configured so that navigating to lamapalusa.com will resolve to our Gandalf server with IP 203.0.113.0. When we start the tic-tac-toe server it will bind on port 7070 and for on the URL /websocket clients can create a WebSocket connection. After two clients are connected they will be paired together and can play with each other. On the root URL / the server returns a simple html page with some info regarding the game and two download links.

First, we will run our app without a proxy and see what it looks like. Afterwards, we will configure nginx as our proxy and see the advantages it can offer.

Running the server on port 7070

We configure a firewall called firewalld that allows ssh connections on port 22 to go through. The firewall will block anything else.

Additionally, as the game server listens on port 7070 we will also open up that port.

dnf install firewalld
firewall-cmd --permanent --zone=public --add-port=7070/tcp
systemctl enable --now firewalld
systemctl status firwalld

If you want to know which services/ports are open on your firewall:

firewall-cmd --list-services
firewall-cmd --list-ports

We also start up our application, for now, we will run it directly on Gandalf. For a production application you would do it differently for example you might use docker but for now, this will be enough for testing. Note as we run the application on our host and it is a jar file we first need to install the Java runtime!

dnf install java-11-openjdk
java -jar GameServer.jar

Now if the tic-tac-toe client uses the URL ws://lamapalusa.com:7070/websocket and we start up our client it should connect with the server.

If we want to navigate to the webpage of the game server (reminder the WebSocket is available on path /websocket and no path aka / returns an html page) we will need to specify our port as well like this http://lamapalusa.com:7070. This is a bit awkward. Normally when you use other URLs you don’t have to specify the path. This is as your browser and websites in general are by default on port 80 or 8080 and you do not need to tell the browser which port when typing the URL into your bar. As we run on the nonstandard port 7070 we have to specify the port in the URL:http://lamapalusa.com:7070 Otherwise the browser will try port 80 which is not used yet.

You could potentially change the port the application uses to 80 or 8080.

Another problem is that we might want to use the root path of our website lamapalusa.com/ as a main page for some html content unrelated to tic-tac-toe. So if we change our app to port 80 we could not do this anymore. We would rather access the tic-tac-toe site on the URL lamapalusa.com/tictactoe and not specify any port at all.

We also haven’t configured SSL, so no secure https or wss. With a proxy we can do both easily, run the application under a specific URL and also configure SSL with it.

Running the application under the URL /tictactoe

In order to use Nginx as our proxy we will need to install and enable it first:

dnf install nginx
systemctl enable --now nginx
systemctl status nginx

If we change any config files we have to reload the nginx, otherwise the new changes won’t be applied:

systemctl reload nginx

Also the firewall needs to be open on port 80,8080 (and later also for 443, 8443).

In order to configure Nginx, there is as usual a configuration to do this. The configuration is at path /etc/nginx/nginx.conf. Depending on your operating system there might also be the folder /etc/nginx/sites-available which contains the Nginx config files. However, the /etc/nginx/nginx.conf is still the main configuration file, it will just include files from the sites-available folder. On Fedora this is not the case so we will directly edit /etc/nginx/nginx.conf.

You don’t need to understand the full configuration but there are two important parts

• a server block with a server_name specified such as lamapalusa.com. If the URL will contain this server name then the block will match. Otherwise, the Nginx will do nothing and return a not found page.

• within each server block there are location directives. For example location /test within the same server block as above would match the URL lamapalusa.com/test and then pass the request along to the application you want.

This allows us to forward requests to different parts / applications within our server using Nginx.

The next step is that we do not expose our port directly. The game server will be available at the URL http://lamapalusa.com/tictactoe for the webpage of the game and the WebSocket URL will now be ws://lamapalusa.com/tictactoe/websocket. We can close port 7070 now as everything will run over our proxy on port 80.

firewall-cmd --permanent --zone=public --remove-port=80/tcp
firewall-cmd --reload

The Nginx config will look something like this, you do not need to understand everything this is just for context. Look at the server {} block with the server name lamapalusa.com. And note also the location <some_url> {} blocks.

### ... more nginx config
server {
   server_name lamapalusa.com;

   ### location block for the websocket
   location /tictactoe/websocket {
        proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_pass http://127.0.0.1:7070/websocket;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_next_upstream error timeout http_502 http_503 http_504;
    }


  
  location /tictactoe {
         proxy_set_header Host $http_host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_pass http://127.0.0.1:7070/;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_next_upstream error timeout http_502 http_503 http_504;
  }

}

### ... more nginx config

Use a proxy to configure SSL for secure http and secure WebSocket connection

An additional advantage is that we can configure an SSL certificate in our webserver and only allow secure connections with https or wss. We will have to change our URLs to https://lamapalusa.com and for our WebSocket connection wss://lamapalusa.com/websocket. Https is on port 443 or 8443 so our firewall will need to let those through.

If we don’t do this http or ws connection will be unencrypted and potentially everyone in the middle of the connection can read our messages.

Without a proxy, every service you run on each port would need to support and be configured so that it uses SSL connections. This is cumbersome and configuring SSL certificates once in our proxy is very convenient.

For configuring our certificate we can use letsencrypt which is a nonprofit service that allows you to quickly get and set up an SSL certificate.

We can use a tool called certbot which will fetch and save the certificate on the filesystem of our server together with a Python library called python3-certbot-nginx which can actually modify your existing nginx proxy configuration and add the required lines which is handy.

dnf install certbot python3-certbot-nginx

Then you can run certbot and it will interactively ask a few things you need to configure:

certbot --nginx -d lamapalusa.com

certbot will also automatically refresh the certificate as this is only valid a little more than a year.

Multiple applications and usages with a single proxy

1. If the domain of the URL is lamapalusa.com and the URL path is /tictactoe, then our proxy will allow websocket connections to the game server. So wss://lamapalusa.com/websocket would be used by the client. Note the additional ‘s’ within wss, as we now have a secure websocket connection using SSL.

2. If the domain of the URL i lamapalusa.com and the URL path is / or anything else besides /tictactoe the / location part will match and the website for our tic tac toe game is served. Note that on our website we cannot have a page with link /websocket as otherwise the /websocket location directive of Nginx would match on no site would be returned as this location is only for the WebSocket. So navigating to the URL https://lamapalusa.com in the browser will render a html page.

3. Having a server is also nice as you can provide download links quite easily also using the Nginx proxy. Nginx has a feature called try_files which will return files in a folder based on the URL of the request. If the path in the URL matches a file inside the folder it will be returned and the download will start at the client. However, in this case, we do not want to make this public but only allow a few selected users to download something. This can be done with Auth Basic. Your browser will even ask you to provide your credentials for the basic authentication. Your server will have a list of users and their hashed passwords configured and it will check if the client’s credentials match and only then allow the connection to go through. A proxy also allows you to restrict access to specific URLs which you couldn’t easily do when they weren’t behind the proxy. You can perform quite fine-grained authentication and authorization for specific URLs or require certain authentication mechanisms. It can also restrict certain IP addresses or only allow a list of IP addresses of a specific range. Therefore a proxy gives you a lot of configurability who can access what. For example, if a user navigates to https://lamapalusa.com/download/folder1/subfolder2/file1.zip our proxy will match server_name lamapalusa.com then the /download location part would match and the rest of the path folder1/subfolder2/file1.zip would be used to look up a file in the given folder. So if you serve the folder /var/www/download it would check is there a file with the relative path ./folder1/subfolder2/file1.zip, so if the file /var/www/download/folder1/subfolder2/file1.zip exists it is returned in the download.

4. We can also use the Nginx to handle multiple domains or subdomains. If DNS is properly configured so that the new domain blog.lamapalusa.com points to the Gandalf server then we can add a server block for the server_name blog.lamapalusa.com and create a location directive / which matches everything and then returns a static html website when accessed.

5. A proxy can also forward requests to another server, for example within an internal network that has no other access to the outside. Here the domain funkylamachat.com has a Nginx entry and the proxy will forward the request to another server within our network which will then hand the request to the chat server application. We also could have added running the chat server on the lamapalusa.com domain and added a location /chat.

Transparency and clear documentation

When setting up a server you will configure quite a few things. When you just expose ports and run applications that bind to them it will be hard for someone else or yourself at a later time to see what actually runs where. Having a proxy such as Nginx makes everything very transparent. If you want to see what kind of services are exposed you can just check the Nginx proxy configuration and see which services receive connections through which paths or domain names.

Having such a configuration acts directly as documentation and is much more concrete and explicit. Everything is written down nicely and someone else can directly understand the system to some degree when checking this file.

URLs with context path

It is important to note that you usually develop applications and use paths relative to the root path /. However, if your application is served under some URL usually called context path such as the tic-tac-toe server receiving requests only on lamapalusa.com/tictactoe it will need to be told that there is some context path.

Our proxy will not pass the location path to the server so /tictactoe will be cut from the URL, so for the app this is nice as it doesn’t matter which base path is used. If we change in our proxy that the app now runs under lamapalusa.com/game nothing would change in the application as it only receives the part after /game path.

However, this can become a problem if your app uses some absolute paths. If the game creates an URL such as lamapalusa.com/downloadGame, our Nginx location directive /tictactoe would not match. The URL created should have included the context path, lamapalusa.com/tictactoe/downloadGame.

Most software that you can host yourself (game servers, webframeworks, wikis) usually allows passing along both the domain name and the context path to the application. So it will be aware if it runs under a certain context path and can include this in the link generation or any absolute paths. However, we can pass the domain name and context path from outside dynamically and it is not hard coded. So the application would still work as if it is run under the root path and the context path is still removed before the rest of the path is handed to the app. The context path would only be included again in link generation.

If you write an application yourself that does not use a framework that handles this for you make sure that you support this by either reading these values from an environment variable, or configuration file or by accepting these parameters in command line arguments.

Also, our game currently has hardcoded that it will connect to port 7070. It is good practice to make this configurable as well with a command line argument or similar. Otherwise (without using docker which would solve this problem) you cannot run two different services that by chance use the same port. So consider allowing port configuration as well.

So running our server would now look like this:

java -jar GameServer.jar --port 9090 --domain-name=lamapalusa.com --context-path=/tictactoe

If you don’t want to deal with context paths or the application you want to host doesn’t allow you to configure them you can simply use a subdomain for the app which would put its paths at the root again. Choose some new subdomain you will only use for this purpose. So you can add a server block just for this specific application and then a location directive that matches everything and just proxy pass to the program.

Summary

Having a proxy on your own VPS gives you a lot of flexibility to configure your own services and map paths to them. You can restrict access to certain paths by using basic auth, the Nginx config file directly acts as system documentation and it gives you a uniform way to configure different kinds of services. However, today there are alternative approaches that manage some parts of this for you which will not require you to set up your own server and learn technologies such as a firewall or any kind of proxy configuration.

Also for running applications there are ways you can separate them cleanly from the host system using docker containers for example. In our example, we just run the game server as a Java application on the machine. With docker containers, each program can have its own container with the things installed, it needs.

If you want to host static content maybe check out something like Netlify, if you want to just deploy an application without configuring a full server Digitalocean for example has ways to directly deploy an application from a git repo. However, in both cases, you will need to configure DNS.

I would still recommend learning how to set up a server as it is a nice learning experience and gives you some flexibility for your own needs.