Setting up a Web Server Using Docker on Oracle Cloud Infrastructure Ampere arm64 Always Free Instance

After successfully migrating from a #DigitalOcean Droplet for my #Mastodon instance I decided to take advantage of the remaining resources still available on #OracleCloud to migrate my web hosting as well. I’m going to be taking a different approach, basically because of the #arm64 setup, and using #Docker for running most of my hosting setup.

Create New OCI Instance

In the event you want to test out an instance, and reuse the associated IP address without having to update public DNS records, you should first create a reserved IP address. Then during the instance creation choose not to assign a public IPv4 address as you’ll assign it after the instance is created.

1. Create a new #Ampere arm64 instance with as many resources as you want.
2. Select #Ubuntu arm64 as the OS. (Note: You can choose a minimal Ubuntu system to keep things lean but will require the addition of other packages. This is my preference so this guide may include installation steps that aren’t needed if you choose a full install.)
3. Edit settings as you see fit, choose to not assign a public IPv4 address is using an existing reserverd IP address. (See note above)

Install Some Required Packages

I would recommend installing the dialog package in order to provide console screens for package configuration.

$ sudo apt install dialog

I prefer to setup my system with #Neovim so that is usually one of the first packages I install to use to edit configuration files during the setup.

$ sudo apt install neovim

After these updates you will most likely be instructed to reboot the server.

$ sudo reboot

Install #Nala and Update System

Recently my preference for system package management has been the nala package manager.

$ sudo apt update
$ sudo apt install nala

Per the nala installation instructions you can setup nala to fetch from the most responsive mirrors closest to you.

$ sudo nala fetch

Afterwards you can use nala to update the system.

$ sudo nala update
$ sudo nale upgrade

Afterwards you should perform a system reboot.

$ sudo reboot

Note: I will be using nala for package management in most cases throughout the rest of this guide.

Install Latest Docker Engine

Docker is included in many distributions but I want to make sure that the latest versions are being installed directly from Docker for continual updates. You can follow the official documentation for installing on Ubuntu and Debian-based distributions.

$ sudo nala remove docker docker-engine docker.io containerd runc

Note: It is OK if this step indicates that there is nothing to remove.

Install Using The Repository

First install Docker required packages.

$ sudo nala install ca-certificates curl gnupg lsb-release

Note: It is OK if this step results in no new packages added.

Setup The Repository

$ sudo mkdir -p /etc/apt/keyrings
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg

$ echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

Install Docker Engine

Check for package updates and install the Docker packages.

$ sudo chmod a+r /etc/apt/keyrings/docker.gpg
$ sudo nala update
$ sudo nala install docker-ce docker-ce-cli containerd.io docker-compose-plugin

After the Docker installation you’ll want to reboot the OCI instance.

$ sudo reboot

Basic Security Configuration

Installing And Configuring fail2ban

This is basically the same setup as I used for the Mastodon instance setup on OCI to setup #fail2ban.

Here’s a quick command rundown.

$ sudo nala install fail2ban
$ sudo vi /etc/fail2ban/jail.local
$ sudo reboot

Installing And Configuring The iptables Linux Firewall

As mentioned in the above post the Linux firewall should already be setup with the Ubuntu system install. I did setup the IPv6 firewall rules to begin allowing traffic via IPv6. I used the initial Mastodon IPv6 configuration.

$ sudo vi /etc/iptables/rules.v6

#  Allow all loopback (lo0) traffic and drop all traffic to 127/8 that doesn't use lo0
-A INPUT -i lo -j ACCEPT
-A INPUT ! -i lo -d ::1/128 -j REJECT

#  Accept all established inbound connections
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

#  Allow all outbound traffic - you can modify this to only allow certain traffic
-A OUTPUT -j ACCEPT

#  Allow HTTP and HTTPS connections from anywhere (the normal ports for websites and SSL).
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT

#  Allow SSH connections
#  The -dport number should be the same port number you set in sshd_config
-A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT

#  Allow ping
-A INPUT -p icmpv6 -j ACCEPT

#  Log iptables denied calls
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7

#  Reject all other inbound - default deny unless explicitly allowed policy
-A INPUT -j REJECT
-A FORWARD -j REJECT

After the configuration edit, you just need to load it into iptables. I rebooted just to make sure everything was set.

$ sudo ip6tables-restore < /etc/iptables/rules.v6
$ sudo reboot

#Portainer Installation with Nginx Proxy Manager and SSL

Figuring out a setup to easily manage SSL, using Let’s Encrypt, as well as handle running services all on Docker took many iterations. If I was going to be using Docker to run OpenLiteSpeed then I didn’t need all of the overhead of many of the web control panels. I tried Cockpit and Virtualmin as I had either used them in the past or they came highly recommended. I came across some documentation on using the Nginx Proxy Manager and it seemed like the lightweight setup I was after. The nice thing about it is that the general setup of it is running it via Docker. I liked this idea as it meant I didn’t have to setup and maintain Nginx locally on the server instance.

Nginx Proxy Manager Installation

References:

• https://nginxproxymanager.com/guide/#quick-setup
• https://medium.com/@devops.ent/nginx-proxy-manager-how-to-installation-and-configuration-using-docker-portainer-1348c67a392e
• https://www.51sec.org/2022/08/12/deploy-docker-docker-compose-portainer-and-npm-nginx-proxy-manager/
• https://www.howtoforge.com/how-to-install-and-use-portainer-for-docker-management-with-nginx-proxy-manager/

$ sudo mkdir -p /var/lib/nginx-proxy-manager/data
$ sudo mkdir /etc/letsencrypt
$ sudo docker network create services-network
$ sudo vi /var/lib/nginx-proxy-manager/docker-compose.yml

docker-compose.yml

version: "3.5"
services:
  npm:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    networks:
     - default
    ports:
      # These ports are in format <host-port>:<container-port>
      - '80:80' # Public HTTP Port
      - '443:443' # Public HTTPS Port
      # - '81:81' # Admin Web Port
      # Add any other Stream port you want to expose
      # - '21:21' # FTP

    # Uncomment the next line if you uncomment anything in the section
    # environment:
      # Uncomment this if you want to change the location of 
      # the SQLite DB file within the container
      # DB_SQLITE_FILE: "/data/database.sqlite"

      # Uncomment this if IPv6 is not enabled on your host
      # DISABLE_IPV6: 'true'

    volumes:
      - /var/lib/nginx-proxy-manager/data:/data
      - /etc/letsencrypt:/etc/letsencrypt
      - /var/lib/letsencrypt:/var/lib/letsencrypt
      - /var/log/letsencrypt:/var/log/letsencrypt

networks:
 default:
   name: services-network

$ sudo docker compose -f /var/lib/nginx-proxy-manager/docker-compose.yml up -d

Add an OCI Ingress Rule for Access

You will need to set Ingress rules for your Oracle Virtual Cloud Network to temporarily allow web traffic for the Nginx Proxy Manager default port 81. The basic run down is as follows:

1. Open the navigation menu and click Networking, and then click Virtual Cloud Networks.
2. Select the VCN you created with your compute instance.
3. With your new VCN displayed, click <your-subnet-name> subnet link.The public subnet information is displayed with the Security Lists at the bottom of the page. A link to the Default Security List for your VCN is displayed.
4. Click the Default Security List link.The default Ingress Rules for your VCN are displayed.
5. Click Add Ingress Rules.An Add Ingress Rules dialog is displayed.
6. Fill in the ingress rule with the following information.Fill in the ingress rule as follows:
   • Stateless: CheckedSource Type: CIDRSource CIDR: 0.0.0.0/0IP Protocol: TCPSource port range: (leave-blank)Destination Port Range: 81Description: Allow HTTP connections
   Click Add Ingress Rules. Now HTTP connections are allowed. Your VCN is configured for Apache server.

Login to the Nginx Proxy Manager at http://<public-ip>:81.

Default Admin User:

Email: admin@example.com
Password: changeme

After you login update/create your admin account.

Secure the Nginx Proxy Manager via SSL

References:

• https://github.com/NginxProxyManager/nginx-proxy-manager/issues/139

• Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
• Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. admin.example.com)
  • Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
• Enter the Docker service name(i.e. npm) of the Nginx Proxy Manager and port 81 for the “Forward Port”.
• Turn on “Block Common Exploits” and “Websockets Support”.
• Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
• Save the new Proxy Host settings.

You should now be able to login via the custom subdomain over SSL. At this point I would remove the OCI Ingress rule added to allow port 81 and restrict all access to only SSL.

Installing Portainer

References:

• https://docs.portainer.io/start/install/server/docker/linux
• https://gusiol.medium.com/nginx-proxy-and-portainer-multiple-applications-in-one-domain-d82efec0750f
• https://www.freecodecamp.org/news/docker-nginx-letsencrypt-easy-secure-reverse-proxy-40165ba3aee2/

Preparing and Starting Portainer

$ sudo mkdir -p /var/lib/portainer/data
$ sudo vi /var/lib/portainer/docker-compose.yml

docker-compose.yml

version: "3.5"
services:
  portainer:
    image: portainer/portainer-ee:latest
    command: -H unix:///var/run/docker.sock
    restart: unless-stopped
    networks:
      - default
    # ports:
      # These ports are in format <host-port>:<container-port>
      # - '8000:8000' # Public HTTP Port
      # - '8080:9000' # Public HTTP Port
      # - '9443:9443' # Public HTTPS Port

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/lib/portainer/data:/data

networks:
 default:
   name: services-network

$ sudo docker compose -f /var/lib/portainer/docker-compose.yml up -d

Securing Portainer via SSL Using Nginx Proxy Manager

The first thing to do is to setup a new Proxy Host in the Nginx Proxy Manager for Portainer to create the SSL certificate.

• Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
• Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. containers.example.com)
  • Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
• Enter the Docker service name(i.e. portainer) of the Portainer service and port 9000 for the “Forward Port”.
• Turn on “Block Common Exploits” and “Websockets Support”.
• Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
• Save the new Proxy Host settings.

Upgrading the Portainer Stack

In order to upgrade Portainer itself you’ll need to stop the Portainer stack, you can do this from the stack details screen in Portainer.

Pull the Latest Portainer Image

$ sudo docker pull portainer/portainer-ee:latest

Startup the Portainer Stack

$ sudo docker compose -f /var/lib/portainer/docker-compose.yml up -d

Installing a MariaDB Server With Docker Using Portainer

Now that Portainer is setup and running we’ll go ahead and get a #MariaDB database server running in order for the web servers, and #WordPress sites, use it. We will also additionally setup phpMyAdmin for web management.

References:

• https://hub.docker.com/_/mariadb
• https://www.youtube.com/watch?v=zyGhnbyOq90&t=12s

Create a Persistent Volume for the Database Storage

Navigate to “Volumes” and give it a name (i.e. mariadb).

Create a new MariaDB Container

1. Navigate to “Containers” and “Add container”.
2. Give it a name (i.e. mariadb)
3. Enter mariadb:10.10 for the image.
4. Map the server ports using “Manual network port publishing” using the standard 3306 port for both host and container.
5. Under the “Advanced container settings” got to “Volumes” and “map additional volume” using /config for the container and choose the previously created volume name from the volume dropdown menu.
6. Navigate to the “Network” section and set the network to the previously created services-network.
7. Then navigate to the “Env” section to define environment variables. Switch to “Advanced Mode” in order to easily add as many needed variables.
   • PUID=1000
   • PGID=1000
   • MYSQL_ROOT_PASSWORD=<password>
8. Navigate to the “Restart policy” and set to “Unless stopped”.
9. Finally click on “Deploy the container”

Setup a phpMyAdmin Service

1. Navigate to “Containers” and “Add container”.
2. Give it a name (i.e. phpmyadmin)
3. Enter phpmyadmin:latest for the image.
4. Map the server ports using “Manual network port publishing” using 8080 for the host port and 80 for the container port.
5. Navigate to the “Network” section and set the network to the previously created services-network.
6. Then navigate to the “Env” section to define environment variables. Switch to “Advanced Mode” in order to easily add as many needed variables.
   • MYSQL_ROOT_PASSWORD=<password>
7. Navigate to the “Restart policy” and set to “Unless stopped”.
8. Finally click on “Deploy the container”

Securing the phpMyAdmin Service with Nginx Proxy Manager

• Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
• Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. db.example.com)
  • Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
• Enter the Docker service name(i.e. phpmyadmin) of the phpMyAdmin server and port 80 for the “Forward Port”.
• Turn on “Block Common Exploits” and “Websockets Support”.
• Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
• Save the new Proxy Host settings.

Installing a Redis Server With Docker Using Portainer

This setup is going to leverage the Portainer “App Templates” feature to quickly spin up a #Redis instance.

• Navigate to “App Templates”.
• Search for “redis”.
• Select the “Redi” template.
• Give the new service a name (i.e. redis), depending on how you plan to use the instance.
• Set the Network to services-network.

Optional

If you’d like to access the Redis server via tools outside of the Docker environment you’ll want to map a port to the host OCI instance. Click on “Show advanced options” to then set the host port to 6379.

Lastly, click on “Deploy the container”.

Installing The OpenLiteSpeed Web Server With Docker Using Portainer

Preliminary Setup Work

We are going to manage the OpenLiteSpeed server data via the Docker host system similar to the setup of Nginx Proxy Manager and Portainer.

$ sudo mkdir /var/lib/openlitespeed

Create The OpenLiteSpeed Container in Portainer

For this setup we are going to use the docker-compose.yml example from the LiteSpeedTech GitHub repository as our guide.

1. Navigate to “Containers” and “Add container”.
2. Give it a name (i.e. openlitespeed)
3. Enter ghcr.io/ndigitals/openlitespeed:1.7.16-lsphp80 for the image. (note: Currently LiteSpeed doesn’t provide an official arm64 Docker image so I ended up building an image specifically for arm64. See: Supporting OpenLiteSpeed on arm64 with Docker)
4. Under the “Advanced container settings” go to “Volumes” and “map additional volume” and add “bin” mappings that follow the list as mentioned in the OLS example repository. The only difference is that we are going to prepend all of the host paths with /var/lib/openlitespeed. The following format is the Docker Compose format of host:container, however in Portainer they are listed in reverse.
   • /var/lib/openlitespeed/lsws/conf:/usr/local/lsws/.conf
   • /var/lib/openlitespeed/lsws/admin-conf:/usr/local/lsws/admin/.conf
   • /var/lib/openlitespeed/sites:/var/www/vhosts
   • /etc/letsencrypt:/root/.acme.sh (note: mark this one Read-only, since the Nginx Proxy Manager takes care of managing the certificates.)
   • /var/lib/openlitespeed/logs:/usr/local/lsws/logs
5. Navigate to the “Network” section and set the network to the previously created services-network.
6. Then navigate to the “Env” section to define environment variables. Switch to “Advanced Mode” in order to easily add as many needed variables.
   • TZ=America/New_York (note: you’ll want to set this to an appropriate timezone for your location.)
7. Navigate to the “Restart policy” and set to “Unless stopped”.
8. Finally click on “Deploy the container”

Securing the OpenLiteSpeed Web Console with Nginx Proxy Manager

• Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
• Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. ols.example.com)
  • Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
• Enter the Docker service name(i.e. openlitespeed) of the OpenLiteSpeed service and https set for the Scheme as well as port 7080 for the “Forward Port”.
• Turn on “Block Common Exploits” and “Websockets Support”.
• Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
• Save the new Proxy Host settings.

Securing the OpenLiteSpeed Web Server Primary Domain Name with Nginx Proxy Manager

• Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
• Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. example.com & www.example.com)
  • Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
• Enter the Docker service name(i.e. openlitespeed) of the OpenLiteSpeed service and port 8088 for the “Forward Port”.
• Turn on “Block Common Exploits” and “Websockets Support”.
• Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
• Save the new Proxy Host settings.

Additional Resources

Many of the follow-up items I listed at the end of the Mastodon on OCI setup guide were also applied here. In addition to those items here are some more things to consider:

• Portainer Authenticated Docker Hub Registry Access – In order to address public Docker Hub rate limiting when pulling Docker images via Portainer you can setup authenticated access. There are still limits when using a free Docker Hub account. (see official Portainer documentation also)
• Real IP Addresses in OLS Behind Nginx Proxy – In order to address issues of only getting the Docker network IP addresses when OLS is behind the Nginx Proxy you need to set Use Client IP in Header to Yes under the OpenLiteSpeed WebAdmin Console > Server Configuration > General Settings.
• Server Update Notifications – In order to ensure that the underlying instance is kept secure it is important that updates, especially security updates, are applied in a timely manager. Apticron is a tool that can be used to send an email to notify you of available updates.
  • After the package installation you’ll want to copy /usr/lib/apticron/apticron.conf to /etc/apticron/apticron.conf and make the required changes.
• Keycloak Docker Container Setup – I had previously setup a Keycloak container under CyberPanel to do some OpenID Connect testing, getting this running under the new setup proved to be challenging since it had been so long ago that I had setup the old one.
  • Running Keycloak in a container
  • Configuring the database – I chose this time to have the Keycloak instance use my existing MariaDB instance.
    • JDBC Database URL Format – It had been some time since messing with Java & JDBC and so I wasn’t sure what the URL format Keycloak needed was so this was helpful.
  • Run in Docker behind Nginx Reverse Proxy – I had some challenges with the configuration of the latest Keycloak version and this helped me identify all of the bits and bobs I needed to get things running.
• Low Disk Space Notifications – OCI doesn’t appear to have alarm monitoring for disk usage, additionally for monitoring an instance you are limited to only 2 alarms on the free tier. So some internal server monitoring setup is required.
  • Send Email Notifications – This was the starting point for my cron job to monitor disk usage.
  • Send Slack Notifications – I leveraged this to also send Slack notifications along with the email notifications in my cron job.