Setting up a Web Server Using Docker on Oracle Cloud Infrastructure Ampere arm64 Always Free Instance
After successfully migrating from a #DigitalOcean Droplet for my #Mastodon instance I decided to take advantage of the remaining resources still available on #OracleCloud to migrate my web hosting as well. I’m going to be taking a different approach, basically because of the #arm64 setup, and using #Docker for running most of my hosting setup.
Create New OCI Instance
In the event you want to test out an instance, and reuse the associated IP address without having to update public DNS records, you should first create a reserved IP address. Then during the instance creation choose not to assign a public IPv4 address as you’ll assign it after the instance is created.
- Create a new #Ampere arm64 instance with as many resources as you want.
- Select #Ubuntu arm64 as the OS. (Note: You can choose a minimal Ubuntu system to keep things lean but will require the addition of other packages. This is my preference so this guide may include installation steps that aren’t needed if you choose a full install.)
- Edit settings as you see fit, choose to not assign a public IPv4 address is using an existing reserverd IP address. (See note above)
Install Some Required Packages
I would recommend installing the dialog
package in order to provide console screens for package configuration.
$ sudo apt install dialog
I prefer to setup my system with #Neovim so that is usually one of the first packages I install to use to edit configuration files during the setup.
$ sudo apt install neovim
After these updates you will most likely be instructed to reboot the server.
$ sudo reboot
Install #Nala and Update System
Recently my preference for system package management has been the nala
package manager.
$ sudo apt update
$ sudo apt install nala
Per the nala
installation instructions you can setup nala to fetch from the most responsive mirrors closest to you.
$ sudo nala fetch
Afterwards you can use nala
to update the system.
$ sudo nala update
$ sudo nale upgrade
Afterwards you should perform a system reboot.
$ sudo reboot
Note: I will be using
nala
for package management in most cases throughout the rest of this guide.
Install Latest Docker Engine
Docker is included in many distributions but I want to make sure that the latest versions are being installed directly from Docker for continual updates. You can follow the official documentation for installing on Ubuntu and Debian-based distributions.
$ sudo nala remove docker docker-engine docker.io containerd runc
Note: It is OK if this step indicates that there is nothing to remove.
Install Using The Repository
First install Docker required packages.
$ sudo nala install ca-certificates curl gnupg lsb-release
Note: It is OK if this step results in no new packages added.
Setup The Repository
$ sudo mkdir -p /etc/apt/keyrings
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
$ echo \
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
Install Docker Engine
Check for package updates and install the Docker packages.
$ sudo chmod a+r /etc/apt/keyrings/docker.gpg
$ sudo nala update
$ sudo nala install docker-ce docker-ce-cli containerd.io docker-compose-plugin
After the Docker installation you’ll want to reboot the OCI instance.
$ sudo reboot
Basic Security Configuration
Installing And Configuring fail2ban
This is basically the same setup as I used for the Mastodon instance setup on OCI to setup #fail2ban.
Here’s a quick command rundown.
$ sudo nala install fail2ban
$ sudo vi /etc/fail2ban/jail.local
$ sudo reboot
Installing And Configuring The iptables
Linux Firewall
As mentioned in the above post the Linux firewall should already be setup with the Ubuntu system install. I did setup the IPv6 firewall rules to begin allowing traffic via IPv6. I used the initial Mastodon IPv6 configuration.
$ sudo vi /etc/iptables/rules.v6
# Allow all loopback (lo0) traffic and drop all traffic to 127/8 that doesn't use lo0
-A INPUT -i lo -j ACCEPT
-A INPUT ! -i lo -d ::1/128 -j REJECT
# Accept all established inbound connections
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# Allow all outbound traffic - you can modify this to only allow certain traffic
-A OUTPUT -j ACCEPT
# Allow HTTP and HTTPS connections from anywhere (the normal ports for websites and SSL).
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
# Allow SSH connections
# The -dport number should be the same port number you set in sshd_config
-A INPUT -p tcp -m state --state NEW --dport 22 -j ACCEPT
# Allow ping
-A INPUT -p icmpv6 -j ACCEPT
# Log iptables denied calls
-A INPUT -m limit --limit 5/min -j LOG --log-prefix "iptables denied: " --log-level 7
# Reject all other inbound - default deny unless explicitly allowed policy
-A INPUT -j REJECT
-A FORWARD -j REJECT
After the configuration edit, you just need to load it into iptables. I rebooted just to make sure everything was set.
$ sudo ip6tables-restore < /etc/iptables/rules.v6
$ sudo reboot
#Portainer Installation with Nginx Proxy Manager and SSL
Figuring out a setup to easily manage SSL, using Let’s Encrypt, as well as handle running services all on Docker took many iterations. If I was going to be using Docker to run OpenLiteSpeed then I didn’t need all of the overhead of many of the web control panels. I tried Cockpit and Virtualmin as I had either used them in the past or they came highly recommended. I came across some documentation on using the Nginx Proxy Manager and it seemed like the lightweight setup I was after. The nice thing about it is that the general setup of it is running it via Docker. I liked this idea as it meant I didn’t have to setup and maintain Nginx locally on the server instance.
Nginx Proxy Manager Installation
References:
- https://nginxproxymanager.com/guide/#quick-setup
- https://medium.com/@devops.ent/nginx-proxy-manager-how-to-installation-and-configuration-using-docker-portainer-1348c67a392e
- https://www.51sec.org/2022/08/12/deploy-docker-docker-compose-portainer-and-npm-nginx-proxy-manager/
- https://www.howtoforge.com/how-to-install-and-use-portainer-for-docker-management-with-nginx-proxy-manager/
$ sudo mkdir -p /var/lib/nginx-proxy-manager/data
$ sudo mkdir /etc/letsencrypt
$ sudo docker network create services-network
$ sudo vi /var/lib/nginx-proxy-manager/docker-compose.yml
docker-compose.yml
version: "3.5"
services:
npm:
image: 'jc21/nginx-proxy-manager:latest'
restart: unless-stopped
networks:
- default
ports:
# These ports are in format <host-port>:<container-port>
- '80:80' # Public HTTP Port
- '443:443' # Public HTTPS Port
# - '81:81' # Admin Web Port
# Add any other Stream port you want to expose
# - '21:21' # FTP
# Uncomment the next line if you uncomment anything in the section
# environment:
# Uncomment this if you want to change the location of
# the SQLite DB file within the container
# DB_SQLITE_FILE: "/data/database.sqlite"
# Uncomment this if IPv6 is not enabled on your host
# DISABLE_IPV6: 'true'
volumes:
- /var/lib/nginx-proxy-manager/data:/data
- /etc/letsencrypt:/etc/letsencrypt
- /var/lib/letsencrypt:/var/lib/letsencrypt
- /var/log/letsencrypt:/var/log/letsencrypt
networks:
default:
name: services-network
$ sudo docker compose -f /var/lib/nginx-proxy-manager/docker-compose.yml up -d
Add an OCI Ingress Rule for Access
You will need to set Ingress rules for your Oracle Virtual Cloud Network to temporarily allow web traffic for the Nginx Proxy Manager default port 81. The basic run down is as follows:
- Open the navigation menu and click Networking, and then click Virtual Cloud Networks.
- Select the VCN you created with your compute instance.
- With your new VCN displayed, click <your-subnet-name> subnet link.The public subnet information is displayed with the Security Lists at the bottom of the page. A link to the Default Security List for your VCN is displayed.
- Click the Default Security List link.The default Ingress Rules for your VCN are displayed.
- Click Add Ingress Rules.An Add Ingress Rules dialog is displayed.
- Fill in the ingress rule with the following information.Fill in the ingress rule as follows:
- Stateless: CheckedSource Type: CIDRSource CIDR: 0.0.0.0/0IP Protocol: TCPSource port range: (leave-blank)Destination Port Range: 81Description: Allow HTTP connections
Login to the Nginx Proxy Manager at http://<public-ip>:81
.
Default Admin User:
Email:
admin@example.com
Password:changeme
After you login update/create your admin account.
Secure the Nginx Proxy Manager via SSL
References:
- Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
- Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. admin.example.com)
- Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
- Enter the Docker service name(i.e.
npm
) of the Nginx Proxy Manager and port81
for the “Forward Port”. - Turn on “Block Common Exploits” and “Websockets Support”.
- Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
- Save the new Proxy Host settings.
You should now be able to login via the custom subdomain over SSL. At this point I would remove the OCI Ingress rule added to allow port 81 and restrict all access to only SSL.
Installing Portainer
References:
Preparing and Starting Portainer
$ sudo mkdir -p /var/lib/portainer/data
$ sudo vi /var/lib/portainer/docker-compose.yml
docker-compose.yml
version: "3.5"
services:
portainer:
image: portainer/portainer-ee:latest
command: -H unix:///var/run/docker.sock
restart: unless-stopped
networks:
- default
# ports:
# These ports are in format <host-port>:<container-port>
# - '8000:8000' # Public HTTP Port
# - '8080:9000' # Public HTTP Port
# - '9443:9443' # Public HTTPS Port
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- /var/lib/portainer/data:/data
networks:
default:
name: services-network
$ sudo docker compose -f /var/lib/portainer/docker-compose.yml up -d
Securing Portainer via SSL Using Nginx Proxy Manager
The first thing to do is to setup a new Proxy Host in the Nginx Proxy Manager for Portainer to create the SSL certificate.
- Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
- Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. containers.example.com)
- Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
- Enter the Docker service name(i.e.
portainer
) of the Portainer service and port9000
for the “Forward Port”. - Turn on “Block Common Exploits” and “Websockets Support”.
- Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
- Save the new Proxy Host settings.
Upgrading the Portainer Stack
In order to upgrade Portainer itself you’ll need to stop the Portainer stack, you can do this from the stack details screen in Portainer.
Pull the Latest Portainer Image
$ sudo docker pull portainer/portainer-ee:latest
Startup the Portainer Stack
$ sudo docker compose -f /var/lib/portainer/docker-compose.yml up -d
Installing a MariaDB Server With Docker Using Portainer
Now that Portainer is setup and running we’ll go ahead and get a #MariaDB database server running in order for the web servers, and #WordPress sites, use it. We will also additionally setup phpMyAdmin for web management.
References:
Create a Persistent Volume for the Database Storage
Navigate to “Volumes” and give it a name (i.e. mariadb
).
Create a new MariaDB Container
- Navigate to “Containers” and “Add container”.
- Give it a name (i.e.
mariadb
) - Enter
mariadb:10.10
for the image. - Map the server ports using “Manual network port publishing” using the standard
3306
port for both host and container. - Under the “Advanced container settings” got to “Volumes” and “map additional volume” using
/config
for the container and choose the previously created volume name from the volume dropdown menu. - Navigate to the “Network” section and set the network to the previously created
services-network
. - Then navigate to the “Env” section to define environment variables. Switch to “Advanced Mode” in order to easily add as many needed variables.
PUID=1000
PGID=1000
MYSQL_ROOT_PASSWORD=<password>
- Navigate to the “Restart policy” and set to “Unless stopped”.
- Finally click on “Deploy the container”
Setup a phpMyAdmin Service
- Navigate to “Containers” and “Add container”.
- Give it a name (i.e.
phpmyadmin
) - Enter
phpmyadmin:latest
for the image. - Map the server ports using “Manual network port publishing” using
8080
for the host port and80
for the container port. - Navigate to the “Network” section and set the network to the previously created
services-network
. - Then navigate to the “Env” section to define environment variables. Switch to “Advanced Mode” in order to easily add as many needed variables.
MYSQL_ROOT_PASSWORD=<password>
- Navigate to the “Restart policy” and set to “Unless stopped”.
- Finally click on “Deploy the container”
Securing the phpMyAdmin Service with Nginx Proxy Manager
- Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
- Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. db.example.com)
- Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
- Enter the Docker service name(i.e.
phpmyadmin
) of the phpMyAdmin server and port80
for the “Forward Port”. - Turn on “Block Common Exploits” and “Websockets Support”.
- Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
- Save the new Proxy Host settings.
Installing a Redis Server With Docker Using Portainer
This setup is going to leverage the Portainer “App Templates” feature to quickly spin up a #Redis instance.
- Navigate to “App Templates”.
- Search for “redis”.
- Select the “Redi” template.
- Give the new service a name (i.e.
redis
), depending on how you plan to use the instance. - Set the Network to
services-network
.
Optional
If you’d like to access the Redis server via tools outside of the Docker environment you’ll want to map a port to the host OCI instance. Click on “Show advanced options” to then set the host port to
6379
.
Lastly, click on “Deploy the container”.
Installing The OpenLiteSpeed Web Server With Docker Using Portainer
Preliminary Setup Work
We are going to manage the OpenLiteSpeed server data via the Docker host system similar to the setup of Nginx Proxy Manager and Portainer.
$ sudo mkdir /var/lib/openlitespeed
Create The OpenLiteSpeed Container in Portainer
For this setup we are going to use the docker-compose.yml
example from the LiteSpeedTech GitHub repository as our guide.
- Navigate to “Containers” and “Add container”.
- Give it a name (i.e.
openlitespeed
) - Enter
ghcr.io/ndigitals/openlitespeed:1.7.16-lsphp80
for the image. (note: Currently LiteSpeed doesn’t provide an official arm64 Docker image so I ended up building an image specifically for arm64. See: Supporting OpenLiteSpeed on arm64 with Docker) - Under the “Advanced container settings” go to “Volumes” and “map additional volume” and add “bin” mappings that follow the list as mentioned in the OLS example repository. The only difference is that we are going to prepend all of the host paths with
/var/lib/openlitespeed
. The following format is the Docker Compose format of host:container, however in Portainer they are listed in reverse./var/lib/openlitespeed/lsws/conf:/usr/local/lsws/.conf
/var/lib/openlitespeed/lsws/admin-conf:/usr/local/lsws/admin/.conf
/var/lib/openlitespeed/sites:/var/www/vhosts
/etc/letsencrypt:/root/.acme.sh
(note: mark this one Read-only, since the Nginx Proxy Manager takes care of managing the certificates.)/var/lib/openlitespeed/logs:/usr/local/lsws/logs
- Navigate to the “Network” section and set the network to the previously created
services-network
. - Then navigate to the “Env” section to define environment variables. Switch to “Advanced Mode” in order to easily add as many needed variables.
TZ=America/New_York
(note: you’ll want to set this to an appropriate timezone for your location.)
- Navigate to the “Restart policy” and set to “Unless stopped”.
- Finally click on “Deploy the container”
Securing the OpenLiteSpeed Web Console with Nginx Proxy Manager
- Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
- Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. ols.example.com)
- Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
- Enter the Docker service name(i.e.
openlitespeed
) of the OpenLiteSpeed service andhttps
set for the Scheme as well as port7080
for the “Forward Port”. - Turn on “Block Common Exploits” and “Websockets Support”.
- Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
- Save the new Proxy Host settings.
Securing the OpenLiteSpeed Web Server Primary Domain Name with Nginx Proxy Manager
- Navigate to the Nginx Proxy Manager “Host” section and choose to “Add Proxy Host”.
- Enter the subdomain you’d like to use to access the Nginx Proxy Manager. (i.e. example.com & www.example.com)
- Note: ensure that this domain name is configured via your DNS services to point to the OCI instance IP address or as a CNAME record.
- Enter the Docker service name(i.e.
openlitespeed
) of the OpenLiteSpeed service and port8088
for the “Forward Port”. - Turn on “Block Common Exploits” and “Websockets Support”.
- Under the “SSL” section choose to “Request a new SSL Certificate” and “Agree to the Let’s Encrypt Terms of Service”.
- Save the new Proxy Host settings.
Additional Resources
Many of the follow-up items I listed at the end of the Mastodon on OCI setup guide were also applied here. In addition to those items here are some more things to consider:
- Portainer Authenticated Docker Hub Registry Access – In order to address public Docker Hub rate limiting when pulling Docker images via Portainer you can setup authenticated access. There are still limits when using a free Docker Hub account. (see official Portainer documentation also)
- Real IP Addresses in OLS Behind Nginx Proxy – In order to address issues of only getting the Docker network IP addresses when OLS is behind the Nginx Proxy you need to set Use Client IP in Header to Yes under the OpenLiteSpeed WebAdmin Console > Server Configuration > General Settings.
- Server Update Notifications – In order to ensure that the underlying instance is kept secure it is important that updates, especially security updates, are applied in a timely manager. Apticron is a tool that can be used to send an email to notify you of available updates.
- After the package installation you’ll want to copy
/usr/lib/apticron/apticron.conf
to/etc/apticron/apticron.conf
and make the required changes.
- After the package installation you’ll want to copy
- Keycloak Docker Container Setup – I had previously setup a Keycloak container under CyberPanel to do some OpenID Connect testing, getting this running under the new setup proved to be challenging since it had been so long ago that I had setup the old one.
- Running Keycloak in a container
- Configuring the database – I chose this time to have the Keycloak instance use my existing MariaDB instance.
- JDBC Database URL Format – It had been some time since messing with Java & JDBC and so I wasn’t sure what the URL format Keycloak needed was so this was helpful.
- Run in Docker behind Nginx Reverse Proxy – I had some challenges with the configuration of the latest Keycloak version and this helped me identify all of the bits and bobs I needed to get things running.
- Low Disk Space Notifications – OCI doesn’t appear to have alarm monitoring for disk usage, additionally for monitoring an instance you are limited to only 2 alarms on the free tier. So some internal server monitoring setup is required.
- Send Email Notifications – This was the starting point for my cron job to monitor disk usage.
- Send Slack Notifications – I leveraged this to also send Slack notifications along with the email notifications in my cron job.
@tim I’ll keep this one handy! Been thinking about testing out Oracle Cloud. Free things lime this make me a bit nervous… Though I do wonder about using it for small client sites.
@Sergio well, this is for just personal stuff. I haven’t priced out the paid services compared to #GCP or #AWS but I might have to see how they compare if #Oracle pulls a #Docker and tells me to pay up or ship out.
aws
docker
gcp
oracle