Rancher server & Rancher agent in high availability on local environment with Vagrant & Virtualbox

Touch4IT Team is responsible for development, hosting and maintaining tens of mobile and web applications. We want to give our clients a feeling that if anything happens, their app will still be up and running. Hosting such a number of projects is not an easy task. Web applications and app backends are built using different technologies, requires different libraries and different versions of these libraries to run properly.

That is exactly why all of these apps and backends are running on Docker. But when you run so many apps on Docker, you need to be able to deploy new versions and new apps without any outages. We also need to have multiple environments for development, staging and production. That is where Docker orchestrators come in handy. There are many orchestrators available, but we are not going to do any comparison here, there are many great articles that covers Docker orchestration overview.

We decided to use Rancher 1.6, which uses Docker orchestrator Cattle, for following reasons:

• simple, yet very functional UI
• working documentation
• production-grade stability
• ease of learn & use
• support with rancher-compose
• ability to deploy containers using CI tools (Gitlab CI) 
• failover and rollback to previous version in case of error
• works on bare metal
• capable of scaling to multiple virtual/physical hosts
• works in high availability

In this article, we are giving you a fully working Rancher playground running in high availability. Let’s have a look at the image below to see how the playground is set up.

We have 2 virtual machines where rancher-server, racher-agent and applications stacks are running. Rancher server is a docker container that runs Rancher UI, API, Cattle orchestrator and basically all the administrative tasks. Rancher agent is a Docker container that registers virtual or physical host where it is running into Rancher server environment.

For the purposes of this playground, we will run 2 server instances in high availability by having both instances on separate virtual machines, 2 agents to have 2 hosts where apps will be deployed twice in case one of the hosts comes down. In the real world you would need to put a load balancer in front of these virtual machines to direct http traffic on ports 80 and 443 to both hosts, have health checks configured and stop sending traffic to host if it is down.

On the mysql-nginx virtual machine we have Nginx, which serves as load balancer for Rancher server and its UI (green arrows). There is also a MySQL database where Rancher server stores configuration of rancher and also application stacks (red arrows). Then again, in real world, you should setup a MySQL Cluster or other form of MySQL replication and have multiple load balancers to achieve high availability status. Rancher agent is communicating with Rancher server through our Nginx proxy (purple arrows).

To run the playground on your local machine, you need to install Git, Vagrant, Virtualbox and Rancher compose. We don’t want you to spend time setting up virtual machines, they are set up for you with Ansible. Check our GitHub repository for more information about how that is done.

Let’s clone our GitHub repository to get the playground source code:

 

 

Add testing domain name to your /etc/hosts file:

 

 

Start all 3 virtual machines, this step will take up to 10 minutes.

 

 

 

 

 

 

After all scripts are finished, Rancher server instances need a couple of minutes to initialize MySQL database and decide which one will become master. When this is done you will be able to visit Rancher UI in your browser on http://rancher.vm.localhost, see screenshots below:
 

 

 

We need to create and start Rancher agents to make both of our virtual machines connect to Rancher server “Default” environment. Click on “Add a host” button, set “Host Registration URL” to http://192.168.70.10 — that is the IP address of our virtual machine running Nginx proxy. Click “Save”.
 

 

Copy “docker run …” command from point 5, screenshot below:

Note: the hash at the end of Rancher agent “docker run” command is different for every Rancher server installation, that’s why it is not automatically done for you.
 

 

Run the command you copied in previous step “docker run …” on both of our Rancher virtual machines: rancher1 and rancher2

 

 

 

 

 

Go to http://rancher.vm.localhost/env/1a5/infra/hosts. It will take up to 10 minutes until both hosts become available and all stacks will be green. It should look as on the screenshot below:
 

 

We need to create environment API key and configure Rancher compose.

Go to http://rancher.vm.localhost/env/1a5/api/keys and follow screenshots below to create new environment API key:

Note: don’t forget to save your new API key and secret, you will need it in next step.
 

 

 

 

 

Let’s set environment variables for rancher compose:

Replace <API_KEY> and <API_SECRET> with your key and secret.

 

 

 

Deploy test Nginx stack to Rancher:

 

 

 

 

When you go to http://rancher.vm.localhost/env/1a5/apps/stacks, you will see your test stack:
 

 

Click on the “test-nginx” service name to edit it. Let’s scale the service up by click in plus button on the left side of the screen. That will start another Nginx container. There will be one on each host. You will see:
 

 

You can visit http://192.168.70.11 and http://192.168.70.12 which will give you the default Nginx welcome page as you can see on the screenshots below:
 

 

Congratulations! You have working playground with Rancher server and Rancher agent in high availability. Test as much as you like. If you break something, do ‘vagrant destroy’ and start over in a no time.