Openshift 4.x — Automation of UPI deployment by Ansible

Why this article?

Sometimes customers have on-premises disconnected private cloud with containerized applications and their vision to provide to multiple edge locations the same cloud environments and the same experience like in central location — cloudlets. Cloudlets not always has a connection to the central cloud so they should be able to live autonomously where connection to the center broken and be able to reconnect once connection restored. In this way containerized applications would be deployed from a central location to the edge locations and even to IOT equipment. In the past customers lost money by managing edge manually, it was error prone, unreliable, expensive and slow and this solution can help to decrease dramatically costs and to minimize risks. On the other hand solution allows customers to start with IOT — new business avenues that were economically unviable and now are due to this solution it is available — increase revenue. This solution addresses this by creating a repeatable solution in an automated infra-as-code way, secure (no humans on machines), secure (RBAC in Tower) — Risk mitigation.

To achieve those goals our customer asked us to provide solution with central management of Edge locations without highly skilled IT personal by providing automation infrastructure for deploying, monitoring and scaling multiple OpenShift Clusters in a disconnected environment. It done by usage of Ansible Tower, including Ansible playbooks from VM provisioning stage up to ready OCP 3.11 Cluster and more.

And now we are in progress of the same solution towards OCP 4.2 Multiple clusters deployment. In current article automation still done in connected environment, we will add additional module for disconnected environment in the near future.

Introduction

Red Hat released Openshift 4.2 recently and before 4.0 and 4.1. One of big changes from Openshift 3.x version is two modes of deployment — IPI (Installer Provided Infrastructure) and UPI (User Provided Infrastructure). In case of IPI it is very easy — you should provide answers to number of interactive questions and Installer will do all the job behind the scene until you will receive up and running cluster. In case of UPI, you should provide the Infrastructure and all configurations required for the Cluster deployment and Installer will only create the cluster. In vSphere environment the only supported deployment mode is UPI and the purpose of this article to show how you can automate the UPI Deployment on vSphere infrastructure.

Prerequisites

1. vSphere Infrastructure
2. Ansible engine (must) and Ansible Tower (optional)
3. DHCP and DNS preconfigured with the relevant IP addresses reservation and DNS zone configuration. DNS also can be configured as Load Balancer Detailed information about it can be found here
4. Firewall and LB Configurations if you decided to use Firewall for security purposes and HA Proxy as Load Balancer. Detailed information about it can be found here
5. Access to the relevant platform Red Hat Openshift Cluster Manager link to download from this link RHCOS ova template, pull secret, download installers, client etc…

Architecture

Logical Diagram of VM Folders and VMs

The Workflow

Visualizer of OCP4.x Full deployment Workflow template in Ansible Tower

To orchestrate the whole deployment process we created Workflow template “ocp4 — Full deployment”

The order of the workflow is:

1. Run “Deploy VMware” template and creates Installer/Helper VM, if it fails it runs the “Destroy VMware” template and exit. If the “Deploy VMware” completed successfully the workflow continues.
2. Run “Ocp4-preinstall”, if it fails the workflow stops. After successful “Ocp4-preinstall” completion the workflow continues.
3. Run “Ocp4- prerequisites”. If it fails the workflow stops. If it works it continues.
4. Run “Ocp4-vm_rhcos_provision”. If this job will fail “Ocp4 — vm_rhcos_destroy” will run and Destroy CoreOS machines, after that template “Ocp4_helper_vm_cleanup” will run and after that workflow stops. If “Ocp4-vm_rhcos_provision” completed successfully it continues.
5. Run “Ocp4-vm_power_on”. If it fails the workflow stops, if it works it continues.
6. Run “Ocp4-rhcos_hostnames”. If it fails the workflow stops, if it works workflow will continue.
7. Run “Ocp4-cluster_deploy”. After this template successful completion workflow ended.

All playbooks and explanations can be found in the GitHub Repository
OCP4 Deployment

Ansible Tower and vSphere preparation before running automation can be found in another article
Openshift 4.x — Automation of UPI deployment by Ansible — Tower and vSphere preparations