ClickOps vs. Crossplane - Deploying to the clouds from a beginner’s perspective
GitOps vs. ClickOps is one of the big topics these days, be it at both KubeCons 2022, where we saw a significant increase of interest in Platform Engineering, or at Civo Navigate last week, where the concept of GitOps was present in many workshops, talks, and discussions as well. As developers are shifting their operations left, taking responsibility for larger and more complex parts of their software stacks, DevOps and Platform teams find themselves building internal developer platforms more often.
One approach to doing this is using fully-fledged frameworks like Backstage, which I wrote about two weeks ago. Another way of enabling developers to spin up needed infrastructure and environments faster is by leveraging Infrastructure as Code (IaC). There are many tools in existence to express your needed infrastructure as code, e.g., Ansible, Terraform, or Pulumi. The tool I’ll be focusing on today will be Crossplane though, which allows Platform/DevOps teams to provide opinionated, pre-configured infrastructure for their teams on demand, utilizing Kubernetes CustomResourceDefinitions
.
Throughout the course of this article, I will cover the arguments claimed most often when talking about GitOps/ClickOps, and explain the basics of Crossplane. Afterward, I’ll conduct a little experiment: I will deploy two Azure Databases for PostgreSQL servers, once via Azure’s web dashboard, and once using Crossplane.
GitOps and ClickOps⌗
Before starting with my own personal experiment, I’ll need to define what I’m referring to when mentioning GitOps or ClickOps, respectively. GitOps is the idea of manifesting your software’s (and infrastructure’s) desired state and configuration as code and committing it to version control, where it will be deployed from using tools such as ArgoCD or Flux.
ClickOps is the opposite - infrastructure and application configuration is handled by the operator/developer manually by executing a series of clicks in a dashboard, web UI, or any other way that requires manual interaction.
Both approaches have pros and cons: GitOps allows you to keep track of changes to your configuration, and the git repository serves as a single source of truth. On the other hand, developers and platform teams must learn new tooling and often configuration languages, e.g., HCL (Hashicorp Configuration Language), when using Terraform.
ClickOps enables less technical users to configure their needed infrastructure and application state by making a human-readable interface available. This comes at a cost: It’s harder to persist and track changes made, and configuration is more likely to diverge once multiple team members configure their shared projects via UI.
It’s undoubtedly no good-vs-bad decision to make and heavily depends on your environment, situation, and priorities. However, let’s look at Crossplane, a tool that can work wonders if you find yourself clicking around too much and deploying infrastructure too little.
Crossplane - a Self-Service Superhero?⌗
Crossplane describes itself as The cloud native control plane framework on its website, so what does that mean, and how does Crossplane achieve it? For understanding this, there are a few concepts introduced by Crossplane we need to take a closer look at:
Providers do exactly what they sound like - they provide functionality for third-party infrastructure solution. Normally, this includes CustomResourceDefinitions
and some controller(s) to reconcile their respective state. A Provider itself can be installed to your cluster already running Crossplane by applying it as a CRD, as shown below with the AWS provider.
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: provider-aws
spec:
package: "xpkg.upbound.io/crossplane-contrib/provider-aws:v0.33.0"
Managed Resources (MRs) are the building blocks of your Crossplane-powered control plane, installed by Providers. They normally describe a 1:1 relationship between some singular piece of infrastructure running somewhere else, e.g. AWS, and the declared state of said resource the MR defines. For example, an MR for an AWS RDS instance could define specifics like the database engine (postgres
), its region (us-east-1
), or the storage size for the RDS (20
):
apiVersion: database.aws.crossplane.io/v1beta1
kind: RDSInstance
metadata:
name: rdspostgresql
spec:
forProvider:
region: us-east-1
dbInstanceClass: db.t2.small
masterUsername: masteruser
allocatedStorage: 20
engine: postgres
engineVersion: "12"
skipFinalSnapshotBeforeDeletion: true
writeConnectionSecretToRef:
namespace: crossplane-system
name: aws-rdspostgresql-conn
Composite Resources (XRs) are to Crossplane what CustomResources
are to Kubernetes: An opinionated description of some resources or configuration to exist. For XRs to work, you will have to create Composite Resource Definitions first, telling Crossplane which MRs an XR will be composed of. Returning to our AWS RDS example above, you’d normally want to deploy firewall rules, ACL, and the likes along with your RDS instance. Instead of creating a lot of MRs, you can define an XR bundling these needs for infrastructure into one easy-to-deploy, Kubernetes-native resource. An example XR for an opinionated PostgreSQL instance in the cloud could look like this:
apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
metadata:
name: my-db
spec:
parameters:
storageGB: 20
compositionRef:
name: production
writeConnectionSecretToRef:
namespace: crossplane-system
name: my-db-connection-details
The interactions between and responsibilities for the different moving parts within a typical Crossplane control plane might be a bit hard to wrap ones head around when just starting out, so let’s have a more schematic look at the design for tying things up:
Here we can see, that most of the moving parts of a typical Crossplane installation are either created by the Platform team or managed by Crossplane automatically. The workflow could look like this:
-
The Platform team installs a provider into a Crossplane-enabled cluster, deploying controller and Managed Resource definitions.
-
The Platform team creates opinionated Composite Resource Definitions based on the installed Managed Resources and their organization’s guidelines and best practices.
-
The Platform team documents the available Composite Resource Definitions and their configuration to their Application teams.
-
The Application teams create Composite Resource Claims, asking Crossplane to provide Composite Resources for them to consume.
-
Crossplane creates the claimed Composite Resources, and, behind the scenes, the required Managed Resources, and starts managing them.
With the basics covered, let’s get to the actual experiment!
Crossplane vs. Microsoft Azure Dashboard⌗
First of all, I need to install Upbound Universal Crossplane on my cluster, because the official providers for AWS, Azure, and GCP won’t work with the ‘standard’ Crossplane. Once this is done, I can go ahead and configure the Azure Provider for Crossplane to use.
Preparations⌗
# Create a new cluster on CivoCloud
civo k8s create crossplane-demo --wait --save --merge --switch
# Install Upbound's CLI and deploy UPX to the cluster
curl -sL "https://cli.upbound.io" | sh
mv up /usr/local/bin/
up uxp install
# Install the Azure Provider
cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
name: upbound-provider-azure
spec:
package: xpkg.upbound.io/upbound/provider-azure:v0.27.0
EOF
According to Crossplane’s documentation, it might take up to five minutes until the provider has been successfully installed and Crossplane displays its status as healthy:
kubectl get providers
NAME INSTALLED HEALTHY PACKAGE AGE
upbound-provider-azure True True xpkg.upbound.io/upbound/provider-azure:v0.16.0 90s
The last thing to do is to create a Kubernetes Secret for Azure and to reference it in a ProviderConfig
. Both steps are explained in the official documentation, and just one of different ways to authenticate with Azure, so I will skip them here.
Creating a Composite Resource⌗
For this experiment, I will go ahead and create an XRD, defining an XR making a PostgreSQL database with opinionated configuration available for end users of my cluster. Its definition looks like this:
apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
name: xdemoazuredbs.database.dbodky.me
spec:
group: database.dbodky.me
names:
kind: XDemoAzureDb
plural: xdemoazuredbs
claimNames:
kind: DemoAzureDb
plural: demoazuredbs
versions:
- name: v1alpha1
served: true
referenceable: true
schema:
openAPIV3Schema:
type: object
properties:
spec:
type: object
properties:
parameters:
type: object
properties:
storageMb:
type: integer
resourceGroup:
type: string
required:
- storageMb
- resourceGroup
required:
- parameters
Thix XRD defines a new claimable Composite Resource XDemoAzureDb
(with claims being of type DemoAzureDb
) which will require end users to define two properties: storageMb
and resourceGroup
.
Creating a Composition⌗
However, this won’t be enough for Crossplane to react to our claims - we’ll need to define a Composition in addition to the XRD so Crossplane knows how to map our claims to Managed Resources:
apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
name: demoazuredb-composition
labels:
crossplane.io/xrd: xdemoazuredbs.database.dbodky.me
provider: upbound-provider-azure
spec:
writeConnectionSecretsToNamespace: upbound-system
compositeTypeRef:
apiVersion: database.dbodky.me/v1alpha1
kind: XDemoAzureDb
resources:
- name: azure-db
base:
apiVersion: dbforpostgresql.azure.upbound.io/v1beta1
kind: Server
spec:
forProvider:
administratorLogin: psqladmin
administratorLoginPasswordSecretRef:
key: example-key
name: example-secret
namespace: upbound-system
geoRedundantBackupEnabled: false
identity:
- type: SystemAssigned
infrastructureEncryptionEnabled: false
location: West Europe
resourceGroupName: default
skuName: GP_Gen5_2
sslEnforcementEnabled: true
storageMb: 5120
version: "11"
patches:
- type: FromCompositeFieldPath
fromFieldPath: spec.parameters.storageMb
toFieldPath: spec.forProvider.storageMb
- type: FromCompositeFieldPath
fromFieldPath: spec.parameters.resourceGroup
toFieldPath: spec.forProvider.resourceGroupName
There’s a lot to unpack here, so let’s go through the definition step by step:
-
in
spec.resources[]
we list the Managed Resources which will get created and managed by Crossplane once the Composite Resource linked to this Composition gets created.
For the sakes of simplicity, our Composite Resource will spin up aServer
from the Azure Provider’sdbforpostgresql.azure.upbound.io/v1beta1
and nothing else. -
in
spec.resources[0].spec.forProvider
we configure the Managed Resource -
in
patches
we define how the properties defined for our Composite Resource Definition should get mapped to the backing Managed Resources of our claimed Composite Resource. Crossplane will then map the properties at creation time
Claiming our Database⌗
Now we established the Composite Resource we want to offer to our end users, and the way it maps to the Managed Resources provided by the Azure Provider - it’s time to claim ourselves a PostgreSQL database on Azure! The claim looks like this:
apiVersion: database.dbodky.me/v1alpha1
kind: DemoAzureDb
metadata:
namespace: default
name: my-demo-db
spec:
parameters:
storageMb: 10240
resourceGroup: crossplane-demo
compositionRef:
name: demoazuredb-composition
writeConnectionSecretToRef:
name: my-db-connection-details
With just a few lines of YAML, our end users can define
-
The amount of storage their database needs in
storageMb
-
The
resourceGroup
to use on Azure (maybe there is one per team, environment, product, etc.) -
The Composition to use with their Composite Resource Definition (there can be multiple Compositions with different settings for the same Composite Resource Definition)
-
where to write connection information to, as a secret
So this is quite neat - all we need our Platform Team to do is to create the Composite Resource Definitions along with Compositions which map to their opinionated way of setting up, e.g., a PostgreSQL database on Azure, and document the settings which can be adjusted by the end users when they claim those Composite Resources. No more endless clicking around in web consoles!
The Result⌗
As I stated in this article’s title, I am a beginner when it comes to public clouds - I haven’t used any of the big hyper scalers for anything beyond private experiments before, and clicking around their web interfaces is always more hassle than fun to me.
So naturally, I was a lot faster creating the Composite Resource Claim; it took me about a minute to write the claim and deploy it, before getting myself a coffee and letting Crossplane do its magic, instead of clicking around Azure’s Web interface for 5-10 minutes:
kubectl apply -f claim.yml
demoazuredb.database.dbodky.me/my-db created
kubectl get demoazuredb --watch
NAME SYNCED READY CONNECTION-SECRET AGE
my-db True True my-db-connection-details 6m
Crossplane can be a powerful tool if your Platform Team has already established default configuration and best practices for infrastructure in the cloud - they can go ahead, create XRDs and compositions and enable their Developer Teams to create arbitrary infrastructure in a self-service fashion, without having to worry about navigating the endless configuration forms on the clouds’ web interfaces.
But even if there are no established patterns yet, Crossplane can help with creating and defining them, prevent configuration drift and enable your Developer Teams to work faster and less reliant on the Platform Team to be available 24/7.