GitOps vs. ClickOps is one of the big topics these days, be it at both KubeCons 2022, where we saw a significant increase of interest in Platform Engineering, or at Civo Navigate last week, where the concept of GitOps was present in many workshops, talks, and discussions as well. As developers are shifting their operations left, taking responsibility for larger and more complex parts of their software stacks, DevOps and Platform teams find themselves building internal developer platforms more often.

One approach to doing this is using fully-fledged frameworks like Backstage, which I wrote about two weeks ago. Another way of enabling developers to spin up needed infrastructure and environments faster is by leveraging Infrastructure as Code (IaC). There are many tools in existence to express your needed infrastructure as code, e.g., Ansible, Terraform, or Pulumi. The tool I’ll be focusing on today will be Crossplane though, which allows Platform/DevOps teams to provide opinionated, pre-configured infrastructure for their teams on demand, utilizing Kubernetes CustomResourceDefinitions.

Throughout the course of this article, I will cover the arguments claimed most often when talking about GitOps/ClickOps, and explain the basics of Crossplane. Afterward, I’ll conduct a little experiment: I will deploy two Azure Databases for PostgreSQL servers, once via Azure’s web dashboard, and once using Crossplane.

GitOps and ClickOps⌗

Before starting with my own personal experiment, I’ll need to define what I’m referring to when mentioning GitOps or ClickOps, respectively. GitOps is the idea of manifesting your software’s (and infrastructure’s) desired state and configuration as code and committing it to version control, where it will be deployed from using tools such as ArgoCD or Flux.

ClickOps is the opposite - infrastructure and application configuration is handled by the operator/developer manually by executing a series of clicks in a dashboard, web UI, or any other way that requires manual interaction.

Both approaches have pros and cons: GitOps allows you to keep track of changes to your configuration, and the git repository serves as a single source of truth. On the other hand, developers and platform teams must learn new tooling and often configuration languages, e.g., HCL (Hashicorp Configuration Language), when using Terraform.

ClickOps enables less technical users to configure their needed infrastructure and application state by making a human-readable interface available. This comes at a cost: It’s harder to persist and track changes made, and configuration is more likely to diverge once multiple team members configure their shared projects via UI.

It’s undoubtedly no good-vs-bad decision to make and heavily depends on your environment, situation, and priorities. However, let’s look at Crossplane, a tool that can work wonders if you find yourself clicking around too much and deploying infrastructure too little.

Crossplane - a Self-Service Superhero?⌗

Crossplane describes itself as The cloud native control plane framework on its website, so what does that mean, and how does Crossplane achieve it? For understanding this, there are a few concepts introduced by Crossplane we need to take a closer look at:

Providers do exactly what they sound like - they provide functionality for third-party infrastructure solution. Normally, this includes CustomResourceDefinitions and some controller(s) to reconcile their respective state. A Provider itself can be installed to your cluster already running Crossplane by applying it as a CRD, as shown below with the AWS provider.

apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws
spec:
  package: "xpkg.upbound.io/crossplane-contrib/provider-aws:v0.33.0"

Managed Resources (MRs) are the building blocks of your Crossplane-powered control plane, installed by Providers. They normally describe a 1:1 relationship between some singular piece of infrastructure running somewhere else, e.g. AWS, and the declared state of said resource the MR defines. For example, an MR for an AWS RDS instance could define specifics like the database engine (postgres), its region (us-east-1), or the storage size for the RDS (20):

apiVersion: database.aws.crossplane.io/v1beta1
kind: RDSInstance
metadata:
  name: rdspostgresql
spec:
  forProvider:
    region: us-east-1
    dbInstanceClass: db.t2.small
    masterUsername: masteruser
    allocatedStorage: 20
    engine: postgres
    engineVersion: "12"
    skipFinalSnapshotBeforeDeletion: true
  writeConnectionSecretToRef:
    namespace: crossplane-system
    name: aws-rdspostgresql-conn

Composite Resources (XRs) are to Crossplane what CustomResources are to Kubernetes: An opinionated description of some resources or configuration to exist. For XRs to work, you will have to create Composite Resource Definitions first, telling Crossplane which MRs an XR will be composed of. Returning to our AWS RDS example above, you’d normally want to deploy firewall rules, ACL, and the likes along with your RDS instance. Instead of creating a lot of MRs, you can define an XR bundling these needs for infrastructure into one easy-to-deploy, Kubernetes-native resource. An example XR for an opinionated PostgreSQL instance in the cloud could look like this:

apiVersion: database.example.org/v1alpha1
kind: XPostgreSQLInstance
metadata:
  name: my-db
spec:
  parameters:
    storageGB: 20
  compositionRef:
    name: production
  writeConnectionSecretToRef:
    namespace: crossplane-system
    name: my-db-connection-details

The interactions between and responsibilities for the different moving parts within a typical Crossplane control plane might be a bit hard to wrap ones head around when just starting out, so let’s have a more schematic look at the design for tying things up:

Overview of Crossplane’s resources interacting with each other

Here we can see, that most of the moving parts of a typical Crossplane installation are either created by the Platform team or managed by Crossplane automatically. The workflow could look like this:

  1. The Platform team installs a provider into a Crossplane-enabled cluster, deploying controller and Managed Resource definitions.

  2. The Platform team creates opinionated Composite Resource Definitions based on the installed Managed Resources and their organization’s guidelines and best practices.

  3. The Platform team documents the available Composite Resource Definitions and their configuration to their Application teams.

  4. The Application teams create Composite Resource Claims, asking Crossplane to provide Composite Resources for them to consume.

  5. Crossplane creates the claimed Composite Resources, and, behind the scenes, the required Managed Resources, and starts managing them.

With the basics covered, let’s get to the actual experiment!

Crossplane vs. Microsoft Azure Dashboard⌗

First of all, I need to install Upbound Universal Crossplane on my cluster, because the official providers for AWS, Azure, and GCP won’t work with the ‘standard’ Crossplane. Once this is done, I can go ahead and configure the Azure Provider for Crossplane to use.

Preparations⌗ 

# Create a new cluster on CivoCloud
civo k8s create crossplane-demo --wait --save --merge --switch

# Install Upbound's CLI and deploy UPX to the cluster
curl -sL "https://cli.upbound.io" | sh
mv up /usr/local/bin/
up uxp install

# Install the Azure Provider
cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: upbound-provider-azure
spec:
  package: xpkg.upbound.io/upbound/provider-azure:v0.27.0
EOF

According to Crossplane’s documentation, it might take up to five minutes until the provider has been successfully installed and Crossplane displays its status as healthy:

kubectl get providers
NAME                     INSTALLED   HEALTHY   PACKAGE                                          AGE
upbound-provider-azure   True        True   xpkg.upbound.io/upbound/provider-azure:v0.16.0   90s

The last thing to do is to create a Kubernetes Secret for Azure and to reference it in a ProviderConfig. Both steps are explained in the official documentation, and just one of different ways to authenticate with Azure, so I will skip them here.

Creating a Composite Resource⌗

For this experiment, I will go ahead and create an XRD, defining an XR making a PostgreSQL database with opinionated configuration available for end users of my cluster. Its definition looks like this:

apiVersion: apiextensions.crossplane.io/v1
kind: CompositeResourceDefinition
metadata:
  name: xdemoazuredbs.database.dbodky.me
spec:
  group: database.dbodky.me
  names:
    kind: XDemoAzureDb
    plural: xdemoazuredbs
  claimNames:
    kind: DemoAzureDb
    plural: demoazuredbs
  versions:
  - name: v1alpha1
    served: true
    referenceable: true
    schema:
      openAPIV3Schema:
        type: object
        properties:
          spec:
            type: object
            properties:
              parameters:
                type: object
                properties:
                  storageMb:
                    type: integer
                  resourceGroup:
                    type: string
                required:
                - storageMb
                - resourceGroup
            required:
            - parameters

Thix XRD defines a new claimable Composite Resource XDemoAzureDb (with claims being of type DemoAzureDb) which will require end users to define two properties: storageMb and resourceGroup.

Creating a Composition⌗

However, this won’t be enough for Crossplane to react to our claims - we’ll need to define a Composition in addition to the XRD so Crossplane knows how to map our claims to Managed Resources:

apiVersion: apiextensions.crossplane.io/v1
kind: Composition
metadata:
  name: demoazuredb-composition
  labels:
    crossplane.io/xrd: xdemoazuredbs.database.dbodky.me
    provider: upbound-provider-azure
spec:
  writeConnectionSecretsToNamespace: upbound-system
  compositeTypeRef:
    apiVersion: database.dbodky.me/v1alpha1
    kind: XDemoAzureDb
  resources:
  - name: azure-db
    base:
      apiVersion: dbforpostgresql.azure.upbound.io/v1beta1
      kind: Server
      spec:
        forProvider:
          administratorLogin: psqladmin
          administratorLoginPasswordSecretRef:
            key: example-key
            name: example-secret
            namespace: upbound-system
          geoRedundantBackupEnabled: false
          identity:
          - type: SystemAssigned
          infrastructureEncryptionEnabled: false
          location: West Europe
          resourceGroupName: default
          skuName: GP_Gen5_2
          sslEnforcementEnabled: true
          storageMb: 5120
          version: "11"
    patches:
    - type: FromCompositeFieldPath
      fromFieldPath: spec.parameters.storageMb
      toFieldPath: spec.forProvider.storageMb
    - type: FromCompositeFieldPath
      fromFieldPath: spec.parameters.resourceGroup
      toFieldPath: spec.forProvider.resourceGroupName

There’s a lot to unpack here, so let’s go through the definition step by step:

  • in spec.resources[] we list the Managed Resources which will get created and managed by Crossplane once the Composite Resource linked to this Composition gets created.
    For the sakes of simplicity, our Composite Resource will spin up a Server from the Azure Provider’s dbforpostgresql.azure.upbound.io/v1beta1 and nothing else.

  • in spec.resources[0].spec.forProvider we configure the Managed Resource

  • in patches we define how the properties defined for our Composite Resource Definition should get mapped to the backing Managed Resources of our claimed Composite Resource. Crossplane will then map the properties at creation time

Claiming our Database⌗

Now we established the Composite Resource we want to offer to our end users, and the way it maps to the Managed Resources provided by the Azure Provider - it’s time to claim ourselves a PostgreSQL database on Azure! The claim looks like this:

apiVersion: database.dbodky.me/v1alpha1
kind: DemoAzureDb
metadata:
  namespace: default
  name: my-demo-db
spec:
  parameters:
    storageMb: 10240
    resourceGroup: crossplane-demo
  compositionRef:
    name: demoazuredb-composition
  writeConnectionSecretToRef:
    name: my-db-connection-details

With just a few lines of YAML, our end users can define

  • The amount of storage their database needs in storageMb

  • The resourceGroup to use on Azure (maybe there is one per team, environment, product, etc.)

  • The Composition to use with their Composite Resource Definition (there can be multiple Compositions with different settings for the same Composite Resource Definition)

  • where to write connection information to, as a secret

So this is quite neat - all we need our Platform Team to do is to create the Composite Resource Definitions along with Compositions which map to their opinionated way of setting up, e.g., a PostgreSQL database on Azure, and document the settings which can be adjusted by the end users when they claim those Composite Resources. No more endless clicking around in web consoles!

The Result⌗

As I stated in this article’s title, I am a beginner when it comes to public clouds - I haven’t used any of the big hyper scalers for anything beyond private experiments before, and clicking around their web interfaces is always more hassle than fun to me.

So naturally, I was a lot faster creating the Composite Resource Claim; it took me about a minute to write the claim and deploy it, before getting myself a coffee and letting Crossplane do its magic, instead of clicking around Azure’s Web interface for 5-10 minutes:

kubectl apply -f claim.yml
demoazuredb.database.dbodky.me/my-db created

kubectl get demoazuredb --watch
NAME    SYNCED   READY   CONNECTION-SECRET          AGE
my-db   True     True    my-db-connection-details   6m

Crossplane can be a powerful tool if your Platform Team has already established default configuration and best practices for infrastructure in the cloud - they can go ahead, create XRDs and compositions and enable their Developer Teams to create arbitrary infrastructure in a self-service fashion, without having to worry about navigating the endless configuration forms on the clouds’ web interfaces.

But even if there are no established patterns yet, Crossplane can help with creating and defining them, prevent configuration drift and enable your Developer Teams to work faster and less reliant on the Platform Team to be available 24/7.