Kubernetes is a powerful platform for deploying, scaling, and managing containerized applications. It has become the de facto standard for container orchestration and is widely used by organizations of all sizes to manage containerized workloads. However, the surface area exposed by Kubernetes can be overwhelming, leading to ambiguous configurations causing confusion and frustration for application engineers and platform engineers alike.
Kubernetes’ immense flexibility and extensibility make it a powerful tool for building platforms. Specifically, platform engineering teams should leverage Kubernetes to build their own platforms that simplify the process of deploying and managing applications for engineers.
Why Build a Platform?
Software engineering organizations adopt software architectures and frameworks to simplify the process of building and delivering software. We use microservice architectures to isolate management of our business domains and speed up development cycles. We use software frameworks that provide us with a set of conventions and best practices to follow, so we can focus on building our business logic. However, we still find ourselves dealing with relatively low-level Kubernetes APIs and vendor tooling to ship our applications.
Consider the following from The Rails Doctrine:
Part of the Rails’ mission is to swing its machete at the thick, and ever growing, jungle of recurring decisions that face developers creating information systems for the web. There are thousands of such decisions that just need to be made once, and if someone else can do it for you, all the better.
Kubernetes at its base is complex. When you throw in an ever-growing ecosystem of tools required to orchestrate a highly available Kubernetes deployment, complexity increases and more decisions have to be made. Why should we expose our application engineers to this complexity?
A Platform on Kubernetes
There is no one-size-fits-all approach for what a platform should look like. The platform’s design is dependent on the needs of the organization and the classes of applications being deployed. Platform Engineers will be familiar with their organization’s needs and are best positioned to design platforms that meet those needs.
There are, however, common patterns that should be used to build platforms on Kubernetes. These are custom resource definitions (CRDs) and controllers, collectively known as the Operator Pattern. CRDs allow you to define custom resources that can be managed by Kubernetes. Controllers are responsible for managing the lifecycle of these custom resources.
Using the operator pattern, platform engineers can design organization-specific platforms that abstract away the complexity of both Kubernetes and the underlying mechanisms required to operate applications at scale without exposing application engineers to the underlying complexity.
There may be multiple use cases within a single organization that require distinct CRDs to deal with a specific class of applications. For example, a platform for deploying web applications may require a CRD for deploying web services, while a platform for deploying data processing applications may require a CRD for deploying data processing engines.
One of the most commonly used patterns is building a containerized application and having it serve traffic to consumers, such as other applications on the same cluster or external clients. In two follow-up posts, we’ll explore the design considerations when authoring a custom resource definition and how to implement a controller in Rust.