Platform Engineering — Exploring the Trend
What would be the ideal environment for developers?
It would be an environment with full access to a computer somewhere in the cloud, with the developer in complete control of the runtime environment, which is available at all times and behaves predictably as he or she has learned that it should. In such an environment, the developer would be able to create as many resources as they needed and could choose everything — from the operating system to supporting programs, such as installed databases.
Cloud services and solutions such as Docker and Managed Services are breakthrough technologies already enabling developers to get closer to this ideal environment. An integral part of such an ideal environment is the application of the DevOps philosophy.
But reality brings many challenges:
- Lack of time
- High costs: sometimes solutions require excessive investments
- Building blocks are not always the optimal solution: despite "dockerization" and managed services, the design of building blocks is complex and takes time to correctly configure
- Requirements of the local software: some organizations have their IT services installed locally for various reasons, which has an impact on the creation of an ideal environment, and increases the amount of the necessary maintenance work
- Developers like new challenges and do not want to waste time on routine tasks
The path to platform engineering
In practice, developers want to delegate some of their tasks. So would the second most ideal environment be the one in which the developer does the configuration and then passes the service on to someone else to manage? The problem here could be inefficient management due to conflicts of interest.
The developers aim to produce as perfect a solution as possible while keeping in mind the efficiency of the development itself. Operational efficiency is not their immediate concern, so if neglected at design and development stages, their solutions (even if chosen individually as the best solutions according to different developers’ standpoints) can cause unexpected rise in the total complexity and cost of operations. In extreme cases, it can threaten the feasibility of operations down the line, necessitating costly redesigns or unsatisfactory compromises.
It is therefore essential that a separate team of specialists is present from the design stage, as it is in their interest to ensure a complete environment that will allow efficient operations in the long run, if not indefinitely. It is a collaboration where the developers take care of the overall development, and the operations team determines which set of options they can use. The operations team must limit these options because they need to support the entire lifecycle of applications. They incur work during all that time. This work is cumulative, for all active applications.
An example would be restricting the deployment of services in virtual environments, except where there is a compelling reason, because the proliferation of stateful virtual machines increases the effort for each change. Each change must be applied to an ever larger number of virtual machines. This is particularly inefficient when some machines can’t be managed with automation technologies, such as Ansible, Chef, Puppet, etc. The team can provide a more manageable alternative, one with a high degree of commonality, such as a Kubernetes or OpenShift cluster, and support its use.
This practice has recently been given a new name — platform engineering.
Let's take a look at how this evolution takes place. As seen in the figure, developers take care of their part and manage the deployments, but within the sites prepared by Platform Engineering.
Developer self-service is essential, and the platform should allow developers to start working on the project without waiting for the platform engineering team to take action on most projects. Sufficient resources must be provided in advance, and a mechanism must be in place to use these resources. Collaboration between the development and platform engineering teams is key to great solutions. A good platform should aim to deploy the best solutions with minimized unnecessary complexity.
When is it time for new technologies?
Ideally, a well-conceived platform engineering approach would be a competitive advantage for an organization, but since in practice it does not generate revenue on its own, cost-effectiveness is usually the priority.
Prioritization is key: key functions and security must be implemented first and foremost. We need to allow for the possibility that some solutions are implemented in a less-than-ideal way. Developers understandably want the latest solutions, but in business application environments, there is also a need to prioritize: today's best technology is tomorrow's legacy, which will require extra hours of maintenance. It is therefore necessary to evaluate whether new technologies bring enough benefits. Especially in business environments, it is a good idea to wait and see if the technologies prove themselves stable and reliable in the long run.