Fargate and my regrets not just using Kubernetes

What do I want at the end of the day from containers

I want to build my application which exposes a REST API, package it as a docker image and deploy it so others can consume the API. Kubernetes or Fargate should be good candidates for this.

Background

I originally started using Kubernetes around 2017 and since there was no budget for OpenShift I set up CentOS Atomic Kubernetes. Looking back at that decision I realize I learnt a lot about the internals of Kubernetes but that is really not the way to do it. When I started using k8s on Azure with Terraform I obviously found that a lot easier, but the maintenance was always technical and it distracted from developing business logic in my opinion.

After that I used AWS EKS which again had a lot of maintenance tasks.

This is my tale of being instrumental at implementing Fargate at two SaaS companies and my regret.

My fundamental issues with k8s in the past

An expert was always needed

• Even in the cloud, Kubernetes needed to be upgraded fairly often. This would require someone to upgrade the master where applicable and upgrade the nodes.
• Incorrect reservations and limits was a simple issue, but easy enough.
• Helm charts are great, but it was a little too easy to install new toys.
• Something always seemed to need attention like evicted pods or new events.

k8s nodes did not scale

Back when I used k8s in AWS and Azure, you had a fixed set of nodes and node scalers were still experimental. This means a constant cost since the number of nodes and node configuration was constant.

  flowchart TB
subgraph AWS["AWS Cloud"]
subgraph EKS["Kubernetes Cluster (EKS)"]
Master["Master Node (Control Plane)"]

            Node1["Worker Node : fixed nCPU and vCPU Memory"]
            Node2["Worker Node 2 : fixed nCPU and vCPU Memory"]
            Node3["Worker Node CPU : fixed nCPU and vCPU Memory"]

            Master --> Node1
            Master --> Node2
            Master --> Node3
        end
    end

Blast radius

If something went wrong with a node or noisy neighbors you would get a storm of issues. For this you would need an expert.

Fargate to the “rescue”

When I started a new position at a SaaS, I suggested Fargate as it was less maintenance and easier to set up than Kubernetes. I used the AWS CDK in Java to configure it like the oversimplified code below.

It seemed like a great idea at the time since the current staff understood Java and Terraform was not familiar.

There was a learning curve, but once set up it was easy enough

Setting up Fargate in the Java CDK was trivial at first, but more complex setups started to get messy. At least initially it seemed like less expertise was needed, but the devil was in the IaC detail.

class ApplicationStack extends Stack {
    public ApplicationStack(final Construct scope, final String id) {
        super(scope, id);

        String pathPrefix;
        if (props.getPartition() != null && !props.getPartition().isEmpty()) {
            pathPrefix = "/" + props.getStage() + "/" + props.getPartition() + "/application";
        } else {
            pathPrefix = "/" + props.getStage() + "/application";
        }
    }
}

class DevStage extends Stage {
    public DevStage(final Construct scope, final String id, final StageProps props) {
        super(scope, id, props);

        new ApplicationStack(this, "DevApplication", new ApplicationStackProps("dev"));
    }
}
class TestStage extends Stage {
    public DevStage(final Construct scope, final String id, final StageProps props) {
        super(scope, id, props);

        new ApplicationStack(this, "TestApplicationPartition1", new ApplicationStackProps("partition1"));
        new ApplicationStack(this, "TestApplicationPartition1", new ApplicationStackProps("partition2"));
    }
}
public class CdkApp {
    static void main(final String[] args) {
        App app = new App();

        new DevStage(app, "Dev", StageProps.builder().build());
        new TestStage(app, "Test", StageProps.builder().build());

        app.synth();
    }
}

AWS Fargate Monthly Pricing — Approx. 24×7 (USD)

These are the possible limited configurations that you can choose from. Choosing less than 0.5 vCPU is extremely slow.

Notes: This excludes a NAT Gateway which is needed when your service reaches out with IPv4 requests outside the VPC. This can be very high if you have high data volumes.

Blast radius

The blast radius was significantly reduced, but if I wanted to deploy a service inside a VPC I would need to set up a load balancer. This seemed like Fargate was very primitive compared to k8s.

What I quickly learnt about CDK compared to a helm chart

Fargate is really difficult to set up in the CDK even when using the level 3 patterns. Not everything fits into the level 3 pattern like having a frontend and backend on a single load balancer. Here is an example for a simple service with a load balancer.

ApplicationLoadBalancedFargateService.Builder.create(this, "MyFargateService")
            .cluster(cluster)
            .cpu(1024)
            .desiredCount(1)
            .taskImageOptions(
                 ApplicationLoadBalancedTaskImageOptions.builder()
                         .image(ContainerImage.fromRegistry("yourimage"))
                         .build())
            .memoryLimitMiB(2048)
            .publicLoadBalancer(true)
            .build();

My second try at Fargate

When I started my next position there was a need for the cloud engineers to understand the IaC and they understood Terraform. So therefore I helped set up Fargate using a Terraform module. Many of the developers needed to learn Terraform and this distracted them from doing application code. The Terraform in the end was hosted in about 20 services in the code’s infra directory. This did allow the developers to use other services in AWS.

My regret

I found myself looking back at the promise of Fargate and how it should have solved my issues.

• Unfortunately, the cost of CPU and memory is prohibitively expensive since you don’t share resources.
• The CDK and Terraform code for Fargate that needed to be written became quite complex. The cost of ownership of this code can be high if you have many teams with slightly different requirements.
• I always felt that it could have been a simple Helm chart, which would have made onboarding developers easier.
• Fargate has not really changed in the last two years, while k8s is evolving.

My takeway on Fargate

• It is ok for very simple services, but you may quicky outgrow it.
• It is not cheap to set up for more than basic used cases.
• Fargate is also not cheap to run since it does not share resources.
• Fargate has vendor lock in and is not evolving.

As a bonus, it’s easy to monitor Kubernetes with sentry-kubernetes.