December 6, 2021
This post will introduce what network segregation is and will show how to implement a basic example with Kubernetes and the Cilium network solution.The Kubernetes cluster used must have a networking solution that supports Network Policies. In this case Cilium was chosen, however other example include Calico, Kube-router, Romana, and Weave Net. The next post will take a more in depth look at transport and application networking models.
Network segregation has been a best practice in networking for a long time. It’s something we always make sure to implement at Compoze Labs. So what is it?
Network segregation is the concept of splitting up a network into separate sub-networks, or “subnets”. Below is a simplified, but typical, network topology for two applications deployed into the same AWS VPC.
Segregating application networks, both east-west (left-right) and north-south (up-down), improves cluster security posture in two ways:
The VPC above is divided into 4 separate subnets, two per application. The public subnets are internet accessible (or routable), that is, servers within those subnets would have both public and private IPs. Servers within the App Subnets would only be assigned private IPs, meaning only servers within the same VPC could be granted network access.
By limiting what is exposed to the public internet (public subnets) we reduce the number of servers that are prone to external attacks. In the above diagram, only the load balancers are exposed in this way.
In addition, notice the grey arrows on either side of the subnets. Those are to depict the direction network traffic is allowed, namely down (north-south). Using networking constructs like firewalls (security groups in AWS) and NACLs, network traffic can be limited to only specific subnets. In this case, because Application A & B have separate subnets, firewall and NACL rules can be created to impose this segregation.
.jpg)
In the event a server in Application A is compromised, only other servers within Application A’s subnets are susceptible to further attack. This means that Application B is protected.
Reducing our external attack surface with containers is similar to what was done in the above example. Kubernetes’ pods and services typically are only routable from within the cluster itself; therefore externally facing load balancers are still required to direct traffic to applications within the cluster.
With Kubernetes, one way to handle this is via an Ingress resource plus an external load balancer (like AWS’ ELB or ALB). This is what the network topology looks like now:
The load balancers are located in the same subnets as before; and the applications are within internal cluster subnets. This ensures only the load balancers are exposed to the big bad internet
However, since a pod should be able to come up on any server in the cluster, the applications no longer know which subnets they will be deployed to. This makes it nearly impossible to use traditional NACLs or firewalls to enforce segregation.
Assume that Applications A and B have been broken up into simple microservices deployed as containers. The applications consist of a a simple Go http server and a client with curl installed.
Since all pods can talk to each other, client B can still communication with server A and client A can communicate with client B. This means that if a pod is compromised, it will have access to any other pod in the cluster. How can they be segregated from one another?
A network policy is a specification of how groups of pods are allowed to communicate with each other and other network endpoints.
Network Policies act as a way to whitelist traffic to a particular pod using labels. Once a network policy selects a particular pod, that pod will only accept connections that are allowed by the policy. Cilium is one of the Kubernetes network solutions that supports network policies.
Here is the deployment YAML file for our sample Kubernetes applications. It describes the application resources as services and pods.
Before any network rules are applied, both client’s can communicate with the both Go servers.
In order to isolate Application A from Application B the following Cilium network policy can be applied.
The endpointSelector specifies to apply the policy to all resources matching labels org: application-a and class: service-a (service A’s labels) as well as to only allow traffic on port 80 from resources with labels org: application-a. In this case, since client A has the appropriate labels it can communicate with service A. However, since client B does not have the required labels, its traffic will be blocked.
By using public and private subnets, in conjunction with Kubernetes’ network policies, cluster security posture is improved by reducing attack surface and limiting areas of impact if a pod is breached.
Of course, this doesn’t take care of all security vulnerabilities, rather it’s merely the first step towards thinking about securely building applications. The next article will walk through how to create even more fine-grained control through the use of HTTP level restriction.
To chat about your project, connect with us at compozelabs.com or shoot us an email at connect@compozelabs.com! Even if you’re not in the market, we enjoy discussions on network security!
