Kubernetes for Windows experiences

Note: this article contains Kubernetes technique terms.

“Windows Container” a technique might be someone know, but no one really knows it. I work on Windows container project start from 2019/07 till now. I really experienced Pros and Cons of Windows Container such as the following.

Why I jumped into Kubernetes world? I would say because of the following Pros.

• Service High Availability and Traffic LoadBalancing
• Service AutoHealing
• Same deployment step
• Service containerization
• gMSA

Before I jump into Windows Container project，I used to work on Kubernetes project. Therefore, I really experienced a lot of fantastic that was not happen in the Linux VM. That is why Docker and Kubernetes that impact whole industry. In the Windows Container that you still can experience some of Pros of Linux Container. The first Pros of Windows Container that is Service High Availability. You just need to increase Replicas number from 1 to 2, Kubernetes will add a Pod immediately without any additional effort. What amazing world. Right? Furthermore, you can leverage Kubernetes Service to achieve traffic LoadBalancing. In the past, you have to purchase expensive hardware device for Layer 4 and Layer 7 LoadBalancing. Now, you could quick achieve it by Kubernetes for Windows solution.

Service AutoHealing. Due to Windows Container is to monitor process status of Pod. When process is running, your Pod is keeping running. When process changed, you Prod will be restart automatically. That is to say, people don’t need to put effort to restart service manually. Furthermore, in the Kubernetes, you could leverage Liveness for Pod health-check. This can help service team to decreases effort for service status check and restart.

Same deployment step. A service start from Dev, QA, Beta, Stage to Production. There are many environment settings. In the Kubernetes, You can try to maximize the consistency of deployment.

Service Containerization. This is why I jumped into Windows Container project reason. Sometime, you might not want to put resource to re-write whole code to Linux version. But you still want to leverage Kubernetes feature for your service. Therefore, Windows Container might be a good choice. If you could control resource well and change minimal of code, and then leverage Kubernetes feature and CI/CD that Windows Container still can give you many Pros. I always believe that we don’t need re-write whole code when service is running well. You should look for Business Opportunity more.

gMSA. I would say, this good reason beside of code re-writing. In some of cases, people might need change Domain password often. If you apply gMSA for your Windows Container. All of password management will be controlled by Windows ActiveDirectory service. This will increase security a lot, and reduce a lot of operation effort.

For the gMSA topic, I will publish another article :D

After Pros, it is time to talk about Cons.

• Windows OS Size is too large
• Windows Node autoscaling is slow
• Windows Pod image size is too large
• Windows Pod autoscaling is slow
• Possible resource optimization
• No build-in Prometheus exporter

Windows OS size is too large and Windows Node AutoScaling is slow. To run a windows OS node well, you might need at least 40GB disk size. This really impact AutoScaling. In my experience, a Windows node needs at least 6 minutes for new Windows node ready. A similar behavior of Linux, you just need around a minute. This impacts Kubernetes functionality.

Windows Pod image size is large and Windows Pod autoscaling is slow. Just similar above node problem. A general case of Windows IIS + .Net Pod, the pod image size is greater then 5GB. Therefore, image pulling time will be your project challenge. Regarding this issue, I would suggest to ensure your Docker image registry downloading speed on your Production Windows Container. This could be big impact of your service AutoHealing. Regarding this problem, I think Microsoft has realized it. I have discovered some 1GB less docker image. So, I believe there are some use case might leverage windows container, especially plus gMSA.

Possible resource optimization. Maybe someone experienced that should be Pros why I put in the Cons? In the Kubernetes, people should achieve resource high optimization, but we still consider Windows OS resource consumption in the reality. Therefore, people often try to increase Pod resource requirement to avoid issue. This will increase your cost. However, we still can leverage monitor to ensure your actual resource requirement.

No build-in Prometheus Exporter. If you tried Prometheus operator, I think you will love Prometheus node exporter. One command to get any thing ready. But, there is no Prometheus node exporter for Windows node for now. Therefore, I leverage Windows exporter by another open source project. For the Windows exporter implementation, I will publish another article.

As the above sharing, I really believe Windows Container is not a maturity solution, but it would be good solution in some scenario. So, as a service team, you have to evaluate Windows Container Pros and Cons firt. Once you have overall plan, Windows Container still can deliver value. Now, I would like to share the following practices for everyone, it might be not best, but still useful.

• Optimize Windows docker image pulling downloading speed.
• Deal a base image for whole projects
• Pre-download base image when pack or building Windows node.
• Don’t always pull image for your Pod

Regarding to the above items that is what I experienced in the Windows Container project. Apply them that might give better Windows Container experience.

Additionally, I will publish another article for How to implement Kubernetes for Windows. Thank you :D