Kubernetes for Windows How to

  • Structure
  • Installation
  • Verification
  • Experiences of How to

Structure

  • At least a Control Plane node.
  • At least a Linux node.
  • At least a Windows node.
get-service kubelet

Installation

  • Cloud provider Kubernetes for Windows, e.g. EKS
  • On-premise Kubernetes for Windows

Cloud provider Kubernetes for Windows

  • prepare an bastion EC2 with EKS permission (attched IAM role).
  • use eksctl to create EKS cluster and Linux node.
  • ensure EKS and Linux node ready.
  • use eksctl to create Windows NodeGroup
eksctl create nodegroup \
--cluster=$CLUST_NAME \
--name $NODE_NAME \
--node-ami-family WindowsServer2019CoreContainer \
--node-type=t3.large \
--nodes-min=1 \
--nodes-max=1 \
--node-private-networking \
--ssh-public-key=$SSHKEY \
--node-volume-size=100 \
--asg-access \
--external-dns-access \
--full-ecr-access \
--alb-ingress-access
eksctl utils install-vpc-controllers --name=$CLUST_NAME --approve
  • update kubeconfig on bastion EC2
aws eks  — region $YOUR_REGION update-kubeconfig  — name $CLUST_NAME
  • After around 10 minutes, you can perform the following command line to check Windows node
kubectl get nodes

On-premise Kubernetes for Windows

  • OS: CentsOS 7 or 8
  • install yum-utils, device-mapper-persistent-data, lvm2
  • download and add yum docker repo
  • install docker-ce, docker-ce-cli, containerd.io
  • start docker service
  • OS: CentOS 7 or 8
  • install nfs-utils, wget
  • disable firewalld
  • disable swap from /etc/fstab
  • download official yum repo for kubctl, kubeadm, kubelet
  • yum install kubectl, kubeadm, kubelet
  • enable kubelet service
  • disable SELinux
  • change docker cgroup to systemd in /etc/docker/daemon.json
  • change k8s cgroup to systemd in /etc/default/kubelet
  • restart docker service
  • change iptables & disable swappiness
  • reload sysctl
  • start/restart kubelet service
  • pre-pull Kubernetes component docker image
  • download kube-flannel.yml, kube-flannel-overlay, kube-proxy-win (reference 4)
  • reboot server
  • kubeadm init. Please follow the below parameters because above YAML files.
kubeadm init --pod-network-cidr=172.31.0.0/16 --service-cidr=172.30.0.0/16 --kubernetes-version=1.18.6
  • install flannel (reference 4)
kubectl apply -f kube-flannel.yml
  • OS should be same as Control Plane
  • install nfs-utils
  • disable firewalld
  • disable swap from /etc/fstab
  • download official yum repo for kubctl, kubeadm, kubelet
  • yum install kubectl, kubeadm, kubelet
  • enable kubelet service
  • disable SELinux
  • change docker cgroup to systemd in /etc/docker/daemon.json
  • change k8s cgroup to systemd in /etc/default/kubelet
  • restart docker service
  • change up tables & disable swappiness
  • reload sysctl
  • start/restart kubelet service
  • reboot server
  • generate kubeadm join command on Control Plane node
kubeadm token create --print-join-command
  • join Kubernetes Cluster. leverage above output of join-command. For example.
kubeadm join {API SERVER} --token XXXXXXXXXXXXXXXX --discovery-token-ca-cert-hash sha256:xxxxxxxxxxxxxxxxxxxxxx
  • disable windows firewall by PowerShell
  • install Docker Msft Module
  • install Docker
  • reboot
  • enable docker service and reboot again
  • OS: Windows 1809 build or greater
  • disable firewall firewall by PowerShell
netsh firewall set opmode mode=disable
  • pre-download container image
docker pull sigwindowstools/kube-proxy:v1.18.6
docker pull sigwindowstools/flannel:0.12.0
docker pull mcr.microsoft.com/windows/nanoserver:1809
docker tag mcr.microsoft.com/windows/nanoserver:1809 microsoft/nanoserver:latest
docker run microsoft/nanoserver:latest
docker pull mcr.microsoft.com/k8s/core/pause:1.2.0
  • download and apply kube-flannel-overlay.yml and kube-proxy-win.yml YAML files (reference 4).
kubectl apply -f kube-flannel-overlay.yml
kubectl apply -f kube-proxy-win.yml
  • install PowerShell activedirectory module (for gMSA. You can skip it, if you don’t need gMSA).
Install-WindowsFeature RSAT-AD-PowerShell
  • Install Powershell CredentialSpec module (for gMSA. You can skip it, if you don’t need gMSA).
Install-Module -Name CredentialSpec
  • change NIC Ethernet0 card name to be Ethernet. Please insert your NIC name instead of Ethernet0.
Rename-NetAdapter "Ethernet0" -NewName Ethernet -ErrorAction SilentlyContinue
  • disable reset of NIC. please don’t disable your connection NIC.
Get-NetAdapter | ? {$_.name -ne "Ethernet"} | Disable-NetAdapter
  • create c:\k directory (practice since 1.14)
mkdir c:\k
  • generate kubeadm join command on Control Plane node
kubeadm token create --print-join-command
  • Please keep the following command line. You might need it when Windows node reboot.
powershell -command "if((docker network ls) -match 'host                nat'){$null}else{docker network create -d nat host}"
  • perform the following command line to check Windows node.
kubectl get nodes

Verification

kubectl apply -f aspnet.yaml
  • Verify Pod status. You can get pod to discover pod name, and then descirbe pod detail. Due to Windows Pod size usually is over 3GB. Therefore, you can see that Cluster is pulling image mcr.microsoft.com/dotnet/framework/aspnet:4.8
# get pods
kubectl get pods
# output
NAME READY STATUS RESTARTS AGE
aspnet-56887957d-4m5xg 0/1 ContainerCreating 0 10m
# describe pods
kubectl describe pods aspnet-56887957d-4m5xg
# output
Name: aspnet-56887957d-4m5xg
Namespace: default
Priority: 0
Node: <Windows NODE NAME>
Start Time: Sat, 28 Nov 2020 11:43:56 +0800
Labels: app=aspnet
pod-template-hash=56887957d
Annotations: <none>
Status: Pending
IP:
Controlled By: ReplicaSet/aspnet-56887957d
Containers:
aspnet:
Container ID:
Image: mcr.microsoft.com/dotnet/framework/aspnet:4.8
Image ID:
Port: 80/TCP
Host Port: 0/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment: <none>
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from default-token-72dc8 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
default-token-72dc8:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-72dc8
Optional: false
QoS Class: BestEffort
Node-Selectors: beta.kubernetes.io/os=windows
Tolerations: node.kubernetes.io/not-ready:NoExecute for 300s
node.kubernetes.io/unreachable:NoExecute for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled <unknown> default-scheduler Successfully assigned default/aspnet-56887957d-4m5xg to xx-xxx-xxxxxx
Normal Pulling 10m kubelet, xx-xxx-xxxxxx Pulling image "mcr.microsoft.com/dotnet/framework/aspnet:4.8"
  • After 10 minutes (depends on your network), you can get pods again to check status. Now, you can see aspnet-XXXXX-XXXX pods is running.
# get pods
kubectl get pods
# output
NAME READY STATUS RESTARTS AGE
aspnet-56887957d-4m5xg 1/1 Running 0 15m
  • perform kubectl to get port number of aspnet service for current verification. As the following result, you get a tcp/31114 number. In the real service, you have to plan network first. Regarding Kubernetes network, I will share it in another article.
# get servicekubectl get service# outputNAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)        AGE
aspnet NodePort 172.30.2.14 <none> 80:31114/TCP 19m
  • Open your borwser and type http://Node_IP:31114 to get the following web page. For the real web service, that is another story.

Experience of How to

Reference

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store