Kubernetes: Getting started

07 August, 2018 — programming — 2 min read

Kubernetes

Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.

This article covers use of KOPS to setup a Kubernetes cluster and is a follow up of kops/aws.md.

Follow the instruction at kops/install.md to install KOPS cli.

All references to example.com or example.in or DOMAINNAME.TLD or whatever, needs to be replaced with relevant domain, subdomain or tld. Like example.com or nginx.example.com

In addition, one needs access to the following.

AWS Access
- Install AWS cli
- Configure AWS access key and secret.
- Optional: create a kops user.
kops installed. Refer to KOPS installation guide relevant to OS.
kubectl installed
Optional: configure DNS
- Mucking around with root domain does not require any work.
configure s3 for state store
build cluster
deploy docker container

Steps for creating Kubernetes cluster

You need aws cli installed and set up key secret of a user which has admin access. This makes things easy, or at least have the following access.

AmazonEC2FullAccess AmazonRoute53FullAccess AmazonS3FullAccess IAMFullAccess AmazonVPCFullAccess

1export AWS_ACCESS_KEY_ID=$(aws configure get aws_access_key_id)
2export AWS_SECRET_ACCESS_KEY=$(aws configure get aws_secret_access_key)

Configuring DNS is optional. These steps in reference to user having access to Route53 and meddling around with root domain.

If you have have a valid Domain following command should give relevant response.

1dig ns subdomain.example.com

Cluster state storage

1aws s3api create-bucket \
2    --bucket prefix-example-com-state-store \
3    --region us-west-2a

change us-east-1 to whatever region you want to associate it to. I believe it is irrelevant in case of S3.

Create cluster

1export NAME=kluster.example.com
2export KOPS_STATE_STORE=s3://prefix-example-com-state-store
3
4kops create cluster \
5    --zones us-west-2a \
6    ${NAME}
7kops update cluster ${NAME} --yes

At this point if everything went through without any errors. You should have a working kubernetes cluster.

To verify

1kubectl get nodes
2kops validate cluster
3kubectl -n kube-system get po

Above should return some valid stuff. To undo whatever has been done so far.

1kops delete cluster --name ${NAME} --yes

Sometimes delete cluster may not work, especially if you have muddled around with some settings on AWS. For eg. launched an EC2 instance or made some change to Route53 or made some change to security group. You can knock of the relevant entry that kops complain about and try the command again.

Moving on to deploy something useful. Lets deploy nginx and expose it as nginx.example.com

To do this, we need to install external DNS. Make a copy of yaml file to your local and put it in subfolder deployment.

Refer to yaml files addressed here on this gist.

1kubectl apply -f deployments/external-dns.yaml

Verify that there are no error in the logs. If you get any error, you may have to manually grant Route53 full access to security group, something like nodes.example.com or master.example.com check iam section, it would have created a couple of new iam group/user/policies.

1kubectl logs -f $(kubectl get po -l app=external-dns -o name)

Deploy nginx

To deploy nginx on kubernetes

1kubectl create -f deployments/nginx.yaml

After sometime, (may be a up to 5 minutes) you should be able to open nginx.example.com in browser.

1kubectl get nodes --show-labels
2kubectl config view
3kubectl get deployments 
4kubectl get svc
5kubectl get ing

Kubernetes Dashboard

Get admin passsword from kubectl config view

1kubectl create -f deployments/kubernetes-dashboard.yaml
2kubectl apply -f deployments/kube-dashboard-access.yaml

Access UI at something like https://api.kluster.example.com

External DNS

Need to grant Route53 permissions to IAM role nodes.kluster.example.com something like that. Do this from AWS console. Something like note that region depends on what region cluster is deployed.

1kubectl apply -f deployments/external-dns.yaml
2kubectl logs -f $(kubectl get po -l app=external-dns -o name)

Private repo

1kubectl create secret docker-registry regcred --docker-server=https://index.docker.io/v1/ --docker-username=DOCKER_USERNAME --docker-password=DOCKER_PASSWORD [email protected]
2kubectl get secret regcred --output="jsonpath={.data.\.dockerconfigjson}" | base64 -D

Papertrail

1kubectl create secret generic papertrail-destination --from-literal=papertrail-destination=syslog://logs2.papertrailapp.com:YOUR_PORT
2kubectl create -f https://help.papertrailapp.com/assets/files/papertrail-logspout-daemonset.yml

Few other things just for reference.

Below are some of the notes I made while getting my hands dirty with Kubernetes.

1# Optional
2ID=$(uuidgen) && aws route53 create-hosted-zone --name k10s.example.in --caller-reference $ID | \
3    jq .DelegationSet.NameServers
4
5# Optional
6aws route53 list-hosted-zones | jq '.HostedZones[] | select(.Name=="example.in.") | .Id'
7
8# Optional
9aws route53 change-resource-record-sets \
10 --hosted-zone-id YOUR_HOSTED_ZONE_ID \
11 --change-batch file://subdomain.json
12
13# Optional
14dig ns k10s.example.in
15
16# Mandatory
17aws s3api create-bucket \
18    --bucket k10s-example-in-state-store \
19    --region us-east-1
20
21# Mandatory
22export NAME=kluster.example.in
23export KOPS_STATE_STORE=s3://k10s-example-in-state-store
24
25kops create cluster \
26    --zones us-west-2a \
27    ${NAME}
28
29# Optional
30kops edit cluster ${NAME} # For editing configs
31
32kops update cluster ${NAME} --yes

Below is output Suggestions:
validate cluster: kops validate cluster
list nodes: kubectl get nodes --show-labels
ssh to the master: ssh -i ~/.ssh/id_rsa [email protected]
the admin user is specific to Debian. If not using Debian please use the appropriate user based on your OS.
read about installing addons at: https://github.com/kubernetes/kops/blob/master/docs/addons.md.

Deploy Kubernetes UI

Web UI (Dashboard) - Kubernetes

1kubectl create -f https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml
2kubectl proxy
3kubectl delete -f deployments/kube-dashboard-access.yaml

kube-dashboard-access.yaml

1apiVersion: rbac.authorization.k8s.io/v1beta1
2kind: ClusterRoleBinding
3metadata:
4  name: kubernetes-dashboard
5  labels:
6    k8s-app: kubernetes-dashboard
7roleRef:
8  apiGroup: rbac.authorization.k8s.io
9  kind: ClusterRole
10  name: cluster-admin
11subjects:
12- kind: ServiceAccount
13  name: kubernetes-dashboard
14  namespace: kube-system

Important commands

1kops validate cluster
2kubectl get nodes
3kubectl -n kube-system get po
4kubectl cluster-info
5
6kubectl run --image=nginx nginx-app --port=80
7
8kubectl expose deployment nginx-app --port=80 --name=nginx-http

1kubectl version
2kubectl get nodes
3kubectl run kubernetes-bootcamp --image=gcr.io/google-samples/kubernetes-bootcamp:v1 --port=8080
4kubectl get deployments
5kubectl proxy # Another terminal
6curl http://localhost:8001/version
7echo $POD_NAME 
8curl http://localhost:8001/api/v1/namespaces/default/pods/$POD_NAME/proxy/
9kubectl logs $POD_NAME
10kubectl exec $POD_NAME env
11kubectl exec -ti $POD_NAME bash
12cat server.js

1kubectl run --image=nginx nginx-app --port=80 --env="DOMAIN=k10s.example.in"
2kubectl cluster-info
3kubectl get nodes
4kubectl get deployment
5kubectl get pods
6kubectl expose deployment nginx-app --type=LoadBalancer
7kubectl get services
8kubectl run docker-node-express --replicas=2 --labels="run=load-balancer-example" --image=ch4nd4n/docker-node-express --port=3000
9kubectl get deployments
10kubectl get deployments docker-node-express 
11kubectl describe deployments docker-node-express
12kubectl expose deployment docker-node-express --type=LoadBalancer --name=docker-node-express-service
13kubectl get services docker-node-express-service
14kubectl describe services docker-node-express-service
15kubectl delete services docker-node-express-service
16kubectl delete deployment docker-node-express

LoadBalancer Ingress: will contain the address to look for

Updating a build using yaml file

1kubectl cluster-info
2# kubectl set image deployments/docker-node-express docker-node-express=ch4nd4n/docker-node-express:1.0.0
3kubectl create -f deployments/docker-node-app.yaml
4kubectl scale deployment/docker-node-express-deployment --replicas=3
5kubectl replace -f deployments/docker-node-app.yaml --force
6kubectl expose deployment docker-node-express-deployment --type=LoadBalancer --name=docker-node-express-service
7kubectl delete -f deployments/docker-node-app.yaml
8kubectl delete service docker-node-express-service
9
10kubectl logs -f $(kubectl get po -l app=external-dns -o name)