[k8s]쿠버네티스(kubernetes) 실습환경 구축

웹에서 제공하는 쿠버네티스 환경(설치없이 쿠버네티스 명령어들을 연습해볼 수 있음)

• 플레이 쿠버네티스: 4시간 시간제한 있음

 

https://labs.play-with-k8s.com/

Play with Kubernetes

 

• 쿠버네티스 플레이그라운드: 노드가 제한적임(자유로운 사용 보장x) 

https://killercoda.com/playgrounds/scenario/kubernetes

Kubernetes Playgrounds | Killercoda

 

 

코드 설치

VAGTANT로 ->버츄얼 박스에 코드를 보냄 -> 가상 머신의 각각의 노드가 올라옴

어떤환경에서든 인터넷만 연결되면 사용할 수 있음

마스터 노드에서 워커노드에게 애플리케이션을 배포를 할꺼야 라고 전달해줌.

 

 

랩 환경 구성

 

준비물!!

총 4개의 어플리케이션이 필요!! 베이그런트, 버츄얼박스, 타비(2024년기준 putty보다 좋다고함), git

1.Vagrant

https://www.vagrantup.com/

Vagrant by HashiCorp

2.VirtualBox

https://www.virtualbox.org/

Oracle VM VirtualBox

3.Tabby

https://tabby.sh/

Tabby - a terminal for a more modern age

 

4.git

https://git-scm.com/download/win

Git - Downloading Package

 

 

Windows PorweShell Code로 다운로드

#VirtualBox
winget install -e --id Oracle.VirtualBox -v 7.0.18
#Vagrant
winget install -e --id Hashicorp.Vagrant -v 2.4.1
#Tabby
winget install -e --id Eugeny.Tabby -v 1.0.207

 

 

Vagrant로 VirtualBox에 masterNode 와 workerNode1,2,3 다운로드

 

Vagrantfile

#Vagrantfile_Code
# -*- mode: ruby -*-
# vi: set ft=ruby :

## configuration variables ##
# max number of worker nodes
N = 3
# each of components to install
k8s_V = '1.30.0-1.1'  # Kubernetes
ctrd_V = '1.6.31-1'   # Containerd 
## /configuration variables ##

Vagrant.configure("2") do |config|

  #====================#
  # Control-Plane Node #
  #====================#

    config.vm.define "cp-k8s-#{k8s_V[0..5]}" do |cfg|
      cfg.vm.box = "sysnet4admin/Ubuntu-k8s"
      cfg.vm.provider "virtualbox" do |vb|
        vb.name = "cp-k8s-#{k8s_V[0..5]}(github_SysNet4Admin)"
        vb.cpus = 2
        vb.memory = 1792
        vb.customize ["modifyvm", :id, "--groups", "/k8s-U#{k8s_V[0..5]}-ctrd-#{ctrd_V[0..2]}(github_SysNet4Admin)"]
      end
      cfg.vm.host_name = "cp-k8s"
      cfg.vm.network "private_network", ip: "192.168.1.10"
      cfg.vm.network "forwarded_port", guest: 22, host: 60010, auto_correct: true, id: "ssh"
      cfg.vm.synced_folder "../data", "/vagrant", disabled: true
      cfg.vm.provision "shell", path: "k8s_env_build.sh", args: [ N, k8s_V[0..3] ]
      cfg.vm.provision "shell", path: "k8s_pkg_cfg.sh", args: [ k8s_V, ctrd_V, "CP"]
      cfg.vm.provision "shell", path: "controlplane_node.sh"
    end

  #==============#
  # Worker Nodes #
  #==============#

  (1..N).each do |i|
    config.vm.define "w#{i}-k8s-#{k8s_V[0..5]}" do |cfg|
      cfg.vm.box = "sysnet4admin/Ubuntu-k8s"
      cfg.vm.provider "virtualbox" do |vb|
        vb.name = "w#{i}-k8s-#{k8s_V[0..5]}(github_SysNet4Admin)"
        vb.cpus = 1
        vb.memory = 1024
        vb.customize ["modifyvm", :id, "--groups", "/k8s-U#{k8s_V[0..5]}-ctrd-#{ctrd_V[0..2]}(github_SysNet4Admin)"]
      end
      cfg.vm.host_name = "w#{i}-k8s"
      cfg.vm.network "private_network", ip: "192.168.1.10#{i}"
      cfg.vm.network "forwarded_port", guest: 22, host: "6010#{i}", auto_correct: true, id: "ssh"
      cfg.vm.synced_folder "../data", "/vagrant", disabled: true
      cfg.vm.provision "shell", path: "k8s_env_build.sh", args: [ N, k8s_V[0..3] ]
      cfg.vm.provision "shell", path: "k8s_pkg_cfg.sh", args: [ k8s_V, ctrd_V, "W" ]
      cfg.vm.provision "shell", path: "worker_nodes.sh"
    end
  end

end

 

 

controlplane_node.sh

#!/usr/bin/env bash

# init kubernetes (w/ containerd)
kubeadm init --token 123456.1234567890123456 --token-ttl 0 \
             --pod-network-cidr=172.16.0.0/16 --apiserver-advertise-address=192.168.1.10 \
             --cri-socket=unix:///run/containerd/containerd.sock

# config for control plane node only 
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

# CNI raw address & config for kubernetes's network 
CNI_ADDR="https://raw.githubusercontent.com/sysnet4admin/IaC/main/k8s/CNI"
kubectl apply -f $CNI_ADDR/172.16_net_calico_v3.26.0.yaml

# kubectl completion on bash-completion dir
kubectl completion bash >/etc/bash_completion.d/kubectl

# alias kubectl to k 
echo 'alias k=kubectl' >> ~/.bashrc
echo "alias ka='kubectl apply -f'" >> ~/.bashrc
echo 'complete -F __start_kubectl k' >> ~/.bashrc

# git clone k8s-code
git clone https://github.com/sysnet4admin/_Lecture_k8s_starter.kit.git
mv /home/vagrant/_Lecture_k8s_starter.kit $HOME
find $HOME/_Lecture_k8s_starter.kit -regex ".*\.\(sh\)" -exec chmod 700 {} \;

# make rerepo-k8s-starter.kit and put permission
cat <<EOF > /usr/local/bin/rerepo-k8s-starter.kit
#!/usr/bin/env bash
rm -rf $HOME/_Lecture_k8s_starter.kit
git clone https://github.com/sysnet4admin/_Lecture_k8s_starter.kit.git $HOME/_Lecture_k8s_starter.kit
find $HOME/_Lecture_k8s_starter.kit -regex ".*\.\(sh\)" -exec chmod 700 {} \;
EOF
chmod 700 /usr/local/bin/rerepo-k8s-starter.kit

# extended k8s certifications all
git clone https://github.com/yuyicai/update-kube-cert.git /tmp/update-kube-cert
chmod 755 /tmp/update-kube-cert/update-kubeadm-cert.sh
/tmp/update-kube-cert/update-kubeadm-cert.sh all --cri containerd
rm -rf /tmp/update-kube-cert
echo "Wait 30 seconds for restarting the Control-Plane Node..." ; sleep 30

 

k8s_env_build.sh

#!/usr/bin/env bash

# avoid 'dpkg-reconfigure: unable to re-open stdin: No file or directory'
export DEBIAN_FRONTEND=noninteractive

# swapoff -a to disable swapping
swapoff -a
# sed to comment the swap partition in /etc/fstab (Rmv blank)
sed -i.bak -r 's/(.+swap.+)/#\1/' /etc/fstab

# add kubernetes repo 
curl \
  -fsSL https://pkgs.k8s.io/core:/stable:/v$2/deb/Release.key \
  | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo \
  "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] \
  https://pkgs.k8s.io/core:/stable:/v$2/deb/ /" \
  | sudo tee /etc/apt/sources.list.d/kubernetes.list

# add docker-ce repo with containerd
curl -fsSL \
  https://download.docker.com/linux/ubuntu/gpg \
  -o /etc/apt/keyrings/docker.asc
echo \
  "deb [arch=$(dpkg --print-architecture) \
  signed-by=/etc/apt/keyrings/docker.asc] \
  https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" \
  | tee /etc/apt/sources.list.d/docker.list > /dev/null

# packets traversing the bridge are processed by iptables for filtering
echo 1 > /proc/sys/net/ipv4/ip_forward
# enable br_filter for iptables 
modprobe br_netfilter

# local small dns & vagrant cannot parse and delivery shell code.
echo "127.0.0.1 localhost" > /etc/hosts # localhost name will use by calico-node
echo "192.168.1.10 cp-k8s" >> /etc/hosts
for (( i=1; i<=$1; i++  )); do echo "192.168.1.10$i w$i-k8s" >> /etc/hosts; done

 

k8s_pkg_cfg.sh

#!/usr/bin/env bash

# avoid 'dpkg-reconfigure: unable to re-open stdin: No file or directory'
export DEBIAN_FRONTEND=noninteractive

# update package list 
apt-get update 

# install NFS 
if [ $3 = 'CP' ]; then
  apt-get install nfs-server nfs-common -y 
elif [ $3 = 'W' ]; then
  apt-get install nfs-common -y 
fi

# install kubernetes
# both kubelet and kubectl will install by dependency
# but aim to latest version. so fixed version by manually
apt-get install -y kubelet=$1 kubectl=$1 kubeadm=$1 containerd.io=$2

# containerd configure to default and cgroup managed by systemd 
containerd config default > /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/g' /etc/containerd/config.toml

# avoid WARN&ERRO(default endpoints) when crictl run  
cat <<EOF > /etc/crictl.yaml
runtime-endpoint: unix:///run/containerd/containerd.sock
image-endpoint: unix:///run/containerd/containerd.sock
EOF

# ready to install for k8s 
systemctl restart containerd ; systemctl enable containerd
systemctl enable --now kubelet

 

 

worker_nodes.sh

#!/usr/bin/env bash

# config for worker nodes only
kubeadm join --token 123456.1234567890123456 \
             --discovery-token-unsafe-skip-ca-verification 192.168.1.10:6443

 

#Vagrantfile 실행 명령어(vagrantfile이 있는 경로에서 명령어 실행)
#위에 파일들을 한곳에 모아놓고 실행(vagrant up을 하게 되면 나머지 파일들이 다딸려온다)
vagrant up