티스토리 뷰

728x90
SMALL

이번 포스팅에서는 Kubernetes의 Pod들이 배치될 Worker 노드를 설치하고 Master Node와 연결하는 과정을 살펴보겠습니다.

지난 Kubernetes 가이드는 아래를 참고하세요.

 

[Container Management] Kubernetes Master Node 설치

[Container Management] Kubernetes Dashboard Install & Setting

Kubernetes Worker Node 설치

1. firewall-cmd를 활용하여 오픈할 방화벽 port를 정의합니다.

firewall-cmd --zone=public --permanent --add-port=10250/tcp

firewall-cmd --zone=public --permanent --add-port=30000-32767/tcp

firewall-cmd --reload

Worker Node에서 사용하는 포트는 다음과 같습니다.

Protocol

Direction

Port Range

Purpose

Used By

TCP

Inbound

10250

Kubelet API

Self, Control plane

TCP

Inbound

30000 - 32767

NodePort Services**

All

2. Docker 및 Kubernetes 설치

[root@kubeworker ~]# cat /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kube*
[root@kubeworker ~]#

kubernetes repository를 추가하여 kubeadm, kubectl, kubelet을 설치합니다.

[root@kubeworker ~]# yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.navercorp.com
 * extras: mirror.navercorp.com
 * updates: mirror.navercorp.com
Resolving Dependencies
--> Running transaction check
---> Package kubeadm.x86_64 0:1.15.1-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.15.1-0.x86_64
--> Processing Dependency: cri-tools >= 1.13.0 for package: kubeadm-1.15.1-0.x86_64
---> Package kubectl.x86_64 0:1.15.1-0 will be installed
---> Package kubelet.x86_64 0:1.15.1-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.15.1-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.15.1-0.x86_64
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-4.el7 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-4.el7.x86_64
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-9.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

============================================================================================================================================================================================================================================
 Package                                                           Arch                                              Version                                                    Repository                                             Size
============================================================================================================================================================================================================================================
Installing:
 kubeadm                                                           x86_64                                            1.15.1-0                                                   kubernetes                                            8.9 M
 kubectl                                                           x86_64                                            1.15.1-0                                                   kubernetes                                            9.5 M
 kubelet                                                           x86_64                                            1.15.1-0                                                   kubernetes                                             22 M
Installing for dependencies:
 conntrack-tools                                                   x86_64                                            1.4.4-4.el7                                                base                                                  186 k
 cri-tools                                                         x86_64                                            1.13.0-0                                                   kubernetes                                            5.1 M
 kubernetes-cni                                                    x86_64                                            0.7.5-0                                                    kubernetes                                             10 M
 libnetfilter_cthelper                                             x86_64                                            1.0.0-9.el7                                                base                                                   18 k
 libnetfilter_cttimeout                                            x86_64                                            1.0.0-6.el7                                                base                                                   18 k
 libnetfilter_queue                                                x86_64                                            1.0.2-2.el7_2                                              base                                                   23 k
 socat                                                             x86_64                                            1.7.3.2-2.el7                                              base                                                  290 k

Transaction Summary
============================================================================================================================================================================================================================================
Install  3 Packages (+7 Dependent packages)

Total download size: 56 M
Installed size: 251 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-4.el7.x86_64.rpm                                                                                                                                                                       | 186 kB  00:00:00     
warning: /var/cache/yum/x86_64/7/kubernetes/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY  0.0 B/s | 186 kB  --:--:-- ETA 
Public key for 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm is not installed
(2/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm                                                                                                               | 5.1 MB  00:00:01     
(3/10): aa386b8f2cac67415283227ccb01dc043d718aec142e32e1a2ba6dbd5173317b-kubeadm-1.15.1-0.x86_64.rpm                                                                                                                 | 8.9 MB  00:00:02     
(4/10): f27b0d7e1770ae83c9fce9ab30a5a7eba4453727cdc53ee96dc4542c8577a464-kubectl-1.15.1-0.x86_64.rpm                                                                                                                 | 9.5 MB  00:00:01     
(5/10): libnetfilter_cthelper-1.0.0-9.el7.x86_64.rpm                                                                                                                                                                 |  18 kB  00:00:00     
(6/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm                                                                                                                                                                  |  23 kB  00:00:00     
(7/10): socat-1.7.3.2-2.el7.x86_64.rpm                                                                                                                                                                               | 290 kB  00:00:00     
(8/10): libnetfilter_cttimeout-1.0.0-6.el7.x86_64.rpm                                                                                                                                                                |  18 kB  00:00:00     
(9/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm                                                                                                           |  10 MB  00:00:01     
(10/10): f5edc025972c2d092ac41b05877c89b50cedaa7177978d9e5e49b5a2979dbc85-kubelet-1.15.1-0.x86_64.rpm                                                                                                                |  22 MB  00:00:03     
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                                       8.7 MB/s |  56 MB  00:00:06     
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key <gc-team@google.com>"
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key <gc-team@google.com>"
 Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
 From       : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                              1/10 
  Installing : libnetfilter_cthelper-1.0.0-9.el7.x86_64                                                                                                                                                                                2/10 
  Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                 3/10 
  Installing : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                               4/10 
  Installing : libnetfilter_cttimeout-1.0.0-6.el7.x86_64                                                                                                                                                                               5/10 
  Installing : conntrack-tools-1.4.4-4.el7.x86_64                                                                                                                                                                                      6/10 
  Installing : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                           7/10 
  Installing : kubelet-1.15.1-0.x86_64                                                                                                                                                                                                 8/10 
  Installing : kubectl-1.15.1-0.x86_64                                                                                                                                                                                                 9/10 
  Installing : kubeadm-1.15.1-0.x86_64                                                                                                                                                                                                10/10 
  Verifying  : kubectl-1.15.1-0.x86_64                                                                                                                                                                                                 1/10 
  Verifying  : libnetfilter_cttimeout-1.0.0-6.el7.x86_64                                                                                                                                                                               2/10 
  Verifying  : cri-tools-1.13.0-0.x86_64                                                                                                                                                                                               3/10 
  Verifying  : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                                                                                                 4/10 
  Verifying  : libnetfilter_cthelper-1.0.0-9.el7.x86_64                                                                                                                                                                                5/10 
  Verifying  : kubelet-1.15.1-0.x86_64                                                                                                                                                                                                 6/10 
  Verifying  : kubeadm-1.15.1-0.x86_64                                                                                                                                                                                                 7/10 
  Verifying  : kubernetes-cni-0.7.5-0.x86_64                                                                                                                                                                                           8/10 
  Verifying  : socat-1.7.3.2-2.el7.x86_64                                                                                                                                                                                              9/10 
  Verifying  : conntrack-tools-1.4.4-4.el7.x86_64                                                                                                                                                                                     10/10 

Installed:
  kubeadm.x86_64 0:1.15.1-0                                                     kubectl.x86_64 0:1.15.1-0                                                     kubelet.x86_64 0:1.15.1-0                                                    

Dependency Installed:
  conntrack-tools.x86_64 0:1.4.4-4.el7  cri-tools.x86_64 0:1.13.0-0  kubernetes-cni.x86_64 0:0.7.5-0  libnetfilter_cthelper.x86_64 0:1.0.0-9.el7  libnetfilter_cttimeout.x86_64 0:1.0.0-6.el7  libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 
  socat.x86_64 0:1.7.3.2-2.el7         

Complete!
[root@kubeworker ~]#
728x90

Kubernetes Master Node 연동

1. swapoff -a

[root@kubeworker ~]# swapoff -a
[root@kubeworker ~]#

2. kubeadm join

연동 방법은 간단합니다. 기존 Master Node에서 kubeadm init 시 생성된 token 정보를 기반으로 join을 시도합니다.


[root@guruson ~]# kubeadm init 
[init] Using Kubernetes version: v1.15.1 
[preflight] Running pre-flight checks 
[preflight] Pulling images required for setting up a Kubernetes cluster 
[preflight] This might take a minute or two, depending on the speed of your internet connection 
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull' 
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml
[kubelet-start] Activating the kubelet service 
[certs] Using certificateDir folder "/etc/kubernetes/pki" 
[certs] Generating "front-proxy-ca" certificate and key 
[certs] Generating "front-proxy-client" certificate and key 
[certs] Generating "etcd/ca" certificate and key 
[certs] Generating "etcd/peer" certificate and key 
[certs] etcd/peer serving cert is signed for DNS names [guruson localhost] and IPs [222.234.124.110 127.0.0.1 ::1] 
[certs] Generating "etcd/healthcheck-client" certificate and key 
[certs] Generating "etcd/server" certificate and key 
[certs] etcd/server serving cert is signed for DNS names [guruson localhost] and IPs [222.234.124.110 127.0.0.1 ::1] 
[certs] Generating "apiserver-etcd-client" certificate and key 
[certs] Generating "ca" certificate and key 
[certs] Generating "apiserver" certificate and key 
[certs] apiserver serving cert is signed for DNS names [guruson kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 222.234.124.110] 
[certs] Generating "apiserver-kubelet-client" certificate and key 
[certs] Generating "sa" key and public key 
[kubeconfig] Using kubeconfig folder "/etc/kubernetes" 
[kubeconfig] Writing "admin.conf" kubeconfig file 
[kubeconfig] Writing "kubelet.conf" kubeconfig file 
[kubeconfig] Writing "controller-manager.conf" kubeconfig file 
[kubeconfig] Writing "scheduler.conf" kubeconfig file 
[control-plane] Using manifest folder "/etc/kubernetes/manifests" 
[control-plane] Creating static Pod manifest for "kube-apiserver" 
[control-plane] Creating static Pod manifest for "kube-controller-manager" 
[control-plane] Creating static Pod manifest for "kube-scheduler" 
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests" 
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s 
[kubelet-check] Initial timeout of 40s passed. 
[apiclient] All control plane components are healthy after 45.506400 seconds 
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace 
[kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster 
[upload-certs] Skipping phase. Please see --upload-certs 
[mark-control-plane] Marking the node guruson as control-plane by adding the label "node-role.kubernetes.io/master=''
[mark-control-plane] Marking the node guruson as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule] 
[bootstrap-token] Using token: deb19a.7yfa212rg0exg0c9 
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles 
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials 
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token 
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster 
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace 
[addons] Applied essential addon: CoreDNS 
[addons] Applied essential addon: kube-proxy 

Your Kubernetes control-plane has initialized successfully! 

To start using your cluster, you need to run the following as a regular user: 

  mkdir -p $HOME/.kube 
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config 
  sudo chown $(id -u):$(id -g) $HOME/.kube/config 

You should now deploy a pod network to the cluster. 
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: 
  https://kubernetes.io/docs/concepts/cluster-administration/addons/ 

Then you can join any number of worker nodes by running the following on each as root: 

kubeadm join 222.234.124.110:6443 --token deb19a.7yfa212rg0exg0c9 \ 
    --discovery-token-ca-cert-hash sha256:20d38dd05c158fe88fecd1b219ba9a5e02e5ea66ad612b404678571d104754c3  
[root@guruson ~]#


혹시나 키 값을 분실하였거나 Expired 되었을 경우 다음과 같이 확인 또는 재 생성이 가능합니다.

a) master node token 값 확인 (master node에서 명령어 실행)

[root@guruson ~]# kubeadm token list
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
y0ilnu.ospwoi1kbxscy9iz   13h       2019-08-04T20:29:54+09:00   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token
[root@guruson ~]# openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
325d28e81247bea994ebf2fa6bccb5af211aa3e6a2a762af800992b0080fb103
[root@guruson ~]#

위와 같이 token을 확인하고 Expired 이전 이라면 해당 그대로 사용이 가능합니다.

적용 방법은 kubeadm join --token [TOKEN] --discovery-token-ca-cert-hash sha256:[OPENSSL_ENCRYPT] [MASTER_APISERVER_IP]:6443

[root@guruson ~]# kubeadm join --token y0ilnu.ospwoi1kbxscy9iz --discovery-token-ca-cert-hash sha256:325d28e81247bea994ebf2fa6bccb5af211aa3e6a2a762af800992b0080fb103 ^C
[root@guruson ~]#

와 같습니다.

b) master node token이 만료되었을 경우 (master node에서 명령어 실행)

[root@guruson ~]# kubeadm token create
bzgy3x.ltfohixm3ijxm67l
[root@guruson ~]# kubeadm token list
TOKEN                     TTL       EXPIRES                     USAGES                   DESCRIPTION                                                EXTRA GROUPS
bzgy3x.ltfohixm3ijxm67l   23h       2019-08-05T08:15:29+09:00   authentication,signing   <none>                                                     system:bootstrappers:kubeadm:default-node-token
y0ilnu.ospwoi1kbxscy9iz   12h       2019-08-04T20:29:54+09:00   authentication,signing   The default bootstrap token generated by 'kubeadm init'.   system:bootstrappers:kubeadm:default-node-token
[root@guruson ~]#

위와 같이 신규로 생성이 가능합니다.

3. Master Node 연결 상태 확인

Master Node와 Workder Node의 연결상태를 확인합니다.

[root@guruson ~]# kubectl get nodes
NAME         STATUS    ROLES     AGE       VERSION
guruson      Ready     master    11h       v1.15.1
kubeworker   Ready     <none>    3m18s     v1.15.1
[root@guruson ~]# 

연결이 정상적으로 Ready 된 것을 확인할 수 있습니다.

4. Workder Node 상태 확인

[root@kubeworker ~]# docker ps -a
CONTAINER ID        IMAGE                   COMMAND                  CREATED             STATUS              PORTS               NAMES
703c6923dadc        weaveworks/weave-npc    "/usr/bin/weave-npc"     53 minutes ago      Up 53 minutes                           k8s_weave-npc_weave-net-prcb5_kube-system_35d5017f-b830-48d7-84df-3a40b8aebd5a_0
f9f84fe7f91e        weaveworks/weave-kube   "/home/weave/launch.   53 minutes ago      Up 53 minutes                           k8s_weave_weave-net-prcb5_kube-system_35d5017f-b830-48d7-84df-3a40b8aebd5a_0
45ae9c7dbfcc        k8s.gcr.io/kube-proxy   "/usr/local/bin/kube   53 minutes ago      Up 53 minutes                           k8s_kube-proxy_kube-proxy-6bgfw_kube-system_d70f96e3-92ac-483a-96f0-209bedd1bfad_0
1cf6f307fee1        k8s.gcr.io/pause:3.1    "/pause"                 53 minutes ago      Up 53 minutes                           k8s_POD_weave-net-prcb5_kube-system_35d5017f-b830-48d7-84df-3a40b8aebd5a_0
fae7fcda7abc        k8s.gcr.io/pause:3.1    "/pause"                 53 minutes ago      Up 53 minutes                           k8s_POD_kube-proxy-6bgfw_kube-system_d70f96e3-92ac-483a-96f0-209bedd1bfad_0
[root@kubeworker ~]# docker images
REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
k8s.gcr.io/kube-proxy   v1.15.1             89a062da739d        2 weeks ago         82.4MB
weaveworks/weave-kube   2.5.2               f04a043bb67a        2 months ago        148MB
weaveworks/weave-npc    2.5.2               5ce48e0d813c        2 months ago        49.6MB
k8s.gcr.io/pause        3.1                 da86e6ba6ca1        19 months ago       742kB
[root@kubeworker ~]# 

Workder Node는 kube-proxy와 weaveworks Docker 이미지가 기동되어 있습니다.

 

이번 포스팅에서는 Master Node와 Worker Node를 연동하는 방법에 대해 살펴보았습니다.

다음 포스팅에서는 실제 Pod를 생성하여 애플리케이션을 디플로이하기 위한 구조를 생성해 보도록 하겠습니다.

728x90
LIST
댓글
댓글쓰기 폼
250x250
Total
1,235,507
Today
1,010
Yesterday
1,980
«   2021/10   »
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            
글 보관함