티스토리 뷰
Harbor는 Docker Repository와 Helm Chart를 함께 관리할 수 있는 오픈소스 리포지토리이다.
Docker Repository 역할을 기본으로 Clair 기반 도커 이미지 취약점 점검, Chartmuseum 기반 Helm Repository를 함께 제공하여 통합된 클라우드 환경기반 repository를 구축할 수 있다.
본 포스팅에서는 Harbor 구축 과정은 물론, Harbor를 통해 Docker Repository, Docker Image 취약점 분석, Helm Chart Repository 등의 기본 기능 및 활용 가능한 다양한 방안을 모색해 보도록 하자.
Harbor Install
Harbor 공식 홈페이지 참조 : https://goharbor.io/docs/2.0.0/install-config/download-installer/
Harbor 공식 GitHub 참조 : https://github.com/goharbor/harbor/releases
1. Download Harbor
(curl -s https://api.github.com/repos/goharbor/harbor/releases/latest | grep browser_download_url | cut -d '"' -f 4 | grep '\.tgz$' | wget -qi -)
[root@ciserver ~]# mkdir harbor
[root@ciserver ~]# cd harbor/
[root@ciserver harbor]# curl -s https://api.github.com/repos/goharbor/harbor/releases/latest | grep browser_download_url | cut -d '"' -f 4 | grep '\.tgz$' | wget -qi -
[root@ciserver harbor]# ls -la
total 1326036
drwxr-xr-x. 2 root root 184 Aug 2 04:25 .
dr-xr-x---. 20 root root 4096 Aug 2 04:23 ..
-rw-r--r--. 1 root root 678912854 Jul 15 00:27 harbor-offline-installer-v1.10.4.tgz
-rw-r--r--. 1 root root 8484 Jul 15 00:37 harbor-online-installer-v1.10.4.tgz
[root@ciserver harbor]#
[root@ciserver harbor]# tar -xzvf harbor-offline-installer-v1.10.4.tgz
harbor/harbor.v1.10.4.tar.gz
harbor/prepare
harbor/LICENSE
harbor/install.sh
harbor/common.sh
harbor/harbor.yml
[root@ciserver harbor]#
[root@ciserver harbor]# ls
LICENSE common common.sh docker-compose.yml harbor.v1.10.4.tar.gz harbor.yml install.sh prepare
[root@ciserver harbor]#
2. Configuration Harbor
1) harbor.yml
a. hostname: 192.168.56.100
b. https 주석
#https:
#port: 443
#certificate: /your/certificate/path
#private_key: /your/private/key/path
- hostname - Harbor 접속 Domain
- https 주석 - https 사용하지 않을 경우 주석 disable 구성
2) common.sh
error "Need to install docker(17.06.0+) first and run this script again."
#exit 1
error "Need to upgrade docker package to 17.06.0+."
#exit 1
- 특정 OS에 맞는 특정 Docker version을 요구할 경우 위 exit 1을 주석처리 하여 기동 할 수 있다. 다만, 가능하면 요구하는 버전에 맞는 Docker 설치를 권고한다.
3. Harbor Install
(./install.sh --with-clair --with-chartmuseum)
[root@ciserver harbor]# ./install.sh --with-clair --with-chartmuseum
[Step 0]: checking if docker is installed ...
Need to upgrade docker package to 17.06.0+.
[Step 1]: checking docker-compose is installed ...
Note: docker-compose version: 1.25.5
[Step 2]: loading Harbor images ...
710700ba4a6b: Loading layer [==================================================>] 34.5 MB/34.5 MB
7fd57902f2bf: Loading layer [==================================================>] 8.465 MB/8.465 MB
9f7a3727b327: Loading layer [==================================================>] 67.5 MB/67.5 MB
b165ecbfa6a0: Loading layer [==================================================>] 3.072 kB/3.072 kB
618609e47ff5: Loading layer [==================================================>] 3.584 kB/3.584 kB
4941a988de67: Loading layer [==================================================>] 68.33 MB/68.33 MB
Loaded image: goharbor/chartmuseum-photon:v1.10.4
c249fd1745d2: Loading layer [==================================================>] 12.24 MB/12.24 MB
6f099dcc4dab: Loading layer [==================================================>] 42.51 MB/42.51 MB
eb32b6d20d4b: Loading layer [==================================================>] 5.632 kB/5.632 kB
5acd92618fef: Loading layer [==================================================>] 40.45 kB/40.45 kB
62b57401b9ca: Loading layer [==================================================>] 42.51 MB/42.51 MB
d7b6ded42cfb: Loading layer [==================================================>] 2.56 kB/2.56 kB
Loaded image: goharbor/harbor-core:v1.10.4
31b3ca7fa226: Loading layer [==================================================>] 63.6 MB/63.6 MB
b9972bab1402: Loading layer [==================================================>] 66.73 MB/66.73 MB
56b3ba4b4a66: Loading layer [==================================================>] 5.632 kB/5.632 kB
1654024d89fe: Loading layer [==================================================>] 2.56 kB/2.56 kB
040ec6bf5851: Loading layer [==================================================>] 2.56 kB/2.56 kB
e93cd0c30c28: Loading layer [==================================================>] 2.56 kB/2.56 kB
aed062c3be21: Loading layer [==================================================>] 2.56 kB/2.56 kB
820d1a1df842: Loading layer [==================================================>] 10.75 kB/10.75 kB
Loaded image: goharbor/harbor-db:v1.10.4
ce217f401320: Loading layer [==================================================>] 8.466 MB/8.466 MB
b324500c7da3: Loading layer [==================================================>] 3.584 kB/3.584 kB
042b5242fe78: Loading layer [==================================================>] 20.94 MB/20.94 MB
87dd45007ea3: Loading layer [==================================================>] 3.072 kB/3.072 kB
651d502d735c: Loading layer [==================================================>] 8.662 MB/8.662 MB
fe72a4614aa1: Loading layer [==================================================>] 30.42 MB/30.42 MB
Loaded image: goharbor/harbor-registryctl:v1.10.4
5de330f38841: Loading layer [==================================================>] 8.46 MB/8.46 MB
0af0ddd91395: Loading layer [==================================================>] 6.239 MB/6.239 MB
3685afd2d128: Loading layer [==================================================>] 16.04 MB/16.04 MB
d8057fcd0a39: Loading layer [==================================================>] 28.25 MB/28.25 MB
0340225731b6: Loading layer [==================================================>] 22.02 kB/22.02 kB
06d8d803f0eb: Loading layer [==================================================>] 50.52 MB/50.52 MB
Loaded image: goharbor/notary-server-photon:v1.10.4
76eab6dc7bf5: Loading layer [==================================================>] 332.6 MB/332.6 MB
c96d1ad1968e: Loading layer [==================================================>] 135.2 kB/135.2 kB
Loaded image: goharbor/harbor-migrator:v1.10.4
7426785037a5: Loading layer [==================================================>] 10.31 MB/10.31 MB
b9a0601e3558: Loading layer [==================================================>] 7.698 MB/7.698 MB
aac781885802: Loading layer [==================================================>] 223.2 kB/223.2 kB
8af4d736a2ab: Loading layer [==================================================>] 195.1 kB/195.1 kB
5fef45ce538d: Loading layer [==================================================>] 15.36 kB/15.36 kB
5f98131a71d5: Loading layer [==================================================>] 3.584 kB/3.584 kB
Loaded image: goharbor/harbor-portal:v1.10.4
528ae1964423: Loading layer [==================================================>] 12.24 MB/12.24 MB
b03ff000935f: Loading layer [==================================================>] 49.37 MB/49.37 MB
Loaded image: goharbor/harbor-jobservice:v1.10.4
6e2646825500: Loading layer [==================================================>] 89.65 MB/89.65 MB
fb20b8d71cf1: Loading layer [==================================================>] 3.072 kB/3.072 kB
d566c1cc124d: Loading layer [==================================================>] 59.9 kB/59.9 kB
c427dc7cb315: Loading layer [==================================================>] 61.95 kB/61.95 kB
Loaded image: goharbor/redis-photon:v1.10.4
6d6ba3b6ec7b: Loading layer [==================================================>] 85.27 MB/85.27 MB
7a5fdfe83ad0: Loading layer [==================================================>] 49.48 MB/49.48 MB
43de16c75891: Loading layer [==================================================>] 2.56 kB/2.56 kB
e27a79d7a642: Loading layer [==================================================>] 1.536 kB/1.536 kB
ab26083ef82a: Loading layer [==================================================>] 157.2 kB/157.2 kB
242de86f59b8: Loading layer [==================================================>] 3.017 MB/3.017 MB
Loaded image: goharbor/prepare:v1.10.4
9fd7cf078b16: Loading layer [==================================================>] 49.93 MB/49.93 MB
bffa9c13b070: Loading layer [==================================================>] 3.584 kB/3.584 kB
5bc5a2da3367: Loading layer [==================================================>] 3.072 kB/3.072 kB
d207162a345a: Loading layer [==================================================>] 2.56 kB/2.56 kB
3f5fa111d1ff: Loading layer [==================================================>] 3.072 kB/3.072 kB
6fac1f97e0a4: Loading layer [==================================================>] 3.584 kB/3.584 kB
39089450a8d3: Loading layer [==================================================>] 12.29 kB/12.29 kB
c43cc9ac71a3: Loading layer [==================================================>] 5.632 kB/5.632 kB
Loaded image: goharbor/harbor-log:v1.10.4
93dfe2d38dda: Loading layer [==================================================>] 115.3 MB/115.3 MB
a2d6890966ca: Loading layer [==================================================>] 12.15 MB/12.15 MB
008d8a39ac95: Loading layer [==================================================>] 3.072 kB/3.072 kB
a06e99290956: Loading layer [==================================================>] 49.15 kB/49.15 kB
6d0c609a7ea0: Loading layer [==================================================>] 3.584 kB/3.584 kB
cc7d9f19817b: Loading layer [==================================================>] 13.03 MB/13.03 MB
Loaded image: goharbor/clair-photon:v1.10.4
0c8c48462931: Loading layer [==================================================>] 8.466 MB/8.466 MB
7c096b7a5806: Loading layer [==================================================>] 9.71 MB/9.71 MB
f18d35335b53: Loading layer [==================================================>] 9.71 MB/9.71 MB
Loaded image: goharbor/clair-adapter-photon:v1.10.4
f55180240dc6: Loading layer [==================================================>] 10.31 MB/10.31 MB
Loaded image: goharbor/nginx-photon:v1.10.4
4a575c1c2167: Loading layer [==================================================>] 8.466 MB/8.466 MB
d0e9899aeeb5: Loading layer [==================================================>] 3.584 kB/3.584 kB
db6d9646f0e0: Loading layer [==================================================>] 3.072 kB/3.072 kB
478d5f29f1a6: Loading layer [==================================================>] 20.94 MB/20.94 MB
1fbbee6ba37e: Loading layer [==================================================>] 21.76 MB/21.76 MB
Loaded image: goharbor/registry-photon:v1.10.4
10bbb8d426b9: Loading layer [==================================================>] 14.61 MB/14.61 MB
91b66eb6b6b0: Loading layer [==================================================>] 28.25 MB/28.25 MB
58956c7bbf02: Loading layer [==================================================>] 22.02 kB/22.02 kB
1c86ba20384f: Loading layer [==================================================>] 49.09 MB/49.09 MB
Loaded image: goharbor/notary-signer-photon:v1.10.4
[Step 3]: preparing environment ...
[Step 4]: preparing harbor configs ...
prepare base dir is set to /root/harbor/harbor
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /secret/keys/secretkey
Generated certificate, key file: /secret/core/private_key.pem, cert file: /secret/registry/root.crt
Generated configuration file: /config/clair/postgres_env
Generated configuration file: /config/clair/config.yaml
Generated configuration file: /config/clair/clair_env
Generated configuration file: /config/clair-adapter/env
Generated configuration file: /config/chartserver/env
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[Step 5]: starting Harbor ...
Creating network "harbor_harbor" with the default driver
Creating network "harbor_harbor-clair" with the default driver
Creating network "harbor_harbor-chartmuseum" with the default driver
Creating harbor-log ... done
Creating registryctl ... done
Creating harbor-portal ... done
Creating chartmuseum ... done
Creating harbor-db ... done
Creating registry ... done
Creating redis ... done
Creating clair ... done
Creating harbor-core ... done
Creating nginx ... done
Creating harbor-jobservice ... done
Creating clair-adapter ... done
----Harbor has been installed and started successfully.----
[root@ciserver harbor]#
위와 같이 clair 도커 이미지 취약점 분석도구와 chartmuseum helm chart repository를 옵션으로 추가하여 함께 구성한다.
구성이 완료되면 다음과 같이 정상 기동되었는지 여부를 확인한다.
[root@ciserver harbor]# netstat -anp | grep 80 | grep LIST
tcp6 0 0 :::80 :::* LISTEN 4056/docker-proxy-c
unix 2 [ ACC ] STREAM LISTENING 16940 805/NetworkManager /var/run/NetworkManager/private-dhcp
[root@ciserver harbor]# ps -efl | grep 4056 | grep -v grep
4 S root 4056 1141 0 80 0 - 27245 futex_ 04:38 ? 00:00:00 /usr/libexec/docker/docker-proxy-current -proto tcp -host-ip 0.0.0.0 -host-port 80 -container-ip 172.19.0.10 -container-port 8080
[root@ciserver harbor]#
port는 harbor.yml에 정의한 port 정보를 기반으로 확인하며, Process의 기동 상태를 점검한다.
이와 같이 기동된 Harbor는 기본 Docker Compose를 기반으로 기동된다. 다음과 같이 Docker Compose로 기동된 Process를 다시한번 확인해 보자.
[root@ciserver harbor]# docker-compose ps
Name Command State Ports
---------------------------------------------------------------------------------------------
chartmuseum ./docker-entrypoint.sh Up (healthy) 9999/tcp
clair ./docker-entrypoint.sh Up (healthy) 6060/tcp, 6061/tcp
clair-adapter /clair-adapter/clair-adapter Up (healthy) 8080/tcp
harbor-core /harbor/harbor_core Up (healthy)
harbor-db /docker-entrypoint.sh Up (healthy) 5432/tcp
harbor-jobservice /harbor/harbor_jobservice ... Up (healthy)
harbor-log /bin/sh -c /usr/local/bin/ ... Up (healthy) 127.0.0.1:1514->10514/tcp
harbor-portal nginx -g daemon off; Up (healthy) 8080/tcp
nginx nginx -g daemon off; Up (healthy) 0.0.0.0:80->8080/tcp
redis redis-server /etc/redis.conf Up (healthy) 6379/tcp
registry /home/harbor/entrypoint.sh Up (healthy) 5000/tcp
registryctl /home/harbor/start.sh Up (healthy)
[root@ciserver harbor]#
Harbor를 구축하기 위해서는 위와 같이 12개의 Docker Process가 기동되어야 하며, 각각 구성하기 번거로운 점을 install.sh을 구성하여 손쉽게 구축할 수 있도록 Harbor는 제공하고 있다.
4. Harbor Log
특정 Harbor의 Log를 확인하고자 할 경우 /var/log/harbor 디렉토리를 확인한다.
[root@ciserver harbor]# ls -la /var/log/harbor/
total 128
drwxr-xr-x. 2 10000 10000 226 Aug 2 04:38 .
drwxr-xr-x. 9 root root 4096 Aug 2 04:38 ..
-rw-r--r--. 1 10000 10000 224 Aug 2 04:38 chartmuseum.log
-rw-r--r--. 1 10000 10000 390 Aug 2 04:38 clair-adapter.log
-rw-r--r--. 1 10000 10000 5493 Aug 2 04:43 clair.log
-rw-r--r--. 1 10000 10000 14225 Aug 2 04:54 core.log
-rw-r--r--. 1 10000 10000 6686 Aug 2 04:38 jobservice.log
-rw-r--r--. 1 10000 10000 23809 Aug 2 04:54 portal.log
-rw-r--r--. 1 10000 10000 6028 Aug 2 04:38 postgresql.log
-rw-r--r--. 1 10000 10000 6913 Aug 2 04:54 proxy.log
-rw-r--r--. 1 10000 10000 4866 Aug 2 04:53 redis.log
-rw-r--r--. 1 10000 10000 18016 Aug 2 04:54 registry.log
-rw-r--r--. 1 10000 10000 15930 Aug 2 04:54 registryctl.log
[root@ciserver harbor]#
5. Harbor Dashboard
마지막으로 위와 같이 Harbor 대시보드를 확인한다. 정상적으로 기동될 경우 Harbor는 정상 설치된 것으로 볼 수 있다.
Harbor Reconfig
위와 같이 구성된 Harbor의 설정을 변경하고 싶을 경우 아래와 같이 적용이 가능하다.
예를 들어 Harbor port를 80에서 8080으로 변경하고 싶을 경우를 가정하고 살펴보자.
1) docker-compose down -v
[root@ciserver harbor]# docker-compose down -v
Stopping harbor-jobservice ... done
Stopping clair-adapter ... done
Stopping nginx ... done
Stopping harbor-core ... done
Stopping clair ... done
Stopping redis ... done
Stopping harbor-db ... done
Stopping registry ... done
Stopping chartmuseum ... done
Stopping harbor-portal ... done
Stopping registryctl ... done
Stopping harbor-log ... done
Removing harbor-jobservice ... done
Removing clair-adapter ... done
Removing nginx ... done
Removing harbor-core ... done
Removing clair ... done
Removing redis ... done
Removing harbor-db ... done
Removing registry ... done
Removing chartmuseum ... done
Removing harbor-portal ... done
Removing registryctl ... done
Removing harbor-log ... done
Removing network harbor_harbor
Removing network harbor_harbor-clair
Removing network harbor_harbor-chartmuseum
[root@ciserver harbor]#
먼저 Harbor를 기동하고 있는 docker-compose를 다운한다.
2) harbor.yml 파일 수정
다음으로 harbor.yml 파일을 재구성한 후 (port 80 → 8080) 저장한다.
3) prepare를 이용한 변경 사항 적용
[root@ciserver harbor]# ./prepare --with-clair --with-chartmuseum
prepare base dir is set to /root/harbor/harbor
WARNING:root:WARNING: HTTP protocol is insecure. Harbor will deprecate http protocol in the future. Please make sure to upgrade to https
Clearing the configuration file: /config/log/logrotate.conf
Clearing the configuration file: /config/log/rsyslog_docker.conf
Clearing the configuration file: /config/nginx/nginx.conf
Clearing the configuration file: /config/core/env
Clearing the configuration file: /config/core/app.conf
Clearing the configuration file: /config/registry/config.yml
Clearing the configuration file: /config/registry/root.crt
Clearing the configuration file: /config/registryctl/env
Clearing the configuration file: /config/registryctl/config.yml
Clearing the configuration file: /config/db/env
Clearing the configuration file: /config/jobservice/env
Clearing the configuration file: /config/jobservice/config.yml
Clearing the configuration file: /config/clair/postgresql-init.d/README.md
Clearing the configuration file: /config/clair/postgres_env
Clearing the configuration file: /config/clair/config.yaml
Clearing the configuration file: /config/clair/clair_env
Clearing the configuration file: /config/clair-adapter/env
Clearing the configuration file: /config/chartserver/env
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
loaded secret from file: /secret/keys/secretkey
Copying offline data file for clair DB
Generated configuration file: /config/clair/postgres_env
Generated configuration file: /config/clair/config.yaml
Generated configuration file: /config/clair/clair_env
Generated configuration file: /config/clair-adapter/env
Generated configuration file: /config/chartserver/env
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input dir
[root@ciserver harbor]#
prepare 적용 시 변경 된 구성을 적용할 수 있도록 reconfiguration이 동작한다.
4) docker-compose up -d
[root@ciserver harbor]# docker-compose up -d
Creating network "harbor_harbor" with the default driver
Creating network "harbor_harbor-clair" with the default driver
Creating network "harbor_harbor-chartmuseum" with the default driver
Creating harbor-log ... done
Creating harbor-db ... done
Creating redis ... done
Creating harbor-portal ... done
Creating registryctl ... done
Creating chartmuseum ... done
Creating registry ... done
Creating clair ... done
Creating harbor-core ... done
Creating nginx ... done
Creating harbor-jobservice ... done
Creating clair-adapter ... done
[root@ciserver harbor]#
위와 같이 docker-compose up으로 Harbor를 기동한다.
5) 변경 사항 확인
위와 같이 8080으로 변경된 상태로 접속이 가능한 것을 확인할 수 있다.
다음 포스팅에서는 Harbor 활용 방안에 대해 살펴보도록 하자.
'⑤ 개발, 데이터베이스 > ⓒ CI CD' 카테고리의 다른 글
Harbor Helm Repository 활용 및 추가기능 (2) | 2020.08.02 |
---|---|
Harbor Docker Repository 활용 (0) | 2020.08.02 |
SonarQube 정적분석 및 Jenkins CI/CD 통합 (0) | 2020.05.02 |
Spinnaker를 활용한 Kubernetes CD 구성하기 (2) | 2020.03.14 |
Kubernetes Jenkins - 자동 배포환경 구성 (2/2) (0) | 2019.12.22 |
- Total
- Today
- Yesterday
- openstack token issue
- JEUS6
- apache
- Da
- 마이크로서비스
- OpenStack
- k8s
- JBoss
- SWA
- kubernetes
- node.js
- aa
- webtob
- 아키텍처
- API Gateway
- aws
- SA
- JEUS7
- 쿠버네티스
- 마이크로서비스 아키텍처
- wildfly
- jeus
- MSA
- 오픈스택
- TA
- Architecture
- Docker
- nodejs
- git
- openstack tenant
일 | 월 | 화 | 수 | 목 | 금 | 토 |
---|---|---|---|---|---|---|
1 | 2 | 3 | 4 | 5 | 6 | 7 |
8 | 9 | 10 | 11 | 12 | 13 | 14 |
15 | 16 | 17 | 18 | 19 | 20 | 21 |
22 | 23 | 24 | 25 | 26 | 27 | 28 |
29 | 30 | 31 |