Kubernetes实录系列记录文档完整目录参考: Kubernetes实录-目录
相关记录链接地址 ：
集群部署配置(1) CentOS7配置docker环境 集群部署配置(2) 配置企业级镜像仓库Harbor
harbor是部署容器私有镜像仓库的事实标准组件，当前版本已经发布到2.0.0(刚刚出炉，还有很多bug要修复)，其中1版本的最新小版本是1.10.3。 本篇记录是我实际配置harbor 1.10.3企业级镜像仓库服务的步骤以及遇到的坑(挫折)，例如harbor使用离线方式配置下载harbor二进制文件遇到墙等。
主机名称ip地址操作系统角色软件版本备注
opsharbor-shqs-110.99.73.38CentOS 7.6docker hubdocker-ce 19.03.11docker-compose 1.26.0harbor 1.10.3测试域名:csrepo.ejuops.com
备注：准备一个域名使用，可以是一个自定义域名不进行解析，直接配置hosts文件，临时验证使用.
一、准备工作
1. 配置系统，以及docker环境
初始化系统以及安装配置docker-ce,参考第1篇记录: CentOS7配置docker环境
2. 配置docker-compose
使用docker-compose版本为1.26.0
curl -L https://github.com/docker/compose/releases/download/1.26.0/docker-compose-
`uname -s`-
`uname -m` > /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose
3. 验证
Docker version 19.03.11, build 42e35e61f3
Active: active
(running
)
docker-compose version 1.26.0, build d4451659
二、生成自签证书
因为我这边是测试以及只是在机房内网使用，因此没有申请证书而是采用配置自签证书的方式。 域名：csrepo.ejuops.com
1. 自签证书生成
mkdir certs
cd certs
openssl genrsa -des3 -passout pass:x -out csrepo.ejuops.com.pass.key 2048
openssl rsa -passin pass:x -in csrepo.ejuops.com.pass.key -out csrepo.ejuops.com.key
openssl req -new -subj
"/C=CN/ST=ShangHai/L=ShangHai/O=eju/OU=ops/CN=csrepo.ejuops.com" -key csrepo.ejuops.com.key -out csrepo.ejuops.com.csr
openssl x509 -req -sha256 -days 3650 -in csrepo.ejuops.com.csr -signkey csrepo.ejuops.com.key -out csrepo.ejuops.com.crt
2. 可以查询证书的一些信息
notBefore
=Jun 19 03:08:01 2020 GMT
notAfter
=Jun 17 03:08:01 2030 GMT
Certificate:
Data:
Version: 1
(0x0
)
Serial Number:
c6:cb:48:7f:8b:b8:bb:55
Signature Algorithm: sha256WithRSAEncryption
Issuer: C
=CN, ST
=ShangHai, L
=ShangHai, O
=eju, OU
=ops, CN
=csrepo.ejuops.com
Validity
Not Before: Jun 19 03:08:01 2020 GMT
Not After
: Jun 17 03:08:01 2030 GMT
Subject: C
=CN, ST
=ShangHai, L
=ShangHai, O
=eju, OU
=ops, CN
=csrepo.ejuops.com
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key:
(2048 bit
)
Modulus:
xxxxxxxx
[这里省略
]
Exponent: 65537
(0x10001
)
Signature Algorithm: sha256WithRSAEncryption
YYYYYYYY
[这里省略
]
到这里证书准备完成，留在这里备用。
三、安装配置harbor
版本：1.10.3 安装模式：离线安装(offline)
1. 下载harbor二进制文件包
备注：国内github.com下载时断时续的，最好开启断点续传。
wget -c --limit-rate
=500K https://github.com/goharbor/harbor/releases/download/v1.10.3/harbor-offline-installer-v1.10.3.tgz
2. 安装harbor准备
备注：本文档所有的操作都在root下进行的。安装目录为/opt/app/harbor
解压到安装目录
~
]\#
mkdir /opt/app
~
]\#
tar zxvf harbor-offline-installer-v1.10.3.tgz -C /opt/app/
将准备好的证书放到安装目录
mv /root/certs /opt/app/harbor/
目录信息
~
]\#
cd /opt/app/harbor
~
]\#
ls -l
drwxr-xr-x 2 root root 127 Jun 19 10:40 certs
-rw-r--r-- 1 root root 3398 Jun 1 19:14 common.sh
-rw-r--r-- 1 root root 670562154 Jun 1 19:15 harbor.v1.10.3.tar.gz
-rw-r--r-- 1 root root 5882 Jun 1 19:14 harbor.yml
-rwxr-xr-x 1 root root 2284 Jun 1 19:14 install.sh
-rw-r--r-- 1 root root 11347 Jun 1 19:14 LICENSE
-rwxr-xr-x 1 root root 1749 Jun 1 19:14 prepare
修改配置文件
hostname: csrepo.ejuops.com
http:
port: 80
https:
port: 443
certificate: /opt/app/harbor/certs/csrepo.ejuops.com.crt
private_key: /opt/app/harbor/certs/csrepo.ejuops.com.key
harbor_admin_password: xxxxxxxxx
database:
password: yyyyyyyyy
data_volume: /data
log:
level: info
location: /data/logs/harbor
更新信息，并下载镜像
prepare base
dir is
set to /opt/app/harbor
Unable to
find image
'goharbor/prepare:v1.10.3' locally
v1.10.3: Pulling from goharbor/prepare
a10779388d4e: Pull complete
12b8569f46a6: Pull complete
298230e0a21f: Pull complete
ea3847dc8bc4: Pull complete
1b52894b31e5: Pull complete
ae5b50c2a4c9: Pull complete
8b279542d577: Pull complete
Digest: sha256:c757f65f753a73618c7bea3b6e5d0ab281bb60b91ae8d85b20f6d8657d703861
Status: Downloaded newer image
for goharbor/prepare:v1.10.3
Generated configuration file: /config/log/logrotate.conf
Generated configuration file: /config/log/rsyslog_docker.conf
Generated configuration file: /config/nginx/nginx.conf
Generated configuration file: /config/core/env
Generated configuration file: /config/core/app.conf
Generated configuration file: /config/registry/config.yml
Generated configuration file: /config/registryctl/env
Generated configuration file: /config/db/env
Generated configuration file: /config/jobservice/env
Generated configuration file: /config/jobservice/config.yml
Generated and saved secret to file: /secret/keys/secretkey
Generated certificate, key file: /secret/core/private_key.pem, cert file: /secret/registry/root.crt
Generated configuration file: /compose_location/docker-compose.yml
Clean up the input
dir
3. 安装harbor
安装harbor
~
]\#
cd /opt/app/harbor
~
]\# ./install.sh
[Step 0
]: checking
if docker is installed
...
[Step 1
]: checking docker-compose is installed
...
[Step 2
]: loading Harbor images
...
[Step 3
]: preparing environment
...
[Step 4
]: preparing harbor configs
...
[Step 5
]: starting Harbor
...
Creating network
"harbor_harbor" with the default driver
Creating harbor-log
...
done
Creating harbor-portal
...
done
Creating redis
...
done
Creating registryctl
...
done
Creating registry
...
done
Creating harbor-db
...
done
Creating harbor-core
...
done
Creating nginx
...
done
Creating harbor-jobservice
...
done
✔ ----Harbor has been installed and started successfully.----
查看一些信息
REPOSITORY TAG IMAGE ID CREATED SIZE
goharbor/chartmuseum-photon v1.10.3 f4cd7df55b4a 2 weeks ago 178MB
goharbor/harbor-migrator v1.10.3 524036d275a6 2 weeks ago 357MB
goharbor/redis-photon v1.10.3 6562f3c58b71 2 weeks ago 119MB
goharbor/clair-adapter-photon v1.10.3 75badac8362a 2 weeks ago 61.2MB
goharbor/clair-photon v1.10.3 3c0b186089bd 2 weeks ago 171MB
goharbor/notary-server-photon v1.10.3 d0f003ba5021 2 weeks ago 143MB
goharbor/notary-signer-photon v1.10.3 304f122f4b13 2 weeks ago 140MB
goharbor/harbor-registryctl v1.10.3 2ee885ed2f2d 2 weeks ago 102MB
goharbor/registry-photon v1.10.3 f470058ab661 2 weeks ago 84.5MB
goharbor/nginx-photon v1.10.3 8c88b468cdb0 2 weeks ago 43.6MB
goharbor/harbor-log v1.10.3 b0c9e48d015e 2 weeks ago 82MB
goharbor/harbor-jobservice v1.10.3 044b1a98e9ac 2 weeks ago 143MB
goharbor/harbor-core v1.10.3 6521cd80151b 2 weeks ago 129MB
goharbor/harbor-portal v1.10.3 7d107a7d6eb8 2 weeks ago 51.7MB
goharbor/harbor-db v1.10.3 7d10c16f9b2e 2 weeks ago 157MB
goharbor/prepare v1.10.3 be46b57a9105 2 weeks ago 163MB
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS
06030163f29e goharbor/harbor-jobservice:v1.10.3
"/harbor/harbor_jobs…" 4 minutes ago Up 4 minutes
(healthy
)
74e2240ed6ba goharbor/nginx-photon:v1.10.3
"nginx -g 'daemon of…" 4 minutes ago Up 4 minutes
(healthy
) 0.0.0.0:80-
>8080/tcp, 0.0.0.0
08a128187c24 goharbor/harbor-core:v1.10.3
"/harbor/harbor_core" 4 minutes ago Up 4 minutes
(healthy
)
58143706b0a1 goharbor/harbor-db:v1.10.3
"/docker-entrypoint.…" 4 minutes ago Up 4 minutes
(healthy
) 5432/tcp
4ba493895bef goharbor/registry-photon:v1.10.3
"/home/harbor/entryp…" 4 minutes ago Up 4 minutes
(healthy
) 5000/tcp
2b9ac9d7a907 goharbor/redis-photon:v1.10.3
"redis-server /etc/r…" 4 minutes ago Up 4 minutes
(healthy
) 6379/tcp
7665b6b28b3e goharbor/harbor-registryctl:v1.10.3
"/home/harbor/start.…" 4 minutes ago Up 4 minutes
(healthy
)
123ed5421dbe goharbor/harbor-portal:v1.10.3
"nginx -g 'daemon of…" 4 minutes ago Up 4 minutes
(healthy
) 8080/tcp
0a1317dc03c5 goharbor/harbor-log:v1.10.3
"/bin/sh -c /usr/loc…" 4 minutes ago Up 4 minutes
(healthy
) 127.0.0.1:1514-
>10514/tcp
Active Internet connections
(only servers
)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:1514 0.0.0.0:* LISTEN 4451/docker-proxy
tcp6 0 0 :::80 :::* LISTEN 5200/docker-proxy
tcp6 0 0 :::443 :::* LISTEN 5184/docker-proxy
四、使用harbor
1. 浏览器访问harbor UI
https://csrepo.ejuops.com
账户：admin
密码：xxxxxx
[实际密码
]
可以看到浏览器地址栏显示是不安全的，这是因为使用的是自签名证书导致的
2. 配置docker环境使用该harbor
docker使用自签证书的私有仓库需要配置自签证书根证书到docker环境，使用openssl配置自签证书的方式有所不同，这里使用的方式参考 配置企业级镜像仓库Harbor 生成自签证书相关内容。 docker环境配置使用自签证书的方式有多种。这里列出2中方式
私有仓库地址自签证书
https://csrepo.ejuops.comcsrepo.ejuops.com.crt
certs
]
-rw-r--r-- 1 root root 1212 Jun 19 10:40 csrepo.ejuops.com.crt
-rw-r--r-- 1 root root 1009 Jun 19 10:40 csrepo.ejuops.com.csr
-rw-r--r-- 1 root root 1675 Jun 19 10:40 csrepo.ejuops.com.key
-rw-r--r-- 1 root root 1743 Jun 19 10:40 csrepo.ejuops.com.pass.key
方法一. 配置系统全局使用该自签证书
yum -y
install ca-certificates
cp csrepo.ejuops.com.crt /usr/share/pki/ca-trust-source/anchors/
update-ca-trust
tail -n23 /etc/ssl/certs/ca-bundle.trust.crt
-----BEGIN TRUSTED CERTIFICATE-----
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
[这里是省略的意思
]
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
-----END TRUSTED CERTIFICATE-----
systemctl restart docker
docker pull hello-world:latest
docker tag hello-world:latest csrepo.ejuops.com/library/hello-world:latest
docker login -u
'admin' -p
'xxxxx' csrepo.ejuops.com
docker push csrepo.ejuops.com/library/hello-world:latest
docker pull csrepo.ejuops.com/library/hello-world:latest
latest: Pulling from library/hello-world
Digest: sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042
Status: Image is up to
date for csrepo.ejuops.com/library/hello-world:latest
csrepo.ejuops.com/library/hello-world:latest
以上说明配置成功了。
方法二. 配置docker服务自己使用自签证书
这里测试的私有仓库域名是： csrepo.ejuops.com
DOCKER_REGISTRY
=csrepo.ejuops.com
mkdir -p /etc/docker/certs.d/
${DOCKER_REGISTRY}
cp ${DOCKER_REGISTRY}.crt /etc/docker/certs.d/
${DOCKER_REGISTRY}/ca.crt
docker pull csrepo.ejuops.com/library/hello-world:latest
latest: Pulling from library/hello-world
Digest: sha256:90659bf80b44ce6be8234e6ff90a1ac34acbeb826903b02cfa0da11c82cbc042
Status: Image is up to
date for csrepo.ejuops.com/library/hello-world:latest
csrepo.ejuops.com/library/hello-world:latest
