[k8s] ETCD Backup&Restore(2)

문제1. ETCD Backup&Restore (k8s)

https://127.0.0.1:2379에서 실행 중인 etcd의 snapshot을 생성하고 snapshot을 /data/etcd-snapshot.db에 저장한다.
그런 다음 /data/etcd-snapshot-previous.db에 있는 기존의 이전 스냅샷을 복원한다.
etcdctl을 사용하여 서버에 연결하기 위해 다음 TLS 인증서/키가 제공된다.

• CA certificate: /etc/kubernetes/pki/etcd/ca.crt
• Client certificate: /etc/kubernetes/pki/etcd/server.crt
• Client key: /etc/kubernetes/pki/etcd/server.key

 

Solution

• https://127.0.0.1:2379에서 실행 중인 etcd의 snapshot을 생성하고 snapshot을 /data/etcd-snapshot.db에 저장한다.

$ ssh k8s-master 
Welcome to Ubuntu 22.04.1 LTS (GNU/Linux 5.15.0-60-generic x86_64)

$ sudo ETCDCTL_API=3 etcdctl \
  --endpoints=https://127.0.0.1:2379 \
  --cacert=/etc/kubernetes/pki/etcd/ca.crt \
  --cert=/etc/kubernetes/pki/etcd/server.crt \
  --key=/etc/kubernetes/pki/etcd/server.key \
  snapshot save /data/etcd-snapshot.db
{"level":"info","ts":"2023-07-10T15:17:38.710+0900","caller":"snapshot/v3_snapshot.go:65","msg":"created temporary db file","path":"/data/etcd-snapshot.db.part"}
{"level":"info","ts":"2023-07-10T15:17:38.728+0900","logger":"client","caller":"v3@v3.5.7/maintenance.go:212","msg":"opened snapshot stream; downloading"}
{"level":"info","ts":"2023-07-10T15:17:38.728+0900","caller":"snapshot/v3_snapshot.go:73","msg":"fetching snapshot","endpoint":"https://127.0.0.1:2379"}
{"level":"info","ts":"2023-07-10T15:17:38.869+0900","logger":"client","caller":"v3@v3.5.7/maintenance.go:220","msg":"completed snapshot read; closing"}
{"level":"info","ts":"2023-07-10T15:17:38.882+0900","caller":"snapshot/v3_snapshot.go:88","msg":"fetched snapshot","endpoint":"https://127.0.0.1:2379","size":"11 MB","took":"now"}
{"level":"info","ts":"2023-07-10T15:17:38.882+0900","caller":"snapshot/v3_snapshot.go:97","msg":"saved","path":"/data/etcd-snapshot.db"}
Snapshot saved at /data/etcd-snapshot.db

# 저장된 스냅샷 확인
$ sudo ls -l /data/etcd-snapshot.db
-rw------- 1 root root 10829856 Jul 10 15:17 /data/etcd-snapshot.db

• 그런 다음 /data/etcd-snapshot-previous.db에 있는 기존의 이전 스냅샷을 복원한다.

# 문제에서 별 말이 없었으므로 data-dir-location은 임의로 지정 (/var/lib/etcd-previous)
$ sudo ETCDCTL_API=3 etcdctl \
 --data-dir /var/lib/etcd-previous \
 snapshot restore /data/etcd-snapshot.db
 
2023-07-10T15:27:12+09:00	info	snapshot/v3_snapshot.go:248	restoring snapshot	{"path": "/data/etcd-snapshot.db" ...
2023-07-10T15:27:12+09:00	info	membership/store.go:141	Trimming membership information from the backend...
2023-07-10T15:27:12+09:00	info	membership/cluster.go:421	added member	{"cluster-id": "cdf818194e3a8c32", ...
2023-07-10T15:27:12+09:00	info	snapshot/v3_snapshot.go:269	restored snapshot	{"path": "/data/etcd-snapshot.db",...

# 복원이 잘 되었는지 확인
$ sudo tree /var/lib/etcd-previous/
/var/lib/etcd-previous/
└── member
    ├── snap
    │   ├── 0000000000000001-0000000000000001.snap
    │   └── db
    └── wal
        └── 0000000000000000-0000000000000000.wal

3 directories, 3 files

# etcd static pod data dir 수정
$ sudo vi /etc/kubernetes/manifests/etcd.yaml 
# /etc/kubernetes/manifests/etcd.yaml 
  volumes:
  - hostPath:
      path: /etc/kubernetes/pki/etcd
      type: DirectoryOrCreate
    name: etcd-certs
  - hostPath:
      path: /var/lib/etcd-previous
      type: DirectoryOrCreate
    name: etcd-data
status: {}