https://github.com/etcd-io/etcd/releases

Etcd的简单使用

API V2与V3区别

  • 事务：ETCD V3提供了多键条件事务（multi-key conditional transactions），应用各种需要使用事务代替原来的Compare-And-Swap操作。
  • 平键空间（Flat key space）：ETCD V3不再使用目录结构，只保留键。例如：”/a/b/c/“是一个键，而不是目录。V3中提供了前缀查询，来获取符合前缀条件的所有键值，这变向实现了V2中查询一个目录下所有子目录和节点的功能。
  • 简洁的响应：像DELETE这类操作成功后将不再返回操作前的值。如果希望获得删除前的值，可以使用事务，来实现一个原子操作，先获取键值，然后再删除。
  • 租约：租约代替了V2中的TTL实现，TTL绑定到一个租约上，键再附加到这个租约上。当TTL过期时，租约将被销毁，同时附加到这个租约上的键也被删除。

安装

手动安装

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
WORKSPACE=/etcd
ETCD_DIR=etcd
ETCD_VER=v3.4.3

# choose either URL
GOOGLE_URL=https://storage.googleapis.com/etcd
GITHUB_URL=https://github.com/etcd-io/etcd/releases/download
DOWNLOAD_URL=${GOOGLE_URL}

rm -f ${WORKSPACE}/etcd-${ETCD_VER}-linux-amd64.tar.gz
rm -rf ${WORKSPACE}/${ETCD_DIR} && mkdir -p ${WORKSPACE}/${ETCD_DIR}

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o ${WORKSPACE}/etcd-${ETCD_VER}-linux-amd64.tar.gz
tar xzvf ${WORKSPACE}/etcd-${ETCD_VER}-linux-amd64.tar.gz -C ${WORKSPACE}/${ETCD_DIR} --strip-components=1
rm -f ${WORKSPACE}/etcd-${ETCD_VER}-linux-amd64.tar.gz

/etcd/etcd/etcd --version
/etcd/etcd/etcdctl version

需要翻墙

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
rm -rf /tmp/etcd-data.tmp && mkdir -p /tmp/etcd-data.tmp && \
  docker rmi gcr.io/etcd-development/etcd:v3.4.3 || true && \
  docker run \
  -p 2379:2379 \
  -p 2380:2380 \
  --mount type=bind,source=/tmp/etcd-data.tmp,destination=/etcd-data \
  --name etcd-gcr-v3.4.3 \
  gcr.io/etcd-development/etcd:v3.4.3 \
  /usr/local/bin/etcd \
  --name s1 \
  --data-dir /etcd-data \
  --listen-client-urls http://0.0.0.0:2379 \
  --advertise-client-urls http://0.0.0.0:2379 \
  --listen-peer-urls http://0.0.0.0:2380 \
  --initial-advertise-peer-urls http://0.0.0.0:2380 \
  --initial-cluster s1=http://0.0.0.0:2380 \
  --initial-cluster-token tkn \
  --initial-cluster-state new \
  --log-level info \
  --logger zap \
  --log-outputs stderr

docker exec etcd-gcr-v3.4.3 /bin/sh -c "/usr/local/bin/etcd --version"
docker exec etcd-gcr-v3.4.3 /bin/sh -c "/usr/local/bin/etcdctl version"
docker exec etcd-gcr-v3.4.3 /bin/sh -c "/usr/local/bin/etcdctl endpoint health"
docker exec etcd-gcr-v3.4.3 /bin/sh -c "/usr/local/bin/etcdctl put foo bar"
docker exec etcd-gcr-v3.4.3 /bin/sh -c "/usr/local/bin/etcdctl get foo"

简单读写

1
2
3
4
5
6
# start a local etcd server
/etcd/etcd/etcd

# write,read to etcd
/etcd/etcd/etcdctl --endpoints=localhost:2379 put foo bar
/etcd/etcd/etcdctl --endpoints=localhost:2379 get foo

Etcd可视化工具

https://github.com/shiguanghuxian/etcd-manage 

1
2
3
4
5
6
7
8
9
10
11
12
cd /etcd/
# 下载
wget https://github.com/shiguanghuxian/etcd-manage/releases/download/1.1.5/etcd-manage_linux_amd64.zip

# 解压
unzip etcd-manage_linux_amd64.zip

# 配置
vi ./config/cfg.toml

# 运行
./etcd-manage

Etcd单机启动

1
2
3
4
mkdir -p /etcd/etcdrun/data.etcd/
mkdir -p /etcd/etcdrun/log/
cd /etcd/etcdrun/
/etcd/etcd/etcd  --data-dir ./data.etcd/  --listen-client-urls http://localhost:2379 --advertise-client-urls http://localhost:2379 & >./log/etcd.log

  • -listen-client-urls

    用于指定etcd和客户端的连接端口

  • -advertise-client-urls

    用于指定etcd服务器之间通讯的端口

etcd有要求，如果-listen-client-urls被设置了，那么就必须同时设置-advertise-client-urls，所以即使设置和默认相同，也必须显式设置.

Etcd单机docker-compose

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
version: '3'

services:
  etcd:
    container_name: etcd-s1
    image: quay.io/coreos/etcd:v3.5.12
    environment:
      - ETCD_NAME=etcd-s1
      - ETCD_DATA_DIR=/var/etcd
      - ETCD_LISTEN_CLIENT_URLS=http://0.0.0.0:2379
      - ETCD_ADVERTISE_CLIENT_URLS=http://0.0.0.0:2379
      - ETCD_LISTEN_PEER_URLS=http://0.0.0.0:2380
      - ETCD_INITIAL_ADVERTISE_PEER_URLS=http://0.0.0.0:2380
      - ETCD_INITIAL_CLUSTER_TOKEN=etcd-cluster
      - ETCD_INITIAL_CLUSTER=etcd-s1=http://0.0.0.0:2380
      - ETCD_INITIAL_CLUSTER_STATE=new
      - ETCD_LOGGER=zap
      - ETCD_LOG_LeveL=info      
    volumes:
      - /data/etcd:/var/etcd
      - "/etc/localtime:/etc/localtime:ro"
    ports:
      - 2379:2379
      - 2380:2380
    restart: always
  etcdkeeper:
    image: deltaprojects/etcdkeeper
    container_name: etcdkeeper
    ports:
      - 8088:8080

Etcd数据操作

1
export ETCDCTL_API=3

etcdctl命令

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
COMMANDS:
        alarm disarm            Disarms all alarms
        alarm list              Lists all alarms
        auth disable            Disables authentication
        auth enable             Enables authentication
        check datascale         Check the memory usage of holding data for different workloads on a given server endpoint.
        check perf              Check the performance of the etcd cluster
        compaction              Compacts the event history in etcd
        defrag                  Defragments the storage of the etcd members with given endpoints
        del                     Removes the specified key or range of keys [key, range_end)
        elect                   Observes and participates in leader election
        endpoint hashkv         Prints the KV history hash for each endpoint in --endpoints
        endpoint health         Checks the healthiness of endpoints specified in `--endpoints` flag
        endpoint status         Prints out the status of endpoints specified in `--endpoints` flag
        get                     Gets the key or a range of keys
        help                    Help about any command
        lease grant             Creates leases
        lease keep-alive        Keeps leases alive (renew)
        lease list              List all active leases
        lease revoke            Revokes leases
        lease timetolive        Get lease information
        lock                    Acquires a named lock
        make-mirror             Makes a mirror at the destination etcd cluster
        member add              Adds a member into the cluster
        member list             Lists all members in the cluster
        member promote          Promotes a non-voting member in the cluster
        member remove           Removes a member from the cluster
        member update           Updates a member in the cluster
        migrate                 Migrates keys in a v2 store to a mvcc store
        move-leader             Transfers leadership to another etcd cluster member.
        put                     Puts the given key into the store
        role add                Adds a new role
        role delete             Deletes a role
        role get                Gets detailed information of a role
        role grant-permission   Grants a key to a role
        role list               Lists all roles
        role revoke-permission  Revokes a key from a role
        snapshot restore        Restores an etcd member snapshot to an etcd directory
        snapshot save           Stores an etcd node backend snapshot to a given file
        snapshot status         Gets backend snapshot status of a given file
        txn                     Txn processes all the requests in one transaction
        user add                Adds a new user
        user delete             Deletes a user
        user get                Gets detailed information of a user
        user grant-role         Grants a role to a user
        user list               Lists all users
        user passwd             Changes password of user
        user revoke-role        Revokes a role from a user
        version                 Prints the version of etcdctl
        watch                   Watches events stream on keys or prefixes

OPTIONS:
      --cacert=""                               verify certificates of TLS-enabled secure servers using this CA bundle
      --cert=""                                 identify secure client using this TLS certificate file
      --command-timeout=5s                      timeout for short running command (excluding dial timeout)
      --debug[=false]                           enable client-side debug logging
      --dial-timeout=2s                         dial timeout for client connections
  -d, --discovery-srv=""                        domain name to query for SRV records describing cluster endpoints
      --discovery-srv-name=""                   service name to query when using DNS discovery
      --endpoints=[127.0.0.1:2379]              gRPC endpoints
  -h, --help[=false]                            help for etcdctl
      --hex[=false]                             print byte strings as hex encoded strings
      --insecure-discovery[=true]               accept insecure SRV records describing cluster endpoints
      --insecure-skip-tls-verify[=false]        skip server certificate verification
      --insecure-transport[=true]               disable transport security for client connections
      --keepalive-time=2s                       keepalive time for client connections
      --keepalive-timeout=6s                    keepalive timeout for client connections
      --key=""                                  identify secure client using this TLS key file
      --password=""                             password for authentication (if this option is used, --user option shouldn't include password)
      --user=""                                 username[:password] for authentication (prompt if password is not supplied)
  -w, --write-out="simple"                      set the output format (fields, json, protobuf, simple, table)

设置、更新key

1
2
3
4
5
# 设置
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 put /test/1 "Hello world1"
# 更新
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 put /test/1 "Hello world2"

获取key

1
2
3
4
5
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 get /test/1
# 匹配前缀查询
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 get /test --prefix

$ /etcd/etcd/etcdctl --endpoints=localhost:2379 --write-out="json" get /test/1

删除key

1
2
3
4
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 del /test/1

# 匹配前缀删除
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 del /test --prefix

监听一个key

1
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 watch /test/1

申请租约

从申请开始计算时间

1
2
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 lease grant 100
lease 694d680edbc7579e granted with TTL(100s)

授权租约

节点的生命伴随着租约到期将会被删除

1
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 put --lease=694d680edbc7579e /key/1 1

租约续约

每当到期将会续约

1
2
3
4
5
6
7
$ /etcd/etcd/etcdctl --endpoints=localhost:2379 lease grant 100
lease 694d680edbc757a2 granted with TTL(100s)

$ /etcd/etcd/etcdctl --endpoints=localhost:2379 lease keep-alive 694d680edbc757a2
lease 694d680edbc757a2 keepalived with TTL(100)
lease 694d680edbc757a2 keepalived with TTL(100)
lease 694d680edbc757a2 keepalived with TTL(100)

集群

查看集群节点

1
2
$ /etcd/etcd/etcdctl member list
8e9e05c52164694d, started, default, http://localhost:2380, http://localhost:2379, false

删除集群中存在的节点

1
$ /etcd/etcd/etcdctl member remove 8e9e05c52164694d

向集群中新加节点

1
$ /etcd/etcd/etcdctl member add etcd3 http://192.168.1.100:2380

更新集群中节点

1
$ /etcd/etcd/etcdctl member update 8e9e05c52164694d

提升集群中节点

1
$ /etcd/etcd/etcdctl member promote 8e9e05c52164694d

备份和恢复

备份

1
$ /etcd/etcd/etcdctl snapshot save backup.db

查看备份文件信息 

1
$ /etcd/etcd/etcdctl --write-out=table snapshot status backup.db

恢复

1
2
3
4
5
etcdctl snapshot restore backup.db \  
-name etcd1 \ 
-initialcluster etcd1=https://192.168.2.10:2380,etcd2=https://192.168.2.11:2380,etcd3=https://192.168.2.12:2380 \
-initial-cluster-token etcd-cluster-0 \
--initial-advertise-peer-urls https://192.168.2.10:2380

etcd命令

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
Usage:

  etcd [flags]
    Start an etcd server.

  etcd --version
    Show the version of etcd.

  etcd -h | --help
    Show the help information about etcd.

  etcd --config-file
    Path to the server configuration file. Note that if a configuration file is provided, other command line flags and environment variables will be ignored.

  etcd gateway
    Run the stateless pass-through etcd TCP connection forwarding proxy.

  etcd grpc-proxy
    Run the stateless etcd v3 gRPC L7 reverse proxy.


Member:
  --name 'default'
    Human-readable name for this member.
  --data-dir '${name}.etcd'
    Path to the data directory.
  --wal-dir ''
    Path to the dedicated wal directory.
  --snapshot-count '100000'
    Number of committed transactions to trigger a snapshot to disk.
  --heartbeat-interval '100'
    Time (in milliseconds) of a heartbeat interval.
  --election-timeout '1000'
    Time (in milliseconds) for an election to timeout. See tuning documentation for details.
  --initial-election-tick-advance 'true'
    Whether to fast-forward initial election ticks on boot for faster election.
  --listen-peer-urls 'http://localhost:2380'
    List of URLs to listen on for peer traffic.
  --listen-client-urls 'http://localhost:2379'
    List of URLs to listen on for client traffic.
  --max-snapshots '5'
    Maximum number of snapshot files to retain (0 is unlimited).
  --max-wals '5'
    Maximum number of wal files to retain (0 is unlimited).
  --quota-backend-bytes '0'
    Raise alarms when backend size exceeds the given quota (0 defaults to low space quota).
  --backend-batch-interval ''
    BackendBatchInterval is the maximum time before commit the backend transaction.
  --backend-batch-limit '0'
    BackendBatchLimit is the maximum operations before commit the backend transaction.
  --max-txn-ops '128'
    Maximum number of operations permitted in a transaction.
  --max-request-bytes '1572864'
    Maximum client request size in bytes the server will accept.
  --grpc-keepalive-min-time '5s'
    Minimum duration interval that a client should wait before pinging server.
  --grpc-keepalive-interval '2h'
    Frequency duration of server-to-client ping to check if a connection is alive (0 to disable).
  --grpc-keepalive-timeout '20s'
    Additional duration of wait before closing a non-responsive connection (0 to disable).

Clustering:
  --initial-advertise-peer-urls 'http://localhost:2380'
    List of this member's peer URLs to advertise to the rest of the cluster.
  --initial-cluster 'default=http://localhost:2380'
    Initial cluster configuration for bootstrapping.
  --initial-cluster-state 'new'
    Initial cluster state ('new' or 'existing').
  --initial-cluster-token 'etcd-cluster'
    Initial cluster token for the etcd cluster during bootstrap.
    Specifying this can protect you from unintended cross-cluster interaction when running multiple clusters.
  --advertise-client-urls 'http://localhost:2379'
    List of this member's client URLs to advertise to the public.
    The client URLs advertised should be accessible to machines that talk to etcd cluster. etcd client libraries parse these URLs to connect to the cluster.
  --discovery ''
    Discovery URL used to bootstrap the cluster.
  --discovery-fallback 'proxy'
    Expected behavior ('exit' or 'proxy') when discovery services fails.
    "proxy" supports v2 API only.
  --discovery-proxy ''
    HTTP proxy to use for traffic to discovery service.
  --discovery-srv ''
    DNS srv domain used to bootstrap the cluster.
  --discovery-srv-name ''
    Suffix to the dns srv name queried when bootstrapping.
  --strict-reconfig-check 'true'
    Reject reconfiguration requests that would cause quorum loss.
  --pre-vote 'false'
    Enable to run an additional Raft election phase.
  --auto-compaction-retention '0'
    Auto compaction retention length. 0 means disable auto compaction.
  --auto-compaction-mode 'periodic'
    Interpret 'auto-compaction-retention' one of: periodic|revision. 'periodic' for duration based retention, defaulting to hours if no time unit is provided (e.g. '5m'). 'revision' for revision number based retention.
  --enable-v2 'false'
    Accept etcd V2 client requests.

Security:
  --cert-file ''
    Path to the client server TLS cert file.
  --key-file ''
    Path to the client server TLS key file.
  --client-cert-auth 'false'
    Enable client cert authentication.
  --client-crl-file ''
    Path to the client certificate revocation list file.
  --client-cert-allowed-hostname ''
    Allowed TLS hostname for client cert authentication.
  --trusted-ca-file ''
    Path to the client server TLS trusted CA cert file.
  --auto-tls 'false'
    Client TLS using generated certificates.
  --peer-cert-file ''
    Path to the peer server TLS cert file.
  --peer-key-file ''
    Path to the peer server TLS key file.
  --peer-client-cert-auth 'false'
    Enable peer client cert authentication.
  --peer-trusted-ca-file ''
    Path to the peer server TLS trusted CA file.
  --peer-cert-allowed-cn ''
    Required CN for client certs connecting to the peer endpoint.
  --peer-cert-allowed-hostname ''
    Allowed TLS hostname for inter peer authentication.
  --peer-auto-tls 'false'
    Peer TLS using self-generated certificates if --peer-key-file and --peer-cert-file are not provided.
  --peer-crl-file ''
    Path to the peer certificate revocation list file.
  --cipher-suites ''
    Comma-separated list of supported TLS cipher suites between client/server and peers (empty will be auto-populated by Go).
  --cors '*'
    Comma-separated whitelist of origins for CORS, or cross-origin resource sharing, (empty or * means allow all).
  --host-whitelist '*'
    Acceptable hostnames from HTTP client requests, if server is not secure (empty or * means allow all).

Auth:
  --auth-token 'simple'
    Specify a v3 authentication token type and its options ('simple' or 'jwt').
  --bcrypt-cost 10
    Specify the cost / strength of the bcrypt algorithm for hashing auth passwords. Valid values are between 4 and 31.

Profiling and Monitoring:
  --enable-pprof 'false'
    Enable runtime profiling data via HTTP server. Address is at client URL + "/debug/pprof/"
  --metrics 'basic'
    Set level of detail for exported metrics, specify 'extensive' to include histogram metrics.
  --listen-metrics-urls ''
    List of URLs to listen on for the metrics and health endpoints.

Logging:
  --logger 'capnslog'
    Specify 'zap' for structured logging or 'capnslog'. [WARN] 'capnslog' will be deprecated in v3.5.
  --log-outputs 'default'
    Specify 'stdout' or 'stderr' to skip journald logging even when running under systemd, or list of comma separated output targets.
  --log-level 'info'
    Configures log level. Only supports debug, info, warn, error, panic, or fatal.

v2 Proxy (to be deprecated in v4):
  --proxy 'off'
    Proxy mode setting ('off', 'readonly' or 'on').
  --proxy-failure-wait 5000
    Time (in milliseconds) an endpoint will be held in a failed state.
  --proxy-refresh-interval 30000
    Time (in milliseconds) of the endpoints refresh interval.
  --proxy-dial-timeout 1000
    Time (in milliseconds) for a dial to timeout.
  --proxy-write-timeout 5000
    Time (in milliseconds) for a write to timeout.
  --proxy-read-timeout 0
    Time (in milliseconds) for a read to timeout.

Experimental feature:
  --experimental-initial-corrupt-check 'false'
    Enable to check data corruption before serving any client/peer traffic.
  --experimental-corrupt-check-time '0s'
    Duration of time between cluster corruption check passes.
  --experimental-enable-v2v3 ''
    Serve v2 requests through the v3 backend under a given prefix.
  --experimental-backend-bbolt-freelist-type 'array'
    ExperimentalBackendFreelistType specifies the type of freelist that boltdb backend uses(array and map are supported types).
  --experimental-enable-lease-checkpoint 'false'
    ExperimentalEnableLeaseCheckpoint enables primary lessor to persist lease remainingTTL to prevent indefinite auto-renewal of long lived leases.
  --experimental-compaction-batch-limit 1000
    ExperimentalCompactionBatchLimit sets the maximum revisions deleted in each compaction batch.
  --experimental-peer-skip-client-san-verification 'false'
    Skip verification of SAN field in client certificate for peer connections.

Unsafe feature:
  --force-new-cluster 'false'
    Force to create a new one-member cluster.

CAUTIOUS with unsafe flag! It may break the guarantees given by the consensus protocol!

TO BE DEPRECATED:

  --debug 'false'
    Enable debug-level logging for etcd. [WARN] Will be deprecated in v3.5. Use '--log-level=debug' instead.
  --log-package-levels ''
    Specify a particular log level for each etcd package (eg: 'etcdmain=CRITICAL,etcdserver=DEBUG').

go实例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/coreos/etcd/clientv3"
)

var (
	dialTimeout    = 5 * time.Second
	requestTimeout = 2 * time.Second
    //endpoints      = []string{"192.168.221.136:2379"}
	endpoints      = []string{"127.0.0.1:2379"}
)

func main() {
	cli, err := clientv3.New(clientv3.Config{
		Endpoints:   endpoints,
		DialTimeout: dialTimeout,
	})
	if err != nil {
		log.Fatal(err)
	}
	defer cli.Close()

	log.Println("存储值")
	if _, err := cli.Put(context.TODO(), "/sensors", `{sensor01:{topic:"w_sensor01"}}`); err != nil {
		log.Fatal(err)
	}

	log.Println("获取值")
	if resp, err := cli.Get(context.TODO(), "/sensors"); err != nil {
		log.Fatal(err)
	} else {
		log.Println("resp: ", resp)
	}

	// see https://github.com/coreos/etcd/blob/master/clientv3/example_kv_test.go#L220
	log.Println("事务&超时")
	ctx, cancel := context.WithTimeout(context.Background(), requestTimeout)
	_, err = cli.Txn(ctx).
		If(clientv3.Compare(clientv3.Value("key"), ">", "abc")). // txn value comparisons are lexical
		Then(clientv3.OpPut("key", "XYZ")).                      // this runs, since 'xyz' > 'abc'
		Else(clientv3.OpPut("key", "ABC")).
		Commit()
	cancel()
	if err != nil {
		log.Fatal(err)
	}

	// see https://github.com/coreos/etcd/blob/master/clientv3/example_watch_test.go
	log.Println("监视")
	rch := cli.Watch(context.Background(), "", clientv3.WithPrefix())
	for wresp := range rch {
		for _, ev := range wresp.Events {
			fmt.Printf("%s %q : %q\n", ev.Type, ev.Kv.Key, ev.Kv.Value)
		}
	}
}

集群配置

配置项

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
--name etcd0 //节点名称，默认为 default

--data-dir //服务运行数据保存的路径，默认为${name}.etcd

--snapshot-count //指定有多少事务（transaction）被提交时，触发截取快照保存到磁盘

--heartbeat-interval //leader 多久发送一次心跳到 followers。默认值是 100ms

--eletion-timeout //重新投票的超时时间，如果 follow 在该时间间隔没有收到心跳包，会触发重新投票，默认为 1000 ms。

--listen-peer-urls  //和同伴通信的地址，比如http://ip:2380，如果有多个，使用逗号分隔。需要所有节点都能够访问，所以不要使用 localhost

--listen-client-urls //对外提供服务的地址：比如http://ip:2379,http://127.0.0.1:2379，客户端会连接到这里和 etcd 交互

--advertise-client-urls：对外公告的该节点客户端监听地址，这个值会告诉集群中其他节点

--initial-advertise-peer-urls http://192.168.2.55:2380 //其他member使用，其他member通过该地址与本member交互信息。一定要保证从其他member能可访问该地址。静态配置方式下，该参数的value一定要同时在--initial-cluster参数中存在。

memberID的生成受--initial-cluster-token和--initial-advertise-peer-urls影响。该节点同伴监听地址，这个值会告诉集群中其他节点

--initial-cluster //集群中所有节点的信息，格式为node1=http://ip1:2380,node2=http://ip2:2380,…，注意：这里的 node1 是节点的 --name 指定的名字；后面的 ip1:2380 是 --initial-advertise-peer-urls 指定的值。

--initial-cluster-state //新建集群的时候，这个值为 new；假如已经存在的集群，这个值为 existing。

--initial-cluster-token//创建集群的 token，这个值每个集群保持唯一。这样的话，如果你要重新创建集群，即使配置和之前一样，也会再次生成新的集群和节点 uuid；否则会导致多个集群之间的冲突，造成未知的错误。

--initial-cluster-token etcd-cluster-2    //用于区分不同集群。本地如有多个集群要设为不同。

--initial-cluster-state new    //用于指示本次是否为新建集群。有两个取值new和existing。如果填为existing，则该member启动时会尝试与其他member交互。集群初次建立时，要填为new，经尝试最后一个节点填existing也正常，其他节点不能填为existing。集群运行过程中，一个member故障后恢复时填为existing，经尝试填为new也正常。

集群启动脚本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
#!/bin/bash
export ETCDCTL_API=3
IP="192.168.221.136"

CLUSTER="s11=http://${IP}:2380,s12=http://${IP}:2480,s13=http://${IP}:2580"

nohup /etcd/etcd/etcd --name s11 --initial-advertise-peer-urls http://${IP}:2380 \
  --listen-peer-urls http://${IP}:2380 \
  --listen-client-urls http://${IP}:2379,http://127.0.0.1:2379 \
  --advertise-client-urls http://${IP}:2379,http://127.0.0.1:2379 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster ${CLUSTER} \
  --initial-cluster-state new > /dev/null 2>&1 &
  
nohup /etcd/etcd/etcd --name s12 --initial-advertise-peer-urls http://${IP}:2480 \
  --listen-peer-urls http://${IP}:2480 \
  --listen-client-urls http://${IP}:2479,http://127.0.0.1:2479 \
  --advertise-client-urls http://${IP}:2479,http://127.0.0.1:2479 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster ${CLUSTER} \
  --initial-cluster-state new > /dev/null 2>&1 &

nohup /etcd/etcd/etcd --name s13 --initial-advertise-peer-urls http://${IP}:2580 \
  --listen-peer-urls http://${IP}:2580 \
  --listen-client-urls http://${IP}:2579,http://127.0.0.1:2579 \
  --advertise-client-urls http://${IP}:2579,http://127.0.0.1:2579 \
  --initial-cluster-token etcd-cluster-1 \
  --initial-cluster ${CLUSTER} \
  --initial-cluster-state new > /dev/null 2>&1 &

查看集群

1
2
3
4
5
$ etcdctl member list

cfdda6f4fce7ec9, started, s13, http://192.168.221.136:2580, http://127.0.0.1:2579,http://192.168.221.136:2579, false
b0a4b6bdd1efe123, started, s11, http://192.168.221.136:2380, http://127.0.0.1:2379,http://192.168.221.136:2379, false
bba82993c0bc3537, started, s12, http://192.168.221.136:2480, http://127.0.0.1:2479,http://192.168.221.136:2479, false

go实例

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
package main

import (
	"context"
	"fmt"
	"log"
	"time"

	"github.com/coreos/etcd/clientv3"
)

var (
	dialTimeout    = 5 * time.Second
	requestTimeout = 2 * time.Second
    endpoints      = []string{"192.168.221.136:2379","192.168.221.136:2479","192.168.221.136:2579"}
)

func main() {
	cli, err := clientv3.New(clientv3.Config{
		Endpoints:   endpoints,
		DialTimeout: dialTimeout,
	})
	if err != nil {
		log.Fatal(err)
	}
	defer cli.Close()

	log.Println("存储值")
	if _, err := cli.Put(context.TODO(), "/sensors", `{sensor01:{topic:"w_sensor01"}}`); err != nil {
		log.Fatal(err)
	}

	log.Println("获取值")
	if resp, err := cli.Get(context.TODO(), "/sensors"); err != nil {
		log.Fatal(err)
	} else {
		log.Println("resp: ", resp)
	}

	// see https://github.com/coreos/etcd/blob/master/clientv3/example_kv_test.go#L220
	log.Println("事务&超时")
	ctx, cancel := context.WithTimeout(context.Background(), requestTimeout)
	_, err = cli.Txn(ctx).
		If(clientv3.Compare(clientv3.Value("key"), ">", "abc")). // txn value comparisons are lexical
		Then(clientv3.OpPut("key", "XYZ")).                      // this runs, since 'xyz' > 'abc'
		Else(clientv3.OpPut("key", "ABC")).
		Commit()
	cancel()
	if err != nil {
		log.Fatal(err)
	}

	// see https://github.com/coreos/etcd/blob/master/clientv3/example_watch_test.go
	log.Println("监视")
	rch := cli.Watch(context.Background(), "", clientv3.WithPrefix())
	for wresp := range rch {
		for _, ev := range wresp.Events {
			fmt.Printf("%s %q : %q\n", ev.Type, ev.Kv.Key, ev.Kv.Value)
		}
	}
}

docker部署etcd集群

配置环境变量

1
2
3
4
5
6
7
8
9
10
11
12
13
14
mkdir -p /var/log/etcd/			      #建议创建etcd日志保存目录
mkdir -p /data/etcd				      #建议创建单独的etcd数据目录
#REGISTRY=quay.io/coreos/etcd:latest  #官方仓库
REGISTRY=willdockerhub/etcd:v3.3.12   #个人仓库
TOKEN=my-etcd-01				      #设置唯一集群ID
CLUSTER_STATE=new				      #设置为新建集群
NAME_1=etcd1
NAME_2=etcd2
NAME_3=etcd3					      #设置三个节点的name
HOST_1=192.168.92.11
HOST_2=192.168.92.12
HOST_3=192.168.92.13			      #设置三个节点的IP
CLUSTER=${NAME_1}=http://${HOST_1}:2380,${NAME_2}=http://${HOST_2}:2380,${NAME_3}=http://${HOST_3}:2380
DATA_DIR=/data/etcd				      #设置集群etcd数据节点

提示：以上所有操作需要在所有节点操作。

节点1执行

1
2
3
4
5
6
7
8
9
10
11
12
13
THIS_NAME=${NAME_1}
THIS_IP=${HOST_1}
docker run -d --restart=always \
  -p 2379:2379 \
  -p 2380:2380 \
  --volume=${DATA_DIR}:/etcd-data \
  --name etcd ${REGISTRY} \
  /usr/local/bin/etcd \
  --data-dir=/etcd-data --name ${THIS_NAME} \
  --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
  --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
  --initial-cluster ${CLUSTER} \
  --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

节点2执行

1
2
3
4
5
6
7
8
9
10
11
12
13
THIS_NAME=${NAME_2}
THIS_IP=${HOST_2}
docker run -d --restart=always \
  -p 2379:2379 \
  -p 2380:2380 \
  --volume=${DATA_DIR}:/etcd-data \
  --name etcd ${REGISTRY} \
  /usr/local/bin/etcd \
  --data-dir=/etcd-data --name ${THIS_NAME} \
  --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
  --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
  --initial-cluster ${CLUSTER} \
  --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

节点3执行

1
2
3
4
5
6
7
8
9
10
11
12
13
THIS_NAME=${NAME_3}
THIS_IP=${HOST_3}
docker run -d --restart=always \
  -p 2379:2379 \
  -p 2380:2380 \
  --volume=${DATA_DIR}:/etcd-data \
  --name etcd ${REGISTRY} \
  /usr/local/bin/etcd \
  --data-dir=/etcd-data --name ${THIS_NAME} \
  --initial-advertise-peer-urls http://${THIS_IP}:2380 --listen-peer-urls http://0.0.0.0:2380 \
  --advertise-client-urls http://${THIS_IP}:2379 --listen-client-urls http://0.0.0.0:2379 \
  --initial-cluster ${CLUSTER} \
  --initial-cluster-state ${CLUSTER_STATE} --initial-cluster-token ${TOKEN}

1.2 验证集群

查看容器状态

1
2
3
[root@localhost ~]# docker ps
CONTAINER ID        IMAGE                        COMMAND                  CREATED              STATUS              PORTS                              NAMES
36b1c9db2321        willdockerhub/etcd:v3.3.12   "/usr/local/bin/et..."   About a minute ago   Up About a minute   0.0.0.0:2379-2380->2379-2380/tcp   etcd

查看集群状态

1
2
3
4
5
[root@localhost ~]# docker exec -it etcd /usr/local/bin/etcdctl cluster-health
member 1d5a95e7cc3da4a2 is healthy: got healthy result from http://192.168.92.11:2379
member 2324302b6450f04d is healthy: got healthy result from http://192.168.92.12:2379
member 7f413d967727fb2b is healthy: got healthy result from http://192.168.92.13:2379
cluster is healthy

查看主机监听端口

1
2
3
[root@localhost ~]# netstat -nltp | grep 2380
tcp6       0      0 :::2380                 :::*                    LISTEN      19601/docker-proxy- 
[root@localhost ~]#