Featured image of post 使用Heketi部署GlusterFS(裸盘模式)

使用Heketi部署GlusterFS(裸盘模式)

    Heketi 是一个管理 GlusterFS 存储集群的 RESTful 管理器,它可以帮助您简化卷的创建、扩展和删除等操作

本次安装为

安装 heketi

1
sudo yum install -y heketi

多台机器在其中一台中安装heketi进行配置

创建免密登录

这里使用了root 作为演示

1
2
ssh-keygen -t rsa
ssh-copy-id -p 22 root@ip

修改heketi配置

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
{
  "_port_comment": "Heketi Server Port Number",
  "port": "12345",

  "_use_auth": "Enable JWT authorization. Please enable for deployment",
  "use_auth": true,   //使用认证 生产开启

  "_jwt": "Private keys for access",
  "jwt": {
    "_admin": "Admin has access to all APIs",
    "admin": {
      "key": "adminpass" //heketi admin pass 修改
    },
    "_user": "User only has access to /volumes endpoint",
    "user": {
      "key": "userpass"  //heketi user pass 修改
    }
  },

  "_glusterfs_comment": "GlusterFS Configuration",
  "glusterfs": {
    "_executor_comment": [
      "Execute plugin. Possible choices: mock, ssh",
      "mock: This setting is used for testing and development.",
      "      It will not send commands to any node.",
      "ssh:  This setting will notify Heketi to ssh to the nodes.",
      "      It will need the values in sshexec to be configured.",
      "kubernetes: Communicate with GlusterFS containers over",
      "            Kubernetes exec api."
    ],
    "executor": "ssh",  //执行器   这里修改为ssh  默认为mock  可选kubernetes 这里没用到

    "_sshexec_comment": "SSH username and private key file information",
    "sshexec": {
      "keyfile": "/root/.ssh/id_rsa", 
      "user": "root",
      "port": "22",
      "fstab": "/etc/fstab"
    },

    "_kubeexec_comment": "Kubernetes configuration",
    "kubeexec": {
      "host" :"https://kubernetes.host:8443",
      "cert" : "/path/to/crt.file",
      "insecure": false,
      "user": "kubernetes username",
      "password": "password for kubernetes user",
      "namespace": "OpenShift project or Kubernetes namespace",
      "fstab": "Optional: Specify fstab file on node.  Default is /etc/fstab"
    },

    "_db_comment": "Database file name",
    "db": "/var/lib/heketi/heketi.db",

    "_loglevel_comment": [
      "Set log level. Choices are:",
      "  none, critical, error, warning, info, debug",
      "Default is warning"
    ],
    "loglevel" : "warning"
  }
}

通过heketi-topology.json创建集群

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
{
  "clusters": [
    {
      "nodes": [
        {
          "node": {
            "hostnames": {
              "manage": [
                "192.168.1.100"
              ],
              "storage": [
                "192.168.1.100"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sdb","/dev/sdc"
          ]
        },
        {
          "node": {
            "hostnames": {
              "manage": [
               "192.168.1.200"
              ],
              "storage": [
                "192.168.1.200"
              ]
            },
            "zone": 1
          },
          "devices": [
            "/dev/sdb","/dev/sdc"
          ]
        }
      ]
    }
  ]
}

注意

devices为存储的设备IP 必须为裸设备 没有格式化的设备路径 多台就添加多个node

通过

1
heketi-cli topology load --json=/etc/heketi/heketi-topology.json

创建集群

创建成功之后通过查看集群

1
heketi-cli  topology info

如自定义了端口和密码

需要添加参数行

1
2
3
4
heketi-cli --user=admin --secret=adminpass --server http://127.0.0.1:38187  topology info


heketi-cli --user=admin --secret=adminpass --server http://127.0.0.1:38187  topology load --json=/etc/heketi/heketi-topology.json

其他相关命令

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
禁用所有设备
heketi-cli topology info | grep Size | awk '{print $1}' | cut -d: -f 2 |xargs -i  device disable {}

删除所有设备
heketi-cli topology info | grep Size | awk '{print $1}' | cut -d: -f 2 |xargs -i  device remove {}

移除所有设备
heketi-cli topology info | grep Size | awk '{print $1}' | cut -d: -f 2 |xargs -i  device delete {}

删除所有节点
heketi-cli node list|awk '{print $1}' | cut -d: -f 2 |xargs -i  node delete {}
1
2
删除集群    
heketi-cli cluster delete clusterid

集群创建完成之后在k8s中挂载储存类即可使用

持久卷k8s在创建pod时会自动创建

使用 Hugo 构建
主题 StackJimmy 设计