csi

Ethereal Lv4
  2024-07-02 16:07:23 2024-07-02 16:07 Created   2024-07-02 16:12:09 2024-07-02 16:12 Updated  

1. 架构

  • CO：container orchestration，容器编排

  • SP：storage vendors，存储供应商

  • 主节点部署一个，从节点部署一个

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
                             CO "Master" Host
+-------------------------------------------+
|                                           |
|  +------------+           +------------+  |
|  |     CO     |   gRPC    | Controller |  |
|  |            +----------->   Plugin   |  |
|  +------------+           +------------+  |
|                                           |
+-------------------------------------------+

                            CO "Node" Host(s)
+-------------------------------------------+
|                                           |
|  +------------+           +------------+  |
|  |     CO     |   gRPC    |    Node    |  |
|  |            +----------->   Plugin   |  |
|  +------------+           +------------+  |
|                                           |
+-------------------------------------------+

Figure 1: The Plugin runs on all nodes in the cluster: a centralized
Controller Plugin is available on the CO master host and the Node
Plugin is available on all of the CO Nodes.
  • 从节点部署多个
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
                            CO "Node" Host(s)
+-------------------------------------------+
|                                           |
|  +------------+           +------------+  |
|  |     CO     |   gRPC    | Controller |  |
|  |            +--+-------->   Plugin   |  |
|  +------------+  |        +------------+  |
|                  |                        |
|                  |                        |
|                  |        +------------+  |
|                  |        |    Node    |  |
|                  +-------->   Plugin   |  |
|                           +------------+  |
|                                           |
+-------------------------------------------+

Figure 2: Headless Plugin deployment, only the CO Node hosts run
Plugins. Separate, split-component Plugins supply the Controller
Service and the Node Service respectively.
  • 从节点部署一个
1
2
3
4
5
6
7
8
9
10
11
12
13
14
                            CO "Node" Host(s)
+-------------------------------------------+
|                                           |
|  +------------+           +------------+  |
|  |     CO     |   gRPC    | Controller |  |
|  |            +----------->    Node    |  |
|  +------------+           |   Plugin   |  |
|                           +------------+  |
|                                           |
+-------------------------------------------+

Figure 3: Headless Plugin deployment, only the CO Node hosts run
Plugins. A unified Plugin component supplies both the Controller
Service and Node Service.
  • 从节点无controller
1
2
3
4
5
6
7
8
9
10
11
12
13
14
                            CO "Node" Host(s)
+-------------------------------------------+
|                                           |
|  +------------+           +------------+  |
|  |     CO     |   gRPC    |    Node    |  |
|  |            +----------->   Plugin   |  |
|  +------------+           +------------+  |
|                                           |
+-------------------------------------------+

Figure 4: Headless Plugin deployment, only the CO Node hosts run
Plugins. A Node-only Plugin component supplies only the Node Service.
Its GetPluginCapabilities RPC does not report the CONTROLLER_SERVICE
capability.

2. 卷生命周期

  • 动态配置卷
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
   CreateVolume +------------+ DeleteVolume
 +------------->|  CREATED   +--------------+
 |              +---+----^---+              |
 |       Controller |    | Controller       v
+++         Publish |    | Unpublish       +++
|X|          Volume |    | Volume          | |
+-+             +---v----+---+             +-+
                | NODE_READY |
                +---+----^---+
               Node |    | Node
            Publish |    | Unpublish
             Volume |    | Volume
                +---v----+---+
                | PUBLISHED  |
                +------------+

Figure 5: The lifecycle of a dynamically provisioned volume, from
creation to destruction.
  • 动态配置卷+STAGE_UNSTAGE_VOLUME能力
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
   CreateVolume +------------+ DeleteVolume
 +------------->|  CREATED   +--------------+
 |              +---+----^---+              |
 |       Controller |    | Controller       v
+++         Publish |    | Unpublish       +++
|X|          Volume |    | Volume          | |
+-+             +---v----+---+             +-+
                | NODE_READY |
                +---+----^---+
               Node |    | Node
              Stage |    | Unstage
             Volume |    | Volume
                +---v----+---+
                |  VOL_READY |
                +---+----^---+
               Node |    | Node
            Publish |    | Unpublish
             Volume |    | Volume
                +---v----+---+
                | PUBLISHED  |
                +------------+

Figure 6: The lifecycle of a dynamically provisioned volume, from
creation to destruction, when the Node Plugin advertises the
STAGE_UNSTAGE_VOLUME capability.
  • 提前配置的卷（只需要验证能力即可，不需要调用创建）
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
    Controller                  Controller
       Publish                  Unpublish
        Volume  +------------+  Volume
 +------------->+ NODE_READY +--------------+
 |              +---+----^---+              |
 |             Node |    | Node             v
+++         Publish |    | Unpublish       +++
|X| <-+      Volume |    | Volume          | |
+++   |         +---v----+---+             +-+
 |    |         | PUBLISHED  |
 |    |         +------------+
 +----+
   Validate
   Volume
   Capabilities

Figure 7: The lifecycle of a pre-provisioned volume that requires
controller to publish to a node (`ControllerPublishVolume`) prior to
publishing on the node (`NodePublishVolume`).
  • 自定义的
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
       +-+  +-+
       |X|  | |
       +++  +^+
        |    |
   Node |    | Node
Publish |    | Unpublish
 Volume |    | Volume
    +---v----+---+
    | PUBLISHED  |
    +------------+

Figure 8: Plugins MAY forego other lifecycle steps by contraindicating
them via the capabilities API. Interactions with the volumes of such
plugins is reduced to `NodePublishVolume` and `NodeUnpublishVolume`
calls.

3. 接口

  • RPC接口分类

    • Identity Service: Both the Node Plugin and the Controller Plugin MUST implement this sets of RPCs.
    • Controller Service: The Controller Plugin MUST implement this sets of RPCs.
    • Node Service: The Node Plugin MUST implement this sets of RPCs.

  • 能力

    • 由Identity Service决定，当调用GetPluginCapabilities时获取。主要分为两种，Service能力和VolumeExpansion能力。CSI插件默认使用Service能力中的CONTROLLER_SERVICE。返回的接口可以是一个指针数组，包含多个能力。当指定CONTROLLER_SERVICE时，自动包含如下能力

      • CREATE_DELETE_VOLUME: 表示插件支持创建和删除卷的操作。

      • PUBLISH_UNPUBLISH_VOLUME: 表示插件支持挂载和卸载卷的操作。

      • LIST_VOLUMES: 表示插件支持列出所有卷的操作。

      • GET_CAPACITY: 表示插件支持获取存储容量的操作。

      • CREATE_DELETE_SNAPSHOT: 表示插件支持创建和删除快照的操作。

      • LIST_SNAPSHOTS: 表示插件支持列出所有快照的操作。

      • CLONE_VOLUME: 表示插件支持克隆卷的操作。

      • EXPAND_VOLUME: 表示插件支持扩展卷的操作。

      • VOLUME_CONDITION: 表示插件支持卷状态的查询。

    • Controller能力

      • ControllerServiceCapability：即上面的，可以指定数组

    • Node能力

      • NodeGetCapabilities：可以指定数组

    • Group Controller能力

      • GroupControllerGetCapabilities：只有CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT一种权限

    • 卷能力

      • 通过在create时带VolumeCapability参数，或者通过ValidateVolumeCapabilities指定。

      • ValidateVolumeCapabilities作用

        • 预先创建的卷

        • 验证能力匹配：ValidateVolumeCapabilities 方法的主要目的是验证卷的能力是否符合要求。即使在创建卷时指定了卷的能力，调用方仍然需要通过 ValidateVolumeCapabilities 方法来确保卷在后续操作中具备所需的能力。这个过程可以增强存储操作的安全性和稳定性。

        • 动态检查：在创建卷时指定的能力可能会与实际情况不符。例如，存储系统的配置可能发生变化，导致卷的能力发生变化。通过在每个操作之前动态检查卷的能力，可以确保操作的顺利进行。

        • 插件互操作性：ValidateVolumeCapabilities 方法允许存储插件根据调用方的请求动态地确定卷的能力。插件可以根据请求中指定的能力来验证卷是否满足要求，从而提高插件之间的互操作性。

  • Controller侧

    • ListVolumes

      • 显示已经加载的卷：LIST_VOLUMES

    • ControllerGetVolume

      • 获取Volume：VOLUME_CONDITION

    • ControllerModifyVolume

      • 修改Volume：MODIFY_VOLUME，可能不需要实现

      • 注意，任何修改或者新建都可选需要传入serect

    • GetCapacity

      • 获取容量：GET_CAPACITY

    • ControllerGetCapabilities

      • 获取能力：必须实现

    • CreateSnapshot

      • 创建快照：CREATE_DELETE_SNAPSHOT

      • ready_to_use：可以立即返回，然后后台处理

    • DeleteSnapshot

      • 删除快照：CREATE_DELETE_SNAPSHOT

    • ListSnapshots

      • 列出快照：LIST_SNAPSHOTS

    • ControllerExpandVolume

      • 扩展容量：EXPAND_VOLUME

      • 受到卷能力VolumeExpansion.ONLINE影响，若ONLINE，则任何时候都可调用，NODEEXPAND在controller之后即可，否则需要等待其就绪，等待其支持的卸载操作进行后再进行

  • Node侧

    • NodeStageVolume/NodeUnstageVolume

      • 预处理

    • NodeGetVolumeStats

      • 获取卷状态：GET_VOLUME_STATS

    • NodeGetCapabilities

      • 获取能力：必须实现

    • NodeGetInfo

      • 获取信息：PUBLISH_UNPUBLISH_VOLUME

    • NodeExpandVolume

      • 扩展容量：EXPAND_VOLUME

      • 参照上述controller扩展容量

  • Group Controller Service RPCs

    • GroupControllerGetCapabilities：若实现集体控制，则必须实现，只有CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT一种权限

    • CreateVolumeGroupSnapshot：CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT

    • DeleteVolumeGroupSnapshot：CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT

    • GetVolumeGroupSnapshot：CREATE_DELETE_GET_VOLUME_GROUP_SNAPSHOT

  • Snapshot Metadata Service RPCs

    • GetMetadataAllocated

    • GetMetadataDelta

  • Title: csi
  • Author: Ethereal
  • Created at: 2024-07-02 16:07:23
  • Updated at: 2024-07-02 16:12:09
  • Link: https://ethereal-o.github.io/2024/07/02/csi/
  • License: This work is licensed under CC BY-NC-SA 4.0.
 Comments
On this page
csi
  1. 1. 架构
  2. 2. 卷生命周期
  3. 3. 接口