Skip to content

CSI Controller CrashLoop when topology tags (region/zone) are assigned to a datastore #3919

@ptrvsrg

Description

@ptrvsrg

What happened?

The vSphere CSI controller enters a CrashLoopBackOff state when topology labels (region and zone) are assigned as tags on a datastore object in vCenter.

The current topology-aware host retrieval logic does not account for the possibility that region/zone tags can also be attached to datastore objects.

What did you expect to happen?

The controller should gracefully handle the case where region/zone tags are present on datastore objects. It should either:

  1. Skip datastore objects when resolving hosts by topology tags, or
  2. Properly handle datastores as valid tagged entities without crashing.

How to reproduce it (as minimally and precisely as possible)

  1. Deploy a vSphere environment with CSI driver installed.
  2. Create vSphere tags for topology categories (e.g., k8s-region and k8s-zone).
  3. Assign these tags to a datastore (in addition to or instead of hosts/clusters).
  4. Configure the CSI driver with topology awareness enabled (Labels in csi-vpshere.conf or via CSINodeTopology).
  5. Attempt to provision a PVC or wait for a full sync cycle.
  6. Observe the CSI controller pod entering CrashLoopBackOff.

Relevant logs

{
  "level": "error",
  "time": "2026-03-12T06:36:36.71438155Z",
  "caller": "common/topology.go:310",
  "msg": "unrecognised entity type found Datastore:datastore-3427.",
  "TraceId": "0c042e13-3505-4434-90a5-61791866dc0a",
  "stacktrace": "sigs.k8s.io/vsphere-csi-driver/v3/pkg/csi/service/common.fetchHosts
      /build/pkg/csi/service/common/topology.go:310
    sigs.k8s.io/vsphere-csi-driver/v3/pkg/csi/service/common.GetHostsForSegment
      /build/pkg/csi/service/common/topology.go:145
    sigs.k8s.io/vsphere-csi-driver/v3/pkg/csi/service/common/placementengine.GetSharedDatastores
      /build/pkg/csi/service/common/placementengine/placement.go:70
    sigs.k8s.io/vsphere-csi-driver/v3/pkg/csi/service/vanilla.(*controller).createBlockVolumeWithPlacementEngineForMultiVC
      /build/pkg/csi/service/vanilla/controller.go:823
    sigs.k8s.io/vsphere-csi-driver/v3/pkg/csi/service/vanilla.(*controller).CreateVolume.func1
      /build/pkg/csi/service/vanilla/controller.go:1536
    sigs.k8s.io/vsphere-csi-driver/v3/pkg/csi/service/vanilla.(*controller).CreateVolume
      /build/pkg/csi/service/vanilla/controller.go:1538
    github.com/container-storage-interface/spec/lib/go/csi._Controller_CreateVolume_Handler
      /go/pkg/mod/github.com/container-storage-interface/spec@v1.11.0/lib/go/csi/csi_grpc.pb.go:444
    google.golang.org/grpc.(*Server).processUnaryRPC
      /go/pkg/mod/google.golang.org/grpc@v1.72.1/server.go:1405
    google.golang.org/grpc.(*Server).handleStream
      /go/pkg/mod/google.golang.org/grpc@v1.72.1/server.go:1815
    google.golang.org/grpc.(*Server).serveStreams.func2.1
      /go/pkg/mod/google.golang.org/grpc@v1.72.1/server.go:1035"
}

Environment

  • vSphere CSI Driver version: >= v3.1.0
  • Kubernetes version: 1.31+
  • vSphere/vCenter version: 8.0.3
  • Installation method: Helm

/kind bug

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions