February 12, 2025

By means of attaining commencement standing from the Cloud Native Computing Foundation, CubeFS reaches an vital breakthrough as a distributed file system created by neighborhood enter. CubeFS’s commencement standing demonstrates its achieved technical sophistication whereas establishing its dependable historical past of managing manufacturing workloads on a big scale. CubeFS offers low-latency file lookups and excessive throughput storage with sturdy safety by way of separate dealing with of metadata and knowledge storage whereas remaining fitted to quite a few kinds of computing workloads. 

The inherent compatibility between CubeFS’s cloud-native design and Kubernetes achieves full automation of deployments along with rolling upgrades in addition to scalable node adaptation to satisfy growing knowledge wants. CubeFS establishes itself as a reliable high-performance answer for container-based organizations desirous to improve their storage methods due to its devoted open-source neighborhood assist and adherence to the CNCF high quality requirements.

Introduction to CubeFS

CubeFS features as a distributed file system that builders worldwide can entry underneath an open-source license. The distribution of file operations happens between MetaNodes, which deal with metadata administration duties, and DataNodes handle knowledge storage duties overseen by the Grasp Node, which coordinates cluster actions. Authored construction achieves fast file searches and maintains excessive knowledge processing pace. When knowledge nodes fail, replication mechanisms safeguard them, leading to extremely dependable assist for important large-scale functions.

Why Deploy on Kubernetes

Kubernetes affords automated container orchestration, scaling, and a constant technique to deploy microservices. By working CubeFS on Kubernetes:

  • You may rapidly add or take away MetaNodes and DataNodes to match storage wants.
  • You profit from Kubernetes options like rolling updates, well being checks, and autoscaling.
  • You may combine with the Container Storage Interface (CSI) for dynamic provisioning of volumes.

Finish-to-Finish Deployment Examples

Under are YAML manifests that illustrate an easy deployment of CubeFS on Kubernetes. They outline PersistentVolumeClaims (PVCs) for every part, plus Deployments or StatefulSets for the Grasp, MetaNodes, and DataNodes. Lastly, they present find out how to mount and use the file system from a pattern pod.

Grasp Setup

Grasp PVC

apiVersion: v1
type: PersistentVolumeClaim
metadata:
  title: cubefs-master-pvc
  labels:
    app: cubefs-master
spec:
  accessModes:
  - ReadWriteOnce
  assets:
    requests:
      storage: 5Gi
  storageClassName: <YOUR_STORAGECLASS_NAME>

Grasp Service

apiVersion: v1
type: Service
metadata:
  title: cubefs-master-svc
  labels:
    app: cubefs-master
spec:
  selector:
    app: cubefs-master
  ports:
    - title: master-port
      port: 17010
      targetPort: 17010
  sort: ClusterIP

Grasp Deployment

apiVersion: apps/v1
type: Deployment
metadata:
  title: cubefs-master-deploy
  labels:
    app: cubefs-master
spec:
  replicas: 1
  selector:
    matchLabels:
      app: cubefs-master
  template:
    metadata:
      labels:
        app: cubefs-master
    spec:
      containers:
      - title: cubefs-master
        picture: cubefs/cubefs:newest
        ports:
        - containerPort: 17010
        volumeMounts:
        - title: master-data
          mountPath: /var/lib/cubefs/grasp
        env:
        - title: MASTER_ADDR
          worth: "0.0.0.0:17010"
        - title: LOG_LEVEL
          worth: "data"
      volumes:
      - title: master-data
        persistentVolumeClaim:
          claimName: cubefs-master-pvc

MetaNode Setup

MetaNode PVC

apiVersion: v1
type: PersistentVolumeClaim
metadata:
  title: cubefs-meta-pvc
  labels:
    app: cubefs-meta
spec:
  accessModes:
  - ReadWriteOnce
  assets:
    requests:
      storage: 10Gi
  storageClassName: <YOUR_STORAGECLASS_NAME>

MetaNode StatefulSet

apiVersion: apps/v1
type: StatefulSet
metadata:
  title: cubefs-meta-sts
  labels:
    app: cubefs-meta
spec:
  serviceName: "cubefs-meta-sts"
  replicas: 2
  selector:
    matchLabels:
      app: cubefs-meta
  template:
    metadata:
      labels:
        app: cubefs-meta
    spec:
      containers:
      - title: cubefs-meta
        picture: cubefs/cubefs:newest
        ports:
        - containerPort: 17011
        volumeMounts:
        - title: meta-data
          mountPath: /var/lib/cubefs/metanode
        env:
        - title: MASTER_ENDPOINT
          worth: "cubefs-master-svc:17010"
        - title: METANODE_PORT
          worth: "17011"
        - title: LOG_LEVEL
          worth: "data"
      volumes:
      - title: meta-data
        persistentVolumeClaim:
          claimName: cubefs-meta-pvc

DataNode Setup

DataNode PVC

apiVersion: v1
type: PersistentVolumeClaim
metadata:
  title: cubefs-data-pvc
  labels:
    app: cubefs-data
spec:
  accessModes:
  - ReadWriteOnce
  assets:
    requests:
      storage: 100Gi
  storageClassName: <YOUR_STORAGECLASS_NAME>

DataNode StatefulSet

apiVersion: apps/v1
type: StatefulSet
metadata:
  title: cubefs-data-sts
  labels:
    app: cubefs-data
spec:
  serviceName: "cubefs-data-sts"
  replicas: 3
  selector:
    matchLabels:
      app: cubefs-data
  template:
    metadata:
      labels:
        app: cubefs-data
    spec:
      containers:
      - title: cubefs-data
        picture: cubefs/cubefs:newest
        ports:
        - containerPort: 17012
        volumeMounts:
        - title: data-chunk
          mountPath: /var/lib/cubefs/datanode
        env:
        - title: MASTER_ENDPOINT
          worth: "cubefs-master-svc:17010"
        - title: DATANODE_PORT
          worth: "17012"
        - title: LOG_LEVEL
          worth: "data"
      volumes:
      - title: data-chunk
        persistentVolumeClaim:
          claimName: cubefs-data-pvc

Consuming CubeFS

With the Grasp, MetaNodes, and DataNodes working, you possibly can mount CubeFS in your workloads. Under is a straightforward pod spec that makes use of a hostPath for demonstration. In follow, it’s possible you’ll choose the CubeFS CSI driver for dynamic quantity provisioning.

apiVersion: v1
type: Pod
metadata:
  title: cubefs-client-pod
spec:
  containers:
  - title: cubefs-client
    picture: cubefs/cubefs:newest
    command: ["/bin/sh"]
    args: ["-c", "while true; do sleep 3600; done"]
    securityContext:
      privileged: true
    volumeMounts:
    - title: cubefs-vol
      mountPath: /mnt/cubefs
  volumes:
  - title: cubefs-vol
    hostPath:
      path: /mnt/cubefs-host
      sort: DirectoryOrCreate

Inside this pod, you’ll run:

mount.cubefs -o grasp=cubefs-master-svc:17010 /mnt/cubefs

Test logs to make sure profitable mounting, and take a look at file I/O operations.

Submit-Deployment Checks

  • Grasp Logs: kubectl logs cubefs-master-deploy-<POD_ID>
  • MetaNode Logs: kubectl logs cubefs-meta-sts-0 and kubectl logs cubefs-meta-sts-1
  • DataNode Logs: kubectl logs cubefs-data-sts-0, and so on.
  • I/O Take a look at: Write and browse information on /mnt/cubefs to substantiate every part is functioning.

Conclusion

By means of its CNCF commencement, CubeFS achieves confirmed enterprise-grade standing as a cloud-native storage system that withstands demanding knowledge workloads. Organizations acquire easy operational storage options that enhance efficiency whereas optimizing useful resource utilization by way of CubeFS’s scalable structure and environment friendly Kubernetes integration, which additionally offers fault tolerance. CubeFS stands as a reliable selection because of options that persistently evolve because of energetic neighborhood backing empowered by CNCF commencement whereas offering key customers assist for contemporary storage options that deal with any knowledge quantity.