Skip to content

Guance Cluster Backup and Recovery

Precautions

This document introduces the backup of Kubernetes configurations (YAML files) using Velero, which does not include data from PVC volumes.

Introduction

Velero is an open-source tool that can securely back up and restore, perform disaster recovery, and migrate Kubernetes cluster resources and persistent volumes.

  • Disaster Recovery Velero reduces recovery time in cases of infrastructure loss, data corruption, and/or service disruptions.

  • Data Migration Velero achieves cluster portability by easily migrating Kubernetes resources from one cluster to another.

  • Data Protection Provides critical data protection features such as scheduled backups, retention policies, and pre/post-backup hooks for custom operations.

  • Backup Cluster Backs up entire or partial clusters' Kubernetes resources and volumes using namespace resources or label selectors.

  • Scheduled Backups Set schedules to automatically initiate backups at regular intervals.

  • Backup Hooks Configure pre-backup and post-backup hooks to execute custom operations before and after Velero backups.

Basic Information and Compatibility

Name Description
Velero Version 1.13.0
Offline Installation Support Yes
Supported Architectures amd64/arm64

Offline Resources

Name Download Link
Velero-cli Amd Download
Arm Download
Velero Image Amd Download
Arm Download

Image Import

 gunzip -c velero-amd64.tar.gz | docker load

gunzip velero-amd64.tar.gz
ctr -n=k8s.io images import velero-amd64.tar

 gunzip -c velero-arm64.tar.gz | docker load

gunzip velero-arm64.tar.gz
ctr -n=k8s.io images import velero-arm64.tar

Prerequisites

  • A deployed Kubernetes cluster. If not deployed, refer to Kubernetes Deployment, and ensure you can use kubectl to control the cluster.

Installing Velero

Object Storage Setup

Create S3 Bucket

Velero requires an object storage bucket for backups, ideally unique to a single Kubernetes cluster (see FAQ for more details). Create an S3 bucket, replacing placeholders appropriately:

BUCKET=<YOUR_BUCKET>
REGION=<YOUR_REGION>
aws s3api create-bucket \
    --bucket $BUCKET \
    --region $REGION \
    --create-bucket-configuration LocationConstraint=$REGION

us-east-1 does not support LocationConstraint. If your region is us-east-1, omit the bucket configuration:

aws s3api create-bucket \
    --bucket $BUCKET \
    --region us-east-1

Set Permissions for Velero

The following method sets permissions via an IAM user. For other methods, refer to Setting Permissions Using kube2iam.

For more information, see AWS Documentation on IAM Users.

  1. Create an IAM user:
aws iam create-user --user-name velero

If you will use Velero to back up multiple clusters with multiple S3 buckets, it's best to create a unique username for each cluster rather than the default velero.

  1. Attach policies to grant velero necessary permissions:
cat > velero-policy.json <<EOF
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeVolumes",
                "ec2:DescribeSnapshots",
                "ec2:CreateTags",
                "ec2:CreateVolume",
                "ec2:CreateSnapshot",
                "ec2:DeleteSnapshot"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:DeleteObject",
                "s3:PutObject",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts"
            ],
            "Resource": [
                "arn:aws:s3:::${BUCKET}/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::${BUCKET}"
            ]
        }
    ]
}
EOF

aws iam put-user-policy \
  --user-name velero \
  --policy-name velero \
  --policy-document file://velero-policy.json
  1. Create an access key for the user:
aws iam create-access-key --user-name velero

Expected output:

{
  "AccessKey": {
        "UserName": "velero",
        "Status": "Active",
        "CreateDate": "2017-07-31T22:24:41.576Z",
        "SecretAccessKey": <AWS_SECRET_ACCESS_KEY>,
        "AccessKeyId": <AWS_ACCESS_KEY_ID>
  }
}
  1. Create a Velero-specific credential file (credentials-velero) in your local directory:
[default]
aws_access_key_id=<AWS_ACCESS_KEY_ID>
aws_secret_access_key=<AWS_SECRET_ACCESS_KEY>

Create OSS Bucket

Velero requires an object storage bucket for backups, ideally unique to a single Kubernetes cluster. Create an OSS bucket, replacing placeholders appropriately:

BUCKET=<YOUR_BUCKET>
REGION=<YOUR_REGION>
ossutil mb oss://$BUCKET \
        --storage-class Standard \
        --acl=private

Create RAM User

  1. Create a user

Refer to the RAM User Guide in Alibaba Cloud documentation.

If you will use Velero to back up multiple clusters with multiple OSS buckets, it's best to create a unique username for each cluster rather than the default velero.

  1. Attach policies to grant velero necessary permissions:

Note, for security reasons, it's best to revoke Velero's delete permissions after completing backup or restore tasks.

{
    "Version": "1",
    "Statement": [
        {
            "Action": [
                "ecs:DescribeSnapshots",
                "ecs:CreateSnapshot",
                "ecs:DeleteSnapshot",
                "ecs:DescribeDisks",
                "ecs:CreateDisk",
                "ecs:Addtags",
                "oss:PutObject",
                "oss:GetObject",
                "oss:DeleteObject",
                "oss:GetBucket",
                "oss:ListObjects",
                "oss:ListBuckets"
            ],
            "Resource": [
                "*"
            ],
            "Effect": "Allow"
        }
    ]
}
3. Create an access key for the user:

Refer to the Creating AK section in Alibaba Cloud documentation.

  1. Create a Velero-specific credential file (credentials-velero) in your installation directory:
ALIBABA_CLOUD_ACCESS_KEY_ID=<ALIBABA_CLOUD_ACCESS_KEY_ID>
ALIBABA_CLOUD_ACCESS_KEY_SECRET=<ALIBABA_CLOUD_ACCESS_KEY_SECRET>

Install Velero CLI and Configuration

Install CLI

wget https://static.guance.com/dataflux/package/velero-v1.13.0-linux-amd64.tar.gz && tar -xvf velero-v1.13.0-linux-amd64.tar.gz && mv velero-v1.13.0-linux-amd64/velero /bin

wget https://static.guance.com/dataflux/package/velero-v1.13.0-linux-arm64.tar.gz && tar -xvf velero-v1.13.0-linux-arm64.tar.gz && mv velero-v1.13.0-linux-arm64/velero /bin

Offline Installation (Optional) CLI

tar -xvf velero-v1.13.0-linux-amd64.tar.gz && mv velero-v1.13.0-linux-amd64/velero /bin

tar -xvf velero-v1.13.0-linux-arm64.tar.gz && mv velero-v1.13.0-linux-arm64/velero /bin

Verify Installation

velero -h

Initialization

Set some environment variables:

BUCKET=<YOUR_BUCKET>
REGION=<YOUR_REGION>
BUCKETPATH=<YOUR_BUCKETPATH>

Execute the initialization command:

velero install \
    --provider aws \
    --image pubrepo.guance.com/googleimages/velero:v1.13.0 \    
    --plugins pubrepo.guance.com/googleimages/velero-plugin-for-aws:v1.9.0 \
    --bucket $BUCKET \
    --backup-location-config region=$REGION \
    --snapshot-location-config region=$REGION \
    --secret-file ./credentials-velero \
    --prefix $BUCKETPATH \    
    --backup-location-config s3ForcePathStyle="true",s3Url=https://s3.$BUCKET.amazonaws.com.cn

If using overseas nodes, modify s3Url to https://s3.$BUCKET.amazonaws.com

Set some environment variables:

BUCKET=<YOUR_BUCKET>
REGION=<YOUR_REGION>
BUCKETPATH=<YOUR_BUCKETPATH>

Execute the initialization command:

velero install \
  --provider alibabacloud \
  --image pubrepo.guance.com/googleimages/velero:v1.13.0 \
  --bucket $BUCKET \
  --secret-file ./credentials-velero \
  --use-volume-snapshots=false \
  --backup-location-config region=$REGION \
  --prefix $BUCKETPATH \
  --plugins pubrepo.guance.com/googleimages/velero-plugin-alibabacloud:v1.9.6-581f313-aliyun

Verification

Create Test Service
kubectl create deployment demo --image=nginx
kubectl get pod
Backup

Execute the backup command:

velero backup create demo

Check the backup command:

$ velero get backup
NAME   STATUS      ERRORS   WARNINGS   CREATED                         EXPIRES   STORAGE LOCATION   SELECTOR
demo   Completed   0        0          2024-03-04 18:31:28 +0800 CST   29d       default            <none>
Delete and Restore Test

Delete the service:

$ kubectl delete -n default deploy demo

deployment.apps "demo" deleted

Restore:

$ velero restore create --from-backup demo --include-namespaces default --selector app=demo

Restore request "demo-20240304184105" submitted successfully.
Run `velero restore describe demo-20240304184105` or `velero restore logs demo-20240304184105` for more details.

velero restore create --from-backup {backup_name} --restore-volumes --include-namespaces {namespace_name} --selector app={app_label}

Check:

$ kubectl get pod
NAME                    READY   STATUS    RESTARTS   AGE
demo-68b4b4d5bf-qxr26   1/1     Running   0          46s

Set Up Scheduled Backups

Back up daily at midnight (without backing up PVC data), retain for 7 days

$ velero create schedule all-guance  --schedule="0 01 * * *"  --ttl 168h
$ velero get schedule

NAME         STATUS    CREATED                         SCHEDULE    BACKUP TTL   LAST BACKUP   SELECTOR   PAUSED
all-guance   Enabled   2024-03-04 18:44:55 +0800 CST   0 1 * * *   168h0m0s     n/a           <none>     false

Uninstallation

rm -f /bin/velero

velero uninstall

Additional Information

velero  get  backup   # Check backups
velero  get  schedule # Check scheduled backups
velero  get  restore  # Check existing restores
velero  get  plugins  # Check plugins
velero restore create --from-backup all-ns-backup  # Restore all cluster backups (will not overwrite existing services)
velero restore create --from-backup all-ns-backup --include-namespaces default,nginx-example # Restore only default and nginx-example namespaces

Velero can restore resources to a different namespace from their backup source. Use the `--namespace-mappings` flag for this.
velero restore create RESTORE_NAME --from-backup BACKUP_NAME --namespace-mappings old-ns-1:new-ns-1,old-ns-2:new-ns-2
For example, the following command restores resources from the `test-velero` namespace to the `test-velero-1` namespace:
velero restore create restore-for-test --from-backup everyday-1-20210203131802 --namespace-mappings test-velero:test-velero-1
# Schedule backup 
velero create schedule prd-aws-df --schedule="0 1 * * *" --ttl 168h

Feedback

Is this page helpful? ×