Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(controller): prevent volume/snapshot deletion if clone exists #606

Conversation

sinhaashish
Copy link
Member

Pull Request template

Please, go through these steps before you submit a PR.

Why is this PR required? What issue does it fix?:
prevent volume/snapshot deletion if clone exists

What this PR does?:
Checks if a clone exists before deleting the snapshot

Does this PR require any upgrade changes?:

If the changes in this PR are manually verified, list down the scenarios covered::

Any additional information for your reviewer? :
Mention if this PR is part of any design or a continuation of previous PRs

Checklist:

  • Fixes #
  • PR Title follows the convention of <type>(<scope>): <subject>
  • Has the change log section been updated?
  • Commit has unit tests
  • Commit has integration tests
  • (Optional) Are upgrade changes included in this PR? If not, mention the issue/PR to track:
  • (Optional) If documentation changes are required, which issue on https://github.com/openebs/website is used to track them:

PLEASE REMOVE BELOW INFORMATION BEFORE SUBMITTING

The PR title message must follow convention:
<type>(<scope>): <subject>.

Where:

  • type is defining if release will be triggering after merging submitted changes, details in CONTRIBUTING.md.
    Most common types are:

    • feat - for new features, not a new feature for build script
    • fix - for bug fixes or improvements, not a fix for build script
    • chore - changes not related to production code
    • docs - changes related to documentation
    • style - formatting, missing semi colons, linting fix etc; no significant production code changes
    • test - adding missing tests, refactoring tests; no production code change
    • refactor - refactoring production code, eg. renaming a variable or function name, there should not be any significant production code changes
  • scope is a single word that best describes where the changes fit.
    Most common scopes are like:

    • data engine (localpv, jiva, cstor)
    • feature (provisioning, backup, restore, exporter)
    • code component (api, webhook, cast, upgrade)
    • test (tests, bdd)
    • chores (version, build, log, travis)
  • subject is a single line brief description of the changes made in the pull request.

@sinhaashish
Copy link
Member Author

Even though i have raised the PR as per the request here

but i find that this feature is not supported by csi .
In our csi test we run the csi test from this repo https://github1s.com/kubernetes-csi/csi-test.

This PR fails on CI as the csi tests expects that the snapshot should be deleted even if the clone exists . Refer here

This is what is reflected in the ci output.

IMHO we should close this request as its not supported by csi.

Your inputs please @avishnu @tiagolobocastro @Abhinandan-Purkait

Copy link
Contributor

@tiagolobocastro tiagolobocastro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even though i have raised the PR as per the request here

but i find that this feature is not supported by csi . In our csi test we run the csi test from this repo https://github1s.com/kubernetes-csi/csi-test.

This PR fails on CI as the csi tests expects that the snapshot should be deleted even if the clone exists . Refer here

This is what is reflected in the ci output.

IMHO we should close this request as its not supported by csi.

Your inputs please @avishnu @tiagolobocastro @Abhinandan-Purkait

A Quick Look on the csi spec seems to show returning failed precondition on deletion is allowed - but would be better to check what K8s expects. IIRC if we do that we'll be blasted with delete retries? CC @Abhinandan-Purkait

That being said, we may be able to decouple the K8s resources from the "backend" resources. IIRC we do that on mayastor?

But I'd like some more context here, why do we want to add this validation. Does the ZFS clone break if we delete the volume/snapshot?

// snapshot delete is not supported if clones exist for this snapshot
if len(cloneList.Items) != 0 {
return status.Errorf(
codes.Internal,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be FAILED_PRECONDITION

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should first evaluate the issue with what happens if we delete the volume/snapshot.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

verified that if we delete the volume i.e pv and pvc the underneath zfs snapshot get deleted. Even though the zfs snapshot cr is available. To rectify this i have raised the PR #607

Before deleting the pvc

root@node-0-290533:~# zfs list -t all
NAME                                                                                                USED  AVAIL     REFER  MOUNTPOINT
zfspv-pool                                                                                          186K  9.20G       24K  /zfspv-pool
zfspv-pool/pvc-dd7c6d67-8451-43c3-bc56-993e9f41b1b3                                                24.5K  4.00G     24.5K  legacy
zfspv-pool/pvc-dd7c6d67-8451-43c3-bc56-993e9f41b1b3@snapshot-cc55c781-c4d5-4f7b-ac84-1b9582b469c6     0B      -     24.5K  -
zfspv-pool/pvc-dd7c6d67-8451-43c3-bc56-993e9f41b1b3@snapshot-19b7e7d1-f81f-435f-ae88-e73aa828ced8     0B      -     24.5K  -

After deleting the pvc

root@node-0-290533:~# zfs list -t all
NAME         USED  AVAIL     REFER  MOUNTPOINT
zfspv-pool   141K  9.20G       24K  /zfspv-pool

@sinhaashish
Copy link
Member Author

When the snapshot is deleted for which the clone exist . The VS and zfssnap gets deleted but the underneath snapshot and volume are intact .

When the last clone is deleted then the volume and snapshot at zfs level gets deleted , thus cleaning up all the related resource and leaving no orphaned volumes.

ashish@ubuntu-32gb-nbg1-2:~$ kubectl get pvc
NAME        STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
csi-zfspv   Bound    pvc-c12ae98a-4ee4-4ff7-b041-33d1a39bea74   4Gi        RWO            openebs-zfspv   <unset>                 11h
ashish@ubuntu-32gb-nbg1-2:~$ kubectl get vs
NAME         READYTOUSE   SOURCEPVC   SOURCESNAPSHOTCONTENT   RESTORESIZE   SNAPSHOTCLASS     SNAPSHOTCONTENT                                    CREATIONTIME   AGE
zfspv-snap   true         csi-zfspv                           4Gi           zfspv-snapclass   snapcontent-83c47462-da98-4546-b473-0e8e04e60e20   11h            11h
ashish@ubuntu-32gb-nbg1-2:~$ kubectl get zfssnap -n openebs
NAME                                            AGE
snapshot-83c47462-da98-4546-b473-0e8e04e60e20   11h
ashish@ubuntu-32gb-nbg1-2:~$ kubectl apply -f ~/sample-data/clone.yaml
persistentvolumeclaim/zfspv-clone created
ashish@ubuntu-32gb-nbg1-2:~$ kubectl get pvc
NAME          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
csi-zfspv     Bound    pvc-c12ae98a-4ee4-4ff7-b041-33d1a39bea74   4Gi        RWO            openebs-zfspv   <unset>                 11h
zfspv-clone   Bound    pvc-19134719-f097-41c3-9a93-7129cd620274   4Gi        RWO            openebs-zfspv   <unset>                 19s



ZFS volume 

root@node-0-303738:~# zfs list -t all
NAME                                                                                                USED  AVAIL     REFER  MOUNTPOINT
zfspv-pool                                                                                          192K  9.20G       24K  /zfspv-pool
zfspv-pool/pvc-19134719-f097-41c3-9a93-7129cd620274                                                   0B     4G       24K  legacy
zfspv-pool/pvc-c12ae98a-4ee4-4ff7-b041-33d1a39bea74                                                  24K  4.00G       24K  legacy
zfspv-pool/pvc-c12ae98a-4ee4-4ff7-b041-33d1a39bea74@snapshot-83c47462-da98-4546-b473-0e8e04e60e20     0B      -       24K  -

Delete the pvc and then the vs

ashish@ubuntu-32gb-nbg1-2:~$ kubectl delete pvc csi-zfspv
persistentvolumeclaim "csi-zfspv" deleted
ashish@ubuntu-32gb-nbg1-2:~$ kubectl delete pvc csi-zfspv
Error from server (NotFound): persistentvolumeclaims "csi-zfspv" not found
ashish@ubuntu-32gb-nbg1-2:~$ kubectl get pvc
NAME          STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS    VOLUMEATTRIBUTESCLASS   AGE
zfspv-clone   Bound    pvc-19134719-f097-41c3-9a93-7129cd620274   4Gi        RWO            openebs-zfspv   <unset>                 39s


ZFS volume 

root@node-0-303738:~# zfs list -t all
NAME                                                                                                USED  AVAIL     REFER  MOUNTPOINT
zfspv-pool                                                                                          192K  9.20G       24K  /zfspv-pool
zfspv-pool/pvc-19134719-f097-41c3-9a93-7129cd620274                                                   0B     4G       24K  legacy
zfspv-pool/pvc-c12ae98a-4ee4-4ff7-b041-33d1a39bea74                                                  24K  4.00G       24K  legacy
zfspv-pool/pvc-c12ae98a-4ee4-4ff7-b041-33d1a39bea74@snapshot-83c47462-da98-4546-b473-0e8e04e60e20     0B      -       24K  -

Once the clone is deleted , every thing gets deleted at zfs level

ashish@ubuntu-32gb-nbg1-2:~$ kubectl delete pvc zfspv-clone
persistentvolumeclaim "zfspv-clone" deleted
ashish@ubuntu-32gb-nbg1-2:~$ kubectl get pvc
No resources found in default namespace.

root@node-0-303738:~# zfs list -t all
NAME         USED  AVAIL     REFER  MOUNTPOINT
zfspv-pool   141K  9.20G       24K  /zfspv-pool

So its works as expected.

@sinhaashish
Copy link
Member Author

Closing the PR as this feature in not supported by csi

@codecov-commenter
Copy link

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 0.00%. Comparing base (8c402d3) to head (93678dd).
Report is 26 commits behind head on develop.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@            Coverage Diff             @@
##           develop   #606       +/-   ##
==========================================
- Coverage    95.99%      0   -96.00%     
==========================================
  Files            1      0        -1     
  Lines          574      0      -574     
==========================================
- Hits           551      0      -551     
+ Misses          19      0       -19     
+ Partials         4      0        -4     
Flag Coverage Δ
bddtests ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants