Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove corrupt/missing sectors #85

Open
1 of 5 tasks
n8maninger opened this issue Jun 16, 2023 · 2 comments
Open
1 of 5 tasks

Remove corrupt/missing sectors #85

n8maninger opened this issue Jun 16, 2023 · 2 comments
Labels
bug Something isn't working feature A new feature request or enhancement

Comments

@n8maninger
Copy link
Member

n8maninger commented Jun 16, 2023

Description

Extends hostd's data integrity functionality allowing early detection and notification of corrupt/missing sectors.

Missing sectors are sectors whose volume is not currently loaded. This should be considered "temporary," as the volume may become available later. The missing sector references should be permanently removed if the volume is later forcibly removed. Corrupt sectors are sectors whose data does not match the expected sector root. This shouldn't happen during regular operation but is possible if a disk malfunctions. References to corrupt sectors should be removed as soon as they are detected, enabling the sector to be reuploaded and repaired.

When a volume is removed, sectors are migrated to a new volume. If the sector is missing or corrupt, removal is immediately stopped. However, when a volume is "force" removed, the integrity check is ignored and references to missing or corrupt sectors are migrated within the database to new locations. However, the sectors do not physically exist on disk. Instead, the reference to the sector should be removed.

  • remove references to missing and corrupt sectors during forced volume removal
  • remove references to corrupt sectors during volume shrink
  • remove references to corrupt sectors during downloads
  • remove references to corrupt sectors during integrity checks
  • show corrupt and missing sectors in the UI

Version

No response

What operating system are you running (e.g. Ubuntu 22.04, macOS, Windows 11)?

No response

Anything else?

No response

@n8maninger n8maninger added bug Something isn't working feature A new feature request or enhancement labels Jun 16, 2023
@n8maninger n8maninger self-assigned this Jun 16, 2023
@n8maninger n8maninger modified the milestones: v0.1.0-beta, v0.2.0-beta Jun 16, 2023
@n8maninger n8maninger modified the milestones: v0.2.0, v0.3.0 Jul 11, 2023
@n8maninger n8maninger modified the milestones: v0.3.0, v0.2.0 Aug 9, 2023
@n8maninger n8maninger removed this from the v0.2.0 milestone Aug 18, 2023
@n8maninger
Copy link
Member Author

Added support for removing lost/corrupt sectors during force remove in #318

@timrettop
Copy link

timrettop commented Jul 5, 2024

Can I make a request associated to this? I can create a separate ticket if you prefer.

I occassionally have concerns about data integrity on certain volumes, it would be nice to be able to check all contracts that have data on that volume. I guess this would not be as useful if there is automated integrity checks, but I'm not sure where development plans are on that.

Oh and thank you so much for this work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working feature A new feature request or enhancement
Projects
Status: Backlog
Status: Backlog
Development

No branches or pull requests

2 participants