Skip to content

Snapshotter cleanup fix - #18

Merged
ambarve merged 1 commit into
kevpar:fork/release/1.4from
ambarve:sn_cleanup_fix
Jun 8, 2021
Merged

ambarve merged 1 commit into
kevpar:fork/release/1.4from
ambarve:sn_cleanup_fix

Conversation

@ambarve

@ambarve ambarve commented Jun 3, 2021

Copy link
Copy Markdown
Collaborator

During snapshotter cleanup lcow/wcow snapshotters try to rename the snapshot directory
before cleaning it up. This rename operation sometimes fails if the sandbox.vhdx file
inside that directory is still open in some process (ideally it should not be open but can
stay open in case of a crash or exception). However, if this rename operation fails we
return that error and then the entire snapshot garbage collection operation fails. Due to
this we end up no cleaning other snapshots which otherwise could have been cleaned up
without any errors. If this goes on for a long time we will end up filling the entire disk
with stale snapshots. This change fixes that issues by continuing the snapshot cleanup if
the rename operation fails because of such open handles.

Signed-off-by: Amit Barve ambarve@microsoft.com

@ambarve
ambarve force-pushed the sn_cleanup_fix branch 2 times, most recently from 9354c1d to b6eb875 Compare June 3, 2021 23:40
Comment thread snapshots/lcow/lcow.go Outdated
Comment thread snapshots/windows/windows.go Outdated
During snapshotter cleanup lcow/wcow snapshotters try to rename the snapshot directory
before cleaning it up. This rename operation sometimes fails if the sandbox.vhdx file
inside that directory is still open in some process (ideally it should not be open but can
stay open in case of a crash or exception) or if the directory is busy for some other
reason. However, if this rename operation fails we return that error and then the entire
snapshot garbage collection operation fails. Due to this we end up not cleaning other
snapshots which otherwise could have been cleaned up without any errors. If this goes on
for a long time we will end up filling the entire disk with stale snapshots. This change
fixes that issue by continuing the snapshot cleanup if the rename operation fails because
of such open handles.

Signed-off-by: Amit Barve <ambarve@microsoft.com>

@msscotb msscotb left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ambarve
ambarve merged commit a828a5e into kevpar:fork/release/1.4 Jun 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants