Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage: Use zero for unbound volumes' total size on the API #14837

Merged
merged 9 commits into from
Jan 29, 2025

Conversation

hamistao
Copy link
Contributor

@hamistao hamistao commented Jan 21, 2025

As per discussed on #14595 (comment), this implements the standard of showing the size of unbound volumes as 0.

To keep the API consistent, also updates getting custom volume state so that total is shown even when usage is not supported and shows 0 to represent unsupporte usage of both custom and instance volumes.

@github-actions github-actions bot added Documentation Documentation needs updating API Changes to the REST API labels Jan 21, 2025
@hamistao hamistao force-pushed the zero_for_unbound_volumes branch 2 times, most recently from 2f6bb0e to 58336c3 Compare January 21, 2025 17:52
@hamistao
Copy link
Contributor Author

hamistao commented Jan 21, 2025

@tomponline In addition, we did not address what value to use for unsupported usage in our last 1-1. But throughout our discussions we stablished that:
1- It would be nice for the value we use for Usage when not supported to match the value for an unset total.
2- Total should be 0.
3- -1 is a value typically used to communicate an error.

For these reasons, volume/disk usage is being set here to be reported as 0 if not supported and -1 if an error ocurred while trying to retrieve its value.

@hamistao hamistao changed the title Storage: Zero for unbound volumes Storage: Use zero for unbound volumes' total size on the API Jan 21, 2025
@hamistao hamistao force-pushed the zero_for_unbound_volumes branch 2 times, most recently from 9042676 to 4417b69 Compare January 21, 2025 18:20
@tomponline
Copy link
Member

tomponline commented Jan 21, 2025

For these reasons, volume/disk usage is being set here to be reported as 0 if not supported and -1 if an error ocurred while trying to retrieve its value.

Does this change any existing usage values compared to lxd 6.2?

@tomponline
Copy link
Member

1- It would be nice for the value we use for Usage when not supported to match the value for an unset total. 2- Total should be 0. 3- -1 is a value typically used to communicate an error.

For these reasons, volume/disk usage is being set here to be reported as 0 if not supported and -1 if an error ocurred while trying to retrieve its value.

What situations are there when usage is not suppprted/unavailable. I suspect we may want to consider unavailable/unsupported as an error (-1) to allow differentiation from zero usage (a possibility as a genuine value).

@hamistao
Copy link
Contributor Author

hamistao commented Jan 21, 2025

Does this change any existing usage values compared to lxd 6.2?

Yes. Similarly to what happened to "total", we went from a null struct to showing -1 for unsupported usage in #14511, that was recently merged and made it into 6.2. So this would be changing that -1 to a 0.

What situations are there when usage is not suppprted/unavailable. I suspect we may want to consider unavailable/unsupported as an error (-1) to allow differentiation from zero usage (a possibility as a genuine value).

dir: doesn't support usage if project quotas are not enabled.
lvm and powerflex: don't support usage for unmounted volumes.
btrfs: doesn't support usage if quotas are disabled on btrfs, not common I suppose.

Also trying to get the usage on a snapshot volume is not supported on dir, lvm and cephfs.

We use a uint64 omitempty for custom volume usage, so perhaps using something similar would be best to keep it consistent across volume types. Like always showing 0 usage in case of error/unsupported.

type StorageVolumeStateUsage struct {
	Used uint64 `json:"used,omitempty" yaml:"used,omitempty"`

	Total int64 `json:"total" yaml:"total"`
}

cc: @tomponline

@tomponline
Copy link
Member

@hamistao does this PR effectively restore the behaviour that we had in LXD 6.1?

@hamistao
Copy link
Contributor Author

@hamistao does this PR effectively restore the behaviour that we had in LXD 6.1?

Not exactly, it keeps the main purpose of the merged PR, which is to show total even when retrieving usage is unsupported, but changes the value that we want to show for unsupported usage and for unbound total from -1 to 0.

@tomponline
Copy link
Member

Thanks. Did you check each storage driver's implementation for total size 0 if unbounded?

@tomponline
Copy link
Member

@hamistao i think we'll land this one for LXD 6.3 to partially restore the previous behaviour.

@hamistao
Copy link
Contributor Author

hamistao commented Jan 28, 2025

I agree.

Thinking practically, although this is just conjecture, I suppose our users will hardly be bothered by this reverting.
I say this because only the value for unsupported usage and the instance state's total are affected, and the latter is so broken and useless that I doubt anyone is currently relying on it.

@tomponline
Copy link
Member

I agree.

Thinking practically, although this is just conjecture, I suppose our users will hardly be bothered by this reverting. I say this because only the value for unsupported usage and the instance state's total are affected, and the latter is so broken and useless that I doubt anyone is currently relying on it.

Yes it didnt make it into any LTS which is good.

@tomponline
Copy link
Member

@hamistao so is this good to merge?

@hamistao
Copy link
Contributor Author

@tomponline I am concerned about this failing test, please give me some time to check on it

@hamistao hamistao force-pushed the zero_for_unbound_volumes branch 3 times, most recently from d3c8650 to b46ce52 Compare January 29, 2025 02:24
…limits"

As per the discussion outlined on canonical#14595 (comment), we are sticking with using 0 to represent unbounded volumes.

This reverts commit 26b8d73.

Signed-off-by: hamistao <[email protected]>
Also renames variable to a more informative `usedBytes`

Signed-off-by: hamistao <[email protected]>
Also renames variable to a more informative `usedBytes`

Signed-off-by: hamistao <[email protected]>
This omitempty is ineffective here since 0 is not a valid value for usage and any error in retrieving usage resulted, up until this point, in a null `StorageVolumeStateUsage` struct. So changing this doesn't break any previously existing behavior and this now matches how we represent unsupported DISK usage, by showing 0 usage.

Signed-off-by: hamistao <[email protected]>
Now Usage is shown as 0 when retrieving it is unsupported.

Signed-off-by: hamistao <[email protected]>
@hamistao hamistao force-pushed the zero_for_unbound_volumes branch from b46ce52 to 857e0a7 Compare January 29, 2025 03:54
@hamistao hamistao marked this pull request as ready for review January 29, 2025 03:54
@hamistao
Copy link
Contributor Author

@tomponline Sorry, I should have pinged you after I fixed the broken tests. This is ready.

Copy link
Member

@tomponline tomponline left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@tomponline
Copy link
Member

@edlerd is this OK for LXD UI's consumption ?

@edlerd
Copy link
Contributor

edlerd commented Jan 29, 2025

@edlerd is this OK for LXD UI's consumption ?

Yes, that seems all right.

@tomponline tomponline merged commit 013d76b into canonical:main Jan 29, 2025
28 checks passed
@tomponline
Copy link
Member

@MusicDin @roosterfish are all volumes in Powerflex and Pure bounded in size? Please can you double check they use the convention specified in this PR? Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API Changes to the REST API Documentation Documentation needs updating
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants