Storage: Standardize volume.size usage across drivers

[hamistao]

On non-block based drivers, filesystem volumes can be unbounded, having access to the entire storage quota of its pool. When creating such volumes on btrfs and dir, the pool's volume.size config is ignored unbounded volumes are created. But on zfs those volumes are bounded by whatever is defined on volume.size, which is inconsistent behavior. To reproduce this, one can run the following on the specified drivers and compare their behaviors:

lxc launch ubuntu:n c -s poolName
lxc storage volume create poolName vol
lxc storage volume attach poolName vol c /mnt/foo
lxc exec c -- df -h

[tomponline]

this doesnt look right, lets discuss.

[tomponline]

lxd/storage/drivers/volume: only apply volume.size on block volumes

This is incorrect.

We need to apply volume.size on filesystem volumes too.

[hamistao]

I was wondering if perhaps zfs is the only correct driver by some pre-existing standard. We can fix this inconsistency the other way around and make btrfs and dir apply volume.size in al volumes.
Unfortunately, however we proceed, I worry some user may be relying on the current behavior.

[tomponline]

When creating such volumes on btrfs and dir, the pool's volume.size config is ignored unbounded volumes are created

@hamistao please can you see if this is a regression, specifically on btrfs by testing lxd 4.0 and upwards and see if at some point in the past it was respected?

dir pools are an exception here, because the majority of the time, even if you specify explicit a size property of a volume (or instance disk device) that size won't be respected unless the underlying directory is on a mount that has project quotas enabled (I understand this to be available on ext4 and xfs filesystems).

So the question is whether we should inherit a pool's volume.size setting and convert that to a size setting for custom volumes and a volatile.rootfs.size setting for instance volumes (where the instance root disk device doesn't have size) and have it not take effect, and yet still be reported in the API, which in turn may be reported to end userrs (such as via the LXD UI).

[hamistao]

@hamistao please can you see if this is a regression, specifically on btrfs by testing lxd 4.0 and upwards and see if at some point in the past it was respected?

On LXD 4.0, btrfs already ignored volume.size when creating a filesystem volume. Here is the output of lxd exec c -- df -h, with c being a container created in a btrfs pool with volume.size=7GiB and a custom volume attached on /mnt/foo, similarly to what is described on the PR's description:

Filesystem       Size  Used Avail Use% Mounted on
/dev/loop24      **30G**  1.1G   29G   4% /
none                492K  4.0K  488K   1% /dev
efivarfs            184K  155K   25K  87% /sys/firmware/efi/efivars
tmpfs               100K     0  100K   0% /dev/lxd
tmpfs               100K     0  100K   0% /dev/.lxd-mounts
tmpfs                12G     0   12G   0% /dev/shm
tmpfs               4.7G  164K  4.7G   1% /run
tmpfs               5.0M     0  5.0M   0% /run/lock
/dev/loop24      **30G**  1.1G   29G   4% /mnt/foo

For comparison, here is the same output when using zfs on 5.0/stable (zfs wasn't present on 4.0)

Filesystem              Size  Used Avail Use% Mounted on
zfs/containers/c        7.5G  483M  **7.0G**   7% /
none                    492K  4.0K  488K   1% /dev
efivarfs                184K  155K   25K  87% /sys/firmware/efi/efivars
tmpfs                   100K     0  100K   0% /dev/lxd
tmpfs                   100K     0  100K   0% /dev/.lxd-mounts
tmpfs                    12G     0   12G   0% /dev/shm
tmpfs                   4.7G  164K  4.7G   1% /run
tmpfs                   5.0M     0  5.0M   0% /run/lock
zfs/custom/default_vol  7.0G  128K  **7.0G**   1% /mnt/foo

I will try and test with dir with project quotas as well

[tomponline]

Thanks @hamistao lets de-prioritize this for now and come back to it once we've finished the roadmap item you have, plus the size reporting issue.