zfs-2.3.0-rc5 patchset #16875
Merged
zfs-2.3.0-rc5 patchset #16875
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sizeof("foo") includes the trailing null byte, so all the output had
nulls through it. Most terminals quietly ignore it, but it makes some
tools misdetect file types and other annoyances.
Easy fix: subtract 1.
Sponsored-by: https://despairlabs.com/sponsor/
Reviewed-by: Brian Behlendorf <[email protected]>
Reviewed-by: Alexander Motin <[email protected]>
Signed-off-by: Rob Norris <[email protected]>
Closes openzfs#16862
Reviewed-by: Tino Reichardt <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: poscat <[email protected]> Closes openzfs#16861
The first time a device returns ENOTSUP in repsonse to a flush request, we set vdev_nowritecache so we don't issue flushes in the future and instead just pretend the succeeded. However, we still return an error for the initial flush, even though we just decided such errors are meaningless! So, when setting vdev_nowritecache in response to a flush error, also reset the error code to assume success. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#16855
It seems there's no good reason for vdev_disk & vdev_geom to explicitly detect no support for flush and set vdev_nowritecache. Instead, just signal it by setting the error to ENOTSUP, and let zio_vdev_io_assess() take care of it in one place. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#16855
The one-shot zfs-mount.service is incorrectly deemed active by Systemd after a systemctl soft-reboot. As such, soft-rebooting prevents zfs mount -a from being ran automatically. This commit makes it so that zfs-mount.service is marked as being undone by the time umount.target is reached, so that zfs.target then pulls it in again and gets it restarted after a soft reboot. Reviewed by: Brian Behlendorf <[email protected]> Signed-off-by: kotauskas <[email protected]> Closes openzfs#16845
We should not dereference rra after the last zio_nowait() is called. It seems very unlikely, but ASAN in ztest managed to catch it. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#16868
This is purely a cosmetic fix which removes a stray "no" from the configure output. Reviewed-by: Tino Reichardt <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#16867
CONFIG_KERNEL_MODE_NEON depends on CONFIG_NEON. Neither is defined on armel. Add a guard to avoid compilation errors. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Shengqi Chen <[email protected]> Closes openzfs#16871
There were checks still in place to verify we could completely use iov_iter's on the Linux side. All interfaces are available as of kernel 4.18, so there is no reason to check whether we should use that interface at this point. This PR completely removes the UIO_USERSPACE type. It also removes the check for the direct_IO interface checks. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Brian Atkinson <[email protected]> Closes openzfs#16856
As of kernel v5.8, pin_user_pages* interfaced were introduced. These interfaces use the FOLL_PIN flag. This is preferred interface now for Direct I/O requests in the kernel. The reasoning for using this new interface for Direct I/O requests is explained in the kernel documenetation: Documentation/core-api/pin_user_pages.rst If pin_user_pages_unlocked is available, the all Direct I/O requests will use this new API to stay uptodate with the kernel API requirements. Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Brian Atkinson <[email protected]> Closes openzfs#16856
amotin
approved these changes
Dec 16, 2024
ikozhukhov
approved these changes
Dec 16, 2024
Harry-Chen
approved these changes
Dec 18, 2024
Update the CI to include FreeBSD 14.2 as a regularly tested platform. Reviewed-by: Tino Reichardt <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Brian Behlendorf <[email protected]> Closes openzfs#16869
It's a percentage and documented as such, but we were showing it as <size>. Sponsored-by: https://despairlabs.com/sponsor/ Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#16881
In openzfs#16869 we added FreeBSD 13.4 STABLE, but forget the special thing, that the virtio nic within FreeBSD 13.x is buggy. This fix adds the needed rtl8139 nic to the VM. Reviewed-by: George Melikov <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Tino Reichardt <[email protected]> Closes openzfs#16885
I guess we've got some long property names since this was first set up! Sponsored-by: https://despairlabs.com/sponsor/ Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#16883
Setting sharenfs and sharesmb properties on a dataset can become costly if there are large number of snapshots, since setting the share properties iterates over all snapshots present for a dataset. If it is the root dataset for which we are trying to set the share property, snapshots for all child datasets and their children will also be iterated. There is no need to iterate over snapshots for share properties because we do not allow share properties or any other property, to be set on a snapshot itself execpt for user properties. This commit skips iterating over snapshots for share properties, instead iterate over all child dataset and their children for share properties. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Umer Saleem <[email protected]> Closes openzfs#16877
VDEV_PROP_USERPROP is equal do VDEV_PROP_INVAL and so is not a real property. That's why vdev_prop_readonly() does not work right for it. In particular it may declare all vdev user properties readonly on FreeBSD. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Rob Norris <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#16890
The count of chunks in a microzap block is stored as an uint16_t (mze_chunkid). Each chunk is 64 bytes, and the first is used to store a header, so there are 32767 usable chunks, which is just under 2M. 1M is the largest power-2-rounded block size under 2M, so we must set the limit there. If it goes higher, the loop in mzap_addent can overflow and fall into the PANIC case. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#16888
Many RAIDZ/dRAID tests filled files doing millions of 100 or even 10 byte writes. It makes very little sense since we are not micro-benchmarking syscalls or VFS layer here, while before the blocks reach the vdev layer absolute majority of the small writes will be aggregated. In some cases I see we spend almost as much time creating the test files as actually running the tests. And sometimes the tests even time out after that. Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#16905
If a vdev userprop is not found, present it as value '-', default source, so it matches the output from pool userprops. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#16887
FreeBSD recently removed non-standard hex numbers support from awk. Neither it supports -n argument, enabling it in gawk. Instead of depending on those rewrite list_file_blocks() fuction to handle the hex math in shell instead of awk. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#11141
procfs might be not mounted on FreeBSD. Plus checking for specific PID might be not exactly reliable. Check for empty list of jobs instead. Premature loop exit can result in failed test and failed cleanup, failing also some following tests. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#11141
This test takes 3 minutes on RELEASE FreeBSD bots, but on CURRENT, probably due to debugging it has in kernel, it does not complete within 10 minutes, ending up killed. As I see all the redacting here happens within the first ~128MB of the file, so I hope it won't matter if there is 1GB of data instead of 2GB. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Tino Reichardt <[email protected]> Signed-off-by:Alexander Motin <[email protected]> Sponsored by: iXsystems, Inc. Closes openzfs#11141
In Linux, block devices currently lack support for `copy_file_range` API because the kernel does not provide the necessary functionality. However, there is an ongoing upstream effort to address this limitation: https://patchwork.kernel.org/project/dm-devel/cover/[email protected]/. We have adopted this upstream kernel patch into the TrueNAS kernel and made some additional modifications to enable block cloning specifically for the zvol block device. This patch implements the platform- independent portions of these changes for inclusion in OpenZFS. This patch does not introduce any new functionality directly into OpenZFS. The `TX_CLONE_RANGE` replay capability is only relevant when zvols are migrated to non-TrueNAS systems that support Clone Range replay in the ZIL. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Ameer Hamza <[email protected]> Closes openzfs#16901
zfs_arc_shrinker_limit was introduced to avoid ARC collapse due to aggressive kernel reclaim. While useful, the current default (10000) is too prone to OOM especially when MGLRU-enabled kernels with default min_ttl_ms are used. Even when no OOM happens, it often causes too much swap usage. This patch sets zfs_arc_shrinker_limit=0 to not ignore kernel reclaim requests. ARC now plays better with both kernel shrinker and pagecache but, should ARC collapse happen again, MGLRU behavior can be tuned or even disabled. Anyway, zfs should not cause OOM when ARC can be released. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Gionatan Danti <[email protected]> Closes openzfs#16909
behlendorf
force-pushed
the
zfs-2.3.0-rc5-staging
branch
from
7894c03 to
51d8d8c
Compare
zfs_vget doesn't zfs_exit when erroring out due to snapdir being disabled. Signed-off-by: Andrew Walker <[email protected]> Reviewed-by: @bmeagherix Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Ameer Hamza <[email protected]> Reviewed-by: Tony Hutter <[email protected]>
Added centos as optional runners via workflow_dispatch removed centos-stream9 from the FULL_OS runner list as CentOS is not officially support by ZFS. This commit will add preliminary support for EL10 and allow testing ZFS ahead of EL10 codebase solidifying in ~6 months Signed-off-by: James Reilly <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Tino Reichardt <[email protected]>
Before we can remove test files, we need to unmount datasets used by test first. See also: zfs_mount_all_mountpoints.ksh Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Toomas Soome <[email protected]> Closes openzfs#16914
cleanup.ksh is assuming we have TESTDIRS set. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Toomas Soome <[email protected]> Closes openzfs#16915
Originally hex value is used as decimal. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Toomas Soome <[email protected]> Closes openzfs#16917
behlendorf
force-pushed
the
zfs-2.3.0-rc5-staging
branch
from
51d8d8c to
7df539b
Compare
This works around /usr/lib/go-1.18/pkg/tool/linux_amd64/link: mapping output file failed: invalid argument It's happened to me under a Linux jail, but it's also happened to other people, see https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=270247#c4 Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: pstef <[email protected]> Closes openzfs#16918
Remove TESTDIRS as it is not set for pam tests. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Toomas Soome <[email protected]> Closes openzfs#16920
It's possible for a vdev to be flagged for async remove after the pool has suspended. If the removed device has been returned when the pool is resumed, the ASYNC_REMOVE task will still run at the end of txg, and remove the device from the pool again. To fix, we clear the async remove flag at reopen, just as we did for the async fault flag in 5de3ac2. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Rob Norris <[email protected]> Closes openzfs#16921
Instead of using hardwired value for SPA_DISCARD_MEMORY_LIMIT, use save_tunable and restore_tunable to restore the pre-test state. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Toomas Soome <[email protected]> Closes openzfs#16919
behlendorf
force-pushed
the
zfs-2.3.0-rc5-staging
branch
from
7df539b to
9fd6d42
Compare
This updates the Makefile to be more correct for parallel make. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Robert Evans <[email protected]> Closes openzfs#16030 Closes openzfs#16922
Similar to what we saw in openzfs#16569, we need to consider that a replacing vdev should not be considered as fully contributing to the redundancy of a raidz vdev even though current IO has enough redundancy. When a failed vdev_probe() is faulting a disk, it now checks if that disk is required, and if so it suspends the pool until the admin can return the missing disks. Sponsored-by: Klara, Inc. Sponsored-by: Wasabi Technology, Inc. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Reviewed-by: Allan Jude <[email protected]> Reviewed-by: Tony Hutter <[email protected]> Signed-off-by: Don Brady <[email protected]> Closes openzfs#16864
openzfs#15793 wanted to make zfs_strerror threadsafe, unfortunately, it turned out that strerror_l() usage was wrong, and also, some libc implementations dont have strerror_l(). zfs_strerror() now simply calls original strerror() and copies the result to a thread-local buffer, then returns that. Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: Alexander Motin <[email protected]> Signed-off-by: Richard Kojedzinszky <[email protected]> Closes openzfs#15793 Closes openzfs#16640 Closes openzfs#16923
behlendorf
force-pushed
the
zfs-2.3.0-rc5-staging
branch
from
9fd6d42 to
e9bf9a9
Compare
Reviewed-by: Brian Behlendorf <[email protected]> Reviewed-by: George Melikov <[email protected]> Signed-off-by: Robert Evans <[email protected]> Closes openzfs#16926
In order to correctly cross-compile, one has to pass ARCH and CROSS_COMPILE make flags to kernel module build calls. Facilitate this in the same way as for custom CC flag by recognizing KERNEL_-prefixed configure environment variables of same name. Reviewed-by: Brian Behlendorf <[email protected]> Signed-off-by: Phil Sutter <[email protected]> Closes openzfs#16924
Signed-off-by: Brian Behlendorf <[email protected]>
behlendorf
force-pushed
the
zfs-2.3.0-rc5-staging
branch
from
e9bf9a9 to
0c88ae6
Compare
|
Tested 2.3.0-rc5 on Buildroot with GLIBC, MUSL, and UCLIBC. It works perfectly! 💪 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation and Context
Initial proposed patchset for zfs-2.3.0-rc5.
Description
Bug fixes, build fixes, ZTS updates.
How Has This Been Tested?
Clean backports from master. Will be retested by the CI.
Types of changes