Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Jorgen Lundman <[email protected]> Set zp->z_blksz to ashift Slowdowns in zfs_log_write() can happen on large blocksize devices. Signed-off-by: Jorgen Lundman <[email protected]> MacOS: Build with n+1 cpus (#22) Signed-off-by: Andrew Innes <[email protected]> macOS: use portable paths for brewed deps (#23) Signed-off-by: Yurii Kolesnykov <[email protected]> Search for C++ and Obj C compilers (#21) Signed-off-by: Andrew Innes <[email protected]> cpuid uses a,b,c,d order. wikipedia paragraph referred to model string, which has a special case ordering. This fixes a bug where old intel would incorretly assume AES/qmul feature was available when it was not. (And vice-verse, but this would just skip asm versions and not panic) Signed-off-by: Jorgen Lundman <[email protected]> Redo cpuid feature detection We already have a proper cpuid feature API used in userland, but Linux took it out for KERNEL use, to plug into the Linux API. We do not have a kernel API to use, so we might as well use the userland one. This makes it nice and consistent, and more easy to read. Signed-off-by: Jorgen Lundman <[email protected]> Use built in cpuid() where possible Fixing a panic when running sysctl -a Signed-off-by: Jorgen Lundman <[email protected]> UpstreamÂ: remove sprintf usage Signed-off-by: Jorgen Lundman <[email protected]> Add IRC notifications Signed-off-by: Jorgen Lundman <[email protected]> xcode 16 compile fixes function argument type changed, adapt ASSERTs assembly bug in clang-16 work around Signed-off-by: Jorgen Lundman <[email protected]> Reduce userland CPU starvation from low-priority I/Os Once we have successfully read from disk we may do significant CPU-work on the data obtained, depending on features like record sizes, checksums, encryption, or compression. A successful write may cause significant CPU-work to be done for a subsequent zio. Because our vdev_disk layer and its use of IOKit is fundamentally asynchronous, on some media modern linearized scrubs and resilvers may "gang up" on bursts of interactive user I/Os. Moreover, all zfs kernel threads are higher priority than the vast majority of userland threads, therefore the latter can be starved of CPU especially for a scrubbing pool which has a vdev count conmparable to the CPU core count and where data was wrtten using expensive checksums like sha256. Practically all our IOKit I/Os are asynchronous, but significant work may be done on the taskq thread, possibly right to the entry into the vdev_disk_io_intr() callback function. We dispatch "background" I/Os into a lower thread-priority and lower thread-count taskq compared to other types of zio. In the callback function itself, for these low-priority I/Os we kpreempt() before before calling zio_delay_interrupt(). For writes, this may impose a system-load-dependent delay on notifying upper layers of zfs that IOKit has moved the buffer towards the physical device, generating backpressure on subsequent writes. For reads, this kpreempt() gives another thread in the system a chance to run before we do potentially heavy-CPU actions (such as checksumming or decyrption) on the data IOKit has obtained from the storage device. Signed-off-by: Jorgen Lundman <[email protected]> iconfig fixes Signed-off-by: Jorgen Lundman <[email protected]> abd_os changes Signed-off-by: Jorgen Lundman <[email protected]> New ASSERTs Signed-off-by: Jorgen Lundman <[email protected]> new KMEM flag Signed-off-by: Jorgen Lundman <[email protected]> issig() now takes no args Signed-off-by: Jorgen Lundman <[email protected]> sprintf is deprecated Signed-off-by: Jorgen Lundman <[email protected]> Add UIO_DIRECT Signed-off-by: Jorgen Lundman <[email protected]> Re-arrange z_blksz in zfs_znode_alloc() Signed-off-by: Jorgen Lundman <[email protected]> Set zp->z_blksz to ashift Slowdowns in zfs_log_write() can happen on large blocksize devices. Signed-off-by: Jorgen Lundman <[email protected]> altool is deprecated, use notarytool Signed-off-by: Jorgen Lundman <[email protected]> pkg-macos OS friendly name after macOS name change Signed-off-by: Jorgen Lundman <[email protected]> Update the pkg installer scripts Signed-off-by: Jorgen Lundman <[email protected]> zfs-tests: support newer macOS readOnly root Since BigSur days, "/" on macOS is a readOnly, sealed archive which means zfs-tests can not mount the expected /testpool (which all scripts assume are at root). We can work around this by leveraging /etc/synthetic.conf to create "virtual directories". They can be mounted on, but not modified. This way *most* pool names are accounted for in zfs-tests (more will come). We also have to make log_must(rm /TESTPOOL) be optional. Signed-off-by: Jorgen Lundman <[email protected]> Revert "Re-arrange z_blksz in zfs_znode_alloc()" This reverts commit 106e765. See next commit Signed-off-by: Jorgen Lundman <[email protected]> Change zfs_log_write() blocksize for performance "len" can be set to zero in the WR_INDIRECT case, if zp->z_blksz is 0. This leads to endless loop. Signed-off-by: Jorgen Lundman <[email protected]> Revert changes to zfs_znode_alloc for reals. Signed-off-by: Jorgen Lundman <[email protected]> Make sure blocksize is not 0 in zfs_log_write() Or the while(resid) loop will run forever. Signed-off-by: Jorgen Lundman <[email protected]> Negative numcpus make taskq stall Handle limited CPU VMs with 1 or 2 cores. The cpus variable would go to -1 and taskq would stall forever. Signed-off-by: Jorgen Lundman <[email protected]> Handle versions without IOSleepWithLeeway() Signed-off-by: Jorgen Lundman <[email protected]> Allow zfs_fallthrough to be defined Signed-off-by: Jorgen Lundman <[email protected]> Don't use typedefs before OSVersion workaround Signed-off-by: Jorgen Lundman <[email protected]> Handle SF_NOUNLINK on versions without it Signed-off-by: Jorgen Lundman <[email protected]> Update pkg_macos.sh to handle notarytool Signed-off-by: Jorgen Lundman <[email protected]> Compile fixes after rebase Signed-off-by: Jorgen Lundman <[email protected]> Export before removing file Due to a macOS quirk, we have to close the files in a filebased pool during some operations, which means the export -f task will fail to open the vdevs again, the pool will be suspended and zpool_export_004_pos will hang forever. By deleting the file after export, everything works as expected. Signed-off-by: Jorgen Lundman <[email protected]> Don't hold zfs_enter() in asyncput This can deadlock during unmount, as we are already holding WRITE lock. Signed-off-by: Jorgen Lundman <[email protected]> zpool freeze can hang in spa_evicting_os_wait It appears upstream can still unmount, and export the pool - to later re-import to unfreeze. However on macOS, export will hang in spa_evicting_os_wait(). It is unclear why we hang, but it could be we go through more syncs during unmount, and that spa_freeze_txg is set to txg + TXG_SIZE Now we clear (restoring UINT64_MAX) spa_freeze_txg in the unmount() call, ensuring unmount and export works. We might not have identical "zpool freeze" as upstream, but as the zpool sources say: * 'freeze' is a vile debugging abomination, so we treat * it as such. Signed-off-by: Jorgen Lundman <[email protected]> Do not install .in files with Makefile Signed-off-by: Jorgen Lundman <[email protected]> clone file can return ENOTSUP for fallback From Sonoma onwards, file_cmds-428 will handle ENOTSUP and fallback to copyfile. Prior to that, we favour EAGAIN as it will indicate to the users that it is a temporary failure. Signed-off-by: Jorgen Lundman <[email protected]> sprintf has been deprecated Signed-off-by: Jorgen Lundman <[email protected]> Move appveyor to .github/workflows/macos-build.yml Signed-off-by: Jorgen Lundman <[email protected]> make install would fail to mkdir first. Signed-off-by: Jorgen Lundman <[email protected]> Fix .yml to start zed, and zfs-tests Signed-off-by: Jorgen Lundman <[email protected]> fix abd, have taskq_wait_synced() wait for threads to be created taskq_wait_synced() did a VERIFY() on whether the taskq's threads were the requested number, but taskq_create() can ultimately return early because taskq_thread_create() is allowed to return when two desired threads are created. fix this race panic. also, taskq_wait_synced() may fail if if num_ecores is nonzero (on Apple Silicon), so create a flag that lets taskq_create_common() deal with the max_ncpus. Make boot_ncpus a variable that's MAX(1, (int)max_ncores - num_ecores). boot_ncpus is used in common code. Modify the alignments and quanta/import sizes of the abd kmem and vmem cache creations. Make DEBUG builds work with KMF_LITE | KMF_BUFCTL on the abd kmem caches. Signed-off-by: Sean Doran <[email protected]> Allow NOTARYTOOL path to be set Apple has decided that copying notarytool from new macOS to old macOS instead of supporting altool. But sadly "xcrun" will not run it as expected, so we allow NOTARYTOOL env var to be set to a path to use. Signed-off-by: Jorgen Lundman <[email protected]> Add copy_file_range() wrapper to fcopyfile(). Signed-off-by: Jorgen Lundman <[email protected]> Compile fixes since last rebase Signed-off-by: Jorgen Lundman <[email protected]> ZIO_TYPE_IOCTL renamed to ZIO_TYPE_FLUSH And also they renamed zfs_file_fallocate() to zfs_file_deallocate() somewhere along the way. Signed-off-by: Jorgen Lundman <[email protected]> Header fix Signed-off-by: Jorgen Lundman <[email protected]> zfs_racct prototype change Signed-off-by: Jorgen Lundman <[email protected]> zfs_log_write() now takes directio boolean Signed-off-by: Jorgen Lundman <[email protected]> zfs_znode_os.c also, change strcpy back to strlcpy yet again, thanks linux. Signed-off-by: Jorgen Lundman <[email protected]> isimd_stat_init simd_stat_fini Signed-off-by: Jorgen Lundman <[email protected]> Signed-off-by: Jorgen Lundman <[email protected]> Use SUBSTFILES for .in files Signed-off-by: Jorgen Lundman <[email protected]> Compile fixes for O_DIRECTIO Signed-off-by: Jorgen Lundman <[email protected]>
- Loading branch information
