From 5ac53c0221dd5e557e00887b47cc522e4b1c398d Mon Sep 17 00:00:00 2001 From: Jorgen Lundman Date: Mon, 23 Oct 2023 09:13:43 +0900 Subject: [PATCH] 6: etc/* up to 2.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Signed-off-by: Jorgen Lundman Set zp->z_blksz to ashift Slowdowns in zfs_log_write() can happen on large blocksize devices. Signed-off-by: Jorgen Lundman MacOS: Build with n+1 cpus (#22) Signed-off-by: Andrew Innes macOS: use portable paths for brewed deps (#23) Signed-off-by: Yurii Kolesnykov Search for C++ and Obj C compilers (#21) Signed-off-by: Andrew Innes cpuid uses a,b,c,d order. wikipedia paragraph referred to model string, which has a special case ordering. This fixes a bug where old intel would incorretly assume AES/qmul feature was available when it was not. (And vice-verse, but this would just skip asm versions and not panic) Signed-off-by: Jorgen Lundman Redo cpuid feature detection We already have a proper cpuid feature API used in userland, but Linux took it out for KERNEL use, to plug into the Linux API. We do not have a kernel API to use, so we might as well use the userland one. This makes it nice and consistent, and more easy to read. Signed-off-by: Jorgen Lundman Use built in cpuid() where possible Fixing a panic when running sysctl -a Signed-off-by: Jorgen Lundman UpstreamÂ: remove sprintf usage Signed-off-by: Jorgen Lundman Add IRC notifications Signed-off-by: Jorgen Lundman xcode 16 compile fixes function argument type changed, adapt ASSERTs assembly bug in clang-16 work around Signed-off-by: Jorgen Lundman Reduce userland CPU starvation from low-priority I/Os Once we have successfully read from disk we may do significant CPU-work on the data obtained, depending on features like record sizes, checksums, encryption, or compression. A successful write may cause significant CPU-work to be done for a subsequent zio. Because our vdev_disk layer and its use of IOKit is fundamentally asynchronous, on some media modern linearized scrubs and resilvers may "gang up" on bursts of interactive user I/Os. Moreover, all zfs kernel threads are higher priority than the vast majority of userland threads, therefore the latter can be starved of CPU especially for a scrubbing pool which has a vdev count conmparable to the CPU core count and where data was wrtten using expensive checksums like sha256. Practically all our IOKit I/Os are asynchronous, but significant work may be done on the taskq thread, possibly right to the entry into the vdev_disk_io_intr() callback function. We dispatch "background" I/Os into a lower thread-priority and lower thread-count taskq compared to other types of zio. In the callback function itself, for these low-priority I/Os we kpreempt() before before calling zio_delay_interrupt(). For writes, this may impose a system-load-dependent delay on notifying upper layers of zfs that IOKit has moved the buffer towards the physical device, generating backpressure on subsequent writes. For reads, this kpreempt() gives another thread in the system a chance to run before we do potentially heavy-CPU actions (such as checksumming or decyrption) on the data IOKit has obtained from the storage device. Signed-off-by: Jorgen Lundman iconfig fixes Signed-off-by: Jorgen Lundman abd_os changes Signed-off-by: Jorgen Lundman New ASSERTs Signed-off-by: Jorgen Lundman new KMEM flag Signed-off-by: Jorgen Lundman issig() now takes no args Signed-off-by: Jorgen Lundman sprintf is deprecated Signed-off-by: Jorgen Lundman Add UIO_DIRECT Signed-off-by: Jorgen Lundman Re-arrange z_blksz in zfs_znode_alloc() Signed-off-by: Jorgen Lundman Set zp->z_blksz to ashift Slowdowns in zfs_log_write() can happen on large blocksize devices. Signed-off-by: Jorgen Lundman altool is deprecated, use notarytool Signed-off-by: Jorgen Lundman pkg-macos OS friendly name after macOS name change Signed-off-by: Jorgen Lundman Update the pkg installer scripts Signed-off-by: Jorgen Lundman zfs-tests: support newer macOS readOnly root Since BigSur days, "/" on macOS is a readOnly, sealed archive which means zfs-tests can not mount the expected /testpool (which all scripts assume are at root). We can work around this by leveraging /etc/synthetic.conf to create "virtual directories". They can be mounted on, but not modified. This way *most* pool names are accounted for in zfs-tests (more will come). We also have to make log_must(rm /TESTPOOL) be optional. Signed-off-by: Jorgen Lundman Revert "Re-arrange z_blksz in zfs_znode_alloc()" This reverts commit 106e7659a63543b4cae47d78fafb9a49a7257bcd. See next commit Signed-off-by: Jorgen Lundman Change zfs_log_write() blocksize for performance "len" can be set to zero in the WR_INDIRECT case, if zp->z_blksz is 0. This leads to endless loop. Signed-off-by: Jorgen Lundman Revert changes to zfs_znode_alloc for reals. Signed-off-by: Jorgen Lundman Make sure blocksize is not 0 in zfs_log_write() Or the while(resid) loop will run forever. Signed-off-by: Jorgen Lundman Negative numcpus make taskq stall Handle limited CPU VMs with 1 or 2 cores. The cpus variable would go to -1 and taskq would stall forever. Signed-off-by: Jorgen Lundman Handle versions without IOSleepWithLeeway() Signed-off-by: Jorgen Lundman Allow zfs_fallthrough to be defined Signed-off-by: Jorgen Lundman Don't use typedefs before OSVersion workaround Signed-off-by: Jorgen Lundman Handle SF_NOUNLINK on versions without it Signed-off-by: Jorgen Lundman Update pkg_macos.sh to handle notarytool Signed-off-by: Jorgen Lundman Compile fixes after rebase Signed-off-by: Jorgen Lundman Export before removing file Due to a macOS quirk, we have to close the files in a filebased pool during some operations, which means the export -f task will fail to open the vdevs again, the pool will be suspended and zpool_export_004_pos will hang forever. By deleting the file after export, everything works as expected. Signed-off-by: Jorgen Lundman Don't hold zfs_enter() in asyncput This can deadlock during unmount, as we are already holding WRITE lock. Signed-off-by: Jorgen Lundman zpool freeze can hang in spa_evicting_os_wait It appears upstream can still unmount, and export the pool - to later re-import to unfreeze. However on macOS, export will hang in spa_evicting_os_wait(). It is unclear why we hang, but it could be we go through more syncs during unmount, and that spa_freeze_txg is set to txg + TXG_SIZE Now we clear (restoring UINT64_MAX) spa_freeze_txg in the unmount() call, ensuring unmount and export works. We might not have identical "zpool freeze" as upstream, but as the zpool sources say: * 'freeze' is a vile debugging abomination, so we treat * it as such. Signed-off-by: Jorgen Lundman Do not install .in files with Makefile Signed-off-by: Jorgen Lundman clone file can return ENOTSUP for fallback From Sonoma onwards, file_cmds-428 will handle ENOTSUP and fallback to copyfile. Prior to that, we favour EAGAIN as it will indicate to the users that it is a temporary failure. Signed-off-by: Jorgen Lundman sprintf has been deprecated Signed-off-by: Jorgen Lundman Move appveyor to .github/workflows/macos-build.yml Signed-off-by: Jorgen Lundman make install would fail to mkdir first. Signed-off-by: Jorgen Lundman Fix .yml to start zed, and zfs-tests Signed-off-by: Jorgen Lundman fix abd, have taskq_wait_synced() wait for threads to be created taskq_wait_synced() did a VERIFY() on whether the taskq's threads were the requested number, but taskq_create() can ultimately return early because taskq_thread_create() is allowed to return when two desired threads are created. fix this race panic. also, taskq_wait_synced() may fail if if num_ecores is nonzero (on Apple Silicon), so create a flag that lets taskq_create_common() deal with the max_ncpus. Make boot_ncpus a variable that's MAX(1, (int)max_ncores - num_ecores). boot_ncpus is used in common code. Modify the alignments and quanta/import sizes of the abd kmem and vmem cache creations. Make DEBUG builds work with KMF_LITE | KMF_BUFCTL on the abd kmem caches. Signed-off-by: Sean Doran Allow NOTARYTOOL path to be set Apple has decided that copying notarytool from new macOS to old macOS instead of supporting altool. But sadly "xcrun" will not run it as expected, so we allow NOTARYTOOL env var to be set to a path to use. Signed-off-by: Jorgen Lundman Add copy_file_range() wrapper to fcopyfile(). Signed-off-by: Jorgen Lundman Compile fixes since last rebase Signed-off-by: Jorgen Lundman ZIO_TYPE_IOCTL renamed to ZIO_TYPE_FLUSH And also they renamed zfs_file_fallocate() to zfs_file_deallocate() somewhere along the way. Signed-off-by: Jorgen Lundman Header fix Signed-off-by: Jorgen Lundman zfs_racct prototype change Signed-off-by: Jorgen Lundman zfs_log_write() now takes directio boolean Signed-off-by: Jorgen Lundman zfs_znode_os.c also, change strcpy back to strlcpy yet again, thanks linux. Signed-off-by: Jorgen Lundman isimd_stat_init simd_stat_fini Signed-off-by: Jorgen Lundman Signed-off-by: Jorgen Lundman Use SUBSTFILES for .in files Signed-off-by: Jorgen Lundman Compile fixes for O_DIRECTIO Signed-off-by: Jorgen Lundman --- .github/workflows/macos-build.yaml | 4 +- .github/workflows/macos-build.yml | 89 +++ .github/workflows/notify-irc.yml | 36 + cmd/zpool/os/macos/zpool_vdev_os.c | 21 + config/zfs-build.m4 | 37 +- configure.ac | 6 +- .../pkg-scripts/postinstall_actions/kextspost | 16 +- .../postinstall_actions/loadplists | 12 +- .../pkg-scripts/postinstall_actions/startzed | 11 +- etc/launchd.d/.gitignore | 1 + etc/launchd.d/Makefile.am | 23 + etc/launchd.d/zpool-import-all.sh.in | 60 ++ etc/launchd/Makefile.am | 4 + etc/launchd/daemons/.gitignore | 5 + etc/launchd/daemons/Makefile.am | 25 + .../org.openzfsonosx.InvariantDisks.plist.in | 16 + .../org.openzfsonosx.zconfigd.plist.in | 20 + .../daemons/org.openzfsonosx.zed.plist.in | 21 + ...org.openzfsonosx.zpool-import-all.plist.in | 18 + .../org.openzfsonosx.zpool-import.plist.in | 24 + etc/paths.d/.gitignore | 1 + etc/paths.d/Makefile.am | 21 + etc/paths.d/zfs-path.in | 2 + include/os/macos/spl/libkern/libkern.h | 6 + include/os/macos/spl/sys/atomic.h | 2 +- include/os/macos/spl/sys/byteorder.h | 1 + include/os/macos/spl/sys/cred.h | 8 +- include/os/macos/spl/sys/debug.h | 188 ++++- include/os/macos/spl/sys/kmem.h | 1 + include/os/macos/spl/sys/mod_os.h | 9 + include/os/macos/spl/sys/param.h | 2 + include/os/macos/spl/sys/proc.h | 2 +- include/os/macos/spl/sys/processor.h | 42 +- include/os/macos/spl/sys/signal.h | 15 +- include/os/macos/spl/sys/simd.h | 5 + include/os/macos/spl/sys/simd_x86.h | 267 +------ include/os/macos/spl/sys/sysmacros.h | 2 +- include/os/macos/spl/sys/taskq.h | 3 + include/os/macos/spl/sys/thread.h | 56 +- include/os/macos/spl/sys/tsd.h | 3 +- include/os/macos/spl/sys/types.h | 42 +- include/os/macos/spl/sys/uio.h | 29 + include/os/macos/spl/sys/vnode.h | 1 - include/os/macos/zfs/sys/abd_impl_os.h | 38 + include/os/macos/zfs/sys/abd_os.h | 57 ++ include/os/macos/zfs/sys/ldi_osx.h | 1 + include/os/macos/zfs/sys/zfs_vfsops_os.h | 3 - lib/libspl/asm-x86_64/atomic.S | 693 ++++++++++++++++++ lib/libspl/include/os/macos/string.h | 2 + .../include/os/macos/sys/kernel_types.h | 2 + lib/libspl/include/os/macos/unistd.h | 39 + lib/libzfs/Makefile.am | 5 +- lib/libzutil/os/macos/zutil_import_os.c | 16 + .../icp/asm-x86_64/modes/aesni-gcm-x86_64.S | 7 + module/icp/asm-x86_64/sha2/sha256-x86_64.S | 10 + module/icp/asm-x86_64/sha2/sha512-x86_64.S | 9 + module/os/macos/Makefile.am | 12 +- module/os/macos/spl/spl-err.c | 4 + module/os/macos/spl/spl-osx.c | 12 +- module/os/macos/spl/spl-proc.c | 8 + module/os/macos/spl/spl-processor.c | 95 +-- module/os/macos/spl/spl-seg_kmem.c | 32 +- module/os/macos/spl/spl-taskq.c | 100 ++- module/os/macos/spl/spl-uio.c | 36 + module/os/macos/zfs/abd_os.c | 549 ++++++-------- module/os/macos/zfs/arc_os.c | 4 +- module/os/macos/zfs/vdev_disk.c | 109 +-- module/os/macos/zfs/vdev_file.c | 39 +- module/os/macos/zfs/vdev_label_os.c | 45 ++ module/os/macos/zfs/zfs_boot.cpp | 1 + module/os/macos/zfs/zfs_file_os.c | 2 +- module/os/macos/zfs/zfs_racct.c | 6 +- module/os/macos/zfs/zfs_vfsops.c | 94 +-- module/os/macos/zfs/zfs_vnops_os.c | 7 +- module/os/macos/zfs/zfs_vnops_osx.c | 24 +- module/os/macos/zfs/zfs_vnops_osx_lib.c | 8 +- .../macos/zfs/{zfs_znode.c => zfs_znode_os.c} | 279 +------ module/os/macos/zfs/zvol_os.c | 2 +- module/zfs/spa.c | 12 +- module/zfs/zfs_log.c | 7 + module/zfs/zfs_vnops.c | 3 +- module/zfs/zfs_znode.c | 3 +- scripts/pkg_macos.sh | 78 +- scripts/zfs-tests.sh | 23 + tests/zfs-tests/include/libtest.shlib | 12 +- .../cli_root/zpool_export/zpool_export.kshlib | 2 +- .../replacement/replace_rebuild.ksh | 2 +- .../replacement/replace_resilver.ksh | 2 +- 88 files changed, 2374 insertions(+), 1277 deletions(-) create mode 100755 .github/workflows/macos-build.yml create mode 100644 .github/workflows/notify-irc.yml create mode 100644 etc/launchd.d/.gitignore create mode 100644 etc/launchd.d/Makefile.am create mode 100755 etc/launchd.d/zpool-import-all.sh.in create mode 100644 etc/launchd/Makefile.am create mode 100644 etc/launchd/daemons/.gitignore create mode 100644 etc/launchd/daemons/Makefile.am create mode 100644 etc/launchd/daemons/org.openzfsonosx.InvariantDisks.plist.in create mode 100644 etc/launchd/daemons/org.openzfsonosx.zconfigd.plist.in create mode 100644 etc/launchd/daemons/org.openzfsonosx.zed.plist.in create mode 100644 etc/launchd/daemons/org.openzfsonosx.zpool-import-all.plist.in create mode 100644 etc/launchd/daemons/org.openzfsonosx.zpool-import.plist.in create mode 100644 etc/paths.d/.gitignore create mode 100644 etc/paths.d/Makefile.am create mode 100644 etc/paths.d/zfs-path.in create mode 100644 include/os/macos/zfs/sys/abd_impl_os.h create mode 100644 include/os/macos/zfs/sys/abd_os.h create mode 100644 lib/libspl/asm-x86_64/atomic.S create mode 100644 module/os/macos/zfs/vdev_label_os.c rename module/os/macos/zfs/{zfs_znode.c => zfs_znode_os.c} (89%) diff --git a/.github/workflows/macos-build.yaml b/.github/workflows/macos-build.yaml index de387c9adbfd..a02d698aa5df 100644 --- a/.github/workflows/macos-build.yaml +++ b/.github/workflows/macos-build.yaml @@ -28,10 +28,10 @@ jobs: - name: configure run: | #https://stackoverflow.com/a/62591864 - ./configure CPPFLAGS="-I/usr/local/opt/gettext/include -I/usr/local/opt/openssl/include" LDFLAGS="-L/usr/local/opt/gettext/lib/ -L/usr/local/opt/openssl/lib" + ./configure CPPFLAGS="-I${HOMEBREW_PREFIX}/opt/gettext/include -I${HOMEBREW_PREFIX}/opt/openssl/include" LDFLAGS="-L${HOMEBREW_PREFIX}/opt/gettext/lib/ -L${HOMEBREW_PREFIX}/opt/openssl/lib" - name: build run: | - make -j 2 + make -j $(($(sysctl -n hw.ncpu)+1)) #- name: install # run: | # sudo make install DESTDIR=/// diff --git a/.github/workflows/macos-build.yml b/.github/workflows/macos-build.yml new file mode 100755 index 000000000000..8d67739de63a --- /dev/null +++ b/.github/workflows/macos-build.yml @@ -0,0 +1,89 @@ +version: 1.0.{build} +branches: + only: + - /macOS_.*/ +image: macOS-Catalina +build_script: +- sh: >- + curl -sflL 'https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-ssh.sh' | bash -e - + + curl -sflL 'https://raw.githubusercontent.com/appveyor/ci/master/scripts/enable-vnc.sh' | bash -e - + + + HOMEBREW_NO_AUTO_UPDATE=1 + + export HOMEBREW_NO_AUTO_UPDATE + + HOMEBREW_NO_INSTALL_CLEANUP=1 + + export HOMEBREW_NO_INSTALL_CLEANUP + + brew install gsed + + echo "Running from .github/workflows/macos-build.yml" + + sh autogen.sh + + + ./configure CPPFLAGS="-I/usr/local/opt/gettext/include -I/usr/local/opt/openssl@1.1/include" LDFLAGS="-L/usr/local/opt/gettext/lib/ -L/usr/local/opt/openssl@1.1/lib" CFLAGS="-Wno-error -g -O2" --sysconfdir=/etc --localstatedir=/var --prefix=/usr/local/zfs/ --sbindir=/usr/local/zfs/bin -libexecdir=/usr/local/zfs/libexec --enable-debug SED=gsed + + + echo "Checking codestyle" + + make cstyle || exit 1 + + + + make -j2 # V=1 + + + + sudo make -j2 V=1 install +test_script: +- sh: >- + # log stream --source --predicate 'sender == "zfs"' --style compact & + + + echo "Loading kernel modules..." + + sudo scripts/load_macos.sh || exit 1 + + sleep 5 + + echo "Running zfs-tests..." + + export SHELL=/bin/ksh + + + sudo /usr/local/zfs/bin/zed -fF & + + sudo mkdir -p /etc/synthetic.d/ + + sudo touch /etc/synthetic.d/zfs-tests + + scripts/zfs-tests.sh -r macOS-CI || echo "Tests not quite right" + + + sysctl kern.stack_depth_max + + + sysctl kstat > kstat.txt + + + 7z a kext.zip /tmp/zfs.kext/ + + 7z a zfs-tests.zip /var/tmp/test_results/current/ kstat.txt + + ls -la + + + echo "Attempting to unload" + + sudo /usr/local/zfs/bin/zpool export -fa + + sudo kextunload -b org.openzfsonosx.zfs || echo "Unload failed". +artifacts: +- path: kext.zip + name: kext +- path: zfs-tests.zip + name: zfstest-log diff --git a/.github/workflows/notify-irc.yml b/.github/workflows/notify-irc.yml new file mode 100644 index 000000000000..f6534257f85e --- /dev/null +++ b/.github/workflows/notify-irc.yml @@ -0,0 +1,36 @@ + +name: Notify IRC + +on: + push: + issues: + types: + - opened + - edited + - closed + issue_comment: + pull_request: + discussion: + types: + - created + - edited + - closed + - answered + discussion_comment: + create: + delete: + +jobs: + notify-irc: + runs-on: ubuntu-latest + + steps: + - name: Notify IRC + uses: openzfsonwindows/notify-irc@v2 + with: + channel: "#openzfs-macos" + server: "irc.libera.chat" + nickname: "zfs_consus" + mirccolor: "true" + eventpath: ${{ github.event_path }} + diff --git a/cmd/zpool/os/macos/zpool_vdev_os.c b/cmd/zpool/os/macos/zpool_vdev_os.c index 01fc90efda82..2bd828194925 100644 --- a/cmd/zpool/os/macos/zpool_vdev_os.c +++ b/cmd/zpool/os/macos/zpool_vdev_os.c @@ -199,3 +199,24 @@ after_zpool_upgrade(zpool_handle_t *zhp) { (void) zhp; } + +int +zpool_power_current_state(zpool_handle_t *zhp, char *vdev) +{ + + (void) zhp; + (void) vdev; + /* Enclosure slot power not supported on macOS yet */ + return (-1); +} + +int +zpool_power(zpool_handle_t *zhp, char *vdev, boolean_t turn_on) +{ + + (void) zhp; + (void) vdev; + (void) turn_on; + /* Enclosure slot power not supported on macOS yet */ + return (ENOTSUP); +} diff --git a/config/zfs-build.m4 b/config/zfs-build.m4 index 7dacf470917e..a615013f3268 100644 --- a/config/zfs-build.m4 +++ b/config/zfs-build.m4 @@ -557,33 +557,17 @@ AC_DEFUN([ZFS_AC_DEFAULT_PACKAGE], [ AC_MSG_CHECKING([default package type]) case "$VENDOR" in -<<<<<<< HEAD alpine|arch|artix|gentoo|lunar|slackware) DEFAULT_PACKAGE=tgz ;; debian|ubuntu) DEFAULT_PACKAGE=deb ;; freebsd) DEFAULT_PACKAGE=pkg ;; + apple) + DEFAULT_PACKAGE=pkg ;; *) # fedora|openeuler|redhat|sles|toss DEFAULT_PACKAGE=rpm ;; -======= - toss) DEFAULT_PACKAGE=rpm ;; - redhat) DEFAULT_PACKAGE=rpm ;; - fedora) DEFAULT_PACKAGE=rpm ;; - gentoo) DEFAULT_PACKAGE=tgz ;; - alpine) DEFAULT_PACKAGE=tgz ;; - arch) DEFAULT_PACKAGE=tgz ;; - sles) DEFAULT_PACKAGE=rpm ;; - slackware) DEFAULT_PACKAGE=tgz ;; - lunar) DEFAULT_PACKAGE=tgz ;; - ubuntu) DEFAULT_PACKAGE=deb ;; - debian) DEFAULT_PACKAGE=deb ;; - freebsd) DEFAULT_PACKAGE=pkg ;; - openeuler) DEFAULT_PACKAGE=rpm ;; - apple) DEFAULT_PACKAGE=pkg ;; - *) DEFAULT_PACKAGE=rpm ;; ->>>>>>> 5b2f117f5... 4: config/* scripts/* contrib/* esac AC_MSG_RESULT([$DEFAULT_PACKAGE]) AC_SUBST(DEFAULT_PACKAGE) @@ -620,7 +604,6 @@ AC_DEFUN([ZFS_AC_DEFAULT_PACKAGE], [ AC_MSG_CHECKING([default init config directory]) case "$VENDOR" in -<<<<<<< HEAD alpine|artix|gentoo) initconfdir=/etc/conf.d ;; @@ -630,24 +613,12 @@ AC_DEFUN([ZFS_AC_DEFAULT_PACKAGE], [ freebsd) initconfdir=$sysconfdir/rc.conf.d ;; + apple) + initconfdir=${prefix}/etc/launchd/launchd.d/ ;; *) # debian|ubuntu initconfdir=/etc/default ;; -======= - alpine) initconfdir=/etc/conf.d ;; - gentoo) initconfdir=/etc/conf.d ;; - toss) initconfdir=/etc/sysconfig ;; - redhat) initconfdir=/etc/sysconfig ;; - fedora) initconfdir=/etc/sysconfig ;; - sles) initconfdir=/etc/sysconfig ;; - openeuler) initconfdir=/etc/sysconfig ;; - ubuntu) initconfdir=/etc/default ;; - debian) initconfdir=/etc/default ;; - freebsd) initconfdir=$sysconfdir/rc.conf.d;; - apple) initconfdir=${prefix}/etc/launchd/launchd.d/ ;; - *) initconfdir=/etc/default ;; ->>>>>>> 5b2f117f5... 4: config/* scripts/* contrib/* esac AC_MSG_RESULT([$initconfdir]) AC_SUBST(initconfdir) diff --git a/configure.ac b/configure.ac index c2158282dfac..5bc2301aacaf 100644 --- a/configure.ac +++ b/configure.ac @@ -55,9 +55,9 @@ AM_PROG_AS AM_PROG_CC_C_O AX_CODE_COVERAGE _AM_PROG_TAR(pax) -AC_PROG_CXX(clang++) -AC_PROG_OBJC(clang) -AC_PROG_OBJCXX(clang++) +AC_PROG_CXX +AC_PROG_OBJC +AC_PROG_OBJCXX ZFS_AC_LICENSE ZFS_AC_CONFIG diff --git a/contrib/macOS/pkg-scripts/postinstall_actions/kextspost b/contrib/macOS/pkg-scripts/postinstall_actions/kextspost index 20c633252be7..84cfa5a83c4b 100755 --- a/contrib/macOS/pkg-scripts/postinstall_actions/kextspost +++ b/contrib/macOS/pkg-scripts/postinstall_actions/kextspost @@ -11,6 +11,8 @@ else fi fi +# kmutil onwards, kextstat prints it Executes kmutil, and no variant +# but that is to stderr, so does not get seen by "wc". if [ $(/usr/sbin/kextstat -b org.openzfsonosx.zfs | wc -l) -gt 1 ] ; then printf "\nUnloading zfs.kext...\n" /sbin/kextunload -b org.openzfsonosx.zfs @@ -31,14 +33,12 @@ printf "\nTouching %s\n" "${kernelextensionsdir}" /usr/bin/killall -HUP kextd # Load kext -printf "\nLoading spl.kext...\n" -/usr/bin/kextutil "${kernelextensionsdir}/zfs.kext" 2>&1 | grep -q "rejected due to system policy" +printf "\nLoading openzfs.kext...\n" +/usr/bin/kextutil "${kernelextensionsdir}/openzfs.kext" 2>&1 | grep -q "rejected due to system policy" -if [[ $? -eq 0 ]]; then - open /System/Library/PreferencePanes/Security.prefPane -else - printf "\nLoading zfs.kext...\n" - /sbin/kextload "${kernelextensionsdir}/zfs.kext" -fi +err=$? +echo "kexutil said $err" + +open /System/Library/PreferencePanes/Security.prefPane exit 0 diff --git a/contrib/macOS/pkg-scripts/postinstall_actions/loadplists b/contrib/macOS/pkg-scripts/postinstall_actions/loadplists index 85cd15f19080..d5e6f9756049 100755 --- a/contrib/macOS/pkg-scripts/postinstall_actions/loadplists +++ b/contrib/macOS/pkg-scripts/postinstall_actions/loadplists @@ -2,28 +2,28 @@ date '+%s' >> /tmp/o3x.log echo $0 >> /tmp/o3x.log -launchctl list | grep org.openzfsonosx.zconfigd 1>/dev/null +launchctl list org.openzfsonosx.zconfigd 1>/dev/null [ $? -eq 0 ] && /bin/launchctl remove org.openzfsonosx.zconfigd /bin/launchctl load -w /Library/LaunchDaemons/org.openzfsonosx.zconfigd.plist -launchctl list | grep org.openzfsonosx.zed.service 1>/dev/null +launchctl list org.openzfsonosx.zed.service 1>/dev/null [ $? -eq 0 ] && /bin/launchctl remove org.openzfsonosx.zed.service -launchctl list | grep org.openzfsonosx.zed 1>/dev/null +launchctl list org.openzfsonosx.zed 1>/dev/null [ $? -eq 0 ] && /bin/launchctl remove org.openzfsonosx.zed /bin/launchctl load -w /Library/LaunchDaemons/org.openzfsonosx.zed.plist -launchctl list | grep org.openzfsonosx.InvariantDisks 1>/dev/null +launchctl list org.openzfsonosx.InvariantDisks 1>/dev/null [ $? -eq 0 ] && /bin/launchctl remove org.openzfsonosx.InvariantDisks /bin/launchctl load -w /Library/LaunchDaemons/org.openzfsonosx.InvariantDisks.plist -launchctl list | grep org.openzfsonosx.zpool-autoimport 1>/dev/null +launchctl list org.openzfsonosx.zpool-autoimport 1>/dev/null [ $? -eq 0 ] && /bin/launchctl remove org.openzfsonosx.zpool-autoimport -launchctl list | grep org.openzfsonosx.zpool-import-all 1>/dev/null +launchctl list org.openzfsonosx.zpool-import-all 1>/dev/null [ $? -eq 0 ] && /bin/launchctl remove org.openzfsonosx.zpool-import-all /bin/launchctl load -w /Library/LaunchDaemons/org.openzfsonosx.zpool-import-all.plist diff --git a/contrib/macOS/pkg-scripts/postinstall_actions/startzed b/contrib/macOS/pkg-scripts/postinstall_actions/startzed index a17445241389..bbe806ea5de9 100755 --- a/contrib/macOS/pkg-scripts/postinstall_actions/startzed +++ b/contrib/macOS/pkg-scripts/postinstall_actions/startzed @@ -1,5 +1,10 @@ #!/bin/bash +# zed is now run by launctl, so this script should not be required. +exit 0 + +ZED=/usr/local/zfs/bin/zed + set +e pgrep zed 1>/dev/null pgrepret=$? @@ -7,11 +12,11 @@ set -e if [ $pgrepret -ne 0 ] ; then echo "zed not started" - if [ -f /usr/sbin/zed ] ; then + if [ -f $ZED ] ; then echo "Starting zed ..." - /usr/sbin/zed + $ZED else - echo "zed binary /usr/sbin/zed not found" + echo "zed binary $ZED not found" fi else echo "zed already running" diff --git a/etc/launchd.d/.gitignore b/etc/launchd.d/.gitignore new file mode 100644 index 000000000000..f11cc9cf5e4f --- /dev/null +++ b/etc/launchd.d/.gitignore @@ -0,0 +1 @@ +zpool-import-all.sh diff --git a/etc/launchd.d/Makefile.am b/etc/launchd.d/Makefile.am new file mode 100644 index 000000000000..853836859a21 --- /dev/null +++ b/etc/launchd.d/Makefile.am @@ -0,0 +1,23 @@ + +noinst_launch_ddir = \ + $(launchdscriptdir)/ + +do_subst = -$(SED) -e 's,@bindir\@,$(bindir),g' \ + -e 's,@runstatedir\@,$(runstatedir),g' \ + -e 's,@sbindir\@,$(sbindir),g' \ + -e 's,@sysconfdir\@,$(sysconfdir),g' \ + -e 's,@launchdscriptdir\@,$(launchdscriptdir),g' + +CLEANFILES += %D%/zpool-import-all.sh + +INSTALL_DATA_HOOKS += zpool-import-all.sh + +zpool-import-all.sh: %D%/zpool-import-all.sh.in + -$(AM_V_at)$(MKDIR_P) $(DESTDIR)/$(launchdscriptdir)/ + -$(AM_V_GEN)$(SED) -e 's,@bindir\@,$(bindir),g' \ + -e 's,@runstatedir\@,$(runstatedir),g' \ + -e 's,@sbindir\@,$(sbindir),g' \ + -e 's,@sysconfdir\@,$(sysconfdir),g' \ + -e 's,@launchdscriptdir\@,$(launchdscriptdir),g' \ + '$<' > $(DESTDIR)/$(launchdscriptdir)/'$@' + chmod +x $(DESTDIR)/$(launchdscriptdir)/'$@' diff --git a/etc/launchd.d/zpool-import-all.sh.in b/etc/launchd.d/zpool-import-all.sh.in new file mode 100755 index 000000000000..7df5eba0ca64 --- /dev/null +++ b/etc/launchd.d/zpool-import-all.sh.in @@ -0,0 +1,60 @@ +#!/bin/bash + +echo "+zpool-import-all.sh" +date +export ZPOOL_IMPORT_ALL_COOKIE=/var/run/org.openzfsonosx.zpool-import-all.didRun +export INVARIANT_DISKS_IDLE_FILE=/var/run/disk/invariant.idle +export TIMEOUT_SECONDS=60 +export MAXIMUM_SLEEP_ITERATIONS=$((${TIMEOUT_SECONDS} * 10)) + +/usr/bin/time /usr/sbin/system_profiler SPParallelATADataType SPCardReaderDataType SPFibreChannelDataType SPFireWireDataType SPHardwareRAIDDataType SPNetworkDataType SPPCIDataType SPParallelSCSIDataType SPSASDataType SPSerialATADataType SPStorageDataType SPThunderboltDataType SPUSBDataType SPNetworkVolumeDataType 1>/dev/null 2>&1 + +/bin/sync + +echo "Waiting up to ${TIMEOUT_SECONDS} seconds for the InvariantDisks idle file ${INVARIANT_DISKS_IDLE_FILE} to exist" + +i=0 +while [ "${i}" -lt "${MAXIMUM_SLEEP_ITERATIONS}" -a ! -e "${INVARIANT_DISKS_IDLE_FILE}" ] +do + i=$((i+1)) + sleep .1 +done + +if [ -e "${INVARIANT_DISKS_IDLE_FILE}" ] +then + echo "Found ${INVARIANT_DISKS_IDLE_FILE} after ${i} iterations of sleeping 0.1 seconds" +else + echo "File ${INVARIANT_DISKS_IDLE_FILE} not found within ${TIMEOUT_SECONDS} seconds" +fi +date + +if [ -f "@sysconfdir@/zfs/zsysctl.conf" ]; then + @sbindir@/zsysctl -f @sysconfdir@/zfs/zsysctl.conf +fi + +# check to see if we have been instructed not to try to import at all +if [ -f /etc/zfs/noautoimport ]; +then + echo "/etc/zfs/noautoimport exits, exiting"; + exit 0; +fi + +date +echo "-zpool-import-all.sh" + +sleep 10 +echo "Loading and starting org.openzfsonosx.zpool-import" +date + +/bin/launchctl load /Library/LaunchDaemons/org.openzfsonosx.zpool-import.plist +/bin/launchctl kickstart -kp system/org.openzfsonosx.zpool-import + +echo Status: $? + +echo "Touching the file ${ZPOOL_IMPORT_ALL_COOKIE}" +touch "${ZPOOL_IMPORT_ALL_COOKIE}" + +date +echo "-zpool-import-all.sh" + +exit 0 diff --git a/etc/launchd/Makefile.am b/etc/launchd/Makefile.am new file mode 100644 index 000000000000..70047aa02e36 --- /dev/null +++ b/etc/launchd/Makefile.am @@ -0,0 +1,4 @@ +launchddaemondir = /Library/LaunchDaemons +launchdscriptdir = ${libexecdir}/zfs/launchd.d + +include $(srcdir)/%D%/daemons/Makefile.am diff --git a/etc/launchd/daemons/.gitignore b/etc/launchd/daemons/.gitignore new file mode 100644 index 000000000000..8b43b0f2a1fd --- /dev/null +++ b/etc/launchd/daemons/.gitignore @@ -0,0 +1,5 @@ +org.openzfsonosx.zconfigd.plist +org.openzfsonosx.zed.plist +org.openzfsonosx.zpool-import.plist +org.openzfsonosx.zpool-import-all.plist +org.openzfsonosx.InvariantDisks.plist diff --git a/etc/launchd/daemons/Makefile.am b/etc/launchd/daemons/Makefile.am new file mode 100644 index 000000000000..a5f758ab0235 --- /dev/null +++ b/etc/launchd/daemons/Makefile.am @@ -0,0 +1,25 @@ + +CLEANFILES += $(launchddaemon_DATA) + +launchddaemon_DATA = \ + $(top_srcdir)/etc/launchd/daemons/org.openzfsonosx.zconfigd.plist \ + $(top_srcdir)/etc/launchd/daemons/org.openzfsonosx.zed.plist \ + $(top_srcdir)/etc/launchd/daemons/org.openzfsonosx.zpool-import.plist \ + $(top_srcdir)/etc/launchd/daemons/org.openzfsonosx.zpool-import-all.plist \ + $(top_srcdir)/etc/launchd/daemons/org.openzfsonosx.InvariantDisks.plist + +%D%/org.openzfsonosx.zconfigd.plist: %D%/org.openzfsonosx.zconfigd.plist.in +%D%/org.openzfsonosx.zed.plist: %D%/org.openzfsonosx.zed.plist.in +%D%/org.openzfsonosx.zpool-import.plist: %D%/org.openzfsonosx.zpool-import.plist.in +%D%/org.openzfsonosx.zpool-import-all.plist: %D%/org.openzfsonosx.zpool-import-all.plist.in +%D%/org.openzfsonosx.InvariantDisks.plist: %D%/org.openzfsonosx.InvariantDisks.plist.in + +$(launchddaemon_DATA): + -$(AM_V_at)$(MKDIR_P) $(@D) + -$(AM_V_GEN)$(SED) -e 's,@bindir\@,$(bindir),g' \ + -e 's,@runstatedir\@,$(runstatedir),g' \ + -e 's,@sbindir\@,$(sbindir),g' \ + -e 's,@sysconfdir\@,$(sysconfdir),g' \ + -e 's,@launchddaemondir\@,$(launchddaemondir),g' \ + -e 's,@launchdscriptdir\@,$(launchdscriptdir),g' \ + '$<' >'$@' diff --git a/etc/launchd/daemons/org.openzfsonosx.InvariantDisks.plist.in b/etc/launchd/daemons/org.openzfsonosx.InvariantDisks.plist.in new file mode 100644 index 000000000000..4e49548aedb7 --- /dev/null +++ b/etc/launchd/daemons/org.openzfsonosx.InvariantDisks.plist.in @@ -0,0 +1,16 @@ + + + + + Label + org.openzfsonosx.InvariantDisks + ProgramArguments + + @sbindir@/InvariantDisks + + RunAtLoad + + KeepAlive + + + diff --git a/etc/launchd/daemons/org.openzfsonosx.zconfigd.plist.in b/etc/launchd/daemons/org.openzfsonosx.zconfigd.plist.in new file mode 100644 index 000000000000..51154e84661a --- /dev/null +++ b/etc/launchd/daemons/org.openzfsonosx.zconfigd.plist.in @@ -0,0 +1,20 @@ + + + + + Label + org.openzfsonosx.zconfigd + ProgramArguments + + @sbindir@/zconfigd + + RunAtLoad + + KeepAlive + + StandardErrorPath + /private/var/log/org.openzfsonosx.zconfigd.err + StandardOutPath + /private/var/log/org.openzfsonosx.zconfigd.log + + diff --git a/etc/launchd/daemons/org.openzfsonosx.zed.plist.in b/etc/launchd/daemons/org.openzfsonosx.zed.plist.in new file mode 100644 index 000000000000..1f62d4e2a9d2 --- /dev/null +++ b/etc/launchd/daemons/org.openzfsonosx.zed.plist.in @@ -0,0 +1,21 @@ + + + + + Label + org.openzfsonosx.zed + ProgramArguments + + @sbindir@/zed + -vfF + + RunAtLoad + + KeepAlive + + StandardErrorPath + /private/var/log/org.openzfsonosx.zed.err + StandardOutPath + /private/var/log/org.openzfsonosx.zed.log + + diff --git a/etc/launchd/daemons/org.openzfsonosx.zpool-import-all.plist.in b/etc/launchd/daemons/org.openzfsonosx.zpool-import-all.plist.in new file mode 100644 index 000000000000..3687eb418e33 --- /dev/null +++ b/etc/launchd/daemons/org.openzfsonosx.zpool-import-all.plist.in @@ -0,0 +1,18 @@ + + + + + Label + org.openzfsonosx.zpool-import-all + ProgramArguments + + @launchdscriptdir@/zpool-import-all.sh + + RunAtLoad + + StandardErrorPath + /private/var/log/org.openzfsonosx.zpool-import-all.err + StandardOutPath + /private/var/log/org.openzfsonosx.zpool-import-all.log + + diff --git a/etc/launchd/daemons/org.openzfsonosx.zpool-import.plist.in b/etc/launchd/daemons/org.openzfsonosx.zpool-import.plist.in new file mode 100644 index 000000000000..ee60285ba19e --- /dev/null +++ b/etc/launchd/daemons/org.openzfsonosx.zpool-import.plist.in @@ -0,0 +1,24 @@ + + + + + Label + org.openzfsonosx.zpool-import + ProgramArguments + + @sbindir@/zpool + import + -a + -d + /var/run/disk/by-id + + RunAtLoad + + LaunchOnlyOnce + + StandardErrorPath + /private/var/log/org.openzfsonosx.zpool-import-all.log + StandardOutPath + /private/var/log/org.openzfsonosx.zpool-import-all.log + + diff --git a/etc/paths.d/.gitignore b/etc/paths.d/.gitignore new file mode 100644 index 000000000000..f916e9fbee04 --- /dev/null +++ b/etc/paths.d/.gitignore @@ -0,0 +1 @@ +zfs-path diff --git a/etc/paths.d/Makefile.am b/etc/paths.d/Makefile.am new file mode 100644 index 000000000000..8c55fcdee5d8 --- /dev/null +++ b/etc/paths.d/Makefile.am @@ -0,0 +1,21 @@ +pathsddir = $(sysconfdir)/paths.d +pathsd_DATA = %D%/zfs-path + +paths_ddir = $(sysconfdir)/paths.d/ +dist_paths_d_DATA = \ + %D%/zfs-path + +SUBSTFILES += $(dist_paths_d_DATA) + +CLEANFILES += $(pathsd_DATA) +CLEANFILES += %D%/zfs-path + +INSTALL_DATA_HOOKS += pathsd-data-hook +pathsd-data-hook: + -$(SED) -e 's,@bindir\@,$(bindir),g' \ + -e 's,@runstatedir\@,$(runstatedir),g' \ + -e 's,@sbindir\@,$(sbindir),g' \ + -e 's,@sysconfdir\@,$(sysconfdir),g' \ + -e 's,@launchddaemondir\@,$(launchddaemondir),g' \ + -e 's,@launchdscriptdir\@,$(launchdscriptdir),g' \ + %D%/zfs-path.in > $(DESTDIR)/$(sysconfdir)/paths.d/zfs-path diff --git a/etc/paths.d/zfs-path.in b/etc/paths.d/zfs-path.in new file mode 100644 index 000000000000..5532f40754b3 --- /dev/null +++ b/etc/paths.d/zfs-path.in @@ -0,0 +1,2 @@ +@bindir@ +@sbindir@ diff --git a/include/os/macos/spl/libkern/libkern.h b/include/os/macos/spl/libkern/libkern.h index b1fd44555ab3..d4bf33cb678b 100644 --- a/include/os/macos/spl/libkern/libkern.h +++ b/include/os/macos/spl/libkern/libkern.h @@ -40,4 +40,10 @@ #undef copyinstr #define copyinstr(U, K, L, D) ddi_copyinstr((U), (K), (L), (D)) +#if defined(MAC_OS_X_VERSION_10_11) && \ + (MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_X_VERSION_10_11) +#else +#define IOSleepWithLeeway(S, D) IOSleep((S)) +#endif + #endif diff --git a/include/os/macos/spl/sys/atomic.h b/include/os/macos/spl/sys/atomic.h index 982fc3cdc837..d94df8e09a2e 100644 --- a/include/os/macos/spl/sys/atomic.h +++ b/include/os/macos/spl/sys/atomic.h @@ -302,7 +302,7 @@ ATOMIC_AND(ushort, ushort_t) type atomic_cas_##name(volatile type *target, type exp, type des) \ { \ __atomic_compare_exchange_n(target, \ - &exp, des, B_FALSE, \ + &exp, des, FALSE, \ __ATOMIC_SEQ_CST, __ATOMIC_SEQ_CST); \ return (exp); \ } diff --git a/include/os/macos/spl/sys/byteorder.h b/include/os/macos/spl/sys/byteorder.h index 498e467f2718..a712d84378c5 100644 --- a/include/os/macos/spl/sys/byteorder.h +++ b/include/os/macos/spl/sys/byteorder.h @@ -33,6 +33,7 @@ #include #include +#include #define LE_16(x) OSSwapHostToLittleInt16(x) #define LE_32(x) OSSwapHostToLittleInt32(x) diff --git a/include/os/macos/spl/sys/cred.h b/include/os/macos/spl/sys/cred.h index 4f97eec56fc4..06a106498fdd 100644 --- a/include/os/macos/spl/sys/cred.h +++ b/include/os/macos/spl/sys/cred.h @@ -28,12 +28,16 @@ #ifndef _SPL_CRED_H #define _SPL_CRED_H +struct ucred; +typedef struct ucred *kauth_cred_t; +typedef struct ucred cred_t; + +#include + #include #include #include -typedef struct ucred cred_t; - #define kcred spl_kcred() #define CRED() (cred_t *)kauth_cred_get() #define KUID_TO_SUID(x) (x) diff --git a/include/os/macos/spl/sys/debug.h b/include/os/macos/spl/sys/debug.h index 7c110503e268..96cb6d143506 100644 --- a/include/os/macos/spl/sys/debug.h +++ b/include/os/macos/spl/sys/debug.h @@ -83,6 +83,13 @@ void spl_backtrace(char *thesignal); int getpcstack(uintptr_t *pcstack, int pcstack_limit); void print_symbol(uintptr_t symbol); +static inline int +spl_assert(const char *buf, const char *file, const char *func, int line) +{ + spl_panic(file, func, line, "%s", buf); + return (0); +} + #ifndef expect #define expect(expr, value) (__builtin_expect((expr), (value))) #endif @@ -99,61 +106,60 @@ void print_symbol(uintptr_t symbol); #define VERIFY(cond) \ (void) (unlikely(!(cond)) && \ - spl_panic(__FILE__, __FUNCTION__, __LINE__, \ - "%s", "VERIFY(" #cond ") failed\n")) + spl_assert("VERIFY(" #cond ") failed\n", \ + __FILE__, __FUNCTION__, __LINE__)) #define VERIFY3B(LEFT, OP, RIGHT) do { \ - boolean_t _verify3_left = (boolean_t)(LEFT); \ - boolean_t _verify3_right = (boolean_t)(RIGHT); \ - if (!(_verify3_left OP _verify3_right)) \ + const boolean_t _verify3_left = (boolean_t)(LEFT); \ + const boolean_t _verify3_right = (boolean_t)(RIGHT); \ + if (unlikely(!(_verify3_left OP _verify3_right))) \ spl_panic(__FILE__, __FUNCTION__, __LINE__, \ "VERIFY3(" #LEFT " " #OP " " #RIGHT ") " \ "failed (%d " #OP " %d)\n", \ - (boolean_t) (_verify3_left), \ - (boolean_t) (_verify3_right)); \ + (boolean_t)_verify3_left, \ + (boolean_t)_verify3_right); \ } while (0) #define VERIFY3S(LEFT, OP, RIGHT) do { \ - int64_t _verify3_left = (int64_t)(LEFT); \ - int64_t _verify3_right = (int64_t)(RIGHT); \ - if (!(_verify3_left OP _verify3_right)) \ + const int64_t _verify3_left = (int64_t)(LEFT); \ + const int64_t _verify3_right = (int64_t)(RIGHT); \ + if (unlikely(!(_verify3_left OP _verify3_right))) \ spl_panic(__FILE__, __FUNCTION__, __LINE__, \ "VERIFY3(" #LEFT " " #OP " " #RIGHT ") " \ "failed (%lld " #OP " %lld)\n", \ - (long long) (_verify3_left), \ - (long long) (_verify3_right)); \ + (long long)_verify3_left, \ + (long long)_verify3_right); \ } while (0) #define VERIFY3U(LEFT, OP, RIGHT) do { \ - uint64_t _verify3_left = (uint64_t)(LEFT); \ - uint64_t _verify3_right = (uint64_t)(RIGHT); \ - if (!(_verify3_left OP _verify3_right)) \ + const uint64_t _verify3_left = (uint64_t)(LEFT); \ + const uint64_t _verify3_right = (uint64_t)(RIGHT); \ + if (unlikely(!(_verify3_left OP _verify3_right))) \ spl_panic(__FILE__, __FUNCTION__, __LINE__, \ "VERIFY3(" #LEFT " " #OP " " #RIGHT ") " \ "failed (%llu " #OP " %llu)\n", \ - (unsigned long long) (_verify3_left), \ - (unsigned long long) (_verify3_right)); \ + (unsigned long long)_verify3_left, \ + (unsigned long long)_verify3_right); \ } while (0) #define VERIFY3P(LEFT, OP, RIGHT) do { \ - uintptr_t _verify3_left = (uintptr_t)(LEFT); \ - uintptr_t _verify3_right = (uintptr_t)(RIGHT); \ - if (!(_verify3_left OP _verify3_right)) \ + const uintptr_t _verify3_left = (uintptr_t)(LEFT); \ + const uintptr_t _verify3_right = (uintptr_t)(RIGHT); \ + if (unlikely(!(_verify3_left OP _verify3_right))) \ spl_panic(__FILE__, __FUNCTION__, __LINE__, \ "VERIFY3(" #LEFT " " #OP " " #RIGHT ") " \ - "failed (%p " #OP " %p)\n", \ - (void *) (_verify3_left), \ - (void *) (_verify3_right)); \ + "failed (%px " #OP " %px)\n", \ + (void *)_verify3_left, \ + (void *)_verify3_right); \ } while (0) -#define VERIFY0(RIGHT) do { \ - int64_t _verify3_left = (int64_t)(0); \ - int64_t _verify3_right = (int64_t)(RIGHT); \ - if (!(_verify3_left == _verify3_right)) \ +#define VERIFY0(RIGHT) do { \ + const int64_t _verify0_right = (int64_t)(RIGHT); \ + if (unlikely(!(0 == _verify0_right))) \ spl_panic(__FILE__, __FUNCTION__, __LINE__, \ - "VERIFY3(0 == " #RIGHT ") " \ + "VERIFY0(" #RIGHT ") " \ "failed (0 == %lld)\n", \ - (long long) (_verify3_right)); \ + (long long)_verify0_right); \ } while (0) #define VERIFY0P(RIGHT) do { \ @@ -161,10 +167,98 @@ void print_symbol(uintptr_t symbol); if (unlikely(!(0 == _verify0_right))) \ spl_panic(__FILE__, __FUNCTION__, __LINE__, \ "VERIFY0P(" #RIGHT ") " \ - "failed (NULL == %p)\n", \ + "failed (NULL == %px)\n", \ (void *)_verify0_right); \ } while (0) +/* + * Note that you should not put any operations you want to always happen + * in the print section for ASSERTs unless you only want them to run on + * debug builds! + * e.g. ASSERT3UF(2, <, 3, "%s", foo(x)), foo(x) won't run on non-debug + * builds. + */ + +#define VERIFY3BF(LEFT, OP, RIGHT, STR, ...) do { \ + const boolean_t _verify3_left = (boolean_t)(LEFT); \ + const boolean_t _verify3_right = (boolean_t)(RIGHT); \ + if (unlikely(!(_verify3_left OP _verify3_right))) \ + spl_panic(__FILE__, __FUNCTION__, __LINE__, \ + "VERIFY3(" #LEFT " " #OP " " #RIGHT ") " \ + "failed (%d " #OP " %d) " STR "\n", \ + (boolean_t)(_verify3_left), \ + (boolean_t)(_verify3_right), \ + __VA_ARGS__); \ + } while (0) + +#define VERIFY3SF(LEFT, OP, RIGHT, STR, ...) do { \ + const int64_t _verify3_left = (int64_t)(LEFT); \ + const int64_t _verify3_right = (int64_t)(RIGHT); \ + if (unlikely(!(_verify3_left OP _verify3_right))) \ + spl_panic(__FILE__, __FUNCTION__, __LINE__, \ + "VERIFY3(" #LEFT " " #OP " " #RIGHT ") " \ + "failed (%lld " #OP " %lld) " STR "\n", \ + (long long)(_verify3_left), \ + (long long)(_verify3_right), \ + __VA_ARGS__); \ + } while (0) + +#define VERIFY3UF(LEFT, OP, RIGHT, STR, ...) do { \ + const uint64_t _verify3_left = (uint64_t)(LEFT); \ + const uint64_t _verify3_right = (uint64_t)(RIGHT); \ + if (unlikely(!(_verify3_left OP _verify3_right))) \ + spl_panic(__FILE__, __FUNCTION__, __LINE__, \ + "VERIFY3(" #LEFT " " #OP " " #RIGHT ") " \ + "failed (%llu " #OP " %llu) " STR "\n", \ + (unsigned long long)(_verify3_left), \ + (unsigned long long)(_verify3_right), \ + __VA_ARGS__); \ + } while (0) + +#define VERIFY3PF(LEFT, OP, RIGHT, STR, ...) do { \ + const uintptr_t _verify3_left = (uintptr_t)(LEFT); \ + const uintptr_t _verify3_right = (uintptr_t)(RIGHT); \ + if (unlikely(!(_verify3_left OP _verify3_right))) \ + spl_panic(__FILE__, __FUNCTION__, __LINE__, \ + "VERIFY3(" #LEFT " " #OP " " #RIGHT ") " \ + "failed (%px " #OP " %px) " STR "\n", \ + (void *) (_verify3_left), \ + (void *) (_verify3_right), \ + __VA_ARGS__); \ + } while (0) + +#define VERIFY0PF(RIGHT, STR, ...) do { \ + const uintptr_t _verify3_left = (uintptr_t)(0); \ + const uintptr_t _verify3_right = (uintptr_t)(RIGHT); \ + if (unlikely(!(_verify3_left == _verify3_right))) \ + spl_panic(__FILE__, __FUNCTION__, __LINE__, \ + "VERIFY0(0 == " #RIGHT ") " \ + "failed (0 == %px) " STR "\n", \ + (long long) (_verify3_right), \ + __VA_ARGS__); \ + } while (0) + +#define VERIFY0F(RIGHT, STR, ...) do { \ + const int64_t _verify3_left = (int64_t)(0); \ + const int64_t _verify3_right = (int64_t)(RIGHT); \ + if (unlikely(!(_verify3_left == _verify3_right))) \ + spl_panic(__FILE__, __FUNCTION__, __LINE__, \ + "VERIFY0(0 == " #RIGHT ") " \ + "failed (0 == %lld) " STR "\n", \ + (long long) (_verify3_right), \ + __VA_ARGS__); \ + } while (0) + +#define VERIFY_IMPLY(A, B) \ + ((void)(likely((!(A)) || (B)) || \ + spl_assert("(" #A ") implies (" #B ")", \ + __FILE__, __FUNCTION__, __LINE__))) + +#define VERIFY_EQUIV(A, B) \ + ((void)(likely(!!(A) == !!(B)) || \ + spl_assert("(" #A ") is equivalent to (" #B ")", \ + __FILE__, __FUNCTION__, __LINE__))) + #define CTASSERT_GLOBAL(x) _CTASSERT(x, __LINE__) #define CTASSERT(x) { _CTASSERT(x, __LINE__); } #define _CTASSERT(x, y) __CTASSERT(x, y) @@ -194,6 +288,13 @@ void print_symbol(uintptr_t symbol); ((void) sizeof ((uintptr_t)(x)), (void) sizeof ((uintptr_t)(z))) #define ASSERT0(x) ((void) sizeof ((uintptr_t)(x))) #define ASSERT0P(x) ((void) sizeof ((uintptr_t)(x))) +#define ASSERT3BF(x, y, z, str, ...) ASSERT3B(x, y, z) +#define ASSERT3SF(x, y, z, str, ...) ASSERT3S(x, y, z) +#define ASSERT3UF(x, y, z, str, ...) ASSERT3U(x, y, z) +#define ASSERT3PF(x, y, z, str, ...) ASSERT3P(x, y, z) +#define ASSERT0PF(x, str, ...) ASSERT0P(x) +#define ASSERT0F(x, str, ...) ASSERT0(x) +#define ASSERTF(x, str, ...) ASSERT(x) #define IMPLY(A, B) \ ((void) sizeof ((uintptr_t)(A)), (void) sizeof ((uintptr_t)(B))) #define EQUIV(A, B) \ @@ -213,16 +314,18 @@ void print_symbol(uintptr_t symbol); #define ASSERT3P VERIFY3P #define ASSERT0 VERIFY0 #define ASSERT0P VERIFY0P +#define ASSERT3BF VERIFY3BF +#define ASSERT3SF VERIFY3SF +#define ASSERT3UF VERIFY3UF +#define ASSERT3PF VERIFY3PF +#define ASSERT0PF VERIFY0PF +#define ASSERT0F VERIFY0F +#define ASSERTF VERIFYF #define ASSERT VERIFY #define ASSERTV(X) X __maybe_unused -#define IMPLY(A, B) \ - ((void)(((!(A)) || (B)) || \ - spl_panic(__FILE__, __FUNCTION__, __LINE__, \ - "(" #A ") implies (" #B ")"))) -#define EQUIV(A, B) \ - ((void)((!!(A) == !!(B)) || \ - spl_panic(__FILE__, __FUNCTION__, __LINE__, \ - "(" #A ") is equivalent to (" #B ")"))) +#define IMPLY VERIFY_IMPLY +#define EQUIV VERIFY_EQUIV + /* END CSTYLED */ #else /* MACOS_ASSERT_SHOULD_PANIC */ @@ -265,6 +368,13 @@ __attribute__((noinline)) int assfail(const char *str, const char *file, #define ASSERT3P(x, y, z) ASSERT3_IMPL(x, y, z, uintptr_t, "%p", (void *)) #define ASSERT0(x) ASSERT3_IMPL(0, ==, x, int64_t, "%lld", (long long)) #define ASSERT0P(x) ASSERT3_IMPL(0, ==, x, uintptr_t, "%p", (void *)) +#define ASSERT3BF(x) ASSERT3_IMPL(0, ==, x, boolean_t, "%p", (void *)) +#define ASSERT3SF(x) ASSERT3_IMPL(0, ==, x, int64_t, "%p", (void *)) +#define ASSERT3UF(x) ASSERT3_IMPL(0, ==, x, uint64_t, "%p", (void *)) +#define ASSERT3PF(x) ASSERT3_IMPL(0, ==, x, uintptr_t, "%p", (void *)) +#define ASSERT0PF(x) ASSERT3_IMPL(0, ==, x, uintptr_t, "%p", (void *)) +#define ASSERT0F(x) ASSERT3_IMPL(0, ==, x, int64_t, "%p", (void *)) +#define ASSERTF(x) ASSERT3_IMPL(0, ==, x, uintptr_t, "%p", (void *)) #define ASSERTV(x) x @@ -290,7 +400,9 @@ spl_implyout(const char *buf, const char *file, const char *func, int line) #endif /* MACOS_ASSERT_SHOULD_PANIC */ #endif /* NDEBUG */ +#if !defined(zfs_fallthrough) #define zfs_fallthrough __attribute__((__fallthrough__)) +#endif #ifdef __cplusplus } diff --git a/include/os/macos/spl/sys/kmem.h b/include/os/macos/spl/sys/kmem.h index f80c2d2e98b1..1bf08aa31013 100644 --- a/include/os/macos/spl/sys/kmem.h +++ b/include/os/macos/spl/sys/kmem.h @@ -53,6 +53,7 @@ extern uint64_t physmem; #define KM_NODEBUG 0x0010 /* NOT IMPLEMENTED ON OSX */ #define KM_NO_VBA 0x0020 /* OSX: don't descend to the bucket layer */ #define KM_VMFLAGS 0x00ff /* flags that must match VM_* flags */ +#define KMC_RECLAIMABLE 0x0 #define KM_FLAGS 0xffff /* all settable kmem flags */ diff --git a/include/os/macos/spl/sys/mod_os.h b/include/os/macos/spl/sys/mod_os.h index 9f25d3a162a3..3b040b38ff16 100644 --- a/include/os/macos/spl/sys/mod_os.h +++ b/include/os/macos/spl/sys/mod_os.h @@ -40,6 +40,9 @@ extern "C" { #define __init __attribute__((unused)) #define __exit __attribute__((unused)) +// Glancing at Linux kernel, module parameters limit: +#define LINUX_MAX_MODULE_PARAM_LEN 1024 + /* * The init/fini functions need to be called, but they are all static */ @@ -214,6 +217,12 @@ extern "C" { #define param_set_max_auto_ashift_args(var) \ CTLTYPE_U64, &var, sizeof (var), param_set_max_auto_ashift, "QU" +#define spa_taskq_read_param_set_args(var) \ + CTLTYPE_STRING, NULL, 0, spa_taskq_read_param, "A" + +#define spa_taskq_write_param_set_args(var) \ + CTLTYPE_STRING, NULL, 0, spa_taskq_write_param, "A" + #define fletcher_4_param_set_args(var) \ CTLTYPE_STRING, NULL, 0, fletcher_4_param, "A" diff --git a/include/os/macos/spl/sys/param.h b/include/os/macos/spl/sys/param.h index 1d01fa47b4a9..161b6e51be54 100644 --- a/include/os/macos/spl/sys/param.h +++ b/include/os/macos/spl/sys/param.h @@ -38,6 +38,8 @@ #define ptob(pages) (pages << PAGE_SHIFT) #define btop(bytes) (bytes >> PAGE_SHIFT) +#define PAGESHIFT PAGE_SHIFT + #define MAXUID UINT32_MAX #endif /* SPL_PARAM_H */ diff --git a/include/os/macos/spl/sys/proc.h b/include/os/macos/spl/sys/proc.h index 132964d9c1c1..8d7971b489e6 100644 --- a/include/os/macos/spl/sys/proc.h +++ b/include/os/macos/spl/sys/proc.h @@ -41,7 +41,7 @@ extern proc_t p0; /* process 0 */ static inline boolean_t zfs_proc_is_caller(proc_t *p) { - return (p == curproc); + return (p == (struct proc *)current_proc()); } #endif /* SPL_PROC_H */ diff --git a/include/os/macos/spl/sys/processor.h b/include/os/macos/spl/sys/processor.h index 993939576ddc..4d3313ded022 100644 --- a/include/os/macos/spl/sys/processor.h +++ b/include/os/macos/spl/sys/processor.h @@ -31,8 +31,48 @@ #include extern uint32_t getcpuid(void); -extern uint64_t spl_cpuid_features(void); + +#if defined(__amd64__) || defined(__i386__) + +#include + +#define __cpuid_count(__level, __count, __eax, __ebx, __ecx, __edx) \ + __asm("cpuid" : "=a"(__eax), "=b" (__ebx), "=c"(__ecx), "=d"(__edx) \ + : "0"(__level), "2"(__count)) + +static inline unsigned int +__get_cpuid_max(unsigned int __ext, unsigned int *__sig) +{ + uint32_t r[4]; + + do_cpuid(__ext, r); + if (__sig) + *__sig = r[ebx]; + return (r[eax]); +} + +/* macOS does have do_cpuid() macro */ +static inline int +__get_cpuid(unsigned int __level, + unsigned int *__eax, unsigned int *__ebx, + unsigned int *__ecx, unsigned int *__edx) +{ + unsigned int __ext = __level & 0x80000000; + uint32_t r[4]; + + if (__get_cpuid_max(__ext, 0) < __level) + return (0); + do_cpuid(__ext, r); + *__eax = r[eax]; + *__ebx = r[ebx]; + *__ecx = r[ecx]; + *__edx = r[edx]; + return (1); +} + +#endif // x86 typedef int processorid_t; +extern int spl_processor_init(void); #endif /* _SPL_PROCESSOR_H */ diff --git a/include/os/macos/spl/sys/signal.h b/include/os/macos/spl/sys/signal.h index f4bf1845e92d..941f9831d1d3 100644 --- a/include/os/macos/spl/sys/signal.h +++ b/include/os/macos/spl/sys/signal.h @@ -28,16 +28,18 @@ #ifndef _SPL_SYS_SIGNAL_H #define _SPL_SYS_SIGNAL_H +struct kthread; +typedef struct kthread *kthread_t; + #include #include_next -#include #define FORREAL 0 /* Usual side-effects */ #define JUSTLOOKING 1 /* Don't stop the process */ struct proc; -extern int thread_issignal(struct proc *, thread_t, sigset_t); +extern int thread_issignal(struct proc *, kthread_t, sigset_t); #define THREADMASK (sigmask(SIGILL)|sigmask(SIGTRAP)|\ sigmask(SIGIOT)|sigmask(SIGEMT)|\ @@ -46,14 +48,9 @@ extern int thread_issignal(struct proc *, thread_t, sigset_t); sigmask(SIGPIPE)|sigmask(SIGKILL)|\ sigmask(SIGTERM)|sigmask(SIGINT)) -static __inline__ int -issig(int why) -{ - return (thread_issignal(current_proc(), current_thread(), - THREADMASK)); -} +extern int issig(); /* Always called with curthread */ -#define signal_pending(p) issig(0) +#define signal_pending(p) issig() #endif /* SPL_SYS_SIGNAL_H */ diff --git a/include/os/macos/spl/sys/simd.h b/include/os/macos/spl/sys/simd.h index cd233ce947a4..e28c222498ed 100644 --- a/include/os/macos/spl/sys/simd.h +++ b/include/os/macos/spl/sys/simd.h @@ -29,6 +29,8 @@ #ifndef _MACOS_SIMD_H #define _MACOS_SIMD_H +#include + #if defined(__amd64__) || defined(__i386__) #include @@ -44,4 +46,7 @@ #define kfpu_fini() do {} while (0) #endif +#define simd_stat_init() do {} while (0) +#define simd_stat_fini() do {} while (0) + #endif diff --git a/include/os/macos/spl/sys/simd_x86.h b/include/os/macos/spl/sys/simd_x86.h index 7b670816889e..a119567338e7 100644 --- a/include/os/macos/spl/sys/simd_x86.h +++ b/include/os/macos/spl/sys/simd_x86.h @@ -85,31 +85,9 @@ #if defined(_KERNEL) #include #include - -#ifdef __APPLE__ -// XNU fpu.h -static inline uint64_t -xgetbv(uint32_t c) -{ - uint32_t mask_hi, mask_lo; - __asm__ __volatile__("xgetbv" : "=a"(mask_lo), "=d"(mask_hi) : "c" (c)); - return (((uint64_t)mask_hi<<32) + (uint64_t)mask_lo); -} - -#endif - -extern uint64_t spl_cpuid_features(void); -extern uint64_t spl_cpuid_leaf7_features(void); - -#endif - -/* - * CPUID feature tests for user-space. Linux kernel provides an interface for - * CPU feature testing. - */ -#if !defined(_KERNEL) - +#else #include +#endif // KERNEL /* * x86 registers used implicitly by CPUID @@ -147,7 +125,9 @@ typedef enum cpuid_inst_sets { AVX512ER, AVX512VL, AES, - PCLMULQDQ + PCLMULQDQ, + MOVBE, + SHANI, } cpuid_inst_sets_t; /* @@ -171,6 +151,8 @@ typedef struct cpuid_feature_desc { #define _AVX512VL_BIT (1U << 31) /* if used also check other levels */ #define _AES_BIT (1U << 25) #define _PCLMULQDQ_BIT (1U << 1) +#define _MOVBE_BIT (1U << 22) +#define _SHANI_BIT (1U << 29) /* * Descriptions of supported instruction sets @@ -198,6 +180,8 @@ static const cpuid_feature_desc_t spl_cpuid_features[] = { [AVX512VL] = {7U, 0U, _AVX512ER_BIT, EBX }, [AES] = {1U, 0U, _AES_BIT, ECX }, [PCLMULQDQ] = {1U, 0U, _PCLMULQDQ_BIT, ECX }, + [MOVBE] = {1U, 0U, _MOVBE_BIT, ECX }, + [SHANI] = {1U, 0U, _SHANI_BIT, EBX }, }; /* @@ -238,11 +222,11 @@ __cpuid_check_feature(const cpuid_feature_desc_t *desc) return (B_FALSE); } -#define CPUID_FEATURE_CHECK(name, id) \ -static inline boolean_t \ -__cpuid_has_ ## name(void) \ -{ \ - return (__cpuid_check_feature(&spl_cpuid_features[id])); \ +#define CPUID_FEATURE_CHECK(name, id) \ +static inline boolean_t \ +__cpuid_has_ ## name(void) \ +{ \ + return (__cpuid_check_feature(&spl_cpuid_features[id])); \ } /* @@ -270,8 +254,8 @@ CPUID_FEATURE_CHECK(avx512er, AVX512ER); CPUID_FEATURE_CHECK(avx512vl, AVX512VL); CPUID_FEATURE_CHECK(aes, AES); CPUID_FEATURE_CHECK(pclmulqdq, PCLMULQDQ); - -#endif /* !defined(_KERNEL) */ +CPUID_FEATURE_CHECK(shani, SHANI); +CPUID_FEATURE_CHECK(movbe, MOVBE); /* @@ -283,11 +267,8 @@ __simd_state_enabled(const uint64_t state) boolean_t has_osxsave; uint64_t xcr0; -#if defined(_KERNEL) - has_osxsave = !!(spl_cpuid_features() & CPUID_FEATURE_OSXSAVE); -#elif !defined(_KERNEL) has_osxsave = __cpuid_has_osxsave(); -#endif + if (!has_osxsave) return (B_FALSE); @@ -301,18 +282,13 @@ __simd_state_enabled(const uint64_t state) #define __ymm_enabled() __simd_state_enabled(_XSTATE_SSE_AVX) #define __zmm_enabled() __simd_state_enabled(_XSTATE_AVX512) - /* * Check if SSE instruction set is available */ static inline boolean_t zfs_sse_available(void) { -#if defined(_KERNEL) - return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE)); -#elif !defined(_KERNEL) return (__cpuid_has_sse()); -#endif } /* @@ -321,11 +297,7 @@ zfs_sse_available(void) static inline boolean_t zfs_sse2_available(void) { -#if defined(_KERNEL) - return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE2)); -#elif !defined(_KERNEL) return (__cpuid_has_sse2()); -#endif } /* @@ -334,11 +306,7 @@ zfs_sse2_available(void) static inline boolean_t zfs_sse3_available(void) { -#if defined(_KERNEL) - return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE3)); -#elif !defined(_KERNEL) return (__cpuid_has_sse3()); -#endif } /* @@ -347,11 +315,7 @@ zfs_sse3_available(void) static inline boolean_t zfs_ssse3_available(void) { -#if defined(_KERNEL) - return (!!(spl_cpuid_features() & CPUID_FEATURE_SSSE3)); -#elif !defined(_KERNEL) return (__cpuid_has_ssse3()); -#endif } /* @@ -360,11 +324,7 @@ zfs_ssse3_available(void) static inline boolean_t zfs_sse4_1_available(void) { -#if defined(_KERNEL) - return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE4_1)); -#elif !defined(_KERNEL) return (__cpuid_has_sse4_1()); -#endif } /* @@ -373,11 +333,16 @@ zfs_sse4_1_available(void) static inline boolean_t zfs_sse4_2_available(void) { -#if defined(_KERNEL) - return (!!(spl_cpuid_features() & CPUID_FEATURE_SSE4_2)); -#elif !defined(_KERNEL) return (__cpuid_has_sse4_2()); -#endif +} + +/* + * Check if SSE4.2 instruction set is available + */ +static inline boolean_t +zfs_osxsave_available(void) +{ + return (__cpuid_has_osxsave()); } /* @@ -387,12 +352,7 @@ static inline boolean_t zfs_avx_available(void) { boolean_t has_avx; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) - return (!!(spl_cpuid_features() & CPUID_FEATURE_AVX1_0)); -#elif !defined(_KERNEL) has_avx = __cpuid_has_avx(); -#endif return (has_avx && __ymm_enabled()); } @@ -404,17 +364,7 @@ static inline boolean_t zfs_avx2_available(void) { boolean_t has_avx2; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX2) - has_avx2 = (!!(spl_cpuid_leaf7_features() & CPUID_LEAF7_FEATURE_AVX2)); -#else - has_avx2 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx2 = __cpuid_has_avx2(); -#endif - return (has_avx2 && __ymm_enabled()); } @@ -424,15 +374,7 @@ zfs_avx2_available(void) static inline boolean_t zfs_bmi1_available(void) { -#if defined(_KERNEL) -#if defined(CPUID_LEAF7_FEATURE_BMI1) - return (!!(spl_cpuid_leaf7_features() & CPUID_LEAF7_FEATURE_BMI1)); -#else - return (B_FALSE); -#endif -#elif !defined(_KERNEL) return (__cpuid_has_bmi1()); -#endif } /* @@ -441,15 +383,7 @@ zfs_bmi1_available(void) static inline boolean_t zfs_bmi2_available(void) { -#if defined(_KERNEL) -#if defined(CPUID_LEAF7_FEATURE_BMI2) - return (!!(spl_cpuid_leaf7_features() & CPUID_LEAF7_FEATURE_BMI2)); -#else - return (B_FALSE); -#endif -#elif !defined(_KERNEL) return (__cpuid_has_bmi2()); -#endif } /* @@ -458,15 +392,7 @@ zfs_bmi2_available(void) static inline boolean_t zfs_aes_available(void) { -#if defined(_KERNEL) -#if defined(HAVE_AES) - return (!!(spl_cpuid_features() & CPUID_FEATURE_AES)); -#else - return (B_FALSE); -#endif -#elif !defined(_KERNEL) return (__cpuid_has_aes()); -#endif } /* @@ -475,15 +401,7 @@ zfs_aes_available(void) static inline boolean_t zfs_pclmulqdq_available(void) { -#if defined(_KERNEL) -#if defined(HAVE_PCLMULQDQ) - return (!!(spl_cpuid_features() & CPUID_FEATURE_PCLMULQDQ)); -#else - return (B_FALSE); -#endif -#elif !defined(_KERNEL) return (__cpuid_has_pclmulqdq()); -#endif } /* @@ -492,11 +410,7 @@ zfs_pclmulqdq_available(void) static inline boolean_t zfs_movbe_available(void) { -#if defined(X86_FEATURE_MOVBE) - return (!!(spl_cpuid_features() & CPUID_FEATURE_MOVBE)); -#else - return (B_FALSE); -#endif + return (__cpuid_has_movbe()); } /* @@ -505,11 +419,7 @@ zfs_movbe_available(void) static inline boolean_t zfs_shani_available(void) { -#if defined(X86_FEATURE_SHA_NI) - return (!!(spl_cpuid_features() & X86_FEATURE_SHA_NI)); -#else - return (B_FALSE); -#endif + return (__cpuid_has_shani()); } /* @@ -534,18 +444,7 @@ static inline boolean_t zfs_avx512f_available(void) { boolean_t has_avx512 = B_FALSE; - - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX512F) && defined(CPUID_LEAF7_FEATURE_AVX512F) - return (!!(spl_cpuid_leaf7_features() & CPUID_LEAF7_FEATURE_AVX512F)); -#else - has_avx512 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx512 = __cpuid_has_avx512f(); -#endif - return (has_avx512 && __zmm_enabled()); } @@ -555,21 +454,7 @@ zfs_avx512cd_available(void) { boolean_t has_avx512 = B_FALSE; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX512F) && defined(HAVE_AVX512CD) && \ - defined(CPUID_LEAF7_FEATURE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512CD) - has_avx512 = (spl_cpuid_leaf7_features() & - (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512CD)) == - (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512CD); -#else - has_avx512 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx512 = __cpuid_has_avx512cd(); -#endif - return (has_avx512 && __zmm_enabled()); } @@ -579,20 +464,7 @@ zfs_avx512er_available(void) { boolean_t has_avx512 = B_FALSE; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX512F) && defined(HAVE_AVX512ER) && \ - defined(CPUID_LEAF7_FEATURE_AVX512ER) - has_avx512 = (spl_cpuid_leaf7_features() & - (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512ER)) == - (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512ER); -#else - has_avx512 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx512 = __cpuid_has_avx512er(); -#endif - return (has_avx512 && __zmm_enabled()); } @@ -602,20 +474,7 @@ zfs_avx512pf_available(void) { boolean_t has_avx512 = B_FALSE; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX512PF) && defined(HAVE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512PF) - has_avx512 = (spl_cpuid_leaf7_features() & - (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512PF)) == - (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512PF); -#else - has_avx512 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx512 = __cpuid_has_avx512pf(); -#endif - return (has_avx512 && __zmm_enabled()); } @@ -625,21 +484,7 @@ zfs_avx512bw_available(void) { boolean_t has_avx512 = B_FALSE; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX512BW) && defined(HAVE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512BW) - has_avx512 = (spl_cpuid_leaf7_features() & - (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512BW)) == - (CPUID_LEAF7_FEATURE_AVX512F | CPUID_LEAF7_FEATURE_AVX512BW); -#else - has_avx512 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx512 = __cpuid_has_avx512bw(); -#endif - return (has_avx512 && __zmm_enabled()); } @@ -649,21 +494,7 @@ zfs_avx512dq_available(void) { boolean_t has_avx512 = B_FALSE; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX512DQ) && defined(HAVE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512DQ) - has_avx512 = (spl_cpuid_leaf7_features() & - (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512DQ)) == - (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512DQ); -#else - has_avx512 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx512 = __cpuid_has_avx512dq(); -#endif - return (has_avx512 && __zmm_enabled()); } @@ -673,21 +504,7 @@ zfs_avx512vl_available(void) { boolean_t has_avx512 = B_FALSE; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX512VL) && defined(HAVE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512VL) - has_avx512 = (spl_cpuid_leaf7_features() & - (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512VL)) == - (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512VL); -#else - has_avx512 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx512 = __cpuid_has_avx512vl(); -#endif - return (has_avx512 && __zmm_enabled()); } @@ -697,21 +514,7 @@ zfs_avx512ifma_available(void) { boolean_t has_avx512 = B_FALSE; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX512IFMA) && defined(HAVE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512IFMA) - has_avx512 = (spl_cpuid_leaf7_features() & - (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512IFMA)) == - (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512IFMA); -#else - has_avx512 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx512 = __cpuid_has_avx512ifma(); -#endif - return (has_avx512 && __zmm_enabled()); } @@ -721,22 +524,8 @@ zfs_avx512vbmi_available(void) { boolean_t has_avx512 = B_FALSE; - return (FALSE); // Currently broken on macOS -#if defined(_KERNEL) -#if defined(HAVE_AVX512VBMI) && defined(HAVE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512F) && \ - defined(CPUID_LEAF7_FEATURE_AVX512VBMI) - has_avx512 = (spl_cpuid_leaf7_features() & - (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512VBMI)) == - (CPUID_LEAF7_FEATURE_AVX512F|CPUID_LEAF7_FEATURE_AVX512VBMI); -#else - has_avx512 = B_FALSE; -#endif -#elif !defined(_KERNEL) has_avx512 = __cpuid_has_avx512f() && __cpuid_has_avx512vbmi(); -#endif - return (has_avx512 && __zmm_enabled()); } diff --git a/include/os/macos/spl/sys/sysmacros.h b/include/os/macos/spl/sys/sysmacros.h index 9c4a6818b301..ed04635da3e3 100644 --- a/include/os/macos/spl/sys/sysmacros.h +++ b/include/os/macos/spl/sys/sysmacros.h @@ -89,7 +89,7 @@ extern "C" { #define is_system_labeled() 0 extern unsigned int max_ncpus; -#define boot_ncpus max_ncpus +extern unsigned int boot_ncpus; extern unsigned int num_ecores; #ifndef RLIM64_INFINITY diff --git a/include/os/macos/spl/sys/taskq.h b/include/os/macos/spl/sys/taskq.h index a7e691e2f070..e38541a6e8cd 100644 --- a/include/os/macos/spl/sys/taskq.h +++ b/include/os/macos/spl/sys/taskq.h @@ -61,6 +61,7 @@ struct taskq_ent; #ifdef __APPLE__ #define TASKQ_TIMESHARE 0x0020 /* macOS dynamic thread priority */ #define TASKQ_REALLY_DYNAMIC 0x0040 /* don't filter out TASKQ_DYNAMIC */ +#define TASKQ_CREATE_SYNCED 0x0080 /* don't deflate ncpus */ #endif /* @@ -92,6 +93,8 @@ extern taskq_t *taskq_create_proc(const char *, int, pri_t, int, int, proc_t *, uint_t); extern taskq_t *taskq_create_sysdc(const char *, int, int, int, proc_t *, uint_t, uint_t); +extern taskq_t *taskq_create_synced(const char *, int, pri_t, int, int, uint_t, + kthread_t ***); extern taskqid_t taskq_dispatch(taskq_t *, task_func_t, void *, uint_t); extern void nulltask(void *); extern void taskq_destroy(taskq_t *); diff --git a/include/os/macos/spl/sys/thread.h b/include/os/macos/spl/sys/thread.h index 52d9c135b8a6..abeca0ba7bdf 100644 --- a/include/os/macos/spl/sys/thread.h +++ b/include/os/macos/spl/sys/thread.h @@ -31,12 +31,12 @@ #include #include -#include #include #include #include #include #include +#include // thread_t #ifdef __cplusplus extern "C" { @@ -51,6 +51,8 @@ extern "C" { #define kthread thread #define kthread_t struct kthread +#include + /* * Thread interfaces */ @@ -75,6 +77,33 @@ typedef void (*thread_func_t)(void *); // Drop the p0 argument, not used. +#if defined(MAC_OS_X_VERSION_10_9) && \ + (MAC_OS_X_VERSION_MIN_REQUIRED <= MAC_OS_X_VERSION_10_9) +/* Missing in 10.9 - none of this will be run, but handles us compile */ +#define THREAD_LATENCY_QOS_POLICY 7 +#define THREAD_LATENCY_QOS_POLICY_COUNT ((mach_msg_type_number_t) \ + (sizeof (thread_latency_qos_policy_data_t) / sizeof (integer_t))) +#define THREAD_THROUGHPUT_QOS_POLICY 8 +#define THREAD_THROUGHPUT_QOS_POLICY_COUNT ((mach_msg_type_number_t) \ + (sizeof (thread_throughput_qos_policy_data_t) / sizeof (integer_t))) +typedef integer_t thread_latency_qos_t; +typedef integer_t thread_throughput_qos_t; +struct thread_throughput_qos_policy { + thread_throughput_qos_t thread_throughput_qos_tier; +}; +struct thread_latency_qos_policy { + thread_latency_qos_t thread_latency_qos_tier; +}; +typedef struct thread_throughput_qos_policy +thread_throughput_qos_policy_data_t; +typedef struct thread_latency_qos_policy +thread_latency_qos_policy_data_t; +typedef struct thread_throughput_qos_policy +*thread_throughput_qos_policy_t; +typedef struct thread_latency_qos_policy +*thread_latency_qos_policy_t; +#endif + #ifdef SPL_DEBUG_THREAD #define thread_create(A, B, C, D, E, F, G, H) \ @@ -130,29 +159,6 @@ extern kthread_t *spl_thread_create_named_with_extpol_and_qos( #endif -#if defined(MAC_OS_X_VERSION_10_9) && \ - (MAC_OS_X_VERSION_MIN_REQUIRED <= MAC_OS_X_VERSION_10_9) -/* Missing in 10.9 - none of this will be run, but handles us compile */ -#define THREAD_LATENCY_QOS_POLICY 7 -#define THREAD_LATENCY_QOS_POLICY_COUNT ((mach_msg_type_number_t) \ - (sizeof (thread_latency_qos_policy_data_t) / sizeof (integer_t))) -#define THREAD_THROUGHPUT_QOS_POLICY 8 -#define THREAD_THROUGHPUT_QOS_POLICY_COUNT ((mach_msg_type_number_t) \ - (sizeof (thread_throughput_qos_policy_data_t) / sizeof (integer_t))) -typedef integer_t thread_latency_qos_t; -typedef integer_t thread_throughput_qos_t; -struct thread_throughput_qos_policy { - thread_throughput_qos_t thread_throughput_qos_tier; -}; -struct thread_latency_qos_policy { - thread_latency_qos_t thread_latency_qos_tier; -}; -typedef struct thread_throughput_qos_policy -thread_throughput_qos_policy_data_t; -typedef struct thread_latency_qos_policy -thread_latency_qos_policy_data_t; -#endif - #define thread_exit spl_thread_exit extern void spl_thread_exit(void) __attribute__((noreturn)); @@ -170,7 +176,7 @@ extern void spl_set_thread_latency(thread_t, thread_latency_qos_policy_data_t *, const char *); #define delay osx_delay -extern void osx_delay(int); +extern void osx_delay(clock_t); #define KPREEMPT_SYNC 0 static inline void kpreempt(int flags) diff --git a/include/os/macos/spl/sys/tsd.h b/include/os/macos/spl/sys/tsd.h index cfc48000a5dd..9a3e3ba0d198 100644 --- a/include/os/macos/spl/sys/tsd.h +++ b/include/os/macos/spl/sys/tsd.h @@ -31,6 +31,7 @@ #define _SPL_TSD_H #include +#include #define TSD_HASH_TABLE_BITS_DEFAULT 9 #define TSD_KEYS_MAX 32768 @@ -41,7 +42,7 @@ typedef void (*dtor_func_t)(void *); extern int tsd_set(uint_t, void *); extern void *tsd_get(uint_t); -extern void *tsd_get_by_thread(uint_t, thread_t); +extern void *tsd_get_by_thread(uint_t, kthread_t *); extern void tsd_create(uint_t *, dtor_func_t); extern void tsd_destroy(uint_t *); extern void tsd_exit(void); diff --git a/include/os/macos/spl/sys/types.h b/include/os/macos/spl/sys/types.h index 2d4ca3459b5d..3b3345832917 100644 --- a/include/os/macos/spl/sys/types.h +++ b/include/os/macos/spl/sys/types.h @@ -32,11 +32,31 @@ #define likely(x) __builtin_expect(!!(x), 1) #define unlikely(x) __builtin_expect(!!(x), 0) +typedef short pri_t; +typedef unsigned long ulong_t; +typedef unsigned long long u_longlong_t; +typedef unsigned long long rlim64_t; +typedef unsigned long long loff_t; +typedef long long longlong_t; +typedef unsigned char uchar_t; +typedef unsigned int uint_t; +typedef unsigned short ushort_t; +typedef void *spinlock_t; +typedef long long offset_t; +typedef struct timespec timestruc_t; /* definition per SVr4 */ +typedef struct timespec timespec_t; +typedef ulong_t pgcnt_t; +typedef unsigned int umode_t; +#define NODEV32 (dev32_t)(-1) +typedef unsigned int dev32_t; +typedef unsigned int minor_t; +typedef short index_t; + #include #include #include_next #include -#include + #include #if !defined(MAC_OS_X_VERSION_10_12) || \ @@ -83,26 +103,6 @@ typedef enum { B_FALSE, B_TRUE } boolean_t; #endif /* NEED_SOLARIS_BOOLEAN */ -typedef short pri_t; -typedef unsigned long ulong_t; -typedef unsigned long long u_longlong_t; -typedef unsigned long long rlim64_t; -typedef unsigned long long loff_t; -typedef long long longlong_t; -typedef unsigned char uchar_t; -typedef unsigned int uint_t; -typedef unsigned short ushort_t; -typedef void *spinlock_t; -typedef long long offset_t; -typedef struct timespec timestruc_t; /* definition per SVr4 */ -typedef struct timespec timespec_t; -typedef ulong_t pgcnt_t; -typedef unsigned int umode_t; -#define NODEV32 (dev32_t)(-1) -typedef uint32_t dev32_t; -typedef uint_t minor_t; -typedef short index_t; - #include #define FCREAT O_CREAT #define FTRUNC O_TRUNC diff --git a/include/os/macos/spl/sys/uio.h b/include/os/macos/spl/sys/uio.h index f7f0d8152080..7d0923c850ea 100644 --- a/include/os/macos/spl/sys/uio.h +++ b/include/os/macos/spl/sys/uio.h @@ -48,6 +48,11 @@ extern "C" { #endif +/* + * uio_extflg: extended flags + */ +#define UIO_DIRECT 0x0001 /* Direct I/O request */ + typedef struct iovec iovec_t; typedef enum uio_seg zfs_uio_seg_t; @@ -65,6 +70,15 @@ typedef enum uio_rw zfs_uio_rw_t; typedef size_t (*zfs_uio_func)(char *addr, uint64_t offset, size_t len, zfs_uio_rw_t rw, const void *privptr); +/* + * This structure is used when doing Direct I/O. + */ +typedef void vm_page_t; +typedef struct { + vm_page_t *pages; + int npages; +} zfs_uio_dio_t; + /* * Hybrid uio, use OS uio for IO and communicating with XNU * and internal uio for ZFS / crypto. The default mode is @@ -81,6 +95,7 @@ typedef struct zfs_uio { const struct iovec *uio_iov; int uio_iovcnt; off_t uio_loffset; + off_t uio_soffset; zfs_uio_seg_t uio_segflg; boolean_t uio_fault_disable; uint16_t uio_fmode; @@ -88,6 +103,7 @@ typedef struct zfs_uio { ssize_t uio_resid; size_t uio_skip; zfs_uio_func uio_iofunc; + zfs_uio_dio_t uio_dio; } zfs_uio_t; @@ -142,6 +158,12 @@ zfs_uio_offset(zfs_uio_t *uio) return (uio->uio_loffset); } +static inline off_t +zfs_uio_soffset(zfs_uio_t *uio) +{ + return (uio->uio_soffset); +} + static inline size_t zfs_uio_resid(zfs_uio_t *uio) { @@ -160,6 +182,12 @@ zfs_uio_setoffset(zfs_uio_t *uio, off_t off) uio->uio_loffset = off; } +static inline void +zfs_uio_setsoffset(zfs_uio_t *uio, offset_t off) +{ + uio->uio_soffset = off; +} + static inline void zfs_uio_advance(zfs_uio_t *uio, size_t size) { @@ -204,6 +232,7 @@ zfs_uio_iovec_init(zfs_uio_t *uio, const struct iovec *iov, uio->uio_iov = iov; uio->uio_iovcnt = nr_segs; uio->uio_loffset = offset; + uio->uio_soffset = offset; uio->uio_segflg = seg; uio->uio_fmode = 0; uio->uio_extflg = 0; diff --git a/include/os/macos/spl/sys/vnode.h b/include/os/macos/spl/sys/vnode.h index e3a8c13810d2..68ded3aaceb6 100644 --- a/include/os/macos/spl/sys/vnode.h +++ b/include/os/macos/spl/sys/vnode.h @@ -42,7 +42,6 @@ #include - // Be aware that Apple defines "typedef struct vnode *vnode_t" and // ZFS uses "typedef struct vnode vnode_t". // uio and vnode wrappers can be removed now. diff --git a/include/os/macos/zfs/sys/abd_impl_os.h b/include/os/macos/zfs/sys/abd_impl_os.h new file mode 100644 index 000000000000..82296365834f --- /dev/null +++ b/include/os/macos/zfs/sys/abd_impl_os.h @@ -0,0 +1,38 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + */ + +#ifndef _ABD_IMPL_OS_H +#define _ABD_IMPL_OS_H + +#ifdef __cplusplus +extern "C" { +#endif + +#define abd_enter_critical(flags) (flags) = ml_set_interrupts_enabled(FALSE) +#define abd_exit_critical(flags) ml_set_interrupts_enabled((flags)) + +#ifdef __cplusplus +} +#endif + +#endif /* _ABD_IMPL_OS_H */ diff --git a/include/os/macos/zfs/sys/abd_os.h b/include/os/macos/zfs/sys/abd_os.h new file mode 100644 index 000000000000..63c4859482fa --- /dev/null +++ b/include/os/macos/zfs/sys/abd_os.h @@ -0,0 +1,57 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright (c) 2014 by Chunwei Chen. All rights reserved. + * Copyright (c) 2016, 2019 by Delphix. All rights reserved. + */ + +#ifndef _ABD_OS_H +#define _ABD_OS_H + +#ifdef _KERNEL +#include +#endif + +#ifdef __cplusplus +extern "C" { +#endif + +struct abd; + +struct abd_scatter { + uint_t abd_offset; + void *abd_chunks[1]; /* actually variable-length */ +}; + +struct abd_linear { + void *abd_buf; +}; + +#ifdef _KERNEL +__attribute__((malloc)) +struct abd *abd_alloc_from_pages(vm_page_t *, unsigned long, uint64_t); +#endif + +#ifdef __cplusplus +} +#endif + +#endif /* _ABD_H */ diff --git a/include/os/macos/zfs/sys/ldi_osx.h b/include/os/macos/zfs/sys/ldi_osx.h index 2d78017c4245..7116ab837e63 100644 --- a/include/os/macos/zfs/sys/ldi_osx.h +++ b/include/os/macos/zfs/sys/ldi_osx.h @@ -41,6 +41,7 @@ #define _SYS_LDI_OSX_H #include +#include /* * OS X - The initialization/destructor functions are available diff --git a/include/os/macos/zfs/sys/zfs_vfsops_os.h b/include/os/macos/zfs/sys/zfs_vfsops_os.h index 09c7b95802a0..80e9f67b7903 100644 --- a/include/os/macos/zfs/sys/zfs_vfsops_os.h +++ b/include/os/macos/zfs/sys/zfs_vfsops_os.h @@ -173,9 +173,6 @@ int zfs_vfs_uuid_gen(const char *osname, uuid_t uuid); #define ZFS_TEARDOWN_DESTROY(zfsvfs) \ rrm_destroy(&(zfsvfs)->z_teardown_lock) -#define ZFS_TEARDOWN_TRY_ENTER_READ(zfsvfs) \ - rw_tryenter(&(zfsvfs)->z_teardown_lock, RW_READER) - #define ZFS_TEARDOWN_ENTER_READ(zfsvfs, tag) \ rrm_enter_read(&(zfsvfs)->z_teardown_lock, tag); diff --git a/lib/libspl/asm-x86_64/atomic.S b/lib/libspl/asm-x86_64/atomic.S new file mode 100644 index 000000000000..bb7dff12cc8e --- /dev/null +++ b/lib/libspl/asm-x86_64/atomic.S @@ -0,0 +1,693 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or http://www.opensolaris.org/os/licensing. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ +/* + * Copyright 2007 Sun Microsystems, Inc. All rights reserved. + * Use is subject to license terms. + */ + + .ident "%Z%%M% %I% %E% SMI" + + .file "%M%" + +#define _ASM +#ifdef __linux__ +#include +#elif __FreeBSD__ +#include +#define SET_SIZE(x) +#elif __APPLE__ +#include +#endif + ENTRY(atomic_inc_8) + ALTENTRY(atomic_inc_uchar) + lock + incb (%rdi) + ret + SET_SIZE(atomic_inc_uchar) + SET_SIZE(atomic_inc_8) + + ENTRY(atomic_inc_16) + ALTENTRY(atomic_inc_ushort) + lock + incw (%rdi) + ret + SET_SIZE(atomic_inc_ushort) + SET_SIZE(atomic_inc_16) + + ENTRY(atomic_inc_32) + ALTENTRY(atomic_inc_uint) + lock + incl (%rdi) + ret + SET_SIZE(atomic_inc_uint) + SET_SIZE(atomic_inc_32) + + ENTRY(atomic_inc_64) + ALTENTRY(atomic_inc_ulong) + lock + incq (%rdi) + ret + SET_SIZE(atomic_inc_ulong) + SET_SIZE(atomic_inc_64) + + ENTRY(atomic_inc_8_nv) + ALTENTRY(atomic_inc_uchar_nv) + movb (%rdi), %al +1: + leaq 1(%rax), %rcx + lock + cmpxchgb %cl, (%rdi) + jne 1b + movzbl %cl, %eax + ret + SET_SIZE(atomic_inc_uchar_nv) + SET_SIZE(atomic_inc_8_nv) + + ENTRY(atomic_inc_16_nv) + ALTENTRY(atomic_inc_ushort_nv) + movw (%rdi), %ax +1: + leaq 1(%rax), %rcx + lock + cmpxchgw %cx, (%rdi) + jne 1b + movzwl %cx, %eax + ret + SET_SIZE(atomic_inc_ushort_nv) + SET_SIZE(atomic_inc_16_nv) + + ENTRY(atomic_inc_32_nv) + ALTENTRY(atomic_inc_uint_nv) + movl (%rdi), %eax +1: + leaq 1(%rax), %rcx + lock + cmpxchgl %ecx, (%rdi) + jne 1b + movl %ecx, %eax + ret + SET_SIZE(atomic_inc_uint_nv) + SET_SIZE(atomic_inc_32_nv) + + ENTRY(atomic_inc_64_nv) + ALTENTRY(atomic_inc_ulong_nv) + movq (%rdi), %rax +1: + leaq 1(%rax), %rcx + lock + cmpxchgq %rcx, (%rdi) + jne 1b + movq %rcx, %rax + ret + SET_SIZE(atomic_inc_ulong_nv) + SET_SIZE(atomic_inc_64_nv) + + ENTRY(atomic_dec_8) + ALTENTRY(atomic_dec_uchar) + lock + decb (%rdi) + ret + SET_SIZE(atomic_dec_uchar) + SET_SIZE(atomic_dec_8) + + ENTRY(atomic_dec_16) + ALTENTRY(atomic_dec_ushort) + lock + decw (%rdi) + ret + SET_SIZE(atomic_dec_ushort) + SET_SIZE(atomic_dec_16) + + ENTRY(atomic_dec_32) + ALTENTRY(atomic_dec_uint) + lock + decl (%rdi) + ret + SET_SIZE(atomic_dec_uint) + SET_SIZE(atomic_dec_32) + + ENTRY(atomic_dec_64) + ALTENTRY(atomic_dec_ulong) + lock + decq (%rdi) + ret + SET_SIZE(atomic_dec_ulong) + SET_SIZE(atomic_dec_64) + + ENTRY(atomic_dec_8_nv) + ALTENTRY(atomic_dec_uchar_nv) + movb (%rdi), %al +1: + leaq -1(%rax), %rcx + lock + cmpxchgb %cl, (%rdi) + jne 1b + movzbl %cl, %eax + ret + SET_SIZE(atomic_dec_uchar_nv) + SET_SIZE(atomic_dec_8_nv) + + ENTRY(atomic_dec_16_nv) + ALTENTRY(atomic_dec_ushort_nv) + movw (%rdi), %ax +1: + leaq -1(%rax), %rcx + lock + cmpxchgw %cx, (%rdi) + jne 1b + movzwl %cx, %eax + ret + SET_SIZE(atomic_dec_ushort_nv) + SET_SIZE(atomic_dec_16_nv) + + ENTRY(atomic_dec_32_nv) + ALTENTRY(atomic_dec_uint_nv) + movl (%rdi), %eax +1: + leaq -1(%rax), %rcx + lock + cmpxchgl %ecx, (%rdi) + jne 1b + movl %ecx, %eax + ret + SET_SIZE(atomic_dec_uint_nv) + SET_SIZE(atomic_dec_32_nv) + + ENTRY(atomic_dec_64_nv) + ALTENTRY(atomic_dec_ulong_nv) + movq (%rdi), %rax +1: + leaq -1(%rax), %rcx + lock + cmpxchgq %rcx, (%rdi) + jne 1b + movq %rcx, %rax + ret + SET_SIZE(atomic_dec_ulong_nv) + SET_SIZE(atomic_dec_64_nv) + + ENTRY(atomic_add_8) + ALTENTRY(atomic_add_char) + lock + addb %sil, (%rdi) + ret + SET_SIZE(atomic_add_char) + SET_SIZE(atomic_add_8) + + ENTRY(atomic_add_16) + ALTENTRY(atomic_add_short) + lock + addw %si, (%rdi) + ret + SET_SIZE(atomic_add_short) + SET_SIZE(atomic_add_16) + + ENTRY(atomic_add_32) + ALTENTRY(atomic_add_int) + lock + addl %esi, (%rdi) + ret + SET_SIZE(atomic_add_int) + SET_SIZE(atomic_add_32) + + ENTRY(atomic_add_64) + ALTENTRY(atomic_add_ptr) + ALTENTRY(atomic_add_long) + lock + addq %rsi, (%rdi) + ret + SET_SIZE(atomic_add_long) + SET_SIZE(atomic_add_ptr) + SET_SIZE(atomic_add_64) + + ENTRY(atomic_sub_8) + ALTENTRY(atomic_sub_char) + lock + subb %sil, (%rdi) + ret + SET_SIZE(atomic_sub_char) + SET_SIZE(atomic_sub_8) + + ENTRY(atomic_sub_16) + ALTENTRY(atomic_sub_short) + lock + subw %si, (%rdi) + ret + SET_SIZE(atomic_sub_short) + SET_SIZE(atomic_sub_16) + + ENTRY(atomic_sub_32) + ALTENTRY(atomic_sub_int) + lock + subl %esi, (%rdi) + ret + SET_SIZE(atomic_sub_int) + SET_SIZE(atomic_sub_32) + + ENTRY(atomic_sub_64) + ALTENTRY(atomic_sub_ptr) + ALTENTRY(atomic_sub_long) + lock + subq %rsi, (%rdi) + ret + SET_SIZE(atomic_sub_long) + SET_SIZE(atomic_sub_ptr) + SET_SIZE(atomic_sub_64) + + ENTRY(atomic_or_8) + ALTENTRY(atomic_or_uchar) + lock + orb %sil, (%rdi) + ret + SET_SIZE(atomic_or_uchar) + SET_SIZE(atomic_or_8) + + ENTRY(atomic_or_16) + ALTENTRY(atomic_or_ushort) + lock + orw %si, (%rdi) + ret + SET_SIZE(atomic_or_ushort) + SET_SIZE(atomic_or_16) + + ENTRY(atomic_or_32) + ALTENTRY(atomic_or_uint) + lock + orl %esi, (%rdi) + ret + SET_SIZE(atomic_or_uint) + SET_SIZE(atomic_or_32) + + ENTRY(atomic_or_64) + ALTENTRY(atomic_or_ulong) + lock + orq %rsi, (%rdi) + ret + SET_SIZE(atomic_or_ulong) + SET_SIZE(atomic_or_64) + + ENTRY(atomic_and_8) + ALTENTRY(atomic_and_uchar) + lock + andb %sil, (%rdi) + ret + SET_SIZE(atomic_and_uchar) + SET_SIZE(atomic_and_8) + + ENTRY(atomic_and_16) + ALTENTRY(atomic_and_ushort) + lock + andw %si, (%rdi) + ret + SET_SIZE(atomic_and_ushort) + SET_SIZE(atomic_and_16) + + ENTRY(atomic_and_32) + ALTENTRY(atomic_and_uint) + lock + andl %esi, (%rdi) + ret + SET_SIZE(atomic_and_uint) + SET_SIZE(atomic_and_32) + + ENTRY(atomic_and_64) + ALTENTRY(atomic_and_ulong) + lock + andq %rsi, (%rdi) + ret + SET_SIZE(atomic_and_ulong) + SET_SIZE(atomic_and_64) + + ENTRY(atomic_add_8_nv) + ALTENTRY(atomic_add_char_nv) + movb (%rdi), %al +1: + movb %sil, %cl + addb %al, %cl + lock + cmpxchgb %cl, (%rdi) + jne 1b + movzbl %cl, %eax + ret + SET_SIZE(atomic_add_char_nv) + SET_SIZE(atomic_add_8_nv) + + ENTRY(atomic_add_16_nv) + ALTENTRY(atomic_add_short_nv) + movw (%rdi), %ax +1: + movw %si, %cx + addw %ax, %cx + lock + cmpxchgw %cx, (%rdi) + jne 1b + movzwl %cx, %eax + ret + SET_SIZE(atomic_add_short_nv) + SET_SIZE(atomic_add_16_nv) + + ENTRY(atomic_add_32_nv) + ALTENTRY(atomic_add_int_nv) + movl (%rdi), %eax +1: + movl %esi, %ecx + addl %eax, %ecx + lock + cmpxchgl %ecx, (%rdi) + jne 1b + movl %ecx, %eax + ret + SET_SIZE(atomic_add_int_nv) + SET_SIZE(atomic_add_32_nv) + + ENTRY(atomic_add_64_nv) + ALTENTRY(atomic_add_ptr_nv) + ALTENTRY(atomic_add_long_nv) + movq (%rdi), %rax +1: + movq %rsi, %rcx + addq %rax, %rcx + lock + cmpxchgq %rcx, (%rdi) + jne 1b + movq %rcx, %rax + ret + SET_SIZE(atomic_add_long_nv) + SET_SIZE(atomic_add_ptr_nv) + SET_SIZE(atomic_add_64_nv) + + ENTRY(atomic_sub_8_nv) + ALTENTRY(atomic_sub_char_nv) + movb (%rdi), %al +1: + movb %sil, %cl + subb %al, %cl + lock + cmpxchgb %cl, (%rdi) + jne 1b + movzbl %cl, %eax + ret + SET_SIZE(atomic_sub_char_nv) + SET_SIZE(atomic_sub_8_nv) + + ENTRY(atomic_sub_16_nv) + ALTENTRY(atomic_sub_short_nv) + movw (%rdi), %ax +1: + movw %si, %cx + subw %ax, %cx + lock + cmpxchgw %cx, (%rdi) + jne 1b + movzwl %cx, %eax + ret + SET_SIZE(atomic_sub_short_nv) + SET_SIZE(atomic_sub_16_nv) + + ENTRY(atomic_sub_32_nv) + ALTENTRY(atomic_sub_int_nv) + movl (%rdi), %eax +1: + movl %esi, %ecx + subl %eax, %ecx + lock + cmpxchgl %ecx, (%rdi) + jne 1b + movl %ecx, %eax + ret + SET_SIZE(atomic_sub_int_nv) + SET_SIZE(atomic_sub_32_nv) + + ENTRY(atomic_sub_64_nv) + ALTENTRY(atomic_sub_ptr_nv) + ALTENTRY(atomic_sub_long_nv) + movq (%rdi), %rax +1: + movq %rsi, %rcx + subq %rax, %rcx + lock + cmpxchgq %rcx, (%rdi) + jne 1b + movq %rcx, %rax + ret + SET_SIZE(atomic_sub_long_nv) + SET_SIZE(atomic_sub_ptr_nv) + SET_SIZE(atomic_sub_64_nv) + + ENTRY(atomic_and_8_nv) + ALTENTRY(atomic_and_uchar_nv) + movb (%rdi), %al +1: + movb %sil, %cl + andb %al, %cl + lock + cmpxchgb %cl, (%rdi) + jne 1b + movzbl %cl, %eax + ret + SET_SIZE(atomic_and_uchar_nv) + SET_SIZE(atomic_and_8_nv) + + ENTRY(atomic_and_16_nv) + ALTENTRY(atomic_and_ushort_nv) + movw (%rdi), %ax +1: + movw %si, %cx + andw %ax, %cx + lock + cmpxchgw %cx, (%rdi) + jne 1b + movzwl %cx, %eax + ret + SET_SIZE(atomic_and_ushort_nv) + SET_SIZE(atomic_and_16_nv) + + ENTRY(atomic_and_32_nv) + ALTENTRY(atomic_and_uint_nv) + movl (%rdi), %eax +1: + movl %esi, %ecx + andl %eax, %ecx + lock + cmpxchgl %ecx, (%rdi) + jne 1b + movl %ecx, %eax + ret + SET_SIZE(atomic_and_uint_nv) + SET_SIZE(atomic_and_32_nv) + + ENTRY(atomic_and_64_nv) + ALTENTRY(atomic_and_ulong_nv) + movq (%rdi), %rax +1: + movq %rsi, %rcx + andq %rax, %rcx + lock + cmpxchgq %rcx, (%rdi) + jne 1b + movq %rcx, %rax + ret + SET_SIZE(atomic_and_ulong_nv) + SET_SIZE(atomic_and_64_nv) + + ENTRY(atomic_or_8_nv) + ALTENTRY(atomic_or_uchar_nv) + movb (%rdi), %al +1: + movb %sil, %cl + orb %al, %cl + lock + cmpxchgb %cl, (%rdi) + jne 1b + movzbl %cl, %eax + ret + SET_SIZE(atomic_and_uchar_nv) + SET_SIZE(atomic_and_8_nv) + + ENTRY(atomic_or_16_nv) + ALTENTRY(atomic_or_ushort_nv) + movw (%rdi), %ax +1: + movw %si, %cx + orw %ax, %cx + lock + cmpxchgw %cx, (%rdi) + jne 1b + movzwl %cx, %eax + ret + SET_SIZE(atomic_or_ushort_nv) + SET_SIZE(atomic_or_16_nv) + + ENTRY(atomic_or_32_nv) + ALTENTRY(atomic_or_uint_nv) + movl (%rdi), %eax +1: + movl %esi, %ecx + orl %eax, %ecx + lock + cmpxchgl %ecx, (%rdi) + jne 1b + movl %ecx, %eax + ret + SET_SIZE(atomic_or_uint_nv) + SET_SIZE(atomic_or_32_nv) + + ENTRY(atomic_or_64_nv) + ALTENTRY(atomic_or_ulong_nv) + movq (%rdi), %rax +1: + movq %rsi, %rcx + orq %rax, %rcx + lock + cmpxchgq %rcx, (%rdi) + jne 1b + movq %rcx, %rax + ret + SET_SIZE(atomic_or_ulong_nv) + SET_SIZE(atomic_or_64_nv) + + ENTRY(atomic_cas_8) + ALTENTRY(atomic_cas_uchar) + movzbl %sil, %eax + lock + cmpxchgb %dl, (%rdi) + ret + SET_SIZE(atomic_cas_uchar) + SET_SIZE(atomic_cas_8) + + ENTRY(atomic_cas_16) + ALTENTRY(atomic_cas_ushort) + movzwl %si, %eax + lock + cmpxchgw %dx, (%rdi) + ret + SET_SIZE(atomic_cas_ushort) + SET_SIZE(atomic_cas_16) + + ENTRY(atomic_cas_32) + ALTENTRY(atomic_cas_uint) + movl %esi, %eax + lock + cmpxchgl %edx, (%rdi) + ret + SET_SIZE(atomic_cas_uint) + SET_SIZE(atomic_cas_32) + + ENTRY(atomic_cas_64) + ALTENTRY(atomic_cas_ulong) + ALTENTRY(atomic_cas_ptr) + movq %rsi, %rax + lock + cmpxchgq %rdx, (%rdi) + ret + SET_SIZE(atomic_cas_ptr) + SET_SIZE(atomic_cas_ulong) + SET_SIZE(atomic_cas_64) + + ENTRY(atomic_swap_8) + ALTENTRY(atomic_swap_uchar) + movzbl %sil, %eax + lock + xchgb %al, (%rdi) + ret + SET_SIZE(atomic_swap_uchar) + SET_SIZE(atomic_swap_8) + + ENTRY(atomic_swap_16) + ALTENTRY(atomic_swap_ushort) + movzwl %si, %eax + lock + xchgw %ax, (%rdi) + ret + SET_SIZE(atomic_swap_ushort) + SET_SIZE(atomic_swap_16) + + ENTRY(atomic_swap_32) + ALTENTRY(atomic_swap_uint) + movl %esi, %eax + lock + xchgl %eax, (%rdi) + ret + SET_SIZE(atomic_swap_uint) + SET_SIZE(atomic_swap_32) + + ENTRY(atomic_swap_64) + ALTENTRY(atomic_swap_ulong) + ALTENTRY(atomic_swap_ptr) + movq %rsi, %rax + lock + xchgq %rax, (%rdi) + ret + SET_SIZE(atomic_swap_ptr) + SET_SIZE(atomic_swap_ulong) + SET_SIZE(atomic_swap_64) + + ENTRY(atomic_set_long_excl) + xorl %eax, %eax + lock + btsq %rsi, (%rdi) + jnc 1f + decl %eax +1: + ret + SET_SIZE(atomic_set_long_excl) + + ENTRY(atomic_clear_long_excl) + xorl %eax, %eax + lock + btrq %rsi, (%rdi) + jc 1f + decl %eax +1: + ret + SET_SIZE(atomic_clear_long_excl) + + /* + * NOTE: membar_enter, and membar_exit are identical routines. + * We define them separately, instead of using an ALTENTRY + * definitions to alias them together, so that DTrace and + * debuggers will see a unique address for them, allowing + * more accurate tracing. + */ + + ENTRY(membar_enter) + mfence + ret + SET_SIZE(membar_enter) + + ENTRY(membar_exit) + mfence + ret + SET_SIZE(membar_exit) + + ENTRY(membar_producer) + sfence + ret + SET_SIZE(membar_producer) + + ENTRY(membar_consumer) + lfence + ret + SET_SIZE(membar_consumer) + +#ifdef __ELF__ +.section .note.GNU-stack,"",%progbits +#endif diff --git a/lib/libspl/include/os/macos/string.h b/lib/libspl/include/os/macos/string.h index c094d836f377..b730921f7738 100644 --- a/lib/libspl/include/os/macos/string.h +++ b/lib/libspl/include/os/macos/string.h @@ -37,4 +37,6 @@ spl_strlcpy(char *__dst, const char *__source, size_t __size) #undef strlcpy #define strlcpy spl_strlcpy +#define strerror_l(X, Y) strerror(X) + #endif /* _LIBSPL_OSX_STRING_H */ diff --git a/lib/libspl/include/os/macos/sys/kernel_types.h b/lib/libspl/include/os/macos/sys/kernel_types.h index 5796351a2024..84e5d0f24a7b 100644 --- a/lib/libspl/include/os/macos/sys/kernel_types.h +++ b/lib/libspl/include/os/macos/sys/kernel_types.h @@ -23,6 +23,8 @@ #ifndef LIBSPL_SYS_KERNEL_TYPES_H #define LIBSPL_SYS_KERNEL_TYPES_H +#define _STRUCT_TIMEVAL32 + /* * Unfortunately, XNU defines uio_t, proc_t and vnode_t differently to * ZFS, so we need to hack around it. diff --git a/lib/libspl/include/os/macos/unistd.h b/lib/libspl/include/os/macos/unistd.h index b9a9e8e16ba2..190c053b9cd1 100644 --- a/lib/libspl/include/os/macos/unistd.h +++ b/lib/libspl/include/os/macos/unistd.h @@ -91,4 +91,43 @@ setproctitle(const char *fmt, ...) (void) fmt; } +#include +#include +inline static ssize_t +copy_file_range(int sfd, loff_t *soff, int dfd, loff_t *doff, + size_t len, unsigned int flags) +{ + (void) len; + (void) flags; + /* + * int fcopyfile(int from, int to, copyfile_state_t state, + * copyfile_flags_t flags); + * + * Does not handle `len`, nor does not update `soff/doff`, + * ignores `flags`. + */ + if (soff && *soff != 0) + if (lseek(sfd, *soff, SEEK_SET) == -1) + return (-1); + if (doff && *doff != 0) + if (lseek(dfd, *doff, SEEK_SET) == -1) + return (-1); + + copyfile_state_t state = copyfile_state_alloc(); + copyfile_flags_t fl = COPYFILE_ALL; + + int result = fcopyfile(sfd, dfd, state, fl); + + off_t bytes_copied = 0; + copyfile_state_get(state, COPYFILE_STATE_COPIED, &bytes_copied); + + copyfile_state_free(state); + + if (result == -1) + return (-1); + + /* Return number of bytes copied */ + return (bytes_copied); +} + #endif diff --git a/lib/libzfs/Makefile.am b/lib/libzfs/Makefile.am index 7d5518045c76..8cb58b69d1ef 100644 --- a/lib/libzfs/Makefile.am +++ b/lib/libzfs/Makefile.am @@ -67,7 +67,10 @@ libzfs_la_LIBADD = \ libzutil.la \ libuutil.la -libzfs_la_LIBADD += -lrt -lm $(LIBCRYPTO_LIBS) $(ZLIB_LIBS) $(LIBFETCH_LIBS) $(LTLIBINTL) +libzfs_la_LIBADD += -lm $(LIBCRYPTO_LIBS) $(ZLIB_LIBS) $(LIBFETCH_LIBS) $(LTLIBINTL) +if !BUILD_MACOS +libzfs_la_LIBADD += -lrt +endif libzfs_la_LDFLAGS = -pthread diff --git a/lib/libzutil/os/macos/zutil_import_os.c b/lib/libzutil/os/macos/zutil_import_os.c index d991f278d72f..f49e6f254c4d 100644 --- a/lib/libzutil/os/macos/zutil_import_os.c +++ b/lib/libzutil/os/macos/zutil_import_os.c @@ -624,8 +624,24 @@ update_vdev_config_dev_strs(nvlist_t *nv) } } +void +update_vdev_config_dev_sysfs_path(nvlist_t *nv, const char *path, + const char *key) +{ + (void) nv; + (void) path; + (void) key; +} + void update_vdevs_config_dev_sysfs_path(nvlist_t *config) { (void) config; } + +int +zpool_disk_wait(const char *path) +{ + (void) path; + return (ENOTSUP); +} diff --git a/module/icp/asm-x86_64/modes/aesni-gcm-x86_64.S b/module/icp/asm-x86_64/modes/aesni-gcm-x86_64.S index 909b2147dff9..a3f8ed5722bb 100644 --- a/module/icp/asm-x86_64/modes/aesni-gcm-x86_64.S +++ b/module/icp/asm-x86_64/modes/aesni-gcm-x86_64.S @@ -56,6 +56,13 @@ /* Apple needs _ */ #if defined (__APPLE__) #define gcm_avx_can_use_movbe _gcm_avx_can_use_movbe + +// There is a bug in Xcode 16.0 +#if defined(__APPLE__) && (__clang_major__ == 16) && (__clang_minor__ == 0) +#define cfi_restore set ignore, +#define cfi_def_cfa_register set ignore, +#endif + #endif .extern gcm_avx_can_use_movbe diff --git a/module/icp/asm-x86_64/sha2/sha256-x86_64.S b/module/icp/asm-x86_64/sha2/sha256-x86_64.S index d3e5e3f0d080..db9b831ac1cd 100644 --- a/module/icp/asm-x86_64/sha2/sha256-x86_64.S +++ b/module/icp/asm-x86_64/sha2/sha256-x86_64.S @@ -24,6 +24,16 @@ #define _ASM #include +#if defined(__APPLE__) && (__clang_major__ == 16) && (__clang_minor__ == 0) +#pragma message("yes") +// There is a bug in Xcode 16.0 +// let sadd #if version == here? +#define cfi_restore set ignore, +#define cfi_def_cfa_register set ignore, +#define cfi_def_cfa set ignore,nothing # +#define cfi_escape set ignore,nothing # +#endif + SECTION_STATIC .balign 64 diff --git a/module/icp/asm-x86_64/sha2/sha512-x86_64.S b/module/icp/asm-x86_64/sha2/sha512-x86_64.S index fbbcca650d10..e4cab031ff50 100644 --- a/module/icp/asm-x86_64/sha2/sha512-x86_64.S +++ b/module/icp/asm-x86_64/sha2/sha512-x86_64.S @@ -21,6 +21,15 @@ #if defined(__x86_64) +#if defined(__APPLE__) && (__clang_major__ == 16) && (__clang_minor__ == 0) +// There is a bug in Xcode 16.0 +// let sadd #if version == here? +#define cfi_restore set ignore, +#define cfi_def_cfa_register set ignore, +#define cfi_def_cfa set ignore,nothing # +#define cfi_escape set ignore,nothing # +#endif + #define _ASM #include diff --git a/module/os/macos/Makefile.am b/module/os/macos/Makefile.am index 1ae3686bfac1..9d4d6e32cd8b 100644 --- a/module/os/macos/Makefile.am +++ b/module/os/macos/Makefile.am @@ -151,9 +151,12 @@ zfs_common_SRCS = \ %D%/../../zfs/dbuf.c \ %D%/../../zfs/dbuf_stats.c \ %D%/../../zfs/ddt.c \ + %D%/../../zfs/ddt_log.c \ + %D%/../../zfs/ddt_stats.c \ %D%/../../zfs/ddt_zap.c \ %D%/../../zfs/dmu.c \ %D%/../../zfs/dmu_diff.c \ + %D%/../../zfs/dmu_direct.c \ %D%/../../zfs/dmu_object.c \ %D%/../../zfs/dmu_objset.c \ %D%/../../zfs/dmu_recv.c \ @@ -249,6 +252,7 @@ zfs_common_SRCS = \ %D%/../../zfs/zfs_rlock.c \ %D%/../../zfs/zfs_sa.c \ %D%/../../zfs/zfs_vnops.c \ + %D%/../../zfs/zfs_znode.c \ %D%/../../zfs/zil.c \ %D%/../../zfs/zio.c \ %D%/../../zfs/zio_checksum.c \ @@ -269,6 +273,7 @@ zfs_os_SRCS = \ %D%/zfs/sysctl_os.c \ %D%/zfs/vdev_disk.c \ %D%/zfs/vdev_file.c \ + %D%/zfs/vdev_label_os.c \ %D%/zfs/zfs_acl.c \ %D%/zfs/zfs_boot.cpp \ %D%/zfs/zfs_ctldir.c \ @@ -284,7 +289,7 @@ zfs_os_SRCS = \ %D%/zfs/zfs_vnops_osx.c \ %D%/zfs/zfs_vnops_osx_lib.c \ %D%/zfs/zfs_vnops_osx_xattr.c \ - %D%/zfs/zfs_znode.c \ + %D%/zfs/zfs_znode_os.c \ %D%/zfs/zio_crypt.c \ %D%/zfs/zvol_os.c \ %D%/zfs/zvolIO.cpp \ @@ -299,7 +304,6 @@ zcommon_SRCS = \ %D%/../../nvpair/nvpair_alloc_fixed.c \ %D%/../../nvpair/nvpair_alloc_spl.c \ %D%/../../unicode/u8_textprep.c \ - %D%/../../unicode/uconv.c \ %D%/../../zcommon/zfs_comutil.c \ %D%/../../zcommon/zfs_deleg.c \ %D%/../../zcommon/zfs_fletcher.c \ @@ -322,7 +326,6 @@ icp_SRCS = \ %D%/../../icp/spi/kcf_spi.c \ %D%/../../icp/io/aes.c \ %D%/../../icp/io/sha2_mod.c \ - %D%/../../icp/io/skein_mod.c \ %D%/../../icp/algs/aes/aes_impl_aesni.c \ %D%/../../icp/algs/aes/aes_impl_aesv8.c \ %D%/../../icp/algs/aes/aes_impl_generic.c \ @@ -333,10 +336,7 @@ icp_SRCS = \ %D%/../../icp/algs/blake3/blake3_generic.c \ %D%/../../icp/algs/blake3/blake3_impl.c \ %D%/../../icp/algs/edonr/edonr.c \ - %D%/../../icp/algs/modes/cbc.c \ %D%/../../icp/algs/modes/ccm.c \ - %D%/../../icp/algs/modes/ctr.c \ - %D%/../../icp/algs/modes/ecb.c \ %D%/../../icp/algs/modes/gcm_generic.c \ %D%/../../icp/algs/modes/gcm.c \ %D%/../../icp/algs/modes/modes.c \ diff --git a/module/os/macos/spl/spl-err.c b/module/os/macos/spl/spl-err.c index 455bf2c8b9d6..ac7e2309100c 100644 --- a/module/os/macos/spl/spl-err.c +++ b/module/os/macos/spl/spl-err.c @@ -29,6 +29,8 @@ #include #include +void IOSleep(int); + void vcmn_err(int ce, const char *fmt, va_list ap) { @@ -49,6 +51,8 @@ vcmn_err(int ce, const char *fmt, va_list ap) printf("SPL: Warning: %s\n", msg); break; case CE_PANIC: + printf("PANIC: %s\n", msg); + IOSleep(4000); PANIC("%s", msg); break; } diff --git a/module/os/macos/spl/spl-osx.c b/module/os/macos/spl/spl-osx.c index b2bada924835..c7ddfe1988e1 100644 --- a/module/os/macos/spl/spl-osx.c +++ b/module/os/macos/spl/spl-osx.c @@ -49,6 +49,7 @@ static utsname_t utsname_static = { { 0 } }; unsigned int max_ncpus = 0; +unsigned int boot_ncpus = 0; unsigned int num_ecores = 0; uint64_t total_memory = 0; uint64_t real_total_memory = 0; @@ -70,9 +71,9 @@ utsname(void) * 1 HZ is 10 milliseconds */ void -osx_delay(int ticks) +osx_delay(clock_t ticks) { - ASSERT3S(ticks, >, 0); + ASSERT3U(ticks, <, SEC_TO_TICK(60)); // ticks are 10 msec units int64_t ticks_to_go = (int64_t)ticks; @@ -108,7 +109,7 @@ osx_delay(int ticks) bool forced_sleep = false; - ASSERT3S(ticks_to_go, >, 0); + ASSERT3U(ticks_to_go, <, SEC_TO_TICK(60)); unsigned milliseconds_remaining = ticks_to_go * 10; if (milliseconds_remaining < 2) { @@ -495,6 +496,9 @@ spl_start(kmod_info_t *ki, void *d) #if defined(__arm64__) num_ecores = (max_ncpus > 4) ? 4 : 0; + boot_ncpus = MAX(1, (int)max_ncpus - (int)num_ecores); +#else + boot_ncpus = max_ncpus; #endif /* @@ -571,7 +575,7 @@ spl_start(kmod_info_t *ki, void *d) spl_kmem_thread_init(); spl_kmem_mp_init(); - spl_cpuid_features(); + spl_processor_init(); return (KERN_SUCCESS); } diff --git a/module/os/macos/spl/spl-proc.c b/module/os/macos/spl/spl-proc.c index 9c90559f0fba..ebb0d408da02 100644 --- a/module/os/macos/spl/spl-proc.c +++ b/module/os/macos/spl/spl-proc.c @@ -28,3 +28,11 @@ struct proc { }; struct proc p0 = {0}; + + +int +issig(int why) +{ + return (thread_issignal(current_proc(), current_thread(), + THREADMASK)); +} diff --git a/module/os/macos/spl/spl-processor.c b/module/os/macos/spl/spl-processor.c index 9b19448f47d9..278281c4f0f5 100644 --- a/module/os/macos/spl/spl-processor.c +++ b/module/os/macos/spl/spl-processor.c @@ -26,25 +26,14 @@ */ #include +#include extern int cpu_number(void); #if defined(__x86_64__) #include - -#define _spl_cpuid(func, a, b, c, d) \ - __asm__ __volatile__( \ - " pushq %%rbx \n" \ - " xorq %%rcx,%%rcx \n" \ - " cpuid \n" \ - " movq %%rbx, %%rsi \n" \ - " popq %%rbx \n" : \ - "=a" (a), "=S" (b), "=c" (c), "=d" (d) : "a" (func)) - -static uint64_t _spl_cpuid_features = 0ULL; -static uint64_t _spl_cpuid_features_leaf7 = 0ULL; -static boolean_t _spl_cpuid_has_xgetbv = B_FALSE; +#include #elif defined(__aarch64__) @@ -85,53 +74,6 @@ getcpuid(void) #endif } - -#if defined(__x86_64__) -uint64_t -spl_cpuid_features(void) -{ - static int first_time = 1; - uint64_t a, b, c, d; - - if (first_time == 1) { - first_time = 0; - // Wikipedia: stored in EAX, EBX, EDX, ECX (in that order). - _spl_cpuid(0, a, b, d, c); - if (a >= 1) { - _spl_cpuid(1, a, b, d, c); - _spl_cpuid_features = d | (c << 32); - - // GETBV is bit 26 in ECX. Apple defines it as: - // CPUID_FEATURE_XSAVE _HBit(26) - // ( ECX & (1 << 26) - // or, (feature & 400000000000000) - _spl_cpuid_has_xgetbv = - _spl_cpuid_features & CPUID_FEATURE_XSAVE; - } - if (a >= 7) { - c = 0; - _spl_cpuid(7, a, b, d, c); - _spl_cpuid_features_leaf7 = b | (c << 32); - } - - - printf("SPL: CPUID 0x%08llx and leaf7 0x%08llx\n", - _spl_cpuid_features, _spl_cpuid_features_leaf7); - - } - - return (_spl_cpuid_features); -} - -uint64_t -spl_cpuid_leaf7_features(void) -{ - return (_spl_cpuid_features_leaf7); -} - -#endif /* x86_64 */ - - #if defined(__aarch64__) /* 4,5,6,7 -> GET_BITS(4, 8) */ @@ -202,12 +144,35 @@ spl_cpuid_id_aa64isar1_el1(void) return (_spl_cpuid_id_aa64isar1_el1); } -uint64_t -spl_cpuid_features(void) +#endif /* aarch64 */ + +int +spl_processor_init(void) { + +#if defined(__aarch64__) spl_cpuid_id_aa64isar0_el1(); spl_cpuid_id_aa64isar1_el1(); - return (0ULL); -} +#endif -#endif /* aarch64 */ +#if defined(__x86) + printf("CPUID: %s%s%s%s%s%s%s\n", + zfs_osxsave_available() ? "osxsave " : "", + zfs_sse_available() ? "sse " : "", + zfs_sse2_available() ? "sse2 " : "", + zfs_sse3_available() ? "sse3 " : "", + zfs_ssse3_available() ? "ssse3 " : "", + zfs_sse4_1_available() ? "sse4.1 " : "", + zfs_sse4_2_available() ? "sse4.2 " : ""); + printf("CPUID: %s%s%s%s%s%s%s\n", + zfs_avx_available() ? "avx " : "", + zfs_avx2_available() ? "avx2 " : "", + zfs_aes_available() ? "aes " : "", + zfs_pclmulqdq_available() ? "pclmulqdq " : "", + zfs_avx512f_available() ? "avx512f " : "", + zfs_movbe_available() ? "movbe " : "", + zfs_shani_available() ? "sha-ni " : ""); +#endif + + return (0); +} diff --git a/module/os/macos/spl/spl-seg_kmem.c b/module/os/macos/spl/spl-seg_kmem.c index 69aaa6c9e37d..b9f8b290edc9 100644 --- a/module/os/macos/spl/spl-seg_kmem.c +++ b/module/os/macos/spl/spl-seg_kmem.c @@ -281,31 +281,27 @@ segkmem_abd_init() /* * OpenZFS does not segregate the abd kmem cache out of the general * heap, leading to large numbers of short-lived slabs exchanged - * between the kmem cache and it's parent. XNU absorbs this with a - * qcache, following its history of absorbing the pre-ABD zio file and - * metadata caches being qcached (which raises the exchanges with the - * general heap from PAGESIZE to 256k). + * between the kmem cache and it's parent. XNU absorbs this with a a + * large minimum request to the parent vmem_caches on large-memory + * MacOS systems. */ extern vmem_t *spl_heap_arena; -#define BIG_SLAB 131072 -#ifdef __arm64__ -#define BIG_BIG_SLAB (BIG_SLAB * 2) -#else -#define BIG_BIG_SLAB BIG_SLAB -#endif +#define BIG_SLAB (PAGESIZE * 16) #define SMALL_RAM_MACHINE (4ULL * 1024ULL * 1024ULL * 1024ULL) if (total_memory >= SMALL_RAM_MACHINE) { abd_arena = vmem_create("abd_cache", NULL, 0, - PAGESIZE, vmem_alloc_impl, vmem_free_impl, spl_heap_arena, - BIG_BIG_SLAB, VM_SLEEP | VMC_NO_QCACHE); + sizeof (void *), + vmem_alloc_impl, vmem_free_impl, spl_heap_arena, + BIG_SLAB, VM_SLEEP | VMC_NO_QCACHE); } else { abd_arena = vmem_create("abd_cache", NULL, 0, - PAGESIZE, vmem_alloc_impl, vmem_free_impl, spl_heap_arena, - 131072, VM_SLEEP | VMC_NO_QCACHE); + sizeof (void *), + vmem_alloc_impl, vmem_free_impl, spl_heap_arena, + PAGESIZE, VM_SLEEP | VMC_NO_QCACHE); } VERIFY3P(abd_arena, !=, NULL); @@ -322,13 +318,15 @@ segkmem_abd_init() if (total_memory >= SMALL_RAM_MACHINE) { abd_subpage_arena = vmem_create("abd_subpage_cache", NULL, 0, - sizeof (void *), vmem_alloc_impl, vmem_free_impl, + sizeof (void *), + vmem_alloc_impl, vmem_free_impl, spl_heap_arena, BIG_SLAB, VM_SLEEP | VMC_NO_QCACHE); } else { abd_subpage_arena = vmem_create("abd_subpage_cache", NULL, 0, - 512, vmem_alloc_impl, vmem_free_impl, abd_arena, - 131072, VM_SLEEP | VMC_NO_QCACHE); + sizeof (void *), + vmem_alloc_impl, vmem_free_impl, abd_arena, + PAGESIZE, VM_SLEEP | VMC_NO_QCACHE); } VERIFY3P(abd_subpage_arena, !=, NULL); diff --git a/module/os/macos/spl/spl-taskq.c b/module/os/macos/spl/spl-taskq.c index 3b58d3335c97..569c65fba942 100644 --- a/module/os/macos/spl/spl-taskq.c +++ b/module/os/macos/spl/spl-taskq.c @@ -1758,9 +1758,9 @@ taskq_resume(taskq_t *tq) } int -taskq_member(taskq_t *tq, kthread_t *thread) +taskq_member(taskq_t *tq, kthread_t *xthread) { - return (tq == (taskq_t *)tsd_get_by_thread(taskq_tsd, thread)); + return (tq == (taskq_t *)tsd_get_by_thread(taskq_tsd, xthread)); } taskq_t * @@ -2401,7 +2401,9 @@ taskq_create_common(const char *name, int instance, int nthreads, pri_t pri, { taskq_t *tq = kmem_cache_alloc(taskq_cache, KM_SLEEP); #ifdef __APPLE__ - uint_t ncpus = max_ncpus - num_ecores; + uint_t ncpus = max_ncpus; + if (!(flags & TASKQ_CREATE_SYNCED)) + ncpus = boot_ncpus; /* possibly deflated by num_ecores */ #else uint_t ncpus = ((boot_max_ncpus == -1) ? max_ncpus : boot_max_ncpus); #endif @@ -2822,6 +2824,98 @@ taskq_bucket_extend(void *arg) #endif /* __APPLE__ */ } +typedef struct taskq_sync_arg { + kthread_t *tqa_thread; + kcondvar_t tqa_cv; + kmutex_t tqa_lock; + int tqa_ready; +} taskq_sync_arg_t; + +static void +taskq_sync_assign(void *arg) +{ + taskq_sync_arg_t *tqa = arg; + + mutex_enter(&tqa->tqa_lock); + tqa->tqa_thread = curthread; + tqa->tqa_ready = 1; + cv_signal(&tqa->tqa_cv); + while (tqa->tqa_ready == 1) + cv_wait(&tqa->tqa_cv, &tqa->tqa_lock); + mutex_exit(&tqa->tqa_lock); +} + +/* + * Create a taskq with a specified number of pool threads. Allocate + * and return an array of nthreads kthread_t pointers, one for each + * thread in the pool. The array is not ordered and must be freed + * by the caller. + */ +taskq_t * +taskq_create_synced(const char *name, int nthreads, pri_t pri, + int minalloc, int maxalloc, uint_t flags, kthread_t ***ktpp) +{ + taskq_t *tq; + taskq_sync_arg_t *tqs = kmem_zalloc(sizeof (*tqs) * nthreads, KM_SLEEP); + kthread_t **kthreads = kmem_zalloc(sizeof (*kthreads) * nthreads, + KM_SLEEP); + + flags &= ~(TASKQ_DYNAMIC | TASKQ_THREADS_CPU_PCT | TASKQ_DC_BATCH); + + tq = taskq_create(name, nthreads, minclsyspri, nthreads, INT_MAX, + flags | TASKQ_PREPOPULATE | TASKQ_CREATE_SYNCED); + + VERIFY(tq != NULL); + + /* wait until our minalloc (nthreads) threads are created */ + mutex_enter(&tq->tq_lock); + for (int i = 1; tq->tq_nthreads != nthreads; i++) { + printf("SPL: %s:%d: waiting for tq_nthreads (%d)" + " to be nthreads (%d), (target = %d, pass %d)\n", + __func__, __LINE__, + tq->tq_nthreads, tq->tq_nthreads_target, nthreads, i); + cv_wait(&tq->tq_wait_cv, &tq->tq_lock); + } + mutex_exit(&tq->tq_lock); + + VERIFY3U(tq->tq_nthreads, ==, nthreads); + + /* spawn all syncthreads */ + for (int i = 0; i < nthreads; i++) { + cv_init(&tqs[i].tqa_cv, NULL, CV_DEFAULT, NULL); + mutex_init(&tqs[i].tqa_lock, NULL, MUTEX_DEFAULT, NULL); + (void) taskq_dispatch(tq, taskq_sync_assign, + &tqs[i], TQ_FRONT); + } + + /* wait on all syncthreads to start */ + for (int i = 0; i < nthreads; i++) { + mutex_enter(&tqs[i].tqa_lock); + while (tqs[i].tqa_ready == 0) + cv_wait(&tqs[i].tqa_cv, &tqs[i].tqa_lock); + mutex_exit(&tqs[i].tqa_lock); + } + + /* let all syncthreads resume, finish */ + for (int i = 0; i < nthreads; i++) { + mutex_enter(&tqs[i].tqa_lock); + tqs[i].tqa_ready = 2; + cv_broadcast(&tqs[i].tqa_cv); + mutex_exit(&tqs[i].tqa_lock); + } + taskq_wait(tq); + + for (int i = 0; i < nthreads; i++) { + kthreads[i] = tqs[i].tqa_thread; + mutex_destroy(&tqs[i].tqa_lock); + cv_destroy(&tqs[i].tqa_cv); + } + kmem_free(tqs, sizeof (*tqs) * nthreads); + + *ktpp = kthreads; + return (tq); +} + static int taskq_kstat_update(kstat_t *ksp, int rw) { diff --git a/module/os/macos/spl/spl-uio.c b/module/os/macos/spl/spl-uio.c index b19c949f9ebd..6f632ede3c1e 100644 --- a/module/os/macos/spl/spl-uio.c +++ b/module/os/macos/spl/spl-uio.c @@ -146,3 +146,39 @@ zfs_uio_prefaultpages(ssize_t n, zfs_uio_t *uio) { return (0); } + +/* + * Check if the uio is page-aligned in memory. + */ +boolean_t +zfs_uio_page_aligned(zfs_uio_t *uio) +{ + for (int i = zfs_uio_iovcnt(uio); i > 0; i--) { + uintptr_t addr = (uintptr_t)zfs_uio_iovbase(uio, i); + size_t size = zfs_uio_iovlen(uio, i); + if ((addr & (PAGE_SIZE - 1)) || (size & (PAGE_SIZE - 1))) { + return (B_FALSE); + } + } + + return (B_TRUE); +} + +void +zfs_uio_free_dio_pages(zfs_uio_t *uio, zfs_uio_rw_t rw) +{ +#if 0 + ASSERT(uio->uio_extflg & UIO_DIRECT); + ASSERT3P(uio->uio_dio.pages, !=, NULL); + ASSERT(zfs_uio_rw(uio) == rw); + + if (rw == UIO_WRITE) + zfs_uio_release_stable_pages(uio); + + vm_page_unhold_pages(&uio->uio_dio.pages[0], + uio->uio_dio.npages); + + kmem_free(uio->uio_dio.pages, + uio->uio_dio.npages * sizeof (vm_page_t)); +#endif +} diff --git a/module/os/macos/zfs/abd_os.c b/module/os/macos/zfs/abd_os.c index 952e8573666d..be63e56e665f 100644 --- a/module/os/macos/zfs/abd_os.c +++ b/module/os/macos/zfs/abd_os.c @@ -30,6 +30,7 @@ #include #include +#include #include #include #include @@ -136,22 +137,36 @@ kmem_cache_t *abd_subpage_cache[SUBPAGE_CACHE_INDICES] = { NULL }; abd_t *abd_zero_scatter = NULL; static char *abd_zero_buf = NULL; -static void -abd_free_chunk(void *c) -{ - kmem_cache_free(abd_chunk_cache, c); -} +/* + * zfs_abd_scatter_min_size is the minimum allocation size to use scatter + * ABD's for. Smaller allocations will use linear ABD's which use + * zio_[data_]buf_alloc(). + * + * Scatter ABD's use at least one page each, so sub-page allocations waste + * some space when allocated as scatter (e.g. 2KB scatter allocation wastes + * half of each page). Using linear ABD's for small allocations means that + * they will be put on slabs which contain many allocations. + * + * Linear ABDs for multi-page allocations are easier to use, and in some cases + * it allows to avoid buffer copying. But allocation and especially free + * of multi-page linear ABDs are expensive operations due to KVA mapping and + * unmapping, and with time they cause KVA fragmentations. + */ +static size_t zfs_abd_scatter_min_size = ABD_PGSIZE + 1; -static inline size_t +kmem_cache_t *abd_chunk_cache; +static kstat_t *abd_ksp; + +static uint_t abd_chunkcnt_for_bytes(size_t size) { - return (P2ROUNDUP(size, zfs_abd_chunk_size) / zfs_abd_chunk_size); + return ((size + PAGE_MASK) >> PAGE_SHIFT); } -static size_t +static inline uint_t abd_scatter_chunkcnt(abd_t *abd) { - VERIFY(!abd_is_linear(abd)); + ASSERT(!abd_is_linear(abd)); return (abd_chunkcnt_for_bytes( ABD_SCATTER(abd).abd_offset + abd->abd_size)); } @@ -159,24 +174,25 @@ abd_scatter_chunkcnt(abd_t *abd) boolean_t abd_size_alloc_linear(size_t size) { - return (B_FALSE); + return (!zfs_abd_scatter_enabled || size < zfs_abd_scatter_min_size); } void abd_update_scatter_stats(abd_t *abd, abd_stats_op_t op) { - size_t n = abd_scatter_chunkcnt(abd); + uint_t n = abd_scatter_chunkcnt(abd); ASSERT(op == ABDSTAT_INCR || op == ABDSTAT_DECR); + int waste = (n << PAGE_SHIFT) - abd->abd_size; if (op == ABDSTAT_INCR) { ABDSTAT_BUMP(abdstat_scatter_cnt); ABDSTAT_INCR(abdstat_scatter_data_size, abd->abd_size); - ABDSTAT_INCR(abdstat_scatter_chunk_waste, - n * ABD_SCATTER(abd).abd_chunk_size - abd->abd_size); + ABDSTAT_INCR(abdstat_scatter_chunk_waste, waste); + arc_space_consume(waste, ARC_SPACE_ABD_CHUNK_WASTE); } else { ABDSTAT_BUMPDOWN(abdstat_scatter_cnt); ABDSTAT_INCR(abdstat_scatter_data_size, -(int)abd->abd_size); - ABDSTAT_INCR(abdstat_scatter_chunk_waste, - abd->abd_size - n * ABD_SCATTER(abd).abd_chunk_size); + ABDSTAT_INCR(abdstat_scatter_chunk_waste, -waste); + arc_space_return(waste, ARC_SPACE_ABD_CHUNK_WASTE); } } @@ -196,101 +212,49 @@ abd_update_linear_stats(abd_t *abd, abd_stats_op_t op) void abd_verify_scatter(abd_t *abd) { + uint_t i, n; + /* - * There is no scatter linear pages in FreeBSD so there is an - * if an error if the ABD has been marked as a linear page. + * There is no scatter linear pages in FreeBSD so there is + * an error if the ABD has been marked as a linear page. */ - VERIFY(!abd_is_linear_page(abd)); - VERIFY3U(ABD_SCATTER(abd).abd_offset, <, - zfs_abd_chunk_size); - VERIFY3U(ABD_SCATTER(abd).abd_offset, <, - ABD_SCATTER(abd).abd_chunk_size); - VERIFY3U(ABD_SCATTER(abd).abd_chunk_size, >=, - SPA_MINBLOCKSIZE); - - size_t n = abd_scatter_chunkcnt(abd); - - if (ABD_SCATTER(abd).abd_chunk_size != ABD_PGSIZE) { - VERIFY3U(n, ==, 1); - VERIFY3U(ABD_SCATTER(abd).abd_chunk_size, <, ABD_PGSIZE); - VERIFY3U(abd->abd_size, <=, ABD_SCATTER(abd).abd_chunk_size); - } + ASSERT(!abd_is_linear_page(abd)); + ASSERT3U(ABD_SCATTER(abd).abd_offset, <, ABD_PGSIZE); + n = abd_scatter_chunkcnt(abd); - for (int i = 0; i < n; i++) { - VERIFY3P( - ABD_SCATTER(abd).abd_chunks[i], !=, NULL); + for (i = 0; i < n; i++) { + ASSERT3P(ABD_SCATTER(abd).abd_chunks[i], !=, NULL); } } -static inline int -abd_subpage_cache_index(const size_t size) -{ - const int idx = size >> SPA_MINBLOCKSHIFT; - - if ((size % SPA_MINBLOCKSIZE) == 0) - return (idx - 1); - else - return (idx); -} - -static inline uint_t -abd_subpage_enclosing_size(const int i) -{ - return (SPA_MINBLOCKSIZE * (i + 1)); -} - void abd_alloc_chunks(abd_t *abd, size_t size) { - VERIFY3U(size, >, 0); - if (size <= (zfs_abd_chunk_size - SPA_MINBLOCKSIZE)) { - const int i = abd_subpage_cache_index(size); - VERIFY3S(i, >=, 0); - VERIFY3S(i, <, SUBPAGE_CACHE_INDICES); - const uint_t s = abd_subpage_enclosing_size(i); - VERIFY3U(s, >=, size); - VERIFY3U(s, <, zfs_abd_chunk_size); - void *c = kmem_cache_alloc(abd_subpage_cache[i], KM_SLEEP); - ABD_SCATTER(abd).abd_chunks[0] = c; - ABD_SCATTER(abd).abd_chunk_size = s; - } else { - const size_t n = abd_chunkcnt_for_bytes(size); + uint_t i, n; - for (int i = 0; i < n; i++) { - void *c = kmem_cache_alloc(abd_chunk_cache, KM_SLEEP); - ABD_SCATTER(abd).abd_chunks[i] = c; - } - ABD_SCATTER(abd).abd_chunk_size = zfs_abd_chunk_size; + n = abd_chunkcnt_for_bytes(size); + for (i = 0; i < n; i++) { + ABD_SCATTER(abd).abd_chunks[i] = + kmem_cache_alloc(abd_chunk_cache, KM_SLEEP); } } void abd_free_chunks(abd_t *abd) { - const uint_t abd_cs = ABD_SCATTER(abd).abd_chunk_size; - - if (abd_cs <= (zfs_abd_chunk_size - SPA_MINBLOCKSIZE)) { - VERIFY3U(abd->abd_size, <, zfs_abd_chunk_size); - VERIFY0(P2PHASE(abd_cs, SPA_MINBLOCKSIZE)); + uint_t i, n; - const int idx = abd_subpage_cache_index(abd_cs); - VERIFY3S(idx, >=, 0); - VERIFY3S(idx, <, SUBPAGE_CACHE_INDICES); - - kmem_cache_free(abd_subpage_cache[idx], - ABD_SCATTER(abd).abd_chunks[0]); - } else { - const size_t n = abd_scatter_chunkcnt(abd); - for (int i = 0; i < n; i++) { - abd_free_chunk(ABD_SCATTER(abd).abd_chunks[i]); - } + n = abd_scatter_chunkcnt(abd); + for (i = 0; i < n; i++) { + kmem_cache_free(abd_chunk_cache, + ABD_SCATTER(abd).abd_chunks[i]); } } abd_t * abd_alloc_struct_impl(size_t size) { - size_t chunkcnt = abd_chunkcnt_for_bytes(size); + uint_t chunkcnt = abd_chunkcnt_for_bytes(size); /* * In the event we are allocating a gang ABD, the size passed in * will be 0. We must make sure to set abd_size to the size of an @@ -300,7 +264,9 @@ abd_alloc_struct_impl(size_t size) */ size_t abd_size = MAX(sizeof (abd_t), offsetof(abd_t, abd_u.abd_scatter.abd_chunks[chunkcnt])); - abd_t *abd = kmem_zalloc(abd_size, KM_PUSHPAGE); + abd_t *abd = kmem_alloc(abd_size, KM_SLEEP); + + ASSERT3P(abd, !=, NULL); ABDSTAT_INCR(abdstat_struct_size, abd_size); return (abd); @@ -313,48 +279,49 @@ abd_free_struct_impl(abd_t *abd) abd_scatter_chunkcnt(abd); ssize_t size = MAX(sizeof (abd_t), offsetof(abd_t, abd_u.abd_scatter.abd_chunks[chunkcnt])); - kmem_free(abd, size); ABDSTAT_INCR(abdstat_struct_size, -size); } /* * Allocate scatter ABD of size SPA_MAXBLOCKSIZE, where - * each chunk in the scatterlist will be set to abd_zero_buf. + * each chunk in the scatterlist will be set to the same area. */ static void abd_alloc_zero_scatter(void) { - size_t n = abd_chunkcnt_for_bytes(SPA_MAXBLOCKSIZE); - abd_zero_buf = kmem_cache_alloc(abd_chunk_cache, KM_SLEEP); - memset(abd_zero_buf, 0, zfs_abd_chunk_size); - abd_zero_scatter = abd_alloc_struct(SPA_MAXBLOCKSIZE); + uint_t i, n; - abd_zero_scatter->abd_flags |= ABD_FLAG_OWNER | ABD_FLAG_ZEROS; + abd_zero_buf = kmem_alloc(ABD_PGSIZE, KM_SLEEP); + memset(abd_zero_buf, 0, ABD_PGSIZE); + + n = abd_chunkcnt_for_bytes(SPA_MAXBLOCKSIZE); + abd_zero_scatter = abd_alloc_struct(SPA_MAXBLOCKSIZE); + abd_zero_scatter->abd_flags |= ABD_FLAG_OWNER; abd_zero_scatter->abd_size = SPA_MAXBLOCKSIZE; ABD_SCATTER(abd_zero_scatter).abd_offset = 0; - ABD_SCATTER(abd_zero_scatter).abd_chunk_size = - zfs_abd_chunk_size; - for (int i = 0; i < n; i++) { + for (i = 0; i < n; i++) { ABD_SCATTER(abd_zero_scatter).abd_chunks[i] = - abd_zero_buf; + __DECONST(void *, abd_zero_buf); } ABDSTAT_BUMP(abdstat_scatter_cnt); - ABDSTAT_INCR(abdstat_scatter_data_size, zfs_abd_chunk_size); + ABDSTAT_INCR(abdstat_scatter_data_size, ABD_PGSIZE); } static void abd_free_zero_scatter(void) { ABDSTAT_BUMPDOWN(abdstat_scatter_cnt); - ABDSTAT_INCR(abdstat_scatter_data_size, -(int)zfs_abd_chunk_size); + ABDSTAT_INCR(abdstat_scatter_data_size, -(int)ABD_PGSIZE); abd_free_struct(abd_zero_scatter); abd_zero_scatter = NULL; - kmem_cache_free(abd_chunk_cache, abd_zero_buf); + + kmem_free(abd_zero_buf, ABD_PGSIZE); + abd_zero_buf = NULL; } static int @@ -382,46 +349,8 @@ abd_kstats_update(kstat_t *ksp, int rw) void abd_init(void) { - /* check if we guessed ABD_PGSIZE correctly */ - ASSERT3U(ABD_PGSIZE, ==, PAGE_SIZE); - -#ifdef DEBUG - /* - * KMF_BUFTAG | KMF_LITE on the abd kmem_caches causes them to waste - * up to 50% of their memory for redzone. Even in DEBUG builds this - * therefore should be KMC_NOTOUCH unless there are concerns about - * overruns, UAFs, etc involving abd chunks or subpage chunks. - * - * Additionally these KMF_ - * flags require the definitions from - */ - - /* - * DEBUGGING: do this - * const int cflags = KMF_BUFTAG | KMF_LITE; - * or - * const int cflags = KMC_ARENA_SLAB; - */ - - int cflags = KMC_ARENA_SLAB; -#else - int cflags = KMC_ARENA_SLAB; -#endif - -#ifdef _KERNEL -/* This must all match spl-seg_kmem.c : segkmem_abd_init() */ -#define SMALL_RAM_MACHINE (4ULL * 1024ULL * 1024ULL * 1024ULL) - - extern uint64_t total_memory; - - if (total_memory < SMALL_RAM_MACHINE) { - cflags = KMC_NOTOUCH; - } -#endif - - abd_chunk_cache = kmem_cache_create("abd_chunk", zfs_abd_chunk_size, - ABD_PGSIZE, - NULL, NULL, NULL, NULL, abd_arena, cflags); + abd_chunk_cache = kmem_cache_create("abd_chunk", ABD_PGSIZE, 0, + NULL, NULL, NULL, NULL, 0, KMC_NODEBUG | KMC_RECLAIMABLE); wmsum_init(&abd_sums.abdstat_struct_size, 0); wmsum_init(&abd_sums.abdstat_scatter_cnt, 0); @@ -439,57 +368,11 @@ abd_init(void) } abd_alloc_zero_scatter(); - - /* - * Check at compile time that SPA_MINBLOCKSIZE is 512, because we want - * to build sub-page-size linear ABD kmem caches at multiples of - * SPA_MINBLOCKSIZE. If SPA_MINBLOCKSIZE ever changes, a different - * layout should be calculated at runtime. - * - * See also the assertions above the definition of abd_subpbage_cache. - */ - - _Static_assert(SPA_MINBLOCKSIZE == 512, - "unexpected SPA_MINBLOCKSIZE != 512"); - - const int step_size = SPA_MINBLOCKSIZE; - for (int bytes = step_size; bytes < ABD_PGSIZE; bytes += step_size) { - char name[36]; - - (void) snprintf(name, sizeof (name), - "abd_subpage_%lu", (ulong_t)bytes); - - const int index = (bytes >> SPA_MINBLOCKSHIFT) - 1; - VERIFY3S(index, >=, 0); - VERIFY3S(index, <, SUBPAGE_CACHE_INDICES); - -#ifdef DEBUG - int csubflags = KMF_LITE; -#else - int csubflags = 0; -#endif -#ifdef _KERNEL - if (total_memory < SMALL_RAM_MACHINE) - csubflags = cflags; -#endif - abd_subpage_cache[index] = - kmem_cache_create(name, bytes, sizeof (void *), - NULL, NULL, NULL, NULL, abd_subpage_arena, csubflags); - - VERIFY3P(abd_subpage_cache[index], !=, NULL); - } } void abd_fini(void) { - const int step_size = SPA_MINBLOCKSIZE; - for (int bytes = step_size; bytes < ABD_PGSIZE; bytes += step_size) { - const int index = (bytes >> SPA_MINBLOCKSHIFT) - 1; - kmem_cache_destroy(abd_subpage_cache[index]); - abd_subpage_cache[index] = NULL; - } - abd_free_zero_scatter(); if (abd_ksp != NULL) { @@ -512,7 +395,7 @@ void abd_free_linear_page(abd_t *abd) { /* - * FreeBSD does not have have scatter linear pages + * FreeBSD does not have scatter linear pages * so there is an error. */ VERIFY(0); @@ -534,64 +417,22 @@ abd_alloc_for_io(size_t size, boolean_t is_metadata) return (abd_alloc_linear(size, is_metadata)); } - -/* - * return an ABD structure that peers into source ABD sabd. The returned ABD - * may be new, or the one supplied as abd. abd and sabd must point to one or - * more zfs_abd_chunk_size (ABD_PGSIZE) chunks, or point to one and exactly one - * smaller chunk. - * - * The [off, off+size] range must be found within (and thus - * fit within) the source ABD. - */ - abd_t * -abd_get_offset_scatter(abd_t *abd, abd_t *sabd, size_t off, size_t size) +abd_get_offset_scatter(abd_t *abd, abd_t *sabd, size_t off, + size_t size) { abd_verify(sabd); - VERIFY3U(off, <=, sabd->abd_size); + ASSERT3U(off, <=, sabd->abd_size); - const uint_t sabd_chunksz = ABD_SCATTER(sabd).abd_chunk_size; + size_t new_offset = ABD_SCATTER(sabd).abd_offset + off; + size_t chunkcnt = abd_chunkcnt_for_bytes( + (new_offset & PAGE_MASK) + size); - const size_t new_offset = ABD_SCATTER(sabd).abd_offset + off; - - /* subpage ABD range checking */ - if (sabd_chunksz != zfs_abd_chunk_size) { - /* off+size must fit in 1 chunk */ - VERIFY3U(off + size, <=, sabd_chunksz); - /* new_offset must be in bounds of 1 chunk */ - VERIFY3U(new_offset, <=, sabd_chunksz); - /* new_offset + size must be in bounds of 1 chunk */ - VERIFY3U(new_offset + size, <=, sabd_chunksz); - } + ASSERT3U(chunkcnt, <=, abd_scatter_chunkcnt(sabd)); /* - * chunkcnt is abd_chunkcnt_for_bytes(size), which rounds - * up to the nearest chunk, but we also must take care - * of the offset *in the leading chunk* - */ - const size_t chunkcnt = (sabd_chunksz != zfs_abd_chunk_size) - ? 1 - : abd_chunkcnt_for_bytes((new_offset % sabd_chunksz) + size); - - /* sanity checks on chunkcnt */ - VERIFY3U(chunkcnt, <=, abd_scatter_chunkcnt(sabd)); - VERIFY3U(chunkcnt, >, 0); - - /* non-subpage sanity checking */ - if (chunkcnt > 1) { - /* compare with legacy calculation of chunkcnt */ - VERIFY3U(chunkcnt, ==, abd_chunkcnt_for_bytes( - P2PHASE(new_offset, zfs_abd_chunk_size) + size)); - /* EITHER subpage chunk (singular) or std chunks */ - VERIFY3U(sabd_chunksz, ==, zfs_abd_chunk_size); - } - - /* - * If an abd struct is provided, it is only the minimum size (and - * almost certainly provided as an abd_t embedded in a larger - * structure). If we need additional chunks, we need to allocate a - * new struct. + * If an abd struct is provided, it is only the minimum size. If we + * need additional chunks, we need to allocate a new struct. */ if (abd != NULL && offsetof(abd_t, abd_u.abd_scatter.abd_chunks[chunkcnt]) > @@ -600,7 +441,7 @@ abd_get_offset_scatter(abd_t *abd, abd_t *sabd, size_t off, size_t size) } if (abd == NULL) - abd = abd_alloc_struct(chunkcnt * sabd_chunksz); + abd = abd_alloc_struct(chunkcnt << PAGE_SHIFT); /* * Even if this buf is filesystem metadata, we only track that @@ -608,44 +449,56 @@ abd_get_offset_scatter(abd_t *abd, abd_t *sabd, size_t off, size_t size) * this case. Therefore, we don't ever use ABD_FLAG_META here. */ - /* update offset, and sanity check it */ - ABD_SCATTER(abd).abd_offset = new_offset % sabd_chunksz; - - VERIFY3U(ABD_SCATTER(abd).abd_offset, <, sabd_chunksz); - VERIFY3U(ABD_SCATTER(abd).abd_offset + size, <=, - chunkcnt * sabd_chunksz); - - ABD_SCATTER(abd).abd_chunk_size = sabd_chunksz; - - if (chunkcnt > 1) { - VERIFY3U(ABD_SCATTER(sabd).abd_chunk_size, ==, - zfs_abd_chunk_size); - } + ABD_SCATTER(abd).abd_offset = new_offset & PAGE_MASK; /* Copy the scatterlist starting at the correct offset */ (void) memcpy(&ABD_SCATTER(abd).abd_chunks, - &ABD_SCATTER(sabd).abd_chunks[new_offset / - sabd_chunksz], + &ABD_SCATTER(sabd).abd_chunks[new_offset >> PAGE_SHIFT], chunkcnt * sizeof (void *)); return (abd); } -static inline size_t -abd_iter_scatter_chunk_offset(struct abd_iter *aiter) +/* + * Allocate a scatter ABD structure from user pages. + */ +abd_t * +abd_alloc_from_pages(vm_page_t *pages, unsigned long offset, uint64_t size) { - ASSERT(!abd_is_linear(aiter->iter_abd)); - return ((ABD_SCATTER(aiter->iter_abd).abd_offset + - aiter->iter_pos) % - ABD_SCATTER(aiter->iter_abd).abd_chunk_size); -} + VERIFY3U(size, <=, DMU_MAX_ACCESS); + ASSERT3U(offset, <, PAGE_SIZE); + ASSERT3P(pages, !=, NULL); -static inline size_t -abd_iter_scatter_chunk_index(struct abd_iter *aiter) -{ - ASSERT(!abd_is_linear(aiter->iter_abd)); - return ((ABD_SCATTER(aiter->iter_abd).abd_offset + aiter->iter_pos) - / ABD_SCATTER(aiter->iter_abd).abd_chunk_size); + /* Until we do DIRECTIO */ + VERIFY3U(0, !=, 0); + abd_t *abd = NULL; +#if 0 + abd_t *abd = abd_alloc_struct(size); + abd->abd_flags |= ABD_FLAG_OWNER | ABD_FLAG_FROM_PAGES; + abd->abd_size = size; + + if ((offset + size) <= PAGE_SIZE) { + /* + * There is only a single page worth of data, so we will just + * use a linear ABD. We have to make sure to take into account + * the offset though. In all other cases our offset will be 0 + * as we are always PAGE_SIZE aligned. + */ + abd->abd_flags |= ABD_FLAG_LINEAR | ABD_FLAG_LINEAR_PAGE; + ABD_LINEAR_BUF(abd) = (char *)zfs_map_page(pages[0], + &abd->abd_u.abd_linear.sf) + offset; + } else { + ABD_SCATTER(abd).abd_offset = offset; + ASSERT0(ABD_SCATTER(abd).abd_offset); + + /* + * Setting the ABD's abd_chunks to point to the user pages. + */ + for (int i = 0; i < abd_chunkcnt_for_bytes(size); i++) + ABD_SCATTER(abd).abd_chunks[i] = pages[i]; + } +#endif + return (abd); } /* @@ -656,10 +509,8 @@ abd_iter_init(struct abd_iter *aiter, abd_t *abd) { ASSERT(!abd_is_gang(abd)); abd_verify(abd); + memset(aiter, 0, sizeof (struct abd_iter)); aiter->iter_abd = abd; - aiter->iter_pos = 0; - aiter->iter_mapaddr = NULL; - aiter->iter_mapsize = 0; } /* @@ -698,54 +549,25 @@ void abd_iter_map(struct abd_iter *aiter) { void *paddr; - size_t offset = 0; ASSERT3P(aiter->iter_mapaddr, ==, NULL); ASSERT0(aiter->iter_mapsize); -#if 0 - /* Panic if someone has changed zfs_abd_chunk_size */ - - IMPLY(!abd_is_linear(aiter->iter_abd), zfs_abd_chunk_size == - ABD_SCATTER(aiter->iter_abd).abd_chunk_size); -#else - /* - * If scattered, VERIFY that we are using ABD_PGSIZE chunks, or we have - * one and only one chunk of less than ABD_PGSIZE. - */ - - if (!abd_is_linear(aiter->iter_abd)) { - if (ABD_SCATTER(aiter->iter_abd).abd_chunk_size != - zfs_abd_chunk_size) { - VERIFY3U( - ABD_SCATTER(aiter->iter_abd).abd_chunk_size, - <, zfs_abd_chunk_size); - VERIFY3U(aiter->iter_abd->abd_size, - <, zfs_abd_chunk_size); - VERIFY3U(aiter->iter_abd->abd_size, - <=, ABD_SCATTER(aiter->iter_abd).abd_chunk_size); - } - } -#endif - /* There's nothing left to iterate over, so do nothing */ if (abd_iter_at_end(aiter)) return; - if (abd_is_linear(aiter->iter_abd)) { - offset = aiter->iter_pos; - aiter->iter_mapsize = aiter->iter_abd->abd_size - offset; - paddr = ABD_LINEAR_BUF(aiter->iter_abd); + abd_t *abd = aiter->iter_abd; + size_t offset = aiter->iter_pos; + if (abd_is_linear(abd)) { + aiter->iter_mapsize = abd->abd_size - offset; + paddr = ABD_LINEAR_BUF(abd); } else { - size_t index = abd_iter_scatter_chunk_index(aiter); - IMPLY(ABD_SCATTER(aiter->iter_abd).abd_chunk_size != ABD_PGSIZE, - index == 0); - offset = abd_iter_scatter_chunk_offset(aiter); - aiter->iter_mapsize = MIN( - ABD_SCATTER(aiter->iter_abd).abd_chunk_size - - offset, - aiter->iter_abd->abd_size - aiter->iter_pos); - paddr = ABD_SCATTER(aiter->iter_abd).abd_chunks[index]; + offset += ABD_SCATTER(abd).abd_offset; + paddr = ABD_SCATTER(abd).abd_chunks[offset >> PAGE_SHIFT]; + offset &= PAGE_MASK; + aiter->iter_mapsize = MIN(ABD_PGSIZE - offset, + abd->abd_size - aiter->iter_pos); } aiter->iter_mapaddr = (char *)paddr + offset; } @@ -786,3 +608,108 @@ abd_cache_reap_now(void) * can be a noop. */ } + +/* + * Borrow a raw buffer from an ABD without copying the contents of the ABD + * into the buffer. If the ABD is scattered, this will alloate a raw buffer + * whose contents are undefined. To copy over the existing data in the ABD, use + * abd_borrow_buf_copy() instead. + */ +void * +abd_borrow_buf(abd_t *abd, size_t n) +{ + void *buf; + abd_verify(abd); + ASSERT3U(abd->abd_size, >=, 0); + if (abd_is_linear(abd)) { + buf = abd_to_buf(abd); + } else { + buf = zio_buf_alloc(n); + } +#ifdef ZFS_DEBUG + (void) zfs_refcount_add_many(&abd->abd_children, n, buf); +#endif + return (buf); +} + +void * +abd_borrow_buf_copy(abd_t *abd, size_t n) +{ + void *buf = abd_borrow_buf(abd, n); + if (!abd_is_linear(abd)) { + abd_copy_to_buf(buf, abd, n); + } + return (buf); +} + +/* + * Return a borrowed raw buffer to an ABD. If the ABD is scattered, this will + * not change the contents of the ABD. If you want any changes you made to + * buf to be copied back to abd, use abd_return_buf_copy() instead. If the + * ABD is not constructed from user pages from Direct I/O then an ASSERT + * checks to make sure the contents of the buffer have not changed since it was + * borrowed. We can not ASSERT the contents of the buffer have not changed if + * it is composed of user pages. While Direct I/O write pages are placed under + * write protection and can not be changed, this is not the case for Direct I/O + * reads. The pages of a Direct I/O read could be manipulated at any time. + * Checksum verifications in the ZIO pipeline check for this issue and handle + * it by returning an error on checksum verification failure. + */ +void +abd_return_buf(abd_t *abd, void *buf, size_t n) +{ + abd_verify(abd); + ASSERT3U(abd->abd_size, >=, n); +#ifdef ZFS_DEBUG + (void) zfs_refcount_remove_many(&abd->abd_children, n, buf); +#endif + if (abd_is_from_pages(abd)) { + if (!abd_is_linear_page(abd)) + zio_buf_free(buf, n); + } else if (abd_is_linear(abd)) { + ASSERT3P(buf, ==, abd_to_buf(abd)); + } else if (abd_is_gang(abd)) { +#ifdef ZFS_DEBUG + /* + * We have to be careful with gang ABD's that we do not ASSERT + * for any ABD's that contain user pages from Direct I/O. See + * the comment above about Direct I/O read buffers possibly + * being manipulated. In order to handle this, we jsut iterate + * through the gang ABD and only verify ABD's that are not from + * user pages. + */ + void *cmp_buf = buf; + + for (abd_t *cabd = list_head(&ABD_GANG(abd).abd_gang_chain); + cabd != NULL; + cabd = list_next(&ABD_GANG(abd).abd_gang_chain, cabd)) { + if (!abd_is_from_pages(cabd)) { + ASSERT0(abd_cmp_buf(cabd, cmp_buf, + cabd->abd_size)); + } + cmp_buf = (char *)cmp_buf + cabd->abd_size; + } +#endif + zio_buf_free(buf, n); + } else { + ASSERT0(abd_cmp_buf(abd, buf, n)); + zio_buf_free(buf, n); + } +} + +void +abd_return_buf_copy(abd_t *abd, void *buf, size_t n) +{ + if (!abd_is_linear(abd)) { + abd_copy_from_buf(abd, buf, n); + } + abd_return_buf(abd, buf, n); +} + +/* Tunable Parameters */ +module_param(zfs_abd_scatter_enabled, int, 0644); +MODULE_PARM_DESC(zfs_abd_scatter_enabled, + "Toggle whether ABD allocations must be linear."); +module_param(zfs_abd_scatter_min_size, int, 0644); +MODULE_PARM_DESC(zfs_abd_scatter_min_size, + "Minimum size of scatter allocations."); diff --git a/module/os/macos/zfs/arc_os.c b/module/os/macos/zfs/arc_os.c index 2af34fa79141..922abe62cbd7 100644 --- a/module/os/macos/zfs/arc_os.c +++ b/module/os/macos/zfs/arc_os.c @@ -169,7 +169,7 @@ arc_memory_throttle(spa_t *spa, uint64_t reserve, uint64_t txg) */ static void arc_kmem_reap_now(void) { - arc_wait_for_eviction(0, B_FALSE); + arc_wait_for_eviction(0, B_FALSE, B_FALSE); /* arc.c will do the heavy lifting */ arc_kmem_reap_soon(); @@ -223,7 +223,7 @@ arc_reclaim_thread(void *unused) (arc_c >> arc_shrink_shift))); } - arc_wait_for_eviction(0, B_FALSE); + arc_wait_for_eviction(0, B_FALSE, B_FALSE); int64_t free_memory = arc_available_memory(); diff --git a/module/os/macos/zfs/vdev_disk.c b/module/os/macos/zfs/vdev_disk.c index c7d457b56661..e5b451733b09 100644 --- a/module/os/macos/zfs/vdev_disk.c +++ b/module/os/macos/zfs/vdev_disk.c @@ -159,8 +159,8 @@ vdev_disk_off_finalize(ldi_handle_t lh, ldi_ev_cookie_t ecookie, * unsuccessful. */ if (ldi_result != LDI_EV_SUCCESS) { - vd->vdev_probe_wanted = B_TRUE; - spa_async_request(vd->vdev_spa, SPA_ASYNC_PROBE); + vd->vdev_fault_wanted = B_TRUE; + spa_async_request(vd->vdev_spa, SPA_ASYNC_FAULT_VDEV); } } @@ -486,6 +486,30 @@ vdev_disk_io_intr(ldi_buf_t *bp) zio->io_size); } + /* + * thread_block / yield if this is an automated / low-priority read or + * write, in order to avoid CPU starvation of user-initiated threads. + * + * kpreempt(KPREEMPT_SYNC) is cheap and fast on a mac with idle cores, + * + * However, on a busy system it effectively asks xnu to impose a + * system-load-dependent delay on the forthcoming insertion of this + * I/O into a z_{rd,wr}_int taskq (which even on a busy many-cored + * system might send the zio without delay to checksuming and other + * expensive pipeline operations). + * + * During a scrub this also tends to increase the difference in queue + * depth and latency (zpool iostat -{q,w}) between scrubq_read and + * [a]syncq I/Os to the same pool. + */ + if (zio->io_priority == ZIO_PRIORITY_SCRUB || + zio->io_priority == ZIO_PRIORITY_REMOVAL || + zio->io_priority == ZIO_PRIORITY_INITIALIZING || + zio->io_priority == ZIO_PRIORITY_TRIM || + zio->io_priority == ZIO_PRIORITY_REBUILD) { + kpreempt(KPREEMPT_SYNC); + } + zio_delay_interrupt(zio); } @@ -543,7 +567,11 @@ vdev_disk_io_strategy(void *arg) case ZIO_TYPE_READ: if (zio->io_priority == ZIO_PRIORITY_SYNC_READ) { flags = B_READ; - } else if (zio->io_priority == ZIO_PRIORITY_SCRUB) { + } else if (zio->io_priority == ZIO_PRIORITY_SCRUB || + zio->io_priority == ZIO_PRIORITY_REMOVAL || + zio->io_priority == ZIO_PRIORITY_INITIALIZING || + zio->io_priority == ZIO_PRIORITY_TRIM || + zio->io_priority == ZIO_PRIORITY_REBUILD) { /* * Signal using B_THROTTLED_IO. * This is safe because our path through @@ -601,7 +629,7 @@ vdev_disk_io_start(zio_t *zio) } switch (zio->io_type) { - case ZIO_TYPE_IOCTL: + case ZIO_TYPE_FLUSH: if (!vdev_readable(vd)) { zio->io_error = SET_ERROR(ENXIO); @@ -609,44 +637,34 @@ vdev_disk_io_start(zio_t *zio) return; } - switch (zio->io_cmd) { - case DKIOCFLUSHWRITECACHE: - - if (zfs_nocacheflush) - break; - - if (vd->vdev_nowritecache) { - zio->io_error = SET_ERROR(ENOTSUP); - break; - } - - zio->io_vsd = dkc = kmem_alloc(sizeof (*dkc), KM_SLEEP); - zio->io_vsd_ops = &vdev_disk_vsd_ops; - - dkc->dkc_callback = vdev_disk_ioctl_done; - dkc->dkc_flag = FLUSH_VOLATILE; - dkc->dkc_cookie = zio; + if (zfs_nocacheflush) + break; - error = ldi_ioctl(dvd->vd_lh, zio->io_cmd, - (uintptr_t)dkc, FKIOCTL, kcred, NULL); + if (vd->vdev_nowritecache) { + zio->io_error = SET_ERROR(ENOTSUP); + break; + } - if (error == 0) { - /* - * The ioctl will be done asychronously, - * and will call vdev_disk_ioctl_done() - * upon completion. - */ - return; - } + zio->io_vsd = dkc = kmem_alloc(sizeof (*dkc), KM_SLEEP); + zio->io_vsd_ops = &vdev_disk_vsd_ops; - zio->io_error = error; + dkc->dkc_callback = vdev_disk_ioctl_done; + dkc->dkc_flag = FLUSH_VOLATILE; + dkc->dkc_cookie = zio; - break; + error = ldi_ioctl(dvd->vd_lh, DKIOCFLUSHWRITECACHE, + (uintptr_t)dkc, FKIOCTL, kcred, NULL); - default: - zio->io_error = SET_ERROR(ENOTSUP); - } /* io_cmd */ + if (error == 0) { + /* + * The ioctl will be done asychronously, + * and will call vdev_disk_ioctl_done() + * upon completion. + */ + return; + } + zio->io_error = error; zio_execute(zio); return; @@ -692,9 +710,10 @@ vdev_disk_io_start(zio_t *zio) } /* - * dispatch async or scrub reads on appropriate taskq, - * dispatch async writes on appropriate taskq, - * do everything else on this thread + * Dispatch scrub and other low-priority reads on a + * lower-thread-priority and lower-thread-number taskq, + * with other async I/Os dispatched to an appropriate + * taskq, and other I/Os into a default taskq for observability. */ if (zio->io_type == ZIO_TYPE_READ) { if (zio->io_priority == ZIO_PRIORITY_ASYNC_READ) { @@ -702,7 +721,15 @@ vdev_disk_io_start(zio_t *zio) vdev_disk_io_strategy, zio, TQ_SLEEP), !=, 0); return; - } else if (zio->io_priority == ZIO_PRIORITY_SCRUB) { + } else if (zio->io_priority == ZIO_PRIORITY_SCRUB || + zio->io_priority == ZIO_PRIORITY_REMOVAL || + zio->io_priority == ZIO_PRIORITY_INITIALIZING || + zio->io_priority == ZIO_PRIORITY_TRIM || + zio->io_priority == ZIO_PRIORITY_REBUILD) { + /* + * dispatch automated / low-priority reads onto a + * lowered-priority taskq + */ VERIFY3U(taskq_dispatch(vdev_disk_taskq_scrub, vdev_disk_io_strategy, zio, TQ_SLEEP), !=, 0); @@ -850,7 +877,7 @@ vdev_disk_init(void) * Apple Silicon we deflate further. */ - const int cpus = MAX(1, max_ncpus - num_ecores - 2); + const int cpus = MAX(1, (int)(max_ncpus - num_ecores - 2)); /* * Keep vdev_disk_taskq_stack as in-order as we can, and use diff --git a/module/os/macos/zfs/vdev_file.c b/module/os/macos/zfs/vdev_file.c index 24a32b1eb547..8e389fd9b4f5 100644 --- a/module/os/macos/zfs/vdev_file.c +++ b/module/os/macos/zfs/vdev_file.c @@ -231,6 +231,8 @@ vdev_file_io_start(zio_t *zio) { vdev_t *vd = zio->io_vd; vdev_file_t *vf = vd->vdev_tsd; + zfs_file_t *fp = vf->vf_file; + int error = 0; #ifdef CLOSE_ON_UNMOUNT if (vf != NULL && vf->vf_file == NULL) { @@ -254,33 +256,34 @@ vdev_file_io_start(zio_t *zio) } #endif - if (zio->io_type == ZIO_TYPE_IOCTL) { + if (zio->io_type == ZIO_TYPE_FLUSH) { if (!vdev_readable(vd)) { - zio->io_error = SET_ERROR(ENXIO); - zio_interrupt(zio); - return; - } - - switch (zio->io_cmd) { - case DKIOCFLUSHWRITECACHE: - zio->io_error = zfs_file_fsync(vf->vf_file, - O_SYNC|O_DSYNC); - break; - default: - zio->io_error = SET_ERROR(ENOTSUP); + /* Drive not there, can't flush */ + error = SET_ERROR(ENXIO); + } else if (zfs_nocacheflush) { + /* Flushing disabled by operator, declare success */ + error = 0; + } else if (vd->vdev_nowritecache) { + /* This vdev not capable of flushing */ + error = SET_ERROR(ENOTSUP); + } else { + /* + * Issue the flush. If successful, the response will + * be handled in the completion callback, so we're done. + */ + zio->io_error = zfs_file_fsync(fp, 0); } + zio->io_error = error; zio_execute(zio); return; - } else if (zio->io_type == ZIO_TYPE_TRIM) { - int mode = 0; + } else if (zio->io_type == ZIO_TYPE_TRIM) { ASSERT3U(zio->io_size, !=, 0); - /* XXX FreeBSD has no fallocate routine in file ops */ - zio->io_error = zfs_file_fallocate(vf->vf_file, - mode, zio->io_offset, zio->io_size); + zio->io_error = zfs_file_deallocate(vf->vf_file, + zio->io_offset, zio->io_size); zio_execute(zio); return; } diff --git a/module/os/macos/zfs/vdev_label_os.c b/module/os/macos/zfs/vdev_label_os.c new file mode 100644 index 000000000000..3d965b89a962 --- /dev/null +++ b/module/os/macos/zfs/vdev_label_os.c @@ -0,0 +1,45 @@ +/* + * CDDL HEADER START + * + * The contents of this file are subject to the terms of the + * Common Development and Distribution License (the "License"). + * You may not use this file except in compliance with the License. + * + * You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE + * or https://opensource.org/licenses/CDDL-1.0. + * See the License for the specific language governing permissions + * and limitations under the License. + * + * When distributing Covered Code, include this CDDL HEADER in each + * file and include the License file at usr/src/OPENSOLARIS.LICENSE. + * If applicable, add the following below this CDDL HEADER, with the + * fields enclosed by brackets "[]" replaced with your own identifying + * information: Portions Copyright [yyyy] [name of copyright owner] + * + * CDDL HEADER END + */ + +/* + * Copyright (c) 2023 by iXsystems, Inc. + */ + +#include +#include +#include +#include +#include + +/* + * Check if the reserved boot area is in-use. + * + * This function always returns 0, as there are no known external uses + * of the reserved area on Linux. + */ +int +vdev_check_boot_reserve(spa_t *spa, vdev_t *childvd) +{ + (void) spa; + (void) childvd; + + return (0); +} diff --git a/module/os/macos/zfs/zfs_boot.cpp b/module/os/macos/zfs/zfs_boot.cpp index 84a1608c0e2e..28a3362a81cd 100644 --- a/module/os/macos/zfs/zfs_boot.cpp +++ b/module/os/macos/zfs/zfs_boot.cpp @@ -105,6 +105,7 @@ extern "C" { #include #include #include +#include #include #include diff --git a/module/os/macos/zfs/zfs_file_os.c b/module/os/macos/zfs/zfs_file_os.c index 96959505db08..358bc90ac954 100644 --- a/module/os/macos/zfs/zfs_file_os.c +++ b/module/os/macos/zfs/zfs_file_os.c @@ -360,7 +360,7 @@ zfs_file_fsync(zfs_file_t *filp, int flags) * OPTIONAL */ int -zfs_file_fallocate(zfs_file_t *fp, int mode, loff_t offset, loff_t len) +zfs_file_deallocate(zfs_file_t *fp, loff_t offset, loff_t len) { int rc; struct flock flck = { 0 }; diff --git a/module/os/macos/zfs/zfs_racct.c b/module/os/macos/zfs/zfs_racct.c index 7b5d510c303f..6fe679842784 100644 --- a/module/os/macos/zfs/zfs_racct.c +++ b/module/os/macos/zfs/zfs_racct.c @@ -22,11 +22,13 @@ #include void -zfs_racct_read(uint64_t size, uint64_t iops) +zfs_racct_read(spa_t *spa, uint64_t size, uint64_t iops, uint32_t flags) { + spa_iostats_read_add(spa, size, iops, flags); } void -zfs_racct_write(uint64_t size, uint64_t iops) +zfs_racct_write(spa_t *spa, uint64_t size, uint64_t iops, uint32_t flags) { + spa_iostats_write_add(spa, size, iops, flags); } diff --git a/module/os/macos/zfs/zfs_vfsops.c b/module/os/macos/zfs/zfs_vfsops.c index ee3d21f0d7c3..264d79a1a98d 100644 --- a/module/os/macos/zfs/zfs_vfsops.c +++ b/module/os/macos/zfs/zfs_vfsops.c @@ -48,6 +48,7 @@ #include #include #include +#include // spa_freeze_txg unsigned int zfs_vnop_skip_unlinked_drain = 0; @@ -2023,7 +2024,6 @@ zfsvfs_teardown(zfsvfs_t *zfsvfs, boolean_t unmounting) dmu_objset_pool(zfsvfs->z_os)), 0); if (++round > 1 && !unmounting) break; - break; /* Only loop once - osx can get stuck */ } } @@ -2161,6 +2161,10 @@ zfs_vfs_unmount(struct mount *mp, int mntflags, vfs_context_t context) /* Save osname for later */ dmu_objset_name(zfsvfs->z_os, osname); + spa_t *spa = dmu_objset_spa(zfsvfs->z_os); + if (spa != NULL) + spa->spa_freeze_txg = UINT64_MAX; + /* * We might skip the sync called in the unmount path, since * zfs_vfs_sync() is generally ignoring xnu's calls, and alas, @@ -2413,7 +2417,7 @@ zfs_vget_internal(zfsvfs_t *zfsvfs, ino64_t ino, vnode_t **vpp) &parent, sizeof (parent)) == 0); if (zap_value_search(zfsvfs->z_os, parent, zp->z_id, - ZFS_DIRENT_OBJ(-1ULL), name) == 0) { + ZFS_DIRENT_OBJ(-1ULL), name, MAXPATHLEN) == 0) { dprintf("vget: set name '%s'\n", name); vnode_update_identity(*vpp, NULL, name, @@ -2941,92 +2945,6 @@ zfs_set_version(zfsvfs_t *zfsvfs, uint64_t newvers) return (0); } -/* - * Read a property stored within the master node. - */ -int -zfs_get_zplprop(objset_t *os, zfs_prop_t prop, uint64_t *value) -{ - uint64_t *cached_copy = NULL; - - /* - * Figure out where in the objset_t the cached copy would live, if it - * is available for the requested property. - */ - if (os != NULL) { - switch (prop) { - case ZFS_PROP_VERSION: - cached_copy = &os->os_version; - break; - case ZFS_PROP_NORMALIZE: - cached_copy = &os->os_normalization; - break; - case ZFS_PROP_UTF8ONLY: - cached_copy = &os->os_utf8only; - break; - case ZFS_PROP_CASE: - cached_copy = &os->os_casesensitivity; - break; - default: - break; - } - } - if (cached_copy != NULL && *cached_copy != OBJSET_PROP_UNINITIALIZED) { - *value = *cached_copy; - return (0); - } - - /* - * If the property wasn't cached, look up the file system's value for - * the property. For the version property, we look up a slightly - * different string. - */ - const char *pname; - int error = ENOENT; - if (prop == ZFS_PROP_VERSION) { - pname = ZPL_VERSION_STR; - } else { - pname = zfs_prop_to_name(prop); - } - - if (os != NULL) { - ASSERT3U(os->os_phys->os_type, ==, DMU_OST_ZFS); - error = zap_lookup(os, MASTER_NODE_OBJ, pname, 8, 1, value); - } - - if (error == ENOENT) { - /* No value set, use the default value */ - switch (prop) { - case ZFS_PROP_VERSION: - *value = ZPL_VERSION; - break; - case ZFS_PROP_NORMALIZE: - case ZFS_PROP_UTF8ONLY: - *value = 0; - break; - case ZFS_PROP_CASE: - *value = ZFS_CASE_SENSITIVE; - break; - case ZFS_PROP_ACLMODE: - *value = ZFS_ACLTYPE_OFF; - break; - default: - return (error); - } - error = 0; - } - - /* - * If one of the methods for getting the property value above worked, - * copy it into the objset_t's cache. - */ - if (error == 0 && cached_copy != NULL) { - *cached_copy = *value; - } - - return (error); -} - /* * Return true if the coresponding vfs's unmounted flag is set. * Otherwise return false. diff --git a/module/os/macos/zfs/zfs_vnops_os.c b/module/os/macos/zfs/zfs_vnops_os.c index a42f93b4e3e3..bd773737baec 100644 --- a/module/os/macos/zfs/zfs_vnops_os.c +++ b/module/os/macos/zfs/zfs_vnops_os.c @@ -1727,8 +1727,6 @@ zfs_readdir(vnode_t *vp, zfs_uio_t *uio, cred_t *cr, int *eofp, return (error); } -static ulong_t zfs_fsync_sync_cnt = 4; - /* Explore if we can use zfs/zfs_vnops.c's zfs_fsync() */ int zfs_fsync(znode_t *zp, int syncflag, cred_t *cr) @@ -1742,16 +1740,15 @@ zfs_fsync(znode_t *zp, int syncflag, cred_t *cr) cluster_push(vp, /* waitdata ? IO_SYNC : */ 0); } - (void) tsd_set(zfs_fsyncer_key, (void *)zfs_fsync_sync_cnt); - if (zfsvfs->z_os->os_sync != ZFS_SYNC_DISABLED && !vnode_isrecycled(ZTOV(zp))) { if ((error = zfs_enter_verify_zp(zfsvfs, zp, FTAG)) != 0) return (error); + atomic_inc_32(&zp->z_sync_writes_cnt); zil_commit(zfsvfs->z_log, zp->z_id); + atomic_dec_32(&zp->z_sync_writes_cnt); zfs_exit(zfsvfs, FTAG); } - tsd_set(zfs_fsyncer_key, NULL); return (0); } diff --git a/module/os/macos/zfs/zfs_vnops_osx.c b/module/os/macos/zfs/zfs_vnops_osx.c index 3f2ca8156123..02405f9d80ec 100644 --- a/module/os/macos/zfs/zfs_vnops_osx.c +++ b/module/os/macos/zfs/zfs_vnops_osx.c @@ -74,8 +74,8 @@ #include #include - - +#include +#include #ifdef _KERNEL #include @@ -2706,7 +2706,7 @@ zfs_pageout(zfsvfs_t *zfsvfs, znode_t *zp, upl_t upl, vm_offset_t upl_offset, err = sa_bulk_update(zp->z_sa_hdl, bulk, count, tx); ASSERT0(err); zfs_log_write(zfsvfs->z_log, tx, TX_WRITE, zp, off, len, 0, - NULL, NULL); + B_FALSE, NULL, NULL); } dmu_tx_commit(tx); @@ -3142,7 +3142,7 @@ zfs_vnop_pageoutv2(struct vnop_pageout_args *ap) &zp->z_pflags, 8); zfs_tstamp_update_setup(zp, CONTENT_MODIFIED, mtime, ctime); zfs_log_write(zfsvfs->z_log, tx, TX_WRITE, zp, ap->a_f_offset, - a_size, 0, NULL, NULL); + a_size, 0, B_FALSE, NULL, NULL); } dmu_tx_commit(tx); @@ -4654,6 +4654,18 @@ zfs_vnop_clonefile(struct vnop_clonefile_args *ap) error = zfs_clone_range(inzp, &inoff, outzp, &outoff, &len, cr); +#if defined(MAC_OS_X_VERSION_14) && \ + (MAC_OS_X_VERSION_MIN_REQUIRED >= MAC_OS_X_VERSION_14) + /* + * From file_cmds-428 (Sonoma) if we return ENOTSUP, it will + * fallback to regular copy. Earlier macOS will just print + * error. We could consider an internal copy_file_range() + * here if we want to. + */ + if (error == EAGAIN) + error = ENOTSUP; +#endif + if (error == 0) { *ap->a_vpp = ZTOV(outzp); } else { @@ -5098,9 +5110,6 @@ zfs_znode_asyncwait(zfsvfs_t *zfsvfs, znode_t *zp) if (zfsvfs == NULL) return (ret); - if ((error = zfs_enter(zfsvfs, FTAG)) != 0) - return (ret); - if (zfsvfs->z_os == NULL) goto out; @@ -5117,7 +5126,6 @@ zfs_znode_asyncwait(zfsvfs_t *zfsvfs, znode_t *zp) mutex_exit(&zp->z_attach_lock); out: - zfs_exit(zfsvfs, FTAG); return (ret); } diff --git a/module/os/macos/zfs/zfs_vnops_osx_lib.c b/module/os/macos/zfs/zfs_vnops_osx_lib.c index 20cc7f8e1f43..316cf0b3e634 100644 --- a/module/os/macos/zfs/zfs_vnops_osx_lib.c +++ b/module/os/macos/zfs/zfs_vnops_osx_lib.c @@ -349,7 +349,7 @@ zfs_getattr_znode_unlocked(struct vnode *vp, vattr_t *vap) // Go find the name. if (zap_value_search(zfsvfs->z_os, parent, zp->z_id, ZFS_DIRENT_OBJ(-1ULL), - vap->va_name) == 0) { + vap->va_name, MAXPATHLEN) == 0) { VATTR_SET_SUPPORTED(vap, va_name); // Might as well keep this name too. strlcpy(zp->z_name_cache, vap->va_name, @@ -648,8 +648,10 @@ zfs_getbsdflags(znode_t *zp) if (zflags & ZFS_TRACKED) bsdflags |= UF_TRACKED; +#ifdef SF_NOUNLINK if (zflags & ZFS_NOUNLINK) bsdflags |= SF_NOUNLINK; +#endif if (zflags & ZFS_SIMMUTABLE) bsdflags |= SF_IMMUTABLE; if (zflags & ZFS_SAPPENDONLY) @@ -704,8 +706,10 @@ zfs_setbsdflags(znode_t *zp, uint32_t ioctl_flags, xvattr_t *xva) XAT_SIMMUTABLE, xoap->xoa_simmutable); FLAG_CHANGE(ZFS_ARCHIVE, SF_ARCHIVED, XAT_ARCHIVE, xoap->xoa_archive); +#ifdef SF_NOUNLINK FLAG_CHANGE(ZFS_NOUNLINK, SF_NOUNLINK, XAT_NOUNLINK, xoap->xoa_nounlink); +#endif // FLAG_CHANGE(ZFS_READONLY, XAT_READONLY, xoap->xoa_readonly); // FLAG_CHANGE(ZFS_SYSTEM, XAT_SYSTEM, xoap->xoa_system); // FLAG_CHANGE(ZFS_REPARSE, XAT_REPARSE, xoap->xoa_reparse); @@ -1950,7 +1954,7 @@ zfs_setattr_generate_id(znode_t *zp, uint64_t val, char *name) /* Lookup filename */ filename = kmem_zalloc(MAXPATHLEN + 2, KM_SLEEP); if (zap_value_search(zfsvfs->z_os, parent, zp->z_id, - ZFS_DIRENT_OBJ(-1ULL), filename) == 0) { + ZFS_DIRENT_OBJ(-1ULL), filename, MAXPATHLEN) == 0) { nameptr = filename; // Might as well keep this name too. diff --git a/module/os/macos/zfs/zfs_znode.c b/module/os/macos/zfs/zfs_znode_os.c similarity index 89% rename from module/os/macos/zfs/zfs_znode.c rename to module/os/macos/zfs/zfs_znode_os.c index af15502c0572..5deb2c89693c 100644 --- a/module/os/macos/zfs/zfs_znode.c +++ b/module/os/macos/zfs/zfs_znode_os.c @@ -29,7 +29,6 @@ /* Portions Copyright 2011 Martin Matuska */ /* Portions Copyright 2013 Jorgen Lundman */ -#ifdef _KERNEL #include #include #include @@ -55,7 +54,6 @@ #include #include #include -#endif /* _KERNEL */ #include #include @@ -67,7 +65,6 @@ #include #include -#include "zfs_prop.h" #include "zfs_comutil.h" /* Used by fstat(1). */ @@ -78,12 +75,6 @@ SYSCTL_INT(_debug_sizeof, OID_AUTO, znode, CTLFLAG_RD, 0, sizeof (znode_t), void zfs_release_sa_handle(sa_handle_t *hdl, dmu_buf_t *db, void *tag); -/* - * Functions needed for userland (ie: libzpool) are not put under - * #ifdef_KERNEL; the rest of the functions have dependencies - * (such as VFS logic) that will not compile easily in userland. - */ -#ifdef _KERNEL /* * This is used by the test suite so that it can delay znodes from being * freed in order to inspect the unlinked set. @@ -2092,278 +2083,12 @@ zfs_create_fs(objset_t *os, cred_t *cr, nvlist_t *zplprops, dmu_tx_t *tx) kmem_free(zfsvfs, sizeof (zfsvfs_t)); } -#endif /* _KERNEL */ - -static int -zfs_sa_setup(objset_t *osp, sa_attr_type_t **sa_table) -{ - uint64_t sa_obj = 0; - int error; - - error = zap_lookup(osp, MASTER_NODE_OBJ, ZFS_SA_ATTRS, 8, 1, &sa_obj); - if (error != 0 && error != ENOENT) - return (error); - - error = sa_setup(osp, sa_obj, zfs_attr_table, ZPL_END, sa_table); - return (error); -} -static int -zfs_grab_sa_handle(objset_t *osp, uint64_t obj, sa_handle_t **hdlp, - dmu_buf_t **db, void *tag) -{ - dmu_object_info_t doi; - int error; - - if ((error = sa_buf_hold(osp, obj, tag, db)) != 0) - return (error); - - dmu_object_info_from_db(*db, &doi); - if (((doi.doi_bonus_type != DMU_OT_SA) && - (doi.doi_bonus_type != DMU_OT_ZNODE)) || - ((doi.doi_bonus_type == DMU_OT_ZNODE) && - (doi.doi_bonus_size < sizeof (znode_phys_t)))) { - sa_buf_rele(*db, tag); - return (SET_ERROR(ENOTSUP)); - } - - error = sa_handle_get(osp, obj, NULL, SA_HDL_PRIVATE, hdlp); - if (error != 0) { - sa_buf_rele(*db, tag); - return (error); - } - return (0); -} - -void -zfs_release_sa_handle(sa_handle_t *hdl, dmu_buf_t *db, void *tag) -{ - sa_handle_destroy(hdl); - sa_buf_rele(db, tag); -} - -/* - * Given an object number, return its parent object number and whether - * or not the object is an extended attribute directory. - */ -static int -zfs_obj_to_pobj(objset_t *osp, sa_handle_t *hdl, sa_attr_type_t *sa_table, - uint64_t *pobjp, int *is_xattrdir) -{ - uint64_t parent; - uint64_t pflags; - uint64_t mode; - uint64_t parent_mode; - sa_bulk_attr_t bulk[3]; - sa_handle_t *sa_hdl; - dmu_buf_t *sa_db; - int count = 0; - int error; - - SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_PARENT], NULL, - &parent, sizeof (parent)); - SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_FLAGS], NULL, - &pflags, sizeof (pflags)); - SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_MODE], NULL, - &mode, sizeof (mode)); - - if ((error = sa_bulk_lookup(hdl, bulk, count)) != 0) - return (error); - - /* - * When a link is removed its parent pointer is not changed and will - * be invalid. There are two cases where a link is removed but the - * file stays around, when it goes to the delete queue and when there - * are additional links. - */ - error = zfs_grab_sa_handle(osp, parent, &sa_hdl, &sa_db, FTAG); - if (error != 0) - return (error); - - error = sa_lookup(sa_hdl, ZPL_MODE, &parent_mode, sizeof (parent_mode)); - zfs_release_sa_handle(sa_hdl, sa_db, FTAG); - if (error != 0) - return (error); - - *is_xattrdir = ((pflags & ZFS_XATTR) != 0) && S_ISDIR(mode); - - /* - * Extended attributes can be applied to files, directories, etc. - * Otherwise the parent must be a directory. - */ - if (!*is_xattrdir && !S_ISDIR(parent_mode)) - return ((EINVAL)); - - *pobjp = parent; - - return (0); -} - -/* - * Given an object number, return some zpl level statistics - */ -static int -zfs_obj_to_stats_impl(sa_handle_t *hdl, sa_attr_type_t *sa_table, - zfs_stat_t *sb) -{ - sa_bulk_attr_t bulk[4]; - int count = 0; - - SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_MODE], NULL, - &sb->zs_mode, sizeof (sb->zs_mode)); - SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_GEN], NULL, - &sb->zs_gen, sizeof (sb->zs_gen)); - SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_LINKS], NULL, - &sb->zs_links, sizeof (sb->zs_links)); - SA_ADD_BULK_ATTR(bulk, count, sa_table[ZPL_CTIME], NULL, - &sb->zs_ctime, sizeof (sb->zs_ctime)); - - return (sa_bulk_lookup(hdl, bulk, count)); -} - -static int -zfs_obj_to_path_impl(objset_t *osp, uint64_t obj, sa_handle_t *hdl, - sa_attr_type_t *sa_table, char *buf, int len) -{ - sa_handle_t *sa_hdl; - sa_handle_t *prevhdl = NULL; - dmu_buf_t *prevdb = NULL; - dmu_buf_t *sa_db = NULL; - char *path = buf + len - 1; - int error; - - *path = '\0'; - sa_hdl = hdl; - - uint64_t deleteq_obj; - VERIFY0(zap_lookup(osp, MASTER_NODE_OBJ, - ZFS_UNLINKED_SET, sizeof (uint64_t), 1, &deleteq_obj)); - error = zap_lookup_int(osp, deleteq_obj, obj); - if (error == 0) { - return (ESTALE); - } else if (error != ENOENT) { - return (error); - } - error = 0; - - for (;;) { - uint64_t pobj = 0; - char component[MAXNAMELEN + 2]; - size_t complen; - int is_xattrdir = 0; - - if (prevdb) - zfs_release_sa_handle(prevhdl, prevdb, FTAG); - - if ((error = zfs_obj_to_pobj(osp, sa_hdl, sa_table, &pobj, - &is_xattrdir)) != 0) - break; - - if (pobj == obj) { - if (path[0] != '/') - *--path = '/'; - break; - } - - component[0] = '/'; - if (is_xattrdir) { - (void) snprintf(component + 1, MAXNAMELEN+1, - ""); - } else { - error = zap_value_search(osp, pobj, obj, - ZFS_DIRENT_OBJ(-1ULL), - component + 1); - if (error != 0) - break; - } - - complen = strlen(component); - path -= complen; - ASSERT(path >= buf); - memcpy(path, component, complen); - obj = pobj; - - if (sa_hdl != hdl) { - prevhdl = sa_hdl; - prevdb = sa_db; - } - error = zfs_grab_sa_handle(osp, obj, &sa_hdl, &sa_db, FTAG); - if (error != 0) { - sa_hdl = prevhdl; - sa_db = prevdb; - break; - } - } - - if (sa_hdl != NULL && sa_hdl != hdl) { - ASSERT(sa_db != NULL); - zfs_release_sa_handle(sa_hdl, sa_db, FTAG); - } - - if (error == 0) - (void) memmove(buf, path, buf + len - path); - - return (error); -} - -int -zfs_obj_to_path(objset_t *osp, uint64_t obj, char *buf, int len) -{ - sa_attr_type_t *sa_table; - sa_handle_t *hdl; - dmu_buf_t *db; - int error; - - error = zfs_sa_setup(osp, &sa_table); - if (error != 0) - return (error); - - error = zfs_grab_sa_handle(osp, obj, &hdl, &db, FTAG); - if (error != 0) - return (error); - - error = zfs_obj_to_path_impl(osp, obj, hdl, sa_table, buf, len); - - zfs_release_sa_handle(hdl, db, FTAG); - return (error); -} - -int -zfs_obj_to_stats(objset_t *osp, uint64_t obj, zfs_stat_t *sb, - char *buf, int len) -{ - char *path = buf + len - 1; - sa_attr_type_t *sa_table; - sa_handle_t *hdl; - dmu_buf_t *db; - int error; - - *path = '\0'; - - error = zfs_sa_setup(osp, &sa_table); - if (error != 0) - return (error); - - error = zfs_grab_sa_handle(osp, obj, &hdl, &db, FTAG); - if (error != 0) - return (error); - - error = zfs_obj_to_stats_impl(hdl, sa_table, sb); - if (error != 0) { - zfs_release_sa_handle(hdl, db, FTAG); - return (error); - } - - error = zfs_obj_to_path_impl(osp, obj, hdl, sa_table, buf, len); - - zfs_release_sa_handle(hdl, db, FTAG); - return (error); -} - void zfs_znode_update_vfs(znode_t *zp) { - ubc_setsize(ZTOV(zp), zp->z_size); + vnode_pager_setsize(ZTOV(zp), zp->z_size); } + ZFS_MODULE_PARAM(zfs, zfs_, unlink_suspend_progress, UINT, ZMOD_RW, "Set to prevent async unlinks "); diff --git a/module/os/macos/zfs/zvol_os.c b/module/os/macos/zfs/zvol_os.c index dc7a67b4de3b..556f24602833 100644 --- a/module/os/macos/zfs/zvol_os.c +++ b/module/os/macos/zfs/zvol_os.c @@ -508,7 +508,7 @@ zvol_os_unmap(zvol_state_t *zv, uint64_t off, uint64_t bytes) dmu_tx_abort(tx); } else { - zvol_log_truncate(zv, tx, off, bytes, B_TRUE); + zvol_log_truncate(zv, tx, off, bytes); dmu_tx_commit(tx); diff --git a/module/zfs/spa.c b/module/zfs/spa.c index 9a534a2b8c7f..55e3321d6b07 100644 --- a/module/zfs/spa.c +++ b/module/zfs/spa.c @@ -1493,11 +1493,13 @@ spa_taskq_param_get(zio_type_t t, char *buf, boolean_t add_newline) for (uint_t q = 0; q < ZIO_TASKQ_TYPES; q++) { const zio_taskq_info_t *zti = &zio_taskqs[t][q]; if (zti->zti_mode == ZTI_MODE_FIXED) - pos += sprintf(&buf[pos], "%s%s,%u,%u", sep, + pos += snprintf(&buf[pos], LINUX_MAX_MODULE_PARAM_LEN, + "%s%s,%u,%u", sep, modes[zti->zti_mode], zti->zti_count, zti->zti_value); else - pos += sprintf(&buf[pos], "%s%s", sep, + pos += snprintf(&buf[pos], LINUX_MAX_MODULE_PARAM_LEN, + "%s%s", sep, modes[zti->zti_mode]); sep = " "; } @@ -7691,10 +7693,12 @@ spa_vdev_attach(spa_t *spa, uint64_t guid, nvlist_t *nvroot, int replacing, * to make it distinguishable from newvd, and unopenable from now on. */ if (strcmp(oldvdpath, newvdpath) == 0) { + int n = strlen(newvdpath) + 5; spa_strfree(oldvd->vdev_path); - oldvd->vdev_path = kmem_alloc(strlen(newvdpath) + 5, + oldvd->vdev_path = kmem_alloc(n, KM_SLEEP); - (void) sprintf(oldvd->vdev_path, "%s/old", + (void) snprintf(oldvd->vdev_path, n, + "%s/old", newvdpath); if (oldvd->vdev_devid != NULL) { spa_strfree(oldvd->vdev_devid); diff --git a/module/zfs/zfs_log.c b/module/zfs/zfs_log.c index cb4534f8f34a..1642682f4026 100644 --- a/module/zfs/zfs_log.c +++ b/module/zfs/zfs_log.c @@ -47,6 +47,10 @@ #include #include +#ifdef __APPLE__ +#include // spa_min_alloc +#endif + /* * These zfs_log_* functions must be called within a dmu tx, in one * of 2 contexts depending on zilog->z_replay: @@ -641,6 +645,9 @@ zfs_log_write(zilog_t *zilog, dmu_tx_t *tx, int txtype, uint64_t gen = 0; ssize_t size = resid; + if (unlikely(blocksize == 0)) + blocksize = SPA_MINBLOCKSIZE; + if (zil_replaying(zilog, tx) || zp->z_unlinked || zfs_xattr_owner_unlinked(zp)) { if (callback != NULL) diff --git a/module/zfs/zfs_vnops.c b/module/zfs/zfs_vnops.c index e491cb3087ca..740c02d03c9a 100644 --- a/module/zfs/zfs_vnops.c +++ b/module/zfs/zfs_vnops.c @@ -92,10 +92,9 @@ static int zfs_dio_enabled = 1; */ static uint64_t zfs_vnops_read_chunk_size = 1024 * 1024; -#ifndef __APPLE__ - static ulong_t zfs_fsync_sync_cnt = 4; +#ifndef __APPLE__ int zfs_fsync(znode_t *zp, int syncflag, cred_t *cr) { diff --git a/module/zfs/zfs_znode.c b/module/zfs/zfs_znode.c index 824db8c689a7..64953523f091 100644 --- a/module/zfs/zfs_znode.c +++ b/module/zfs/zfs_znode.c @@ -218,7 +218,8 @@ zfs_obj_to_path_impl(objset_t *osp, uint64_t obj, sa_handle_t *hdl, component[0] = '/'; if (is_xattrdir) { - strcpy(component + 1, ""); + strlcpy(component + 1, "", + ZAP_MAXNAMELEN_NEW); } else { error = zap_value_search(osp, pobj, obj, ZFS_DIRENT_OBJ(-1ULL), component + 1, diff --git a/scripts/pkg_macos.sh b/scripts/pkg_macos.sh index 6b5e761678fa..0aa3f2593ade 100755 --- a/scripts/pkg_macos.sh +++ b/scripts/pkg_macos.sh @@ -52,6 +52,16 @@ OS=$(sw_vers | awk '{if ($1 == "ProductVersion:") print $2;}') OS=$(echo "$OS" | awk -F . '{if ($1 == 10) print $1"."$2; else print $1}') RC=$(grep Release: META | awk '$2 ~ /rc/ { print $2}') +friendly=$(awk '/SOFTWARE LICENSE AGREEMENT FOR macOS/' '/System/Library/CoreServices/Setup Assistant.app/Contents/Resources/en.lproj/OSXSoftwareLicense.rtf' | awk -F 'macOS ' '{print $NF}' | tr -d '\\') +if [ -z "$friendly" ]; then + friendly=$(awk '/SOFTWARE LICENSE AGREEMENT FOR OS X/' '/System/Library/CoreServices/Setup Assistant.app/Contents/Resources/en.lproj/OSXSoftwareLicense.rtf' | awk -F 'OS X ' '{print $NF}' | awk '{print substr($0, 0, length($0)-1)}') +fi +if [ -z "$friendly" ]; then + friendly=$(awk '/SOFTWARE LICENSE AGREEMENT FOR macOS/' '/System/Library/CoreServices/Setup Assistant.app/Contents/Resources/en.lproj/OSXSoftwareLicense.rtf' | awk -F 'macOS ' '{print $NF}' | awk '{print substr($0, 0, length($0)-1)}') +fi + +friendly=$(echo "$friendly" | tr ' ' '.') + function usage { echo "$0: Create installable pkg for macOS" @@ -59,6 +69,7 @@ function usage echo " Options:" echo " -l skip make install step, using a folder of a previous run" echo " -L copy and fix external libraries" + echo " -A use obsolete altool to notarize" exit } @@ -66,6 +77,7 @@ while getopts "hlL" opt; do case $opt in l ) skip_install=1 ;; L ) fix_libraries=1 ;; + A ) use_altool=1 ;; h ) usage exit 2 ;; @@ -108,9 +120,24 @@ if [ -f "${BASE_DIR}/../config.status" ]; then prefix=$(grep 'S\["prefix"\]' "${BASE_DIR}/../config.status" | tr '=' ' ' | awk '{print $2;}' | tr -d '"') fi +if [ -z "$NOTARYTOOL" ]; then + xcrun notarytool > /dev/null 2>&1 + if [ $? != 0 ]; then + use_altool=1 + else + NOTARYTOOL="$(xcrun -f notarytool)" + fi +fi + echo "Version is $version" echo "Prefix set to $prefix" echo "RC, if set: $RC" +echo "OS name: $friendly" +if [ -n "$use_altool" ]; then + echo "notarize: altool" +else + echo "notarize: notarytool ($NOTARYTOOL)" +fi echo "" sleep 3 @@ -205,7 +232,7 @@ function do_codesign [ x"$OS" == x"10.11" ] || [ x"$OS" == x"10.10" ] || [ x"$OS" == x"10.9" ]; then - extra="" + extra="--signature-size=12000" else extra="--options runtime" fi @@ -267,6 +294,9 @@ function do_prune "./${prefix}/share/zfs-macos/runfiles" \ "./${prefix}/share/zfs-macos/test-runner" \ "./${prefix}/share/zfs-macos/zfs-tests" \ +"./${prefix}/share/zfs/runfiles" \ +"./${prefix}/share/zfs/test-runner" \ +"./${prefix}/share/zfs/zfs-tests" \ "./${prefix}/src" popd || fail "failed to popd" @@ -280,7 +310,7 @@ function do_prune function copy_fix_libraries { echo "Fixing external libraries ... " - fixlib=$(otool -L ${codesign_files} | egrep '/usr/local/opt/|/opt/local/lib/' |awk '{print $1;}' | grep '\.dylib$' | sort | uniq) + fixlib=$(otool -L ${codesign_files} | egrep '/usr/local/opt/|/opt/local/lib/|/opt/local/libexec/' |awk '{print $1;}' | grep '\.dylib$' | sort | uniq) # Add the libs into codesign list - both to be codesigned, and updated # between themselves (libssl depends on libcrypt) @@ -314,7 +344,7 @@ function copy_fix_libraries for file in $codesign_files do chmod u+w "${file}" - src=$(otool -L "$file" | awk '{print $1;}' | grep "${name}.dylib") + src=$(otool -L "$file" | grep -v ":$" | awk '{print $1;}' | grep "${name}.dylib") install_name_tool -change "${src}" "${prefix}/lib/${name}.dylib" "${file}" done done @@ -322,7 +352,7 @@ function copy_fix_libraries # Upload .pkg file # Staple .pkg file -function do_notarize +function do_notarize_altool { echo "Uploading PKG to Apple ..." @@ -361,6 +391,39 @@ function do_notarize } +# Upload .pkg file +# Staple .pkg file +function do_notarize_notarytool +{ + echo "Uploading PKG to Apple ..." + + TFILE="out-altool.xml" + RFILE="req-altool.xml" + # xcrun altool --notarize-app -f my_package_new.pkg --primary-bundle-id org.openzfsonosx.zfs -u lundman@lundman.net -p "$PKG_NOTARIZE_KEY" --output-format xml > ${TFILE} + $NOTARYTOOL submit --wait --apple-id lundman@lundman.net --team-id "735AM5QEU3" --password "$PKG_NOTARIZE_KEY" my_package_new.pkg + + echo "Stapling PKG ..." + xcrun stapler staple my_package_new.pkg + ret=$? + xcrun stapler validate -v my_package_new.pkg + + if [ $ret != 0 ]; then + echo "Failed to notarize: $ret" + grep "https://" ${RFILE} + exit 1 + fi + +} + +function do_notarize +{ + if [ -n "$use_altool" ]; then + do_notarize_altool + else + do_notarize_notarytool + fi +} + echo "Pruning install area ..." do_prune @@ -397,13 +460,6 @@ if [ $ret != 0 ]; then fail "pkgbuild failed" fi -friendly=$(awk '/SOFTWARE LICENSE AGREEMENT FOR macOS/' '/System/Library/CoreServices/Setup Assistant.app/Contents/Resources/en.lproj/OSXSoftwareLicense.rtf' | awk -F 'macOS ' '{print $NF}' | tr -d '\\') -if [ -z "$friendly" ]; then - friendly=$(awk '/SOFTWARE LICENSE AGREEMENT FOR OS X/' '/System/Library/CoreServices/Setup Assistant.app/Contents/Resources/en.lproj/OSXSoftwareLicense.rtf' | awk -F 'OS X ' '{print $NF}' | awk '{print substr($0, 0, length($0)-1)}') -fi - -friendly=$(echo "$friendly" | tr ' ' '.') - # Now fiddle with pkg to make it nicer productbuild --synthesize --package ./my_package.pkg distribution.xml diff --git a/scripts/zfs-tests.sh b/scripts/zfs-tests.sh index dbeae14d946f..f4031545b078 100755 --- a/scripts/zfs-tests.sh +++ b/scripts/zfs-tests.sh @@ -650,6 +650,29 @@ if [ "$UNAME" = "Darwin" ]; then # BigSur gets even harder. sudo /sbin/mount -uw / export SHELL=ksh + # We can guess common pool names used by the testers, + # and add to synthetic + if [ ! -f "/etc/synthetic.d/zfs-tests" ]; then + sudo mkdir -p /etc/synthetic.d + sudo touch /etc/synthetic.conf + sudo touch /etc/synthetic.d/zfs-tests + sudo chmod 666 /etc/synthetic.d/zfs-tests + cat << EOF > /etc/synthetic.d/zfs-tests +testpool +testpool1 +testpool2 +logsm_import +lgcypool +ldnpool +zonepool +perfpool +EOF + sudo chmod 444 /etc/synthetic.d/zfs-tests + echo "" + echo "Please reboot to activate /etc/synthetic.d/zfs-tests" + echo "" + exit 0 + fi fi # # Verify the ZFS module stack is loaded. diff --git a/tests/zfs-tests/include/libtest.shlib b/tests/zfs-tests/include/libtest.shlib index beed55304ff4..c4af06ed66ff 100644 --- a/tests/zfs-tests/include/libtest.shlib +++ b/tests/zfs-tests/include/libtest.shlib @@ -304,7 +304,7 @@ function default_setup_noexit if poolexists $TESTPOOL ; then destroy_pool $TESTPOOL fi - [[ -d /$TESTPOOL ]] && rm -rf /$TESTPOOL + [[ -d /$TESTPOOL ]] && rm -f /$TESTPOOL log_must zpool create -f $TESTPOOL $disklist else reexport_pool @@ -506,7 +506,7 @@ function default_mirror_setup_noexit log_fail "$func: No parameters passed" [[ -z $secondary ]] && \ log_fail "$func: No secondary partition passed" - [[ -d /$TESTPOOL ]] && rm -rf /$TESTPOOL + [[ -d /$TESTPOOL ]] && rm -f /$TESTPOOL log_must zpool create -f $TESTPOOL mirror $@ log_must zfs create $TESTPOOL/$TESTFS log_must zfs set mountpoint=$TESTDIR $TESTPOOL/$TESTFS @@ -543,7 +543,7 @@ function default_raidz_setup_noexit log_fail "A raid-z requires a minimum of two disks." fi - [[ -d /$TESTPOOL ]] && rm -rf /$TESTPOOL + [[ -d /$TESTPOOL ]] && rm -f /$TESTPOOL log_must zpool create -f $TESTPOOL raidz $disklist log_must zfs create $TESTPOOL/$TESTFS log_must zfs set mountpoint=$TESTDIR $TESTPOOL/$TESTFS @@ -1697,7 +1697,7 @@ function create_pool #pool devs_list fi if is_global_zone ; then - [[ -d /$pool ]] && rm -rf /$pool + [[ -d /$pool ]] && rm -f /$pool log_must zpool create -f $pool $@ fi @@ -1731,7 +1731,7 @@ function destroy_pool #pool log_must_busy zpool destroy -f $pool [[ -d $mtpt ]] && \ - log_must rm -rf $mtpt + rm -f $mtpt else log_note "Pool does not exist. ($pool)" return 1 @@ -1789,7 +1789,7 @@ function destroy_dataset # dataset [args] mtpt=$(get_prop mountpoint "$dataset") log_must_busy zfs destroy $args $dataset - [ -d $mtpt ] && log_must rm -rf $mtpt + [ -d $mtpt ] && rm -f $mtpt else log_note "Dataset does not exist. ($dataset)" return 1 diff --git a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.kshlib b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.kshlib index 54f805ea71a4..347829102d51 100644 --- a/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.kshlib +++ b/tests/zfs-tests/tests/functional/cli_root/zpool_export/zpool_export.kshlib @@ -27,6 +27,6 @@ function zpool_export_cleanup { - [[ -d $TESTDIR0 ]] && log_must rm -rf $TESTDIR0 default_cleanup + [[ -d $TESTDIR0 ]] && log_must rm -rf $TESTDIR0 } diff --git a/tests/zfs-tests/tests/functional/replacement/replace_rebuild.ksh b/tests/zfs-tests/tests/functional/replacement/replace_rebuild.ksh index ea1fabdbbeb9..77fcc94b63ce 100755 --- a/tests/zfs-tests/tests/functional/replacement/replace_rebuild.ksh +++ b/tests/zfs-tests/tests/functional/replacement/replace_rebuild.ksh @@ -148,7 +148,7 @@ for type in "" "mirror" "draid"; do log_must eval "zpool iostat -v $TESTPOOL1 | grep \"$REPLACEFILE\"" destroy_pool $TESTPOOL1 - log_must rm -rf /$TESTPOOL1 + rm -f /$TESTPOOL1 done done diff --git a/tests/zfs-tests/tests/functional/replacement/replace_resilver.ksh b/tests/zfs-tests/tests/functional/replacement/replace_resilver.ksh index bdca3c772584..9fc2f4c16aeb 100755 --- a/tests/zfs-tests/tests/functional/replacement/replace_resilver.ksh +++ b/tests/zfs-tests/tests/functional/replacement/replace_resilver.ksh @@ -145,7 +145,7 @@ for type in "" "raidz" "mirror" "draid"; do log_must eval "zpool iostat -v $TESTPOOL1 | grep \"$REPLACEFILE\"" destroy_pool $TESTPOOL1 - log_must rm -rf /$TESTPOOL1 + rm -f /$TESTPOOL1 done done