Ensure Compatibility with BSD sha1sum implementation

[otteryc]

Recent macOS updates introduced /sbin/sha1sum, which is not GNU's sha1sum and is incompatible with its implementation. This causes the sha1sum test in make artifact to always pass because the test currently relies on detecting the string "FAILED" in the stderr output.

Apple's sha1sum does not support the -c (checksum verification) option, making it unsuitable for verifying artifact integrity. Below is a demonstration of the differences between Apple's and GNU's sha1sum:

MACOS$ echo "CONTENT" > file
MACOS$ sha1sum file
7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file
MACOS$ echo "NOT ORIGINAL CONTENT" > file
MACOS$ echo "7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file" | sha1sum -c
usage: sha1sum [-bctwz] [files ...]
MACOS$ sha1sum --version
sha1sum (Darwin) 1.0
MACOS$ gsha1sum --version # by brew install coreutils
gsha1sum --version
sha1sum (GNU coreutils) 9.6
Copyright (C) 2025 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Ulrich Drepper, Scott Miller, and David Madore.

LINUX$ echo "CONTENT" > file
LINUX$ sha1sum file
7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file
LINUX$ echo "NOT ORIGINAL CONTENT" > file
LINUX$ echo "7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file" | sha1sum -c
file: FAILED
sha1sum: WARNING: 1 computed checksum did NOT match
LINUX$ sha1sum --version
sha1sum (GNU coreutils) 9.4
Copyright (C) 2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Ulrich Drepper, Scott Miller, and David Madore.

[jserv]

You can explicitly specify - in sha1sum on macOS. That is,

$ echo "7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file" | sha1sum -c -

It works on both GNU/Linux and macOS.

[jserv]

Quote from Wikipedia:

In 2016, with the release of macOS 10.12 Sierra, the name was changed from OS X to macOS with the purpose of aligning it with the branding of Apple's other primary operating systems: iOS, watchOS, and tvOS.

Here, refer to "macOS" instead of "MacOS."

[otteryc]

You can explicitly specify - in sha1sum on macOS. That is,

$ echo "7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file" | sha1sum -c -

It works on both GNU/Linux and macOS.

Specifying - in sha1sum works on both GNU/Linux and macOS for existing files, however the behavior is still not identical when the file does not exist. Here's an example:

MACOS$ echo "CONTENT" > file
MACOS$ sha1sum file
7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file
MACOS$ rm file
MACOS$ echo "7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file" | sha1sum -c -
sha1sum:  file: No such file or directory

LINUX$ sha1sum file
7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file
LINUX$ rm file
LINUX$ echo "7a2ed3b06b4ecf87d3a31966d8c17a9afb6634d9  file" | sha1sum -c
sha1sum: file: No such file or directory
file: FAILED open or read
sha1sum: WARNING: 1 listed file could not be read

As shown above, when the file does not exist, sha1sum outputs "No such file or directory", but the "FAILED" string is not printed on macOS.

[jserv]

Specifying - in sha1sum works on both GNU/Linux and macOS for existing files, however the behavior is still not identical when the file does not exist.

We can always check the existence with shell commands, and that is why GNU's and FreeBSD's (which macOS takes from) implementations have different perspectives: GNU coreutils comes with various powerful features while FreeBSD's implementation takes conservative approaches and prioritizes on robustness. In this case, switching to gsha1sum involves additional documentation and corresponding checks for people using FreeBSD and macOS.

[otteryc]

We can always check the existence with shell commands, and that is why GNU's and FreeBSD's (which macOS takes from) implementations have different perspectives: GNU coreutils comes with various powerful features while FreeBSD's implementation takes conservative approaches and prioritizes on robustness. In this case, switching to gsha1sum involves additional documentation and corresponding checks for people using FreeBSD and macOS.

Thank you for the insightful comment!
Given the scenario, I would like to know your preference regarding the following patch or the alternative approach of completing the documentation and adding corresponding checks for FreeBSD and macOS users.

diff --git a/mk/external.mk b/mk/external.mk
index 05f5576..bbf91aa 100644
--- a/mk/external.mk
+++ b/mk/external.mk
@@ -68,7 +68,7 @@ define verify
                 | sort \
                 | $(SHA1SUM) \
                 | cut -f 1 -d ' ' > $(SHA1_FILE2) && cmp $(SHA1_FILE1) $(SHA1_FILE2))), \
-            ($(eval VERIFIER := echo "$(strip $(1))  $(strip $(2))" | $(SHA1SUM) -c)) \
+            ($(eval VERIFIER := (ls $(2) >/dev/null 2>&1 || echo FAILED) && echo "$(strip $(1))  $(strip $(2))" | $(SHA1SUM) -c -)) \
     ))
     $(eval _ := $(shell $(VERIFIER) 2>&1))

[jserv]

I would like to know your preference regarding the following patch or the alternative approach of completing the documentation and adding corresponding checks for FreeBSD and macOS users.

Upon careful review of the rv32emu documentation, you will find that using 'brew' is optional. macOS and FreeBSD users can build from source to access minimal features. This flexibility is crucial for a small project like rv32emu, as users should not need to constantly reference documentation. Regarding sha1sum compatibility, different implementations can be used straightforwardly based on their behaviors.

[jserv]

Thank @otteryc for contributing!