Lua 5.4.1 #65993

carlocab · 2020-12-01T12:33:32Z

This is another attempt at #65976. See also #57133.

Apologies for starting from scratch at this one, but I think I messed up
one of my previous commits in the earlier PR and I don't have enough
git-fu to fix it.

Build Failures (10.14, 10.15)

Build Failures (Big Sur)

hyperkit fails only on Big Sur. See #65000

To revisit?

These formulae still depend on [email protected]. Possible patches that will allow upgrading to [email protected] or alternatives to this in parentheses.

corsixth ([RDY] lua 5.4 support CorsixTH/CorsixTH#1686)
~~instead (replace [email protected] with luajit, this seems to be preferred: docs)~~
lsyncd
~~osrm-backend (Bugfix Lua 5.4 not working Project-OSRM/osrm-backend#5783)~~
~~sile (sile --version error: "Please install a bit library" sile-typesetter/sile#1115, perhaps with https://luarocks.org/modules/siffiejoe/bit32)~~

[email protected] is currently keg_only, unlike [email protected]. If this is undesirable, its executables will need to be renamed to avoid conflict with lua.

On luarocks:

~~It still has hardcoded version information in the tests. (this probably won't cause problems for a while though, if at all.)~~
~~It lacks a test for [email protected], but has one for [email protected]. I tried copying the old test, but couldn't get it to work.~~

lua patches:
They've been around for ages. Would be good to know if they're still needed and, if they are, to migrate them to Homebrew/formula-patches.

I'll pursue these in separate PRs once this current one is merged.

fxcoudert · 2020-12-01T12:46:58Z

OK got it. In formula_renames.json, remove this line:

  "[email protected]": "lua",

Formula/lua.rb

SMillerDev · 2020-12-01T13:48:39Z

Formula/lua.rb

@@ -93,68 +92,68 @@ def caveats
  end

  test do
-    system "#{bin}/lua", "-e", "print ('Ducks are cool')"
+    assert_match "Homebrew is awesome!", shell_output("#{bin}/lua -e \"print ('Homebrew is awesome!')\"")
  end
 end

 __END__


It seems like this patch will be around for a while, is it submitted upstream already? And/or can we migrate it to Homebrew/formula-patches?

This patch has been around for quite a while: Homebrew/legacy-homebrew#5043

I think it's somewhat unique to the way Homebrew packages things? Though I don't see how it could hurt upstream, and probably only helps.

I don't mind migrating it to Homebrew/formula-patches. Do you mind if I pursue that in a separate PR, if (fingers crossed) this one gets merged?

I only just realised exactly how old this patch is. It's really alarming.

I'm going to try to see if I can track down someone familiar with lua internals who can have a look at the patch.

fxcoudert · 2020-12-01T14:11:09Z

So, weirdly enough, out of the 32 formulas that depend on lua, only 23 have broken linkage:

brew linkage --test clingo
brew linkage --test corsixth
brew linkage --test freeswitch
brew linkage --test gnuplot
brew linkage --test highlight
brew linkage --test imapfilter
brew linkage --test instead
brew linkage --test kyua
brew linkage --test lcm
brew linkage --test lsyncd
brew linkage --test lutok
brew linkage --test macvim
brew linkage --test neomutt
brew linkage --test onscripter
brew linkage --test osrm-backend
brew linkage --test pdns
brew linkage --test pdnsrec
brew linkage --test rpm
brew linkage --test sile
brew linkage --test tracebox
brew linkage --test vim
brew linkage --test vis
brew linkage --test weechat

Does this mean that the others (fennel kyoto-tycoon lmod luarocks ntopng osm2pgsql widelands wordgrinder z.lua) do not actually have a runtime dependency on lua? If not, they should be made build-only dependencies.

Regarding the others, they need to be revision bumped (e.g., brew bump-revision clingo --message "for lua")

carlocab · 2020-12-01T14:15:16Z

Ok, I'm going to try a revision bump on all the test failures (only for 10.14 for now, will work on any discrepancies with later versions after).

If that doesn't work, then I'm going to switch their dependency to [email protected]. (Hopefully, my version bump to 5.3.6 didn't break anything.)

Let me know if there's actually a better direction I should go.

fxcoudert · 2020-12-01T14:25:35Z

I'm going to try a revision bump on all the test failures

Only the linkage failures, for now, please. Some test failures could be dependents that don't actually need to be revision-bumped.

If that doesn't work, then I'm going to switch their dependency to [email protected]

Yes.

Let me know if there's actually a better direction I should go

While testing proceeds, we need to understand the 9 formulas that don't link to lua: do they really have a runtime dependency, or could it be a build dependency?

gromgit · 2020-12-01T15:20:14Z

A quick check of Lua usage in the "unbroken linkage" fomulae:

fennel: static linking
kyoto-tycoon: static linking
lmod: scripts
luarocks: scripts
ntopng: static linking
osm2pgsql: static linking
widelands: static linking
wordgrinder: static linking
z.lua: script

All "static linking" formulae can probably be converted to requiring Lua at build-time only. I can take care of this.

fxcoudert · 2020-12-01T15:37:28Z

Thanks @gromgit
The static linking cases need to be made build-only, and revision-bumped. The scripts do not require any change.

gromgit · 2020-12-01T15:39:10Z

All "static linking" formulae can probably be converted to requiring Lua at build-time only. I can take care of this.

@fxcoudert , should my work be folded into this effort, or can it be done in separate PRs? I ask because out of the 6 targeted formulae, only two (fennel and wordgrinder) have meaningful test cases, so this will take a while. (kyoto-tycoon doesn't have a test block at all.)

Formula/lua.rb

SMillerDev · 2020-12-01T18:21:01Z

Error: GitHub API Error: API rate limit exceeded for 207.254.26.156. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)
Try again in 1 minute 28 seconds, or create a personal access token:
https://github.com/settings/tokens/new?scopes=gist,public_repo&description=Homebrew
echo 'export HOMEBREW_GITHUB_API_TOKEN=your_token_here' >> ~/.zshrc

Yeah, that happens sometimes. Probably the next run will be better.

carlocab · 2020-12-01T19:05:24Z

Some related open PRs:

~~#66004 gnuplot (revision bump here)~~
#61931 rpm (revision bump here)
#65843 tracebox (revision bump ~~+ depends on "[email protected]" here~~)

alerque · 2020-12-01T19:24:42Z

@carlocab SILE should be ready to fly with Lua 5.4. We test every release it on a full Lua matrix and I'm running it on Lua 5.4 myself (on Arch Linux). All its respective Rock dependencies are likewise 5.4 compatible.

I don't see the actual failure log so this is a stab in the dark here, but from @fxcoudert's comment above:

Does this mean that the others (fennel kyoto-tycoon lmod luarocks ntopng osm2pgsql widelands wordgrinder z.lua) do not actually have a runtime dependency on lua? If not, they should be made build-only dependencies.

Could it be that the linkage detector doesn't understand what's going on here? There are two main types of Lua modules: plain Lua code ones that are scripts parsed at runtime and C ones that are compiled into binaries. The linkage check might only be funding the latter, but many projects that fall into the former still need to know at build time what version of Lua they are going to be running under because they make language and path adjustments to match. They need Lua at run time, but they also need to know some things about it at build time.

SILE for example has differt dependencies depending on the Lua version. Lua 5.2 for example provides a bit32 module built in, while 5.1, 5.3, and 5.4 need it to be included as an independent module.

LuaRocks also does some path stuff differently depending on what the runtime environment is expected to be.

fxcoudert · 2020-12-01T20:54:22Z

@carlocab for the two crake-based projects that cannot find [email protected], you need to set the variables LUA_INCLUDE_DIR and LUA_LIBRARIES, I think: https://cmake.org/cmake/help/latest/module/FindLua.html

See how wireshark does this for [email protected]

gromgit · 2020-12-03T15:10:16Z

I made a mistake with the final formula on my list (fennel). It's actually a gargantuan Lua script, and it passed CI here, so I'm skipping it.

Also add proper test. Followup to #65993. Closes #66152. Signed-off-by: Sean Molenaar <[email protected]> Signed-off-by: BrewTestBot <[email protected]>

carlocab · 2020-12-03T20:30:01Z

@gromgit at least you don't have mistakes that got merged... #66186

Follows up Homebrew#65993

Follows up #65993 Closes #66287. Signed-off-by: FX Coudert <[email protected]>

Follow up to Homebrew#65993. See also lsyncd/lsyncd#502. I've fixed a bunch of configuration and compile errors, but I'm left with the following linker error: [100%] Linking C executable lsyncd /usr/local/Cellar/cmake/3.19.1/bin/cmake -E cmake_link_script CMakeFiles/lsyncd.dir/link.txt --verbose=1 /usr/local/Homebrew/Library/Homebrew/shims/mac/super/clang -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.0.sdk -Wl,-search_paths_first -Wl,-headerpad_max_install_names CMakeFiles/lsyncd.dir/lsyncd.c.o CMakeFiles/lsyncd.dir/runner.c.o CMakeFiles/lsyncd.dir/defaults.c.o CMakeFiles/lsyncd.dir/fsevents.c.o -o lsyncd /usr/local/opt/lua/lib/liblua.5.4.dylib Undefined symbols for architecture x86_64: "_lua_objlen", referenced from: _l_exec in lsyncd.c.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[2]: *** [lsyncd] Error 1 make[1]: *** [CMakeFiles/lsyncd.dir/all] Error 2 make: *** [all] Error 2 Does anyone have any ideas?

Follow up to Homebrew#65993

Follow up to #65993 Closes #66407. Signed-off-by: FX Coudert <[email protected]> Signed-off-by: BrewTestBot <[email protected]>

BrewTestBot added the automerge-skip `brew pr-automerge` will skip this pull request label Dec 1, 2020

SMillerDev reviewed Dec 1, 2020

View reviewed changes