Mac M1 build support proposal (minus rocksdb option) #1743

devnexen · 2021-01-23T17:51:27Z

No description provided.

robertbindar · 2021-01-26T11:24:50Z

Hi @devnexen! Thanks a lot for this patch, I was really curious how we're doing in terms of compiling on the new M1 macs. Can you please give me a bit more details about what this patch is fixing? Is it some compile errors?

devnexen · 2021-01-26T11:45:22Z

A little compiling error indeed but mostly crc32 intrinsics support detection in addition

cvicentiu · 2021-01-26T13:00:51Z

I had a look at the patch and it looks safe to me. We are due for an M1 mac soon, so we'll be able to provide CI support for it when that happens. For now, I cross checked it with this discussion here:
google/cpu_features#121

We'll be able to test this probably after the 10.5.9 release, but I think this patch is safe to merge regardless. I'll let it up to @robertbindar to see if he can double check with an M1 mac today.

an3l · 2021-01-26T13:06:45Z

@robertbindar during the testing of the patch, please note that PR #1730 is also adding similar change, so if the changes are ok, I suggest to first merge PR #1730 and after this patch.

devnexen · 2021-01-26T13:12:48Z

Note I also applied a change in rocksdb facebook/rocksdb#7893 and seemingly had been accepted

robertbindar · 2021-01-28T13:22:57Z

Hey @devnexen! I'm not familiar with this area of code, but I trust your expertise on this and on top of that, the code added here seems to be enclosed in the right ifdefs to not have potential conflicts with other areas of the server. I'll give the patch a quick test run to see builds on other platforms are indeed not influenced and then happily merge it.

Is the server building clean on your M1 with this patch on?

devnexen · 2021-01-28T13:32:42Z

yes indeed it does build.

lonnylot · 2021-01-29T01:02:26Z

Hey @robertbindar I tried to build this PR and I couldn't. Here is my output from cmake ../server: https://gist.github.com/lonnylot/4bbff8ec6fcfb5f28304d59335b95af2

I don't know enough about the MariaDB dev env. to really debug it, but I just wanted to share. Let me know if this is something obvious to fix and I'll be happy to try again.

robertbindar · 2021-01-29T08:59:08Z

Hey @lonnylot , thanks for working on this. It seems to me that cmake cannot find libiconv, do you have it installed on your system?
Edit: not sure if this is its name on osx too though

lonnylot · 2021-01-29T12:02:36Z

So I have iconv:

lonnykapelushnik@Lonnys-MBP server % which iconv
/usr/bin/iconv
lonnykapelushnik@Lonnys-MBP server % iconv --version
iconv (GNU libiconv 1.11)
Copyright (C) 2000-2006 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Bruno Haible.

Should this satisfy the requirement? (Sorry, I haven't compiled anything in awhile and really don't recall all of the nuances)

devnexen · 2021-01-29T12:07:26Z

You need to make sure your environment points to the right location of the dependencies (by that I mean not to confuse dependencies built for Rosetta vs the ones for ARM). Here an example :

echo $CPPFLAGS -I/opt/homebrew/include -I/opt/homebrew/include/harfbuzz -I/opt/homebrew/opt/openssl/include -I/opt/homebrew/opt/lapack/include -I/opt/homebrew/opt/libiconv/include ... ... other headers location you might need ... echo $LDFLAGS -L/opt/homebrew/lib -Wl,-rpath,/opt/homebrew/lib -L/opt/homebrew/opt/llvm/lib -L/opt/homebrew/opt/openssl/lib -L/opt/homebrew/opt/lapack/lib -L/opt/homebrew/opt/libiconv/lib ... other libraries you might need

lonnylot · 2021-01-30T07:09:53Z

@devnexen Any idea how I could tell which are built for ARM vs. Rosetta?

I brew install libiconv and set them in those two env. vars, but still same error. So I'm guessing it's coming from another library/app that's for Rosetta?

lonnykapelushnik@Lonnys-MBP server % echo $CPPFLAGS                                                      
-I/opt/homebrew/opt/libiconv/include
lonnykapelushnik@Lonnys-MBP server % echo $LDFLAGS                                                       
-L/opt/homebrew/opt/libiconv/lib

lonnykapelushnik@Lonnys-MBP server % cmake ../server
...

-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
ICONV_LIBRARIES (ADVANCED)
    linked by target "mariadbclient" in directory /Users/lonnykapelushnik/Development/server/libmariadb/libmariadb
    linked by target "libmariadb" in directory /Users/lonnykapelushnik/Development/server/libmariadb/libmariadb

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

devnexen · 2021-01-30T07:41:43Z

@devnexen Any idea how I could tell which are built for ARM vs. Rosetta?

Yes homebrew for Rosetta installs in the same location just like the older Mac Intel ie /usr/local whereas for ARM it is into /opt/homebrew

I brew install libiconv and set them in those two env. vars, but still same error. So I'm guessing it's coming from another library/app that's for Rosetta?

lonnykapelushnik@Lonnys-MBP server % echo $CPPFLAGS                                                      
-I/opt/homebrew/opt/libiconv/include
lonnykapelushnik@Lonnys-MBP server % echo $LDFLAGS                                                       
-L/opt/homebrew/opt/libiconv/lib

lonnykapelushnik@Lonnys-MBP server % cmake ../server
...

-- Configuring done
CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
ICONV_LIBRARIES (ADVANCED)
    linked by target "mariadbclient" in directory /Users/lonnykapelushnik/Development/server/libmariadb/libmariadb
    linked by target "libmariadb" in directory /Users/lonnykapelushnik/Development/server/libmariadb/libmariadb

-- Generating done
CMake Generate step failed.  Build files cannot be regenerated correctly.

Try to set CXXFLAGS with same values as CPPFLAGS then. You might need more than just libiconv path tough

lonnylot · 2021-01-30T14:27:35Z

Wow - I was on the wrong branch. Sorry for all the noise.

But now I'm on the mac_m1_support_upd branch and make results in the below error. I'm guessing it's related to facebook/rocksdb#7714. I'll have to look into it later though.

[ 83%] Building CXX object storage/rocksdb/CMakeFiles/rocksdblib.dir/rocksdb/util/crc32c.cc.o
/Users/lonnykapelushnik/Development/server/storage/rocksdb/rocksdb/util/crc32c.cc:500:18: error: use of undeclared identifier 'isSSE42'
  has_fast_crc = isSSE42();
                 ^
/Users/lonnykapelushnik/Development/server/storage/rocksdb/rocksdb/util/crc32c.cc:1230:7: error: use of undeclared identifier 'isSSE42'
  if (isSSE42()) {
      ^
/Users/lonnykapelushnik/Development/server/storage/rocksdb/rocksdb/util/crc32c.cc:1231:9: error: use of undeclared identifier 'isPCLMULQDQ'
    if (isPCLMULQDQ()) {
        ^
3 errors generated.
make[2]: *** [storage/rocksdb/CMakeFiles/rocksdblib.dir/rocksdb/util/crc32c.cc.o] Error 1
make[1]: *** [storage/rocksdb/CMakeFiles/rocksdblib.dir/all] Error 2

devnexen · 2021-01-30T14:35:32Z

Yes I believe the git hash in here is pretty old https://github.com/facebook/rocksdb/tree/bba5e7bc21093d7cfa765e1280a7c4fdcd284288
some changes went through since as you noticed and another should land later maybe
facebook/rocksdb#7893

robertbindar · 2021-01-30T14:45:17Z

Hey @devnexen and @lonnylot! I've received my m1 this morning and I managed to build this patch with no problems.

Unfortunately @lonnylot I probably won't be able to help you much to set up your dev environment, I'm very newbie in this environment, my process seems to have worked pretty easy (installed Xcode and macports, installed MariaDB dependencies with macports, then just ran a cmake and make. We can try to figure it out together if you're still interested to get a working setup.

@devnexen I'll merge PR #1730 first as it was created quite a long time ago, and although it is a subset of yours, I think the contributor still deserves GitHub credits for the effort. Can you please rebase against 10.5 once I merged #1730? Thanks a lot, if you don't have the time, I can do the rebase myself no worries.

robertbindar · 2021-01-30T15:07:42Z

Thanks a lot @devnexen and @lonnylot for helping me get these patches in time for 10.5.9! Great work and see you around 🎉

grooverdan · 2021-02-04T01:22:31Z

rocksdb note - found http://lists.askmonty.org/pipermail/commits/2021-February/014468.html though its not on any branch yet.
The rocksdb commit facebook/rocksdb@cdd8b09 is a Apple Silicon fix. So maybe not too much longer there.

robertbindar self-assigned this Jan 26, 2021

an3l mentioned this pull request Jan 26, 2021

Apple Silicon is a 64-bit platform #1730

Closed

Mac M1 build support proposal (minus rocksdb option)

3a98fbc

robertbindar merged commit b124158 into MariaDB:10.5 Jan 30, 2021

fxcoudert mentioned this pull request Jan 30, 2021

mariadb: fix for ARM Homebrew/homebrew-core#70060

Closed

jdblischak mentioned this pull request Apr 25, 2024

Remove conda-build overdepending warnings conda-forge/libtiledb-sql-feedstock#161

Merged

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mac M1 build support proposal (minus rocksdb option) #1743

Mac M1 build support proposal (minus rocksdb option) #1743

devnexen commented Jan 23, 2021

robertbindar commented Jan 26, 2021

devnexen commented Jan 26, 2021

cvicentiu commented Jan 26, 2021

an3l commented Jan 26, 2021

devnexen commented Jan 26, 2021

robertbindar commented Jan 28, 2021

devnexen commented Jan 28, 2021

lonnylot commented Jan 29, 2021

robertbindar commented Jan 29, 2021 •

edited

Loading

lonnylot commented Jan 29, 2021

devnexen commented Jan 29, 2021 •

edited

Loading

lonnylot commented Jan 30, 2021

devnexen commented Jan 30, 2021 •

edited

Loading

lonnylot commented Jan 30, 2021

devnexen commented Jan 30, 2021

robertbindar commented Jan 30, 2021

robertbindar commented Jan 30, 2021

grooverdan commented Feb 4, 2021

Mac M1 build support proposal (minus rocksdb option) #1743

Mac M1 build support proposal (minus rocksdb option) #1743

Conversation

devnexen commented Jan 23, 2021

robertbindar commented Jan 26, 2021

devnexen commented Jan 26, 2021

cvicentiu commented Jan 26, 2021

an3l commented Jan 26, 2021

devnexen commented Jan 26, 2021

robertbindar commented Jan 28, 2021

devnexen commented Jan 28, 2021

lonnylot commented Jan 29, 2021

robertbindar commented Jan 29, 2021 • edited Loading

lonnylot commented Jan 29, 2021

devnexen commented Jan 29, 2021 • edited Loading

lonnylot commented Jan 30, 2021

devnexen commented Jan 30, 2021 • edited Loading

lonnylot commented Jan 30, 2021

devnexen commented Jan 30, 2021

robertbindar commented Jan 30, 2021

robertbindar commented Jan 30, 2021

grooverdan commented Feb 4, 2021

robertbindar commented Jan 29, 2021 •

edited

Loading

devnexen commented Jan 29, 2021 •

edited

Loading

devnexen commented Jan 30, 2021 •

edited

Loading