Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

symbol not found #190

Closed
xiuliren opened this issue Nov 4, 2021 · 15 comments
Closed

symbol not found #190

xiuliren opened this issue Nov 4, 2021 · 15 comments

Comments

@xiuliren
Copy link

xiuliren commented Nov 4, 2021 •
edited
Loading

it is probably some library version conflict in my environment.
When I installed z5py, libboost 1.77 was installed.
has anyone seen this error and has a solution?

import z5py
  File "/Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/__init__.py", line 1, in <module>
    from .file import File, N5File, ZarrFile, S3File
  File "/Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/file.py", line 6, in <module>
    from . import _z5py
ImportError: dlopen(/Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so, 2): Symbol not found: __ZN5boost10filesystem4path25m_path_iterat
or_incrementERNS1_8iteratorE
  Referenced from: /Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so
  Expected in: /Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/../../../libboost_filesystem.dylib
 in /Users/jwu/opt/anaconda3/envs/cf/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so
@jakirkham
Copy link

Would suggest including enough info to explain how things were installed so someone else can reproduce. If Conda, conda list along with commands and channel used. Similar info for other install mechanisms

@xiuliren
Copy link
Author

xiuliren commented Nov 4, 2021 •
edited
Loading

this is the output of conda info:

conda info

     active environment : cf
    active env location : /Users/jwu/opt/anaconda3/envs/cf
            shell level : 2
       user config file : /Users/jwu/.condarc
 populated config files : /Users/jwu/.condarc
          conda version : 4.10.3
    conda-build version : 3.18.11
         python version : 3.8.3.final.0
       virtual packages : __osx=10.16=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /Users/jwu/opt/anaconda3  (writable)
      conda av data dir : /Users/jwu/opt/anaconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/osx-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/osx-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/osx-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /Users/jwu/opt/anaconda3/pkgs
                          /Users/jwu/.conda/pkgs
       envs directories : /Users/jwu/opt/anaconda3/envs
                          /Users/jwu/.conda/envs
               platform : osx-64
             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.8.3 Darwin/20.6.0 OSX/10.16
                UID:GID : 1647:20
             netrc file : None
           offline mode : False

I removed the whole conda environment and reinstalled z5py, still got the same error. I used python 3.8 this time. These are the packages I got.

 Package           Version  Build              Channel                  Size
───────────────────────────────────────────────────────────────────────────────
  Install:
───────────────────────────────────────────────────────────────────────────────

  + blosc            1.21.0  he49afe7_0         conda-forge/osx-64     920 KB
  + boost-cpp        1.77.0  hff03dee_0         conda-forge/osx-64      16 MB
  + libblas           3.9.0  12_osx64_openblas  conda-forge/osx-64      12 KB
  + libcblas          3.9.0  12_osx64_openblas  conda-forge/osx-64      12 KB
  + libgfortran       5.0.0  9_3_0_h6c81a4c_23  conda-forge/osx-64      19 KB
  + libgfortran5      9.3.0  h6c81a4c_23        conda-forge/osx-64       2 MB
  + liblapack         3.9.0  12_osx64_openblas  conda-forge/osx-64      12 KB
  + libopenblas      0.3.18  openmp_h3351f45_0  conda-forge/osx-64       9 MB
  + llvm-openmp      12.0.1  hda6cdc1_1         conda-forge/osx-64     287 KB
  + numpy            1.21.3  py38h49b9922_1     conda-forge/osx-64       6 MB
  + z5py              2.0.9  py38h2720e7c_0     conda-forge/osx-64     619 KB

  Change:
───────────────────────────────────────────────────────────────────────────────

  - certifi       2021.10.8  py38hecd8cb5_0     installed
  + certifi       2021.10.8  py38h50d1736_1     conda-forge/osx-64     145 KB
  - openssl          1.1.1l  h9ed2024_0         installed
  + openssl          1.1.1l  h0d85af4_0         conda-forge/osx-64       2 MB

  Summary:

  Install: 11 packages
  Change: 2 packages

z5py used to work in my laptop with python 3.7. I need to upgrade to 3.8 or 3.9 since another package used new features of python.

@jakirkham
Copy link

Also please include conda list

@xiuliren
Copy link
Author

xiuliren commented Nov 5, 2021 

# packages in environment at /Users/jwu/opt/anaconda3/envs/cf:
#
# Name                    Version                   Build  Channel
attrs                     21.2.0                   pypi_0    pypi
blosc                     1.21.0               he49afe7_0    conda-forge
boost-cpp                 1.77.0               hff03dee_0    conda-forge
boto3                     1.19.10                  pypi_0    pypi
botocore                  1.22.11                  pypi_0    pypi
brotli                    1.0.9                    pypi_0    pypi
brotlipy                  0.7.0           py38h96a0964_1003    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
c-ares                    1.18.1               h0d85af4_0    conda-forge
ca-certificates           2021.10.26           hecd8cb5_2
cachetools                4.2.4                    pypi_0    pypi
certifi                   2021.10.8        py38h50d1736_1    conda-forge
cffi                      1.14.6           py38h2125817_0
chardet                   4.0.0            py38h50d1736_1    conda-forge
charset-normalizer        2.0.0              pyhd8ed1ab_0    conda-forge
chunkflow                 1.0.8                     dev_0    <develop>
click                     8.0.3                    pypi_0    pypi
cloud-files               4.3.0                    pypi_0    pypi
cloud-volume              5.3.2                    pypi_0    pypi
colorama                  0.4.4              pyh9f0ad1d_0    conda-forge
compressed-segmentation   2.0.1                    pypi_0    pypi
compresso                 2.3.1                    pypi_0    pypi
conda                     4.10.3           py38h50d1736_3    conda-forge
conda-package-handling    1.7.3            py38h96a0964_1    conda-forge
connected-components-3d   3.3.0                    pypi_0    pypi
crc32c                    2.2.post0                pypi_0    pypi
cryptography              35.0.0           py38h56c4533_2    conda-forge
cycler                    0.11.0                   pypi_0    pypi
deflate                   0.3.0                    pypi_0    pypi
dijkstra3d                1.12.0                   pypi_0    pypi
dill                      0.3.4                    pypi_0    pypi
docutils                  0.18                     pypi_0    pypi
dracopy                   0.0.19                   pypi_0    pypi
edt                       2.1.1                    pypi_0    pypi
fasteners                 0.16.3                   pypi_0    pypi
fastremap                 1.12.2                   pypi_0    pypi
fill-voids                2.0.2                    pypi_0    pypi
fpzip                     1.1.5                    pypi_0    pypi
gevent                    21.8.0                   pypi_0    pypi
google-api-core           2.2.2                    pypi_0    pypi
google-apitools           0.5.32                   pypi_0    pypi
google-auth               2.3.3                    pypi_0    pypi
google-cloud-core         2.1.0                    pypi_0    pypi
google-cloud-storage      1.42.3                   pypi_0    pypi
google-crc32c             1.3.0                    pypi_0    pypi
google-resumable-media    2.1.0                    pypi_0    pypi
googleapis-common-protos  1.53.0                   pypi_0    pypi
greenlet                  1.1.2                    pypi_0    pypi
h5py                      3.5.0                    pypi_0    pypi
httplib2                  0.20.2                   pypi_0    pypi
icu                       68.2                 he49afe7_0    conda-forge
idna                      3.1                pyhd3deb0d_0    conda-forge
imageio                   2.10.1                   pypi_0    pypi
inflection                0.5.1                    pypi_0    pypi
iniconfig                 1.1.1                    pypi_0    pypi
jmespath                  0.10.0                   pypi_0    pypi
joblib                    1.1.0                    pypi_0    pypi
json5                     0.9.6                    pypi_0    pypi
jsonschema                3.2.0                    pypi_0    pypi
kimimaro                  3.0.0                    pypi_0    pypi
kiwisolver                1.3.2                    pypi_0    pypi
krb5                      1.19.2               hcfbf3a7_3    conda-forge
libarchive                3.5.2                h2b60450_1    conda-forge
libblas                   3.9.0           12_osx64_openblas    conda-forge
libcblas                  3.9.0           12_osx64_openblas    conda-forge
libcurl                   7.79.1               hf45b732_1    conda-forge
libcxx                    12.0.0               h2f01273_0
libedit                   3.1.20210910         hca72f7f_0
libev                     4.33                 haf1e3a3_1    conda-forge
libffi                    3.3                  hb1e8313_2
libgfortran               5.0.0           9_3_0_h6c81a4c_23    conda-forge
libgfortran5              9.3.0               h6c81a4c_23    conda-forge
libiconv                  1.16                 haf1e3a3_0    conda-forge
liblapack                 3.9.0           12_osx64_openblas    conda-forge
libnghttp2                1.43.0               h6f36284_1    conda-forge
libopenblas               0.3.18          openmp_h3351f45_0    conda-forge
libsolv                   0.7.19               hcf210ce_5    conda-forge
libssh2                   1.10.0               h52ee1ee_2    conda-forge
libxml2                   2.9.12               h93ec3fd_0    conda-forge
llvm-openmp               12.0.1               hda6cdc1_1    conda-forge
lz4-c                     1.9.3                he49afe7_1    conda-forge
lzo                       2.10              haf1e3a3_1000    conda-forge
mamba                     0.17.0           py38h0c98490_0    conda-forge
markdown                  3.3.4                    pypi_0    pypi
matplotlib                3.4.3                    pypi_0    pypi
multiprocess              0.70.12.2                pypi_0    pypi
ncurses                   6.3                  hca72f7f_0
networkx                  2.6.3                    pypi_0    pypi
neuroglancer              2.22                     pypi_0    pypi
numpy                     1.21.3           py38h49b9922_1    conda-forge
oauth2client              4.1.3                    pypi_0    pypi
openssl                   1.1.1l               h0d85af4_0    conda-forge
orjson                    3.6.4                    pypi_0    pypi
packaging                 21.2                     pypi_0    pypi
pandas                    1.3.4                    pypi_0    pypi
pathos                    0.2.8                    pypi_0    pypi
pillow                    8.4.0                    pypi_0    pypi
pip                       21.2.4           py38hecd8cb5_0
pluggy                    1.0.0                    pypi_0    pypi
posix-ipc                 1.0.5                    pypi_0    pypi
pox                       0.3.0                    pypi_0    pypi
ppft                      1.6.6.4                  pypi_0    pypi
protobuf                  3.19.1                   pypi_0    pypi
psutil                    5.8.0                    pypi_0    pypi
py                        1.11.0                   pypi_0    pypi
pyasn1                    0.4.8                    pypi_0    pypi
pyasn1-modules            0.2.8                    pypi_0    pypi
pycosat                   0.6.3           py38h96a0964_1009    conda-forge
pycparser                 2.20               pyh9f0ad1d_2    conda-forge
pynrrd                    0.4.2                    pypi_0    pypi
pyopenssl                 21.0.0             pyhd8ed1ab_0    conda-forge
pyparsing                 2.4.7                    pypi_0    pypi
pyrsistent                0.18.0                   pypi_0    pypi
pysimdjson                4.0.2                    pypi_0    pypi
pysocks                   1.7.1            py38h50d1736_4    conda-forge
pytest                    6.2.5                    pypi_0    pypi
python                    3.8.12               h88f2d9e_0
python-dateutil           2.8.2                    pypi_0    pypi
python-jsonschema-objects 0.4.0                    pypi_0    pypi
python_abi                3.8                      2_cp38    conda-forge
pytz                      2021.3                   pypi_0    pypi
pywavelets                1.1.1                    pypi_0    pypi
readline                  8.1                  h9ed2024_0
reproc                    14.2.3               h0d85af4_0    conda-forge
reproc-cpp                14.2.3               he49afe7_0    conda-forge
requests                  2.26.0             pyhd8ed1ab_0    conda-forge
rsa                       4.7.2                    pypi_0    pypi
ruamel_yaml               0.15.80         py38h96a0964_1006    conda-forge
s3transfer                0.5.0                    pypi_0    pypi
scikit-image              0.18.3                   pypi_0    pypi
scikit-learn              1.0.1                    pypi_0    pypi
scipy                     1.7.1                    pypi_0    pypi
setuptools                58.0.4           py38hecd8cb5_0
simplejpeg                1.6.2                    pypi_0    pypi
six                       1.16.0             pyh6c4a22f_0    conda-forge
sockjs-tornado            1.0.7                    pypi_0    pypi
sqlite                    3.36.0               hce871da_0
tenacity                  8.0.1                    pypi_0    pypi
threadpoolctl             3.0.0                    pypi_0    pypi
tifffile                  2021.11.2                pypi_0    pypi
tinybrain                 1.2.1                    pypi_0    pypi
tk                        8.6.11               h7bc2e8c_0
toml                      0.10.2                   pypi_0    pypi
tornado                   6.1                      pypi_0    pypi
tqdm                      4.62.3             pyhd8ed1ab_0    conda-forge
urllib3                   1.26.7             pyhd8ed1ab_0    conda-forge
wheel                     0.37.0             pyhd3eb1b0_1
xz                        5.2.5                h1de35cc_0
yaml                      0.2.5                haf1e3a3_0    conda-forge
z5py                      2.0.9            py38h2720e7c_0    conda-forge
zlib                      1.2.11               h1de35cc_3
zmesh                     0.5.0                    pypi_0    pypi
zope-event                4.5.0                    pypi_0    pypi
zope-interface            5.4.0                    pypi_0    pypi
zstandard                 0.16.0                   pypi_0    pypi
zstd                      1.5.0                h582d3a0_0    conda-forge

@jakirkham
Copy link

It looks like this mixing packages from defaults and conda-forge in an incompatible way. That is likely leading to these symbol loading errors. Would set strict channel priority and create a fresh environment

@constantinpape
Copy link
Owner

@jakirkham I am currently trying to build using std::filesystem instead of boost::filesystem to get rid of the boost linking all together and make z5py easier to install.
However, the build is failing, I think due to mac builds with c++14, see conda-forge/z5py-feedstock#47 (comment). Could you maybe have a look in the issue? Thanks!

@xiuliren
Copy link
Author

xiuliren commented Nov 5, 2021

@jakirkham the default you mentioned is the pypi?
So I should install all the packages from conda-forge rather than mixing with pypi?

@jakirkham
Copy link

No defaults is a different channel. If you run conda config --set show_channel_urls true, then it will show up on the lines that currently don't list a channel.

@xiuliren
Copy link
Author

xiuliren commented Nov 5, 2021 •
edited
Loading

I created a fresh environment but still got the same symbol error.

>>> import z5py
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/__init__.py", line 1, in <module>
    from .file import File, N5File, ZarrFile, S3File
  File "/Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/file.py", line 6, in <module>
    from . import _z5py
ImportError: dlopen(/Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so, 2): Symbol not found: __ZN5boost10filesystem4path25m_path_iterator_incrementERNS1_8iteratorE
  Referenced from: /Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so
  Expected in: /Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/../../../libboost_filesystem.dylib
 in /Users/jwu/opt/anaconda3/envs/z5py/lib/python3.9/site-packages/z5py/_z5py.cpython-39-darwin.so

all the packages are from conda-forge

# packages in environment at /Users/jwu/opt/anaconda3/envs/z5py:
#
# Name                    Version                   Build  Channel
blosc                     1.21.0               he49afe7_0    conda-forge
boost-cpp                 1.77.0               hf3dc895_1    conda-forge
bzip2                     1.0.8                h0d85af4_4    conda-forge
ca-certificates           2021.10.8            h033912b_0    conda-forge
icu                       69.1                 he49afe7_0    conda-forge
libblas                   3.9.0           12_osx64_openblas    conda-forge
libcblas                  3.9.0           12_osx64_openblas    conda-forge
libcxx                    12.0.1               habf9029_0    conda-forge
libffi                    3.4.2                he49afe7_4    conda-forge
libgfortran               5.0.0           9_3_0_h6c81a4c_23    conda-forge
libgfortran5              9.3.0               h6c81a4c_23    conda-forge
liblapack                 3.9.0           12_osx64_openblas    conda-forge
libopenblas               0.3.18          openmp_h3351f45_0    conda-forge
libzlib                   1.2.11            h9173be1_1013    conda-forge
llvm-openmp               12.0.1               hda6cdc1_1    conda-forge
lz4-c                     1.9.3                he49afe7_1    conda-forge
ncurses                   6.2                  h2e338ed_4    conda-forge
numpy                     1.21.3           py39h7eed0ac_1    conda-forge
openssl                   3.0.0                h0d85af4_2    conda-forge
pip                       21.3.1             pyhd8ed1ab_0    conda-forge
python                    3.9.7           h38b4d05_3_cpython    conda-forge
python_abi                3.9                      2_cp39    conda-forge
readline                  8.1                  h05e3726_0    conda-forge
setuptools                58.5.3           py39h6e9494a_0    conda-forge
sqlite                    3.36.0               h23a322b_2    conda-forge
tk                        8.6.11               h5dbffcc_1    conda-forge
tzdata                    2021e                he74cb21_0    conda-forge
wheel                     0.37.0             pyhd8ed1ab_1    conda-forge
xz                        5.2.5                haf1e3a3_1    conda-forge
z5py                      2.0.9            py39hdfa2d75_0    conda-forge
zlib                      1.2.11            h9173be1_1013    conda-forge
zstd                      1.5.0                h582d3a0_0    conda-forge

@constantinpape
Copy link
Owner

My guess is that the boost version is incompatible with the one z5py was built with. In theory conda-forge should pin the correct version, but maybe this fails here?

I am trying to update the conda-forge build to use std::filesystem, so that linking to boost is not necessary any longer in conda-forge/z5py-feedstock#47.
However, the build fails, despite working in the CI in this repository. Any help there would be highly appreciated.

@constantinpape
Copy link
Owner

@jingpengw there is a z5py version available on conda-forge now that does not rely on boost.
Could you try installing z5py>=2.0.11 and see if that fixes the issue?

@xiuliren
Copy link
Author

xiuliren commented Nov 5, 2021

can not find this version.


PackagesNotFoundError: The following packages are not available from current channels:

  - z5py=2.0.11

Current channels:

  - https://conda.anaconda.org/conda-forge/osx-64
  - https://conda.anaconda.org/conda-forge/noarch
  - https://repo.anaconda.com/pkgs/main/osx-64
  - https://repo.anaconda.com/pkgs/main/noarch
  - https://repo.anaconda.com/pkgs/free/osx-64
  - https://repo.anaconda.com/pkgs/free/noarch
  - https://repo.anaconda.com/pkgs/r/osx-64
  - https://repo.anaconda.com/pkgs/r/noarch

@constantinpape
Copy link
Owner

It's listed here: https://anaconda.org/conda-forge/z5py/files. But was only added ~ 30 minutes ago.
It sometimes takes a while for new packages to become available fully (not sure why, maybe some caches need to be updated). Could you please try again tomorrow or next week and report back?

@jakirkham
Copy link

conda-forge mirrors packages to a CDN (hence the delay)

@xiuliren
Copy link
Author

xiuliren commented Nov 6, 2021

I just tried, it works! thanks a lot! @constantinpape @jakirkham

@xiuliren xiuliren closed this as completed Nov 6, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jakirkham @constantinpape @xiuliren