dylib support

[rob9315]

It would be nice to have dylib support for when there is a shared dylib dependency between multiple rust targets, right now this error message is displayed as it ignores dylibs.

found no targets in 197 packages

The idea was to compile a shared rust dependency as a dynamic library because compilation artifacts aren't shared when using corrosion in different catkin packages. This however doesn't work as corrosion will refuse to compile the dylib. Using a cdylib is not an acceptable tradeoff as it would require making the library c-ffi-safe which would not be needed with either shared compilation artifacts or a dylib which was compiled with the same compiler as the bin using it.

[jschwe]

Could you elaborate a bit more on the intended usage of this feature?
Do you want to treat the dylib artifact as a regular IMPORTED SHARED library on the CMake side, with the intention of using CMake facilities for packaging / installing of your Rust binary + dylib?

On the cargo side of things - If a bin crate has a dependency on a dylib only library, do you need to call cargo build for the library separately? Intuitively, I would have expected that cargo would compile the dylib dependency and put it somewhere in the build directory of the bin crate.

[rob9315]

I would like to save on compilation time, as such it corrosion should probably treat the dylib artifact as a regular IMPORTED SHARED library, yes. Alternatively a respecting rlib as a build option and somehow (re)using those artifacts would be perfect as well.
Yes, if a bin crate has a dependency on a dylib only library, you need to call cargo build for the library separately and provide the libxyz.so beside the bin at runtime. When both are in a single workspace, they both end up in the same folder and it all works out. When using corrosion and specifying the dylib as a dependency, cargo does not build the dylib crate as well (intended behavior) and instead expects the libxyz.so file at runtime. That however does not get installed by corrosion, neither does this work with cdylibs because of #64.

This is what happens when you build a dylib with cargo build.

$ tree dylib
dylib
├── Cargo.lock
├── Cargo.toml
├── src
│   └── lib.rs
└── target
    ├── CACHEDIR.TAG
    └── release
        ├── build
        ├── deps
        │   ├── dylib.d
        │   └── libdylib.so
        ├── examples
        ├── incremental
        ├── libdylib.d
        └── libdylib.so

I set up a test project corrosion-dylib for testing the different options for rust-to-rust catkin compilation with corrosion. I surely got a lot wrong there but I basically tried to build it how I would assume things to hypothetically work.

[rob9315]

dylib and cdylib are mutually exclusive so dylib could just be treated like cdylib and then for dynamic libraries only #64 would be a blocker I assume. Out of interest, could rust static libraries, rlibs somehow be supported? That would be the actual solution to the problem of sharing build artifacts with different cmake invocations.

[jschwe]

I would like to save on compilation time,

Have you tried sccache? It works quite well in my experience and can cache results across multiple machines if setup correctly.

Alternatively a respecting rlib as a build option and somehow (re)using those artifacts would be perfect as well.

In my opinion this would be a cargo issue.

I set up a test project corrosion-dylib

Thanks - I currently don't have much bandwidth - but I'll have a look eventually.

dylib and cdylib are mutually exclusive so dylib could just be treated like cdylib

I would be very hesitant - I don't want to get strange bug reports from users using dylibs with binaries of a different Rust version.
If we add this I want it to be extremely clear to the user that dylib and cdylib are two very different things, and people don't by accident use dylib.

Out of interest, could rust static libraries, rlibs somehow be supported?

Probably not in the short term. In the longer term hopefully an rlib format is stabilized. See this Pre-RFC and/or rust-lang/rust#73632

[rob9315]

Of course there should be a big distinction being made with dylibs and cdylibs. With my particular usecase there is luckily no risk of rust versions clashing.
sccache is my short term solution, however it would be nice to allow sccache to cache the artifacts by allowing rlibs to build without any installation steps for now so that when you have bin1 and bin2 depending on rlib they can then reuse the cached artifacts. Caching across multiple machines is not needed or desired here.

Thank you for this great project!

[jschwe]

If you are using Rust >= 1.64, you could put cdylib into your Cargo.toml package manifest (so that corrosion imports the crate) and then override the crate type by adding --crate-type=dylib to the build flags (corrosion_set_cargo_flags(target_name --crate-type=dylib)).

[russelltg]

I would like dylib support for the usecase of getting corrosion to

1. copy the dylib to the LIBRARY_OUTPUT_DIRECTORY after users of it are built (staticlib, cdylib or bin)
2. for packaging--I use cpack and would like to consume the dependency metadata (ie, make sure staticlib targets that link to dylib targets, (I use -Zstaticlib-allow-rdylib-deps) have proper cpack dependencies setup for them)

Is there interest in this? I'm happy to contribute it.

Also, potentially could pass --crate-type for targets that are both dylib and rlib based on BUILD_SHARED_LIBS?

[jschwe]

1. copy the dylib to the LIBRARY_OUTPUT_DIRECTORY after users of it are built (staticlib, cdylib or bin)

Is the "after users of it are built (staticlib, cdylib or bin)“ bit important? I imagine that would be quite a bit more work than just copying dylibs to the LIBRARY_OUTPUT_DIRECTORY.

2. for packaging

I would love to see some contributions to improve the packaging story. Ideally this would include some short documentation to help out other users and some tests to ensure installing works as intended.

I use cpack and would like to consume the dependency metadata

Okay, so it seems you do need the dependency information. I see the use case, but maybe you could write a bit more how you would implement this, which edge-cases there are and what is in/out of scope before you start working on a PR.

Also, potentially could pass --crate-type for targets that are both dylib and rlib based on BUILD_SHARED_LIBS?

Until the rlib format is stabilized, corrosion won't use rlibs, so there is no point in ever doing --crate-type=rlib. Do you have any numbers on how much compile-time can be saved by skipping the build of the rlib? I'd imagine it would not be much.