[FEATURE REQUEST] - For android (IOS?) builds change dependency to libgstreamer_android.so instead of discrete libs

[fuzing]

Please see here for context, including comments from @sdroege
GStreamer #2999
Cerbero #453

When building for android (IOS?) the recommended deployment is a single shared lib (typically called libgstreamer_android.so), but the GStreamer rust bindings depend upon discrete libraries instead of the mono-lib created for the android platform.

For android builds, these discrete dependencies should be replaced with the mono-lib.

[sdroege]

@bilelmoussaoui Any opinions here? We would need to add some cargo feature that switches the #[link] attributes to use libgstreamer_android.so for all the libraries. The pkg-config part is something that should be solved in GStreamer itself though.

[bilelmoussaoui]

This only affects libraries in gtk-rs-core i believe right? and gstreamer of course

[sdroege]

Until someone ports GTK to Android/iOS, yes. Also other GNOME libraries, like libsoup, which are not desktop/GTK-related.

[bilelmoussaoui]

Are you aware of any library that requires similar treatment but links against a separate .so? just wondering if we should make gir hardcode the libgstreamer_android.so behind a gstreamer/android feature or we want to make it configurable.

[bilelmoussaoui]

I can do the gir bits if you want, should be pretty easy

[sdroege]

There's also 'gstreamer-full', a way to build GStreamer with all deps and plugins into a single shared library (or even static library) via meson. And generally you can do such things relatively easily with meson, so I would be surprised if there are not other people doing such things out there.

[bilelmoussaoui]

I see. So the link with name has to be configurable somehow.

[bilelmoussaoui]

Maybe also finish gtk-rs/gir#1298 while we are at it

[sdroege]

Yeah that is kind of related. Also the things @pbor needs for gradually porting code over to Rust.

[fuzing]

Are you aware of any library that requires similar treatment but links against a separate .so? just wondering if we should make gir hardcode the libgstreamer_android.so behind a gstreamer/android feature or we want to make it configurable.

Just a quick comment on this one. Having this configurable (even for Androis/IOS?) as opposed to hardcoded is likely a wiser choice insofar as it is possible to use the discrete libs on the android platform (I was doing it this way early on). I'm guessing that most folks use the documented mono-lib/libgstreamer_android.so strategy, but it's probably wise to anticipate that there are others using the discrete libs.

[bilelmoussaoui]

The problem is that only way to configure that would be through build.rs which is meh. I guess that is fine for now as we use it anyways

[fuzing]

Yeah - agree. Thanks for your work on this..... it's much appreciated.

[sdroege]

How would you configure it through build.rs?

[sdroege]

The #[link] attributes can actually be overridden from the command-line: https://doc.rust-lang.org/rustc/command-line-arguments.html#-l-link-the-generated-crate-to-a-native-library

[bilelmoussaoui]

How would you configure it through build.rs?

Have a feature that would switch from using the hardcoded default dylib name to something you pass with an env variable? not sure if it that idea can even work, didn't thought too much about it

[fuzing]

@bilelmoussaoui and @sdroege - setting an environment variable would likely be a great solution (with the name of the shared monolib) - this can be checked in build.rs

[fuzing]

The #[link] attributes can actually be overridden from the command-line: https://doc.rust-lang.org/rustc/command-line-arguments.html#-l-link-the-generated-crate-to-a-native-library

I'm not familiar enough with these settings to know if these are "cumulative" (i.e. libraries are added to the list of those linked to), or overridden/replaced. If one can truly override/replace the other/default library linkage settings then this might be a great option. In my experience these settings are typically added (switches/flags/etc.) to the toolchain compilation/linkage steps.

[fuzing]

@sdroege and @bilelmoussaoui
I ran the following experiment which worked for my use case.

I have the following dependencies in my rust package (among others) -

gio = { git = "https://github.com/gtk-rs/gtk-rs-core" }
glib = { git = "https://github.com/gtk-rs/gtk-rs-core" }
gst = { package = "gstreamer", git = "https://gitlab.freedesktop.org/gstreamer/gstreamer-rs" }
gst-sys = { package = "gstreamer-sys", git = "https://gitlab.freedesktop.org/gstreamer/gstreamer-rs" }
gst-video = { package = "gstreamer-video", git = "https://gitlab.freedesktop.org/gstreamer/gstreamer-rs" }
gst-base = { package = "gstreamer-base", git = "https://gitlab.freedesktop.org/gstreamer/gstreamer-rs" }
gst-app = { package = "gstreamer-app", git = "https://gitlab.freedesktop.org/gstreamer/gstreamer-rs" }

My rust code and these dependencies are compiled and bundled into my own shared library ("libnative.so"). Then, in my android Java/Kotlin code, as part of bootstrap, the following occurs (kotlin):

  System.loadLibrary("gstreamer_android")  // the gstreamer mono-lib - dlopen and also run the JNI_OnLoad initializer
  System.loadLibrary("android_glue") // extra glue code for my use-case - irrelevant in this example
  System.loadLibrary("native") // my rust code, and the rust gstreamer bindings dependencies

  // Elsewhere, the GStreamer.init() occurs in bootstrap code - all normal android startup/setup

However, the above fails with a message similar to:

10-10 10:15:02.735  3717  3717 E GeneratedPluginsRegister: Caused by: java.lang.UnsatisfiedLinkError: dlopen failed: library "libgobject-2.0.so" not found: needed by /data/app/~~XB4c_YqPzVl8UI9_l1Osqg==/com.somecompany.some-application-Ioh0HPpzZnCF8bJ6zq-HTQ==/lib/arm64/libnative.so in namespace clns-4

So, one or more of the rust bindings packages is dependent upon the libgobject-2.0 shared library and the dlopen fails. Notably, all of the symbols from the libgobject-2.0 library are already included (and loaded) within my libgstreamer_android.so library but the gstreamer rust dependencies expect these symbols in their own shared libraries (libgobject-2.0 etc.). Bundling libgobject-2.0.so as a shared library with my android project solves the dependency, but then the next dependency fails (i.e. to be expected). As mentioned before, this is likely to lead to a cascade of other problems because all symbols will be included twice (i.e. once in the monolib, and once in another shared lib).

However the following works:

Inserting the following into my build.rs seemed to force the toolchain to check libgstreamer_android.so for symbols before anything else

    //
    // for android, link against the libgstreamer_android.so shared monolib
    // This is a kludge for now - as it only works/solves for the arm64-v8a arch android build
    // solve for all others though by creating sub-variants based on 'arch'
    //
    match os.as_str() {
        "android" => {
            // place where android build toolchain places the libgstreamer_android.so file
            // as long as this points to the local build, it should solve/satiate dependencies for gstreamer, as they are all included
            // inside the libgstreamer_android.so mono-lib
            println!("cargo:rustc-link-search=/home/<some-username>/<some-project>/android/gst-android-build/arm64-v8a");
            println!("cargo:rustc-link-lib=dylib=gstreamer_android");
        },
        _ => {}
    }

This compiles and runs within android, and symbols for all of the rust bindings packages are now satisfied/satiated from libgstreamer_android.so

Because I haven't looked in detail at the dependencies coded into the rust gstreamer bindings packages, I'm not sure whether this is a stable solution (i.e. ordering of dependencies may be problematic, especially within the Android eco-system, which requires both native and Java bootstrapping, but this does appear to have solved my dependency issue). I plan to test this more fully this week, and will provide an update later on.

[sdroege]

Can be closed now with gtk-rs/gir#1600