pattern is too long minimatch error during electron/universal auto-detect unpack logic

[mmaietta]

Hello all, I've encountered this bug repeatedly on universal builds

The issue seems to stem from the electron/universal unpack detection.

  ⨯ pattern is too long  failedTask=build stackTrace=TypeError: pattern is too long
    at assertValidPattern (/Users/mikemaietta/Development/electron-builder-test/node_modules/minimatch/minimatch.js:279:11)
    at minimatch (/Users/mikemaietta/Development/electron-builder-test/node_modules/minimatch/minimatch.js:116:3)
    at shouldUnpackPath (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:152:8)
    at handleFile (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:171:22)
    at next (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:199:5)
    at createPackageFromFiles (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:203:2)
    at Object.createPackageWithOptions (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/asar/src/asar.ts:57:10)
    at mergeASARs (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/universal/src/asar-utils.ts:215:5)
    at makeUniversalApp (/Users/mikemaietta/Development/electron-builder-test/node_modules/@electron/universal/src/index.ts:245:7)

Steps to repro:

• Install realm to electron-quick-start project
• Assemble project with electron-builder (which internally uses electron/asar) using universal target

Internal logic:

1. electron-builder is executing electron/asar with unpack minimatch (logged from within electron/asar's shouldUnpackPath internal function for debugging purposes')

 {node_modules/realm,/private/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/t-wh5lzR/asar-app-2/node_modules/realm/prebuilds/node/realm.node}

4. 2 asars are successfully generated with electron/asar
5. execute electron/universal on the two .apps.What seems to happen is that electron/universal is autodetecting what to unpack, and passes in a minimatch with over 2.8k minimatch entries
   

   universal/src/asar-utils.ts

   Lines 96 to 109 in caa0567

   	function buildUnpacked(a: string, fileList: Set<string>): void {
   	for (const file of fileList) {
   	const stat = asar.statFile(a, file);
   	if (!('unpacked' in stat) || !stat.unpacked) {
   	continue;
   	}
   	if ('files' in stat) {
   	continue;
   	}
   	unpackedFiles.add(file);
   	}
   	}

Excerpt:

/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/PrivacyInfo.xcprivacy
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/RealmJS.podspec
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/bindgen/src/realm_js_jsi_helpers.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/bindgen/src/realm_js_node_helpers.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/bindgen/vendor/realm-core/bindgen/src/realm_helpers.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/CMakeLists.txt
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/build.gradle
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/settings.gradle
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/AndroidManifest.xml
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/cpp/io_realm_react_RealmReactModule.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/cpp/jni_utils.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/cpp/jni_utils.hpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/cpp/platform.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/java/io/realm/react/RealmReactModule.java
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/java/io/realm/react/RealmReactPackage.java
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/android/src/main/java/io/realm/react/util/SSLHelper.java
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/apple/RealmReactModule.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/apple/RealmReactModule.mm
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/apple/platform.mm
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/jsi_externs.hpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/jsi_init.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/jsi_init.h
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/react_scheduler.cpp
/var/folders/y3/r1xf5xzn5pddms2gslqh14sh0000gn/T/x64-w0n63V/node_modules/realm/binding/jsi/react_scheduler.h
...waaaay more entries like this

Proposed resolution options:

1. Only pass in the common root folder of each File Set (in this case /node_modules/realm) to minimatch (this requires changes to electron/universal detection logic)
2. Bypass minimatch with more explicit control over packaging options, similar as to asar ordering logic. (requires changes in electron/asar) Options:
   a. Implement property that allows us to pass unpackPaths: string[] which allows explicit control
   b. Leverage options.ordering configuration of electron/asar to pass in JSON config for each file to be unpacked

Personally, I'd highly advocate for 2b, as it's relatively light of a change that is not breaking. AND it also makes sure this minimatch pattern is too long issue won't be ever be a roadblock for advanced users. electron-builder has a very complex copy-unpack-detection logic flow and we've hit the pattern is too long a few times previously, which we solved by implementing the approach of #1 before passing the unpack minimatch into electron/asar, but I can imagine this issue will be hit again if I'm just receiving it on electron/universal for an electron-quick-start project installing only realm.

Implementation Proposal:

Right now, the "ordering" file is simply of format:

<file name>:<anything>

Right now, electron/asar only pulls the filename value before the :
https://github.com/electron/asar/blob/121efebd2d02e374713cb513eb56a0e3db9534e9/src/asar.ts#L92-L95

The proposal is to leverage <anything> to be JSON.stringify of Type:

{
	unpack: boolean // the required property for resolve this issue
	// allows for future properties to be added for advanced usage without a breaking change
}

Taking this approach allows future configuration options to be allowed on a per-file basis should electron/asar need/desire to support more advanced usage

---

[UPDATE] PR Fix electron/asar#350