7.5.0 (#438)

* CHANGELOG, TODO, project.clj
* README, updated comrades
* support multiple 'interface' values in .toc files (#430)
* toc.clj, introduces 'interface-version-list' field.
  it contains all possible interface versions parsed from the 'interface' field.
* toc.clj, split 'parse-addon-toc' into '-parse-addon-toc'.
  new private fn is less strict, interpreting and extrapolating values, warning on bad values etc.
  new fn does validation and discards data if it's invalid.
  this was done so we don't have to repeat parsing steps elsewhere.
* toc.clj, the templated '## # interface' value is now considered and included in the interface-version-list.
* github-api.clj, removed duplicated parsing of :interface values in favour of that in toc.clj
* github-api.clj, order of extracted gametrack values is now deterministic.
* utils, fixed bug in interface-version-to-game-version
 - '110000' was returning '1.0.000' instead of '11.0.0'
* toc.clj, fixed bug where data would fail to validate and be discarded.
 - it was possible for multiple distinct interface-version values to become duplicate game tracks values (like [:retail :retail])
* jfx.clj, the 'WoW' (interface version) column now supports multiple values.
 - default column width values increased to accommodate them.
* specs.clj, replaces toc :interface-version with :interface-version-list
* core.clj, fixed a typo in the import/export logic and the test helper that was using :toc/game-track instead of :-toc/game-track
* toc.clj, replaced :-toc/game-track with :-toc/game-track-list
* addon.clj, a single set of toc data can now belong to many game tracks
 - this means the first instance of 'classic' toc data (for example) will be used even if multiple toc files support classic. crazy.
* tukui, curseforge, removed their 'expand-summary' implementations as they were using :interface-version
* jfx.clj, addon data detail pane now accommodates many interface values.
* toc.clj, removed a warning when the presence of a game track in the filename doesn't match the game tracks parsed from the data.
 - the user can't really do anything about it, I had at least one false-positive and with many game tracks to now check it seems needlessly expensive.
* curseforge, tukui, removes test fixtures referenced by removed code.
* catalogue.clj, fixed possibilty of multiple game tracks in toc2summary
* specs, source-updates, removes 'release-label' and 'interface-version'.
 these appear to have been introduced for curseforge and only used there.
* linted
* review feedback
* deps bump
* constants, adds entries for Dragonflight patch releases.
* constants, adds entry for The War Within.
* replaces references to old wiki with new wiki.
* fixed bug in emergency 'short' catalogue generation
 it would include addons from BFA instead of stopping at Shadowlands
* install addon from file, preserve 'ignore' and 'pin' flags (#434)
* cli, adds test 'install-addon--ignore-then-update-from-file'
  - it recreates what is being seen in issue #433
* core.clj, download-time tests duplicated at install-time.
  - but with different consequences.
* cli.clj, installing an addon from a file now goes through 'core', not 'addon'.
* core, relaxes restriction on what core/install-addon considers installable.
* addon.clj, special handling for nfo flags when installing addon.
  - if an addon is ignored then the new nfo data will inherit the ignore flag.  
  - if an addon is pinned then thew new nfo data will remove the pinned-version flag.
  - it shouldn't be possible to reach this logic right now, core/install-addon should prevent that.
* core.clj, install-addon, replaced 'test-only?' parameter with map of options.
* addon.clj, fixed bug where ignore check fails on single dir addons
  - it assumed all addons had a :group-addons field, but that only exists for addons that have been grouped because there are more than one of them.
  - pretty serious bug.
* core/install-addon, adds option to bypass ignore and pinned flags
* core/remove-many-addons, now refuses to remove ignored addons
* addon/remove-addon, just warns when ignored addon is being removed
* cli/install-addons-from-file, now accepts a map of installation options
* jfx/zip-file-picker, changed default behaviour when installing from file
  - ignored addons are now overwritten
  - pinned addons are now unpinned
* github, improve asset classification when release.json not present (#437)
* github-api, handle case where one unclassified asset can be classified as retail.
* review feedback
* updated constants for The War Within
* updated constants for the release date of previous expansion
* 7.5.0

CHANGELOG.md

@@ -4,6 +4,39 @@ All notable changes to this project will be documented in this file. This change
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 7.5.0 - 2024-09-07
+
+### Added
+
+* added another heuristic for guessing which asset in a Github release supports retail.
+    - in this case, if there is a single remaining unclassified asset, and there are other classified assets and nothing has been classified as retail yet, classify that asset as retail.
+        - yes, this is guessing, and it won't always be correct, but it's probably true most of the time.
+    - thanks @jake770321
+* error handling for a few cases where uncaught exceptions were being swallowed.
+
+### Changed
+
+* installing an addon by file now overwrites/updates ignored addons and unpins pinned addons.
+    - this accommodates a use case where Strongbox is used to install a select few addons manually from unsupported sources (like Curseforge) but then the new addons match against older versions in the catalogue from wowi.
+    - please open a feature request if you want this behaviour adjustable.
+* updated internal constants for The War Within.
+* updated internal constant for the date of the previous expansion.
+    - affects how much the emergency 'short' catalogue gets truncated from the 'full' catalogue.
+        - emergency catalogue is used when Github hosted catalogue is unavailable, which is rare.
+* trimmed the '(Flatpak)' from the 'Strongbox (Flatpak)' on the `.desktop` files distributed by Flatpak.
+    - thanks @deadmeu
+
+### Fixed
+
+* fixed bug in emergency 'short' catalogue generation.
+    - it would include addons from BFA instead of stopping at Shadowlands.
+* fixed bug where installing a file manually would bypass ignored and pinned addon checks.
+    - it would tell you it was refusing to install, then install it anyway.
+    - thanks @akanouras
+* fixed bug where ignore check bypassed on ungrouped/single directory addons.
+    - logic assumed all addons were grouped, but grouping only happens when an addon unzip to multiple directories.
+    - pretty big bug!
+
 ## 7.4.0 - 2024-07-19
 
 ### Added

README.md

@@ -71,7 +71,7 @@ Java users can run Strongbox using the `.jar` from the [releases](https://github
 
 ## Usage
 
-Strongbox works by matching installed addons to a catalogue of addons available online.
+Strongbox works by matching installed addons to a [catalogue of addons](https://github.com/ogri-la/strongbox-catalogue) available online.
 
 Some addons are matched directly to the catalogue but others may require you to search the catalogue and re-install them 
 before that match can be made.
@@ -144,7 +144,7 @@ This software interacts with the following remote hosts:
 
 These hosts *may* redirect requests to other hosts.
 
-These interactions use a HTTP user agent header unique to Strongbox so that it may be identified easily:
+These interactions use a HTTP user-agent header specific to Strongbox so that it may be identified easily:
 
     "strongbox/x.x (https://github.com/ogri-la/strongbox)"
 
@@ -174,11 +174,11 @@ Running Strongbox with `--debug` will generate a file called `debug.log` that I
 
 ### classic and retail addon support
 
-"Classic", "Classic (The Burning Crusade)", "Classic (Wrath of the Lich King)" and "Retail" versions of WoW are all
+"Classic", "Classic (The Burning Crusade)", "Classic (Wrath of the Lich King)", etc and "Retail" versions of WoW are all
 distinct addon systems.
 
-Some addons support all systems in a single download, some support classic as an alternate build of the same addon, 
-some addons support classic only, some addons have been split up into multiple addons. There is a lot of variation.
+Some addon authors support all systems in a single download, some support classic as an alternate build of the same addon, 
+some addons support classic only, some addons have been split up into separate addons. There is a lot of variation.
 
 Click the drop-down next to your addon directory and select either `retail`, `classic` or `classic (TBC)`.
 
@@ -193,7 +193,7 @@ the same addon directory. If an addon is available for multiple addon systems it
 * `classic (WotLK)` will prefer `classic (WotLK)` addons, then `classic (Cata)`, then `classic (TBC)`, then `classic` then `retail`
 * `classic (Cata)` will prefer `classic (Cata)` addons, then `classic (WotLK)`, then `classic (TBC)`, then `classic` then `retail`
 
-If uncertain which addon system an installed addon supports, look at the `WoW` column value on the `installed` tab and 
+If uncertain which addon systems an installed addon supports, look at the `WoW` column values on the `installed` tab and 
 compare it to the `Version` value in the list of WoW [public client builds](https://warcraft.wiki.gg/wiki/Public_client_builds).
 
 ### install addons from URL
@@ -202,7 +202,8 @@ Strongbox supports installing addons using URLs from addon hosts.
 
 Addons installed this way are always available regardless of the catalogue you've selected.
 
-For example, addons imported from Github will be available even if you otherwise use wowinterface.com addons exclusively.
+For example, addons installed directly from Github will be available even if you otherwise use wowinterface.com addons 
+exclusively.
 
 Click `File` from the top menu and select `Import addon` and paste the URL of the addon you want to install.
 
@@ -220,8 +221,7 @@ When you search for an addon you are searching a list of thousands of addons tha
 Click `Catalogue` from the top menu and choose your preferred catalogue.
 
 The default catalogue is the 'short' catalogue. It contains all addons from all supported hosts that have been *updated* 
-since *the beginning of the previous expansion*. This is currently Battle For Azeroth, released 2018-08-14 and the 
-catalogue has approximately 2.8k addons.
+since *the beginning of the [previous expansion](https://warcraft.wiki.gg/wiki/Expansion#World_of_Warcraft)*.
 
 The 'full' catalogue contains all addons from all supported hosts, ever, and is approximately 7.2k addons large. It 
 contains many unmaintained addons.
@@ -231,7 +231,7 @@ There are also per-host catalogues, like a 'wowinterface' catalogue, and Strongb
 Catalogues are updated weekly. New addons released during the week will not be present until the next week. Addons can 
 be installed using its URL in these cases.
 
-The 'user' catalogue is a little different. It's initially empty but grows as addons are starred while searching or 
+The 'user' catalogue is a little different. Its initially empty but grows as addons are starred while searching or 
 imported from addon hosts, like Github. 
 These addons also appear in search results. Individual addons from the user catalogue are checked for new releases 
 normally, but the catalogue itself can only be updated manually.

TODO.md

@@ -6,14 +6,61 @@ see CHANGELOG.md for a more formal list of changes by release
 
 ## done
 
-* support multiple interface values
-* bump any game versions
-* bump any deps
+* issue 433: preserve ignore and pin flags when addon updated manually via strongbox
+    - usecase: 
+        - old addon exists on wowi
+        - new addon exists on curse
+        - user installs new addon from curse and then 'ignores' it so nothing affects it
+        - user installs updated addon from curse and ignore flag is lost
+    - problems:
+        - if the addon was ignored, it shouldn't have been possible to overwrite it to begin with
+        - it seems reasonable to update addons in this way
+    - done
+
+bug:
+    - attempting to overwrite ignored addon results in "refusing to delete ignored addon: /path/to/addon/dir" with no addon name
+    - it looks like it installed it anyway
+    - done
+
+* import addon from github 
+    - getting a 'no retail release' when there is a clear retail rfelease available
+    - done
+
+* update constants for war within
+    - done
+
+* Desktop Entry Name should not include "(Flatpak)" 
+    - https://github.com/flathub/la.ogri.strongbox/issues/7
+    - done
 
 ## todo
 
-
 ## todo bucket (no particular order)
+
+* gui, when installing an addon from file, allow the following options
+    - test-only?
+    - unpin-pinned?
+    - overwrite-ignored?
+
+bug: clear button isn't clearing search
+
+bug: 
+    select github + wowi
+    search for 'alto'
+        - 3 results, 2 github, 1 wowi
+    deselect 'github'
+        - no results
+    select gitlab (so wowi and gitlab are selected)
+        - wowi result
+
+bug: search, next and previous buttons should take me to top of search results
+
+
+* patch tooltip, support multiple game versions
+
+* flathub, remove '(Flathub)' from .desktop file
+    - https://github.com/flathub/la.ogri.strongbox/issues/7
+
 * remove support for parsing templated .toc keyvals: '# ## Key: Value'
 
 * gui, 'set-icon' is taking a long time to do it's thing.

pom.xml

@@ -4,7 +4,7 @@
   <groupId>ogri-la</groupId>
   <artifactId>strongbox</artifactId>
   <packaging>jar</packaging>
-  <version>7.4.0</version>
+  <version>7.5.0</version>
   <name>strongbox</name>
   <description>World Of Warcraft Addon Manager</description>
   <url>https://github.com/ogri-la/strongbox</url>
@@ -18,7 +18,7 @@
     <url>https://github.com/ogri-la/strongbox</url>
     <connection>scm:git:git://github.com/ogri-la/strongbox.git</connection>
     <developerConnection>scm:git:ssh://[email protected]/ogri-la/strongbox.git</developerConnection>
-    <tag>822521feaf85c7579ac3ecff6eb849e91abbed94</tag>
+    <tag>744a0157d35656f2eab69f2ad981aafc5c2dd474</tag>
   </scm>
   <build>
     <sourceDirectory>src</sourceDirectory>
@@ -250,4 +250,4 @@
 <!-- This file was autogenerated by Leiningen.
   Please do not edit it directly; instead edit project.clj and regenerate it.
   It should not be considered canonical data. For more information see
-  https://github.com/technomancy/leiningen -->
+  https://codeberg.org/leiningen/leiningen -->

project.clj

@@ -1,4 +1,4 @@
-(defproject ogri-la/strongbox "7.4.0"
+(defproject ogri-la/strongbox "7.5.0"
   :description "World Of Warcraft Addon Manager"
   :url "https://github.com/ogri-la/strongbox"
   :license {:name "GNU Affero General Public License (AGPL)"

src/strongbox/addon.clj

@@ -79,10 +79,13 @@
   (if (:ignore? addon)
     ;; if addon is being ignored, refuse to remove addon.
     ;; note: `group-addons` will add a top level `:ignore?` flag if any addon in a bundle is being ignored.
-    (error "refusing to delete ignored addon:" install-dir)
+    ;; 2024-08-25: behaviour changed. this is not the place to prevent ignored addons from being removed.
+    ;; see `core/install-addon`, `core/remove-many-addons`
+    ;;(error "refusing to delete ignored addon:" (:label addon))
+    (warn "deleting ignored addon:" (:label addon)))
 
-    (doseq [grouped-addon (flatten-addon addon)]
-      (-remove-addon! install-dir (:dirname grouped-addon) (:group-id addon)))))
+  (doseq [grouped-addon (flatten-addon addon)]
+    (-remove-addon! install-dir (:dirname grouped-addon) (:group-id addon))))
 
 ;;
 
@@ -276,8 +279,13 @@
 (defn-spec install-addon (s/or :ok (s/coll-of ::sp/extant-file), :error ::sp/empty-coll)
   "installs an addon given an addon description, a place to install the addon and the addon zip file itself.
   handles suspicious looking bundles, conflicts with other addons, uninstalling previous addon version and updating nfo files.
-  returns a list of nfo files that were written to disk."
-  [addon :addon/nfo-input-minimum, install-dir ::sp/writeable-dir, downloaded-file ::sp/archive-file]
+
+  relies on `core/install-addon` to block installations that would overwrite ignored or pinned addons.
+  if it's gotten this far ignored/pinned addons will be deleted
+  and the new addon will be unzipped over the top.
+
+  returns a list of nfo files that were written to disk, if any."
+  [addon :addon/nfo-input-minimum, install-dir ::sp/writeable-dir, downloaded-file ::sp/archive-file, opts ::sp/install-opts]
   (let [nom (or (:label addon) (:name addon) (fs/base-name downloaded-file))
         version (:version addon)
 
@@ -288,6 +296,15 @@
 
         zipfile-entries (zip/zipfile-normal-entries downloaded-file)
         toplevel-dirs (zip/top-level-directories zipfile-entries)
+
+        toplevel-nfo (->> toplevel-dirs ;; [{:path "EveryAddon/", ...}, ...]
+                          (map :path) ;; ["EveryAddon/", ...]
+                          (map fs/base-name) ;; ["EveryAddon", ...]
+                          (map #(nfo/read-nfo-file install-dir %))) ;; [`(read-nfo-file install-dir "EveryAddon"), ...]
+
+        contains-nfo-with-ignored-flag (utils/any (map :ignore? toplevel-nfo))
+        contains-nfo-with-pinned-version (utils/any (map :pinned-version toplevel-nfo))
+
         primary-dirname (determine-primary-subdir toplevel-dirs)
 
         ;; let the user know if there are bundled addons and they don't share a common prefix
@@ -306,10 +323,28 @@
                         (let [addon-dirname (:path zipentry)
                               primary? (= addon-dirname (:path primary-dirname))
                               new-nfo-data (nfo/derive addon primary?)
+                              ;; if any of the addons this addon is replacing are being ignored,
+                              ;; the new nfo will be ignored too.
+                              new-nfo-data (if contains-nfo-with-ignored-flag
+                                             (nfo/ignore new-nfo-data)
+                                             new-nfo-data)
+
+                              ;; if any of the addons this addon is replacing are pinned,
+                              ;; the pin is removed. We've just modified them and they are no longer at that version.
+                              new-nfo-data (if contains-nfo-with-pinned-version
+                                             (nfo/unpin new-nfo-data)
+                                             new-nfo-data)
+
                               new-nfo-data (nfo/add-nfo install-dir addon-dirname new-nfo-data)]
                           (nfo/write-nfo! install-dir addon-dirname new-nfo-data)))
 
         ;; write the nfo files, return a list of all nfo files written
+        ;; todo: if a zip file is being installed then we can't rely on `remove-addon!` having been called,
+        ;; but `remove-completely-overwritten-addons` will have been called and *may* have removed the
+        ;; addon *if* the new addon is a superset of the old one.
+        ;; this leads to the possibility of a new addon that has dropped a subdir or added a new one (like a rename)
+        ;; being skipped and orphaning the original subdir.
+        ;; this means we could hit `unzip-addon` with the original addon still fully intact.
         update-nfo-files (fn []
                            (mapv update-nfo-fn toplevel-dirs))
 
@@ -343,12 +378,16 @@
     (suspicious-bundle-check)
 
     ;; todo: remove support for v1 addons in 2.0.0 ;; todo!
-    ;; when is it not valid? when importing v1 addons. v2 addons need 'padding' as well :(
+    ;; when is it not valid?
+    ;; * when importing v1 addons. v2 addons need 'padding' as well :(
+    ;; * when installing from a file and we have nothing more than a generated ID value
     (when (s/valid? :addon/toc addon)
       (remove-addon! install-dir addon))
 
     (remove-completely-overwritten-addons)
 
+    ;; `addon/install-addon` is all about installing an addon, not checking whether it's safe to do so.
+    ;; use `core/install-addon` for safety checks.
     (unzip-addon)
     (update-nfo-files)))
 
@@ -388,7 +427,7 @@
 (defn-spec ignored-dir-list (s/coll-of ::sp/dirname)
   "returns a list of unique addon directory names (including grouped addons) that are not being ignored"
   [addon-list (s/nilable :addon/installed-list)]
-  (->> addon-list (filter :ignore?) (map :group-addons) flatten (map :dirname) (remove nil?) set))
+  (->> addon-list (filter :ignore?) (map flatten-addon) flatten (map :dirname) (remove nil?) set))
 
 (defn-spec overwrites-ignored? boolean?
   "returns `true` if given archive file would unpack over *any* ignored addon.

src/strongbox/cli.clj

@@ -312,10 +312,11 @@
                  (let [downloaded-file (core/download-addon-guard-affective addon install-dir)
                        existing-dirs (addon/dirname-set addon)
                        updated-dirs (zipfile-locks downloaded-file)
-                       locks-needed (clojure.set/union existing-dirs updated-dirs)]
+                       locks-needed (clojure.set/union existing-dirs updated-dirs)
+                       opts {}]
                    (swap! new-dirs into updated-dirs)
                    (utils/with-lock current-locks locks-needed
-                     (core/install-addon-affective addon install-dir downloaded-file)
+                     (core/install-addon-guard-affective addon install-dir opts downloaded-file)
                      (core/refresh-addon addon))))]
     (run! #(joblib/create-addon-job! queue-atm % job-fn) updateable-addon-list)
     (joblib/run-jobs! queue-atm core/num-concurrent-downloads)
@@ -363,7 +364,7 @@
           error-messages
           (logging/buffered-log
            :warn
-           (addon/install-addon addon (core/selected-addon-dir) downloaded-file))]
+           (core/install-addon addon (core/selected-addon-dir) downloaded-file))]
       (core/refresh)
       {:label (fs/base-name downloaded-file)
        :error-messages error-messages}))
@@ -372,7 +373,7 @@
   "installs/updates a list of addon zip files in parallel.
   does a clever refresh check afterwards to try and prevent a full refresh from happening.
   very similar code to `install-update-these-in-parallel`."
-  [download-file-list (s/coll-of ::sp/extant-archive-file)]
+  [download-file-list (s/coll-of ::sp/extant-archive-file), opts ::sp/install-opts]
   (let [queue-atm (core/get-state :job-queue)
         install-dir (core/selected-addon-dir)
         current-locks (atom #{})
@@ -389,7 +390,7 @@
                      (let [error-messages
                            (logging/buffered-log
                             :warn
-                            (let [results (addon/install-addon addon install-dir downloaded-file)]
+                            (let [results (core/install-addon addon install-dir downloaded-file opts)]
                               (when-let [installed-addon-dir (some-> results first fs/parent str)]
                                 (core/refresh-addon* installed-addon-dir))))]
 

src/strongbox/constants.clj

@@ -3,9 +3,9 @@
 (def mascot "ᕙ[°▿°]ᕗ")
 
 (def release-of-previous-expansion
-  "'Shadowlands', released October 13, 2020. Used to shorten the 'full' catalogue.
-  https://warcraft.wiki.gg/wiki/Public_client_builds"
-  "2018-08-14T00:00:00Z")
+  "'Dragonflight', released November 28th/29th 2022. Used to shorten the 'full' catalogue.
+  https://warcraft.wiki.gg/wiki/Expansion#World_of_Warcraft"
+  "2022-11-28T00:00:00Z")
 
 (def release-of-wow-classic
   "the date wow classic went live (addon development may have started before that). Used to guess possible game tracks when it's ambiguous.
@@ -16,7 +16,7 @@
 ;; used as a placeholder for an addon's supported version when we're forced to guess.
 ;; don't fret too much about patch versions. These values don't affect much.
 ;; https://warcraft.wiki.gg/wiki/Public_client_builds
-(def latest-retail-game-version "10.0.0")
+(def latest-retail-game-version "11.0.0")
 (def latest-classic-game-version "1.14.3")
 (def latest-classic-tbc-game-version "2.5.4")
 (def latest-classic-wotlk-game-version "3.4.0")
@@ -25,7 +25,7 @@
 ;; interface version to use if .toc file is missing one.
 ;; assume addon is compatible with the most recent version of retail (see above).
 ;; these values need to match the latest-* values above.
-(def default-interface-version 100000)
+(def default-interface-version 110000)
 (def default-interface-version-classic 11400)
 
 ;; take all of the game tracks to the right of your position