removes tukui support (#408)

* removes tukui support
ogri-la · Jun 10, 2023 · 7f7049b · 7f7049b
1 parent 0caec66
commit 7f7049b
Show file tree

Hide file tree

Showing 18 changed files with 447 additions and 322 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -67,6 +67,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Removed
 
+* Tukui support. Tukui addons are:
+    - no longer checked for updates.
+    - excluded from being imported.
+    - excluded from the user-catalogue.
+    - no longer scraped from the tukui.org API into a catalogue.
+    - no longer present in the 'full' or 'short' catalogues.
+    - excluded from search results.
+    - removed from the 'emergency' (built-in, hardcoded) catalogue (used when remote catalogues are unavailable).
+    - removed from lists of available addon hosts to switch an addon between.
+
 ## 6.1.2 - 2023-05-16
 
 * issue #402, fixed a freezing bug in the search results, introduced in 5.1.0 (2022-03-02).

diff --git a/README.md b/README.md
@@ -1,30 +1,27 @@
 # strongbox, a World of Warcraft addon manager
 
-`strongbox` is an **open source**, **[advertisement free](#recognition)** and **[privacy respecting](#privacy)** addon manager for World of Warcraft.
+`strongbox` is an **open source**, **[advertisement free](#recognition)** and **[privacy respecting](#privacy)** addon
+manager for World of Warcraft.
 
 It runs on Linux and macOS.
 
-It supports addons hosted by ~Curseforge,~ wowinterface.com, Tukui, Github and Gitlab.
+It supports addons hosted by ~Curseforge,~ wowinterface.com, ~Tukui~, Github and Gitlab.
 
 ---
 
-***Notice***: tukui.org hosted addons no longer receive updates as of 2023-06-01 except `elvui` and `tukui`.
+***Notice***: tukui.org no longer hosts addons except `elvui` and `tukui` as of **2023-06-01**.
 
 Unfortunately the location of these two addons has changed as well as details around their access.
 
-I've decied to drop support for tukui.org and instead mirror the `tukui` and `elvui` releases on Github so they will be
-incorporated into the Github catalogue.
-
-In the meantime `File -> Import addon` and pasting the below URLs will install them:
-
-* https://github.com/ogri-la/tukui
-* https://github.com/ogri-la/elvui
+I've dropped support for tukui.org in **7.0.0** and instead mirror the `tukui` and `elvui` releases on Github. This
+allows them to be incorporated into the Github catalogue.
 
 ---
 
 ***Notice***: Curseforge addons no longer receive updates as of version **5.0.0**, released Feb 1st, 2022.
 
-Use the *"Source"* and *"Find similar"* actions from the addon context menu ([added **4.9.0**](https://github.com/ogri-la/strongbox/releases)) to help migrate addons away from Curseforge.
+Use the *"Source"* and *"Find similar"* actions from the addon context menu
+([added **4.9.0**](https://github.com/ogri-la/strongbox/releases)) to help migrate addons away from Curseforge.
 
 I also maintain a list of [other addon managers](https://ogri-la.github.io/wow-addon-managers/).
 
@@ -82,7 +79,7 @@ Afterwards, use the `Update all` button to update all addons with new versions a
 * [install addons from multiple sources](#install-addons-from-multiple-sources):
     - ~Curseforge~
     - wowinterface.com
-    - Tukui
+    - ~Tukui~
     - Github (using *releases*)
     - Gitlab (using *releases*)
 * [import and export lists of addons](#import-and-export-lists-of-addons)
@@ -121,9 +118,7 @@ that it's my privilege to offer this small piece back.
 
 This software interacts with the following remote hosts:
 
-* ~Overwolf/Twitch/Curseforge [Addons API](https://addons-ecs.forgesvc.net/) and its [CDN](https://edge.forgecdn.net/)~
 * [wowinterface.com](https://wowinterface.com)
-* [www.tukui.org](https://www.tukui.org/api.php)
 * [api.github.com](https://developer.github.com/v3/repos/releases)
     - to download repository and release data for addons hosted on Github
     - to download the latest `strongbox` release information
@@ -162,7 +157,8 @@ bug. *Some* of the details it contains are:
 
 ### classic and retail addon support
 
-"Classic", "Classic (The Burning Crusade)", "Classic (Wrath of the Lich King)" and "Retail" versions of WoW are all distinct addon systems.
+"Classic", "Classic (The Burning Crusade)", "Classic (Wrath of the Lich King)" and "Retail" versions of WoW are all
+distinct addon systems.
 
 Some addons support all systems in a single download, some support classic as an alternate build of the same addon, 
 some addons support classic only, some addons have been split up into multiple addons. There is a lot of variation.
@@ -196,9 +192,7 @@ Click `File` from the top menu and select `Import addon` and paste the URL of th
 
 Strongbox supports searching for addons from the following addon hosts:
 
-* ~[Curseforge](https://www.curseforge.com/wow/addons)~
 * [wowinterface.com](https://wowinterface.com/addons.php)
-* [Tukui](https://www.tukui.org)
 * [Github](https://github.com)
 
 Click the `search` tab and start typing.
@@ -238,9 +232,7 @@ Click the `Update all` button next to your addon directory.
 
 Strongbox supports installing addons from the following addon hosts:
 
-* ~[Curseforge](https://www.curseforge.com/wow/addons)~
 * [wowinterface.com](https://wowinterface.com/addons.php)
-* [Tukui](https://www.tukui.org)
 * [Github](https://www.github.com)
 * [Gitlab](https://gitlab.com)
 
@@ -327,7 +319,6 @@ Right-click an addon and select `Release`.
 
 Strongbox currently supports installing previous releases for:
 
-* ~Curseforge~
 * Github
 * Gitlab
 
@@ -408,6 +399,6 @@ Prior to `1.0.0`, `strongbox` was known as `wowman`. The [AUR package](https://a
 
 ## License
 
-Copyright © 2018-2022 Torkus
+Copyright © 2018-2023 Torkus
 
 Distributed under the GNU Affero General Public Licence, version 3 [with additional permissions](LICENCE.txt#L665)
diff --git a/TODO.md b/TODO.md
@@ -112,21 +112,31 @@ see CHANGELOG.md for a more formal list of changes by release
 * size column should be right aligned
     - done
 
-## todo
-
-
-
-* update screenshots
-
 * remove tukui
     - I'm not keeping all this logic for two addons
-    - stop catalogue addons from being merged
-        -...
+    - stop catalogue addons from being merged into full catalogue
+        - done
     - can the addons be mirrored/hosted on github?
-        - ...
+        - done
+    - follow curseforge removal pr
+        - done
+
+## todo
+
+* update screenshots
 
 ## todo bucket (no particular order)
 
+* search results, if there are addons from the same host (github) with the same name (tukui), disambiguate them
+    - 'tukui' in the search results shouldn't mean 'ogri-la/tukui' if 'tukui.org/tukui' is also available
+        - which it isn't, but that's not the point.
+
+* expand-summary, can the game track wrangling logic be made generic rather than per-host?
+    - this would tie in with returning *all* releases from a host
+    - it would insulate host logic from selected game tracks and game track strictness
+    - it would handle pinned logic as well
+        - currently handled in catalogue
+
 * user catalogue, refreshing may guarantee exceeding github limit.
     - if we know this, add a warning? refuse? stop?
 

diff --git a/cloverage.clj b/cloverage.clj
@@ -7,8 +7,7 @@
     [http :as http]
     [joblib :as joblib]
     [logging :as logging]
-    [core :as core]
-    [catalogue :as catalogue]]
+    [core :as core]]
    [clojure.test :as test]
    [cloverage.coverage :as c]))
 
@@ -24,7 +23,6 @@
                   http/*default-pause* 1 ;; ms
                   http/*default-attempts* 1
                   ;;joblib/tick-delay joblib/*tick*
-                  catalogue/host-disabled? (constantly false)
                   utils/folder-size-bytes (constantly 0)
                   constants/max-user-catalogue-age 9999]
       (core/reset-logging!)

diff --git a/repl/strongbox/user.clj b/repl/strongbox/user.clj
@@ -5,6 +5,7 @@
    [clojure.test]
    [clojure.tools.namespace.repl :as tn :refer [refresh]]
    [strongbox
+    [addon :as addon]
     [constants :as constants]
     [main :as main :refer [restart stop]]
     [catalogue :as catalogue]
@@ -35,7 +36,6 @@
                   ;;cli/install-update-these-in-parallel cli/install-update-these-serially
                   ;;core/check-for-updates core/check-for-updates-serially
                   ;; for testing purposes, no addon host is disabled
-                  catalogue/host-disabled? (constantly false)
                   utils/folder-size-bytes (constantly 0)
                   constants/max-user-catalogue-age 9999
                   ]

diff --git a/src/strongbox/addon.clj b/src/strongbox/addon.clj
@@ -18,6 +18,12 @@
 
 (def dummy-dirname "not-the-addon-dir-you-are-looking-for")
 
+(defn-spec host-disabled? boolean?
+  "returns `true` if the addon host has been disabled"
+  [addon map?]
+  (or (-> addon :source (= "curseforge"))
+      (-> addon :source (utils/in? sp/tukui-source-list))))
+
 (defn-spec -remove-addon! nil?
   "safely removes the given `addon-dirname` from `install-dir`.
   if the given `addon-dirname` is a mutual dependency with another addon, just remove it's entry from
@@ -155,11 +161,9 @@
 (defn-spec merge-toc-nfo (s/or :ok map?, :empty nil?)
   "merges `toc` data with `nfo` data with special handling for the `source-map-list`."
   [toc (s/nilable map?), nfo (s/nilable map?)]
-  (let [curse? (fn [source-map]
-                 (= (:source source-map) "curseforge"))
-        source-map-list (some->> (merge-lists (extract-source-map-list toc)
+  (let [source-map-list (some->> (merge-lists (extract-source-map-list toc)
                                               (extract-source-map-list nfo))
-                                 (remove curse?)
+                                 (remove host-disabled?)
                                  vec
                                  (assoc {} :source-map-list))]
     (merge toc nfo source-map-list)))

diff --git a/src/strongbox/catalogue.clj b/src/strongbox/catalogue.clj
@@ -15,11 +15,6 @@
     [gitlab-api :as gitlab-api]
     [github-api :as github-api]]))
 
-(defn-spec host-disabled? boolean?
-  "returns `true` if the addon host has been disabled"
-  [addon map?]
-  (-> addon :source (= "curseforge")))
-
 (defn-spec -expand-summary (s/or :ok :addon/expanded, :error nil?)
   "fetches updates from the addon host for the given `addon` and `game-track`.
   does *not* support multiple game tracks or warning the user, see `expand-summary`.
@@ -62,24 +57,32 @@
         game-track* game-track
         game-track (some #{game-track} sp/game-tracks)] ;; :retail => :retail, :unknown-game-track => nil
     (cond
-      (not game-track) (error (format "unsupported game track '%s'." (str game-track*)))
-      (host-disabled? addon) (warn (utils/message-list (str "addon host 'curseforge' was disabled " constants/curseforge-cutoff-label ".")
-                                                       ["use 'Source' and 'Find similar' from the addon context menu for alternatives."]))
-      :else (if-let [source-updates (if strict?
-                                      (-expand-summary addon game-track)
-                                      (utils/first-nn (partial -expand-summary addon) (get track-map game-track)))]
-              source-updates
-
-              ;; "no 'Retail' release found on github"
-              ;; "no 'Classic' release found on wowinterface"
-              ;; "no 'Classic (TBC)', 'Classic' or 'Retail' release found on github"
-              (let [single-template "no '%s' release found on %s."
-                    multi-template "no '%s', '%s', '%s' or '%s' release found on %s."
-                    msg (if strict?
-                          (format single-template (sp/game-track-labels-map game-track) (:source addon))
-                          (apply format multi-template (conj (mapv #(sp/game-track-labels-map %) (get track-map game-track))
-                                                             (:source addon))))]
-                (warn msg))))))
+      (not game-track)
+      (error (format "unsupported game track '%s'." (str game-track*)))
+
+      (strongbox.addon/host-disabled? addon)
+      (if (= (:source addon) "curseforge")
+        (warn (utils/message-list (str "addon host 'curseforge' was disabled " constants/curseforge-cutoff-label ".")
+                                  ["use 'Source' and 'Find similar' from the addon context menu for alternatives."]))
+        (warn (utils/message-list (str "addon host 'tukui' was disabled " constants/tukui-cutoff-label ".")
+                                  ["use 'Source' and 'Find similar' from the addon context menu for alternatives."])))
+
+      :else
+      (if-let [source-updates (if strict?
+                                (-expand-summary addon game-track)
+                                (utils/first-nn (partial -expand-summary addon) (get track-map game-track)))]
+        source-updates
+
+        ;; "no 'Retail' release found on github"
+        ;; "no 'Classic' release found on wowinterface"
+        ;; "no 'Classic (TBC)', 'Classic' or 'Retail' release found on github"
+        (let [single-template "no '%s' release found on %s."
+              multi-template "no '%s', '%s', '%s' or '%s' release found on %s."
+              msg (if strict?
+                    (format single-template (sp/game-track-labels-map game-track) (:source addon))
+                    (apply format multi-template (conj (mapv #(sp/game-track-labels-map %) (get track-map game-track))
+                                                       (:source addon))))]
+          (warn msg))))))
 
 ;;
 

diff --git a/src/strongbox/config.clj b/src/strongbox/config.clj
@@ -28,7 +28,11 @@
   (let [curse-idx 3]
     (utils/drop-idx -default-catalogue-list--v2 curse-idx)))
 
-(def -default-catalogue-list -default-catalogue-list--v3)
+(def -default-catalogue-list--v4
+  (let [tukui-idx 2]
+    (utils/drop-idx -default-catalogue-list--v3 tukui-idx)))
+
+(def -default-catalogue-list -default-catalogue-list--v4)
 
 (def default-cfg
   {:addon-dir-list []
@@ -124,6 +128,12 @@
   (let [new-catalogue-list (vec (remove #(= :curseforge (:name %)) (:catalogue-location-list cfg)))]
     (assoc cfg :catalogue-location-list new-catalogue-list)))
 
+(defn-spec remove-tukui-catalogue map?
+  "removes the tukui catalogue from the user config."
+  [cfg map?]
+  (let [new-catalogue-list (vec (remove #(= :tukui (:name %)) (:catalogue-location-list cfg)))]
+    (assoc cfg :catalogue-location-list new-catalogue-list)))
+
 (defn-spec handle-column-preferences map?
   "handles upgrading of the default column list.
   if the config is using the v1 defaults, upgrade to v2 defaults."
@@ -174,6 +184,7 @@
                 remove-invalid-catalogue-location-entries
                 add-github-catalogue
                 remove-curseforge-catalogue
+                remove-tukui-catalogue
                 handle-column-preferences
                 strip-unspecced-keys)
         message (format "configuration from %s is invalid and will be ignored: %s"

diff --git a/src/strongbox/constants.clj b/src/strongbox/constants.clj
@@ -64,6 +64,7 @@
 (def glyph-map glyph-map--fontawesome)
 
 (def curseforge-cutoff-label "Feb 1st, 2022")
+(def tukui-cutoff-label "June 1st, 2023")
 
 (def releases
   "https://wowpedia.fandom.com/wiki/Patch"

diff --git a/src/strongbox/core.clj b/src/strongbox/core.clj
@@ -690,7 +690,7 @@
       :short (catalogue/shorten-catalogue catalogue)
       ;;:curseforge (catalogue/filter-catalogue catalogue "curseforge")
       :wowinterface (catalogue/filter-catalogue catalogue "wowinterface")
-      :tukui (catalogue/filter-catalogue catalogue "tukui")
+      ;;:tukui (catalogue/filter-catalogue catalogue "tukui")
       :github (catalogue/filter-catalogue catalogue "github") ;; todo: add a test for this. I expect all derivable catalogues to be available
       nil)))
 
@@ -733,9 +733,7 @@
   "returns the contents of the user catalogue or `nil` if it doesn't exist."
   []
   (let [catalogue (catalogue/read-catalogue (paths :user-catalogue-file) {:bad-data? nil})
-        curse? (fn [addon]
-                 (-> addon :source (= "curseforge")))
-        new-summary-list (->> catalogue :addon-summary-list (remove curse?) vec)]
+        new-summary-list (->> catalogue :addon-summary-list (remove addon/host-disabled?) vec)]
     (when catalogue
       (catalogue/new-catalogue new-summary-list))))
 
@@ -1221,6 +1219,9 @@
                               (= source "curseforge")
                               (error (str "addon host 'curseforge' was disabled " constants/curseforge-cutoff-label "."))
 
+                              (utils/in? source sp/tukui-source-list)
+                              (error (str "addon host 'tukui' was disabled " constants/tukui-cutoff-label "."))
+
                               :else
                               ;; look in the current catalogue. emit an error if we fail
                               (or (:catalogue-match (-find-first-in-db (or (get-state :db) []) addon-summary-stub match-on-list))
@@ -1615,9 +1616,7 @@
                                                  :invalid-data? nil-me
                                                  :transform-map {:game-track keyword}})
 
-        curse? (fn [addon]
-                 (some-> addon :source (= "curseforge")))
-        addon-list (remove curse? addon-list)
+        addon-list (remove addon/host-disabled? addon-list)
 
         full-data? (fn [addon]
                      (utils/all (mapv #(contains? addon %) [:source :source-id :name])))

diff --git a/src/strongbox/specs.clj b/src/strongbox/specs.clj
@@ -270,6 +270,7 @@
 (s/def :addon/category string?)
 (s/def :addon/category-list (s/coll-of :addon/category))
 
+(def tukui-source-list #{"tukui" "tukui-classic" "tukui-classic-tbc" "tukui-classic-wotlk"})
 (s/def :addon/source (s/or :known #{"curseforge" "wowinterface" "github" "gitlab" "tukui" "tukui-classic" "tukui-classic-tbc" "tukui-classic-wotlk"}
                            :unknown string?))
 (s/def :addon/source-id (s/or ::integer-id? int? ;; tukui has negative ids