tools/boards.txt.py rewrite

cores/esp8266/FlashMap.h

@@ -1,8 +1,6 @@
+// - DO NOT EDIT - autogenerated by boards.txt.py
 
-// - do not edit - autogenerated by boards.txt.py
-
-#ifndef __FLASH_MAP_H
-#define __FLASH_MAP_H
+#pragma once
 
 #include <stdint.h>
 #include <stddef.h>
@@ -55,5 +53,3 @@ typedef struct
         { .eeprom_start = 0x4027b000, .fs_start = 0x4027b000, .fs_end = 0x4027b000, .fs_block_size = 0x0, .fs_page_size = 0x0, .flash_size_kb = 512 }, \
     }
 
-#endif // __FLASH_MAP_H
-

doc/faq/a05-board-generator.rst

@@ -19,24 +19,22 @@ available in the IDE menu.
 How can I run the script ?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Python needs to be installed on your system.
-
-The script is located in the ``tools`` subdirectory of the core's root installation.
-It needs to be run from the root directory,
+Python needs to be installed on your system.  Either call the script directly, or use specific `python` binary.
+Running without any parameters will show a command line help.
 
 ::
 
-    $ tools/boards.txt.py
+    $ python tools/boards.txt.py
+    $ py -3 tools/boards.txt.py
 
-::
 
-    C:\...> tools\boards.txt.py
-    C:\...> python tools\boards.txt.py
+The Core location depends on your environment.  For example, using Windows:
 
-Running without parameters will show the command line help.  They are
-generally self-explanatory.  Running with the parameters will show no output but will generate a new boards.txt file (and a backup boards.txt.orig).
+* for Arduino IDE git installation, it is %USERPROFILE%\Documents\Arduino\hardware\esp8266com\esp8266\
 
-The core root directory varies depending on your development environment.  In Windows, core root is found under your home directory; for Arduino it is in AppData\Local\Arduino15\packages\esp8266\hardware\esp8266\2.4.2\ for PlatformIO it is in .platformio\packages\framework-arduinoespressif8266.
+* for Arduino IDE boards manager installation, it is usually %LOCALAPPDATA%\Arduino15\packages\esp8266\hardware\esp8266\VERSION\
+
+* for PlatformIO, it is either %USERPROFILE%\.platformio\packages\framework-arduinoespressif8266 or C:\.platformio
 
 
 What can I do with it ?
@@ -50,9 +48,9 @@ As of today you can:
 
 * increase available flash space by disabling floats in ``*printf`` functions
 
-* change led pin ``LED_BUILTIN`` for the two generic boards
+* change led pin ``LED_BUILTIN`` for the generic boards
 
-* create an abridged boards.txt file
+* re-create boards.txt file with a different set of boards, or create a boards.local.txt
 
 
 When do I need to mess with it ?
@@ -65,7 +63,7 @@ when possible.  It needs to be edited for:
   board (definition, description) can be updated or added to the existing
   list.
 
-* Memory mapping for ldscripts (flash and spiffs size combinations)
+* Memory mapping for ldscripts (flash and filesystem size combinations)
 
 
 Why is my pull-request failing continuous-integration ?
@@ -75,64 +73,58 @@ The generator is able to update a number of files (see list in help), and
 global coherency can be checked by the continuous integration facilities.
 
 After a modification in the generator, it is **mandatory** to regenerate all
-files (option ``--allgen``) and add them in the pull-request.
+files and include them in the pull-request.
+
+::
+
+    ./tools/boards.txt.py generate --all
+    git add -u -p
+    git commit
 
 
 How to create an abridged boards.txt file
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 The list of boards presented by the IDE has gotten quite long. You can reduce
-the ESP8266 boards shown by the IDE to a favorites list. This can
-be done by generating a new boards.txt file using the ``--filter <file>``
-option.
+the number of ESP8266 boards used in the IDE with a favorites list. This can
+be done by generating a new boards.txt file using the ``--include <file>``
+or ``--exclude <file>`` options.
 
 Start by getting a current list of boards supported by boards.txt.py.
 This command will write a list of supported board names to favorites.txt.
 
 ::
 
-    ./tools/boards.txt.py --boardnames >favorites.txt
+    ./tools/boards.txt.py names > favorites.txt
 
-Edit favorites.txt, keeping the name of the boards you want generated in
-boards.txt.
+Edit favorites.txt, keeping only the names you want to keep.
 
 to generate a new abridged boards.txt run:
 
 ::
 
-   ./tools/boards.txt.py --boardsgen --filter favorites.txt
-
-
-You can turn the process around by creating a list of boards, you do not want
-to be generated. To do this we use the ``--xfilter <file>`` option.
-
-to generate this abridged boards.txt run:
-
-::
-
-    ./tools/boards.txt.py --boardsgen --xfilter favorites.txt
-
+   ./tools/boards.txt.py --include favorites.txt generate --boards
 
-Yet another option, you can split the boards between boards.txt and
-boards.local.txt.
 
-The commands below will generate a boards.txt file that omits the boards named
-in favorites.txt, and generates a boards.local.txt ( via option ``--boardslocalgen`` ) that only contains boards
+The commands below will overwrite the boards.txt file and omit every board named
+in favorites.txt, and the next one will generate a boards.local.txt that only contains boards
 named in favorites.txt.
 
 ::
 
-    ./tools/boards.txt.py --boardsgen --xfilter favorites.txt
-    ./tools/boards.txt.py --boardslocalgen --filter favorites.txt
+    ./tools/boards.txt.py --exclude favorites.txt generate --boards
+    ./tools/boards.txt.py --include favorites.txt generate --boards --boards-file boards.local.txt
 
 Additional Notes:
 
-1. The boards.txt file will always contain the generic and esp8285 boards.
+1. Arduino IDE requires at least one board to be specified in boards.txt.
+
+2. Using filtering you could omit our default boards - ``generic`` and ``esp8285``.
 
-2. If boards.txt file exist and no backup copy named boards.txt.orig exist, the current boards.txt will be renamed to boards.txt.orig. Otherwise, the existing boards.txt is over-written when you generate a new boards.txt file. Similar behavior for when generating a new boards.local.txt.
+3. By default, generators will overwrite existing files. Use ``--output=stdout`` to redirect output to console.
 
-3. The boards in the boards.txt file will be in the order they were listed in your favorites file, specified by option ``--filter <file>``.
+4. Use ``--output=file-with-orig`` to preserve existing files as ``<filename>.orig``.
 
-4. It is outside the scope of this document, but you could manually edit any boards.txt file to have fewer boards. One last observation, the Arduino IDE appears to need at least one board in a board.txt file.
+5. The boards in the boards.txt file will be in the order they were listed in your favorites file, specified by option ``--include <file>``. By default, boards are sorted alphabetically.
 
 `FAQ list :back: <readme.rst>`__

doc/faq/readme.rst

@@ -46,7 +46,7 @@ How can I get some extra KBs in flash ?
 * Using ``*printf()`` with floats is enabled by default.  Some KBs of flash can
   be saved by using the option ``--nofloat`` with the boards generator:
 
-  ``./tools/boards.txt.py --nofloat --boardsgen``
+  ``./tools/boards.txt.py --no-float generate --boards``
 
 * Use the debug level option ``NoAssert-NDEBUG`` (in the Tools menu)
 
@@ -55,21 +55,7 @@ How can I get some extra KBs in flash ?
 About WPS
 ~~~~~~~~~
 
-From release 2.4.2 and ahead, not using WPS will give an extra ~4.5KB in
-heap.
-
-In release 2.4.2 only, WPS is disabled by default and the board generator is
-required to enable it:
-
-``./tools/boards.txt.py --allowWPS --boardsgen``
-
-`Read more <a05-board-generator.rst>`__.
-
-For platformIO (and maybe other build environments), you will also need to add the build flag: -D NO_EXTRA_4K_HEAP
-
-This manual selection is not needed starting from 2.5.0 (and in git
-version).  WPS is always available, and not using it will give an extra
-~4.5KB compared to releases until 2.4.1 included.
+Starting from release 2.5.0, WPS support is enabled automatically.
 
 This Arduino library doesn't work on ESP. How do I make it work?
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

package/build_boards_manager_package.sh

@@ -146,8 +146,8 @@ if [ ! -z "${DOC_URL}" ]; then
         .packages[0].platforms[0].help.online = \"${DOC_URL}\""
 fi
 
-cat ${srcdir}/package/package_esp8266com_index.template.json | \
-    jq "${jq_arg}" > package_esp8266com_index.json
+jq "${jq_arg}" ${srcdir}/package/package_esp8266com_index.template.json \
+    > package_esp8266com_index.json
 
 # Use Github API token, if available
 curl_gh_token_arg=()

package/package_esp8266com_index.template.json

@@ -24,109 +24,109 @@
                      "name": "Generic ESP8285 Module"
                   },
                   {
-                     "name": "Lifely Agrumino Lemon v4"
+                     "name": "4D Systems gen4 IoD Range"
                   },
                   {
-                     "name": "ESPDuino (ESP-13 Module)"
+                     "name": "Adafruit Feather HUZZAH ESP8266"
                   },
                   {
-                     "name": "Adafruit Feather HUZZAH ESP8266"
+                     "name": "Amperka WiFi Slot"
                   },
                   {
-                     "name": "WiFi Kit 8"
+                     "name": "Arduino"
                   },
                   {
-                     "name": "Invent One"
+                     "name": "DOIT ESP-Mx DevKit (ESP8285)"
                   },
                   {
-                     "name": "XinaBox CW01"
+                     "name": "Digistump Oak"
                   },
                   {
-                     "name": "ESPresso Lite 1.0"
+                     "name": "ESPDuino (ESP-13 Module)"
                   },
                   {
-                     "name": "ESPresso Lite 2.0"
+                     "name": "ESPectro Core"
                   },
                   {
-                     "name": "Phoenix 1.0"
+                     "name": "ESPino (ESP-12 Module)"
                   },
                   {
-                     "name": "Phoenix 2.0"
+                     "name": "ESPresso Lite 1.0"
                   },
                   {
-                     "name": "NodeMCU 0.9 (ESP-12 Module)"
+                     "name": "ESPresso Lite 2.0"
                   },
                   {
-                     "name": "NodeMCU 1.0 (ESP-12E Module)"
+                     "name": "ITEAD Sonoff"
                   },
                   {
-                     "name": "Olimex MOD-WIFI-ESP8266(-DEV)"
+                     "name": "Invent One"
                   },
                   {
-                     "name": "SparkFun ESP8266 Thing"
+                     "name": "LOLIN(WEMOS) D1 R2 & mini"
                   },
                   {
-                     "name": "SparkFun ESP8266 Thing Dev"
+                     "name": "LOLIN(WEMOS) D1 mini (clone)"
                   },
                   {
-                     "name": "SparkFun Blynk Board"
+                     "name": "LOLIN(WEMOS) D1 mini Lite"
                   },
                   {
-                     "name": "SweetPea ESP-210"
+                     "name": "LOLIN(WEMOS) D1 mini Pro"
                   },
                   {
-                     "name": "LOLIN(WEMOS) D1 R2 & mini"
+                     "name": "LOLIN(WeMos) D1 R1"
                   },
                   {
-                     "name": "LOLIN(WEMOS) D1 mini (clone)"
+                     "name": "Lifely Agrumino Lemon v4"
                   },
                   {
-                     "name": "LOLIN(WEMOS) D1 mini Pro"
+                     "name": "NodeMCU 0.9 (ESP-12 Module)"
                   },
                   {
-                     "name": "LOLIN(WEMOS) D1 mini Lite"
+                     "name": "NodeMCU 1.0 (ESP-12E Module)"
                   },
                   {
-                     "name": "LOLIN(WeMos) D1 R1"
+                     "name": "Olimex MOD-WIFI-ESP8266(-DEV)"
                   },
                   {
-                     "name": "ESPino (ESP-12 Module)"
+                     "name": "Phoenix 1.0"
                   },
                   {
-                     "name": "ThaiEasyElec's ESPino"
+                     "name": "Phoenix 2.0"
                   },
                   {
-                     "name": "WifInfo"
+                     "name": "Schirmilabs Eduino WiFi"
                   },
                   {
-                     "name": "Arduino"
+                     "name": "Seeed Wio Link"
                   },
                   {
-                     "name": "4D Systems gen4 IoD Range"
+                     "name": "SparkFun Blynk Board"
                   },
                   {
-                     "name": "Digistump Oak"
+                     "name": "SparkFun ESP8266 Thing"
                   },
                   {
-                     "name": "WiFiduino"
+                     "name": "SparkFun ESP8266 Thing Dev"
                   },
                   {
-                     "name": "Amperka WiFi Slot"
+                     "name": "SweetPea ESP-210"
                   },
                   {
-                     "name": "Seeed Wio Link"
+                     "name": "ThaiEasyElec's ESPino"
                   },
                   {
-                     "name": "ESPectro Core"
+                     "name": "WiFi Kit 8"
                   },
                   {
-                     "name": "Schirmilabs Eduino WiFi"
+                     "name": "WiFiduino"
                   },
                   {
-                     "name": "ITEAD Sonoff"
+                     "name": "WifInfo"
                   },
                   {
-                     "name": "DOIT ESP-Mx DevKit (ESP8285)"
+                     "name": "XinaBox CW01"
                   }
                ],
                "toolsDependencies": [

tests/ci/build_boards.sh

@@ -1,14 +1,15 @@
 #!/bin/bash
 #
-# CI job which checks that boards.txt and package_esp8266com_index.template.json are up to date
+# CI job which checks that files generated with boards.txt.py are up to date
 
 set -ev
 
 cd $TRAVIS_BUILD_DIR
 
-tools/boards.txt.py --boardsgen --ldgen --packagegen --docgen
+tools/boards.txt.py test
+tools/boards.txt.py generate --all
 
 git diff --exit-code -- boards.txt \
                         doc/boards.rst \
                         tools/sdk/ld/
-git diff --exit-code -w -- package/package_esp8266com_index.template.json
+git diff --exit-code -w -- package/package_esp8266com_index.boards.json