ESP8266 memory layout - irom0_0_seg address

[sszczep]

Got a question regarding the following linker script bundled with Arduino core:

/* Flash Split for 4M chips */
/* sketch @0x40200000 (~1019KB) (1044464B) */
/* empty  @0x402FEFF0 (~2052KB) (2101264B) */
/* spiffs @0x40500000 (~1000KB) (1024000B) */
/* eeprom @0x405FB000 (4KB) */
/* rfcal  @0x405FC000 (4KB) */
/* wifi   @0x405FD000 (12KB) */

MEMORY
{
  dport0_0_seg :                        org = 0x3FF00000, len = 0x10
  dram0_0_seg :                         org = 0x3FFE8000, len = 0x14000
  irom0_0_seg :                         org = 0x40201010, len = 0xfeff0
}

PROVIDE ( _FS_start = 0x40500000 );
PROVIDE ( _FS_end = 0x405FA000 );
PROVIDE ( _FS_page = 0x100 );
PROVIDE ( _FS_block = 0x2000 );
PROVIDE ( _EEPROM_start = 0x405fb000 );
/* The following symbols are DEPRECATED and will be REMOVED in a future release */
PROVIDE ( _SPIFFS_start = 0x40500000 );
PROVIDE ( _SPIFFS_end = 0x405FA000 );
PROVIDE ( _SPIFFS_page = 0x100 );
PROVIDE ( _SPIFFS_block = 0x2000 );

INCLUDE "local.eagle.app.v6.common.ld"

Why irom0_0_seg is defined as org = 0x40201010, len = 0xfeff0 but not org = 0x40200000, len = 0x100000? Why the offset of 0x1010? Also shouldn't empty start from 0x40300000?

An extra question - why the FS region isn't defined right after the irom?

[earlephilhower]

Why irom0_0_seg is defined as org = 0x40201010, len = 0xfeff0

The bootloader, eboot, lives in the first 4K of flash.

Also shouldn't empty start from 0x40300000?

Probably, but it's just a comment and not used or defined anywhere so it's a no-op.

An extra question - why the FS region isn't defined right after the irom?

OTA image before flashing by eboot lives in the empty space.

See the online docs, there is a flash memory map with more detail.

[sszczep]

Since it looks like you are keeping two copies of everything, why even overwrite at all? Just make your boot loader pick either slot A or slot B and run it in place?

I considered it. Unfortunately eboot doesn't support it and I would need to create a custom bootloader. Also I'm not sure if that would work - OTA_CODE is not contained within 0x40200000 - 0x40300000, can you even jump from bootloader to addresses beyond 0x40300000? Also the other issue - I would need to alter FS address. LittleFS would have to be aware of the address depending on which rom you are booting into.

I've tried looking at eboot source code but there is a lot of magic happening there:

1. irom0_0_seg : org = 0x40240000, len = 0x32000 why those values specifically? Shouldn't it start at 0x40200000 and take at most 0x1010 bytes as defined in code linker?
2. #define APP_START_OFFSET 0x1000 why the offset of 4KB?

Im not sure what I would need to change in order to move code partition to 0x40202000 as in the layout sent earlier.

[earlephilhower]

I think you're basically writing a new eboot bootloader to implement what you're trying to do, so feel free to make whatever flash memory map makes sense. The ROM will only jump to the 1st app (i.e. the bootloader), and the rest is up to you.

Also, take a look at rboot, https://github.com/raburton/rboot . I don't know if it's been maintained, but it implements something very much like what you're trying to do.

[sszczep]

I understand now.

Regarding my previous questions:

1. Still not sure why and I don't think it really matters.
2. APP_START_OFFSET is an offset from 0x40200000. It points to the code file header (https://docs.espressif.com/projects/esptool/en/latest/esp8266/advanced-topics/firmware-image-format.html).

Also some of the things have clarified in the meantime:

1. Resulting binary file contains both eboot (0x000000 - 0x001000) and code (0x001000 - 0x100000)
2. irom0_0_seg starts from 0x40201010 because the code segment in binary file is offset by 0x001000 + 16 bytes for file header

I've decided to keep the original bootloader and irom0_0_seg start address. I have successfully managed to merge both firmware and fs into single binary so I finally have something to work with.

Once again, thank you for your help, wouldn't get that without you. Cheers!

[trullock]

@sszczep I'm in a similar situation, would you share your solution?

[trullock]

@earlephilhower This is slightly off topic but I'm missing how this works in order for me to progress with a similar situation to @sszczep

You said:

OTA image before flashing by eboot lives in the empty space.

Could you help me understand how OTA works? I cant find a good explaination. Are you saying the bits are downloaded into the empty partition, and then they are moved to irom0, overwriting the previous binary?

The rboot docs reveal this insight:

This allows individual roms to be up to 1MB in size, so long as they do not straddle an 8Mbit boundary on the flash. This means you could have four 1MB roms or 8 512KB roms on a 32Mbit flash (such as on the ESP-12), or a combination. Note, however, that you could not have, for example, a 512KB rom followed immediately by a 1MB rom because the 2nd rom would then straddle an 8MBit boundary.

I want to have an 800KB binary, which supports OTA. Can I shrink the irom0 partition to 800KB, and have 800KB of empty space for OTA? the OTA section would span an 8Mb boundary, but does that matter here as its not being loaded by the boot loader?

Any help greatly appreciated

Cheers

[earlephilhower]

Could you help me understand how OTA works? I cant find a good explaination. Are you saying the bits are downloaded into the empty partition, and then they are moved to irom0, overwriting the previous binary?

It's simplest to think of OTA as just an(the) app that the on-chip ROM jumps to on power-up. Eboot here will check for a flag saying "copy new code". If the flag is not set, Eboot will just jump to a fixed flash address to start the real application. If the flag is set, it will copy the staged data (located after end of prior app and before the beginning of a filesystem) to the main app address and then jump to it to start.

As for the rboot stuff, sorry but I never really looked into it. You might have better luck putting this question in their repo. If they're doing some magic with the on-chip XIP cache you may be limited to 512K boundaries by the HW itself. If it's via SW, then theoretically it doesn't matter and you could have any setup you wanted (practically, though, they may only have 8 slots or something they track).

[trullock]

@earlephilhower thanks bery much for your response. That helps me understand it a bit better. When you refer to eboot, is this the implementation you speak of?

Arduino/cores/esp8266/Updater.cpp

Line 344 in 8b33e2e

if (_command == U_FLASH) {

Updater.begin() looks to start writing bytes at the next available sector in program space, then updater.end() calls the eboot copy raw command to copy the bits over the top of the existing program. Am I reading that right?

Thanks again

[earlephilhower]

Not quite. That bit just writes the copy into the space between the end of the current app and the beginning of the FS, sets the "copy some data" flag, and reboots. Eboot proper is here: https://github.com/esp8266/Arduino/tree/master/bootloaders/eboot . That's the bit that does the actual OTA overwrite of the app.

[sszczep]

@trullock
This is how I defined partitions:

/* NAME             HEX START      HEX END   HEX LEN      SIZE_B   SIZE_KB */
/* ======================================================================= */
/* BOOTLOADER      0x40200000   0x40201000   0x01000      4096 B      4 KB */
/* CODE            0x40201000   0x40300000   0xFF000   1044480 B   1020 KB */
/* FS              0x40300000   0x403FE000   0xFE000   1040384 B   1016 KB */
/* OTA_BOOTLOADER  0x403FE000   0x403FF000   0x01000      4096 B      4 KB */
/* OTA_CODE        0x403FF000   0x404FE000   0xFF000   1044480 B   1020 KB */
/* OTA_FS          0x404FE000   0x405FC000   0xFE000   1040384 B   1016 KB */
/* EEPROM (UNUSED) 0x405FB000   0x405FC000   0x01000      4096 B      4 KB */
/* RF_CAL          0x405FC000   0x405FD000   0x01000      4096 B      4 KB */
/* WIFI            0x405FD000   0x40600000   0x03000     12288 B     12 KB */

/* EEPROM overlaps with OTA_FS. It is fine as we do not use EEPROM anywhere. */

MEMORY
{
  dport0_0_seg :                        org = 0x3FF00000, len = 0x00010
  dram0_0_seg :                         org = 0x3FFE8000, len = 0x14000
  irom0_0_seg :                         org = 0x40201010, len = 0xFEFF0
}

PROVIDE ( _FS_start = 0x40300000 );
PROVIDE ( _FS_end = 0x403FE000 );
PROVIDE ( _FS_page = 0x100 );
PROVIDE ( _FS_block = 0x2000 );
PROVIDE ( _OTA_start = 0x403FE000 );
PROVIDE ( _OTA_end = 0x405FC000 );
PROVIDE ( _EEPROM_start = 0x405FB000 );

Single binary image contains bootloader, code segment and filesystem, 2040KB in total. Each of those partitions have their corresponding OTA_* counterpart.

How it works in my case:

1. Build both firmware and filesystem image
2. Merge both binaries into single one: esptool.py --chip ESP8266 merge_bin -o firmware.bin --flash_mode dio --flash_size 4MB 0x0 .pio/build/debug/firmware.bin 0x100000 .pio/build/debug/littlefs.bin
3. Upload the resulting binary using html file form. It's being flashed to address 0x403FE000 - 0x405FC000 (2040KB).
4. Write command to eboot - copy 2040KB from (0x403FE000 - 0x40200000 = 0x1FE000) to 0x0
   note that those addresses start at 0x40200000, hence the offset
5. After the reboot of the device, eboot will automatically read the command and flash new memory region. Once it's done everything should be fine.

Also make sure to check out the eboot and OTA* implementations.

@earlephilhower please correct me if I'm wrong