docs/memory.txt: Clarify and expand priority/overlap documentation

The documentation of how overlapping memory regions behave and how
the priority system works was rather brief, and confusion about
priorities seems to be quite common for developers trying to understand
how the memory region system works, so expand and clarify it.
This includes a worked example with overlaps, documentation of the
behaviour when an overlapped container has "holes", and mention
that it's valid for a region to have both MMIO callbacks and
subregions (and how this interacts with priorities when it does).

Signed-off-by: Peter Maydell <[email protected]>
Reviewed-by: Michael S. Tsirkin <[email protected]>
Message-id: [email protected]
Signed-off-by: Anthony Liguori <[email protected]>

docs/memory.txt

@@ -52,6 +52,15 @@ MemoryRegion):
   hole".  Aliases may point to any type of region, including other aliases,
   but an alias may not point back to itself, directly or indirectly.
 
+It is valid to add subregions to a region which is not a pure container
+(that is, to an MMIO, RAM or ROM region). This means that the region
+will act like a container, except that any addresses within the container's
+region which are not claimed by any subregion are handled by the
+container itself (ie by its MMIO callbacks or RAM backing). However
+it is generally possible to achieve the same effect with a pure container
+one of whose subregions is a low priority "background" region covering
+the whole address range; this is often clearer and is preferred.
+Subregions cannot be added to an alias region.
 
 Region names
 ------------
@@ -85,6 +94,49 @@ you can use memory_region_add_subregion_overlap() both to specify a region
 that must sit 'above' any others (with a positive priority) and also a
 background region that sits 'below' others (with a negative priority).
 
+If the higher priority region in an overlap is a container or alias, then
+the lower priority region will appear in any "holes" that the higher priority
+region has left by not mapping subregions to that area of its address range.
+(This applies recursively -- if the subregions are themselves containers or
+aliases that leave holes then the lower priority region will appear in these
+holes too.)
+
+For example, suppose we have a container A of size 0x8000 with two subregions
+B and C. B is a container mapped at 0x2000, size 0x4000, priority 1; C is
+an MMIO region mapped at 0x0, size 0x6000, priority 2. B currently has two
+of its own subregions: D of size 0x1000 at offset 0 and E of size 0x1000 at
+offset 0x2000. As a diagram:
+
+        0      1000   2000   3000   4000   5000   6000   7000    8000
+        |------|------|------|------|------|------|------|-------|
+  A:    [                                                       ]
+  C:    [CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC]
+  B:                  [                          ]
+  D:                  [DDDDD]
+  E:                                [EEEEE]
+
+The regions that will be seen within this address range then are:
+        [CCCCCCCCCCCC][DDDDD][CCCCC][EEEEE][CCCCC]
+
+Since B has higher priority than C, its subregions appear in the flat map
+even where they overlap with C. In ranges where B has not mapped anything
+C's region appears.
+
+If B had provided its own MMIO operations (ie it was not a pure container)
+then these would be used for any addresses in its range not handled by
+D or E, and the result would be:
+        [CCCCCCCCCCCC][DDDDD][BBBBB][EEEEE][BBBBB]
+
+Priority values are local to a container, because the priorities of two
+regions are only compared when they are both children of the same container.
+This means that the device in charge of the container (typically modelling
+a bus or a memory controller) can use them to manage the interaction of
+its child regions without any side effects on other parts of the system.
+In the example above, the priorities of D and E are unimportant because
+they do not overlap each other. It is the relative priority of B and C
+that causes D and E to appear on top of C: D and E's priorities are never
+compared against the priority of C.
+
 Visibility
 ----------
 The memory core uses the following rules to select a memory region when the
@@ -94,11 +146,19 @@ guest accesses an address:
   descending priority order
   - if the address lies outside the region offset/size, the subregion is
     discarded
-  - if the subregion is a leaf (RAM or MMIO), the search terminates
+  - if the subregion is a leaf (RAM or MMIO), the search terminates, returning
+    this leaf region
   - if the subregion is a container, the same algorithm is used within the
     subregion (after the address is adjusted by the subregion offset)
-  - if the subregion is an alias, the search is continues at the alias target
+  - if the subregion is an alias, the search is continued at the alias target
     (after the address is adjusted by the subregion offset and alias offset)
+  - if a recursive search within a container or alias subregion does not
+    find a match (because of a "hole" in the container's coverage of its
+    address range), then if this is a container with its own MMIO or RAM
+    backing the search terminates, returning the container itself. Otherwise
+    we continue with the next subregion in priority order
+- if none of the subregions match the address then the search terminates
+  with no match found
 
 Example memory map
 ------------------