Introduce moveBefore() state-preserving atomic move API

[domfarolino]

This PR introduces a new DOM API on the Node interface: moveBefore(). It mirrors insertBefore() in shape, but defers to a new DOM manipulation primitive that this PR adds in service of this new API: the "move" primitive. The move primitive contains some of the DOM tree bookkeeping steps from the remove primitive, as well as the insert primitive, and does three more interesting things:

1. Calls the moving steps hook with the moved node and the old parent (possibly null, just like the removing steps)
2. Queues a custom element callback reaction for the connectedMoveCallback()
3. Queues two back-to-back mutation record tasks: one for removal from the old parent; one for insertion into the new one

The power of the move primitive comes from the fact that the algorithm does not defer to the traditional insert and removal primitives, and therefore does not invoke the removing steps and insertion steps. This allows most state to be preserved by default (i.e., we don't tear down iframes, or close dialogs). Sometimes, the insertion/removing step overrides in other specifications have steps that do need to be performed during a move anyways. These specifications are expected to override the moving steps hook and perform the necessary work accordingly. See whatwg/html#10657 for HTML.

Remaining tasks (some will be PRs in other standards):

• Custom element integration
• Keep popovers open
• Don't call post-connection steps if state-preserving atomic move is in progress
• Don't call becomes connected / becomes browsing-context
• Only disconnect subframes on removal when state-preserving atomic move is not in progress
• Keep dialogs open: see removing steps
• img/source: this shouldn't count as a relevant mutation
• Preserve fullscreen
• Preserve focus
  • Need to resolve focusin event semantics
• ~[ ] Don't reset animations / transitions. See here
  • Maybe nothing needs to be done here. Given how element removals are handled, the spec does NOT require transitions to be removed from the UA's set of running transitions for moved nodes since they are never removed from the Document.~
• [ ] Preserve text-selection. See set the selection range. Edit: Nothing needs to be done here. Selection metadata (i.e., selectionStart and kin) is preserved by default in browsers, consistent with HTML (no action is taken on removal). The UI behavior of the selection not being highlighted is a side-effect of the element losing focus
• Selection API: don't reset the Document's selection
  • Updates to the selection range should happen according to how the DOM Standard primitives update ranges. The Selection API specification admits as much, by deferring to the insert and removal algorithms. Therefore, we should reference the move primitive from the Selection API specification, and ensure that the move primitive in this DOM Standard PR updates live ranges correctly: Reference the move primitive in DOM mutations section w3c/selection-api#341
  • selectionchange event: We've decided to allow selectionchange event to still fire, since it is queued in a task. No changes for this part are required.
• Pointer event state reset: see here
  • The specification does not reset an element's previously-set pointer capture upon DOM node insertion or removal. While this seems like a gap in that specification ([DOM integration] No state is reset on node removal w3c/pointerevents#526), it appears that nothing needs to be done in that specification to satisfy this preservation semantic.
  • Test: https://github.com/web-platform-tests/wpt/blob/master/dom/nodes/moveBefore/tentative/pointer-events.html
• Hide input/select picker: here
  • Nothing in the specification seems to hook into DOM for removal when the picker dialog is shown, so no work needs to be done in this PR. See "Show the picker" steps never dismisses the picker upon node removal html#10743 for a follow-up.
• Preserve pointer lock: here
  • When a "pointer-lock target becomes disconnected", pointer lock is exited. Since the "becomes disconnected" hook is not run during an atomic move, pointer lock is preserved naturally.
• Containment: keep last remembered size(see here)

• At least two implementers are interested (and none opposed):
  • Chromium: https://issues.chromium.org/issues/40150299
  • WebKit: DOM State-preserving move (Node.prototype.moveBefore) WebKit/standards-positions#375
  • Gecko: DOM State-preserving move (Node.prototype.moveBefore) mozilla/standards-positions#1053
• Tests are written and can be reviewed and commented upon at:
  • https://github.com/web-platform-tests/wpt/tree/master/dom/nodes/moveBefore/tentative (need to mark as non-tentative)
• Implementation bugs are filed:
  • Chromium: https://issues.chromium.org/issues/40150299
  • Gecko: https://bugzilla.mozilla.org/show_bug.cgi?id=1923880
  • WebKit: https://bugs.webkit.org/show_bug.cgi?id=281223
• MDN issue is filed: …
• The top of this comment includes a clear commit message to use.

(See WHATWG Working Mode: Changes for more details.)

---

Preview | Diff

[annevk]

I think the mutation record needs some more design work. I would expect it to capture the information of a remove and an insert at the same time. Perhaps it needs to be a new object, though we could further overload the existing MutationRecord as well I guess. At least I think you need:

• old target
• target
• moved node (I'm not sure you can ever move multiple at this point, but maybe we should allow for it in the mutation record design?)
• old previous sibling
• old next sibling
• previous sibling
• next sibling

Would be good to know what @smaug---- thinks and maybe @ajklein even wants to chime in.

[annevk]

I think it would be cleaner if we introduced "move" instead of overloading "insert", though I'm willing to be convinced. At least I always viewed this as introducing "move" as the third primitive following "insert" and "remove".

[domfarolino]

Yeah I think initially I saw it as a separate primitive too, but the more I looked at it, the more the difference between the two seemed really subtle. It mostly has to do with MutationObservers (and half of the relevant logic here is tucked away in the "remove" primitive) and not running the post-connection steps. So I feel like we'd end up with a near line-by-line copy of "insert", modulo one or two small differences. I'll take another look to see if my intuition is accurate, but I do kinda suspect this is where we'd end up.

[annevk]

It'd be good to have more details. Because you can't move a DocumentFragment so a vast number of checks don't apply. Adopt won't ever run.

There might be a number of range and shadow tree checks that end we end up duplicating, but perhaps that calls for abstracting those instead. At least to me a state flag seems rather unappealing.

[domfarolino]

It's possible there is enough from "insert" that we'd omit in "move", to make it creation worth it, sure.

At least to me a state flag seems rather unappealing.

With a separate "move" primitive and the decision to throw when we can't "move", we could get rid of the state variable that's currently in "insert" (i.e., <var>statePreservingAtomicMoveInProgress</var>). But the state bool on Document is what other specifications will refer to in their removal steps to react to a move appropriately, so I'm not sure we can get rid of that one.

[annevk]

Hmm, wouldn't we give other specifications "move steps"?

[domfarolino]

I thought about this a bit more. I think we're happy to create a new "move" primitive algorithm following the "insert" and "remove" algorithms. As you mentioned, it will be a simpler algorithm than "insert" since we'll avoid the DocumentFragment stuff — oh and also the post-connection stuff, among maybe more things.

There's still the discussion around introducing either (a) new "move" steps/hook that other specs provide to react to moves, vs (b) a Document-scoped boolean that the new "move" algorithm sets, for existing insertion/removal steps to use, to not reset certain state inside those steps.

Option (b) is the direction of the current PR, which we think makes sense after implementing this in Chromium. I think this is because so far we've found no behavior that we specifically want to do during a move that isn't done by insertion/removal — so any "move steps" we provided would just be empty. Does that make sense?

Now, I think the real question here is whether we want to:

1. Keep running the insertion/removal steps during a "move" operation, and just pepper if during atomic move is true, don't run some of these substeps! wherever state would ordinarily be reset
   • Cancel out the iframe removal steps
   • Cancel img relevant mutations, to avoid re-firing load events
2. Simply not run insertion/removal hooks at all during a move. This is only feasible if we determine that there are absolutely no steps in any spec's existing insertion/removal hooks that need to run during a move. We'll have to do an audit to determine this, which I'm starting now.

So the way I see it, those are probably our options, as opposed to creating new move steps which I think from our findings so far, would be unused. (We'll course-correct if this finding changes).

[smaug----]

• old target

• target

• moved node (I'm not sure you can ever move multiple at this point, but maybe we should allow for it in the mutation record design?)

Shouldn't the target node be all the time the same, it is just the siblings which change.
So we'd need only oldPreviousSibling and oldNextSibling. Oh, hmm, this isn't only about moving children but moving anything.

If this is really just remove and add back elsewhere, we could just reuse the existing childList MutationRecords, one for remove, one for adding node back, and possibly just add a flag to MutationRecord that it was about move.

(movedNodes is a bit confusing, since it seems to depend on the connectedness of the relevant nodes and it is apparently empty for the removal part. And it is unclear to me why we need the connectedness check. This is about basic DOM tree operations, and I'd assume those to work the same way whether or not the node is connected)

[annevk]

Creating two separate mutation records that a consumer would have to merge to (fully) understand it's a move seems suboptimal?

I agree that it should probably work for disconnected nodes as well, but I don't think we want to support a case where the shadow-including root changes.

[ajklein]

It's been a long time since I've thought about this stuff, but I'm inclined to agree with @smaug---- that creating a new type of MutationRecord feels unnecessary. Users of MutationObserver already have to do coalescing if they want to make sense of the stream of changes they observe. There are already other move-like operations, such as appending a child that's already somewhere else in the tree, which today generates two records.

[smaug----]

Hmm, why do we have this limitation. Why not allow move within the same tree? Basically why the condition about shadow-including root isn't enough?

[domfarolino]

I believe we discussed this at TPAC. The idea is that moveBefore() is used to preserve state, so in cases where state cannot be preserved, we should not just fall back to usual insertBefore()-style behavior. Instead, we'll throw and let the developer handle this case themselves. Basically we let moveBefore() succeed only when we're sure we can perform a state-preserving move, and when disconnected destinations or origins are involved, the state cannot be preserved.

[smaug----]

But what state is there to preserve when you're in a disconnected subtree?

[domfarolino]

There are three cases:

• If moving from connected -> disconnected, then there's lots of state to preserve, and you can't preserve any of it.
• If moving going from disconnected -> connected, then there's probably no state to preserve — I could see us wanting to enable this case. I don't feel strongly though!
• If going from disconnected -> disconnected, then there's also probably no state to preserve. I could also see us wanting to enable this case, but throwing and forcing developers to use insertBefore() seems fine to me too.

It seems simplest to treat all of the above cases the same, with the lowest common denominator being the first case above. I am sympathetic to allowing the last two cases though. @annevk What do you think?

[smaug----]

I would definitely expect a move operation to succeed if one stays within a same tree, whether or not your connected to document.

[annevk]

I don't see how connected <> disconnected would work. It would need a completely different code path, no?

I can see connected <> connected and disconnected <> disconnected. Someone (either Noam or Dominic) argued against disconnected <> disconnected earlier, but it seems reasonable to allow as well.

[noamr]

I'm ok with both. One can say that disconnected<>disconnected doesn't have any state to preserve, so whether we throw or do the same as insertBefore is a judgement call.

[annevk]

We should not do the same as insertBefore. It should run the moving steps (whether anyone cares about those for disconnected is a different question).

[noamr]

Sure, I mean that it should be observably the same in the disconnected<>disconnected case.

[smaug----]

But what if the move moves end node to be before the start node. That would leave the Range to an odd state, no?

[domfarolino]

I don't think so. It should be equivalent to when a range's start container is set (to the first time) to a node after the end container as well. Here are tests for this, which include intersectsNode assertions to make sure the Range's state is not peculiar: https://github.com/web-platform-tests/wpt/blob/master/dom/nodes/moveBefore/tentative/live-range-updates.html.

[smaug----]

Ok, so something else still updates Range boundary points so that start is before end point? The note is then a bit confusing.

[domfarolino]

Ok, so something else still updates Range boundary points so that start is before end point?

No, a Range's start does not have to be before its end, in DOM tree order. Just like if you create a new Range whose start comes after its end in DOM tree order, the start's position will be "after" its end, which is observable via intersectsNode() since nodes that are after end but before start, will no longer intersectsNode() the Range. The same goes for when moveBefore() moves things around such that the range's start is after its end, in tree order.

[smaug----]

Wait, how do you create a Range which has its start after end? setStart/End etc will modify the other boundary point if the order isn't right.

[noamr]

@smaug---- perhaps a useful middle ground can be that we preserve the selection only if the moved node is an ancestor of the entire range. This way if the range is moved while "intact", the selection would be preserved, but if parts of it are moved it's considered "broken" and it's dissolved. This is useful when moving whole elements which might have bits of text selected inside them.

[smaug----]

Not sure what you mean with "wouldn't be able to preserve selection". Using the normal removal/addition rules would preserve selection the usual way. The range just gets modified.

[domfarolino]

I mean that it wouldn't be able to "preserve" selection at the previous start and end containers. Ordinarily, we can't preserve selection anchored at those containers, since those containers get removed from the DOM. But with moveBefore(), those containers never leave the DOM, so we have an opportunity to do something better by keeping ranges anchored at those containers, which is a higher-fidelity preservation.

[smaug----]

Container approach (not modifying the range if full contained) would be still problematic if the container was moved from light dom to shadow dom, and the range was a range which one can access from selection.getRangeAt(). Selection should not reveal nodes in shadow dom.
Also it might be a bit surprising if selection is preserved in some cases but not in others.

Maybe, unrelated to moveBefore, there should be some helper methods to create a StaticRange from Range, and then update selection later, if/when needed, using that StaticRange. Hmm, though StaticRange isn't quite enough, since one needs also direction.

[noamr]

I'm OK with having the constraint of breaking the selection if moved between tree scopes