Document event life cycle

[Krinkle]

We have:

• a list of callback methods at https://qunitjs.com/api/callbacks/
• a list of events at https://qunitjs.com/api/callbacks/QUnit.on/
• module hooks https://qunitjs.com/api/QUnit/module/#hooks
• global hooks at https://qunitjs.com/api/QUnit/hooks/

While each is well-documented, and describes when it runs, and how (sync or async), we don't have yet provided an explicit overview of the overall order of events. Where this matters most is:

• between QUnit.begin() and QUnit.on('runStart') which logically run at the same time. Idem for QUnit.done() and QUnit.on('runEnd'). Consider a sentence like "QUnit.config.urlConfig modifications must be made before QUnit.begin". Does that mean runStart is fine, or is that too late? And is that supported or accidental?
• between hooks.afterEach and QUnit.testDone(), where the order is not obvious per se, ref New test reporter hook #1475.

Life cycle

I wrote up the following to get us started.

• QUnit.on('runStart') ^{[reporter, sync]}
• QUnit.begin() ^{[plugin, async-await]}
• For each module:
  • QUnit.on('suiteStart') ^{[reporter, sync]}
  • QUnit.moduleStart() ^{[plugin, async-await]}
  • For each test:
    • QUnit.on('testStart') ^{[reporter, sync]}
    • QUnit.testStart() ^{[plugin, async-await]}
    • hooks.before() ^{[testing, async-await]}
    • global QUnit.hooks.beforeEach() ^{[plugin, async-await]}
    • hooks.beforeEach() ^{[testing, async-await]}
    • QUnit.test() function ^{[testing, async-await]}
    • For each assertion:
      • QUnit.log() ^{[plugin, sync]}
    • hooks.afterEach() ^{[testing, async-await]}
    • global QUnit.hooks.afterEach() ^{[plugin, async-await]}
    • hooks.after() ^{[testing, async-await]}
    • QUnit.on('testEnd') ^{[reporter, sync]}
    • QUnit.testDone() ^{[plugin, async-await]}
  • QUnit.on('suiteEnd') ^{[reporter, sync]}
  • QUnit.moduleDone() ^{[plugin, async-await]}
• QUnit.on('runEnd') ^{[reporter, sync]}
• QUnit.done() ^{[plugin, async-await]}

Would a table be clearer?

Event	Purpose	Note
QUnit.on('runStart')	✏️ reporter event [sync]
QUnit.begin()	⚙️ plugin callback [async-await]
📦 QUnit.on('suiteStart')	✏️ reporter event [sync]	For each module
📦 QUnit.moduleStart()	⚙️ plugin callback [async-await]	For each module
↦ QUnit.on('testStart')	✏️ reporter event [sync]	For each test
↦ QUnit.testStart()	⚙️ plugin callback [async-await]	For each test
↦ hooks.before()	testing [async-await]	Before first test in a module
↦ global QUnit.hooks.beforeEach()	⚙️ plugin [async-await]	For each test
↦ hooks.beforeEach()	testing [async-await]	For each test
↦ QUnit.test() function	testing [async-await]	For each test
↦ QUnit.log()	⚙️ plugin callback [sync]	For each assertion
↦ hooks.afterEach()	testing [async-await]	For each test
↦ global QUnit.hooks.afterEach()	⚙️ plugin [async-await]	For each test
↦ hooks.after()	testing [async-await]	After last test in a module
↦ QUnit.on('testEnd')	✏️ reporter event [sync]	For each test
↦ QUnit.testDone()	⚙️ plugin callback [async-await]	For each test
📦 QUnit.on('suiteEnd')	✏️ reporter event [sync]	For each module
📦 QUnit.moduleDone()	⚙️ plugin callback [async-await]	For each module
QUnit.on('runEnd')	✏️ reporter event [sync]
QUnit.done()	⚙️ plugin callback [async-await]

[Krinkle]

Inspired by #1328 (@raycohen), where afaik we did not change this order so this page could be version agnostic I think? (e.g. applicable to both QUnit 2 and 3).

[BillyRayPreachersSon]

• For each test:
  • hooks.before()
  • global QUnit.hooks.beforeEach()
  • hooks.beforeEach()
  • QUnit.test() function

This is what I'm seeing, but it's not what the current docs imply should happen:

Hooks that run before a test, are ordered from outer-most to inner-most, in the order that they are added. This means that a test will first run any global beforeEach hooks, then the hooks of parent modules, and finally the hooks added to the current module that the test is part of.

This says to me that a top-level beforeEach hook (the hooks of parent modules) will run before a nested before hook (finally the hooks added to the current module), but that just isn't the case. Take this example:

import { module, test } from 'qunit';

module('Top-level module', function(hooks) {
  hooks.before(function() {
    console.log('>> top-level before');
  });

  hooks.beforeEach(function() {
    console.log('>> top-level beforeEach');
  });

  module('Nested module', function(hooks) {
    hooks.before(function() {
      console.log('>> nested before');
    });

    hooks.beforeEach(function() {
      console.log('>> nested beforeEach');
    });

    test('log', function() {});
  });
});

If the docs are correct, and the hooks of parent modules are run before the hooks added to the current module that the test is part of, I would expect to see this:

>> top-level before
>> top-level beforeEach
>> nested before
>> nested beforeEach

However, what I actually see is:

>> top-level before
>> nested before
>> top-level beforeEach
>> nested beforeEach

Frustratingly, I actually want the expected behaviour in one of my suites, but regardless, it would be good to have clarity on the order - so thanks for documenting it 🙂 .

[Krinkle]

@BillyRayPreachersSon That text is referring to order within a single hook. "before", "beforeEach", "afterEach", and "after", are four different hooks.

This is important as it allows children to build on the work of parents, yet also allow each test to cleanly inherent the module state. For compatibility, and based on many years of use cases being expressed in satisfactory ways, we'd not likely change this. But, those docs can definitely be clearer.

If you provide a more detailed example of what you're trying to do, I'd be happy to review that for you and see if we can make it more intuitive. I'm sure you don't need help to make it work, but I wouldn't want you to feel the end result is counter-intuitive. Hence I'd like to see an example, and look for what may've led to that as an expectation, and to perhaps improve something aside or in addition (without breaking change) to better accommodate that!