Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(#9585): teach Sentinel sign language #9658

Merged
merged 16 commits into from
Nov 25, 2024
34 changes: 20 additions & 14 deletions sentinel/server.js
Original file line number Diff line number Diff line change
Expand Up @@ -35,20 +35,26 @@ const waitForApi = () => new Promise(resolve => {
});

logger.info('Running server checks...');
serverChecks
.check(environment.couchUrl)
.then(waitForApi)
.then(() => {
// Even requiring this boots translations, so has to be required after
// api has booted

(async () => {
try {
await serverChecks.check(environment.couchUrl);
await waitForApi();

const config = require('./src/config');
return config.init().then(() => {
require('./src/schedule').init();
logger.info('startup complete.');
});
})
.catch(err => {
logger.error('Fatal error intialising sentinel');
await config.init();

const schedule = require('./src/schedule');
schedule.init();

logger.info('startup complete.');

const processHooks = require('./src/process-hooks');
processHooks.init();

} catch (err) {
logger.error('Fatal error initialising sentinel');
logger.error('%o', err);
process.exit(1);
});
}
})();
42 changes: 21 additions & 21 deletions sentinel/src/lib/feed.js
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ const MAX_QUEUE_SIZE = 100;

let request;
let processed = 0;
let processing;

const enqueue = change => changeQueue.push(change);

Expand Down Expand Up @@ -100,33 +101,32 @@ changeQueue.drain(() => {
resumeProcessing();
});

/**
* Start listening from the last processed seq. Will restart
* automatically on error.
*/
const listen = () => {
processing = true;
changeQueue.resume();
return resumeProcessing();
};

module.exports = {
/**
* Stops listening for changes. Must be restarted manually
* by calling listen.
*/
const cancel = () => {
processing = false;
changeQueue.pause();
if (request) {
request.cancel && request.cancel();
request = null;
}
};

/**
* Start listening from the last processed seq. Will restart
* automatically on error.
*/
module.exports = {
listen,
cancel,

/**
* Stops listening for changes. Must be restarted manually
* by calling listen.
*/
cancel: () => {
changeQueue.pause();
if (request) {
request.cancel && request.cancel();
request = null;
}
},

// exposed for testing
_changeQueue: changeQueue,
_transitionsLib: transitionsLib,
_enqueue: enqueue,
toggle: () => processing ? cancel() : listen()
};
13 changes: 13 additions & 0 deletions sentinel/src/process-hooks.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
const feed = require('./lib/feed');
const schedule = require('./schedule');

module.exports = {
init: () => {
process.on('SIGUSR1', () => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I struggled to find any documentation on this but I think we can call the signals whatever we want, right? If so let's use clearer names. What about SIGLISTEN, SIGSTOPLISTENING, and SIGRUNTASKS? Or even SIGSENTINELLISTEN etc? It's getting long but at least it'll be clear that it's a custom signal and not a Node thing?

Also I think SIGUSR1 is reserved and according to this documentation will trigger Node into debugging mode: https://nodejs.org/en/learn/getting-started/debugging - so if we can't rename as above, let's use a different number at least.

Copy link
Member Author

@dianabarsan dianabarsan Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately, you can't send custom signals:

➜  ~ kill -s SIGRUNTASKS 26360
kill: unknown signal: SIGRUNTASKS
kill: type kill -l for a list of signals

So you need to select one of the signals in the list. There are some random ones that might not kill the process, but most do, and it felt weird to send a SIGURG to stop feed listening.
This is why there's a toggle to stop/start processing, because we can't send custom signals and there are only two which are "user customizable" - SIGUSR1 and SIGUSR2, which I am using.

Copy link
Contributor

@garethbowen garethbowen Nov 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ugh that's disgusting. Ok. The only other way I can see to do it would be to have a single signal which then looks somewhere else (file or db) to reload config, but that almost defeats the purpose!

Let's go with what you have.

feed.toggle();
});
process.on('SIGCONT', () => {
schedule.runTasks();
});
}
};
1 change: 1 addition & 0 deletions sentinel/src/schedule/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,4 @@ exports.init = () => {
runTasks();
interval = setInterval(runTasks, RUN_EVERY_MS);
};
exports.runTasks = runTasks;
5 changes: 1 addition & 4 deletions sentinel/src/transitions.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,5 @@ module.exports = {
/**
* Loads the transitions and starts watching for db changes.
*/
loadTransitions: loadTransitions,

// exposed for testing
_transitionsLib: transitionsLib,
loadTransitions,
};
Loading
Loading