[16.0][ADD] queue_job_chunk: add module

[clementmbr]

Adds the notion of queue job chunks, essentially a queue job with some metadata.
Useful to store the JSON received by an API request before their treatment in Odoo.

cc @florian-dacosta @kevinkhao @bealdav @sebastienbeau @rvalyi

[clementmbr]

@bealdav following your suggestion I moved the PR OCA/sale-channel#20 to this repo.
@rvalyi I squashed the commit and cleaned history

[clementmbr]

@kevinkhao coming back to your remark on the last PR, what do you mean by :
if "pdb" in config.get("dev_mode"): ?

I've never seen something like this, but I'll be happy to learn with an example!
Thanks!

[kevinkhao]

@clementmbr
odoo-bin --dev=pdb will evaluate to True

[rvalyi]

Adds the notion of queue job chunks, essentially a queue job with some metadata. Useful to store the JSON received by an API request before their treatment in Odoo.

cc @florian-dacosta @kevinkhao @bealdav @sebastienbeau @rvalyi

Hello, please pardon me if I'm wrong, but isn't it a bit the same idea of the split method that was merged recently in the queue_job module #658 ? If not, how does it differ?
In fact @paradoxxxzero even did a comment suggesting it was a bit similar in #658 (comment)

cc @sebastienbeau @simahawk @clementmbr

[clementmbr]

@rvalyi

Hello, please pardon me if I'm wrong, but isn't it a bit the same idea of the split method that was merged recently in the queue_job module #658 ? If not, how does it differ?

From what I understand the main difference with @paradoxxxzero 's split() method is that the chunk allows a more in depth control of the incoming data, with the fields like data_str, state_info and the stack_trace in case of errors, which is vital when importing Sale Orders like in sale_import_base.

So it looks like different use cases.

[florian-dacosta]

Yes, I think the goal here is not to use queue job directly because it is a very technical object.
I guess it is a bit in the same spirit than https://github.com/OCA/server-tools/tree/16.0/attachment_queue

We want to use queue job and all the machinery to run stuff automatically and all, but we want to have a separate object which allow to :

• Have a dedicated menu, allowing to filter stuff and to make it more understandable to user than a queue.job
• Avoid the deletion of the data after one month
• Do not overcomplicate queue job object with new fields specific to one use case (as the payload received here between other stuff
• Possibility to edit the payload in case of mistake
• ...

Now, @rvalyi has a point... if it is not used to split jobs...why is it named queue_job_chunk? It does not really make sense.
Also, do we really want something so generic without other use cases presently ? Probably not.

I think this should be fully refactored doing the following :

• Rename it sale_import_payload or something similar/more specific to its real current use case.
• Rename the model accordingly
• Move the menu to the "Sale App", probably with a new sale group
• Change reference field by many2one toward sale.channel and remove obsolete field (like model_name, record_id, processor...
• Make it more userfriendly (Show create_date, change order to display more recent first, add default filters (on create date, on payload, on error, maybe more ?),
• Log changes on payload, maybe error also to have some history if the payload is retried...
• Log a message on sale order when created from sale_import_payload so we can easily find which was the payload (and easily be sure that the sale order was created from a payload)
• Move it back to sale_channel
• Make a migraton script. Since the module will change name, I guess we can put a pre_init_hook, which will check if the old module was installed and do the migration if it is the case.
• ... Maybe I forgot some stuff, but it can be improved more later

Nice to have but can wait I guess

• Implement a configurable system to purge it, maybe with a default retention time of 1 year for instance