Why the emphasis on Incremental materialization?

[Mac-lp3]

Hi,

I notice in the docs that the hub/sat/link macro descriptions tend to include the line "should be created using incremental materialization."

I just want to know the reason behind that. Is it just to conserve resources/compute power? Or do we loose some data if we use a "kill & fill" style materialization?

Thanks,
Mike

[tkirschke]

Hi @Mac-lp3 and thanks for reaching out!

In Data Vault 2.0 one of the aims of the raw vault is to completely historize the data. If you have disappearances in your source data, you would loose this data, if your Raw Vault would be completely repopulated every day.

On the other hand, incremental loading can indeed improve the loading performance of larger tables.

Best regards
Tim