-
-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
introduce data aggregation (#194) #234
base: develop
Are you sure you want to change the base?
Conversation
575425b
to
ac6cb56
Compare
This column will be used for aggregation and is filled with 1 initially. The number of hits for a specific date/target/referrer is thus no longer the number of entries, but the sum of all hits.
We now aggregate statistics data in the cleanup routine preserving only distinct entries for each date/referrer/target combination with the sum of all hits.
If Statify is extended by custom columns the aggregation routine will fail. To support such scenarios, we introduce a boolean hook, so the previous behavior can be preserved without breaking compatibility.
ac6cb56
to
fd58722
Compare
fd58722
to
fe5e993
Compare
fe5e993
to
cdf4f5a
Compare
cdf4f5a
to
18b8a63
Compare
18b8a63
to
f8e356a
Compare
f8e356a
to
fe3dd71
Compare
These changes would require changes to keep the functionality of Statify - Extended Evaluation, but hopefully helps to improve the performance with large Statify tables (I've installs with 1,000,000+ entries): patrickrobrecht/extended-evaluation-for-statify#29. |
48543bb
to
e789310
Compare
Kudos, SonarCloud Quality Gate passed! 0 Bugs No Coverage information |
As previously discussed in various places this feature might need some additional planning.
|
We add a new column named
hits
to the database table with default value of 1.The tracking routine is preserved as is, i.e. it will insert a single row per hit.
The cleanup routine (cron job) is extended with an aggregation function that walks through the database and groups distinct date/referrer/target combinations with the sum of all hits.
before:
after:
The output is generated from
SUM(hits)
instead ofCOUNT(*)
then.We also introduce a boolean book
statify__skip_aggregation
(defaults tofalse
) to keep the previous behavior and disable the new aggregation.This might be necessary if the inserted entry is extended with custom columns, e.g. in a
statify__visit_saved
hook. If this is the case, aggregation will fail, because we don't know about additional columns. We could of course add another hook for custom aggregation columns and dynamically build up the statements, but for most users this won't be necessary and it can be extended in future releases as well.