Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

br: make schdule support compact #6033

Merged
merged 46 commits into from
Jan 23, 2025
Merged

br: make schdule support compact #6033

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
255e208
mod types
RidRisR Dec 16, 2024
9321854
don't need optional
RidRisR Dec 16, 2024
5996cef
rule of compact interval
RidRisR Dec 16, 2024
3fd2660
update ts
RidRisR Dec 17, 2024
9f827a8
build compact cr
RidRisR Dec 18, 2024
88df77d
ready for compact
RidRisR Dec 18, 2024
0dd2a35
add compact control
RidRisR Dec 18, 2024
7cbfc41
make generate
RidRisR Dec 18, 2024
9a4e2c1
get startTs and endTs
RidRisR Jan 7, 2025
7b51fd3
add test
RidRisR Jan 7, 2025
77611ac
progress
RidRisR Jan 7, 2025
d67e2e6
handle error
RidRisR Jan 7, 2025
4e20876
delete last compact
RidRisR Jan 8, 2025
26ba08a
why?
RidRisR Jan 8, 2025
eecc891
fix more
RidRisR Jan 9, 2025
bb8eec3
make compact run
RidRisR Jan 9, 2025
1292fa4
add time convert
RidRisR Jan 9, 2025
6802b19
handle failed compact
RidRisR Jan 9, 2025
1903be3
handle last compact
RidRisR Jan 9, 2025
5784bdb
lnt
RidRisR Jan 14, 2025
fa3f39e
Update pkg/backup/backupschedule/backup_schedule_manager.go
RidRisR Jan 14, 2025
1f4cad2
fix comments
RidRisR Jan 15, 2025
4801f23
add GC for compact CR
RidRisR Jan 15, 2025
f7361c0
add comment for calEndTs
RidRisR Jan 15, 2025
3a12df0
remove deleteLastcompactJob （unused）
RidRisR Jan 15, 2025
c10b4e5
lint
RidRisR Jan 15, 2025
8b69ce0
truncate must less than compact
RidRisR Jan 15, 2025
094829c
remove unused func
RidRisR Jan 16, 2025
ca232d4
return err if log already exists
RidRisR Jan 20, 2025
50b7845
Merge branch 'master' into schdule
RidRisR Jan 21, 2025
662bec7
mod comment
RidRisR Jan 21, 2025
cdedfb9
lint
RidRisR Jan 21, 2025
4049ea1
use bm.now to replace schedule
RidRisR Jan 21, 2025
65f058a
lint
RidRisR Jan 21, 2025
1cbd73b
change logic of interval
RidRisR Jan 21, 2025
16a6627
rename some fields
RidRisR Jan 22, 2025
b64d59f
refactor calEndTs
RidRisR Jan 22, 2025
4ec0b3f
add dup log backup check
RidRisR Jan 22, 2025
0f177e3
add log backup existance check
RidRisR Jan 22, 2025
5201e03
fix checkpoint is nil
RidRisR Jan 22, 2025
17b356b
return if no compact
RidRisR Jan 22, 2025
2992a15
remove calLastCompactTs in GC
RidRisR Jan 22, 2025
7b0dcc6
saparate checkLogBackupStatus()
RidRisR Jan 23, 2025
f708689
refactor log backup logic
RidRisR Jan 23, 2025
7a0141d
move compact part
RidRisR Jan 23, 2025
a5705ef
add comments
RidRisR Jan 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
197 changes: 189 additions & 8 deletions docs/api-references/docs.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,9 @@ StorageProvider
<p>
(Members of <code>StorageProvider</code> are embedded into this type.)
</p>
<p>StorageProvider configures where and how backups should be stored.</p>
<p>StorageProvider configures where and how backups should be stored.
*** Note: This field should generally not be left empty, unless you are certain the storage provider
*** can be obtained from another source, such as a schedule CR.</p>
</td>
</tr>
<tr>
Expand Down Expand Up @@ -226,7 +228,9 @@ BRConfig
</em>
</td>
<td>
<p>BRConfig is the configs for BR</p>
<p>BRConfig is the configs for BR
*** Note: This field should generally not be left empty, unless you are certain the BR config
*** can be obtained from another source, such as a schedule CR.</p>
</td>
</tr>
<tr>
Expand Down Expand Up @@ -647,6 +651,17 @@ string
</tr>
<tr>
<td>
<code>compactSpan</code></br>
<em>
string
</em>
</td>
<td>
<p>CompactSpan is to specify how long backups we want to compact.</p>
</td>
</tr>
<tr>
<td>
<code>backupTemplate</code></br>
<em>
<a href="#backupspec">
Expand Down Expand Up @@ -674,6 +689,20 @@ BackupSpec
</tr>
<tr>
<td>
<code>compactBackupTemplate</code></br>
<em>
<a href="#compactspec">
CompactSpec
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>CompactBackupTemplate is the specification of the compact backup structure to get scheduled.</p>
</td>
</tr>
<tr>
<td>
<code>storageClassName</code></br>
<em>
string
Expand Down Expand Up @@ -710,6 +739,37 @@ string
<p>ImagePullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images.</p>
</td>
</tr>
<tr>
<td>
<code>br</code></br>
<em>
<a href="#brconfig">
BRConfig
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>BRConfig is the configs for BR</p>
</td>
</tr>
<tr>
<td>
<code>StorageProvider</code></br>
<em>
<a href="#storageprovider">
StorageProvider
</a>
</em>
</td>
<td>
<p>
(Members of <code>StorageProvider</code> are embedded into this type.)
</p>
<em>(Optional)</em>
<p>StorageProvider configures where and how backups should be stored.</p>
</td>
</tr>
</table>
</td>
</tr>
Expand Down Expand Up @@ -3446,6 +3506,7 @@ string
<h3 id="brconfig">BRConfig</h3>
<p>
(<em>Appears on:</em>
<a href="#backupschedulespec">BackupScheduleSpec</a>,
<a href="#backupspec">BackupSpec</a>,
<a href="#compactspec">CompactSpec</a>,
<a href="#restorespec">RestoreSpec</a>)
Expand Down Expand Up @@ -3938,6 +3999,17 @@ string
</tr>
<tr>
<td>
<code>compactSpan</code></br>
<em>
string
</em>
</td>
<td>
<p>CompactSpan is to specify how long backups we want to compact.</p>
</td>
</tr>
<tr>
<td>
<code>backupTemplate</code></br>
<em>
<a href="#backupspec">
Expand Down Expand Up @@ -3965,6 +4037,20 @@ BackupSpec
</tr>
<tr>
<td>
<code>compactBackupTemplate</code></br>
<em>
<a href="#compactspec">
CompactSpec
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>CompactBackupTemplate is the specification of the compact backup structure to get scheduled.</p>
</td>
</tr>
<tr>
<td>
<code>storageClassName</code></br>
<em>
string
Expand Down Expand Up @@ -4001,6 +4087,37 @@ string
<p>ImagePullSecrets is an optional list of references to secrets in the same namespace to use for pulling any of the images.</p>
</td>
</tr>
<tr>
<td>
<code>br</code></br>
<em>
<a href="#brconfig">
BRConfig
</a>
</em>
</td>
<td>
<em>(Optional)</em>
<p>BRConfig is the configs for BR</p>
</td>
</tr>
<tr>
<td>
<code>StorageProvider</code></br>
<em>
<a href="#storageprovider">
StorageProvider
</a>
</em>
</td>
<td>
<p>
(Members of <code>StorageProvider</code> are embedded into this type.)
</p>
<em>(Optional)</em>
<p>StorageProvider configures where and how backups should be stored.</p>
</td>
</tr>
</tbody>
</table>
<h3 id="backupschedulestatus">BackupScheduleStatus</h3>
Expand Down Expand Up @@ -4032,6 +4149,17 @@ string
</tr>
<tr>
<td>
<code>lastCompact</code></br>
<em>
string
</em>
</td>
<td>
<p>LastCompact represents the last compact</p>
</td>
</tr>
<tr>
<td>
<code>logBackup</code></br>
<em>
string
Expand All @@ -4043,6 +4171,19 @@ string
</tr>
<tr>
<td>
<code>logBackupStartTs</code></br>
<em>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta">
Kubernetes meta/v1.Time
</a>
</em>
</td>
<td>
<p>LogBackupStartTs represents the start time of log backup</p>
</td>
</tr>
<tr>
<td>
<code>lastBackupTime</code></br>
<em>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta">
Expand All @@ -4056,6 +4197,32 @@ Kubernetes meta/v1.Time
</tr>
<tr>
<td>
<code>lastCompactTs</code></br>
<em>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta">
Kubernetes meta/v1.Time
</a>
</em>
</td>
<td>
<p>LastCompactTs represents the endTs of the last compact</p>
</td>
</tr>
<tr>
<td>
<code>nextCompactEndTs</code></br>
<em>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta">
Kubernetes meta/v1.Time
</a>
</em>
</td>
<td>
<p>NextCompactEndTs represents the scheduled endTs of next compact</p>
</td>
</tr>
<tr>
<td>
<code>allBackupCleanTime</code></br>
<em>
<a href="https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#time-v1-meta">
Expand Down Expand Up @@ -4193,7 +4360,9 @@ StorageProvider
<p>
(Members of <code>StorageProvider</code> are embedded into this type.)
</p>
<p>StorageProvider configures where and how backups should be stored.</p>
<p>StorageProvider configures where and how backups should be stored.
*** Note: This field should generally not be left empty, unless you are certain the storage provider
*** can be obtained from another source, such as a schedule CR.</p>
</td>
</tr>
<tr>
Expand Down Expand Up @@ -4230,7 +4399,9 @@ BRConfig
</em>
</td>
<td>
<p>BRConfig is the configs for BR</p>
<p>BRConfig is the configs for BR
*** Note: This field should generally not be left empty, unless you are certain the BR config
*** can be obtained from another source, such as a schedule CR.</p>
</td>
</tr>
<tr>
Expand Down Expand Up @@ -5429,7 +5600,9 @@ StorageProvider
<p>
(Members of <code>StorageProvider</code> are embedded into this type.)
</p>
<p>StorageProvider configures where and how backups should be stored.</p>
<p>StorageProvider configures where and how backups should be stored.
*** Note: This field should generally not be left empty, unless you are certain the storage provider
*** can be obtained from another source, such as a schedule CR.</p>
</td>
</tr>
<tr>
Expand Down Expand Up @@ -5507,7 +5680,9 @@ BRConfig
</em>
</td>
<td>
<p>BRConfig is the configs for BR</p>
<p>BRConfig is the configs for BR
*** Note: This field should generally not be left empty, unless you are certain the BR config
*** can be obtained from another source, such as a schedule CR.</p>
</td>
</tr>
<tr>
Expand Down Expand Up @@ -5697,6 +5872,7 @@ string
<h3 id="compactspec">CompactSpec</h3>
<p>
(<em>Appears on:</em>
<a href="#backupschedulespec">BackupScheduleSpec</a>,
<a href="#compactbackup">CompactBackup</a>)
</p>
<p>
Expand Down Expand Up @@ -5765,7 +5941,9 @@ StorageProvider
<p>
(Members of <code>StorageProvider</code> are embedded into this type.)
</p>
<p>StorageProvider configures where and how backups should be stored.</p>
<p>StorageProvider configures where and how backups should be stored.
*** Note: This field should generally not be left empty, unless you are certain the storage provider
*** can be obtained from another source, such as a schedule CR.</p>
</td>
</tr>
<tr>
Expand Down Expand Up @@ -5843,7 +6021,9 @@ BRConfig
</em>
</td>
<td>
<p>BRConfig is the configs for BR</p>
<p>BRConfig is the configs for BR
*** Note: This field should generally not be left empty, unless you are certain the BR config
*** can be obtained from another source, such as a schedule CR.</p>
</td>
</tr>
<tr>
Expand Down Expand Up @@ -16571,6 +16751,7 @@ More info: <a href="https://kubernetes.io/docs/concepts/storage/persistent-volum
<h3 id="storageprovider">StorageProvider</h3>
<p>
(<em>Appears on:</em>
<a href="#backupschedulespec">BackupScheduleSpec</a>,
<a href="#backupspec">BackupSpec</a>,
<a href="#compactspec">CompactSpec</a>,
<a href="#restorespec">RestoreSpec</a>)
Expand Down
Loading
Loading