Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add CRD for Transparent Memory Offloading #62

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: add CRD for Transparent Memory Offloading #62

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
294 changes: 294 additions & 0 deletions config/crd/bases/config.katalyst.kubewharf.io_transparentmemoryoffloadingconfigurations.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,294 @@
---
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.9.0
creationTimestamp: null
name: transparentmemoryoffloadingconfigurations.config.katalyst.kubewharf.io
spec:
group: config.katalyst.kubewharf.io
names:
kind: TransparentMemoryOffloadingConfiguration
listKind: TransparentMemoryOffloadingConfigurationList
plural: transparentmemoryoffloadingconfigurations
shortNames:
- tmo
singular: transparentmemoryoffloadingconfiguration
scope: Namespaced
versions:
- additionalPrinterColumns:
- jsonPath: .metadata.creationTimestamp
name: AGE
type: date
- jsonPath: .spec.nodeLabelSelector
name: SELECTOR
type: string
- jsonPath: .spec.priority
name: PRIORITY
type: string
- jsonPath: .spec.ephemeralSelector.nodeNames
name: NODES
type: string
- jsonPath: .spec.ephemeralSelector.lastDuration
name: DURATION
type: string
- jsonPath: .status.conditions[?(@.type=="Valid")].status
name: VALID
type: string
- jsonPath: .status.conditions[?(@.type=="Valid")].reason
name: REASON
type: string
- jsonPath: .status.conditions[?(@.type=="Valid")].message
name: MESSAGE
type: string
name: v1alpha1
schema:
openAPIV3Schema:
description: TransparentMemoryOffloadingConfiguration is the Schema for the
configuration API used by Transparent Memory Offloading
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
type: string
metadata:
type: object
spec:
description: TransparentMemoryOffloadingConfigurationSpec defines the
desired state of TransparentMemoryOffloadingConfiguration
properties:
config:
description: Config is custom field for TMO configuration
properties:
CgroupConfig:
description: CgroupConfig is a configuration for manipulating
TMO on specified cgroups
items:
properties:
cgroupPath:
description: CgroupPath is an cgroupV2 absolute path, e.g.
/sys/fs/cgroup/hdfs
type: string
configDetail:
description: ConfigDetail is configuration details of TMO
properties:
enableSwap:
description: EnableSwap is whether to enable swap to
offloading anon pages
type: boolean
enableTMO:
description: EnableTMO is whether to enable TMO on target
objective
type: boolean
interval:
description: Interval is the minimum duration the objectives
got memory reclaimed by TMO
type: string
policyName:
description: PolicyName is used to specify the policy
for calculating memory offloading size
type: string
psiPolicy:
description: PSIPolicyConf is configurations of a TMO
policy which reclaim memory by PSI
properties:
maxProbe:
description: MaxProbe limits the memory offloading
size in one cycle, it's a ratio of memory usage.
type: number
psiAvg60Threshold:
description: PSIAvg60Threshold indicates the threshold
of memory pressure. If observed pressure exceeds
this threshold, memory offloading will be paused.
type: number
type: object
refaultPolicy:
description: RefaultPolicy is configurations of a TMO
policy which reclaim memory by refault
properties:
maxProbe:
description: MaxProbe limits the memory offloading
size in one cycle, it's a ratio of memory usage.
type: number
reclaimAccuracyTarget:
description: ReclaimAccuracyTarget indicates the
desired level of precision or accuracy in offloaded
pages.
type: number
reclaimScanEfficiencyTarget:
description: ReclaimScanEfficiencyTarget indicates
the desired level of efficiency in scanning and
identifying memory pages that can be offloaded.
type: number
type: object
type: object
required:
- cgroupPath
- configDetail
type: object
type: array
x-kubernetes-list-map-keys:
- cgroupPath
x-kubernetes-list-type: map
qosLevelConfig:
description: QoSLevelConfig is a configuration for manipulating
TMO on Different QoS Level
items:
properties:
configDetail:
description: ConfigDetail is configuration details of TMO
properties:
enableSwap:
description: EnableSwap is whether to enable swap to
offloading anon pages
type: boolean
enableTMO:
description: EnableTMO is whether to enable TMO on target
objective
type: boolean
interval:
description: Interval is the minimum duration the objectives
got memory reclaimed by TMO
type: string
policyName:
description: PolicyName is used to specify the policy
for calculating memory offloading size
type: string
psiPolicy:
description: PSIPolicyConf is configurations of a TMO
policy which reclaim memory by PSI
properties:
maxProbe:
description: MaxProbe limits the memory offloading
size in one cycle, it's a ratio of memory usage.
type: number
psiAvg60Threshold:
description: PSIAvg60Threshold indicates the threshold
of memory pressure. If observed pressure exceeds
this threshold, memory offloading will be paused.
type: number
type: object
refaultPolicy:
description: RefaultPolicy is configurations of a TMO
policy which reclaim memory by refault
properties:
maxProbe:
description: MaxProbe limits the memory offloading
size in one cycle, it's a ratio of memory usage.
type: number
reclaimAccuracyTarget:
description: ReclaimAccuracyTarget indicates the
desired level of precision or accuracy in offloaded
pages.
type: number
reclaimScanEfficiencyTarget:
description: ReclaimScanEfficiencyTarget indicates
the desired level of efficiency in scanning and
identifying memory pages that can be offloaded.
type: number
type: object
type: object
qosLevel:
description: QoSLevel is either of reclaimed_cores, shared_cores,
dedicated_cores, system_cores
type: string
required:
- configDetail
- qosLevel
type: object
type: array
x-kubernetes-list-map-keys:
- qosLevel
x-kubernetes-list-type: map
type: object
ephemeralSelector:
description: EphemeralSelector is a selector for temporary use only
properties:
lastDuration:
description: define the duration this configuration will last
from creationTimestamp. must and only set when NodeNames already
set
type: string
nodeNames:
description: Specific nodes' name the configurations will be effected.
items:
type: string
type: array
type: object
nodeLabelSelector:
description: NodeLabelSelector select nodes to apply these configurations,
the priority and node label selector must be matched according to
KatalystCustomConfig.spec.nodeLabelSelectorAllowedKeyList, otherwise
it will not be synced.
type: string
priority:
description: Priority is used by one node matched by NodeLabelSelector
of more than one configuration, and the higher priority will be
considered. The priority only be supported when NodeLabelSelector
set
format: int32
type: integer
revisionHistoryLimit:
default: 3
description: RevisionHistoryLimit is the maximum number of revisions
that will be maintained in the resource's revision history. The
revision history consists of all revisions not represented by a
currently applied Spec version. The default value is 3.
format: int64
type: integer
required:
- config
type: object
status:
properties:
collisionCount:
description: Count of hash collisions for this cr. The kcc controller
uses this field as a collision avoidance mechanism when it needs
to create the name for the newest ControllerRevision.
format: int32
type: integer
conditions:
description: Represents the latest available observations of a config's
current state.
items:
properties:
lastTransitionTime:
description: Last time the condition transit from one status
to another.
format: date-time
type: string
message:
description: message is a human-readable explanation containing
details about the transition
type: string
reason:
description: reason is the reason for the condition's last transition.
type: string
status:
description: Status of the condition, one of True, False, Unknown.
type: string
type:
description: Type of config condition
type: string
required:
- status
- type
type: object
type: array
observedGeneration:
description: The most recent generation observed by the kcc controller.
format: int64
type: integer
type: object
type: object
served: true
storage: true
subresources:
status: {}
18 changes: 18 additions & 0 deletions pkg/apis/config/v1alpha1/register.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,8 +20,17 @@ import (
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
"k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"

workloadapi "github.com/kubewharf/katalyst-api/pkg/apis/workload/v1alpha1"
)

func init() {
// We only register manually written functions here. The registration of the
// generated functions takes place in the generated files. The separation
// makes the code compile even when the generated files are missing.
workloadapi.SchemeBuilder.Register(addSPDKnownTypes)
}

const (
// GroupName is the group name used in this package
GroupName string = "config.katalyst.kubewharf.io"
Expand All @@ -36,6 +45,7 @@ const (
ResourceNameCustomNodeConfigs = "customnodeconfigs"
ResourceNameAdminQoSConfigurations = "adminqosconfigurations"
ResourceNameAuthConfigurations = "authconfigurations"
ResourceNameTMOConfigurations = "transparentmemoryoffloadingconfigurations"
)

// Resource takes an unqualified resource and returns a Group qualified GroupResource
Expand All @@ -50,6 +60,12 @@ var (
AddToScheme = SchemeBuilder.AddToScheme
)

func addSPDKnownTypes(scheme *runtime.Scheme) error {
scheme.AddKnownTypes(workloadapi.SchemeGroupVersion,
&TransparentMemoryOffloadingIndicators{})
return nil
}

// Adds the list of known types to the given scheme.
func addKnownTypes(scheme *runtime.Scheme) error {
scheme.AddKnownTypes(SchemeGroupVersion,
Expand All @@ -63,6 +79,8 @@ func addKnownTypes(scheme *runtime.Scheme) error {
&AdminQoSConfigurationList{},
&AuthConfiguration{},
&AuthConfigurationList{},
&TransparentMemoryOffloadingConfiguration{},
&TransparentMemoryOffloadingConfigurationList{},
)

metav1.AddToGroupVersion(scheme, SchemeGroupVersion)
Expand Down
Loading
Loading