Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Synced kserve upstream master branch into odh kserve master branch. #476

Closed
wants to merge 37 commits into from
+23,090 −5,997
Closed

Synced kserve upstream master branch into odh kserve master branch. #476

Changes from 1 commit
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
b836fc8
Remove redundant namespace yaml (#4148)
greenmoon55 Dec 23, 2024
cb38637
Fix Localmodel agent build (#4150)
greenmoon55 Dec 23, 2024
41a1fd7
Fix model server fails to gracefully shutdown (#4116)
sivanantha321 Dec 24, 2024
07edb56
Ensure root model directory exists and add protection for jobs create…
yuzisun Dec 24, 2024
61b0746
Enable transformer deeper readiness check tests (#4121)
sivanantha321 Dec 26, 2024
5e5bff8
Update HuggingFace server dependencies versions (#4147)
AyushSawant18588 Dec 26, 2024
874a4a4
Add workflow for verifying go mod (#4137)
sivanantha321 Dec 26, 2024
ff13ef8
Fix for CVE-2024-52304 - aiohttp upgrade (#4113)
andyi2it Dec 27, 2024
7bec6fb
Allow other engine builders other than docker (#3906)
spolti Dec 30, 2024
8f79c44
Add localmodelnode crd to helm chart (#4161)
greenmoon55 Jan 2, 2025
5b718bc
Fixes Non-linear parsing of case-insensitive content (#4158)
spolti Jan 3, 2025
3e842b8
Helm chart - option to run daemonset as root (#4164)
greenmoon55 Jan 3, 2025
7b2681f
Replace nodeGroup with nodeGroups in charts/kserve-crd (#4166)
ritzdevp Jan 5, 2025
3597fa7
Add affinity and tolerations to localmodel daemonset (#4173)
greenmoon55 Jan 7, 2025
cb8a657
Fix s3 download PermanentRedirectError for legacy s3 endpoint (#4157)
bentohset Jan 8, 2025
654d314
Make label and annotation propagation configurable (#4030)
spolti Jan 8, 2025
b7155b3
Add ModelCache e2e test (#4136)
sivanantha321 Jan 11, 2025
717c6ae
Update vllm to 0.6.6 (#4176)
rajatvig Jan 12, 2025
1b64da9
[bugfix] fix s3 storage download filename bug (#4162)
anencore94 Jan 13, 2025
8a32b6f
Add hf to storageuri prefix list (#4184)
tjandy98 Jan 16, 2025
990e412
Add Support for OpenAI-compatible Embeddings API (#4129)
FabianScheidt Jan 16, 2025
d377350
fix: typo in _construct_http_status_error method (#4190)
Mgla96 Jan 16, 2025
a1ad62b
Fix raw logger e2e test (#4185)
sivanantha321 Jan 19, 2025
b164ea0
Feat: Support configuring isvc resource defaults (#4032)
andyi2it Jan 21, 2025
6653b89
keep replicas when autoscaler set external (#4196)
Jooho Jan 22, 2025
6f842d6
Increase kserve controller readiness probe time period (#4200)
sivanantha321 Jan 22, 2025
1d0afe1
Fix golangci-lint binary path selection based on GOBIN (#4198)
Jooho Jan 22, 2025
96f43de
Add option to disable volume management in localModel config (#4186)
ritzdevp Jan 22, 2025
ea1f055
set MaxUnavailable(0%)/MaxSurge(100%) for rollingUpdate in multinode …
Jooho Jan 24, 2025
549ab62
Gracefully shutdown the router server (#3367)
sivanantha321 Jan 25, 2025
5aedc95
Add workflow for manual huggingface vLLM image publish (#4092)
sivanantha321 Jan 25, 2025
7f4cad9
Gateway API Migration - Raw Deployment (#3952)
sivanantha321 Jan 25, 2025
b80c906
add make goal to build huggingface cpu image (#4202)
spolti Jan 26, 2025
c7b6ad0
Cleanup the filepath in createNewFile to avoid path traversal issue (…
hdefazio Jan 26, 2025
4c940e1
Enhance multinode health_check python and manifests (#4197)
Jooho Jan 26, 2025
41a0904
Publish 0.15-rc0 release (#4213)
yuzisun Jan 27, 2025
bb9c9d8
Merge kserve upstream master branch into odh kserve master branch.
mholder6 Jan 29, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Loading
Loading status checks…
Allow other engine builders other than docker (kserve#3906) 
* Allow other engine builds other than docker

chore:	This change will allow user to use other engine builds.
	In addition, for Podman/Buildah engine builder, users can
	set the target architecture, useful when using ARM and
	there is a need to build x86 Container images.
	Can be done with `ENGINE=podman ARCH="--arch x86_64" make docker...`

Signed-off-by: Spolti <[email protected]>

* review changes

Signed-off-by: Spolti <[email protected]>

---------

Signed-off-by: Spolti <[email protected]>
@spolti
spolti authored Dec 30, 2024

Verified

This commit was signed with the committer’s verified signature.
Verified
Learn about vigilant mode
commit 7bec6fb5865f445714e06c3b230da530693a3b5b
84 changes: 45 additions & 39 deletions Makefile
View file
Original file line number Diff line number Diff line change
@@ -28,6 +28,12 @@ ENVTEST_K8S_VERSION = 1.29
SUCCESS_200_ISVC_IMG ?= success-200-isvc
ERROR_404_ISVC_IMG ?= error-404-isvc

ENGINE ?= docker
# Empty string for local build when using podman, it allows to build different architectures
# to use do: ENGINE=podman ARCH="--arch x86_64" make docker-build-something
ARCH ?=


## Location to install dependencies to
LOCALBIN ?= $(shell pwd)/bin
$(LOCALBIN):
@@ -207,8 +213,8 @@ bump-version:
@hack/prepare-for-release.sh $(PRIOR_VERSION) $(NEW_VERSION)

# Build the docker image
docker-build: test
docker buildx build . -t ${IMG}
docker-build:
${ENGINE} buildx build ${ARCH} . -t ${IMG}
@echo "updating kustomize image patch file for manager resource"

# Use perl instead of sed to avoid OSX/Linux compatibility issue:
@@ -220,115 +226,115 @@ docker-push:
docker push ${IMG}

docker-build-agent:
docker buildx build -f agent.Dockerfile . -t ${KO_DOCKER_REPO}/${AGENT_IMG}
${ENGINE} buildx build ${ARCH} -f agent.Dockerfile . -t ${KO_DOCKER_REPO}/${AGENT_IMG}

docker-build-router:
docker buildx build -f router.Dockerfile . -t ${KO_DOCKER_REPO}/${ROUTER_IMG}
${ENGINE} buildx build ${ARCH} -f router.Dockerfile . -t ${KO_DOCKER_REPO}/${ROUTER_IMG}

docker-push-agent:
docker push ${KO_DOCKER_REPO}/${AGENT_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${AGENT_IMG}

docker-push-router:
docker push ${KO_DOCKER_REPO}/${ROUTER_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${ROUTER_IMG}

docker-build-sklearn:
cd python && docker buildx build --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${SKLEARN_IMG} -f sklearn.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${SKLEARN_IMG} -f sklearn.Dockerfile .

docker-push-sklearn: docker-build-sklearn
docker push ${KO_DOCKER_REPO}/${SKLEARN_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${SKLEARN_IMG}

docker-build-xgb:
cd python && docker buildx build --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${XGB_IMG} -f xgb.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${XGB_IMG} -f xgb.Dockerfile .

docker-push-xgb: docker-build-xgb
docker push ${KO_DOCKER_REPO}/${XGB_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${XGB_IMG}

docker-build-lgb:
cd python && docker buildx build --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${LGB_IMG} -f lgb.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${LGB_IMG} -f lgb.Dockerfile .

docker-push-lgb: docker-build-lgb
docker push ${KO_DOCKER_REPO}/${LGB_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${LGB_IMG}

docker-build-pmml:
cd python && docker buildx build --build-arg BASE_IMAGE=${PMML_BASE_IMG} -t ${KO_DOCKER_REPO}/${PMML_IMG} -f pmml.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} --build-arg BASE_IMAGE=${PMML_BASE_IMG} -t ${KO_DOCKER_REPO}/${PMML_IMG} -f pmml.Dockerfile .

docker-push-pmml: docker-build-pmml
docker push ${KO_DOCKER_REPO}/${PMML_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${PMML_IMG}

docker-build-paddle:
cd python && docker buildx build --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${PADDLE_IMG} -f paddle.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${PADDLE_IMG} -f paddle.Dockerfile .

docker-push-paddle: docker-build-paddle
docker push ${KO_DOCKER_REPO}/${PADDLE_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${PADDLE_IMG}

docker-build-custom-model:
cd python && docker buildx build -t ${KO_DOCKER_REPO}/${CUSTOM_MODEL_IMG} -f custom_model.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${CUSTOM_MODEL_IMG} -f custom_model.Dockerfile .

docker-push-custom-model: docker-build-custom-model
docker push ${KO_DOCKER_REPO}/${CUSTOM_MODEL_IMG}

docker-build-custom-model-grpc:
cd python && docker buildx build -t ${KO_DOCKER_REPO}/${CUSTOM_MODEL_GRPC_IMG} -f custom_model_grpc.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${CUSTOM_MODEL_GRPC_IMG} -f custom_model_grpc.Dockerfile .

docker-push-custom-model-grpc: docker-build-custom-model-grpc
docker push ${KO_DOCKER_REPO}/${CUSTOM_MODEL_GRPC_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${CUSTOM_MODEL_GRPC_IMG}

docker-build-custom-transformer:
cd python && docker buildx build -t ${KO_DOCKER_REPO}/${CUSTOM_TRANSFORMER_IMG} -f custom_transformer.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${CUSTOM_TRANSFORMER_IMG} -f custom_transformer.Dockerfile .

docker-push-custom-transformer: docker-build-custom-transformer
docker push ${KO_DOCKER_REPO}/${CUSTOM_TRANSFORMER_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${CUSTOM_TRANSFORMER_IMG}

docker-build-custom-transformer-grpc:
cd python && docker buildx build -t ${KO_DOCKER_REPO}/${CUSTOM_TRANSFORMER_GRPC_IMG} -f custom_transformer_grpc.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${CUSTOM_TRANSFORMER_GRPC_IMG} -f custom_transformer_grpc.Dockerfile .

docker-push-custom-transformer-grpc: docker-build-custom-transformer-grpc
docker push ${KO_DOCKER_REPO}/${CUSTOM_TRANSFORMER_GRPC_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${CUSTOM_TRANSFORMER_GRPC_IMG}

docker-build-aif:
cd python && docker buildx build -t ${KO_DOCKER_REPO}/${AIF_IMG} -f aiffairness.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${AIF_IMG} -f aiffairness.Dockerfile .

docker-push-aif: docker-build-aif
docker push ${KO_DOCKER_REPO}/${AIF_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${AIF_IMG}

docker-build-art:
cd python && docker buildx build -t ${KO_DOCKER_REPO}/${ART_IMG} -f artexplainer.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${ART_IMG} -f artexplainer.Dockerfile .

docker-push-art: docker-build-art
docker push ${KO_DOCKER_REPO}/${ART_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${ART_IMG}

docker-build-storageInitializer:
cd python && docker buildx build --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${STORAGE_INIT_IMG} -f storage-initializer.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} --build-arg BASE_IMAGE=${BASE_IMG} -t ${KO_DOCKER_REPO}/${STORAGE_INIT_IMG} -f storage-initializer.Dockerfile .

docker-push-storageInitializer: docker-build-storageInitializer
docker push ${KO_DOCKER_REPO}/${STORAGE_INIT_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${STORAGE_INIT_IMG}

docker-build-qpext:
docker buildx build -t ${KO_DOCKER_REPO}/${QPEXT_IMG} -f qpext/qpext.Dockerfile .
${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${QPEXT_IMG} -f qpext/qpext.Dockerfile .

docker-build-push-qpext: docker-build-qpext
docker push ${KO_DOCKER_REPO}/${QPEXT_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${QPEXT_IMG}

deploy-dev-qpext: docker-build-push-qpext
kubectl patch cm config-deployment -n knative-serving --type merge --patch '{"data": {"queue-sidecar-image": "${KO_DOCKER_REPO}/${QPEXT_IMG}"}}'

docker-build-success-200-isvc:
cd python && docker buildx build -t ${KO_DOCKER_REPO}/${SUCCESS_200_ISVC_IMG} -f success_200_isvc.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${SUCCESS_200_ISVC_IMG} -f success_200_isvc.Dockerfile .

docker-push-success-200-isvc: docker-build-success-200-isvc
docker push ${KO_DOCKER_REPO}/${SUCCESS_200_ISVC_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${SUCCESS_200_ISVC_IMG}

docker-build-error-node-404:
cd python && docker buildx build -t ${KO_DOCKER_REPO}/${ERROR_404_ISVC_IMG} -f error_404_isvc.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${ERROR_404_ISVC_IMG} -f error_404_isvc.Dockerfile .

docker-push-error-node-404: docker-build-error-node-404
docker push ${KO_DOCKER_REPO}/${ERROR_404_ISVC_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${ERROR_404_ISVC_IMG}

docker-build-huggingface:
cd python && docker buildx build -t ${KO_DOCKER_REPO}/${HUGGINGFACE_SERVER_IMG} -f huggingface_server.Dockerfile .
cd python && ${ENGINE} buildx build ${ARCH} -t ${KO_DOCKER_REPO}/${HUGGINGFACE_SERVER_IMG} -f huggingface_server.Dockerfile .

docker-push-huggingface: docker-build-huggingface
docker push ${KO_DOCKER_REPO}/${HUGGINGFACE_SERVER_IMG}
${ENGINE} push ${KO_DOCKER_REPO}/${HUGGINGFACE_SERVER_IMG}

test-qpext:
cd qpext && go test -v ./... -cover
@@ -342,8 +348,8 @@ $(ENVTEST): $(LOCALBIN)
test -s $(LOCALBIN)/setup-envtest || GOBIN=$(LOCALBIN) go install sigs.k8s.io/controller-runtime/tools/setup-envtest@latest

apidocs:
docker buildx build -f docs/apis/Dockerfile --rm -t apidocs-gen . && \
docker run -it --rm -v $(CURDIR)/pkg/apis:/go/src/github.com/kserve/kserve/pkg/apis -v ${PWD}/docs/apis:/go/gen-crd-api-reference-docs/apidocs apidocs-gen
${ENGINE} buildx build ${ARCH} -f docs/apis/Dockerfile --rm -t apidocs-gen . && \
${ENGINE} run -it --rm -v $(CURDIR)/pkg/apis:/go/src/github.com/kserve/kserve/pkg/apis -v ${PWD}/docs/apis:/go/gen-crd-api-reference-docs/apidocs apidocs-gen

.PHONY: check-doc-links
check-doc-links: