Skip to content

Commit

Permalink
OCI Artifact changes.
Browse files Browse the repository at this point in the history 
Changes the specf to adhere to OCI Artifact guidance.
Adds a section about the tar layers. Some minor
editing changes.And reproducibility section
  • Loading branch information
@gorkem
gorkem committed Jan 13, 2025
1 parent fe73e2e commit ef8c140
Showing 1 changed file with 85 additions and 23 deletions.
108 changes: 85 additions & 23 deletions docs/spec.md
View file
Original file line number Diff line number Diff line change
@@ -1,57 +1,112 @@
# Model Format Specification

The specification defines an open standard for packaging and distribution Artificial Intelligence models as OCI artifacts, adhering to [the OCI image specification](https://github.com/opencontainers/image-spec/blob/main/spec.md#image-format-specification).
The specification defines an open standard for packaging and distribution of Artificial Intelligence models as OCI artifacts, adhering to [the OCI image specification][image-spec].

The goal of this specification is to outline a blueprint and enable the creation of interoperable solutions for packaging and retrieving AI/ML models by leveraging the existing OCI ecosystem, thereby facilitating efficient model management, deployment and serving in cloud-native environments.

## Use Cases

* An OCI Registry could storage and manage AI/ML model artifacts with model versions, metadata, and parameters retrievable and displayable.
* An OCI Registry can store and manage AI/ML model artifacts, making model versions, metadata, and parameters both retrievable and easily displayed.
* A Data Scientist can package models together with their metadata (e.g., format, precision) and upload them to a registry, facilitating collaboration with MLOps Engineers while streamlining the deployment process to efficiently deliver models into production.
* A model serving/deployment platform can read model metadata (e.g., format, precision) from a registry to understand the AI/ML model details, identify the required server runtime
(as well as startup parameters, necessary resources, etc.), and serve the model in Kubernetes by [mounting it directly as a volume source](https://kubernetes.io/blog/2024/08/16/kubernetes-1-31-image-volume-source/)
without needing to pre-download it in an init-container or bundle it within the server runtime container.

## Overview

At a high level, the Model Format Specification is based on the [OCI Image Format Specification](https://github.com/opencontainers/image-spec/blob/main/spec.md#image-format-specification) and incorporates [all its components](https://github.com/opencontainers/image-spec/blob/main/spec.md#understanding-the-specification). The key distinction lies in extending the [OCI Image Manifest Specification](https://github.com/opencontainers/image-spec/blob/main/manifest.md) to accommodate artifact usage specifically tailored for AI/ML models.
At a high level, the Model Format Specification is based on the [OCI Image Format Specification][image-spec] and incorporates [all its components](https://github.com/opencontainers/image-spec/blob/main/spec.md#understanding-the-specification).

### OCI Image Manifest Specification For Model Artifacts

The image manifest of model artifacts follows the [OCI Image Manifest Specification][image-manifest] and adheres to the [artifacts guidance](https://github.com/opencontainers/image-spec/blob/main/artifacts-guidance.md).

```JSON
{
"schemaVersion": 2,
"mediaType": "application/vnd.cnai.model.manifest.v1+json",
"config": {
"mediaType": "application/vnd.cnai.model.config.v1+json",
"digest": "sha256:d5815835051dd97d800a03f641ed8162877920e734d3d705b698912602b8c763",
"size": 301
},
"layers": [
{
"mediaType": "application/vnd.cnai.model.layer.v1.tar",
"digest": "sha256:3f907c1a03bf20f20355fe449e18ff3f9de2e49570ffb536f1a32f20c7179808",
"size": 30327160
},
{
"mediaType": "application/vnd.cnai.model.layer.v1.tar",
"digest": "sha256:6d923539c5c208de77146335584252c0b1b81e35c122dd696fe6e04ed03d7411",
"size": 5018536960
},
{
"mediaType": "application/vnd.cnai.model.doc.v1.tar",
"digest": "sha256:5e236ec37438b02c01c83d134203a646cb354766ac294e533a308dd8caa3a11e",
"size": 23040
}
]
}
```

* **`mediaType`** *string*

### Extended OCI Image Manifest Specification For Model Artifacts
This REQUIRED property MUST be `application/vnd.cnai.model.manifest.v1+json`.

The image manifest of model artifacts follows the [OCI Image Manifest Specification](https://github.com/opencontainers/image-spec/blob/main/manifest.md) and adheres to the [guidelines for artifacts usage](https://github.com/opencontainers/image-spec/blob/main/manifest.md#guidelines-for-artifact-usage). Specifically, it leverages the extensible `artifactType` and `annotations` properties to define attributes specific to model artifacts.
* **`layers`** *array of objects*

![manifest](./img/manifest.svg)
* **`mediaType`** *string*

* **`artifactType`** *string*
This is a REQUIRED property. Implemenations MUST support at least the following media types:

This REQUIRED property MUST be `application/vnd.cnai.model.manifest.v1+json`.
* `application/vnd.cnai.model.layer.v1.tar`: The layer is a [tar archive][tar-archive] that contains the model weight file. If the model has multiple weight files, they SHOULD be packaged into separate layers.

* **`layers`** *array of objects*
* `application/vnd.cnai.model.doc.v1.tar`: The layer is a [tar archive][tar-archive] that includes documentation files like `README.md`, `LICENSE`, etc.

* **`mediaType`** *string*
* `application/vnd.cnai.model.code.v1.tar`: The layer is a [tar archive][tar-archive] that includes code artifacts like scripts, code files etc.

* `application/vnd.cnai.model.dataset.v1.tar`: The layer is a [tar archive][tar-archive] that includes datasets that may be needed for the lifecycle of AI/ML models.

* **`annotations`** *string-string map*

This REQUIRED property MUST be one of the [OCI Image Media Types](https://github.com/opencontainers/image-spec/blob/main/media-types.md) designated for [layers](https://github.com/opencontainers/image-spec/blob/main/layer.md).
Otherwise, it will not be compatible with the container runtime.
This OPTIONAL property contains arbitrary attributes for the layer. For metadata specific to models, implementations SHOULD use the predefined annotation keys as outlined in the [Layer Annotation Keys](./annotations.md#layer-annotation-keys).

* **`artifactType`** *string*
## Guidance on Layers

This REQUIRED property MUST be at least the following media types:
This section describes how to serialize AI/ML artifacts into a blob called a layer.

* `application/vnd.cnai.model.layer.v1.tar`: The layer is a [tar archive](https://en.wikipedia.org/wiki/Tar_(computing)) that contains the model weight file. If the model has multiple weight files, they SHOULD be packaged into separate layers.
* `application/vnd.cnai.model.layer.v1.tar+gzip`: The layer is a [tar archive](https://en.wikipedia.org/wiki/Tar_(computing)) compressed with [gzip](https://datatracker.ietf.org/doc/html/rfc1952) that contains the model weight file.
If the model has multiple weight files, they SHOULD be packaged in separate layers.
**Implementers' note**: It is recommended to package weight files without compression to avoid unnecessary overhead of decompression by the container runtime as model weight files are typically uncompressible.

*Implementers note*: It is recommended to package weight files without compression to avoid unnecessary overhead of decompression by the container runtime as model weight files are typically already compressed.
* `application/vnd.cnai.model.doc.v1.tar`: The layer is a [tar archive](https://en.wikipedia.org/wiki/Tar_(computing)) that includes documentation files like `README.md`, `LICENSE`, etc.
* `application/vnd.cnai.model.config.v1.tar`: The layer is a [tar archive](https://en.wikipedia.org/wiki/Tar_(computing)) that includes additional configuration files such as `config.json`，`tokenizer.json`, `generation_config.json`, etc.
### `+gzip` Media Types

* **`annotations`** *string-string map*
The `application/vnd.cnai.model.layer.v1.tar+gzip` represents an `application/vnd.cnai.model.layer.v1.tar` payload which has been compressed with [gzip][rfc1952_2]. The mediaTypes `application/vnd.cnai.model.doc.v1.tar+gzip`, `application/vnd.cnai.model.code.v1.tar+gzip`, `application/vnd.cnai.model.dataset.v1.tar+gzip` refer to the gzip compressed payloads of their corresponding type.

This OPTIONAL property contains arbitrary attributes for the layer. For metadata specific to models, implementations SHOULD use the predefined annotation keys as outlined in the [Layer Annotation Keys](./annotations.md#layer-annotation-keys).
### `+zstd` Media Types

The `application/vnd.cnai.model.layer.v1.tar+zstd` represents an `application/vnd.cnai.model.layer.v1.tar` payload which has been compressed with the [zstd][rfc8478] algorithm. The mediaTypes `application/vnd.cnai.model.doc.v1.tar+zstd`, `application/vnd.cnai.model.code.v1.tar+zstd`, `application/vnd.cnai.model.dataset.v1.tar+zstd` refer to the zstd compressed payloads of their corresponding type.

### File Attributes

Where supported, MUST include file attributes

* Modification Time (`mtime`)
* User ID (`uid`)
* User Name (`uname`) should be ignored on platforms that support User ID (`uid`)
* Group ID (`gid`)
* Group Name (`gname`) should be ignored on platforms that support Group ID (`gid`)
* Mode (`mode`)

### Reproducibility

To ensure tar layers are packaged in a reproducible way, implementation SHOULD adhere to the following guidance:

* If the archive includes multiple files, files should be added to the archive in lexicographical order.
* File metadata (such as modification time, owner/group id) should be set to known, constant values rather than the current values on disk.
* Platform/implementation specific metadata should be omitted from the archive.

## Workflow

As the model format specification conforms to the [OCI Image Specification](https://github.com/opencontainers/image-spec/blob/main/layer.md), it naturally aligns with the standard [OCI distribution workflow](https://github.com/opencontainers/distribution-spec/blob/main/spec.md).
The model format specification naturally aligns with the standard [OCI distribution specification][distribution-spec].

This section outlines the typical workflow for a model OCI artifact, which consists of two main stages: `BUILD & PUSH` and `PULL & SERVE`.

Expand All @@ -68,3 +123,10 @@ The generated artifact can then be pushed to OCI registries (e.g., Harbor, Docke
Once the model artifact is stored in an OCI registry, the container runtime (e.g., containerd, CRI-O) can pull it from the OCI registry and mount it as a read-only volume during the model serving process, if required.

![pull-serve](./img/pull-and-serve.png)

[image-spec]: https://github.com/opencontainers/image-spec/blob/main/spec.md#image-format-specification
[rfc1952_2]: https://tools.ietf.org/html/rfc1952
[tar-archive]: https://en.wikipedia.org/wiki/Tar_(computing)
[image-manifest]: https://github.com/opencontainers/image-spec/blob/main/manifest.md
[rfc8478]: https://tools.ietf.org/html/rfc8478
[distribution-spec]: https://github.com/opencontainers/distribution-spec/blob/main/spec.md

0 comments on commit ef8c140

Please sign in to comment.