diff --git a/docs/chains/doc_gen/API-reference.mdx b/docs/chains/doc_gen/API-reference.mdx index 909b33838..d45c26dec 100644 --- a/docs/chains/doc_gen/API-reference.mdx +++ b/docs/chains/doc_gen/API-reference.mdx @@ -38,11 +38,11 @@ chainlet instance is provided. **Parameters:** -| Name | Type | Description | -|----------------|----------------------|--------------------------------------------------------------------------------------------------------------| -| `chainlet_cls` | *Type[ChainletBase]* | The chainlet class of the dependency. | -| `retries` | *int* | The number of times to retry the remote chainlet in case of failures (e.g. due to transient network issues). | -| `timeout_sec` | *int* | Timeout for the HTTP request to this chainlet. | +| Name | Type | Description | +|----------------|----------------------------------------------------------|--------------------------------------------------------------------------------------------------------------| +| `chainlet_cls` | *Type[[ChainletBase](#class-truss-chains-chainletbase)]* | The chainlet class of the dependency. | +| `retries` | *int* | The number of times to retry the remote chainlet in case of failures (e.g. due to transient network issues). | +| `timeout_sec` | *int* | Timeout for the HTTP request to this chainlet. | * **Returns:** A “symbolic marker” to be used as a default argument in a chainlet’s initializer. @@ -65,7 +65,7 @@ context instance is provided. A “symbolic marker” to be used as a default argument in a chainlet’s initializer. -### *class* `truss_chains.DeploymentContext(Generic[UserConfigT])` +### *class* `truss_chains.DeploymentContext` Bases: `pydantic.BaseModel` @@ -85,11 +85,7 @@ an access token for downloading model weights). | `chainlet_to_service` | *Mapping[str,[ServiceDescriptor](#class-truss-chains-servicedescriptor)]* | A mapping from chainlet names to service descriptors. This is used create RPCs sessions to dependency chainlets. It contains only the chainlet services that are dependencies of the current chainlet. | | `secrets` | *Mapping[str,str]* | A mapping from secret names to secret values. It contains only the secrets that are listed in `remote_config.assets.secret_keys` of the current chainlet. | | `user_env` | *Mapping[str,str]* | These values can be provided to the deploy command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | - -#### chainlet_to_service: -Mapping[str, [ServiceDescriptor](#class-truss-chains-servicedescriptor)]* - -#### data_dir *: Path | None* +| `environment` | *[Environment](#class-truss-chains-definitions-environment)\|None* | The environment that the chainlet is deployed in. None if the chainlet is not associated with an environment. | #### get_baseten_api_key() @@ -107,11 +103,26 @@ Mapping[str, [ServiceDescriptor](#class-truss-chains-servicedescriptor)]* * **Return type:** [*ServiceDescriptor*](#class-truss-chains-servicedescriptor) -#### secrets *: Mapping[str, str]* +### *class* `truss_chains.definitions.Environment` + +Bases: `pydantic.BaseModel` + +The environment the chainlet is deployed in. + +* **Parameters:** + **name** (*str*) – The name of the environment. -#### user_config *: UserConfigT* +### *class* `truss_chains.ChainletOptions` -#### user_env *: Mapping[str, str]* +Bases: `pydantic.BaseModel` + + +**Parameters:** + +| Name | Type | Description | +|------------------------|--------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `enable_b10_tracing` | *bool* | enables baseten-internal trace data collection. This helps baseten engineers better analyze chain performance in case of issues. It is independent of a potentially user-configured tracing instrumentation. Turning this on, could add performance overhead. | +| `env_variables` | *Mapping[str,str]* | static environment variables available to the deployed chainlet. | ### *class* `truss_chains.RPCOptions` @@ -147,9 +158,9 @@ class MyChainlet(ChainletBase): **Parameters:** -| Name | Type | Description | -|-------|----------------------|---------------------| -| `cls` | *Type[ChainletBase]* | The chainlet class. | +| Name | Type | Description | +|-------|----------------------------------------------------------|---------------------| +| `cls` | *Type[[ChainletBase](#class-truss-chains-chainletbase)]* | The chainlet class. | * **Return type:** *Type*[*ChainletBase*] @@ -182,12 +193,13 @@ class MyChainlet(chains.ChainletBase): **Parameters:** -| Name | Type | Description | -|----------------|--------------------------------------------------|-------------| -| `docker_image` | *[DockerImage](#class-truss-chains-dockerimage)* | | -| `compute` | *[Compute](#class-truss-chains-compute)* | | -| `assets` | *[Assets](#class-truss-chains-assets)* | | -| `name` | *str\|None* | | +| Name | Type | Description | +|----------------|----------------------------------------------------------|-------------| +| `docker_image` | *[DockerImage](#class-truss-chains-dockerimage)* | | +| `compute` | *[Compute](#class-truss-chains-compute)* | | +| `assets` | *[Assets](#class-truss-chains-assets)* | | +| `name` | *str\|None* | | +| `options` | *[ChainletOptions](#class-truss-chains-chainletoptions)* | | ### *class* `truss_chains.DockerImage` @@ -220,11 +232,12 @@ Bases: `Enum` Default images, curated by baseten, for different python versions. If a Chainlet uses GPUs, drivers will be included in the image. -#### PY310 *= 'py310'* - -#### PY311 *= 'py311'* +| Enum Member | Value | +|-------------|---------| +| `PY310` | *py310* | +| `PY311 ` | *py311* | +| `PY39` | *py39* | -#### PY39 *= 'py39'* ### *class* `truss_chains.CustomImage` @@ -312,15 +325,16 @@ Deploys a chain remotely (with all dependent chainlets). **Parameters:** -| Name | Type | Description | -|-------------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -| `entrypoint` | *Type[ChainletBase]* | The chainlet class that serves as the entrypoint to the chain. | -| `chain_name` | *str* | The name of the chain. | -| `publish` | *bool* | Whether to publish the chain as a published deployment (it is a draft deployment otherwise) | -| `promote` | *bool* | Whether to promote the chain to be the production deployment (this implies publishing as well). | -| `user_env` | *Mapping[str,str]\|None* | These values can be provided to the push command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | -| `only_generate_trusses` | *bool* | Used for debugging purposes. If set to True, only the the underlying truss models for the chainlets are generated in `/tmp/.chains_generated`. | -| `remote` | *str\|None* | name of a remote config in .trussrc. If not provided, it will be inquired. | +| Name | Type | Description | +|-------------------------|----------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `entrypoint` | *Type[[ChainletBase](#class-truss-chains-chainletbase)]* | The chainlet class that serves as the entrypoint to the chain. | +| `chain_name` | *str* | The name of the chain. | +| `publish` | *bool* | Whether to publish the chain as a published deployment (it is a draft deployment otherwise) | +| `promote` | *bool* | Whether to promote the chain to be the production deployment (this implies publishing as well). | +| `user_env` | *Mapping[str,str]\|None* | These values can be provided to the push command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | +| `only_generate_trusses` | *bool* | Used for debugging purposes. If set to True, only the the underlying truss models for the chainlets are generated in `/tmp/.chains_generated`. | +| `remote` | *str\|None* | name of a remote config in .trussrc. If not provided, it will be inquired. | +| `environment` | *str\|None* | The name of an environment to promote deployment into. | * **Returns:** A chain service handle to the deployed chain. @@ -442,7 +456,7 @@ Context manager local debug execution of a chain. The arguments only need to be provided if the chainlets explicitly access any the corresponding fields of -[`DeploymentContext`](#class-truss-chains-deploymentcontext-generic-userconfigt). +[`DeploymentContext`](#class-truss-chains-deploymentcontext). **Parameters:** @@ -552,11 +566,11 @@ Factory method, convenient to be used in chainlet’s `__init__`-method. **Parameters:** -| Name | Type | Description | -|---------------|----------------------------------------------------------------------------------|-------------------------------------------------------------------| -| `predict_url` | *str* | URL to predict endpoint of another chain / truss model. | -| `context` | *[DeploymentContext](#class-truss-chains-deploymentcontext-generic-userconfigt)* | Deployment context object, obtained in the chainlet’s `__init__`. | -| `options` | *[RPCOptions](#class-truss-chains-rpcoptions)* | RPC options, e.g. retries. | +| Name | Type | Description | +|---------------|--------------------------------------------------------------|-------------------------------------------------------------------| +| `predict_url` | *str* | URL to predict endpoint of another chain / truss model. | +| `context` | *[DeploymentContext](#class-truss-chains-deploymentcontext)* | Deployment context object, obtained in the chainlet’s `__init__`. | +| `options` | *[RPCOptions](#class-truss-chains-rpcoptions)* | RPC options, e.g. retries. | ### *class* `truss_chains.RemoteErrorDetail` diff --git a/docs/chains/doc_gen/generate_reference.py b/docs/chains/doc_gen/generate_reference.py index feb6ae483..2f09fc5c0 100644 --- a/docs/chains/doc_gen/generate_reference.py +++ b/docs/chains/doc_gen/generate_reference.py @@ -31,6 +31,7 @@ # "truss_chains.definitions.AssetSpec", # "truss_chains.definitions.ComputeSpec", "truss_chains.remote.ChainService", + "truss_chains.definitions.Environment", ] @@ -42,6 +43,8 @@ "truss_chains.depends", "truss_chains.depends_context", "truss_chains.DeploymentContext", + "truss_chains.definitions.Environment", + "truss_chains.ChainletOptions", "truss_chains.RPCOptions", "truss_chains.mark_entrypoint", ], diff --git a/docs/chains/doc_gen/generated-reference.mdx b/docs/chains/doc_gen/generated-reference.mdx index cd9a14b8d..911457158 100644 --- a/docs/chains/doc_gen/generated-reference.mdx +++ b/docs/chains/doc_gen/generated-reference.mdx @@ -89,11 +89,14 @@ an access token for downloading model weights). | `chainlet_to_service` | *Mapping[str,[ServiceDescriptor](#truss_chains.ServiceDescriptor* | A mapping from chainlet names to service descriptors. This is used create RPCs sessions to dependency chainlets. It contains only the chainlet services that are dependencies of the current chainlet. | | `secrets` | *MappingNoIter[str,str]* | A mapping from secret names to secret values. It contains only the secrets that are listed in `remote_config.assets.secret_keys` of the current chainlet. | | `user_env` | *Mapping[str,str]* | These values can be provided to the deploy command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | +| `environment` | *[Environment](#truss_chains.definitions.Environment* | The environment that the chainlet is deployed in. None if the chainlet is not associated with an environment. | #### chainlet_to_service *: Mapping[str, [ServiceDescriptor](#truss_chains.ServiceDescriptor)]* #### data_dir *: Path | None* +#### environment *: [Environment](#truss_chains.definitions.Environment) | None* + #### get_baseten_api_key() * **Return type:** @@ -112,6 +115,32 @@ an access token for downloading model weights). #### user_env *: Mapping[str, str]* +### *class* `truss_chains.definitions.Environment` + +Bases: `pydantic.BaseModel` + +The environment the chainlet is deployed in. + +* **Parameters:** + **name** (*str*) – The name of the environment. +#### name *: str* + +### *class* `truss_chains.ChainletOptions` + +Bases: `pydantic.BaseModel` + + +**Parameters:** + +| Name | Type | Description | +|------|------|-------------| +| `enable_b10_tracing` | *bool* | enables baseten-internal trace data collection. This helps baseten engineers better analyze chain performance in case of issues. It is independent of a potentially user-configured tracing instrumentation. Turning this on, could add performance overhead. | +| `env_variables` | *Mapping[str,str]* | static environment variables available to the deployed chainlet. | + +#### enable_b10_tracing *: bool* + +#### env_variables *: Mapping[str, str]* + ### *class* `truss_chains.RPCOptions` Bases: `pydantic.BaseModel` @@ -187,6 +216,7 @@ class MyChainlet(chains.ChainletBase): | `compute` | *[Compute](#truss_chains.Compute* | | | `assets` | *[Assets](#truss_chains.Assets* | | | `name` | *str\|None* | | +| `options` | *[ChainletOptions](#truss_chains.ChainletOptions* | | #### assets *: [Assets](#truss_chains.Assets)* @@ -207,6 +237,8 @@ class MyChainlet(chains.ChainletBase): #### name *: str | None* +#### options *: [ChainletOptions](#truss_chains.ChainletOptions)* + ### *class* `truss_chains.DockerImage` Bases: `pydantic.BaseModel` @@ -377,6 +409,7 @@ Deploys a chain remotely (with all dependent chainlets). | `user_env` | *Mapping[str,str]\|None* | These values can be provided to the push command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | | `only_generate_trusses` | *bool* | Used for debugging purposes. If set to True, only the the underlying truss models for the chainlets are generated in `/tmp/.chains_generated`. | | `remote` | *str\|None* | name of a remote config in .trussrc. If not provided, it will be inquired. | +| `environment` | *str\|None* | The name of an environment to promote deployment into. | * **Returns:** A chain service handle to the deployed chain. @@ -435,8 +468,7 @@ This property must be externally populated. Queries the statuses of all chainlets in the chain. * **Returns:** - List of `DeployedChainlet`, `(name, is_entrypoint, status, logs_url)` - for each chainlet. + List of `DeployedChainlet` for each chainlet. * **Return type:** list[*DeployedChainlet*] diff --git a/docs/chains/doc_gen/reference.patch b/docs/chains/doc_gen/reference.patch index 2bb8db588..b31761583 100644 --- a/docs/chains/doc_gen/reference.patch +++ b/docs/chains/doc_gen/reference.patch @@ -1,5 +1,5 @@ ---- docs/chains/doc_gen/generated-reference.mdx 2024-08-06 16:49:38.423081000 -0700 -+++ docs/chains/doc_gen/API-reference.mdx 2024-08-06 16:43:37.299866932 -0700 +--- docs/chains/doc_gen/generated-reference.mdx 2024-11-04 11:41:28.619593764 -0800 ++++ docs/chains/doc_gen/API-reference.mdx 2024-11-04 11:53:15.657920649 -0800 @@ -24,31 +24,28 @@ dependency of another chainlet. The return value of `depends` is intended to be used as a default argument in a chainlet’s `__init__`-method. @@ -27,11 +27,11 @@ -| `retries` | *int* | The number of times to retry the remote chainlet in case of failures (e.g. due to transient network issues). | -| `timeout_sec` | *int* | Timeout for the HTTP request to this chainlet. | - -+| Name | Type | Description | -+|----------------|----------------------|--------------------------------------------------------------------------------------------------------------| -+| `chainlet_cls` | *Type[ChainletBase]* | The chainlet class of the dependency. | -+| `retries` | *int* | The number of times to retry the remote chainlet in case of failures (e.g. due to transient network issues). | -+| `timeout_sec` | *int* | Timeout for the HTTP request to this chainlet. | ++| Name | Type | Description | ++|----------------|----------------------------------------------------------|--------------------------------------------------------------------------------------------------------------| ++| `chainlet_cls` | *Type[[ChainletBase](#class-truss-chains-chainletbase)]* | The chainlet class of the dependency. | ++| `retries` | *int* | The number of times to retry the remote chainlet in case of failures (e.g. due to transient network issues). | ++| `timeout_sec` | *int* | Timeout for the HTTP request to this chainlet. | * **Returns:** A “symbolic marker” to be used as a default argument in a chainlet’s initializer. @@ -57,15 +57,14 @@ -* **Return type:** - [*DeploymentContext*](#truss_chains.DeploymentContext) --### *class* `truss_chains.DeploymentContext` -+### *class* `truss_chains.DeploymentContext(Generic[UserConfigT])` + ### *class* `truss_chains.DeploymentContext` -Bases: `pydantic.BaseModel`, `Generic`[`UserConfigT`] +Bases: `pydantic.BaseModel` Bundles config values and resources needed to instantiate Chainlets. -@@ -82,15 +78,16 @@ +@@ -82,20 +78,14 @@ **Parameters:** @@ -76,6 +75,13 @@ -| `chainlet_to_service` | *Mapping[str,[ServiceDescriptor](#truss_chains.ServiceDescriptor* | A mapping from chainlet names to service descriptors. This is used create RPCs sessions to dependency chainlets. It contains only the chainlet services that are dependencies of the current chainlet. | -| `secrets` | *MappingNoIter[str,str]* | A mapping from secret names to secret values. It contains only the secrets that are listed in `remote_config.assets.secret_keys` of the current chainlet. | -| `user_env` | *Mapping[str,str]* | These values can be provided to the deploy command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | +-| `environment` | *[Environment](#truss_chains.definitions.Environment* | The environment that the chainlet is deployed in. None if the chainlet is not associated with an environment. | +- +-#### chainlet_to_service *: Mapping[str, [ServiceDescriptor](#truss_chains.ServiceDescriptor)]* +- +-#### data_dir *: Path | None* +- +-#### environment *: [Environment](#truss_chains.definitions.Environment) | None* +| Name | Type | Description | +|-----------------------|---------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| +| `data_dir` | *Path\|None* | The directory where the chainlet can store and access data, e.g. for downloading model weights. | @@ -83,35 +89,61 @@ +| `chainlet_to_service` | *Mapping[str,[ServiceDescriptor](#class-truss-chains-servicedescriptor)]* | A mapping from chainlet names to service descriptors. This is used create RPCs sessions to dependency chainlets. It contains only the chainlet services that are dependencies of the current chainlet. | +| `secrets` | *Mapping[str,str]* | A mapping from secret names to secret values. It contains only the secrets that are listed in `remote_config.assets.secret_keys` of the current chainlet. | +| `user_env` | *Mapping[str,str]* | These values can be provided to the deploy command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | ++| `environment` | *[Environment](#class-truss-chains-definitions-environment)\|None* | The environment that the chainlet is deployed in. None if the chainlet is not associated with an environment. | --#### chainlet_to_service *: Mapping[str, [ServiceDescriptor](#truss_chains.ServiceDescriptor)]* -+#### chainlet_to_service: -+Mapping[str, [ServiceDescriptor](#class-truss-chains-servicedescriptor)]* - - #### data_dir *: Path | None* + #### get_baseten_api_key() -@@ -101,12 +98,16 @@ +@@ -104,16 +94,14 @@ #### get_service_descriptor(chainlet_name) -* **Parameters:** - **chainlet_name** (*str*) +-* **Return type:** +- [*ServiceDescriptor*](#truss_chains.ServiceDescriptor) +- +-#### secrets *: MappingNoIter[str, str]* +**Parameters:** -+ + +-#### user_config *: UserConfigT* +| Name | Type | Description | +|-----------------|-------|---------------------------| +| `chainlet_name` | *str* | The name of the chainlet. | -+ - * **Return type:** -- [*ServiceDescriptor*](#truss_chains.ServiceDescriptor) + +-#### user_env *: Mapping[str, str]* ++* **Return type:** + [*ServiceDescriptor*](#class-truss-chains-servicedescriptor) --#### secrets *: MappingNoIter[str, str]* -+#### secrets *: Mapping[str, str]* + ### *class* `truss_chains.definitions.Environment` + +@@ -123,7 +111,6 @@ + + * **Parameters:** + **name** (*str*) – The name of the environment. +-#### name *: str* + + ### *class* `truss_chains.ChainletOptions` + +@@ -132,14 +119,10 @@ - #### user_config *: UserConfigT* + **Parameters:** + +-| Name | Type | Description | +-|------|------|-------------| +-| `enable_b10_tracing` | *bool* | enables baseten-internal trace data collection. This helps baseten engineers better analyze chain performance in case of issues. It is independent of a potentially user-configured tracing instrumentation. Turning this on, could add performance overhead. | +-| `env_variables` | *Mapping[str,str]* | static environment variables available to the deployed chainlet. | +- +-#### enable_b10_tracing *: bool* +- +-#### env_variables *: Mapping[str, str]* ++| Name | Type | Description | ++|------------------------|--------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| ++| `enable_b10_tracing` | *bool* | enables baseten-internal trace data collection. This helps baseten engineers better analyze chain performance in case of issues. It is independent of a potentially user-configured tracing instrumentation. Turning this on, could add performance overhead. | ++| `env_variables` | *Mapping[str,str]* | static environment variables available to the deployed chainlet. | -@@ -120,15 +121,10 @@ + ### *class* `truss_chains.RPCOptions` + +@@ -149,15 +132,10 @@ **Parameters:** @@ -131,7 +163,7 @@ ### `truss_chains.mark_entrypoint` -@@ -140,18 +136,23 @@ +@@ -169,18 +147,23 @@ Example usage: @@ -149,9 +181,9 @@ - **cls** (*Type* *[**ChainletT* *]*) +**Parameters:** + -+| Name | Type | Description | -+|-------|----------------------|---------------------| -+| `cls` | *Type[ChainletBase]* | The chainlet class. | ++| Name | Type | Description | ++|-------|----------------------------------------------------------|---------------------| ++| `cls` | *Type[[ChainletBase](#class-truss-chains-chainletbase)]* | The chainlet class. | + * **Return type:** - *Type*[*ChainletT*] @@ -159,7 +191,7 @@ # Remote Configuration -@@ -165,7 +166,7 @@ +@@ -194,7 +177,7 @@ This is specified as a class variable for each chainlet class, e.g.: @@ -168,7 +200,7 @@ import truss_chains as chains -@@ -181,31 +182,12 @@ +@@ -210,34 +193,13 @@ **Parameters:** @@ -178,6 +210,7 @@ -| `compute` | *[Compute](#truss_chains.Compute* | | -| `assets` | *[Assets](#truss_chains.Assets* | | -| `name` | *str\|None* | | +-| `options` | *[ChainletOptions](#truss_chains.ChainletOptions* | | - - -#### assets *: [Assets](#truss_chains.Assets)* @@ -197,16 +230,19 @@ - *ComputeSpec* - -#### name *: str | None* -+| Name | Type | Description | -+|----------------|--------------------------------------------------|-------------| -+| `docker_image` | *[DockerImage](#class-truss-chains-dockerimage)* | | -+| `compute` | *[Compute](#class-truss-chains-compute)* | | -+| `assets` | *[Assets](#class-truss-chains-assets)* | | -+| `name` | *str\|None* | | +- +-#### options *: [ChainletOptions](#truss_chains.ChainletOptions)* ++| Name | Type | Description | ++|----------------|----------------------------------------------------------|-------------| ++| `docker_image` | *[DockerImage](#class-truss-chains-dockerimage)* | | ++| `compute` | *[Compute](#class-truss-chains-compute)* | | ++| `assets` | *[Assets](#class-truss-chains-assets)* | | ++| `name` | *str\|None* | | ++| `options` | *[ChainletOptions](#class-truss-chains-chainletoptions)* | | ### *class* `truss_chains.DockerImage` -@@ -213,35 +195,23 @@ +@@ -245,35 +207,23 @@ Configures the docker image in which a remoted chainlet is deployed. @@ -253,7 +289,24 @@ ### *class* `truss_chains.BasetenImage` -@@ -264,42 +234,35 @@ +@@ -282,11 +232,12 @@ + Default images, curated by baseten, for different python versions. If a Chainlet + uses GPUs, drivers will be included in the image. + +-#### PY310 *= 'py310'* ++| Enum Member | Value | ++|-------------|---------| ++| `PY310` | *py310* | ++| `PY311 ` | *py311* | ++| `PY39` | *py39* | + +-#### PY311 *= 'py311'* +- +-#### PY39 *= 'py39'* + + ### *class* `truss_chains.CustomImage` + +@@ -296,42 +247,35 @@ **Parameters:** @@ -297,6 +350,7 @@ -| `gpu` | *str\|Accelerator\|None* | GPU accelerator type, e.g. “A10G”, “A100”, refer to the [truss config](https://docs.baseten.co/reference/config#resources-accelerator) for more choices. | -| `gpu_count` | *int* | Number of GPUs to allocate. | -| `predict_concurrency` | *int\|Literal['cpu_count']* | Number of concurrent requests a single replica of a deployed chainlet handles. | +- +| Name | Type | Description | +|-----------------------|-----------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------| +| `cpu_count` | *int* | Minimum number of CPUs to allocate. | @@ -305,14 +359,13 @@ +| `gpu_count` | *int* | Number of GPUs to allocate. | +| `predict_concurrency` | *int\|Literal['cpu_count']* | Number of concurrent requests a single replica of a deployed chainlet handles. | -- -Concurrency concepts are explained in [this guide](https://docs.baseten.co/deploy/guides/concurrency#predict-concurrency). # noqa: E501 +Concurrency concepts are explained +in [this guide](https://docs.baseten.co/deploy/guides/concurrency#predict-concurrency). It is important to understand the difference between predict_concurrency and the concurrency target (used for autoscaling, i.e. adding or removing replicas). Furthermore, the `predict_concurrency` of a single instance is implemented in -@@ -310,18 +273,13 @@ +@@ -342,18 +286,13 @@ - With a threadpool if it’s a synchronous function. This requires that the threads don’t have significant CPU load (due to the GIL). @@ -332,7 +385,7 @@ import truss_chains as chains from truss import truss_config -@@ -335,27 +293,13 @@ +@@ -367,27 +306,13 @@ See [truss caching guide](https://docs.baseten.co/deploy/guides/model-cache#enabling-caching-for-a-model) for more details on caching. @@ -365,7 +418,7 @@ # Core -@@ -368,69 +312,34 @@ +@@ -400,75 +325,41 @@ **Parameters:** @@ -378,15 +431,17 @@ -| `user_env` | *Mapping[str,str]\|None* | These values can be provided to the push command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | -| `only_generate_trusses` | *bool* | Used for debugging purposes. If set to True, only the the underlying truss models for the chainlets are generated in `/tmp/.chains_generated`. | -| `remote` | *str\|None* | name of a remote config in .trussrc. If not provided, it will be inquired. | -+| Name | Type | Description | -+|-------------------------|--------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| -+| `entrypoint` | *Type[ChainletBase]* | The chainlet class that serves as the entrypoint to the chain. | -+| `chain_name` | *str* | The name of the chain. | -+| `publish` | *bool* | Whether to publish the chain as a published deployment (it is a draft deployment otherwise) | -+| `promote` | *bool* | Whether to promote the chain to be the production deployment (this implies publishing as well). | -+| `user_env` | *Mapping[str,str]\|None* | These values can be provided to the push command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | -+| `only_generate_trusses` | *bool* | Used for debugging purposes. If set to True, only the the underlying truss models for the chainlets are generated in `/tmp/.chains_generated`. | -+| `remote` | *str\|None* | name of a remote config in .trussrc. If not provided, it will be inquired. | +-| `environment` | *str\|None* | The name of an environment to promote deployment into. | ++| Name | Type | Description | ++|-------------------------|----------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| ++| `entrypoint` | *Type[[ChainletBase](#class-truss-chains-chainletbase)]* | The chainlet class that serves as the entrypoint to the chain. | ++| `chain_name` | *str* | The name of the chain. | ++| `publish` | *bool* | Whether to publish the chain as a published deployment (it is a draft deployment otherwise) | ++| `promote` | *bool* | Whether to promote the chain to be the production deployment (this implies publishing as well). | ++| `user_env` | *Mapping[str,str]\|None* | These values can be provided to the push command and customize the behavior of deployed chainlets. E.g. for differentiating between prod and dev version of the same chain. | ++| `only_generate_trusses` | *bool* | Used for debugging purposes. If set to True, only the the underlying truss models for the chainlets are generated in `/tmp/.chains_generated`. | ++| `remote` | *str\|None* | name of a remote config in .trussrc. If not provided, it will be inquired. | ++| `environment` | *str\|None* | The name of an environment to promote deployment into. | * **Returns:** A chain service handle to the deployed chain. @@ -447,7 +502,14 @@ Queries the statuses of all chainlets in the chain. -@@ -446,18 +355,23 @@ + * **Returns:** +- List of `DeployedChainlet` for each chainlet. ++ List of `DeployedChainlet`, `(name, is_entrypoint, status, logs_url)` ++ for each chainlet. + * **Return type:** + list[*DeployedChainlet*] + +@@ -478,18 +369,23 @@ Invokes the entrypoint with JSON data. @@ -475,7 +537,7 @@ Link to status page on Baseten. -@@ -479,12 +393,12 @@ +@@ -511,12 +407,12 @@ You can now in `root/sub_package/chainlet.py` point to the requirements file like this: @@ -490,7 +552,7 @@ This helper uses the directory of the immediately calling module as an absolute reference point for resolving the file location. Therefore, you MUST NOT wrap the instantiation of `make_abs_path_here` into a -@@ -492,7 +406,7 @@ +@@ -524,7 +420,7 @@ Ok: @@ -499,7 +561,7 @@ def foo(path: AbsPath): abs_path = path.abs_path -@@ -502,7 +416,7 @@ +@@ -534,7 +430,7 @@ Not Ok: @@ -508,7 +570,7 @@ def foo(path: str): dangerous_value = make_abs_path_here(path).abs_path -@@ -510,8 +424,15 @@ +@@ -542,8 +438,15 @@ foo("./somewhere") ``` @@ -526,14 +588,14 @@ * **Return type:** *AbsPath* -@@ -520,24 +441,24 @@ +@@ -552,24 +455,24 @@ Context manager local debug execution of a chain. The arguments only need to be provided if the chainlets explicitly access any the -corresponding fields of `DeploymentContext`. - +corresponding fields of -+[`DeploymentContext`](#class-truss-chains-deploymentcontext-generic-userconfigt). ++[`DeploymentContext`](#class-truss-chains-deploymentcontext). **Parameters:** @@ -560,7 +622,7 @@ import os import truss_chains as chains -@@ -563,7 +484,8 @@ +@@ -595,7 +498,8 @@ print(result) ``` @@ -570,7 +632,7 @@ for more details. ### *class* `truss_chains.ServiceDescriptor` -@@ -575,22 +497,13 @@ +@@ -607,22 +511,13 @@ **Parameters:** @@ -586,20 +648,20 @@ -#### options *: [RPCOptions](#truss_chains.RPCOptions)* - -#### predict_url *: str* -- --### *class* `truss_chains.StubBase` +| Name | Type | Description | +|---------------|------------------------------------------------|-------------| +| `name` | *str* | | +| `predict_url` | *str* | | +| `options` | *[RPCOptions](#class-truss-chains-rpcoptions)* | | +-### *class* `truss_chains.StubBase` +- -Bases: `ABC` +## *class* `truss_chains.StubBase` Base class for stubs that invoke remote chainlets. -@@ -598,17 +511,18 @@ +@@ -630,17 +525,18 @@ in user-code for wrapping a deployed truss model into the chains framework, e.g. like that: @@ -620,7 +682,7 @@ resp = await self._remote.predict_async( json_payload={"audio": audio_b64}) return WhisperOutput(text=resp["text"], language=resp["language"]) -@@ -625,28 +539,24 @@ +@@ -657,28 +553,24 @@ ) ``` @@ -650,15 +712,15 @@ -| `context` | *[DeploymentContext](#truss_chains.DeploymentContext* | Deployment context object, obtained in the chainlet’s `__init__`. | -| `options` | *[RPCOptions](#truss_chains.RPCOptions* | RPC options, e.g. retries. | - -+| Name | Type | Description | -+|---------------|----------------------------------------------------------------------------------|-------------------------------------------------------------------| -+| `predict_url` | *str* | URL to predict endpoint of another chain / truss model. | -+| `context` | *[DeploymentContext](#class-truss-chains-deploymentcontext-generic-userconfigt)* | Deployment context object, obtained in the chainlet’s `__init__`. | -+| `options` | *[RPCOptions](#class-truss-chains-rpcoptions)* | RPC options, e.g. retries. | ++| Name | Type | Description | ++|---------------|--------------------------------------------------------------|-------------------------------------------------------------------| ++| `predict_url` | *str* | URL to predict endpoint of another chain / truss model. | ++| `context` | *[DeploymentContext](#class-truss-chains-deploymentcontext)* | Deployment context object, obtained in the chainlet’s `__init__`. | ++| `options` | *[RPCOptions](#class-truss-chains-rpcoptions)* | RPC options, e.g. retries. | ### *class* `truss_chains.RemoteErrorDetail` -@@ -658,20 +568,13 @@ +@@ -690,20 +582,13 @@ **Parameters:** @@ -686,7 +748,7 @@ #### format() -@@ -680,7 +583,3 @@ +@@ -712,7 +597,3 @@ * **Return type:** str diff --git a/truss-chains/truss_chains/__init__.py b/truss-chains/truss_chains/__init__.py index 3377d4b06..69c1493c8 100644 --- a/truss-chains/truss_chains/__init__.py +++ b/truss-chains/truss_chains/__init__.py @@ -22,6 +22,7 @@ from truss_chains.definitions import ( Assets, BasetenImage, + ChainletOptions, Compute, CustomImage, DeploymentContext, @@ -47,6 +48,7 @@ "Assets", "BasetenImage", "ChainletBase", + "ChainletOptions", "Compute", "CustomImage", "DeploymentContext",