Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[EDU-474] Studios doc audit #407

Open
wants to merge 37 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
f02d68b
[EDU-474] Data Studios doc audit
justinegeffen Jan 29, 2025
5682462
Merge branch 'master' into justinegeffen-edu-474
justinegeffen Jan 31, 2025
9e9aff5
Audit
justinegeffen Feb 10, 2025
f93d4a1
Merge branch 'master' into justinegeffen-edu-474
justinegeffen Feb 14, 2025
5a6bf87
Merge branch 'master' into justinegeffen-edu-474
justinegeffen Feb 14, 2025
369ba0b
Merge branch 'justinegeffen-edu-474' of https://github.com/seqeralabs…
justinegeffen Feb 14, 2025
e8b77de
Merge branch 'master' into justinegeffen-edu-474
justinegeffen Feb 17, 2025
7b771af
Data Studios > Studios plus redirects
justinegeffen Feb 17, 2025
5350744
Renaming images
justinegeffen Feb 17, 2025
b163577
Minor fixes
justinegeffen Feb 17, 2025
e94f72e
Studios
justinegeffen Feb 17, 2025
657e5c8
Changes
justinegeffen Feb 17, 2025
93dc6e4
revert two changes
justinegeffen Feb 17, 2025
87336d1
Two fixes
justinegeffen Feb 17, 2025
5ae9dd4
.
justinegeffen Feb 17, 2025
6fc7c46
Troubleshooting updates
justinegeffen Feb 18, 2025
8f87123
build fix
justinegeffen Feb 18, 2025
6ffafea
Updated enterprise deployment docs
justinegeffen Feb 18, 2025
97fa4ed
Merge branch 'master' into justinegeffen-edu-474
justinegeffen Feb 19, 2025
6d20aaf
Fixing broken internal links.
justinegeffen Feb 19, 2025
0dbcae7
Fixed broken internal links
justinegeffen Feb 19, 2025
e1eca08
broken internal link
justinegeffen Feb 19, 2025
c0038cc
Fixed homepage links
justinegeffen Feb 19, 2025
4510235
Additional naming changes and fixes
justinegeffen Feb 19, 2025
fde0a69
Fixed redirects
justinegeffen Feb 19, 2025
ab6d148
Redirect fixes
justinegeffen Feb 19, 2025
faeffb7
Redirects
justinegeffen Feb 19, 2025
fa0e035
Edits
justinegeffen Feb 19, 2025
c4025e6
Rewording
justinegeffen Feb 19, 2025
e6c3d25
Rewording
justinegeffen Feb 19, 2025
efa819c
Apply suggestions from code review
justinegeffen Feb 25, 2025
8b23fef
Update platform_versioned_docs/version-24.3/troubleshooting_and_faqs/…
justinegeffen Feb 25, 2025
f363fe9
update branch and fix conflicts
justinegeffen Feb 25, 2025
3a81fc7
Additional renaming and fixing
justinegeffen Feb 25, 2025
58430b1
Fixed broken internal links
justinegeffen Feb 25, 2025
962abb9
broken link
justinegeffen Feb 25, 2025
37e562d
Broken link
justinegeffen Feb 25, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 8 additions & 10 deletions platform_versioned_docs/version-24.2/data_studios/custom-envs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ tags: [data, studios]

In addition to the Seqera-provided container template images, you can provide your own custom container environments by augmenting the Seqera-provided images with a list of Conda packages or by providing your own base container template image.

Data Studios uses the [Wave][wave-home] service to build custom container template images.
Studios use the [Wave][wave-home] service to build custom container template images.

## Conda packages

Expand All @@ -21,7 +21,7 @@ Data Studios uses the [Wave][wave-home] service to build custom container templa

### Conda package syntax {#conda-package-syntax}

When adding a new data studio session, you can customize its configuration to install a list of Conda packages to the template image. The supported schema is identical to that used by the Conda `environment.yml` file. For more information on the Conda environment file, see [Creating an environment file manually][env-manually].
When adding a new Studio, you can customize its configuration to install a list of Conda packages to the template image. The supported schema is identical to that used by the Conda `environment.yml` file. For more information on the Conda environment file, see [Creating an environment file manually][env-manually].

```yaml title="Example environment.yml file"
channels:
Expand All @@ -33,7 +33,7 @@ dependencies:
- seaborn
```

To create a data studio with custom Conda packages, see [Add a data studio][add-ds].
To create a Studio with custom Conda packages, see [Add a Studio][add-s].

## Custom container template image {#custom-containers}

Expand All @@ -45,14 +45,12 @@ Public container registries are supported by default. Amazon Elastic Container R

### Prerequisites

- Access to a container image repository, either a public container registry or a private Amazon ECR repository
- A container template image
- Access to a container image repository, either a public container registry or a private Amazon ECR repository.
- A container template image.

### Dockerfile configuration {#dockerfile}

For your custom template container image, you must use a Seqera-provided base image and include several additional build steps for compatibility with Data Studios.

To create a data studio with a custom template image, see [Add a data studio][add-ds].
For your custom template container image, you must use a Seqera-provided base image and include several additional build steps for compatibility with Studios. To create a Studio with a custom template image, see [Add a Studio][add-ds].

#### Ports

Expand Down Expand Up @@ -109,14 +107,14 @@ If the custom container template image build fails, the data studio session has

To inspect the status of an ongoing build, or a successful or failed build, complete the following steps:

1. Select the **Data Studios** tab in Seqera Platform.
1. Select the **Studios** tab in Seqera Platform.
1. From the list of data studio sessions, select the name of the session with **building** or **build-failed** status that you want to inspect, and then select **View**.
1. In the **Details** tab, scroll to **Build reports** and select **Summary** to open the Wave service container build report for your build.
1. Optional: If the build failed, select the **Error report** tab to view the errors associated with the build failure.


<!-- links -->
[add-ds]: ./managing.mdx#add-a-data-studio
[add-s]: ./managing.mdx#add-a-studio
[wave]: ../enterprise/configuration/wave.mdx

[wave-home]: https://seqera.io/wave/
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Studios is a unified platform where you can host a combination of container imag
On Seqera Cloud, the free tier permits only one running session at a time. To run simultaneous sessions, [contact Seqera][contact] for a Seqera Cloud Pro license.

:::note
Data Studios is currently in **public preview** and is available from Seqera Platform v24.1. [Contact Seqera support](https://support.seqera.io) if you experience any problems during the deployment process. Data Studios in Enterprise is not enabled by default. You can enable Data Studios in the [environment variables configuration](../enterprise/data-studios.mdx).
Studios is currently in **public preview** and is available from Seqera Platform v24.1. [Contact Seqera support](https://support.seqera.io) if you experience any problems during the deployment process. Studios in Enterprise is not enabled by default. You can enable it in the [environment variables configuration](../enterprise/studios.mdx).
:::

## Requirements
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ The DB or Redis volume is persistent after a Docker restart by default. Use the

- To configure the Seqera pipeline resource optimization service (`groundswell`), see [Pipeline resource optimization](./configuration/pipeline_optimization.mdx).

- To deploy with Data Studios, see [Data studios deployment](../enterprise/data-studios.mdx).
- To deploy with Studios, see [Data studios deployment](../enterprise/studios.mdx).

4. Deploy the application and wait for it to initialize (this process takes a few minutes):

Expand All @@ -43,10 +43,10 @@ For more information on configuration, see [Configuration options](./configurati

## Optional features

### Data Studios
### Studios

[Data Studios](../data_studios/index.mdx) is an interactive analysis environment available in organizational workspaces. To enable Data Studios, see [Data Studios deployment](./data-studios.mdx).
[Studios](../data_studios/index.mdx) is an interactive analysis environment available in organizational workspaces. To enable Studios, see [Studios deployment](./studios.mdx).

:::note
Data Studios is currently in **public preview** and is available from Seqera Platform v24.1. If you experience any problems during the deployment process please contact your account executive. Data Studios in Enterprise is not installed by default.
Studios is currently in **public preview** and is available from Seqera Platform v24.1. If you experience any problems during the deployment process please contact your account executive. Data Studios in Enterprise is not installed by default.
:::
10 changes: 5 additions & 5 deletions platform_versioned_docs/version-24.2/enterprise/kubernetes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@ Seqera Enterprise is distributed as a collection of Docker containers available
```bash
kubectl create secret docker-registry cr.seqera.io \
--docker-server=cr.seqera.io \
--docker-username='<YOUR NAME>' \
--docker-password='<YOUR SECRET>'
--docker-username='<DOCKER_USER>' \
--docker-password='<DOCKER_PASSWORD>'
```

The credential `name` contains a dollar `$` character. Wrap the name in single quotes to prevent the Linux shell from interpreting this value as an environment variable.
Expand Down Expand Up @@ -250,12 +250,12 @@ Define a set of credentials for the resource optimization database in the `tower

The initContainers will wait until both the Seqera and pipeline optimization service databases are ready before starting the migration in the Seqera database and finally starting the resource optimization container.

### Data Studios
### Studios

[Data Studios](../data_studios/index.mdx) is an interactive analysis environment available in organizational workspaces. To enable Data Studios, see [Data Studios deployment](./data-studios.mdx).
[Data Studios](../data_studios/index.mdx) is an interactive analysis environment available in organizational workspaces. To enable Data Studios, see [Studios deployment](./studios.mdx).

:::note
Data Studios is currently in **public preview** and is available from Seqera Platform v24.1. If you experience any problems during the deployment process [contact Seqera support](https://support.seqera.io). Data Studios in Enterprise is not installed by default.
Studios is currently in **public preview** and is available from Seqera Platform v24.1. If you experience any problems during the deployment process [contact Seqera support](https://support.seqera.io). Studios in Enterprise is not installed by default.
:::

### Database console
Expand Down
Original file line number Diff line number Diff line change
@@ -1,39 +1,39 @@
---
title: "Data Studios deployment"
title: "Studios deployment"
description: Deploy Seqera Platform with Data studios
date: "12 Jun 2024"
tags: [docker, compose, kubernetes, data, studios, deployment]
---

Enable Data Studios as part of your Seqera Platform instance. You must have Data Explorer enabled to use Data Studios. Only the Amazon AWS public cloud is currently supported.
Enable Studios as part of your Seqera Platform instance. You must have Data Explorer enabled to use Data Studios. Only the Amazon AWS public cloud is currently supported.

:::caution
You must complete the upgrade of your Platform Enterprise installation to version 24.1.4 before enabling and configuring Data Studios.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You must complete the upgrade of your Platform Enterprise installation to version 24.1.4 before enabling and configuring Data Studios.
You must complete the upgrade of your Platform Enterprise installation to version 24.2 before enabling and configuring Studios.

Let's start requiring to upgrade to v24.2 so we start collecting telemetry data.

:::

## DNS configuration

Each data studio is reachable at a unique URL that includes a randomly generated subdomain name. For example: `https://abcd.example.com/`, where `example.com` is your Seqera base domain name.
Each Studio is reachable at a unique URL that includes a randomly generated subdomain name. For example: `https://abcd.example.com/`, where `example.com` is your Seqera base domain name.

Provide a wildcard TLS certificate to allow for uniquely generated subdomains. A wildcard certificate common name includes `*.` in the domain name, such as `*.example.com`, thereby securing any subdomain name at this level.

Data Studios uses the following set of domains and subdomains:
Studios uses the following set of domains and subdomains:

- The domain that you set for `TOWER_SERVER_URL`, such as `example.com`.
- A wildcard subdomain that you must configure specifically for Data Studios. This wildcard subdomain is the parent for each unique data studios session URL, such as `abcd.example.com`.
- A wildcard subdomain that you must configure specifically for Studios. This wildcard subdomain is the parent for each unique Studios session URL, such as `abcd.example.com`.
- The connection proxy, defined by `CONNECT_PROXY_URL`. This URL is a first-level subdomain of your `TOWER_SERVER_URL`. For example, `https://connect.example.com`.

## Data Studios workspace availability
## Studios workspace availability

You can configure which organizational workspaces have access to Data Studios. This configuration is set in the `tower.yml` file. The `tower.data-studio.allowed-workspaces` field supports the following options:
You can configure which organizational workspaces have access to Studios. This configuration is set in the `tower.yml` file. The `tower.data-studio.allowed-workspaces` field supports the following options:

- `allowed-workspaces: []`: Disables Data Studios. This is the default if the `allowed-workspaces` field is not specified.
- `allowed-workspaces: [ <WORKSPACE_ID>,<WORKSPACE_ID> ]`: Enables Data Studios for the comma-separated list of organizational workspace IDs.
- `allowed-workspaces: null`: Enables Data Studios for all organizational workspaces.
- `allowed-workspaces: []`: Disables Studios. This is the default if the `allowed-workspaces` field is not specified.
- `allowed-workspaces: [ <WORKSPACE_ID>,<WORKSPACE_ID> ]`: Enables Studios for the comma-separated list of organizational workspace IDs.
- `allowed-workspaces: null`: Enables Studios for all organizational workspaces.

## Available data studio environment images
## Available Studio environment images

Each of the provided Data Studios environments includes a particular version of the underlying data analytics software package.
Each of the provided Studios environments includes a particular version of the underlying data analytics software package.

To quickly identify which version of the software an image includes, the version string for each container is in the form of `<software_version>-<seqera_version>`. For example, if the version string for the RStudio Server is `4.4.1-0.7.4`, version `4.4.1` is the RStudio version and `0.7.4` is the version of this Seqera-built container image.

Expand All @@ -56,7 +56,7 @@ This guide assumes that all services will be run in the same container as the re

### Procedure

1. Download the Data Studios [environment configuration file](./_templates/docker/data-studios.env).
1. Download the Studios [environment configuration file](./_templates/docker/data-studios.env).

1. Create an initial OIDC registration token, which can be any secure random string. For example, using openssl:

Expand Down Expand Up @@ -92,12 +92,12 @@ This guide assumes that all services will be run in the same container as the re

1. Open `tower.env` in an editor and set the following variables:

- `TOWER_DATA_EXPLORER_ENABLED`: Set `true` to enable Data Explorer. You must enable Data Explorer to mount data inside a data studio instance.
- `TOWER_DATA_STUDIO_CONNECT_URL`: The URL of the Data Studios connect proxy, such as `https://connect.example.com/`.
- `TOWER_DATA_EXPLORER_ENABLED`: Set `true` to enable Data Explorer. You must enable Data Explorer to mount data inside a Studio.
- `TOWER_DATA_STUDIO_CONNECT_URL`: The URL of the Studios connect proxy, such as `https://connect.example.com/`.
- `TOWER_OIDC_REGISTRATION_INITIAL_ACCESS_TOKEN`: The same value set in the `oidc_registration_token` environment variable.
- `TOWER_OIDC_PEM_PATH`: The file path to a PEM certificate used for signing the OIDC tokens for the OpenID connect provider, mounted as a volume inside the container.

1. Edit the `tower.yml` file and include the following snippet to enable Data Studios in all organization workspaces:
1. Edit the `tower.yml` file and include the following snippet to enable Studios in all organization workspaces:

```yaml
tower:
Expand All @@ -113,15 +113,15 @@ This guide assumes that all services will be run in the same container as the re
docker ps
```

1. To confirm that Data Studios is available, log in to your Platform instance and navigate to an organizational workspace that has Data Studios enabled. The **Data Studios** tab is included with the available tabs.
1. To confirm that Studios is available, log in to your Platform instance and navigate to an organizational workspace that has Studios enabled. The **Studios** tab is included with the available tabs.

## Kubernetes

This procedure describes how to configure Data Studios for Kubernetes deployments of Seqera Platform.
This procedure describes how to configure Studios for Kubernetes deployments of Seqera Platform.

### Procedure

1. Download the Kubernetes manifests for the Data Studios service:
1. Download the Kubernetes manifests for the Studios service:

- [Proxy](./_templates/k8s/data_studios/proxy.yml)
- [Server](./_templates/k8s/data_studios/server.yml)
Expand All @@ -143,7 +143,7 @@ This procedure describes how to configure Data Studios for Kubernetes deployment
1. Edit the `proxy.yml` file and set the following variables:

- `CONNECT_REDIS_ADDRESS`: The hostname or IP address of the Redis server configured for Platform.
- `CONNECT_PROXY_URL`: A URL for the connect proxy subdomain. We recommend you set a first-level subdomain of your Platform installation domain (`PLATFORM_URL` below) for your connect proxy, to be able to use the same wildcard TLS certificate for all data studio instance URLs and avoid additional domain nesting. For example, `https://connect.example.com`.
- `CONNECT_PROXY_URL`: A URL for the connect proxy subdomain. We recommend you set a first-level subdomain of your Platform installation domain (`PLATFORM_URL` below) for your connect proxy, to be able to use the same wildcard TLS certificate for all Studio session URLs and avoid additional domain nesting. For example, `https://connect.example.com`.
- `PLATFORM_URL`: The base URL for your Platform installation, such as `https://example.com/`.
- `CONNECT_OIDC_CLIENT_REGISTRATION_TOKEN`: The same value as the `oidc_registration_token` value created previously.

Expand Down Expand Up @@ -174,7 +174,7 @@ This procedure describes how to configure Data Studios for Kubernetes deployment
base64_pem=$(cat data-studios-rsa.pem | base64)
```

1. Create a Secret file named `secret.yml` and set the `oidc.pem` key by pasting the contents of the base64-encoded public/private key pair:
1. Create a secret file named `secret.yml` and set the `oidc.pem` key by pasting the contents of the base64-encoded public/private key pair:

```yaml
apiVersion: v1
Expand All @@ -186,17 +186,17 @@ This procedure describes how to configure Data Studios for Kubernetes deployment
oidc.pem: <BASE64_ENCODED_PEM_FILE>
```

1. Create the Secret:
1. Create the secret:

```
kubectl apply -f secret.yml
```

1. Edit the `tower-svc.yml` file and uncomment the `volumes.cert-volume`, `volumeMounts.cert-volume`, and `env.TOWER_OIDC_PEM_PATH` fields so that the public/private key pair is available to Platform.
1. Edit the `tower-src.yml` file and uncomment the `volumes.cert-volume`, `volumeMounts.cert-volume`, and `env.TOWER_OIDC_PEM_PATH` fields so that the public/private key pair is available to Platform.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@endre-seqera, this relates to https://seqera.atlassian.net/browse/EDU-491. Please could you review and confirm it's the correct fix?


1. Edit the ConfigMap named `platform-backend-cfg` in the `configmap.yml` for Platform by editing the following environment variables:

- `TOWER_DATA_STUDIO_CONNECT_URL`: The URL of the Data Studios connect proxy, such as `https://connect.example.com/`.
- `TOWER_DATA_STUDIO_CONNECT_URL`: The URL of the Studios connect proxy, such as `https://connect.example.com/`.
- `TOWER_OIDC_REGISTRATION_INITIAL_ACCESS_TOKEN`: The same value as the `oidc_registration_token` value created previously.

1. Edit the ConfigMap named `tower-yml` in the `configmap.yml` and include the following snippet:
Expand Down Expand Up @@ -236,12 +236,12 @@ This procedure describes how to configure Data Studios for Kubernetes deployment
kubectl scale --replicas=1 deployment/backend
```

1. Apply the Data Studios manifests:
1. Apply the Studios manifests:

```
kubectl apply -f ingress.aks.yml proxy.yml server.yml
```

It can take several minutes for Kubernetes to apply your changes, during which new pods are rolled out.

1. To confirm that Data Studios is available, log in to your Platform instance and navigate to an organizational workspace that has Data Studios enabled. The **Data Studios** tab is included with the available tabs.
1. To confirm that Studios is available, log in to your Platform instance and navigate to an organizational workspace that has Studios enabled. The **Studios** tab is included with the available tabs.
6 changes: 3 additions & 3 deletions platform_versioned_docs/version-24.3/cli/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ https://api.cloud.seqera.io/workspaces/97652229034xxx/datasets/6vBGj6aWWpBuLpGKj

Run `tw studios -h` to view the list of supported operations.

Manage [Studio sessions](../data_studios/index.mdx) hosted in Seqera Platform. Studio sessions allow interactive analysis using Jupyter, RStudio, VSCode, and Xpra. Additional custom analysis environments can be defined as needed.
Manage [Studio sessions](../studios/index.mdx) hosted in Seqera Platform. Studio sessions allow interactive analysis using Jupyter, RStudio, VSCode, and Xpra. Additional custom analysis environments can be defined as needed.

:::note
Most Studio operations require workspace `MAINTAIN` permissions.
Expand Down Expand Up @@ -329,7 +329,7 @@ Build reports | NA

Run `tw studios templates -h` to view the required and optional fields for session templates.

Templates are the container image templates used as the foundation for Studio sessions. See [Container image templates](../data_studios/index.mdx#container-image-templates) for more information.
Templates are the container image templates used as the foundation for Studio sessions. See [Container image templates](../studios/index.mdx#container-image-templates) for more information.

```bash
tw studios templates -w community/showcase
Expand Down Expand Up @@ -391,7 +391,7 @@ tw studios stop -i 13083356 -w community/showcase

Run `tw studios checkpoints -h` to view the required and optional fields for viewing checkpoints for a session.

List all checkpoints for an existing Studio session in a workspace. See [Session checkpoints](../data_studios/index.mdx#session-checkpoints) for more information.
List all checkpoints for an existing Studio session in a workspace. See [Session checkpoints](../studios/index.mdx#session-checkpoints) for more information.

```bash
tw studios checkpoints -i 19a3abbd -w community/showcase
Expand Down
Loading