Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: prepare sdf-beta 6 #332

Merged
merged 4 commits into from
Jan 17, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdf/SDF_VERSION
Original file line number Diff line number Diff line change
@@ -1 +1 @@
sdf-beta5
sdf-beta6
2 changes: 1 addition & 1 deletion sdf/_embeds/install-sdf.bash
Original file line number Diff line number Diff line change
@@ -1 +1 @@
fvm install sdf-beta5
fvm install sdf-beta6
3 changes: 2 additions & 1 deletion sdf/cli/build.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,13 @@ Usage: sdf build [OPTIONS]

Options:
--dev set runtime to use dev mode [env: DEV=]
--build-profile <BUILD_PROFILE> set the build profile [default: release]
-h, --help Print help
```

Where:
* `--dev` development profile to be used for runtime mode

* `--build-profile` - sets the build profile [debug, optimized, release]
#### Examples

##### sdf build
Expand Down
3 changes: 1 addition & 2 deletions sdf/cli/run.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ Where:
* `--port` sets the port for the web server to listen on
* `--env` sets environment variables to be passed to operators
* `--skip-running` - compiles components and exists without running the dataflow
* `--build-profile` - sets the build profile
* `--build-profile` - sets the build profile [debug, optimized, release]
* `--prod` - sets runtime to apply prod specific parameters
* `--force-update` - forces the update of the project dependencies

Expand Down Expand Up @@ -130,5 +130,4 @@ Show the detailed information:

Please see the section of [sql mode].


[sql mode]: ./run_sql.mdx
10 changes: 8 additions & 2 deletions sdf/cli/test.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,12 @@ Commands:

Options:
--dev set runtime to use dev mode [env: DEV=]
--build-profile <BUILD_PROFILE> set build profile [default: release]
```

Where:
* `--dev` - development profile to be used for building code
* `--build-profile` - sets the build profile [debug, optimized, release]


#### Example
Expand All @@ -102,16 +104,20 @@ Arguments:

Options:
-v, --value <VALUE> Record value mapped to the `value` input of the function
--value-file <VALUE_FILE> record value from file
-k, --key <KEY> Record key mapped to the `key` input of the function
--key-file <KEY_FILE> key value from file
-s, --state-key <STATE_KEY> The key of the state the function runner uses to look-up
state object
-o, --output <OUTPUT> Output types, where v: is value (default), and kv: is
key-value [default: v]
```

Where:
* `--value` maps ot the input parameter of the function
* `--key` maps to the `optional` input parameter of the function
* `--value` maps to the input parameter of the function
* `--value-file` maps a file content to the input parameter of the function
* `--key` maps to the `optional` input key parameter of the function
* `--key-file` maps a file content to the `optional` input key parameter of the function
* `--state-key` asks by the engine to look-up a state
* `--output` ask the engine to show values-only or key-values

Expand Down
1 change: 1 addition & 0 deletions sdf/composition/overview.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,7 @@ functions:
type: <input-type>
output:
type: <output-type>
optional: <boolean, default to false>

dev:
converter: <converter-props>
Expand Down
2 changes: 1 addition & 1 deletion sdf/concepts/operators.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,7 @@ window:
top_words.push(word_count);
}
},
_ => return Err(anyhow::anyhow!("unexpected schema")),
_ => return Err(sdfg::anyhow::anyhow!("unexpected schema")),
}
Ok(top_words)
}
Expand Down
23 changes: 8 additions & 15 deletions sdf/whatsnew.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ sidebar_position: 20
import CodeBlock from '@theme/CodeBlock';
import InstallFvm from '!!raw-loader!./_embeds/install-sdf.bash';

# Beta 5
# Beta 6

## Upgrading

To upgrade CLI to the beta4, run the following command:
To upgrade CLI to this version, run the following command:

<CodeBlock language="bash">{InstallFvm}</CodeBlock>

Expand All @@ -24,27 +24,20 @@ $ sdf worker create <host-worker-name>

For upgrading cloud workers, please contact [InfinyOn support](#infinyon-support).

## Featured change

- Added [sql mode] to interactive shell. With this change, user should be able to run SQL queries (including JOINS) in states of dataflow. The states that support the queries are the [dataframe states]. In particular, when the state has a window context, the queries are againts the last flushed state.

### CLI changes

- `sdf run` not longer accepts `--dev`. Development mode is now the default for `sdf run`. If you want to run in non-development mode use `--prod`.
- Added --build-profile to sdf build and sdf test subcommands.
- Added ability to pass file as input for `sdf test`.

### Improvements

- Added capability to run complex queries like join on states in operator context through the [sql function].
- Performance improvements.
- Improved error messages when nested types definitions are wrong.
- Allow publishing packages with no functions.
- Remove the need of saving the WIT generated files in SDF packages.

### Changes
- Replaced dashes in tables. Previously, when the state name has dashes in it, we were escaping the state name in sql context with quotes. From sdf-beta5, we should access them using `_` instead of `-` on the table name in order to avoid the escaping.
- Added validation to check that output of filter-map operators are optional.
- Removed `anyhow` dependencies from operators rust code, it is reexported now in the `sdfg` crate, use with `sdfg::anyhow`.

## InfinyOn Support

For any questions or issues, please contact InfinyOn support at [email protected] or https://discordapp.com/invite/bBG2dTz

[sql mode]: cli/run.mdx#sql-mode
[sql function]: concepts/sql.mdx#sql-function
[dataframe states]: concepts/sql.mdx#dataframe-states
2 changes: 1 addition & 1 deletion sdf_versioned_docs/version-sdf-beta5/whatsnew.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import InstallFvm from '!!raw-loader!./_embeds/install-sdf.bash';

## Upgrading

To upgrade CLI to the beta4, run the following command:
To upgrade CLI to the beta5, run the following command:

<CodeBlock language="bash">{InstallFvm}</CodeBlock>

Expand Down
Loading