Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Control plane support for Extractor regex replace #9124

Merged
merged 82 commits into from
Mar 14, 2024
Merged
Show file tree
Hide file tree
Changes from 80 commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
da9a877
update envoy transformation API w changes as of 55ddd1bfc4d9976bc953c…
ben-taussig-solo Jan 31, 2024
b8e54e4
add new settings to user-facing API
ben-taussig-solo Jan 31, 2024
ac366bd
update transformation plugin + add unit tests
ben-taussig-solo Jan 31, 2024
496f89c
bump envoy-gloo version to pre-release hash
ben-taussig-solo Jan 31, 2024
762589f
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Jan 31, 2024
112fc4f
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 1, 2024
c9c5a57
unit testing updates
ben-taussig-solo Feb 1, 2024
c321292
check regex validity in transformation plugin
ben-taussig-solo Feb 1, 2024
6982323
use table based tests
ben-taussig-solo Feb 1, 2024
153ff62
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 1, 2024
204ff38
add e2e tests
ben-taussig-solo Feb 2, 2024
f216a4e
Merge branch 'extractor_regex_replace' of github.com:solo-io/gloo int…
ben-taussig-solo Feb 2, 2024
315389e
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 2, 2024
97030ab
update comment, add DO_NOT_SUBMIT tag
ben-taussig-solo Feb 2, 2024
7e0acaa
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 2, 2024
e9b9b38
testing updates
ben-taussig-solo Feb 2, 2024
cdfa246
Merge branch 'extractor_regex_replace' of github.com:solo-io/gloo int…
ben-taussig-solo Feb 2, 2024
ce6e920
create error type
ben-taussig-solo Feb 2, 2024
befc7a4
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 2, 2024
4e58fdb
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 2, 2024
cdad29c
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 2, 2024
e3ef571
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 9, 2024
755cd37
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 9, 2024
722356a
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 9, 2024
1a642b7
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 20, 2024
e2972d1
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 21, 2024
d4b5aec
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 23, 2024
63c62f9
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 23, 2024
4195bf3
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 23, 2024
029030c
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 23, 2024
1ac4358
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 26, 2024
d16c4a8
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 26, 2024
9bd9397
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 27, 2024
a001c66
support dynamic_metadata API field
ben-taussig-solo Feb 27, 2024
1602aba
support dynamic_metadata API field in user-facing API
ben-taussig-solo Feb 27, 2024
a4fe385
Revert "support dynamic_metadata API field in user-facing API"
ben-taussig-solo Feb 29, 2024
3fdda2e
Revert "support dynamic_metadata API field"
ben-taussig-solo Feb 29, 2024
7782665
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Feb 29, 2024
facc9f0
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 1, 2024
2262946
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 1, 2024
3502710
update inline documentation of transformation filter to match envoy-g…
ben-taussig-solo Mar 4, 2024
f91cf1d
Merge branch 'extractor_regex_replace' of github.com:solo-io/gloo int…
ben-taussig-solo Mar 4, 2024
54db51a
add changelog entry
ben-taussig-solo Mar 4, 2024
e4300ed
use pre-release ENVOY_GLOO_IMAGE
ben-taussig-solo Mar 4, 2024
40c41a8
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 4, 2024
173a2b5
remove DO_NOT_MERGE, add new test against default mode
ben-taussig-solo Mar 4, 2024
a84d261
Merge branch 'extractor_regex_replace' of github.com:solo-io/gloo int…
ben-taussig-solo Mar 4, 2024
8a774d2
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 4, 2024
8792c39
add documentation for new feature
ben-taussig-solo Mar 4, 2024
c2ffc17
clean up plugin.go todos and commentary
ben-taussig-solo Mar 5, 2024
5031e53
update external API documentation
ben-taussig-solo Mar 5, 2024
4f05f8d
remove remaining TODO in plugin.test.go
ben-taussig-solo Mar 5, 2024
8c72187
unfocus tests in plugin_test.go
ben-taussig-solo Mar 5, 2024
4de96ec
let control plane be dumb about regex semantics; already covered by e…
ben-taussig-solo Mar 7, 2024
d8004fe
update proto documentation for new modes
ben-taussig-solo Mar 7, 2024
0c3b9d9
docs updates
ben-taussig-solo Mar 7, 2024
7780092
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 7, 2024
833414a
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 7, 2024
c6bc647
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 7, 2024
9616094
Adding changelog file to new location
Mar 8, 2024
a0c03c0
Deleting changelog file from old location
Mar 8, 2024
ac7bf5d
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 8, 2024
3df1eb0
bump ENVOY_GLOO_IMAGE to 1.28.1-patch2
ben-taussig-solo Mar 11, 2024
8faa77a
documentation updates
ben-taussig-solo Mar 11, 2024
23dca24
update user guide, .proto inline documentation
ben-taussig-solo Mar 11, 2024
66ec5ec
Merge branch 'extractor_regex_replace' of github.com:solo-io/gloo int…
ben-taussig-solo Mar 11, 2024
3913f6d
remove unintentially committed bananaToOrange test
ben-taussig-solo Mar 11, 2024
7bd8644
resolve typos in .proto files
ben-taussig-solo Mar 11, 2024
559914b
update tests with new defaults, move envoy transformation validation …
ben-taussig-solo Mar 11, 2024
0897766
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 12, 2024
33409d8
Adding changelog file to new location
Mar 12, 2024
0d3722f
Deleting changelog file from old location
Mar 12, 2024
404827a
update envoy-gloo dep to point to new 1.27 release
ben-taussig-solo Mar 13, 2024
659f5b1
Merge branch 'extractor_regex_replace' of github.com:solo-io/gloo int…
ben-taussig-solo Mar 13, 2024
7623116
empty commit to kick build
ben-taussig-solo Mar 13, 2024
1c33947
clean up modeName check when generating extractor errors
ben-taussig-solo Mar 13, 2024
291917c
remove duplicated default extraction mode handler
ben-taussig-solo Mar 13, 2024
0f16510
Merge branch 'extractor_regex_replace' of github.com:solo-io/gloo int…
ben-taussig-solo Mar 13, 2024
7997cd4
validate new default mode handling in new test
ben-taussig-solo Mar 13, 2024
8ae1973
Merge refs/heads/main into extractor_regex_replace
soloio-bulldozer[bot] Mar 13, 2024
8aae66d
update language in .proto file documentation
ben-taussig-solo Mar 13, 2024
78cb1b5
Merge branch 'extractor_regex_replace' of github.com:solo-io/gloo int…
ben-taussig-solo Mar 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ VERSION ?= 1.0.1-dev

SOURCES := $(shell find . -name "*.go" | grep -v test.go)

ENVOY_GLOO_IMAGE ?= quay.io/solo-io/envoy-gloo:1.27.2-patch1
ENVOY_GLOO_IMAGE ?= quay.io/solo-io/envoy-gloo:1.27.3-patch2
LDFLAGS := "-X github.com/solo-io/gloo/pkg/version.Version=$(VERSION)"
GCFLAGS := all="-N -l"

Expand Down
11 changes: 11 additions & 0 deletions changelog/v1.17.0-beta13/extractor-regex-replace.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
changelog:
- type: DEPENDENCY_BUMP
resolvesIssue: false
issueLink: https://github.com/solo-io/gloo/issues/8706
dependencyOwner: envoy-gloo
dependencyRepo: solo-io
dependencyTag: 1.27.3-patch2
description: >-
Upgrade envoy-gloo version to pull in transformation filter updates.
With this bump, transformation filter extractors now support regex
replace/replace all operations on extracted values.
Original file line number Diff line number Diff line change
Expand Up @@ -270,6 +270,76 @@ Extracted values can be used in two ways:
- If `advancedTemplates` is set to `true`, format the string such as `{{ extraction(my-extractor) }}`.
- You can use them in conjunction with the `mergeExtractorsToBody` body transformation type to merge them into the body.

##### Extractor Modes

Starting with Gloo Edge 1.17, extractors support multiple different modes of operation. These modes enable more powerful and flexible extraction capabilities. You can specify the mode of operation for an extractor using the `mode` field. The supported modes are:
- `Extract` (default): Extracts the value of the specified capturing group from the source. This is the standard behavior of extractors prior to this version.
- `Single Replace`: Replaces the value of the n-th capturing group, where n is the value selected by `subgroup`, with the text specified in `replacement_text`.
- `Replace All`: Replaces all occurrences of the pattern specified in the `regex` field within the source with the text specified in `replacement_text`.

{{% notice warning %}}
The semantics of the `regex` field vary by mode and must be carefully considered.
{{% /notice %}}

- `Extract`: The `regex` must match the entire source. Only the specified `subgroup` is extracted.
- `Single Replace`: The `regex` must match the entire source for replacement to occur. The `subgroup` specifies which part of the match to replace.
- `Replace All`: The `regex` can match any part of the source. All matches are replaced, and configuration which specifies a `subgroup` will be rejected.

##### Extractor Configuration Validation

To ensure correct configuration, Gloo Edge performs validation based on the extractor's mode:
- For **Extract** mode, the `replacement_text` must not be set. If set, the configuration will be rejected.
- For **Single Replace** and **Replace All** modes, the `replacement_text` must be set. If not set, the configuration will be rejected.
- For **Replace All** mode, the `subgroup` must not be set, as it is not used. If set, the configuration will be rejected.

##### Extractor Example - Single Replace Mode

Define an extractor that replaces the first occurrence of the item `banana` with `orange` in a list of fruits. The exact composition of the list is unknown, but the format is the same, such as a comma-separated list.
Fruits: `[apple, pear, banana, pineapple, etc.]`
```

```yaml
extractors:
BananaToOrange:
body: {}
mode: SINGLE_REPLACE
replacement_text: 'orange'
regex: '.*(banana).*'
subgroup: 1
```
In this configuration:
- `BananaToOrange` is the name of the extractor.
- `body: {}` indicates that the request/response body will be used as input to the extractor.
- `regex: '.*(banana).*'` is designed to match the entire source, and capture the word `banana` in the first capturing group.
- `mode: SINGLE_REPLACE` specifies the operation mode, indicating a single replace operation.
- `replacement_text: 'orange'` replaces the content in the first capturing group with the text `orange`.
- `subgroup: 1` specifies that the replacement should only apply to the first captured group, which is banana in this case.

In this case, if the extractor processes a body containing the text `Fruits: [apple, pear, banana, pineapple]`

The result will be `Fruits: [apple, pear, orange, pineapple]`

#### Extractor Example - Replace All Mode

Define an extractor that replaces all occurrences of the pattern `foo` with the text `bar` in the source.
```yaml
extractors:
FooToBar:
body: {}
regex: 'foo'
mode: REPLACE_ALL
replacement_text: 'bar'
```

In this configuration:
- `FooToBar` is the name of the extractor.
- `body: {}` indicates that the request/response body will be used as input to the extractor.
- `regex: 'foo'` specifies the pattern to match.
- `mode: REPLACE_ALL` specifies the mode of operation.
- `replacement_text: 'bar'` specifies the text to replace the matched pattern with.

In this case, if the extractor processes a body containing the text `foo foo foo`, the result will be `bar bar bar`.

##### headers
Use this attribute to apply templates to request/response headers. It consists of a map where each key determines the name of the resulting header, while the corresponding value is a {{< protobuf display="template" name="envoy.api.v2.filter.http.InjaTemplate" >}} which will determine the value.

Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading
Loading