Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(dashboard): Digest liquid helper and popover handler #7439

Merged
merged 69 commits into from
Feb 3, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
997c1f8
fix(api): Allow arbitrary variables on the payload namespace
SokratisVidros Dec 23, 2024
feaf04c
wup:
scopsy Dec 23, 2024
163ad78
fix: items
scopsy Dec 23, 2024
b2111ec
Update email-subject.tsx
scopsy Dec 23, 2024
65733c8
fix: hello world
scopsy Dec 23, 2024
624383a
fix: re order
scopsy Dec 23, 2024
75d2c7d
fix: state
scopsy Dec 23, 2024
faa410b
fix: refactor field editor
scopsy Dec 23, 2024
96acb70
j
scopsy Dec 23, 2024
80ed1f3
fix: update other field
scopsy Dec 23, 2024
6ec81ab
fix: refactor
scopsy Dec 23, 2024
25141c8
fix: item
scopsy Dec 23, 2024
c32cd7e
fix:
scopsy Dec 23, 2024
2c3e6a9
fix: cursor
scopsy Dec 23, 2024
bf9324a
Update field-editor.tsx
scopsy Dec 23, 2024
fefc2cc
fix: a
scopsy Dec 24, 2024
96ffc1b
Merge branch 'next' into pills-for-all-inputs
scopsy Dec 25, 2024
ed3eaae
Merge branch 'next' into pills-for-all-inputs
scopsy Dec 27, 2024
c6be4ab
Merge branch 'next' into pills-for-all-inputs
scopsy Dec 30, 2024
ec924ed
fix: open prs
scopsy Dec 30, 2024
e717bc8
fix: design
scopsy Dec 30, 2024
e56a5c6
fix:
scopsy Dec 30, 2024
c83af5c
Update variable-popover.tsx
scopsy Dec 30, 2024
93c1d46
fix: it worked
scopsy Dec 30, 2024
50a25c9
fix: refactor
scopsy Dec 30, 2024
74b7ba4
fix: done
scopsy Dec 30, 2024
e78ee36
Merge branch 'next' into pills-for-all-inputs
scopsy Dec 30, 2024
b378aac
fix: add some personality
scopsy Dec 30, 2024
c0f0c3e
fix: view
scopsy Dec 30, 2024
05842f0
Merge branch 'pills-for-all-inputs' of https://github.com/novuhq/novu…
scopsy Dec 30, 2024
48b448c
fix: popover
scopsy Dec 30, 2024
62c48c2
fixes: asdas
scopsy Dec 30, 2024
ecc8a8c
fix: reusability
scopsy Dec 30, 2024
fb0901f
fix: refactor
scopsy Dec 30, 2024
95f249b
feat: add comments
scopsy Dec 30, 2024
92f915f
fix: types
scopsy Dec 30, 2024
f970981
fix: wip for digest liquid
scopsy Dec 30, 2024
f97c433
fix: remove unused
scopsy Dec 30, 2024
44c678a
Merge branch 'next' into pills-for-all-inputs
scopsy Jan 1, 2025
2bd08b0
Update field-editor.tsx
scopsy Jan 1, 2025
a20ee80
Merge branch 'next' into pills-for-all-inputs
scopsy Jan 2, 2025
7209172
fix: close on blur
scopsy Jan 2, 2025
64564ef
fix: refactor
scopsy Jan 3, 2025
d00d441
fix: working state
scopsy Jan 3, 2025
36da57b
fix: auto complete
scopsy Jan 3, 2025
c10c935
fix: liquid
scopsy Jan 3, 2025
d355467
fix: pr comments
scopsy Jan 3, 2025
a148965
fix: initial values
scopsy Jan 3, 2025
71d6d08
Update variable-pill-widget.ts
scopsy Jan 3, 2025
d1d512a
fix: items
scopsy Jan 3, 2025
17164ed
fix: popover
scopsy Jan 3, 2025
bda722a
fix: command refactor
scopsy Jan 3, 2025
cc3b33b
improve usememo
scopsy Jan 3, 2025
a8c1816
fix: revie
scopsy Jan 3, 2025
967be19
fix: minor issues
scopsy Jan 3, 2025
dc7d907
Merge branch 'next' into pills-for-all-inputs
scopsy Jan 3, 2025
3601b0d
Merge branch 'pills-for-all-inputs' into digest-liquid-helper
scopsy Jan 6, 2025
b76fbfa
fix:
scopsy Jan 6, 2025
a88ba03
Update liquid-parser.ts
scopsy Jan 6, 2025
076d365
fix
scopsy Jan 6, 2025
43bbbaa
fix: items
scopsy Jan 6, 2025
eea4a0e
Merge branch 'next' into digest-liquid-helper
scopsy Jan 16, 2025
a0b3f91
fix: remove
scopsy Jan 16, 2025
81d94b4
fix: review
scopsy Jan 17, 2025
a2b8f7a
Merge branch 'next' into digest-liquid-helper
scopsy Jan 19, 2025
e656ad4
Merge branch 'next' into digest-liquid-helper
scopsy Jan 26, 2025
4fdab39
Merge branch 'next' into digest-liquid-helper
scopsy Jan 27, 2025
1045e1e
Merge branch 'next' into digest-liquid-helper
scopsy Feb 3, 2025
650a5ca
Update utils.ts
scopsy Feb 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
import { Template, Liquid, RenderError, LiquidError } from 'liquidjs';
import { isValidTemplate, extractLiquidExpressions } from './parser-utils';
import { Liquid, LiquidError, RenderError, Template } from 'liquidjs';
import { extractLiquidExpressions, isValidTemplate } from './parser-utils';

const LIQUID_CONFIG = {
strictVariables: true,
Expand Down Expand Up @@ -123,10 +123,14 @@ function processLiquidRawOutput(rawOutputs: string[]): TemplateVariables {
}

function parseByLiquid(rawOutput: string): TemplateVariables {
const parserEngine = new Liquid(LIQUID_CONFIG);

// Register digest filter for validation of digest transformers
parserEngine.registerFilter('digest', () => '');

Comment on lines +126 to +129
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently we have 2 version of liquid one in the framework and one in the API, didn't do it in this PR but worth considering consolidating them somehow or exporting from the framework the parser

const validVariables: Variable[] = [];
const invalidVariables: Variable[] = [];
const engine = new Liquid(LIQUID_CONFIG);
const parsed = engine.parse(rawOutput) as unknown as Template[];
const parsed = parserEngine.parse(rawOutput) as unknown as Template[];

parsed.forEach((template: Template) => {
if (isOutputToken(template)) {
Expand Down
2 changes: 1 addition & 1 deletion apps/api/src/app/workflows-v2/util/utils.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
/* eslint-disable no-param-reassign */
import { JSONSchemaDto } from '@novu/shared';
import difference from 'lodash/difference';
import isArray from 'lodash/isArray';
import isObject from 'lodash/isObject';
import reduce from 'lodash/reduce';
import { JSONSchemaDto } from '@novu/shared';
import { MAILY_ITERABLE_MARK } from '../../environments-v1/usecases/output-renderers/maily-to-liquid/maily.types';

export function findMissingKeys(requiredRecord: object, actualRecord: object) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -308,4 +308,17 @@ export const FILTERS: Filters[] = [
example: '"fun%20%26%20games" | url_decode → fun & games',
sampleValue: 'fun%20%26%20games',
},
{
label: 'Digest',
value: 'digest',
hasParam: true,
description: 'Format a list of names with optional key path and separator',
example: 'events | digest: 2, "name", ", " → John, Jane and 3 others',
params: [
{ placeholder: 'Max names to show', type: 'number' },
{ placeholder: 'Object key path (optional)', type: 'string' },
{ placeholder: 'Custom separator (optional)', type: 'string' },
],
sampleValue: '[{ name: "John" }, { name: "Jane" }]',
},
];
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
import { useMemo } from 'react';
import { FILTERS } from '../constants';
import { Filters, FilterWithParam } from '../types';

type SuggestionGroup = {
label: string;
filters: Filters[];
};

export function useSuggestedFilters(variableName: string, currentFilters: FilterWithParam[]): SuggestionGroup[] {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These provide filter suggestion based on common variable names, patterns of data

return useMemo(() => {
const currentFilterValues = new Set(currentFilters.map((f) => f.value));
const suggestedFilters: Filters[] = [];

const addSuggestions = (filterValues: string[]) => {
const newFilters = FILTERS.filter((f) => filterValues.includes(f.value) && !currentFilterValues.has(f.value));

suggestedFilters.push(...newFilters);
};

if (isStepsEventsPattern(variableName)) {
addSuggestions(['digest']);
}

if (isDateVariable(variableName)) {
addSuggestions(['date']);
}

if (isNumberVariable(variableName)) {
addSuggestions(['round', 'floor', 'ceil', 'abs', 'plus', 'minus', 'times', 'divided_by']);
}

if (isArrayVariable(variableName)) {
addSuggestions(['first', 'last', 'join', 'map', 'where', 'size']);
}

if (isTextVariable(variableName)) {
addSuggestions(['upcase', 'downcase', 'capitalize', 'truncate', 'truncatewords']);
}

return suggestedFilters.length > 0 ? [{ label: 'Suggested', filters: suggestedFilters }] : [];
}, [variableName, currentFilters]);
}

function isDateVariable(name: string): boolean {
const datePatterns = ['date', 'time', 'created', 'updated', 'timestamp', 'scheduled'];

return datePatterns.some((pattern) => name.toLowerCase().includes(pattern));
}

function isNumberVariable(name: string): boolean {
const numberPatterns = ['count', 'amount', 'total', 'price', 'quantity', 'number', 'sum', 'age'];

return numberPatterns.some((pattern) => name.toLowerCase().includes(pattern));
}

function isArrayVariable(name: string): boolean {
const arrayPatterns = ['list', 'array', 'items', 'collection', 'set', 'group', 'events'];

return arrayPatterns.some((pattern) => name.toLowerCase().includes(pattern));
}

function isTextVariable(name: string): boolean {
const textPatterns = ['name', 'title', 'description', 'text', 'message', 'content', 'label'];

return textPatterns.some((pattern) => name.toLowerCase().includes(pattern));
}

function isStepsEventsPattern(name: string): boolean {
return /^steps\..*\.events$/.test(name);
}
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
CommandInput,
CommandItem,
CommandList,
CommandSeparator,
} from '@/components/primitives/command';
import { FormControl, FormItem } from '@/components/primitives/form/form';
import { Input } from '@/components/primitives/input';
Expand All @@ -21,8 +22,9 @@ import { FilterItem } from './components/filter-item';
import { FilterPreview } from './components/filter-preview';
import { ReorderFiltersGroup } from './components/reorder-filters-group';
import { useFilterManager } from './hooks/use-filter-manager';
import { useSuggestedFilters } from './hooks/use-suggested-filters';
import { useVariableParser } from './hooks/use-variable-parser';
import type { FilterWithParam, VariablePopoverProps } from './types';
import type { Filters, FilterWithParam, VariablePopoverProps } from './types';
import { formatLiquidVariable, getDefaultSampleValue } from './utils';

export function VariablePopover({ variable, onUpdate }: VariablePopoverProps) {
Expand Down Expand Up @@ -84,6 +86,7 @@ export function VariablePopover({ variable, onUpdate }: VariablePopoverProps) {
onUpdate: setFilters,
});

const suggestedFilters = useSuggestedFilters(name, filters);
const filteredFilters = useMemo(() => getFilteredFilters(searchQuery), [getFilteredFilters, searchQuery]);

const currentLiquidValue = useMemo(
Expand Down Expand Up @@ -173,6 +176,25 @@ export function VariablePopover({ variable, onUpdate }: VariablePopoverProps) {

<CommandList className="max-h-[300px]">
<CommandEmpty>No filters found</CommandEmpty>
{suggestedFilters.length > 0 && !searchQuery && (
<>
<CommandGroup heading="Suggested">
{suggestedFilters[0].filters.map((filterItem: Filters) => (
<CommandItem
key={filterItem.value}
onSelect={() => {
handleFilterToggle(filterItem.value);
setSearchQuery('');
setIsCommandOpen(false);
}}
>
<FilterItem filter={filterItem} />
</CommandItem>
))}
</CommandGroup>
{suggestedFilters.length > 0 && <CommandSeparator />}
</>
)}
{filteredFilters.length > 0 && (
<CommandGroup>
{filteredFilters.map((filter) => (
Expand Down
3 changes: 3 additions & 0 deletions packages/framework/src/client.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { Liquid } from 'liquidjs';
import { digest } from './filters/digest';

import { ChannelStepEnum, PostActionEnum } from './constants';
import {
Expand Down Expand Up @@ -76,9 +77,11 @@ export class Client {
this.apiUrl = builtOpts.apiUrl;
this.secretKey = builtOpts.secretKey;
this.strictAuthentication = builtOpts.strictAuthentication;

this.templateEngine.registerFilter('json', (value, spaces) =>
stringifyDataStructureWithSingleQuotes(value, spaces)
);
this.templateEngine.registerFilter('digest', digest);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@SokratisVidros any name suggestions?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Digest is fine, as it points to the step.

}

private buildOptions(providedOptions?: ClientOptions) {
Expand Down
67 changes: 67 additions & 0 deletions packages/framework/src/filters/digest.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
type NestedObject = Record<string, unknown>;

function getNestedValue(obj: NestedObject, path: string): string {
const value = path.split('.').reduce((current: unknown, key) => {
if (current && typeof current === 'object') {
return (current as Record<string, unknown>)[key];
}

return undefined;
}, obj);

if (value === null || value === undefined) return '';
if (typeof value === 'string') return value;
if (typeof value === 'number' || typeof value === 'boolean') return String(value);
if (typeof value === 'object') {
const stringified = JSON.stringify(value);

return stringified === '{}' ? '' : stringified;
}

return '';
}

/**
* Format a list of items for digest notifications with configurable behavior
* Default formatting:
* - 1 item: "John"
* - 2 items: "John and Josh"
* - 3 items: "John, Josh and Sarah"
* - 4+ items: "John, Josh and 2 others"
*
* @param array The array of items to format
* @param maxNames Maximum names to show before using "others"
* @param keyPath Path to extract from objects (e.g., "name" or "profile.name")
* @param separator Custom separator between names (default: ", ")
* @returns Formatted string
*
* Examples:
* {{ actors | digest }} => "John, Josh and 2 others"
* {{ actors | digest: 2 }} => "John, Josh and 3 others"
* {{ users | digest: 2, "name" }} => For array of {name: string}
* {{ users | digest: 2, "profile.name", "•" }} => "John • Josh and 3 others"
*/
export function digest(array: unknown, maxNames = 2, keyPath?: string, separator = ', '): string {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if that's the best name 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about summarize?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I think it would be better to have they keyPath required? As it might be a common way to receive here an array of objects vs array of strings

if (!Array.isArray(array) || array.length === 0) return '';

const values = keyPath
? array.map((item) => {
if (typeof item !== 'object' || !item) return '';

return getNestedValue(item as NestedObject, keyPath);
})
: array;

if (values.length === 1) return values[0];
if (values.length === 2) return `${values[0]} and ${values[1]}`;

if (values.length === 3 && maxNames >= 3) {
return `${values[0]}, ${separator}${values[1]} and ${values[2]}`;
}

// Use "others" format for 4+ items or when maxNames is less than array length
const shownItems = values.slice(0, maxNames);
const othersCount = values.length - maxNames;

return `${shownItems.join(separator)} and ${othersCount} ${othersCount === 1 ? 'other' : 'others'}`;
}
Loading