TAN-3812 Authoring assistance prototype #10341

EdwinKato · 2025-02-13T12:45:42Z

No description provided.

…input

cl-dev-bot · 2025-02-13T12:49:39Z

	Warnings
⚠️	The changelog is empty. What should I put in the changelog?
⚠️	The branch name contains no Jira issue key (case-sensitive)

	Messages
📖	Notion issue: TAN-3812
📖	Run the e2e tests
📖	Check translation progress

Generated by 🚫 dangerJS against 8f245c0

…lab into FE-add-authoring

notion-workspace · 2025-02-19T11:39:41Z

✨ Authoring assistance prototype

…lab into FE-add-authoring

sebastienhoorens · 2025-02-20T10:03:46Z

back/app/services/authoring_assistance_service.rb

+  def duplicate_inputs_response(authoring_assistance_response)
+    service = SimilarIdeasService.new(authoring_assistance_response.idea)
+    service.upsert_embedding!
+    threshold = 0.4


We will try to find a way to return only ideas that are "sufficiently" similar next week.

sebastienhoorens · 2025-02-20T10:05:45Z

back/app/services/custom_free_prompt_service.rb

+    return false if body_description.blank?
+
+    html = multiloc_service.t(body_description)
+    Nokogiri::HTML(html).text


I would like to compare if the results are better when stripping HTML or when keeping HTML.

sebastienhoorens · 2025-02-20T10:53:54Z

back/engines/commercial/analysis/app/services/analysis/phase_to_text.rb

+    def execute(phase, **options)
+      super.merge(
+        'Project title' => @multiloc_service.t(phase.project.title_multiloc),
+        'Project description' => Nokogiri::HTML(@multiloc_service.t(phase.project.description_multiloc)).text, # TODO: Deal with content builder layouts


Keeping this as TODO as Simon is currently working on a solution for this (see https://go-vocal.slack.com/archives/C016C2EHURY/p1740047548222469).

sebastienhoorens · 2025-02-20T10:56:48Z

back/spec/acceptance/authoring_assistance_responses_spec.rb

+
+    let(:user) { create(:super_admin) }
+
+    post 'web_api/v1/ideas/:idea_id/authoring_assistance_responses' do


As agreed, I didn't spend time on testing all cases for the prototype. What could have been tested still otherwise:

Non-super-admins are not authorized.

Case when regenerate is true.

Case when creating the first response.

Check if created activity is logged.

CustomFreePromptService.

I think for sanity I think you should just add one other test to ensure that non-super-admins cannot access this endpoint. It's quite important that while it is experimental that there is no back door to it IMO

…nceData interface

sebastienhoorens · 2025-02-21T13:11:43Z

back/app/services/authoring_assistance_service.rb

+  end
+
+  def analyze!
+    duplicate_inputs_thread = start_thread { duplicate_inputs_response(_1) }


I had a chat with Koen around this today. We agreed that this is fine for the prototype, but for the feature we'll expose to the users we need to be careful and should probably consider a polling or pushing mechanism instead of a simple POST request.

sebastienhoorens

All looks good to me on the frontend side. I'm just wondering if we need to add translations to Crowdin.

sebastienhoorens · 2025-02-21T13:47:36Z

front/app/api/authoring_assistance/useAuthoringAssistance.ts

+
+import fetcher from 'utils/cl-react-query/fetcher';
+
+import areaKeys from './keys';


Just checking, this has nothing to do with areas right?

This is just the name you've given here to the default export from keys.ts - maybe cut and paste from somewhere else? Should just be able to rename this to authoringAssistanceKeys - it'll make more sense then!

sebastienhoorens · 2025-02-21T13:59:35Z

front/app/containers/IdeasShow/components/AIInsights/messages.ts

@@ -0,0 +1,30 @@
+// This code is a prototype for input authoring. Clean-up will follow after the prototype phase.
+
+import { defineMessages } from 'react-intl';


As this is a prototype that is only exposed to super admins, I wonder if we should bother translators with this? Could it make sense to just always use this in English? Or is this a very bad idea? 😅

Yeah, think we can merge without getting translations done - once it's on master then the strings will appear in crowdin and will likely get translated anyway. But they don't need to be done before merging. The prompt only works in english anyway - is that right?

It should still return responses if you write the custom free prompt in another language, but perhaps it won't work that well, and the language of the response would be less predictable.

sebastienhoorens · 2025-02-21T14:03:34Z

front/app/containers/IdeasShow/components/AIInsights/index.tsx

+    isLoading,
+  } = useAddAuthoringAssistance();
+  const promptData = authoringResponse?.data;
+  const similarIdeaIds =


I don't fully understand how you can already access this data if the request didn't happen yet?

sebastienhoorens · 2025-02-21T14:14:48Z

back/engines/commercial/analysis/config/prompts/custom_free_prompt.erb

+  <%= input_text %>
+</content>
+
+<%= custom_free_prompt %>


@kogre It would probably be valuable to have your feedback on the prompt as well.

Might be nice to be able to tweak the top prompt A human user...etc through the browser too to see what difference that makes. Maybe later if we're not seeing good results

I like this idea! If we need to do more experimenting, it could be interesting to try this.

jamesspeake

Generally looks good (as a prototype) and is well separated from any impact on the rest of the application so I think we can merge and start trying on production. A few little comments - main one is to add one more test for non-super-admins.

Looking through this generally though, I stand by my comments about trying to bring more of this functionality together centrally for future releases - particularly some of the core methods that currently sits in Analysis and gets pulled into multiple other engines. That'll probably take a bit of co-ordinating with Koen and others though

jamesspeake · 2025-02-25T16:33:14Z

back/engines/commercial/analysis/config/prompts/custom_free_prompt.erb

+  <%= input_text %>
+</content>
+
+<%= custom_free_prompt %>


Might be nice to be able to tweak the top prompt A human user...etc through the browser too to see what difference that makes. Maybe later if we're not seeing good results

jamesspeake · 2025-02-25T16:36:55Z

back/lib/tasks/single_use/20241122_populate_embeddings.rake

Why are you updating a single use rake file? Maybe this file is not single use and should be moved?

Good point. At first, it was indeed intended to be used just once. But now we need it again for the prototype, and perhaps we'll need it again for i1: similar ideas. I'll move it out of single_use and added this as a discussion point for the next weekly meeting.

jamesspeake · 2025-02-25T16:39:05Z

back/spec/acceptance/authoring_assistance_responses_spec.rb

+
+    let(:user) { create(:super_admin) }
+
+    post 'web_api/v1/ideas/:idea_id/authoring_assistance_responses' do


I think for sanity I think you should just add one other test to ensure that non-super-admins cannot access this endpoint. It's quite important that while it is experimental that there is no back door to it IMO

jamesspeake · 2025-02-25T16:43:20Z

front/app/api/authoring_assistance/useAuthoringAssistance.ts

+
+import fetcher from 'utils/cl-react-query/fetcher';
+
+import areaKeys from './keys';


This is just the name you've given here to the default export from keys.ts - maybe cut and paste from somewhere else? Should just be able to rename this to authoringAssistanceKeys - it'll make more sense then!

jamesspeake · 2025-02-25T16:46:07Z

front/app/containers/IdeasShow/components/AIInsights/index.tsx

+
+import messages from './messages';
+
+const AIInsights = ({ ideaId }: { ideaId: string }) => {


I might be tempted to name this AuthoringAssistanePrototype instead so it's quite clear for now

jamesspeake · 2025-02-25T16:48:03Z

front/app/containers/IdeasShow/components/AIInsights/messages.ts

@@ -0,0 +1,30 @@
+// This code is a prototype for input authoring. Clean-up will follow after the prototype phase.
+
+import { defineMessages } from 'react-intl';


Yeah, think we can merge without getting translations done - once it's on master then the strings will appear in crowdin and will likely get translated anyway. But they don't need to be done before merging. The prompt only works in english anyway - is that right?

jamesspeake · 2025-02-25T16:55:28Z

back/app/policies/authoring_assistance_response_policy.rb

@@ -0,0 +1,5 @@
+class AuthoringAssistanceResponsePolicy < ApplicationPolicy
+  def create?
+    user&.active? && user.super_admin?


As a prototype is it also worth checking the feature flag here on the backend, so we know it's only ever invoked on platforms we want it to be? Too much maybe?

sebastienhoorens and others added 14 commits February 10, 2025 19:32

initial authoring assistance feature flag and model

a02dcf1

initial endpoint for authoring assistance responses (untested)

92cd31d

spec for getting responses + fixes

8b0f42a

initial author assistance service (toxicity detection)

97fe195

fixed attributes passing to toxicity detection

32d6b46

fix offense

3905c51

initial version for duplicate inputs in authoring

65b9b3c

add authoring assistance query keys for response handling

d744320

add types for authoring assistance data and response handling

7f371f9

add hook for authoring assistance mutation with query invalidation

adc67b2

add AIInsights component for authoring assistance with custom prompt …

bc6b94f

…input

add messages for AIInsights component

f0ce1ff

add AIInsights component to IdeasShow for enhanced idea presentation

1341590

Translations updated by CI (extract-intl)

6e4c06b

sebastienhoorens and others added 13 commits February 13, 2025 15:01

initial custom free prompt in authoring

f33af54

Merge branch 'TAN-3812-authoring-assistance' into FE-add-authoring

13b4cee

remove unneeded IdeaCard component from AIInsights

c09f307

fix issue with getting custom form

f0e8bf6

fix issue with getting custom form

d96cd5d

cleanup: exists? and default toxicity attributes

b98f80a

spec fix

f58331d

Merge branch 'master' into TAN-3812-authoring-assistance

ec2934a

manual merge

3f67d0f

Merge branch 'FE-add-authoring' of github.com:CitizenLabDotCo/citizen…

15fc42f

…lab into FE-add-authoring

initial version of phase2text for custom free prompt

fefa5a1

fix offenses

ca1392f

Merge branch 'TAN-3812-authoring-assistance' into FE-add-authoring

350e525

sebastienhoorens changed the title ~~Frontend authoring prototype~~ TAN-3812 Authoring assistance prototype Feb 19, 2025

sebastienhoorens added 3 commits February 20, 2025 10:40

fix offenses

4c6fc2d

Merge branch 'FE-add-authoring' of github.com:CitizenLabDotCo/citizen…

7ac5882

…lab into FE-add-authoring

minor changes

11cbd4d

sebastienhoorens reviewed Feb 20, 2025

View reviewed changes

sebastienhoorens and others added 7 commits February 20, 2025 11:58

minor

b3fd1a7

minor

a385483

refactor: rename toxicity_ai_reason to ai_reason in IAuthoringAssista…

93d8e1d

…nceData interface

feat: add toxicity insights to AIInsights component

1f2d2d0

feat: update Title component to use variant prop in AIInsights

2a3e803

Translations updated by CI (extract-intl)

4107660

similarity between projects or not

18d3d69

sebastienhoorens marked this pull request as ready for review February 21, 2025 13:09

sebastienhoorens reviewed Feb 21, 2025

View reviewed changes

Merge branch 'master' into FE-add-authoring

43b9475

sebastienhoorens approved these changes Feb 21, 2025

View reviewed changes

sebastienhoorens requested a review from jamesspeake February 21, 2025 14:13

sebastienhoorens reviewed Feb 21, 2025

View reviewed changes

sebastienhoorens added 6 commits February 21, 2025 17:55

watch out when editing duplicate detection query

fe76aba

watch out when editing duplicate detection query

c2322db

Merge branch 'master' into FE-add-authoring

2378fca

use GPT4o for free text prompt

17c844b

use GPT4o for free text prompt

ddbb19f

feedback oostende: dont show similar ideas of same author

fedf07e

jamesspeake approved these changes Feb 25, 2025

View reviewed changes

sebastienhoorens added 3 commits February 26, 2025 14:39

manual merge

7ee2ebe

requested: added spec for non-super-admins

267b02d

deal with layouts in phase2txt + move embeddings tasks out of single use

8f245c0

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

TAN-3812 Authoring assistance prototype #10341

TAN-3812 Authoring assistance prototype #10341

EdwinKato commented Feb 13, 2025

cl-dev-bot commented Feb 13, 2025 •

edited

Loading

notion-workspace bot commented Feb 19, 2025

sebastienhoorens Feb 20, 2025

sebastienhoorens Feb 20, 2025

sebastienhoorens Feb 20, 2025

sebastienhoorens Feb 20, 2025

jamesspeake Feb 25, 2025

sebastienhoorens Feb 21, 2025

sebastienhoorens left a comment

sebastienhoorens Feb 21, 2025

jamesspeake Feb 25, 2025

sebastienhoorens Feb 21, 2025

jamesspeake Feb 25, 2025

sebastienhoorens Feb 26, 2025 •

edited

Loading

sebastienhoorens Feb 21, 2025

sebastienhoorens Feb 21, 2025

jamesspeake Feb 25, 2025

sebastienhoorens Feb 26, 2025

jamesspeake left a comment

jamesspeake Feb 25, 2025

jamesspeake Feb 25, 2025

sebastienhoorens Feb 26, 2025

jamesspeake Feb 25, 2025

jamesspeake Feb 25, 2025

jamesspeake Feb 25, 2025

jamesspeake Feb 25, 2025

jamesspeake Feb 25, 2025


		let(:user) { create(:super_admin) }

		post 'web_api/v1/ideas/:idea_id/authoring_assistance_responses' do


		import fetcher from 'utils/cl-react-query/fetcher';

		import areaKeys from './keys';

		@@ -0,0 +1,30 @@
		// This code is a prototype for input authoring. Clean-up will follow after the prototype phase.

		import { defineMessages } from 'react-intl';


		import messages from './messages';

		const AIInsights = ({ ideaId }: { ideaId: string }) => {

TAN-3812 Authoring assistance prototype #10341

Are you sure you want to change the base?

TAN-3812 Authoring assistance prototype #10341

Conversation

EdwinKato commented Feb 13, 2025

cl-dev-bot commented Feb 13, 2025 • edited Loading

notion-workspace bot commented Feb 19, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebastienhoorens left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sebastienhoorens Feb 26, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jamesspeake left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cl-dev-bot commented Feb 13, 2025 •

edited

Loading

sebastienhoorens Feb 26, 2025 •

edited

Loading