Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TAN-3873] Support new sentiment_linear_scale custom field #10357

Open
wants to merge 57 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
6b5a034
WIP support new sentiment_linear_scale custom field type in BE
amanda-anderson Feb 17, 2025
dc48834
Fix lint
amanda-anderson Feb 18, 2025
915a13d
Merge branch 'master' into TAN-3873-be-new-sentiment-linear-scale-field
amanda-anderson Feb 18, 2025
0ab43bd
Add spec and remove ToDo comments
amanda-anderson Feb 18, 2025
9ff7fad
Add missing BE specs for sentiment_linear_scale
amanda-anderson Feb 18, 2025
7379635
Include ask_follow_up in json schema response and spens
amanda-anderson Feb 18, 2025
f9d6bd8
Update specs and add missing support for ask_follow_up attribute
amanda-anderson Feb 18, 2025
33a0049
Fix lint issues
amanda-anderson Feb 18, 2025
15d480e
WIP initial support for adding new sentiment question to form builder…
amanda-anderson Feb 20, 2025
e9fbe06
Tweaks to BE code
amanda-anderson Feb 20, 2025
b62aeca
Translations updated by CI (extract-intl)
Feb 20, 2025
19d433e
Simplify serializer for linear scale labels
amanda-anderson Feb 20, 2025
bd29e33
Fix syntax
amanda-anderson Feb 20, 2025
7e5b1e1
Update specs
amanda-anderson Feb 20, 2025
6f8f344
Fix failing spec
amanda-anderson Feb 20, 2025
b38e70a
Fix spec
amanda-anderson Feb 20, 2025
128f4be
Merge branch 'master' into TAN-3873-be-new-sentiment-linear-scale-field
amanda-anderson Feb 20, 2025
c0909b3
Include sentiment_linear_scale in importing list
amanda-anderson Feb 20, 2025
338c988
Do not allow sentiment question in pdf print
amanda-anderson Feb 20, 2025
706810e
Merge branch 'TAN-3873-be-new-sentiment-linear-scale-field' into TAN-…
amanda-anderson Feb 20, 2025
7b46599
Merge branch 'TAN-3874-support-sentiment-question-form-builder' of gi…
amanda-anderson Feb 20, 2025
40fba16
Support sentiment question in form builder
amanda-anderson Feb 20, 2025
2f0e19e
Fix comment
amanda-anderson Feb 20, 2025
54b3466
Translations updated by CI (extract-intl)
Feb 20, 2025
69d5d29
WIP Initial support for the follow_up custom field for sentiment ques…
amanda-anderson Feb 21, 2025
d5d05a4
Merge branch 'TAN-3873-be-new-sentiment-linear-scale-field' into TAN-…
amanda-anderson Feb 21, 2025
9208b15
Merge branch 'TAN-3874-support-sentiment-question-form-builder' of gi…
amanda-anderson Feb 21, 2025
6a4e283
WIP Support sentiment linear scale in front office
amanda-anderson Feb 24, 2025
7e932e1
Make follow up field a text area and use correct label value
amanda-anderson Feb 24, 2025
5e750eb
Fix from review
amanda-anderson Feb 24, 2025
95126f3
Merge branch 'TAN-3873-be-new-sentiment-linear-scale-field' into TAN-…
amanda-anderson Feb 24, 2025
c3d65a8
Merge branch 'TAN-3874-support-sentiment-question-form-builder' into …
amanda-anderson Feb 24, 2025
9caae48
Fix rubocop issue
amanda-anderson Feb 24, 2025
79b4ed3
Merge branch 'TAN-3873-be-new-sentiment-linear-scale-field' into TAN-…
amanda-anderson Feb 24, 2025
c5fe819
Merge branch 'TAN-3874-support-sentiment-question-form-builder' into …
amanda-anderson Feb 24, 2025
10676b6
Improve accessibility
amanda-anderson Feb 24, 2025
8e644bf
Code cleanup
amanda-anderson Feb 24, 2025
9dbf8b7
Code cleanup
amanda-anderson Feb 24, 2025
723eabd
Remove part of comment
amanda-anderson Feb 24, 2025
01903be
Basic support for initial results view
amanda-anderson Feb 24, 2025
7a4c911
Use correct initial label values for sentiment_linear_scale fields
amanda-anderson Feb 24, 2025
ebd9854
Translations updated by CI (extract-intl)
Feb 24, 2025
e7e8e9a
Fixes from code review
amanda-anderson Feb 26, 2025
219a53d
Fixes from code review
amanda-anderson Feb 26, 2025
74a3880
Fix lint issue
amanda-anderson Feb 26, 2025
2358f29
Merge branch 'TAN-3873-be-new-sentiment-linear-scale-field' into TAN-…
amanda-anderson Feb 26, 2025
6915aca
Merge branch 'TAN-3874-support-sentiment-question-form-builder' of gi…
amanda-anderson Feb 26, 2025
0df400e
Code cleanup from code review
amanda-anderson Feb 26, 2025
cd19746
Merge branch 'TAN-3874-support-sentiment-question-form-builder' into …
amanda-anderson Feb 26, 2025
d48af9e
Code cleanup and tests after code review
amanda-anderson Feb 26, 2025
235ad96
Add types
amanda-anderson Feb 26, 2025
e454792
Merge pull request #10405 from CitizenLabDotCo/TAN-3875-support-senti…
amanda-anderson Feb 26, 2025
492cfda
Merge pull request #10395 from CitizenLabDotCo/TAN-3874-support-senti…
amanda-anderson Feb 26, 2025
2c12550
Merge in master
amanda-anderson Feb 26, 2025
6696232
Merge branch 'TAN-3873-be-new-sentiment-linear-scale-field' of github…
amanda-anderson Feb 26, 2025
fb013a0
Add optional label in to placeholder for follow up field
amanda-anderson Feb 26, 2025
89370a9
Translations updated by CI (extract-intl)
Feb 26, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 29 additions & 2 deletions back/app/models/custom_field.rb
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@
# linear_scale_label_9_multiloc :jsonb not null
# linear_scale_label_10_multiloc :jsonb not null
# linear_scale_label_11_multiloc :jsonb not null
# ask_follow_up :boolean default(FALSE), not null
#
# Indexes
#
Expand Down Expand Up @@ -66,7 +67,7 @@ class CustomField < ApplicationRecord
INPUT_TYPES = %w[
checkbox date file_upload files html html_multiloc image_files linear_scale rating multiline_text multiline_text_multiloc
multiselect multiselect_image number page point line polygon select select_image shapefile_upload text text_multiloc
topic_ids section cosponsor_ids ranking matrix_linear_scale
topic_ids section cosponsor_ids ranking matrix_linear_scale sentiment_linear_scale
].freeze
CODES = %w[
author_id birthyear body_multiloc budget domicile gender idea_files_attributes idea_images_attributes
Expand Down Expand Up @@ -122,6 +123,10 @@ def includes_other_option?
options.any?(&:other)
end

def ask_follow_up?
ask_follow_up
end

def support_free_text_value?
%w[text multiline_text text_multiloc multiline_text_multiloc html_multiloc].include?(input_type) || (support_options? && includes_other_option?)
end
Expand All @@ -143,7 +148,7 @@ def supports_geojson?
end

def supports_linear_scale?
%w[linear_scale matrix_linear_scale rating].include?(input_type)
%w[linear_scale matrix_linear_scale sentiment_linear_scale rating].include?(input_type)
end

def supports_matrix_statements?
Expand Down Expand Up @@ -224,6 +229,10 @@ def linear_scale?
input_type == 'linear_scale'
end

def sentiment_linear_scale?
input_type == 'sentiment_linear_scale'
end

def rating?
input_type == 'rating'
end
Expand Down Expand Up @@ -303,6 +312,24 @@ def other_option_text_field
)
end

def follow_up_text_field
return unless ask_follow_up?

follow_up_field_key = "#{key}_follow_up"
title_multiloc = MultilocService.new.i18n_to_multiloc(
'custom_fields.ideas.ask_follow_up_field.title',
locales: CL2_SUPPORTED_LOCALES
)

CustomField.new(
key: follow_up_field_key,
input_type: 'multiline_text',
title_multiloc: title_multiloc,
required: false,
enabled: true
)
end

def ordered_options
@ordered_options ||= if random_option_ordering
options.shuffle.sort_by { |o| o.other ? 1 : 0 }
Expand Down
6 changes: 5 additions & 1 deletion back/app/serializers/web_api/v1/custom_field_serializer.rb
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,10 @@ class WebApi::V1::CustomFieldSerializer < WebApi::V1::BaseSerializer
object.dropdown_layout_type?
}

attribute :ask_follow_up, if: proc { |object, _params|
object.input_type == 'sentiment_linear_scale'
}

attribute :constraints do |object, params|
if params[:constraints]
params[:constraints][object.code&.to_sym] || {}
Expand All @@ -45,7 +49,7 @@ class WebApi::V1::CustomFieldSerializer < WebApi::V1::BaseSerializer
:linear_scale_label_9_multiloc,
:linear_scale_label_10_multiloc,
:linear_scale_label_11_multiloc,
if: proc { |object, _params| object.linear_scale? || object.supports_matrix_statements? }
if: proc { |object, _params| object.linear_scale? || object.sentiment_linear_scale? || object.supports_matrix_statements? }

attributes :select_count_enabled, :maximum_select_count, :minimum_select_count, if: proc { |object, _params|
object.multiselect?
Expand Down
5 changes: 5 additions & 0 deletions back/app/services/custom_field_params_service.rb
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@ def reject_other_text_values(extra_field_values)
extra_field_values.delete key
end
end

if key.end_with? '_follow_up'
key.delete_suffix '_follow_up'
extra_field_values.delete key # TODO: - Figure out how to accept follow-up values
end
end
end
end
4 changes: 4 additions & 0 deletions back/app/services/field_visitor_service.rb
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,10 @@ def visit_ranking(field)
default(field)
end

def visit_sentiment_linear_scale(field)
default(field)
end

def visit_page(field)
default(field)
end
Expand Down
1 change: 1 addition & 0 deletions back/app/services/idea_custom_fields_service.rb
Original file line number Diff line number Diff line change
Expand Up @@ -188,6 +188,7 @@ def insert_other_option_text_fields(fields)
fields.each do |field|
all_fields << field
all_fields << field.other_option_text_field if field.other_option_text_field
all_fields << field.follow_up_text_field if field.follow_up_text_field
end
all_fields
end
Expand Down
9 changes: 9 additions & 0 deletions back/app/services/json_schema_generator_service.rb
Original file line number Diff line number Diff line change
Expand Up @@ -262,6 +262,14 @@ def visit_linear_scale(field)
}
end

def visit_sentiment_linear_scale(_field)
{
type: 'number',
minimum: 1,
maximum: 5
}
end

def visit_rating(field)
{
type: 'number',
Expand Down Expand Up @@ -317,6 +325,7 @@ def generate_for_current_locale(fields)

accu[field.key] = field_schema
accu[field.other_option_text_field.key] = visit(field.other_option_text_field) if field.other_option_text_field
accu[field.follow_up_text_field.key] = visit(field.follow_up_text_field) if field.follow_up_text_field
end
{
type: 'object',
Expand Down
1 change: 1 addition & 0 deletions back/app/services/project_copy_service.rb
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,7 @@ def yml_custom_fields(shift_timestamps: 0)
'answer_visible_to' => field.answer_visible_to,
'hidden' => field.hidden,
'maximum' => field.maximum,
'ask_follow_up' => field.ask_follow_up,
'linear_scale_label_1_multiloc' => field.linear_scale_label_1_multiloc,
'linear_scale_label_2_multiloc' => field.linear_scale_label_2_multiloc,
'linear_scale_label_3_multiloc' => field.linear_scale_label_3_multiloc,
Expand Down
21 changes: 13 additions & 8 deletions back/app/services/survey_results_generator_service.rb
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,10 @@ def visit_linear_scale(field)
visit_select_base(field)
end

def visit_sentiment_linear_scale(field)
visit_select_base(field)
end

def visit_matrix_linear_scale(field)
core_field_attributes(field).merge({
multilocs: { answer: build_scaled_input_multilocs(field) },
Expand Down Expand Up @@ -165,8 +169,8 @@ def visit_select_base(field)
query = inputs
query = query.joins(:author) if group_mode == 'user_field'
if group_field
raise "Unsupported group field type: #{group_field.input_type}" unless %w[select linear_scale rating].include?(group_field.input_type)
raise "Unsupported question type: #{field.input_type}" unless %w[select multiselect linear_scale rating multiselect_image].include?(field.input_type)
raise "Unsupported group field type: #{group_field.input_type}" unless %w[select linear_scale sentiment_linear_scale rating].include?(group_field.input_type)
raise "Unsupported question type: #{field.input_type}" unless %w[select multiselect linear_scale sentiment_linear_scale rating multiselect_image].include?(field.input_type)

query = query.select(
select_field_query(field, as: 'answer'),
Expand All @@ -181,7 +185,7 @@ def visit_select_base(field)
end

# Sort correctly
answers = answers.sort_by { |a| -a[:count] } unless %w[linear_scale rating].include?(field.input_type)
answers = answers.sort_by { |a| -a[:count] } unless %w[linear_scale sentiment_linear_scale rating].include?(field.input_type)
answers = answers.sort_by { |a| a[:answer] == 'other' ? 1 : 0 } # other should always be last

# Build response
Expand All @@ -191,7 +195,7 @@ def visit_select_base(field)
def select_field_query(field, as: 'answer')
table = field.resource_type == 'User' ? 'users' : 'ideas'

if %w[select linear_scale rating].include? field.input_type
if %w[select linear_scale sentiment_linear_scale rating].include? field.input_type
"COALESCE(#{table}.custom_field_values->'#{field.key}', 'null') as #{as}"
elsif %w[multiselect multiselect_image].include? field.input_type
%{
Expand Down Expand Up @@ -246,6 +250,7 @@ def build_select_response(answers, field)
})

attributes[:textResponses] = get_text_responses("#{field.key}_other") if field.other_option_text_field
# TODO: Get text responses for sentiment questions with follow up questions.
attributes[:legend] = generate_answer_keys(group_field) if group_field

attributes
Expand All @@ -258,7 +263,7 @@ def get_multilocs(field)
end

def get_option_multilocs(field)
if %w[linear_scale rating].include?(field.input_type)
if %w[linear_scale sentiment_linear_scale rating].include?(field.input_type)
return build_scaled_input_multilocs(field)
end

Expand All @@ -274,7 +279,7 @@ def build_scaled_input_multilocs(field)
{ title_multiloc: locales.index_with { |_locale| value.to_s } }
end

format_labels = %w[linear_scale matrix_linear_scale].include?(field.input_type)
format_labels = %w[linear_scale sentiment_linear_scale matrix_linear_scale].include?(field.input_type)

answer_multilocs.each_key do |value|
labels = field.nth_linear_scale_multiloc(value).transform_values do |label|
Expand All @@ -290,7 +295,7 @@ def build_scaled_input_multilocs(field)
def get_option_logic(field)
return {} if field.logic.blank?

is_linear_or_rating = %w[linear_scale rating].include?(field.input_type)
is_linear_or_rating = %w[linear_scale sentiment_linear_scale rating].include?(field.input_type)
options = if is_linear_or_rating
# Create a unique ID for this linear scale option in the full results so we can filter logic
(1..field.maximum).map { |value| { id: "#{field.id}_#{value}", key: value } }
Expand Down Expand Up @@ -390,7 +395,7 @@ def group_query(query, group: false)
end

def generate_answer_keys(field)
(%w[linear_scale rating].include?(field.input_type) ? (1..field.maximum).to_a : field.options.map(&:key)) + [nil]
(%w[linear_scale sentiment_linear_scale rating].include?(field.input_type) ? (1..field.maximum).to_a : field.options.map(&:key)) + [nil]
end

# Convert stored user keys for domicile field to match the options keys eg "f6319053-d521-4b28-9d71-a3693ec95f45" => "north_london_8rg"
Expand Down
11 changes: 11 additions & 0 deletions back/app/services/ui_schema_generator_service.rb
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,17 @@ def visit_linear_scale(field)
end
end

def visit_sentiment_linear_scale(field)
default(field).tap do |ui_field|
ui_field[:options][:ask_follow_up] = field.ask_follow_up
ui_field[:options][:linear_scale_label1] = multiloc_service.t(field.linear_scale_label_1_multiloc)
ui_field[:options][:linear_scale_label2] = multiloc_service.t(field.linear_scale_label_2_multiloc)
ui_field[:options][:linear_scale_label3] = multiloc_service.t(field.linear_scale_label_3_multiloc)
ui_field[:options][:linear_scale_label4] = multiloc_service.t(field.linear_scale_label_4_multiloc)
ui_field[:options][:linear_scale_label5] = multiloc_service.t(field.linear_scale_label_5_multiloc)
end
end

def visit_matrix_linear_scale(field)
visit_linear_scale(field).tap do |ui_field|
ui_field[:options][:statements] = field.matrix_statements.map do |statement|
Expand Down
2 changes: 2 additions & 0 deletions back/config/locales/en.yml
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,8 @@ en:
description:
other_input_field:
title: Type your answer
ask_follow_up_field:
title: Tell us why (optional)
custom_forms:
categories:
main_content:
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
class AddFollowUpToCustomFields < ActiveRecord::Migration[7.1]
def change
add_column :custom_fields, :ask_follow_up, :boolean, default: false, null: false
end
end
8 changes: 6 additions & 2 deletions back/db/structure.sql
Original file line number Diff line number Diff line change
Expand Up @@ -1564,7 +1564,8 @@ CREATE TABLE public.phases (
manual_votes_count integer DEFAULT 0 NOT NULL,
manual_voters_amount integer,
manual_voters_last_updated_by_id uuid,
manual_voters_last_updated_at timestamp(6) without time zone
manual_voters_last_updated_at timestamp(6) without time zone,
native_survey_method character varying
);
Copy link
Contributor Author

@amanda-anderson amanda-anderson Feb 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jamesspeake Do I need to be concerned about the changes in this file? 🤔 Aside from the ask_follow_up, they're not related to my other changes, but doing a migrate/rebuild doesn't seem to change this file for me locally..



Expand Down Expand Up @@ -2097,7 +2098,8 @@ CREATE TABLE public.custom_fields (
linear_scale_label_8_multiloc jsonb DEFAULT '{}'::jsonb NOT NULL,
linear_scale_label_9_multiloc jsonb DEFAULT '{}'::jsonb NOT NULL,
linear_scale_label_10_multiloc jsonb DEFAULT '{}'::jsonb NOT NULL,
linear_scale_label_11_multiloc jsonb DEFAULT '{}'::jsonb NOT NULL
linear_scale_label_11_multiloc jsonb DEFAULT '{}'::jsonb NOT NULL,
ask_follow_up boolean DEFAULT false NOT NULL
);


Expand Down Expand Up @@ -6826,6 +6828,8 @@ ALTER TABLE ONLY public.ideas_topics
SET search_path TO public,shared_extensions;

INSERT INTO "schema_migrations" (version) VALUES
('20250217295025'),
('20250211103910'),
('20250204143605'),
('20250120125531'),
('20250117121004'),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -416,6 +416,7 @@ def update_all_params
:dropdown_layout,
:page_layout,
:map_config_id,
:ask_follow_up,
{ title_multiloc: CL2_SUPPORTED_LOCALES,
description_multiloc: CL2_SUPPORTED_LOCALES,
linear_scale_label_1_multiloc: CL2_SUPPORTED_LOCALES,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -745,6 +745,78 @@
})
end

example 'Update sentiment_linear_scale field' do
field_to_update = create(:custom_field_sentiment_linear_scale, resource: custom_form)
create(:custom_field, resource: custom_form) # field to destroy
request = {
custom_fields: [
{
input_type: 'page',
page_layout: 'default'
},
{
id: field_to_update.id,
title_multiloc: { 'en' => 'Select a value from the scale' },
description_multiloc: { 'en' => 'Description of question' },
required: true,
enabled: true,
maximum: 5,
ask_follow_up: true,
linear_scale_label_1_multiloc: { 'en' => 'Lowest' },
linear_scale_label_2_multiloc: { 'en' => 'Low' },
linear_scale_label_3_multiloc: { 'en' => 'Neutral' },
linear_scale_label_4_multiloc: { 'en' => 'High' },
linear_scale_label_5_multiloc: { 'en' => 'Highest' },
linear_scale_label_6_multiloc: {},
linear_scale_label_7_multiloc: {},
linear_scale_label_8_multiloc: {},
linear_scale_label_9_multiloc: {},
linear_scale_label_10_multiloc: {},
linear_scale_label_11_multiloc: {}
},
final_page
]
}
do_request request

assert_status 200
json_response = json_parse(response_body)
expect(json_response[:data].size).to eq 3
expect(json_response[:data][1]).to match({
attributes: {
code: nil,
created_at: an_instance_of(String),
description_multiloc: { en: 'Description of question' },
enabled: true,
input_type: 'sentiment_linear_scale',
key: an_instance_of(String),
ordering: 1,
required: true,
title_multiloc: { en: 'Select a value from the scale' },
updated_at: an_instance_of(String),
maximum: 5,
ask_follow_up: true,
linear_scale_label_1_multiloc: { en: 'Lowest' },
linear_scale_label_2_multiloc: { en: 'Low' },
linear_scale_label_3_multiloc: { en: 'Neutral' },
linear_scale_label_4_multiloc: { en: 'High' },
linear_scale_label_5_multiloc: { en: 'Highest' },
linear_scale_label_6_multiloc: {},
linear_scale_label_7_multiloc: {},
linear_scale_label_8_multiloc: {},
linear_scale_label_9_multiloc: {},
linear_scale_label_10_multiloc: {},
linear_scale_label_11_multiloc: {},
logic: {},
random_option_ordering: false,
constraints: {}
},
id: an_instance_of(String),
type: 'custom_field',
relationships: { options: { data: [] }, resource: { data: { id: custom_form.id, type: 'custom_form' } } }
})
end

example 'Update select field with logic' do
field_to_update = create(:custom_field_select, :with_options, resource: custom_form)
survey_end_page = create(:custom_field_page, key: 'survey_end', resource: custom_form)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ class CustomField < Base
random_option_ordering
dropdown_layout
page_layout
ask_follow_up
]

# Enigmatic comment from the previous implementation:
Expand Down
Loading