-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
cf46da6
commit a224e66
Showing
7 changed files
with
215 additions
and
179 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,4 +1,4 @@ | ||
| # Speakers | ||
| # Committee | ||
|
|
||
| <table> | ||
| <tr> | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,22 @@ | ||
| # Home | ||
| # Welcome to the ACL Ethics Website | ||
|
|
||
| ## 🌟 Welcome! 🌟 | ||
|
|
||
| We are delighted to have you here at the **ACL Ethics Website**. Our mission is to promote ethical practices and foster a community of integrity within the field of computational linguistics. | ||
|
|
||
| ### What You Will Find Here | ||
|
|
||
| - **[Schedule](/schedule)**: Stay updated with our latest events, workshops, and meetings. | ||
| - **[Committee](/committee)**: Find out more about the members of the ACL Ethics Committee and their roles. | ||
| - **[Tutorials](/tutorials)**: Explore past tutorials from ACL conferences we have developed. | ||
| - **[Resources](/resources)**: Access to a variety of resources and guidelines for research and publication. | ||
|
|
||
| ### Get Involved | ||
|
|
||
| We encourage you to participate and contribute to our growing community. Your insights and experiences are invaluable in shaping a more ethical future for all. | ||
|
|
||
| --- | ||
|
|
||
| Thank you for visiting, and we look forward to your active participation! | ||
|
|
||
| **The ACL Ethics Committee** |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,177 +1,3 @@ | ||
| # Resources | ||
|
|
||
| Materials for the EACL 2023 Ethics Tutorial: **Understanding Ethics in NLP Authoring and Reviewing**. | ||
|
|
||
| _This repository archives the materials in a reusable form, related materials (ACL Stakeholder Survey) and also links to other contemporary resources. If you see things missing or needing maintenance, please file an issue or a pull request._ | ||
|
|
||
| Authors: | ||
|
|
||
| * [Luciana Benotti](https://benotti.github.io/) (Universidad Nacional de Córdoba) | ||
|
|
||
| * [Karën Fort](https://members.loria.fr/KFort/) (Sorbonne Université and LORIA) | ||
|
|
||
| * [Min-Yen Kan](http://www.comp.nus.edu.sg/~kanmy) (National University of Singapore) | ||
|
|
||
| * [Yulia Tsvetkov](https://homes.cs.washington.edu/~yuliats/) (University of Washington) | ||
|
|
||
| ### How to run your own tutorial or lesson? | ||
|
|
||
| We recommend using our base materials here and adapting them to your setting. Minimally, you need to create your own shortlinks and URL codes to your version of the materials before running, as the archived materials reference our EACL 2023 instance. | ||
|
|
||
| For our tutorial setting in an audience participatory style and approximately 20+ participants and 4 organisers, it was important to: | ||
|
|
||
| * limit the number of groups so that each group could present within the session's time limit; | ||
|
|
||
| * designate a particular organiser to facilitate virtual participants; | ||
|
|
||
| * have live edit access to online documents, to allow participant leads to note take and present. | ||
|
|
||
| Other formats might consider: | ||
|
|
||
| * Lecture only: use and adapt the lecture materials from the first and seventh segments of the tutorial. | ||
|
|
||
| * Experiential only: use and adapt Segments 2-6, which asks participants to read and critique abstracts that bring up common ethical issues in our community. | ||
|
|
||
| * Student (Homework) Assignment: curate sources from the accompanying [Ethics Reading List](#erl) that the committee maintains, and ask participants to read and write their reflections on. | ||
|
|
||
| **If you run a course, tutorial or other session based on these materials, we'd love to hear from you! Please get in contact with us, and we may also (with your permission) list your course or materials in the [Resource](#r) segment. Also please do cite our tutorial abstract as a means of acknowledging the helpfulness of the materials. Thank you!** | ||
|
|
||
| ``` | ||
| @inproceedings{benotti-etal-2023-understanding, | ||
| title = "Understanding Ethics in {NLP} Authoring and Reviewing", | ||
| author = {Benotti, Luciana and | ||
| Fort, Kar{\"e}n and | ||
| Kan, Min-Yen and | ||
| Tsvetkov, Yulia}, | ||
| booktitle = "Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics: Tutorial Abstracts", | ||
| month = may, | ||
| year = "2023", | ||
| address = "Dubrovnik, Croatia", | ||
| publisher = "Association for Computational Linguistics", | ||
| url = "https://aclanthology.org/2023.eacl-tutorials.4", | ||
| pages = "19--24", | ||
| abstract = "With NLP research now quickly being transferred into real-world applications, it is important to be aware of and think through the consequences of our scientific investigation. Such ethical considerations are important in both authoring and reviewing. This tutorial will equip participants with basic guidelines for thinking deeply about ethical issues and review common considerations that recur in NLP research. The methodology is interactive and participatory, including case studies and working in groups. Importantly, the participants will be co-building the tutorial outcomes and will be working to create further tutorial materials to share as public outcomes.", | ||
| } | ||
| ``` | ||
| ## Table of Contents | ||
| (these are internal links to the sections below) | ||
|
|
||
| * [Introduction](#i) | ||
| * [Tutorial Slides](#ts) | ||
| * [Tutorial Recording](#tr) | ||
| * [Activity](#a) | ||
| * [Resources](#r) | ||
|
|
||
| ### <a id="i">Introduction</a> | ||
|
|
||
| In 2022, the ACL Ethics Committee (AEC) decided to propose to run a tutorial on ethics and its impact on ethics in both authoring and reviewing aspects for the community of CL/NLP scholars and practitioners. As part of this process, the committee chairs and Luciana Benotti (a member of the committee) put together a the [proposal file](https://github.com/acl-org/ethics-tutorial/blob/main/proposal/Ethics_Tutorial_Proposal.pdf) and submitted it to the joint call for tutorial proposals. | ||
|
|
||
| The tutorial was eventually accepted to run at [EACL 2023](http://2023.eacl.org) (Dubrovnik, Croatia, 2-6 May 2023) on Saturday, 6 May as part of the 1/2-day AM events. We produced a [teaser](https://github.com/acl-org/ethics-tutorial/blob/main/teaser/teaser.mp4) to accompany the [published tutorial abstract](https://aclanthology.org/2023.eacl-tutorials.4/). | ||
|
|
||
| The tutorial and its materials* were presented in English. It was structured as per the proposal, in seven segments, each approximately 30 minutes long. It was participatory in nature, requiring the audience to work in groups on synthetic problematic abstracts written by the proposers that represent common ethical issues experienced by ethics review chairs. This required a good facilitator-to-group ratio, to ensure all of the participants had a chance to reflect, contribute and be heard, and for the facilitators to keep the session on track. | ||
|
|
||
| *<sub>(Due to the sensitive nature of our EACL 2023 participants' contribution, we have scrubbed the QR coded and shortlinked materials of their presence; if any party wants to access the original document we have these archived but not in this repository nor linked to here; please contact us directly)</sub> | ||
|
|
||
| |**Segment Topic** |**Led By** | | ||
| |--------------------------------------------------------------|-----------------------| | ||
| |1. Introduction and Foundations for Ethics |Presenters | | ||
| |2. Case Studies: Problematic Ethical Research — First reading |Participants | | ||
| |3. Structured Interaction / Dialogue |Presenters,Participants| | ||
| |4. Case studies — Second reading (Rotation) |Participants | | ||
| |5. Group Presentations Group |Leads | | ||
| |6. Summary and Common Issues |Presenters | | ||
| |7. Discussing and Troubleshooting Ethics and Further Resources|Presenters | | ||
|
|
||
| We conducted the tutorial in an active classroom style, where participants were self-organized into small groups (in our instance, 2 groups of about 10 participants each), electing leads for subsequent group presentation and worked through the exercises to record their reactions to the materials and identify the issues and conducive outcomes. | ||
|
|
||
| ### <a id="ts">Tutorial Slides</a> | ||
|
|
||
| After introducing the organisers and the format and resources of the tutorial, the first segment featured two lecture segments, both focused on introductory and foundational material on ethics. | ||
|
|
||
| The presentation deck for the tutorial lives in a public, shared Google Slides permalink: [http://bit.ly/eacl23-ethics-slides](http://bit.ly/eacl23-ethics-slides) but for posterity, we have also archived forms here in this repository: | ||
|
|
||
| * <a href="https://github.com/acl-org/ethics-tutorial/blob/main/slides/Understanding%20Ethics%20in%20NLP%20Authoring%20and%20Reviewing%20%E2%80%93%20Presentation%20Deck.pdf">PDF, local copy</a> - created from the Google Slides origin deck | ||
| * <a href="https://github.com/acl-org/ethics-tutorial/blob/main/slides/Understanding%20Ethics%20in%20NLP%20Authoring%20and%20Reviewing%20%E2%80%93%20Presentation%20Deck.pptx">Microsoft Powerpoint, local copy</a> - created from the Google Slides origin deck | ||
|
|
||
| Segment 1A (Slides 6-39) covers the definition and scope of ethics; its origins; virtue, deontological and utilitarian forms of ethics; experiments and policy reactions to ethical violation (Nuremburg Trials and Code, Belmont Report). _This segment was presented by Karën Fort._ | ||
|
|
||
| Segment 1B (Slides 40-70) focuses on appropriate use and application, biases and evaluation, covering hypothetical and real instances of ethical problems in recent research (Chicken dilemma, IQ prediction, sexual orientation prediction -- the "Gaydar" paper, and how AI and NLP technology influences people both directly and indirectly. _This segment was presented by Yulia Tsvetkov._ | ||
|
|
||
| Segment 7A (Slides 86-99) reprised work from Benotti and Blackburn (2022) [_Ethical Consideration Sections in Natural Language Processing Papers_](https://aclanthology.org/2022.emnlp-main.299/), which examined such written ECS in NLP papers to analyze the issues covered in terms of 1) research benefits, 2) potential harms, and 3) vulnerable groups affected. It covers the differences between geographical region's reporting of such issues. _This segment was presented by Luciana Benotti._ | ||
|
|
||
| The remainder of the slides (Segment 7B) are framing materials to help introduce the tutorial, and set up the abstract critique exercise. _This segment was presented by Min-Yen Kan and all presenters._ | ||
|
|
||
| ### <a id="tr">Tutorial Recording</a> | ||
|
|
||
| The files of the recordings are too large to fit into this repository, but the below links to the segments of the scrubbed recordings. | ||
|
|
||
| * (Segment 1A) - [Introduction and Foundations for Ethics - Part 1](https://drive.google.com/file/d/1iqCfM0ew_hkcMTmPb7vvEhyS0Rx-y0me/view?usp=sharing) - Karën; 22m49s | ||
| * (Segment 1A) - [Introduction and Foundations for Ethics - Part 1 (with Q/A)](https://drive.google.com/file/d/1ZYBnhhif_Z_8pUeH2kedaFY_zZ5F3ez2/view?usp=sharing) - Karën; 40m17s | ||
|
|
||
| * (Segment 1B) - [Introduction and Foundations for Ethics - Part 2](https://drive.google.com/file/d/1blwYuSCsSsII7xRGICif72xYwma7_IXl/view?usp=sharing) - Yulia; 27m18s | ||
| * (Segment 1B) - [Introduction and Foundations for Ethics - Part 2 (with Q/A)](https://drive.google.com/file/d/1BlVrqVHxfEikH9J0Plchw42qgmBobVqm/view?usp=sharing) - Yulia; 46m47s | ||
|
|
||
| * (Segment 7A) - [Ethical Consideration Sections](https://drive.google.com/file/d/1ECBaGAOtf__KS0zm4FA6KBxqAIDiC1oL/view?usp=sharing) - Luciana; 17m42s | ||
|
|
||
| * (Segment 7B) - [Conclusion](https://drive.google.com/file/d/1rNiNRRGZcpnJy-9R_lutb4P1_EBghPbp/view?usp=sharing) - Min; 4m54s | ||
|
|
||
| ### <a id="a">Activity</a> | ||
|
|
||
| The hands-on activity in the tutorial starts with a 5-minute introduction of the activity and a short review of the individual abstracts, which are also described in the slides. Participants were grouped into random groups, which had to elect a scribe and a presenter. The scribe serves as a secretariat for typing in the notes from their group; and the presenter is delegated as the person in the group to present the findings to the entire tutorial audience. Each group was assigned one of the abstracts to read and critique. | ||
|
|
||
| The critique of the abstracts is run in two phases. In the first phase, groups discuss first to prepare a single slide for silent sharing with other groups (Segment 2). In this second phase (Segment 4), the groups could enlarge their thinking by reflecting on either additional abstracts, the single shared slide from each group, or both. | ||
|
|
||
| After both phases were finished, each group presented elements of their findings, with the faciltators structuring and probing for issues and clarity on different aspects. | ||
|
|
||
| #### Materials | ||
|
|
||
| The activity was run using a scribe document where the organizers communicated information (the abstracts below and the instructions) and solicited the audience's feedback on the overall session. | ||
|
|
||
| * <a href="https://github.com/acl-org/ethics-tutorial/blob/main/activity/[Template]%20Understanding%20Ethics%20in%20NLP%20Authoring%20and%20Reviewing%20%E2%80%94%20Scribe%20Document.pdf">Template Scribe Document (.pdf)</a> - scrubbed | ||
| * <a href="https://github.com/acl-org/ethics-tutorial/blob/main/activity/[Template]%20Understanding%20Ethics%20in%20NLP%20Authoring%20and%20Reviewing%20%E2%80%94%20Scribe%20Document.docx">Template Scribe Document (.docx)</a> - scrubbed | ||
| * <a href="https://docs.google.com/document/d/1iNpIeXkelcpGl982fCj69RD6oQ61oQ0ijaf7VlcQUjc/edit#heading=h.5wqi5yqsvt45">Template Scribe Document (live Google Doc)</a> - view-only; scrubbed | ||
|
|
||
| However, for the group activities, we used an editable online presentation deck such that each group could create 1-3 slides (1 slide max in the first phease) in their respective subgroups. | ||
|
|
||
| * <a href="https://github.com/acl-org/ethics-tutorial/blob/main/activity/[Template]%20Understanding%20Ethics%20in%20NLP%20Authoring%20and%20Reviewing%20%E2%80%94%20Activity%20Deck.pdf">Template Group Slide Presentation (.pdf)</a> -scrubbed | ||
| * <a href="https://docs.google.com/presentation/d/1BOF8yXdk7JsBxhkCT0jHjdKTEDaqEKmtkLBwpAZK_-0/edit?usp=sharing">Template Group Slide Presentation (live Google Slide link)</a> - view only; scrubbed | ||
|
|
||
| We provide 6 abstracts below with a gloss of their topical concern. In our EACL 2023 instance, we ran only the first three, due to the smaller number of participants. | ||
|
|
||
| 1. [Facial Recognition](https://github.com/acl-org/ethics-tutorial/blob/main/abstracts/1.txt) | ||
| 2. [Social Media Dataset Collection](https://github.com/acl-org/ethics-tutorial/blob/main/abstracts/2.txt) | ||
| 3. [Cost-prohibitive Language Models](https://github.com/acl-org/ethics-tutorial/blob/main/abstracts/3.txt) | ||
| 4. [Language Resource Collection from Protected Groups](https://github.com/acl-org/ethics-tutorial/blob/main/abstracts/4.txt) | ||
| 5. [Multilingual Sentiment and Crowdsourced Annotation](https://github.com/acl-org/ethics-tutorial/blob/main/abstracts/5.txt) | ||
| 6. [Large language model use in healthcare](https://github.com/acl-org/ethics-tutorial/blob/main/abstracts/6.txt) | ||
|
|
||
| ### <a id="r">Resources</a> | ||
|
|
||
| * The ACL Ethics Stakeholder Survey - Prior to the tutorial, the AEC also conducted a survey of our ACL stakeholders, to help priortize ethics needs of our community. This tutorial is a direct result of that mandate. To be presented live at ACL (Toronto, Canada) on 11 July 2023. | ||
| * Full report for the survey (draft): [Yes, We Care (more)! Results of the 2021 Ethics and Natural Language Processing Survey](https://github.com/acl-org/ethics-tutorial/blob/main/survey/yes_we_care_more_results_of_the_2021_ethics_and_natural_language_processing_survey.pdf) | ||
| * [Associated presentation slides](https://github.com/acl-org/ethics-tutorial/blob/main/survey/230724-ACL-Ethics-Commitee-Panel.pdf) | ||
| * The ACL Ethics Reading List - [https://github.com/acl-org/ethics-reading-list](https://github.com/acl-org/ethics-reading-list) - also mentioned in Slide 110 | ||
|
|
||
| The tutorial (Slide 109) also concluded with a list of faculty courses (by our presenters, certainly there are more out there) that have also taught similar topics. For expediency we list them directly here: | ||
|
|
||
| * (In French and English) Karën https://members.loria.fr/KFort/teaching/nancy-tours-etc/ | ||
| * (In English) Yulia | ||
| https://courses.cs.washington.edu/courses/cse582/23sp/ | ||
| * (In Spanish) Benotti and Alonso Alemany. Ética Práctica para Ciencia de Datos. | ||
| https://sites.google.com/view/etica-practica-cd/ | ||
|
|
||
| #### Copyright and Acknowledgements | ||
|
|
||
| All materials in this repo are [CC-BY-4.0](https://github.com/acl-org/ethics-tutorial/blob/main/LICENSE-CC-BY-4.0.md) | ||
|
|
||
| Our presenters would also like to thank the entire ACL Ethics Committee (AEC) for their support and endorsement of the process. | ||
|
|
||
| * Luciana Benotti, committee member (Americas) [she/her] 2021–2024 | ||
| * Mark Drezde, committee member (Americas) [he/him] 2021–2024 | ||
| * Karën Fort, co-chair (Europe/Africa) [she/her] 2021–2026 | ||
| * Pascale Fung, committee member (Asia/Oceania) [she/her] 2021–2024 | ||
| * Dirk Hovy, committee member (Europe/Africa) [he/him] 2021–2024 | ||
| * Min-Yen Kan, co-chair (Asia/Oceania) [he/him] 2021–2026 | ||
| * Jin-Dong Kim, committee member (Asia/Oceania) [he/him] 2021–2024 | ||
| * Malvina Nissim, committee member (Europe/Africa) [she/her] 2021–2024 | ||
| * Yulia Tsvetkov, co-chair (Americas) [she/her] 2021–2026 | ||
| In this page, we will compile all relevant materials necessary for crafting ethical consideration sections for ACL papers. This includes guidelines for ethical research, best practices, and case studies to help researchers navigate diverse situations. By providing comprehensive resources, we aim to support authors in addressing ethical issues effectively and ensuring their work adheres to the highest standards of integrity. Whether you are new to ethical research or looking to refine your approach, this page will serve as a valuable reference to guide you through the process of ethical decision-making in your research endeavors. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1 +1,3 @@ | ||
| # Schedule | ||
|
|
||
| In this page, we will post all past and upcoming events related to the ACL Ethics Committee. This will serve as a comprehensive schedule where members and interested parties can stay informed about important dates and activities. Whether it's meetings, workshops, or conferences, you will find all relevant information here. Our goal is to ensure transparency and keep everyone updated on the committee's efforts and initiatives. By regularly checking this page, you can stay engaged with the ongoing work and contribute to the ethical discourse within the ACL community. |
Oops, something went wrong.