ACCORNS or Admin Control Center Overseeing RAG Needed for SCUIRREL is the instructor-facing backend application to SCUIRREL. Here you can set-up, manage and monitor SCUIRREL and the students' interactions with it.
If you are looking for more details on how to analyse the data collected by SCUIRREL or ACCORNS, please read the researcher guide
If you are a developer, please read the devloper guide for specific implementation details
Navigate to the Vector database tab
Instructors can upload text-based materials relevant to the topics students should discuss with SCUIRREL. During the conversation, SCUIRREL will refer to these materials using retrieval augmented generation (RAG) to improve the quality an accuracy of the conversation.The files are processed and stored in a special vector database (part of the app itself).
You need to have uploaded at least one document before you can start configuring topics and concepts. This is because SCUIRREL will try to limit itself to draw knowledge from the background information you provide instead of relying on its inherent knowledge as large language model (LLM). The application should accept most text based file formats (e.g. .txt, .pdf, .docx, .md) and even powerpoint slides (not extensively tested).
- Click the Pick a file button at the bottom of the page
- Select a file from your local machine and upload it
- Depending on the size of the file, it might take several minutes to process, during which additional uploads are not possible. Don't exit the app until finished
- When the files has been processed successfully, it will appear in the table at the top of the screen and you can click it to view an AI generated summary
NOTE: The uploaded material will be sent to the LLM (e.g. ChatGPT), so please ensure that you have the necessary permissions to share this content and read the privacy policies to know what is being done with your data on their side.
Navigate to the User management tab
There are three types of access codes used to manage access to both SCUIRREL and ACCORNS apps and their functionality:
- Account creation code
- Password reset code
- Group access code
Note: the group access code is created and managed in the groups tab (see below)
Students, instructors and admins need to be given an access code before they can access ACCORNS or SCUIRREL. You can generate these codes by providing the following information in the User management tab form:
- Number of codes to generate (you can always generate more if needed)
- Role
- Student codes allow students to create an account with SCUIRREL access
- Instructor codes allow instructors to create an account with ACCORNS and SCUIRREL access. Instructors can be invited to join groups and manage them in ACCORNS (see below)
- Admins codes (can only be generated by other admins) will allow new admins to join and fully manage all ACCORNS and SCUIRREL accounts and see everything from all instructors
- Note: Optional note why you generated the codes, can help to keep track of unused codes
Once codes have been generated, you will be able to download them as a CSV file and can distribute them. You can keep track of unused codes in the relevant table to monitor how many have been used.
Anyone with ACCORNS access can help users (both SCUIRREL and ACCORNS) with a password reset. A user who has forgotten their password will request a reset code, which will appear in the relevant table. If someone reaches out to you, just ask their username and you can give them the reset code. Once they have successfully reset their password, the code will disappear from the table.
For more info on how to reset a password, see the student guide
Navigate to the groups tab
The applications have been designed to be used by multiple instructors at once. To prevent all content to be mixed, groups can to be created to separate topics and student access by class or any other relevant grouping.
To create a new group, click the New group button and provide a group name.
Once a group is created, you can add additional instructors (group managers) or students to it by generating group access codes and distributing them. NOte that each code can only be used once. To generate the code provide the following:
- Number of codes to generate (you can always generate more if needed)
- Role
- Student codes allow students to join the group in SCUIRREL and interact with all its assigned topics
- Admins codes will allow another instructor or admin to join and manage the group in ACCORNS
- Note: Optional note why you generated the codes, can help to keep track of unused codes
Once codes have been generated, you will be able to download them as a CSV file and can distribute them to students / co-instructors. You can keep track of unused codes in the relevant table to monitor how many have been used.
If you have been given an admin-level group code by a co-instructor, you can click the Join group button at the top of the Groups page. Once you have joined, you will be able to manage the group and all its topics
Navigate to the topics tab
Each topic you create will be a choice for students to discuss with SCUIRREL. You can compare a topic to a textbook chapter. The scope should be something that can be revised in a short conversation of a few minutes.
- Select a group under which the topic will be stored (you can create new groups in the groups tab)
- Depending on the add / edit / archive button clicked, you will be able to create or modify an existing topic
It is recommended to only edit a topic in case of typos or small changes that do not significantly change the covered content (create new one otherwise)
For each topic, you have to create one or more concepts. A concept can be seen as a topic-related fact or piece of information that you want SCUIRREL to check with your students. Compare this to bullet points summarising a textbook chapter.
- You can add, edit or archive concepts as needed using the provided buttons underneath the table of concepts
- Early experiments suggest that about 5-10 concepts per topic is a good number to keep the conversation engaging but not too long, but feel free to experiment with this and provide us with feedback
- You can reorder topics by clicking the respective button. KNow that the order is important as it guides the conversation
The way SCUIRREL works is that it will use the topic as the main conversation thread, and systematically check each of the concepts (in order) with the student by walking through the list you created. Whilst this is happening, a second LLM will monitor the conversation and decide if a student demonstrates enough understanding of a concept to move on to the next one. Once all concepts belonging to a topic have been covered, SCUIRREL will end the conversation.
Navigate to the Quiz questions tab
For each topic, you can also create quiz questions that students can take to verify their topic understanding. The LLM will help generate these questions for you based on the concepts you provided for a topic and the uploaded materials. To generate a new question:
- Select a topic, and click the generate quiz question button
- The LLM will try and generate a question, 4 possible answers and an explanation why each answer is correct or incorrect
- It is possible the LLM fails at this task in which case you might see an error message and can try again later
- Once generated, you will have the option to review everything and edit any part of the question, answer or explanation to ensure it is correct and good for assessment.
- Questions can be archived at any time, in which case they will no longer be shown to students
Given this is a more summative type of assessment we opted to have the instructor validate each question instead of having the LLM directly generate them for the student. This is also useful from a monitoring / research perspective to compare the conversation with SCUIRREL to the performance on the quiz questions. For more info, we refer to the researcher documentation
This project was designed to help student gauge their background on this topic before they come into class, or revisit it after class. For this reason, we do not recommend using this application to teach brand new material, but rather serve as an aid for students. You as an instructor should still be responsible for what's being taught :)