meeting notes for 24/08, 31/08

meeting_notes/Reginald 24_08_23.md

@@ -0,0 +1,23 @@
+# Reginald 24/08/23
+
+## Notes
+- Quick overview of existing Reginald Work
+    - Reginald (Handbook): fine-tune a Q&A LLM using the REG handbook
+    - Reginald (Llama): use something called llama-index
+    - Quick recap of the type of questions it answered
+- Discussion of work to do this month
+    - Mainly focus on model development
+    - Getting onto Turing slack would be nice, but important to get the model working well first
+        - Not the end of the world with the bot existing in another Slack workspace for now
+        - Time better spent to learn about LLMs and playing around with libraries rather than looking into technicalities of slack and IT policy
+    - Models to look at
+        - RAG
+        - Using LLama2 rather than OpenAI API
+
+## Actions
+- Start playing around with llama-index again and read up on alternative models
+- Look into using an open-source model rather than OpenAI
+- Admin for Ryan:
+    - Get Yi-Ling added to Reginald repo
+    - Add Rosie to Azure subscription
+    - Create regular calendar invite

meeting_notes/Reginald 31_08_23.md

@@ -0,0 +1,35 @@
+# Reginald 31/08/23
+
+## Notes
+- Rosie been working with using llama-index with LLama2 (currently running locally with use of [llama-cpp](https://github.com/abetlen/llama-cpp-python))
+    - Went through Rosie's development notebook
+- Also had a play around with llama-index chat engine
+    - Went through some examples
+    - With the chat engine, the model will try to determine if it should query the database (i.e. construct a prompt with some text from the database) or should it just send something to the LLM
+        - Can we investigate more about _how_ it does this?
+- Is it possible to obtain the prompt to the LLM
+    - We know that llama-index does some prompt engineering, but would it be possible to look at that?
+    - Also in cases when it tries to refine the answer - does it actually give much benefit?
+- Yi-Ling has been looking at llama-index and ideas of how to evaluate the models to see if it's using the knowledge base effectively
+    - We should compile some "model" question and answer pairs that we hope our model to get right
+    - Start to think about being able to systematically compare models and approaches
+- Discussion about documentation
+    - Start uploading meeting notes
+    - Start documenting the models we try and our experiences with them
+
+## Actions
+- Continue with chat engine hacking
+    - There are several chat engine choices given by llama-index
+        - Compare different approaches and figure out what is the most appropriate
+        - Try to figure out when the model is querying and when it is 
+        - Try to figure out how it makes the decision of whether or not to just have a conversation or not
+        - How does this work with small context lengths?
+            - How is it remembering/tracking conversation history?
+- Maybe start working on Azure rather than local
+    - Could be able to run a larger model
+    - Might not need GPU if only inference and only for dev
+    - Maybe GPU long term?
+- Admin:
+    - Start meeting note uploads (Ryan)
+    - Start model documentation (all)
+    - Start project board (Rosie, Levan)