Gml 1824 regression test with cypher support and no hallucination and usefulness check #248
Conversation
Gml 1793 UI dialog
Minor UI fixes
url sources as links
…-in-docker-compose GML-1796 dependency cycle detecte
Render markdown
…ementation + UI styling updates (#229) * [GML-1779-show-response-as-table-or-graph] graph interface added - mock data * GML-1797 - Investigate ECC Updated ECC to let workers work take a processing node after the expire window. Added more logging and options during configuration * Lots of cleanup of the ECC * [GML-1779-show-response-as-table-or-graph] updates to graph and table, styling updates * [GML-1779-show-response-as-table-or-graph] table vis advancements * [GML-1779-show-response-as-table-or-graph] re-do updates --------- Co-authored-by: Ahmed Albadri <[email protected]> Co-authored-by: RobRossmiller-TG <[email protected]>
* condition tweak for table vis * cleanup, styling tweaks
* condition tweak for table vis * cleanup, styling tweaks * caret typewriter animation edit
…umber-of-transactions
…umber-of-transactions
luzhoutg
requested review from
billshitg,
parkererickson-tg and
RobRossmiller-TG
| @@ -19,6 +19,10 @@ def __init__(self, config): | |||
| from langchain.chat_models import ChatOpenAI | |||
|
|
|||
| model_name = config["llm_model"] | |||
| if graphname == "Transaction_Fraud": | |||
There was a problem hiding this comment.
This shouldn't be here
There was a problem hiding this comment.
If I remember correctly, @RobRossmiller-TG added it to ensure better performance for demo. since gpt-4 works better on transaction fraud and inquiryAI.
There was a problem hiding this comment.
Yeah I don't think that we should ship that though. For the demo it worked well.
There was a problem hiding this comment.
sounds good. got it removed now
There was a problem hiding this comment.
It was removed in the dev branch unless I missed it. Is this based on the latest dev?
There was a problem hiding this comment.
Yes. I also rebase it. It should be based on the latest dev branch now.
copilot/app/agent/agent.py
Outdated
| @@ -156,7 +165,7 @@ def question_for_agent( | |||
| def make_agent(graphname, conn, use_cypher, ws: WebSocket = None, supportai_retriever="hnsw_overlap") -> TigerGraphAgent: | |||
| if llm_config["completion_service"]["llm_service"].lower() == "openai": | |||
| llm_service_name = "openai" | |||
| llm_provider = OpenAI(llm_config["completion_service"]) | |||
| llm_provider = OpenAI(llm_config["completion_service"], graphname=graphname) | |||
There was a problem hiding this comment.
Remove the graphname info here too
luzhoutg
changed the title
luzhoutg
changed the title
Sure. I have added cypher support back as optional now. |
copilot/app/agent/agent.py
Outdated
| @@ -90,7 +91,7 @@ def __init__( | |||
| self.embedding_store, | |||
| self.mq2s, | |||
| self.gen_func, | |||
| cypher_gen_tool=self.cypher_tool, | |||
| # cypher_gen_tool=self.cypher_tool, | |||
There was a problem hiding this comment.
Just double check, is cypher support still available and optional?
There was a problem hiding this comment.
good catch. just added it back. should be good to go.
…hen asking how many
| for key, value in output.items(): | ||
| logger.info(f"testing steps {key}: {value}") |
There was a problem hiding this comment.
Missed this line. Do we need to log all the questions and generated answer?
There was a problem hiding this comment.
We can comment it out. It was trying to understand which step went wrong during testing. I will remove it in next PR.
luzhoutg
deleted the
GML-1824-regression-test-with-out-cypher-support-only
branch
regression test with cypher support only but not hallucination and usefulness check