diff --git a/.github/.codecov.yml b/.github/.codecov.yml index 3caf71568..bdedab057 100644 --- a/.github/.codecov.yml +++ b/.github/.codecov.yml @@ -6,6 +6,7 @@ ignore: - ".github" # ignore the .github directory - "docs" # ignore the tests directory - "figs" # ignore the figs directory + - "ui" # ignore the ui directory coverage: status: diff --git a/.github/workflows/cli_tests.yml b/.github/workflows/cli_tests.yml index 6c9cf5422..932a298fc 100644 --- a/.github/workflows/cli_tests.yml +++ b/.github/workflows/cli_tests.yml @@ -20,7 +20,7 @@ jobs: strategy: max-parallel: 5 matrix: - os: [ubuntu-latest, macos-13] + os: [ubuntu-latest, macos-latest] runs-on: ${{ matrix.os }} @@ -38,7 +38,7 @@ jobs: run: | python -m pip install --upgrade pip python -m pip install uv - uv sync --extra test --extra chat + uv sync --extra test --extra api - name: Test with pytest run: | uv run pytest tests/cli/test_install.py --cov=. --cov-report=xml diff --git a/.github/workflows/mypy.yml b/.github/workflows/mypy.yml index 59ed2f621..a3a96d25d 100644 --- a/.github/workflows/mypy.yml +++ b/.github/workflows/mypy.yml @@ -35,7 +35,7 @@ jobs: run: | python -m pip install --upgrade pip python -m pip install uv - uv sync --extra test --extra chat + uv sync --extra test --extra api - name: Type-checking package with mypy run: | # Run this mypy instance against our main package. diff --git a/.github/workflows/tests.sh b/.github/workflows/tests.sh index 332f9d1e9..d6aa6b5af 100644 --- a/.github/workflows/tests.sh +++ b/.github/workflows/tests.sh @@ -1 +1 @@ -uv run --extra test --extra chat pytest --ignore tests/cli --cov=. --cov-report=xml +uv run --extra test --extra api pytest --ignore tests/cli --cov=. --cov-report=xml diff --git a/.github/workflows/tests_in_docker.yml b/.github/workflows/tests_in_docker.yml index 836642b44..4b22e1aad 100644 --- a/.github/workflows/tests_in_docker.yml +++ b/.github/workflows/tests_in_docker.yml @@ -28,7 +28,7 @@ jobs: - name: Docker Compose run: docker compose -f .devcontainer/docker-compose.yml up -d - name: Run tests - run: docker compose -f .devcontainer/docker-compose.yml run --rm -u root -v /home/runner/work/sotopia/sotopia:/workspaces/sotopia devcontainer /bin/sh -c "cd /workspaces/sotopia; ls; uv sync --extra test --extra chat; uv run pytest --ignore tests/cli --cov=. --cov-report=xml" + run: docker compose -f .devcontainer/docker-compose.yml run --rm -u root -v /home/runner/work/sotopia/sotopia:/workspaces/sotopia devcontainer /bin/sh -c "cd /workspaces/sotopia; ls; uv sync --extra test --extra api; uv run pytest --ignore tests/cli --cov=. --cov-report=xml" - name: Upload coverage report to Codecov uses: codecov/codecov-action@v4.0.1 with: diff --git a/docs/pages/concepts/evaluation_dimension.md b/docs/pages/concepts/evaluation_dimension.md new file mode 100644 index 000000000..f86b7a894 --- /dev/null +++ b/docs/pages/concepts/evaluation_dimension.md @@ -0,0 +1,116 @@ +## Overview + +Evaluation dimensions are used to evaluate the quality of social interactions. +In original Sotopia paper, there are 7 dimensions to evaluate the quality of social interactions, where we named them as `sotopia` evaluation dimensions: +- believability +- relationship +- knowledge +- secret +- social rules +- financial and material benefits +- goal + +The `SotopiaDimensions` can be used directly without initializing the database. It provides a set of predefined evaluation dimensions that are ready to use for evaluating social interactions. For example, + +```python +from sotopia.envs.parallel import ParallelSotopiaEnv +from sotopia.envs.evaluators import EvaluationForTwoAgents, ReachGoalLLMEvaluator, RuleBasedTerminatedEvaluator, SotopiaDimensions + +env = ParallelSotopiaEnv( + env_profile=env_profile, + model_name=model_names["env"], + action_order="round-robin", + evaluators=[ + RuleBasedTerminatedEvaluator(max_turn_number=20, max_stale_turn=2), + ], + terminal_evaluators=[ + ReachGoalLLMEvaluator( + model_names["env"], + EvaluationForTwoAgents[SotopiaDimensions], # type: ignore + # TODO check how to do type annotation + ), + ], + ) +``` + + +However we observe under many use cases people may want to evaluate with customized evaluation metrics, so we provide a way to build custom evaluation dimensions. +For a quick reference, you can directly check out the `examples/use_custom_dimensions.py`. + +### CustomEvaluationDimension +The [`CustomEvaluationDimension`](/python_API/database/evaluation_dimensions) is a class that can be used to create a custom evaluation dimension. +There are four parameters: +- name: the name of the dimension +- description: the description of the dimension +- range_low: the minimum score of the dimension (should be an integer) +- range_high: the maximum score of the dimension (should be an integer) + +### CustomEvaluationDimensionList +The [`CustomEvaluationDimensionList`](/python_API/database/evaluation_dimensions) is a class that can be used to create a custom evaluation dimension list based on the existing dimensions. It helps one to group multiple dimensions together for a specific use case. +There are two parameters: +- name: the name of the dimension list +- dimension_pks: the primary keys of the dimensions in the dimension list + +### EvaluationDimensionBuilder +The [`EvaluationDimensionBuilder`](/python_API/database/evaluation_dimensions) is a class that can be used to generate a custom evaluation dimension model based on the existing dimensions. + + +## Usage +### Initialize the database +The default evaluation metric is still `SotopiaDimensions` in `sotopia.env.evaluators`.There is no `CustomEvaluationDimension` in the database by default. To initialize the database, please refer to `examples/use_custom_dimensions.py`. + + +### Use the custom evaluation dimensions +After you initialize your customized evaluation dimensions, you can choose to use any one of these methods provided below: + +#### Method 1: Choose dimensions by names +```python +evaluation_dimensions = ( + EvaluationDimensionBuilder.select_existing_dimension_model_by_name( + ["transactivity", "verbal_equity"] + ) +) +``` + +#### Method 2: Directly choose the grouped evaluation dimension list +```python +evaluation_dimensions = ( + EvaluationDimensionBuilder.select_existing_dimension_model_by_list_name( + "sotopia" + ) +) +``` + +#### Method 3: Build a custom evaluation dimension model temporarily +We provide multiple ways to build a custom evaluation dimension model with `EvaluationDimensionBuilder`, specifically: +- `generate_dimension_model`: build an evaluation dimension from existing dimension primary keys. +- `generate_dimension_model_from_dict`: build an evaluation dimension from a dictionary that specifies the parameters of the `CustomEvaluationDimension`. For example +```json +[ + { + "name": "believability", + "description": "The believability of the interaction", + "range_low": 0, + "range_high": 10 + }, + ... +] +``` +- `select_existing_dimension_model_by_name`: build an evaluation dimension from existing dimension names. For example `['believability', 'goal']` +- `select_existing_dimension_model_by_list_name`: build an evaluation dimension from existing `CustomEvaluationDimensionList` list names. For example, directly use `sotopia`. + + +After you get the evaluation dimension model, you can pass it as a parameter for the `Evaluator`, for example, +```python +evaluation_dimensions = ( + EvaluationDimensionBuilder.select_existing_dimension_model_by_list_name( + "sotopia" + ) +) +terminal_evaluators=[ + ReachGoalLLMEvaluator( + model_names["env"], + EvaluationForTwoAgents[evaluation_dimensions], # type: ignore + ), +], +``` diff --git a/docs/pages/concepts/generation.mdx b/docs/pages/concepts/generation.mdx index f4f58eb6b..1b43db436 100644 --- a/docs/pages/concepts/generation.mdx +++ b/docs/pages/concepts/generation.mdx @@ -47,4 +47,8 @@ In this example, we generate a list of the first `n` prime numbers with the `gpt Apart from using api endpoints from LLM providers like OpenAI, Together AI, Azure, etc., you can also use custom model with OpenAI compatible endpoints. You will need to set the model name to `custom/@url`, and CUSTOM_API_KEY to the API key of the custom model. -For an example, check out `examples/generation_api/custom_model.py`. + +For example, if you want to use the `llama3.2` model for an agent from [Meta](https://www.meta.com/llama/), and you host the model on [LiteLLM](https://github.com/BerriAI/litellm) proxy server (e.g., Proxy running on `http://0.0.0.0:4000`). Then you can set the model name to `model_name="custom/llama3.2:1b@http:0.0.0.0:4000"` +to call the model in the [`LLMAgent`](/python_API/agents/llm_agent#llmagent). + +For more information, check out `examples/generation_api/custom_model.py`. diff --git a/docs/pages/contribution/contribution.md b/docs/pages/contribution/contribution.md index 970218686..33803adfe 100644 --- a/docs/pages/contribution/contribution.md +++ b/docs/pages/contribution/contribution.md @@ -133,7 +133,7 @@ Please refer to [Dev Containers](https://containers.dev/supporting#editors) to s You can also set up the development environment without Dev Containers. There are three things you will need to set up manually: -- Python and uv: Please start from an environment supporting Python 3.10+ and install uv using `pip install uv; uv sync --all-extra`. +- Python and uv: Please start from an environment supporting Python 3.10+ and install uv using `pip install uv; uv sync --all-extras`. (Note that this will install all the extra dependencies) - Redis: Please refer to introduction page for the set up of Redis. - Local LLM (optional): If you don't have access to model endpoints (e.g. OpenAI, Anthropic or others), you can use a local model. You can use Ollama, Llama.cpp, vLLM or many others which support OpenAI compatible endpoints. diff --git a/docs/pages/examples/deployment.md b/docs/pages/examples/deployment.md new file mode 100644 index 000000000..6a8185701 --- /dev/null +++ b/docs/pages/examples/deployment.md @@ -0,0 +1,6 @@ +# Deploy Sotopia Python API to Modal +We offer a script to deploy Sotopia Python API to [Modal](https://modal.com/). +To do so, simply go to the `sotopia/sotopia/ui` directory and run the following command: +```bash +modal deploy sotopia/ui/modal_api_server.py +``` diff --git a/docs/pages/index.mdx b/docs/pages/index.mdx index d7ac669ec..bf109560a 100644 --- a/docs/pages/index.mdx +++ b/docs/pages/index.mdx @@ -117,7 +117,7 @@ export REDIS_OM_URL="redis://localhost:6379" ``` if you are developing Sotopia using uv, you can sync your dependency with ```bash - uv sync --extra examples --extra chat + uv sync --extra examples --extra api ``` @@ -144,13 +144,18 @@ or manual setup: Docker is my thing. - Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/troubleshooting.md) to initiate the redis server with the Sotopia data. + Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. If you want to use the existing data in Sotopia, you can download the `dump.rdb` file from [here](https://cmu.box.com/shared/static/xiivc5z8rnmi1zr6vmk1ohxslylvynur). Feel free to check more datasets related to Sotopia [here](https://huggingface.co/collections/cmu-lti/sotopia-65f312c1bd04a8c4a9225e5b). + + After downloading the `dump.rdb` file, make a `redis-data` folder in an desired `` directory. And then you can start the server with the following command: + ```bash + docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 -v /redis-data:/data/ redis/redis-stack:latest + ``` The `REDIS_OM_URL` need to be set before loading and saving agents: ```bash conda env config vars set REDIS_OM_URL="redis://user:password@host:port" ``` - + No, I don't want to use Docker. diff --git a/docs/pages/python_API/database/evaluation_dimensions.md b/docs/pages/python_API/database/evaluation_dimensions.md new file mode 100644 index 000000000..4a826a555 --- /dev/null +++ b/docs/pages/python_API/database/evaluation_dimensions.md @@ -0,0 +1,54 @@ +# `evaluation_dimensions.py` + +This module provides classes and utilities for defining and managing custom evaluation dimensions within the Sotopia environment. It includes classes for individual dimensions, lists of dimensions, and a builder for creating dimension models. + +## Classes + +### `CustomEvaluationDimension` + +Represents a custom evaluation dimension with specific attributes such as name, description, and score range. + +#### Attributes +- `name`: `str`. The name of the dimension. +- `description`: `str`. A brief description of the dimension. +- `range_low`: `int`. The minimum score for the dimension. +- `range_high`: `int`. The maximum score for the dimension. + +### `CustomEvaluationDimensionList` + +Groups multiple custom evaluation dimensions together. + +#### Attributes +- `name`: `str`. The name of the dimension list. +- `dimension_pks`: `list[str]`. A list of primary keys for the dimensions included in the list. + +### `EvaluationDimensionBuilder` + +Provides utility methods to create and manage evaluation dimension models. + +#### Methods +- `create_range_validator(low: int, high: int)`: Creates a validator for score ranges. + + **Arguments:** + - `low`: `int`. The minimum score allowed. + - `high`: `int`. The maximum score allowed. + +- `build_dimension_model(dimension_ids: list[str])`: Builds a dimension model from primary keys. + + **Arguments:** + - `dimension_ids`: `list[str]`. A list of dimension primary keys. + +- `build_dimension_model_from_dict(dimensions: list[dict[str, Union[str, int]]])`: Builds a dimension model from a dictionary. + + **Arguments:** + - `dimensions`: `list[dict[str, Union[str, int]]]`. A list of dictionaries specifying dimension attributes. + +- `select_existing_dimension_model_by_name(dimension_names: list[str])`: Selects a dimension model by dimension names. + + **Arguments:** + - `dimension_names`: `list[str]`. A list of dimension names. + +- `select_existing_dimension_model_by_list_name(list_name: str)`: Selects a dimension model by list name. + + **Arguments:** + - `list_name`: `str`. The name of the dimension list. diff --git a/examples/experiment_eval.py b/examples/experiment_eval.py index 1c01ef8a6..82fe4bbd0 100644 --- a/examples/experiment_eval.py +++ b/examples/experiment_eval.py @@ -17,6 +17,7 @@ EnvAgentComboStorage, EnvironmentProfile, EpisodeLog, + EvaluationDimensionBuilder, ) from sotopia.envs.evaluators import ( EvaluationForTwoAgents, @@ -34,6 +35,7 @@ ) from sotopia.server import run_async_server from sotopia_conf.gin_utils import parse_gin_flags, run +# from sotopia.database import EvaluationDimensionBuilder _DEFAULT_GIN_SEARCH_PATHS = [ os.path.dirname(os.path.dirname(os.path.abspath(__file__))) @@ -109,6 +111,18 @@ def _iterate_env_agent_combo_not_in_db( tag: str | None = None, ) -> Generator[EnvAgentCombo[Observation, AgentAction], None, None]: """We iterate over each environment and return the **first** env-agent combo that is not in the database.""" + # loading evaluation metric + try: + evaluation_dimensions = EvaluationDimensionBuilder.select_existing_dimension_model_by_list_name( + "sotopia" + ) # Initialize your customized dimension, please refer to `examples/use_custom_dimensions.py` + except Exception as e: + print( + "No customized evaluation dimensions found, using default SotopiaDimensions", + e, + ) + evaluation_dimensions = SotopiaDimensions + if not env_ids: env_ids = list(EnvironmentProfile.all_pks()) for env_id in env_ids: @@ -123,6 +137,11 @@ def _iterate_env_agent_combo_not_in_db( ) assert env_agent_combo_storage_list first_env_agent_combo_storage_to_run: EnvAgentComboStorage | None = None + + env_agent_combo_storage_list = sorted( + env_agent_combo_storage_list, key=lambda x: str(x.pk) + ) + for env_agent_combo_storage in env_agent_combo_storage_list: env_agent_combo_storage = cast( EnvAgentComboStorage, env_agent_combo_storage @@ -147,7 +166,8 @@ def _iterate_env_agent_combo_not_in_db( terminal_evaluators=[ ReachGoalLLMEvaluator( model_names["env"], - EvaluationForTwoAgents[SotopiaDimensions], + EvaluationForTwoAgents[evaluation_dimensions], # type: ignore + # TODO check how to do type annotation ), ], ) @@ -183,10 +203,14 @@ def run_async_server_in_batch( logger.removeHandler(rich_handler) # we cannot get the exact length of the generator, we just give an estimate of the length - env_agent_combo_iter = _iterate_env_agent_combo_not_in_db(model_names=model_names) + env_agent_combo_iter = _iterate_env_agent_combo_not_in_db( + model_names=model_names, tag=tag + ) env_agent_combo_iter_length = sum(1 for _ in env_agent_combo_iter) - env_agent_combo_iter = _iterate_env_agent_combo_not_in_db(model_names=model_names) + env_agent_combo_iter = _iterate_env_agent_combo_not_in_db( + model_names=model_names, tag=tag + ) env_agent_combo_batch: list[EnvAgentCombo[Observation, AgentAction]] = [] while True: diff --git a/examples/experimental/group_discussion_agents/group_discussion_agents.py b/examples/experimental/group_discussion_agents/group_discussion_agents.py index e4b3c0c2c..8ef55e5c5 100644 --- a/examples/experimental/group_discussion_agents/group_discussion_agents.py +++ b/examples/experimental/group_discussion_agents/group_discussion_agents.py @@ -2,7 +2,7 @@ from aact import Message, NodeFactory from aact.messages import Text, Tick, DataModel, DataModelFactory from sotopia.agents.llm_agent import ainput -from sotopia.experimental.agents import BaseAgent +from sotopia.experimental.agents.base_agent import BaseAgent from sotopia.generation_utils import agenerate from sotopia.generation_utils.generate import StrOutputParser diff --git a/examples/experimental/interview_openhands/interview_openhands.toml b/examples/experimental/interview_openhands/interview_openhands.toml new file mode 100644 index 000000000..2cecbb751 --- /dev/null +++ b/examples/experimental/interview_openhands/interview_openhands.toml @@ -0,0 +1,99 @@ +redis_url = "redis://localhost:6379/0" +extra_modules = ["examples.experimental.interview_openhands.llm_agent", "examples.experimental.nodes.scene_context_node", "examples.experimental.nodes.chat_print_node"] + + +[[nodes]] +node_name = "Jack" +node_class = "llm_agent" + +[nodes.node_args] +query_interval = 5 +output_channel = "Jack:Jane" +input_text_channels = ["Jane:Jack"] +input_env_channels = ["Scene:Jack", "Runtime:Agent"] +input_tick_channel = "tick/secs/1" +goal = "Your goal is to effectively test Jane's technical ability and finally decide if she has passed the interview. Make sure to also evaluate her communication skills, problem-solving approach, and enthusiasm." +model_name = "gpt-4o-mini" +agent_name = "Jack" + +[[nodes]] +node_name = "Jane" +node_class = "llm_agent" + +[nodes.node_args] +query_interval = 7 +output_channel = "Jane:Jack" +input_text_channels = ["Jack:Jane"] +input_env_channels = ["Scene:Jane", "Runtime:Agent"] +input_tick_channel = "tick/secs/1" +goal = "Your goal is to do well in the interview by demonstrating your technical skills, clear communication, and enthusiasm for the position. Stay calm, ask clarifying questions when needed, and confidently explain your thought process." +model_name = "gpt-4o-mini" +agent_name = "Jane" + +[[nodes]] +node_name = "tick" +node_class = "tick" + +[[nodes]] +node_name = "JaneScene" +node_class = "scenario_context" + +[nodes.node_args] +input_tick_channel = "tick/secs/1" +output_channels = ["Scene:Jane"] +env_scenario = """ +You are Jane, a college senior at Stanford University interviewing for a Software Engineering Intern position at Fintech company. You are currently sitting in an office with your interviewer, Jack. +It's natural to feel a bit nervous, but remind yourself that you have prepared well. + +### Goals: +1. **Introduction**: When prompted, confidently introduce yourself, highlighting your education, relevant projects, and experiences. +2. **Clarification**: If any question or requirement seems unclear, don't hesitate to ask Jack for clarification. +3. **Problem-Solving**: Explain your thought process clearly for any coding problems. Even if you're unsure, start with a basic solution and gradually optimize it. +4. **Communication**: Be articulate in your explanations. Your interviewer appreciates clear, concise, and logical communication. +5. **Coding**: Write your code in a file in the /workspace directory. Make sure to justify each part of your solution. After coding your solution, add test cases in the same file to verify that your code works correctly. Explain how your test cases cover different scenarios and edge cases. +6. **Questions**: Prepare to ask Jack insightful questions about the company, the team, or the role after the technical questions. + +Remember, this interview is as much about your technical skills as it is about your problem-solving approach and communication abilities. +""" + +[[nodes]] +node_name = "JackScene" +node_class = "scenario_context" + +[nodes.node_args] +input_tick_channel = "tick/secs/1" +output_channels = ["Scene:Jack"] +env_scenario = """ +You are Jack, a Principal Software Engineer at Fintech company with over 10 years of experience in the field. +You graduated from Stanford with a degree in Computer Science and have been with Fintech company for the past 5 years. +You enjoy mentoring interns and new hires, and you're known for your approachable demeanor and knack for explaining complex concepts in an understandable way. +Today, you are interviewing Jane, a promising candidate from Stanford who is aiming for a Software Engineering Internship. + +### Goals: +1. **Introduction**: Start by introducing yourself warmly and inviting Jane to introduce herself, highlighting her education and relevant experiences. +2. **Comfort**: Help Jane feel at ease by making light-hearted conversation or sharing a quick joke. +3. **Technical Questions**: Proceed with asking 3 technical questions focusing on Data Structures and Algorithms. Make sure to: + - Clearly specify the problem statement. + - Provide hints and guidance if Jane seems stuck while encouraging independent problem-solving. +4. **Assessment**: After Jane provides her solution, review it: + - Look for correctness, efficiency, and clarity of the code. + - Ask Jane to explain her solution and discuss any optimizations. + - Run test cases and provide feedback. +5. **Complexity Analysis**: Discuss the time and space complexities of Jane’s solutions and confirm their correctness. +6. **Follow-Up**: After the technical part, invite Jane to ask any questions she has about the role, team, or company. +7. **Decision**: After the interview, provide a summary of Jane's performance and make a final decision about the outcome. + +This interview not only evaluates Jane’s technical skills but also her communication, problem-solving approach, and fit for the team. +""" + +[[nodes]] +node_name = "chat_print" +node_class = "chat_print" + +[nodes.node_args.print_channel_types] +"Jane:Jack" = "agent_action" +"Jack:Jane" = "agent_action" +"Agent:Runtime" = "agent_action" + +[nodes.node_args] +env_agents = ["Jack", "Jane"] diff --git a/examples/experimental/interview_openhands/llm_agent.py b/examples/experimental/interview_openhands/llm_agent.py new file mode 100644 index 000000000..421a48232 --- /dev/null +++ b/examples/experimental/interview_openhands/llm_agent.py @@ -0,0 +1,448 @@ +import logging +import sys +from enum import Enum +from rich.logging import RichHandler +from pydantic import Field + +from typing import Optional + +from aact import Message, NodeFactory +from aact.messages import Text, Tick, DataModel +from aact.messages.registry import DataModelFactory + +from sotopia.experimental.agents.base_agent import BaseAgent + +from sotopia.generation_utils import agenerate +from sotopia.generation_utils.generate import StrOutputParser + +import json + +# Check Python version +if sys.version_info >= (3, 11): + pass +else: + pass + +# Configure logging +FORMAT = "%(asctime)s - %(levelname)s - %(name)s - %(message)s" +logging.basicConfig( + level=logging.WARNING, + format=FORMAT, + datefmt="[%X]", + handlers=[RichHandler()], +) + + +class ActionType(Enum): + NONE = "none" + SPEAK = "speak" + NON_VERBAL = "non-verbal" + LEAVE = "leave" + THOUGHT = "thought" + BROWSE = "browse" + BROWSE_ACTION = "browse_action" + READ = "read" + WRITE = "write" + RUN = "run" + + def __str__(self) -> str: + return self.value + + def __eq__(self, other: object) -> bool: + if isinstance(other, ActionType): + return self.value == other.value + elif isinstance(other, str): + return self.value == other + else: + return NotImplemented + + +@DataModelFactory.register("agent_action") +class AgentAction(DataModel): + agent_name: str = Field(description="the name of the agent") + action_type: ActionType = Field( + description="whether to speak at this turn or choose to not do anything" + ) + argument: str = Field( + description="the utterance if choose to speak, the expression or gesture if choose non-verbal communication, or the physical action if choose action" + ) + path: Optional[str] = Field(description="path of file") + + def to_natural_language(self) -> str: + action_descriptions = { + ActionType.NONE: "did nothing", + ActionType.SPEAK: f'said: "{self.argument}"', + ActionType.THOUGHT: f'thought: "{self.argument}"', + ActionType.BROWSE: f'browsed: "{self.argument}"', + ActionType.RUN: f'ran: "{self.argument}"', + ActionType.READ: f'read: "{self.argument}"', + ActionType.WRITE: f'wrote: "{self.argument}"', + ActionType.NON_VERBAL: f"[{self.action_type.value}] {self.argument}", + ActionType.LEAVE: "left the conversation", + } + + return action_descriptions.get(self.action_type, "performed an unknown action") + + +@NodeFactory.register("llm_agent") +class LLMAgent(BaseAgent[AgentAction | Tick | Text, AgentAction]): + def __init__( + self, + input_text_channels: list[str], + input_tick_channel: str, + input_env_channels: list[str], + output_channel: str, + query_interval: int, + agent_name: str, + goal: str, + model_name: str, + redis_url: str, + ): + super().__init__( + [ + (input_text_channel, AgentAction) + for input_text_channel in input_text_channels + ] + + [ + (input_tick_channel, Tick), + ] + + [(input_env_channel, Text) for input_env_channel in input_env_channels], + [(output_channel, AgentAction)], + redis_url, + ) + self.output_channel = output_channel + self.query_interval = query_interval + self.count_ticks = 0 + self.message_history: list[tuple[str, str, str]] = [] + self.name = agent_name + self.model_name = model_name + self.goal = goal + + async def send(self, message: AgentAction) -> None: + if message.action_type in ("speak", "thought"): + await self.r.publish( + self.output_channel, + Message[AgentAction](data=message).model_dump_json(), + ) + + elif message.action_type in ("browse", "browse_action", "write", "read", "run"): + await self.r.publish( + "Agent:Runtime", + Message[AgentAction](data=message).model_dump_json(), + ) + + def _format_message_history( + self, message_history: list[tuple[str, str, str]] + ) -> str: + ## TODO: akhatua Fix the mapping of action to be gramatically correct + return "\n".join( + (f"{speaker} {action} {message}") + for speaker, action, message in message_history + ) + + def get_action_template(self, selected_actions: list[ActionType]) -> str: + """ + Returns the action template string with selected actions. + + Args: + selected_actions (list[ActionType]): List of ActionType enum members to include in the template. + + Returns: + str: The action template with the selected actions. + """ + base_template = """ You are talking to another agent. + You are {agent_name}.\n + {message_history}\nand you plan to {goal}. + ## Action + What is your next thought or action? Your response must be in JSON format. + + It must be an object, and it must contain two fields: + * `action`, which is one of the actions below + * `args`, which is a map of key-value pairs, specifying the arguments for that action + """ + + action_descriptions = { + str( + ActionType.SPEAK + ): """`speak` - you can talk to the other agents to share information or ask them something. Arguments: + * `content` - the message to send to the other agents (should be short)""", + str( + ActionType.THOUGHT + ): """`thought` - only use this rarely to make a plan, set a goal, record your thoughts. Arguments: + * `content` - the message you send yourself to organize your thoughts (should be short). You cannot think more than 2 turns.""", + str( + ActionType.NONE + ): """`none` - you can choose not to take an action if you are waiting for some data""", + str( + ActionType.NON_VERBAL + ): """`non-verbal` - you can choose to do a non verbal action + * `content` - the non veral action you want to send to other agents. eg: smile, shrug, thumbs up""", + str(ActionType.BROWSE): """`browse` - opens a web page. Arguments: + * `url` - the URL to open, when you browse the web you must use `none` action until you get some information back. When you get the information back you must summarize the article and explain the article to the other agents.""", + str( + ActionType.BROWSE_ACTION + ): """`browse_action` - actions you can take on a web browser + * `command` - the command to run. You have 15 available commands. These commands must be a single string value of command + Options for `command`: + `command` = goto(url: str) + Description: Navigate to a url. + Examples: + goto('http://www.example.com') + + `command` = go_back() + Description: Navigate to the previous page in history. + Examples: + go_back() + + `command` = go_forward() + Description: Navigate to the next page in history. + Examples: + go_forward() + + `command` = noop(wait_ms: float = 1000) + Description: Do nothing, and optionally wait for the given time (in milliseconds). + You can use this to get the current page content and/or wait for the page to load. + Examples: + noop() + noop(500) + + `command` = scroll(delta_x: float, delta_y: float) + Description: Scroll horizontally and vertically. Amounts in pixels, positive for right or down scrolling, negative for left or up scrolling. Dispatches a wheel event. + Examples: + scroll(0, 200) + scroll(-50.2, -100.5) + + `command` = fill(bid, value) + Description: Fill out a form field. It focuses the element and triggers an input event with the entered text. It works for ,