-
Notifications
You must be signed in to change notification settings - Fork 7
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Signed-off-by: Dmitri Saberi <[email protected]>
- Loading branch information
1 parent
1dfc26f
commit d094a68
Showing
39 changed files
with
5,641 additions
and
6,467 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -22,4 +22,5 @@ dist-ssr | |
*.njsproj | ||
*.sln | ||
*.sw? | ||
.env | ||
.env | ||
.vercel |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,47 +1,66 @@ | ||
<div align="center"> | ||
<h1 align="center">Flux</h1> | ||
<p align="center"> | ||
Graph-based LLM power tool for exploring many completions in parallel. | ||
<br /> | ||
<br /> | ||
<a href="https://twitter.com/transmissions11/status/1640775967856803840">Announcement</a> | ||
· | ||
<a href="http://flux.paradigm.xyz">Try Online</a> | ||
· | ||
<a href="https://github.com/paradigmxyz/flux/issues">Report a Bug</a> | ||
</p> | ||
|
||
<img src="./docs/source/_static/logo.png" alt="Branches Logo" width=200></img> | ||
|
||
# Branches | ||
|
||
Prototype advanced LLM algorithms for reasoning and planning. | ||
|
||
[Try Online](http://code-gen-tree.vercel.app) • | ||
[Report a Bug](https://github.com/normal-computing/branches/issues) • | ||
[Stay tuned](#stay-tuned-for) | ||
|
||
</div> | ||
|
||
<br /> | ||
![Branches in action: Tree-search visualization for code generation with self-correction in the HumanEval benchmark](https://storage.googleapis.com/normal-blog-artifacts/systerm2/tot_demo.gif) | ||
|
||
![A screenshot of a Flux workspace.](/public/meta-full.png) | ||
***Tree-search visualization during code generation.** We visualize a reasoning algorithm which learns from feedback, automatically correcting itself by analyzing error tracebacks to refine its solutions. In this case, we benchmark Python programming problems from the HumanEval dataset.* | ||
|
||
## About | ||
|
||
Flux is a power tool for interacting with large language models (LLMs) that **generates multiple completions per prompt in a tree structure and lets you explore the best ones in parallel.** | ||
Branches is an AI tool for graph-based prototyping of advanced algorithms for LLM reasoning and planning -- like Tree of Thoughts and Reflexion. Branches is adapted from [Flux](https://github.com/paradigmxyz/flux). | ||
|
||
Flux's tree structure allows you to: | ||
Designed for researchers and developers, it allows users to directly interact with AI reasoning processes, streamlining the exploration of complex coding challenges and strategic problem-solving. | ||
|
||
- Get a wider variety of creative responses | ||
### Code Generation (HumanEval) | ||
|
||
- Test out different prompts with the same shared context | ||
Branches automatically expands decision trees to solve programming problems from the [HumanEval dataset](https://huggingface.co/datasets/openai_humaneval), visualizing reasoning chains and facilitating self-correction through error tracebacks. This is found on the `main` branch and is currently hosted. | ||
|
||
- Use inconsistencies to identify where the model is uncertain | ||
### Game of 24 | ||
Branches includes a specialized evaluation mechanism for the [Game of 24 puzzle](https://en.wikipedia.org/wiki/24_(puzzle)), leveraging a scoring system to enhance breadth-first search (BFS) by prioritizing promising paths. This is found on the `game-of-24` branch. | ||
|
||
It also provides a robust set of keyboard shortcuts, allows setting the system message and editing GPT messages, autosaves to local storage, uses the OpenAI API directly, and is 100% open source and MIT licensed. | ||
## Features | ||
|
||
## Usage | ||
- [x] 🌳 **Automated Tree Expansion**: Leveraging Tree of Thoughts for dynamic expansion in problem-solving. | ||
- [x] 🧠 **Pre-loaded Prompts**: Curated for search-based reasoning to solve specific problems. | ||
- [x] 💻 **Code Interpretation**: Instant execution and error analysis for self-correcting AI-generated code. | ||
- [x] 🔍 **Scoring Mechanism**: Advanced BFS for the Game of 24 with node evaluation for search optimization. | ||
- [x] 📊 **Interactive Visualization**: Graphical representation of tree searches for easy analysis and education. Largely adapted from [Flux](https://github.com/paradigmxyz/flux). | ||
|
||
Visit [flux.paradigm.xyz](https://flux.paradigm.xyz) to try Flux online or follow the instructions below to run it locally. | ||
## Usage | ||
|
||
## Running Locally | ||
To get started with Branches, you can either visit [code-gen-tree.vercel.app](https://code-gen-tree.vercel.app) for the hosted version or run it locally by following the instructions below. | ||
|
||
## Deploy to Vercel | ||
```sh | ||
git clone https://github.com/paradigmxyz/flux.git | ||
npm install | ||
npm run dev | ||
npm i -g vercel | ||
vercel | ||
``` | ||
|
||
## Stay Tuned For | ||
|
||
Our commitment to enhancing Branches continues, with exciting new developments on the way: | ||
|
||
- More reasoning and planning algorithms beyond the defaults ([#10](https://github.com/normal-computing/branches/issues/10)) | ||
- Node Value Editing and Regenerate Subtree Functionality ([#5](https://github.com/normal-computing/branches/issues/5)) | ||
- UI Color Fixes and Customization Features ([#6](https://github.com/normal-computing/branches/issues/6)) | ||
- Address Model/UI Timeout Issues ([#7](https://github.com/normal-computing/branches/issues/7)) | ||
- Enhance Game of 24 Logic, Model Cost Tracking, and Prompt Engineering ([#8](https://github.com/normal-computing/branches/issues/8)) | ||
|
||
## Contributing | ||
|
||
See the [open issues](https://github.com/paradigmxyz/flux/issues) for a list of proposed features (and known issues). | ||
Your contributions make Branches better. Whether it’s bug reports, new features, or feedback, we welcome it all! Report bugs or request features by creating an issue [here](https://github.com/normal-computing/Branches/issues). | ||
|
||
## License | ||
|
||
Branches is open-source and continues to uphold the [MIT license](LICENSE). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,40 @@ | ||
from http import HTTPStatus | ||
import json | ||
from concurrent.futures import ThreadPoolExecutor | ||
from human_eval.execution import check_correctness | ||
from flask import Flask, request, jsonify | ||
|
||
|
||
app = Flask(__name__) | ||
|
||
executor = ThreadPoolExecutor(max_workers=5) | ||
|
||
|
||
@app.route("/execute", methods=["POST"]) | ||
def execute(): | ||
data = request.json | ||
|
||
problem = data.get("problem", "") | ||
completion = data.get("completion", "") | ||
timeout = data.get("timeout", 5.0) | ||
args = (problem, completion, timeout) | ||
|
||
if not completion: | ||
response = jsonify({"error": "No completion provided"}) | ||
response.status_code = HTTPStatus.BAD_REQUEST | ||
return response | ||
|
||
try: | ||
future = executor.submit(check_correctness, problem, completion, timeout) | ||
result = future.result() | ||
return jsonify({"result": result}) | ||
except Exception as e: | ||
response = jsonify({"error": str(e)}) | ||
response.status_code = HTTPStatus.INTERNAL_SERVER_ERROR | ||
return response | ||
|
||
|
||
# check if a 500 error code is thrown | ||
@app.errorhandler(500) | ||
def internal_error(error): | ||
return "500 error: {}".format(str(error)), 500 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
openai | ||
git+https://github.com/arunpatro/human-eval.git@pipgit | ||
Flask |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.