Skip to content

Issues: UKGovernmentBEIS/inspect_ai

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

44 Open 294 Closed
44 Open 294 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Use the anyio compatibility framework, so downstream integrations can use asyncio or trio?
#1423 opened Feb 27, 2025 by Zac-HD
1
Real-time model output evaluation failed, moved to dummy solver in AI Inspect
#1419 opened Feb 27, 2025 by sudarshan-kamath
3
Support Claude Sonnet 3.7 computer use
#1418 opened Feb 27, 2025 by AlexLloyd0
1
support for custom command line parameters (e.g. -U foo=bar)
#1402 opened Feb 26, 2025 by jjallaire-aisi
1
inspect view does not know how to render tool calls with arguments of type dict[str, str]
#1387 opened Feb 25, 2025 by JasonGross
@dragonstyle
Filter by sample metadata in log viewer
#1378 opened Feb 24, 2025 by tadamcz
Search message contents in log viewer
#1377 opened Feb 24, 2025 by tadamcz
The "Value" type in score is overly restrictive for dictonaries
#1333 opened Feb 17, 2025 by max-kaufmann
@dragonstyle
4
Side by side comparison of two models
#1327 opened Feb 16, 2025 by yassersouri
@dragonstyle
5
Scorers which don't return a numeric Score cause crashes in the log viewer
#1324 opened Feb 15, 2025 by max-kaufmann
2
[feature request] progress of forked solvers / subtasks are not visible in terminal viewer
#1296 opened Feb 12, 2025 by skinnerjc
2
[feature request] set default cache policy
#1292 opened Feb 11, 2025 by kaifronsdal
1
MATH dataset used for mathematics tutorial no longer accessible
#1283 opened Feb 10, 2025 by ChiWilliams
1
improved retry behavior
#1277 opened Feb 10, 2025 by jjallaire
1
Custom Displays
#1267 opened Feb 7, 2025 by Baidicoot
1
Log viewer: scroll bar resets when using the scorer drop-down
#1220 opened Jan 30, 2025 by tadamcz
@dragonstyle
[Feature request] Support for LiteLLM and LiteLLM proxy
#1213 opened Jan 30, 2025 by danrr
Execute --runslow tests in CI
#1212 opened Jan 30, 2025 by jjallaire-aisi
@art-dsit
1
Tutorial: Mathematics dataset not available in HF anymore
#1183 opened Jan 24, 2025 by guiem
Broader notion of retryable exception
#1174 opened Jan 22, 2025 by tadamcz
10
Feature Request: Grouped output metrics & exportable result files for benchmarks
#1172 opened Jan 22, 2025 by owenparsons
@dragonstyle
3
Enable sandboxes to connect to existing Docker or k8s instances
#1153 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
Enforcing cost limits
#1152 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
2
Broader notions of "model" for evals
#1149 opened Jan 19, 2025 by jjallaire
@jjallaire
Bridge to convert from compose.yaml to HELM values
#1148 opened Jan 19, 2025 by jjallaire
@craigwalton-dsit
1
ProTip! What’s not been updated in a month: updated:<2025-01-28.