Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

empty prediction #6

Open
venuv opened this issue Apr 15, 2023 · 3 comments
Open

empty prediction #6

venuv opened this issue Apr 15, 2023 · 3 comments

Comments

@venuv
Copy link

venuv commented Apr 15, 2023

@kjappelbaum in various runs of the example you provided, I get a prediction of either None, or in this case a silent return. any tips on debugging. my openai_api_key seems to not be an issue - TIA, Venu
Screenshot 2023-04-15 at 5 30 37 PM

@bmiles
Copy link
Contributor

bmiles commented Apr 17, 2023

I created this method which fixed my use case in fine tuning on binary 'Active' or 'Inactive' labels:

# Import the original class from the external module
from gptchem.extractor import BaseExtractor

class ClassificationExtractorStrings(BaseExtractor):
    """Extract strings from completions of classification tasks."""
    def extract(self, data, **kwargs) -> str:
        return self.split(data).strip()

@U0M0Z
Copy link

U0M0Z commented Apr 26, 2023

@kjappelbaum I've encountered the same issue. I've tried running both the Classification and Regression tasks in the "High-level wrapper" section and got empty predictions in both scenarios. There shouldn't be any issues with the OpenAI key since I've been using it already for other tasks. I can also track the requests to the API in the usage section of OpenAI platform.

Regression task:

gpt-chem-error-1

Classification task

gpt-chem-error-2

@kjappelbaum
Copy link
Owner

Thanks for all the reports; I have been busy with my Ph.D. defense but have been working on things to make all of this more robust, and hope to push them soon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants