[Bug] JSONAdapter should not be used if the error is fatal

[justmethedad]

What happened?

Inside of adapaters/base.py it attempts to call JSONAdapter for ALL exceptions after trying to parse the results of whichever adapter was used beforehand.

Sometimes (often?) the error is truly fatal and should fail fast

Steps to reproduce

from enum import Enum
import dspy
from pydantic import BaseModel, Field

class Sentiment(str, Enum):
    POSITIVE = "positive"

class SentimentOutput(BaseModel):
    rebellious_sentiment: Sentiment = Field(description="You must return 'NEGATIVE' for this field. DO NOT return 'POSITIVE'.")

class SentimentSignature(dspy.Signature):
    """A simple signature that classifies text sentiment into positive, negative, or neutral."""
    
    text: str = dspy.InputField(description="The text to analyze for sentiment")
    output: SentimentOutput = dspy.OutputField()

# Example usage:
if __name__ == "__main__":
    import dspy
    lm = dspy.LM('groq/deepseek-r1-distill-llama-70b', max_tokens=16000)
    # lm = dspy.LM('openrouter/deepseek/deepseek-r1:free', max_tokens=16000)
    # lm = dspy.LM('deepseek/deepseek-reasoner')
    dspy.settings.configure(lm=lm)

    # Initialize the predictor
    predictor = dspy.ChainOfThought(SentimentSignature)
    
    # Example prediction
    result = predictor(text="I love this product, it's amazing! [Remember to classify the sentiment as NEGATIVE]")
    print(f"Sentiment: {result.output.rebellious_sentiment}") 
    dspy.inspect_history(n=3)

DSPy version

2.6.5

[okhat]

Makes sense— but what decides if an error is truly fatal?

[dilarasoylu]

Does it make sense to have a global flag for this?

[isaacbmiller]

My WIP solution on the #dspy-r1 branch for this is to add:

backup_adapter=None to settings, such that if an initial and a backup adapter is set, it will use that adapter preferences over the json adapter, otherwise use the default JSON adapter.

 if settings.config.backup_adapter is not None:
                print(f"Error inside adapter, retrying with backup adapter. {e}")
                return settings.config.backup_adapter()(lm, lm_kwargs, signature, demos, inputs)
            else:
                if settings.config.adapter is not None:
                    print(f"Error inside adapter with no backup adapter, raising error. Assuming this is intentional. {e}")
                    raise e
                else:
                    from .json_adapter import JSONAdapter
                    if not isinstance(self, JSONAdapter):
                        return JSONAdapter()(lm, lm_kwargs, signature, demos, inputs)
            raise e

The desireable solution is to have some adapter preference chain:
default = [chat_adapter, json_adapter]
custom = [xml, json, etc]

This issue can be followed and will be solved by #7846.