PR: Add Support for Groq's Hosted Models via Groq's OpenAI Compatibility API

[jackspirou]

• Add Groq model options via Groq's OpenAI API compatibility
• Update documentation accordingly
• Fix example

Fixes #682

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.46%. Comparing base (699f397) to head (e85e425).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #683      +/-   ##
==========================================
+ Coverage   98.44%   98.46%   +0.01%     
==========================================
  Files          24       24              
  Lines        1353     1364      +11     
==========================================
+ Hits         1332     1343      +11     
  Misses         15       15              
  Partials        6        6              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[sashabaranov]

@jackspirou thank you for the PR!

I don't think Groq needs any special treatment, as there are multiple services with OpenAI-compatible API now.

It requires a minimal four-line change to the basic example we have in README to make it work with Groq (or any other service like Ollama for that matter). I'm totally open to expanding our README with such an example, though!

--- basic.go	2024-03-15 14:40:43
+++ groq.go	2024-03-15 14:40:29
@@ -7,11 +7,14 @@
 )

 func main() {
-	client := openai.NewClient("your token")
+	config := openai.DefaultConfig("token")
+	config.BaseURL = "https://api.groq.com/openai/v1"
+	client := openai.NewClientWithConfig(config)
+
 	resp, err := client.CreateChatCompletion(
 		context.Background(),
 		openai.ChatCompletionRequest{
-			Model: openai.GPT3Dot5Turbo,
+			Model: "mixtral-8x7b-32768",
 			Messages: []openai.ChatCompletionMessage{
 				{
 					Role:    openai.ChatMessageRoleUser,```

[jackspirou]

@sashabaranov - Thanks for the response and thanks for the example! I didn't understand the forest for the trees here. I agree with your assessment above, given the example you shared.

I'll update this PR to take a stab at a simple readme update per your advice.

One side-note - it might be nice to update the pattern of:
openai.NewClient(authToken string) to something like
openai.NewClient(authToken string, optons ...func(*Client)).

I think it might have provided some hints for me to see the forest for the trees in this case. Maybe it's something you've already considered.

[donvito]

was this released?