Question about Streaming on Assistants API #93
-
|
Hi, See the function name is "§HANGUP" output. But the name should be "HANGUP“. How the "§" comes? it comes up random. Some time dose not appear. Sometimes dose. |
Beta Was this translation helpful? Give feedback.
Answered by
sashirestela
Apr 22, 2024
Replies: 9 comments 3 replies
-
|
@hlzhangxt thanks for using simple-openai. Please, could you share your complete request data? |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Here: |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
And also, to process the function request, I code: See I have to create a chatFunction from Function to run it. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@hlzhangxt I've tried your code several times but I cannot replicate the issue. This is the code: package io.github.sashirestela.openai.playground;
import com.fasterxml.jackson.annotation.JsonPropertyDescription;
import io.github.sashirestela.openai.SimpleOpenAI;
import io.github.sashirestela.openai.domain.assistant.AssistantFunction;
import io.github.sashirestela.openai.domain.assistant.AssistantRequest;
import io.github.sashirestela.openai.domain.assistant.AssistantTool;
import io.github.sashirestela.openai.domain.assistant.Events;
import io.github.sashirestela.openai.domain.assistant.TextContent;
import io.github.sashirestela.openai.domain.assistant.ThreadMessageDelta;
import io.github.sashirestela.openai.domain.assistant.ThreadMessageRequest;
import io.github.sashirestela.openai.domain.assistant.ThreadRequest;
import io.github.sashirestela.openai.domain.assistant.ThreadRunRequest;
import io.github.sashirestela.openai.domain.chat.tool.ChatFunction;
import io.github.sashirestela.openai.function.Functional;
public class ReplicateFunctionNameIssue {
public static void main(String[] args) {
var openai = SimpleOpenAI.builder().apiKey(System.getenv("OPENAI_API_KEY")).build();
var assistantService = openai.assistants();
var threadService = openai.threads();
var assistant = assistantService.create(AssistantRequest.builder()
.model("gpt-4-0125-preview")
.name("Nova Assistant")
.instructions(
"You are a Heights Digital Repair Assistant working for the Heights Real Estate Company . Your name is Nova. You are now answering the tenant call for repair issue. Please response as soon as possible and as short as possible! If you can not get responses from the tenant for more than 2 times, please hangup to end the conversation.\r\nIf the tenant speaks a non-English language, You should speak the tenant's language too. After you answered the tenant call, you do the following steps(Please note that you can skip any step if you had already got the answer information of the step) :\r\n1, Answers with greeting; Ask how are they doing with standard reply.\r\n2, Ask who are they speaking to; Ask the building address and unit#.\r\n After you got the building address from the tenant, the address maybe not exactly same with address in the attached building address json file. So you need to retrieve the right address in the file and confim it with the tenant. And also retrieve the building code. \r\n But if you can not get the right address in the file, ask tenant to speak the address again to check and confirm.\r\n And after you get the super name by invoking the getsuper function, you should tell the tenant the super name.\r\n3, Asks name of person they are speaking to.\r\n4, Confirm identity of person spoken to.\r\n5, Asks what is the service issue.\r\n6, Asks What type of repair is this? Electrical, plumbing, painting or something else?\r\n7, Asks When did problem start.\r\n8, Asks Was this reported to the super. \r\n9, If it was reported to the super then asks the following questions: \r\n Reported on what date?\r\n What happened after it was reported\r\n Asks Was this condition ever inspected?\r\n10. Ask Can I conference in the super right now to help schedule an appointment?\r\n If the tenant agrees, let the tenant on hold please, then start calling( invoke conference super function) the super into the conference.\r\n or else say good bye and goodwill to end the conversation.")
.tool(AssistantTool.builder().function(AssistantFunction.function(ChatFunction.builder()
.name("HANGUP")
.description(
"while caller or called person wants to end the conversation, hangup to disconnect the call")
.functionalClass(HANGUP.class)
.build())).build())
.tool(AssistantTool.builder().function(AssistantFunction.function(ChatFunction.builder()
.name("GETSUPER")
.description(
"After retrieved building address confrimed with the tenant ,the building code from the attached building json file and the unit number with the tenant name the tenant provided, invoke this function to get super's details including name and phone number, and also return the building address, building code and unit number with the tenant name.")
.functionalClass(GETSUPER.class)
.build())).build())
.tool(AssistantTool.builder().function(AssistantFunction.function(ChatFunction.builder()
.name("CONFSUPER")
.description(
"After the tenant agrees that we make an appointment with the super, then we invoke this function to call super to let the super join the conference")
.functionalClass(CONFSUPER.class)
.build())).build())
.tool(AssistantTool.builder().function(AssistantFunction.function(ChatFunction.builder()
.name("SAVEAPPOINTMENTDATE")
.description(
"After confirmed the appointment date and time with the super and the tenant, invoke this function to save them")
.functionalClass(SAVEAPPOINTMENTDATE.class)
.build())).build())
.tool(AssistantTool.builder().function(AssistantFunction.function(ChatFunction.builder()
.name("SETSPANISHLANG")
.description(
"If you have detected that the tenant speaks Spanish, invoke this function")
.functionalClass(SETSPANISHLANG.class)
.build())).build())
.tool(AssistantTool.RETRIEVAL)
.build()).join();
var assistantId = assistant.getId();
System.out.println("Assistant was created with id: " + assistantId);
var thread = threadService.create(ThreadRequest.builder()
.message(ThreadMessageRequest.builder()
.role("user")
.content("Hi, can you help me?")
.build())
.build()).join();
var threadId = thread.getId();
System.out.println("Thread was created with id: " + threadId);
var stream = threadService.createRunStream(threadId, ThreadRunRequest.builder()
.assistantId(assistantId)
.build()).join();
System.out.println("-".repeat(80));
stream.filter(event -> event.getName().equals(Events.THREAD_MESSAGE_DELTA))
.map(event -> ((TextContent) ((ThreadMessageDelta) event.getData())
.getDelta().getContent().get(0)).getValue())
.forEach(System.out::print);
System.out.println();
var threadRun = threadService.getRunList(threadId).join();
threadRun.forEach(System.out::println);
var deletedThread = threadService.delete(threadId).join();
var deletedAssistant = assistantService.delete(assistantId).join();
System.out.println("Thread was deleted: " + deletedThread.getDeleted());
System.out.println("Assistant was deleted: " + deletedAssistant.getDeleted());
}
public static class HANGUP implements Functional {
@Override
public Object execute() {
return "DONE";
}
}
public static class GETSUPER implements Functional {
@JsonPropertyDescription("the confimed building address from the file attached, for example: 61 Main Street")
public String building_addr;
@JsonPropertyDescription("the building code retrieved from the attached building json file ")
public String building_code;
@JsonPropertyDescription("the tenant name the tenant provided")
public String tenant_name;
@JsonPropertyDescription("the unit number the tenant provided")
public String unit_no;
@Override
public Object execute() {
return "DONE";
}
}
public static class CONFSUPER implements Functional {
@Override
public Object execute() {
return "DONE";
}
}
public static class SAVEAPPOINTMENTDATE implements Functional {
@JsonPropertyDescription("the appointment date confirmed with the super and the tenant, the format is mm/dd/yyyy")
public String appmtdate;
@JsonPropertyDescription("the appointment time confirmed with the super and the tenant, the format is hh:mm in 24 hours")
public String appmttime;
@Override
public Object execute() {
return "DONE";
}
}
public static class SETSPANISHLANG implements Functional {
@Override
public Object execute() {
return "DONE";
}
}
}This is the log result, where you can check in the line 246 the function name is correct: I'm afraid that your issue cannot be replicated. Maybe it is something about your local configuration or with the OpenAI model. I would suggest using the latest model gpt-4-turbo-2024-04-09. If you don't have more observations, I think we could close this issue. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Thanks for patience of test. If we don't submit tool ouput of the function result, the stream will stuck in Require_Action. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@hlzhangxt I'm going to convert this issue into a Discussion item, it is more appropriate. In general, if you have questions, use the Discussion section instead of creating an issue. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
yeah, no promblem. |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@hlzhangxt I've prepared an example using the latest version of simple-openai (2.3.0). There you can learn how to handle the scenario that you asked: package io.github.sashirestela.openai.playground;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Stream;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyDescription;
import io.github.sashirestela.cleverclient.Event;
import io.github.sashirestela.openai.OpenAIBeta;
import io.github.sashirestela.openai.SimpleOpenAI;
import io.github.sashirestela.openai.domain.assistant.AssistantFunction;
import io.github.sashirestela.openai.domain.assistant.AssistantRequest;
import io.github.sashirestela.openai.domain.assistant.AssistantTool;
import io.github.sashirestela.openai.domain.assistant.Events;
import io.github.sashirestela.openai.domain.assistant.TextContent;
import io.github.sashirestela.openai.domain.assistant.ThreadMessageDelta;
import io.github.sashirestela.openai.domain.assistant.ThreadMessageRequest;
import io.github.sashirestela.openai.domain.assistant.ThreadMessageRequest.Role;
import io.github.sashirestela.openai.domain.assistant.ThreadRequest;
import io.github.sashirestela.openai.domain.assistant.ThreadRun;
import io.github.sashirestela.openai.domain.assistant.ThreadRunRequest;
import io.github.sashirestela.openai.domain.assistant.ToolOutputSubmission;
import io.github.sashirestela.openai.domain.chat.tool.ChatFunction;
import io.github.sashirestela.openai.function.FunctionExecutor;
import io.github.sashirestela.openai.function.Functional;
public class DemoAssistantFunctionStream {
private SimpleOpenAI openai;
private OpenAIBeta.Assistants assistantService;
private OpenAIBeta.Threads threadService;
private FunctionExecutor functionExecutor;
private String assistantId;
private String threadId;
public DemoAssistantFunctionStream() {
openai = SimpleOpenAI.builder().apiKey(System.getenv("OPENAI_API_KEY")).build();
assistantService = openai.assistants();
threadService = openai.threads();
functionExecutor = new FunctionExecutor();
}
public void prepareConversation() {
List<ChatFunction> functionList = new ArrayList<>();
functionList.add(ChatFunction.builder()
.name("getCurrentTemperature")
.description("Get the current temperature for a specific location")
.functionalClass(CurrentTemperature.class)
.build());
functionList.add(ChatFunction.builder()
.name("getRainProbability")
.description("Get the probability of rain for a specific location")
.functionalClass(RainProbability.class)
.build());
functionExecutor.enrollFunctions(functionList);
var assistant = assistantService.create(AssistantRequest.builder()
.name("World Assistant")
.model("gpt-4-turbo")
.instructions(
"You are a skilled tutor on geo-politic topics.")
.tool(AssistantTool.builder().function(AssistantFunction.function(functionList.get(0))).build())
.tool(AssistantTool.builder().function(AssistantFunction.function(functionList.get(1))).build())
.build()).join();
assistantId = assistant.getId();
System.out.println("Assistant was created with id: " + assistantId);
var thread = threadService.create(ThreadRequest.builder().build()).join();
threadId = thread.getId();
System.out.println("Thread was created with id: " + threadId);
System.out.println();
}
public void runConversation() {
var myMessage = "Hi, can you help me?";
System.out.println(myMessage);
while (!myMessage.toLowerCase().equals("exit")) {
threadService.createMessage(threadId, ThreadMessageRequest.builder()
.role(Role.USER)
.content(myMessage)
.build());
var runStream = threadService.createRunStream(threadId, ThreadRunRequest.builder()
.assistantId(assistantId)
.build()).join();
handleRunEvents(runStream);
myMessage = System.console().readLine("\nAsk a question (write 'exit' to finish):");
}
}
private void handleRunEvents(Stream<Event> runStream) {
runStream.forEach(event -> {
switch (event.getName()) {
case Events.THREAD_RUN_CREATED:
case Events.THREAD_RUN_COMPLETED:
case Events.THREAD_RUN_REQUIRES_ACTION:
var run = (ThreadRun) event.getData();
System.out.println("=====>> Thread Run: id=" + run.getId() + ", status=" + run.getStatus());
if (run.getStatus().equals("requires_action")) {
var toolCalls = run.getRequiredAction().getSubmitToolOutputs().getToolCalls();
var toolOutputs = functionExecutor.executeAll(toolCalls);
var runSubmitToolStream = threadService.submitToolOutputsStream(threadId, run.getId(),
ToolOutputSubmission.builder()
.toolOutputs(toolOutputs)
.stream(true)
.build())
.join();
handleRunEvents(runSubmitToolStream);
}
break;
case Events.THREAD_MESSAGE_DELTA:
var msgDelta = (ThreadMessageDelta) event.getData();
var content = msgDelta.getDelta().getContent().get(0);
if (content instanceof TextContent) {
var textContent = (TextContent) content;
System.out.print(textContent.getValue());
}
break;
case Events.THREAD_MESSAGE_COMPLETED:
System.out.println();
break;
default:
break;
}
});
}
public void cleanConversation() {
var deletedThread = threadService.delete(threadId).join();
var deletedAssistant = assistantService.delete(assistantId).join();
System.out.println("Thread was deleted: " + deletedThread.getDeleted());
System.out.println("Assistant was deleted: " + deletedAssistant.getDeleted());
}
public static void main(String[] args) {
var demo = new DemoAssistantFunctionStream();
demo.prepareConversation();
demo.runConversation();
demo.cleanConversation();
}
public static class CurrentTemperature implements Functional {
@JsonPropertyDescription("The city and state, e.g., San Francisco, CA")
@JsonProperty(required = true)
public String location;
@JsonPropertyDescription("The temperature unit to use. Infer this from the user's location.")
@JsonProperty(required = true)
public String unit;
@Override
public Object execute() {
return Math.random() * 45;
}
}
public static class RainProbability implements Functional {
@JsonPropertyDescription("The city and state, e.g., San Francisco, CA")
@JsonProperty(required = true)
public String location;
@Override
public Object execute() {
return Math.random() * 100;
}
}
}The output of that code is: |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
@hlzhangxt Thanks for using simple-openai. If you think this project is valuable to you, there are a few ways to show us your love:
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks. |
Beta Was this translation helpful? Give feedback.
Answer selected by
sashirestela
-
|
@hlzhangxt First, ChatCompletions does not handle submitToolOutputs, so, I suppose that you are asking for an example to handle ToolCalls with Functions in streaming mode. I've prepared the example and I had to make some minor changes to the library in order to support it. Use the latest version of simple-openai. The example is similar to the previous example, a chat waiting for user messages and using two functions about temperature and rain probability: package io.github.sashirestela.openai.playground;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.fasterxml.jackson.annotation.JsonPropertyDescription;
import io.github.sashirestela.openai.OpenAI;
import io.github.sashirestela.openai.SimpleOpenAI;
import io.github.sashirestela.openai.domain.chat.ChatRequest;
import io.github.sashirestela.openai.domain.chat.ChatResponse;
import io.github.sashirestela.openai.domain.chat.Choice;
import io.github.sashirestela.openai.domain.chat.message.ChatMsg;
import io.github.sashirestela.openai.domain.chat.message.ChatMsgAssistant;
import io.github.sashirestela.openai.domain.chat.message.ChatMsgResponse;
import io.github.sashirestela.openai.domain.chat.message.ChatMsgSystem;
import io.github.sashirestela.openai.domain.chat.message.ChatMsgTool;
import io.github.sashirestela.openai.domain.chat.message.ChatMsgUser;
import io.github.sashirestela.openai.domain.chat.tool.ChatFunction;
import io.github.sashirestela.openai.domain.chat.tool.ChatToolCall;
import io.github.sashirestela.openai.function.FunctionExecutor;
import io.github.sashirestela.openai.function.Functional;
import java.util.ArrayList;
import java.util.List;
import java.util.stream.Stream;
public class DemoChatCompletionFunctionStream {
private SimpleOpenAI openai;
private OpenAI.ChatCompletions chatService;
private FunctionExecutor functionExecutor;
private List<ChatMsg> messages;
private int indexTool;
private StringBuilder content;
private StringBuilder functionArgs;
public DemoChatCompletionFunctionStream() {
openai = SimpleOpenAI.builder().apiKey(System.getenv("OPENAI_API_KEY")).build();
chatService = openai.chatCompletions();
functionExecutor = new FunctionExecutor();
messages = new ArrayList<>();
}
public void prepareConversation() {
List<ChatFunction> functionList = new ArrayList<>();
functionList.add(ChatFunction.builder()
.name("getCurrentTemperature")
.description("Get the current temperature for a specific location")
.functionalClass(CurrentTemperature.class)
.build());
functionList.add(ChatFunction.builder()
.name("getRainProbability")
.description("Get the probability of rain for a specific location")
.functionalClass(RainProbability.class)
.build());
functionExecutor.enrollFunctions(functionList);
messages.add(new ChatMsgSystem("You are a skilled tutor on geo-politic topics."));
}
public void runConversation() {
var userMessage = "Hi, can you help me?";
System.out.println(userMessage);
messages.add(new ChatMsgUser(userMessage));
while (!userMessage.toLowerCase().equals("exit")) {
var chatResponseStream = chatService.createStream(ChatRequest.builder()
.model("gpt-4-turbo")
.messages(messages)
.tools(functionExecutor.getToolFunctions())
.temperature(0.2)
.stream(true)
.build()).join();
indexTool = -1;
content = new StringBuilder();
functionArgs = new StringBuilder();
var response = getResponse(chatResponseStream);
if (response.getMessage().getContent() != null) {
messages.add(new ChatMsgAssistant(response.getMessage().getContent()));
}
if (response.getFinishReason().equals("tool_calls")) {
messages.add(response.getMessage());
for (var chatToollCall : response.getMessage().getToolCalls()) {
var result = functionExecutor.execute(chatToollCall.getFunction());
messages.add(new ChatMsgTool(result.toString(), chatToollCall.getId()));
}
continue;
}
userMessage = System.console().readLine("\n\nAsk a question (or write 'exit' to finish): ");
messages.add(new ChatMsgUser(userMessage));
}
}
private Choice getResponse(Stream<ChatResponse> chatResponseStream) {
var choice = new Choice();
choice.setIndex(0);
var chatMsgResponse = new ChatMsgResponse();
List<ChatToolCall> toolCalls = new ArrayList<>();
chatResponseStream.forEach(chatResponseChunk -> {
var innerChoice = chatResponseChunk.getChoices().get(0);
var delta = innerChoice.getMessage();
if (delta.getRole() != null) {
chatMsgResponse.setRole(delta.getRole());
} else if (delta.getContent() != null && !delta.getContent().isEmpty()) {
content.append(delta.getContent());
System.out.print(delta.getContent());
} else if (delta.getToolCalls() != null) {
var toolCall = delta.getToolCalls().get(0);
if (toolCall.getIndex() != indexTool) {
if (toolCalls.size() > 0) {
toolCalls.get(toolCalls.size() - 1).getFunction().setArguments(functionArgs.toString());
functionArgs = new StringBuilder();
}
toolCalls.add(toolCall);
indexTool++;
} else {
functionArgs.append(toolCall.getFunction().getArguments());
}
} else {
if (content.length() > 0) {
chatMsgResponse.setContent(content.toString());
}
if (toolCalls.size() > 0) {
toolCalls.get(toolCalls.size() - 1).getFunction().setArguments(functionArgs.toString());
chatMsgResponse.setToolCalls(toolCalls);
}
choice.setMessage(chatMsgResponse);
choice.setFinishReason(innerChoice.getFinishReason());
}
});
return choice;
}
public static void main(String[] args) {
var demo = new DemoChatCompletionFunctionStream();
demo.prepareConversation();
demo.runConversation();
}
public static class CurrentTemperature implements Functional {
@JsonPropertyDescription("The city and state, e.g., San Francisco, CA")
@JsonProperty(required = true)
public String location;
@JsonPropertyDescription("The temperature unit to use. Infer this from the user's location.")
@JsonProperty(required = true)
public String unit;
@Override
public Object execute() {
return Math.random() * 45;
}
}
public static class RainProbability implements Functional {
@JsonPropertyDescription("The city and state, e.g., San Francisco, CA")
@JsonProperty(required = true)
public String location;
@Override
public Object execute() {
return Math.random() * 100;
}
}
}The output is: If you have more questions, I suggest you create another entry on the Discussion section. Well, after all this work to help you, I really hope you could write a brief review on your social networks (perhaps Linkedin) about your experience with simple-openai and tag me, could you? |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
No problems. Thanks. |
Beta Was this translation helpful? Give feedback.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@hlzhangxt I've prepared an example using the latest version of simple-openai (2.3.0). There you can learn how to handle the scenario that you asked: