Repository navigation
Bug: An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id' #256
Copy link
Copy link
Open
Labels
bugSomething isn't workingSomething isn't workingtriageNeeds triage from engineering teamNeeds triage from engineering team
Description
Describe the bug
Got the following error using Semantic Kernel in Java with automatic function calling using 4o* OpenAI models: "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'".
To Reproduce
Steps to reproduce the behavior:
- Unzip provided example, build it and execute it:
mvn clean install
export OPENAI_API_KEY=sk-...
java -jar target/lights-example-1.0-SNAPSHOT.jar- Enter two consecute inputs: "get the lights" and "turn them on"
- See error
SLF4J(W): No SLF4J providers were found.
SLF4J(W): Defaulting to no-operation (NOP) logger implementation
SLF4J(W): See https://www.slf4j.org/codes.html#noProviders for further details.
User > get the lights
Getting lights
Assistant > Here are the current states of the lights:
1. **Table Lamp**: Off
2. **Porch Light**: Off
3. **Chandelier**: On
User > turn them on
Exception in thread "main" com.azure.core.exception.HttpResponseException: Status code 400, "{
"error": {
"message": "An assistant message with 'tool_calls' must be followed by tool messages responding to each 'tool_call_id'. The following tool_call_ids did not have response messages: call_Mor7WOBzlb1Ao9ryIczup9mS, call_ejzgT6RAATaWBqmJbrzbzAMP",
"type": "invalid_request_error",
"param": "messages",
"code": null
}
}"
at com.azure.core.implementation.http.rest.RestProxyBase.instantiateUnexpectedException(RestProxyBase.java:388)
at com.azure.core.implementation.http.rest.AsyncRestProxy.lambda$ensureExpectedStatus$1(AsyncRestProxy.java:135)
at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:113)
at reactor.core.publisher.MonoUsing$MonoUsingSubscriber.onNext(MonoUsing.java:231)
at reactor.core.publisher.FluxHandle$HandleSubscriber.onNext(FluxHandle.java:126)
at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onNext(FluxMap.java:224)
at reactor.core.publisher.FluxDoFinally$DoFinallySubscriber.onNext(FluxDoFinally.java:113)
at reactor.core.publisher.FluxHandleFuseable$HandleFuseableSubscriber.onNext(FluxHandleFuseable.java:191)
at reactor.core.publisher.FluxContextWrite$ContextWriteSubscriber.onNext(FluxContextWrite.java:107)
at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1839)
at reactor.core.publisher.MonoCollectList$MonoCollectListSubscriber.onComplete(MonoCollectList.java:129)
at reactor.core.publisher.FluxPeek$PeekSubscriber.onComplete(FluxPeek.java:260)
at reactor.core.publisher.FluxMap$MapSubscriber.onComplete(FluxMap.java:144)
at reactor.netty.channel.FluxReceive.onInboundComplete(FluxReceive.java:415)
at reactor.netty.channel.ChannelOperations.onInboundComplete(ChannelOperations.java:439)
at reactor.netty.channel.ChannelOperations.terminate(ChannelOperations.java:493)
at reactor.netty.http.client.HttpClientOperations.onInboundNext(HttpClientOperations.java:789)
at reactor.netty.channel.ChannelOperationsHandler.channelRead(ChannelOperationsHandler.java:114)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at com.azure.core.http.netty.implementation.AzureSdkHandler.channelRead(AzureSdkHandler.java:224)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:346)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:318)
at io.netty.channel.CombinedChannelDuplexHandler.channelRead(CombinedChannelDuplexHandler.java:251)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.handler.ssl.SslHandler.unwrap(SslHandler.java:1475)
at io.netty.handler.ssl.SslHandler.decodeNonJdkCompatible(SslHandler.java:1349)
at io.netty.handler.ssl.SslHandler.decode(SslHandler.java:1389)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:530)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:469)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1407)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:918)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:788)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:724)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:650)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:562)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:994)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:1583)
Suppressed: java.lang.Exception: #block terminated with an error
at reactor.core.publisher.BlockingSingleSubscriber.blockingGet(BlockingSingleSubscriber.java:100)
at reactor.core.publisher.Mono.block(Mono.java:1742)
at org.example.LightsExample.main(LightsExample.java:86)
Expected behavior
The lights should be turned on.
Platform
- OS: Mac M1
- IDE: IntelliJ Idea or command line
- Language: Java
- Source: Maven repository, version 1.3.0
Additional context
The problem seems to be reproduced with models 4o and 4o-mini, but not with "gpt-3.5-turbo-0125". Change constant MODEL_ID in class LightsExample to see the difference.lights-example.zip
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingtriageNeeds triage from engineering teamNeeds triage from engineering team
Activity
d3r3kk commentedon Oct 30, 2024
John (@johnoliver) does this have anything to do with our underlying Azure OpenAI connection/SDK?
johnoliver commentedon Oct 31, 2024
It is likely a bug in the failure to invoke the requested tool, will take a look
johnoliver commentedon Nov 11, 2024
So I did recreate a few failures with this scenario, mostly around the tool invocation not using the correct types for tool arguments, with #263 the sample works for me, let me know if it works for you after merge