Repository navigation
Conversation
fixes google#156) In Vertex AI, multi-turn tool-calling requests fail with HTTP 400 ('Requests ending with a model turn are not supported') when a tool result (ToolMessage) is followed by new screen observation text (HumanMessage). LangChain's Vertex serializer merges consecutive user-turn messages into a single Content object with parts ordered as [functionResponse, text], which Vertex Gemini 3.x rejects. This normalizes outbound message copies in RobustChatModelWrapper and acomplete for Vertex AI endpoints by positioning HumanMessage observations before ToolMessages within tool turns, yielding [text, functionResponse] part ordering and populating tool names on ToolMessages from preceding AIMessage tool calls. The underlying stored conversation history remains intact and unchanged.
There was a problem hiding this comment.
Code Review
This pull request introduces message normalization for Vertex AI Gemini models to prevent HTTP 400 errors caused by incorrect ordering of tool responses and user observations. It adds a normalize_vertex_messages utility, integrates it into the robust chat model wrapper and completion functions, and includes comprehensive unit tests. The review feedback suggests using getattr to safely access tool_call_id on messages to avoid potential AttributeErrors, and extracting the duplicated input preparation logic into a static helper method to improve maintainability.
…s tool_call_id Addresses review comments from @gemini-code-assist: - Extract duplicated input preparation logic into RobustChatModelWrapper._prepare_vertex_inputs static method - Use getattr(m, 'tool_call_id', None) on ToolMessage to avoid potential AttributeError with mock objects
Author
|
Thank you for the review! I've addressed the feedback in commit
|
Author
|
@gemini-code-assist please review the latest updates in commit |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #156.
Problem
In ARTEMIS's mobile workflows using Vertex AI Gemini (e.g.
gemini-3.8-flash), the first model turn succeeds and executes a UI action. However, when ARTEMIS sends the follow-up request with the tool result and the next screen observation, Vertex AI returns:This occurs because ARTEMIS appends a
ToolMessage(tool execution result) followed by aHumanMessage(new screen observation) to its conversation history. LangChain's Vertex serializer (_parse_chat_history_gemini) convertsToolMessagetofunctionResponseand merges consecutive user-turn messages into a singleContent(role="user")object with parts ordered as[functionResponse, text]. Vertex Gemini 3.x rejects this part ordering.Solution
normalize_vertex_messages):artemis/services/llm.py, produces a normalized copy of the message list for Vertex AI invocations whereHumanMessageobservations in a tool-calling turn are positioned beforeToolMessageresults.ToolMessage.namefrom the precedingAIMessage.tool_callsmap if missing.TranscriptLedgerandStatewithout mutation.RobustChatModelWrapper(complete(),ainvoke(),invoke()) andacomplete()to automatically detect Vertex AI endpoints (is_vertex_providerorChatVertexAIclass name) and apply the normalization before request execution.is_vertex_provider()inartemis.llm.googleto cleanly check for Vertex AI provider configurations.Validation
tests/unit/test_vertex_message_normalizer.py(7 tests) covering:_parse_chat_history_geminipart ordering verification ([text, function_response]).RobustChatModelWrapperandacompletegateway integration.test_vertex_message_normalizer.py,test_google_provider.py, andtest_llm_gateway.py.ruff checkandruff format --checkpassed cleanly across all changed files.