mirror of
https://github.com/ggerganov/llama.cpp
synced 2026-03-07 23:59:26 +01:00
3dadc88b58
* common : fix Step-3.5-Flash format detection and thinking support Step-3.5-Flash uses the same XML-style tool call format as Qwen3-Coder (<tool_call><function=...><parameter=...>) but its Jinja template lacks the bare <function> and plural <parameters> markers that the detection logic previously required. This caused it to fall through to Hermes 2 Pro, which doesn't call func_args_not_string(), so arguments stayed as JSON strings and templates using arguments|items crashed. Additionally, the Qwen3-Coder-XML format handler had no thinking support. Models like Step-3.5-Flash that unconditionally emit <think> in their generation prompt need the same thinking_forced_open handling that Nemotron v3 and Hermes 2 Pro already have, otherwise reasoning_content is never separated from content in API responses. Changes: - Relax Qwen3-Coder XML detection to only require the 3 shared markers - Tighten Nemotron v3 branch to also require bare <function> and plural <parameters>, preventing Step-3.5-Flash from being misrouted via <think> - Add thinking_forced_open support to Qwen3-Coder-XML init function - Add <think>/</think> to preserved tokens - Fix build_grammar_xml_tool_call to handle thinking_forced_open in the grammar root rule, allowing </think> before tool calls - Add Step-3.5-Flash chat template and format detection test Builds on: https://github.com/ggml-org/llama.cpp/pull/19283 * chat : route Step-3.5-Flash to Nemotron v3 PEG parser, add tests Step-3.5-Flash uses the same XML tool call format as Qwen3-Coder and Nemotron 3 Nano (<tool_call>/<function=...>/<parameter=...>) but with unconditional <think> output. Route it to the Nemotron v3 PEG parser for streaming and schema-aware parameter parsing. Detection: templates with <think> + XML tool tags use Nemotron v3 PEG parser; templates without <think> (Qwen3-Coder) use GBNF grammar. Tests cover: basic messages, tool calls with/without thinking content, parallel tool calls, code string parameters, optional </parameter> closing tags, and JSON schema response format. * chat : remove dead thinking code from qwen3_coder_xml Remove thinking handling code that became unreachable after routing Step-3.5-Flash to the Nemotron v3 PEG parser. Qwen3-Coder has no <think> in its template, so the thinking_forced_open logic, preserved tokens, and grammar prefix were dead paths. |
||
|---|---|---|
| .. | ||
| jinja | ||
| arg.cpp | ||
| arg.h | ||
| base64.hpp | ||
| build-info.cpp.in | ||
| chat-parser-xml-toolcall.cpp | ||
| chat-parser-xml-toolcall.h | ||
| chat-parser.cpp | ||
| chat-parser.h | ||
| chat-peg-parser.cpp | ||
| chat-peg-parser.h | ||
| chat.cpp | ||
| chat.h | ||
| CMakeLists.txt | ||
| common.cpp | ||
| common.h | ||
| console.cpp | ||
| console.h | ||
| debug.cpp | ||
| debug.h | ||
| download.cpp | ||
| download.h | ||
| http.h | ||
| json-partial.cpp | ||
| json-partial.h | ||
| json-schema-to-grammar.cpp | ||
| json-schema-to-grammar.h | ||
| llguidance.cpp | ||
| log.cpp | ||
| log.h | ||
| ngram-cache.cpp | ||
| ngram-cache.h | ||
| ngram-map.cpp | ||
| ngram-map.h | ||
| ngram-mod.cpp | ||
| ngram-mod.h | ||
| peg-parser.cpp | ||
| peg-parser.h | ||
| preset.cpp | ||
| preset.h | ||
| regex-partial.cpp | ||
| regex-partial.h | ||
| sampling.cpp | ||
| sampling.h | ||
| speculative.cpp | ||
| speculative.h | ||
| unicode.cpp | ||
| unicode.h | ||