Wire Apple On-Device's tool-calling path into ChatViewModel (steps 4+6)

Fourth and sixth pieces of the Apple Intelligence tool-calling work
(see /Users/rune/.claude/plans/apple-intelligence-tool-calling-plan.md).

generateAppleOnDeviceToolResponse() is a new, dedicated dispatch path
— routed to instead of the generic generateAIResponseWithTools() when
currentProvider == .appleOnDevice, since LanguageModelSession's
persistent, framework-driven tool loop doesn't fit the stateless,
manually-looped chatWithToolMessages contract every other provider
uses. Builds the tool list + system prompt once per turn, calls
AppleFoundationProvider.respondWithTools() exactly once, and renders
the result through the exact same showSystemMessage/
flushToolCallSummary/updateToolCallMessage path every other provider's
tool loop already uses, so the "🔧 Calling: …" UI looks identical
regardless of which provider is actually running.

New appleSessionKey gives Apple On-Device a stable per-conversation
identity even before a chat is ever saved (currentConversationId is
nil until first ⌘S) — an ephemeral UUID regenerated on New Chat and
Clear Chat, which also now explicitly resets the cached tool session
on both of those actions so a stale session is never reused for what
the user sees as a different conversation.

Dormant until step 7 flips capabilities.tools for Apple On-Device —
modelSupportTools is still false today, so this new dispatch branch
never actually fires yet. 425 tests, stable across two consecutive
full runs, zero regressions in the existing dispatch/reset paths this
touched.
This commit is contained in:
2026-08-28 11:39:27 +02:00
parent 24bf23e71b
commit c7681d227f
+131 -2
View File
@@ -112,6 +112,15 @@ private final class ConversationSaveAccessory: NSObject {
}
#endif
/// Accumulates `ToolCallDetail`s reported by `AppleToolCallDidFinish` callbacks across a single
/// Apple On-Device tool-calling turn. A reference type because the callback is `@escaping` and may
/// fire after the `Task` that created it has suspended at an `await` a bare local `var` isn't safe
/// to mutate from an escaping closure captured that way.
@MainActor
private final class ToolCallDetailCollector {
var details: [ToolCallDetail] = []
}
@Observable
@MainActor
class ChatViewModel {
@@ -157,6 +166,16 @@ class ChatViewModel {
var currentConversationName: String? = nil
private var savedMessageCount: Int = 0
// Identity key for Apple On-Device's persistent tool-calling session (AppleFoundationProvider,
// see the Apple Intelligence tool-calling plan §2) distinct from currentConversationId, which
// is nil until the conversation is first saved (S-only persistence, no autosave). Falls back to
// this ephemeral, ChatViewModel-scoped identity for a not-yet-saved chat so the on-device session
// cache still correctly distinguishes "still the same conversation" from "a different one," and
// is regenerated whenever the visible transcript is reset (New Chat, Clear Chat) so a stale
// session is never reused for what the user sees as a different conversation.
private var appleChatSessionId = UUID()
private var appleSessionKey: String { currentConversationId?.uuidString ?? appleChatSessionId.uuidString }
// Per-conversation notes.md (see ConversationNotesService)
var notesEnabled: Bool = false
var notesFilename: String? = nil
@@ -448,6 +467,8 @@ Don't narrate future actions ("Let me...") - just use the tools.
savedMessageCount = 0
notesEnabled = false
notesFilename = nil
appleChatSessionId = UUID()
(providerRegistry.getProvider(for: .appleOnDevice) as? AppleFoundationProvider)?.resetToolSession()
}
/// Re-sync local state from SettingsService (called when Settings sheet dismisses)
@@ -583,6 +604,8 @@ Don't narrate future actions ("Let me...") - just use the tools.
messages.removeAll()
sessionStats.reset()
MCPService.shared.resetBashSessionApproval()
appleChatSessionId = UUID()
(providerRegistry.getProvider(for: .appleOnDevice) as? AppleFoundationProvider)?.resetToolSession()
showSystemMessage("Chat cleared")
}
@@ -1034,8 +1057,18 @@ Don't narrate future actions ("Let me...") - just use the tools.
}
let modelSupportTools = selectedModel?.capabilities.tools ?? false
if modelSupportTools && (anytypeActive || bashActive || personalDataActive || researchAgentsActive || externalMCPActive || mailActive || paperlessActive || (mcpActive && !mcp.allowedFolders.isEmpty)) {
generateAIResponseWithTools(provider: provider, modelId: modelId)
let anyToolActive = anytypeActive || bashActive || personalDataActive || researchAgentsActive || externalMCPActive || mailActive || paperlessActive || (mcpActive && !mcp.allowedFolders.isEmpty)
if modelSupportTools && anyToolActive {
// Apple On-Device gets its own path: LanguageModelSession is a persistent, stateful
// session that runs its own internal tool-call loop inside one .respond(to:) call a
// fundamentally different shape from every other provider's chatWithToolMessages, which
// ChatViewModel drives as a stateless, manually-looped, full-history-replayed-per-call
// conversation. See the Apple Intelligence tool-calling plan, §3.
if currentProvider == .appleOnDevice, let appleProvider = provider as? AppleFoundationProvider {
generateAppleOnDeviceToolResponse(provider: appleProvider, modelId: modelId)
} else {
generateAIResponseWithTools(provider: provider, modelId: modelId)
}
return
}
@@ -1954,6 +1987,102 @@ Don't narrate future actions ("Let me...") - just use the tools.
}
}
/// Apple On-Device's tool-calling path. Distinct from `generateAIResponseWithTools` because
/// `LanguageModelSession` is a persistent, stateful session that runs its own internal
/// multi-round tool-call loop inside one `.respond(to:)` call Confab doesn't drive the loop
/// manually the way it does for every other provider. See the Apple Intelligence tool-calling
/// plan (/Users/rune/.claude/plans/apple-intelligence-tool-calling-plan.md), §2-§5.
private func generateAppleOnDeviceToolResponse(provider: AppleFoundationProvider, modelId: String) {
Log.ui.info("generateAppleOnDeviceToolResponse: model=\(modelId)")
isGenerating = true
streamingTask?.cancel()
let conversationKey = appleSessionKey
let tools = MCPService.shared.getToolSchemas(onlineMode: onlineMode)
let baseInstructions = effectiveSystemPrompt
// Excludes the just-appended current-turn user message only consulted by
// AppleFoundationProvider if it actually has to rebuild the session, for best-effort
// continuity (same text-summary fallback Phase 1's makeSession(for:) already uses).
let priorMessages = Array(messages.dropLast())
let userMessageText = messages.last(where: { $0.role == .user })?.content ?? ""
streamingTask = Task {
let startTime = Date()
var wasCancelled = false
// AppleToolCallDidFinish is an escaping @MainActor closure that may be invoked after
// this Task suspends at the `await` below a bare local `var` isn't safe to mutate from
// it, so collect into a small MainActor-isolated reference type instead (mirrors the
// pattern already proven in AppleDynamicToolTests' CallbackRecorder).
let collector = ToolCallDetailCollector()
let onWillStart: AppleToolCallWillStart = { [weak self] toolName in
self?.currentToolActivity = String(localized: "🔧 Calling: \(toolName)")
}
let onDidFinish: AppleToolCallDidFinish = { detail in
collector.details.append(detail)
}
do {
let (response, didRebuild) = try await provider.respondWithTools(
conversationId: conversationKey,
tools: tools,
baseInstructions: baseInstructions,
priorMessagesForRebuildReplay: priorMessages,
userMessage: userMessageText,
onWillStart: onWillStart,
onDidFinish: onDidFinish
)
if Task.isCancelled { wasCancelled = true }
flushToolCallSummary(collector.details)
if didRebuild {
showSystemMessage("🔄 Tools changed — starting a fresh on-device session")
}
let responseTime = Date().timeIntervalSince(startTime)
let assistantMessage = Message(
role: .assistant,
content: applyNotesUpdateIfNeeded(response.content),
tokens: nil,
cost: nil,
timestamp: Date(),
attachments: nil,
responseTime: responseTime,
wasInterrupted: wasCancelled,
modelId: modelId
)
messages.append(assistantMessage)
isGenerating = false
streamingTask = nil
} catch {
let responseTime = Date().timeIntervalSince(startTime)
flushToolCallSummary(collector.details)
let isCancellation = Task.isCancelled || wasCancelled || error is CancellationError
if isCancellation {
let assistantMessage = Message(
role: .assistant,
content: "",
timestamp: Date(),
responseTime: responseTime,
wasInterrupted: true,
modelId: modelId
)
messages.append(assistantMessage)
} else {
Log.api.error("Apple On-Device tool generation failed: \(error.localizedDescription)")
showSystemMessage("\(Self.friendlyErrorMessage(from: error))")
}
isGenerating = false
streamingTask = nil
}
}
}
@discardableResult
private func showSystemMessage(_ text: String.LocalizationValue) -> UUID {
let message = Message(