Dynamic Tool Selection for Claude Agents (GUI-based ‘Skill’ Management)

When Claude starts opening files, clicking buttons, or firing off shell commands, the terminal window you used during prototyping vanishes the moment you need to see *what* actually happened. One mis‑typed path, a stalled network request, or an unexpected 429 response can leave your desktop in a limbo state, and you have no visual cue to intervene. The result? A silent “agent gone rogue” episode that’s hard to debug and even harder to trust in production.

Building a thin GUI around Claude’s new Computer‑Use API solves that problem. A palette of selectable skills, live streaming of Claude’s thoughts, and per‑tool status bars give you the real‑time observability you miss in a pure CLI. Below is a production‑grade walk‑through that takes you from a fresh virtualenv to a frozen‑executable, with every threading pitfall and partial‑failure scenario ironed out.

Dynamic tool selection for Claude Agents via a GUI

Dynamic tool selection for Claude Agents via a GUI involves building a desktop interface where users can visually manage and trigger specialized “skills” or tools. In 2026, this is done using the Anthropic Python SDK, a GUI framework like PyQt6, and a thread‑safe control layer that connects UI events to Claude’s API, parses Tool‑Use blocks, executes functions, and streams results back to the interface.

Why GUI‑based Skill Management is the Future for Claude Agents

Beyond the Chat Terminal: The Need for Interactive Control

A chat terminal gives you a linear log of prompts and responses, but no way to pause, cancel, or inspect the side‑effects of a running tool. Modern Claude agents can launch browsers, edit files, and call custom Python functions. Without a UI that surfaces those side‑effects, you’re flying blind.

Lifecycle of a GUI Agent: Prototype, Test, Deploy, Iterate

1. **Prototype** – Rapidly wire a tool palette in PyQt Designer.
2. **Test** – Use the built‑in logger to verify each tool’s input/output.
3. **Deploy** – Freeze the app with `PyInstaller` or `briefcase` for a single‑file installer.
4. **Iterate** – Hot‑reload the Tool Registry when a new `.py` file appears in the `tools/` folder.

2026 Landscape: Anthropic’s Computer Use and GUI SDKs

Claude 3.5 Sonnet Desktop introduced streamed “tool‑use” blocks that embed JSON payloads in the response stream. The SDK now emits `MessageChunk` objects with a `tool_use` field, making it trivial to hook directly into a Python event loop. The only missing piece is a UI that can process those chunks without freezing the main thread.

The 2026 Tech Stack for a Production‑Ready Claude Agent GUI

Core: Anthropic Python SDK 1.5+ and the `messages.create` API

# anthro_sdk_version.py
# Anthropic SDK >= 1.5.0 required for streaming tool‑use blocks
from anthropic import Anthropic, AsyncClient

Frontend Framework Showdown: PyQt6 vs Tkinter vs Custom Tauri

• **PyQt6/PySide6** – Native widgets, mature threading model (`QThread` & signals), great for desktop‑heavy workloads.
• **Tkinter** – Still ships with Python but feels dated; lacks modern styling and high‑DPI support.
• **Tauri 2.0** – Rust core + WebView; excellent for web‑devs, but integrating Python tools needs a bridge (e.g., `tauri-plugin-python`). For a pure‑Python stack, PyQt6 wins on stability and community support.

**My take:** I stick with PyQt6 for all serious Claude desktop agents. Its signal/slot system lets you move long‑running API calls to a worker thread with less boilerplate than `asyncio`‑only solutions.

State Management for Tools: Threading, Queues, and the Event Loop

• **QThread** for the Claude streaming client.
• **`queue.Queue`** to safely hand tool execution requests from UI to the worker.
• **Signals** (`tool_started`, `tool_finished`, `tool_error`) to update the GUI in the main thread.

Step‑by‑Step: Building the Dynamic Tool Selection Interface

Project Setup: Virtual Environment and SDK Installation

# bash
python3 -m venv .venv
source .venv/bin/activate
pip install "anthropic>=1.5.0" PyQt6==6.6.2 tqdm

Create the project skeleton:

mkdir claude_gui_agent && cd claude_gui_agent
mkdir tools logs
touch main.py tool_registry.py execution_engine.py ui.py

UI Layer: Creating the Tool Palette, Agent Console, and Log Panels

Below is a minimal PyQt6 layout that satisfies the three core panes. Save it as `ui.py`.

# ui.py
# PyQt6 UI – version 6.6.2
import sys
from PyQt6.QtWidgets import (
    QApplication, QWidget, QVBoxLayout, QHBoxLayout,
    QListWidget, QTextEdit, QPushButton, QLabel, QSplitter
)
from PyQt6.QtCore import Qt, pyqtSignal, pyqtSlot, QObject

class GuiSignals(QObject):
    send_prompt = pyqtSignal(str)               # UI -> Claude worker
    tool_selected = pyqtSignal(str)             # UI -> Registry
    append_log = pyqtSignal(str)                # Worker -> UI

class MainWindow(QWidget):
    def __init__(self, signals: GuiSignals):
        super().__init__()
        self.signals = signals
        self.setWindowTitle("Claude Agent Desktop")
        self.setMinimumSize(900, 600)
        self._build_ui()

    def _build_ui(self):
        main_layout = QVBoxLayout(self)

        # Top splitter: Tool palette | Console
        top_split = QSplitter(Qt.Orientation.Horizontal)

        # Tool palette
        palette = QListWidget()
        palette.addItems([])  # Filled dynamically later
        palette.itemClicked.connect(self._on_tool_click)
        top_split.addWidget(palette)

        # Console
        console = QTextEdit()
        console.setReadOnly(False)
        console.setPlaceholderText("Enter prompt for Claude…")
        main_layout.addWidget(top_split)

        # Bottom panel: Log output
        log = QTextEdit()
        log.setReadOnly(True)
        log.setMaximumHeight(150)

        # Send button
        send_btn = QPushButton("Send")
        send_btn.clicked.connect(lambda: self._emit_prompt(console.toPlainText()))

        # Assemble lower controls
        lower = QHBoxLayout()
        lower.addWidget(send_btn)
        lower.addStretch()

        main_layout.addWidget(console)
        main_layout.addLayout(lower)
        main_layout.addWidget(QLabel("Execution Log"))
        main_layout.addWidget(log)

        # Expose widgets for external updates
        self.palette = palette
        self.console = console
        self.log = log

        # Connect signals
        self.signals.append_log.connect(self._append_log)

    @pyqtSlot(str)
    def _append_log(self, txt: str):
        self.log.append(txt)

    def _emit_prompt(self, txt: str):
        if txt.strip():
            self.signals.send_prompt.emit(txt)
            self.console.clear()

    def _on_tool_click(self, item):
        tool_name = item.text()
        self.signals.tool_selected.emit(tool_name)

Backend Core: Building the Tool Registry and Execution Engine

`tool_registry.py` discovers Python modules in the `tools/` directory that expose a `description` string and a callable `run(**kwargs)`.

# tool_registry.py
# Tool Registry – version 1.0.0
import importlib.util
import json
import os
from pathlib import Path
from typing import Dict, Callable

TOOL_DIR = Path(__file__).parent / "tools"

class ToolRegistry:
    def __init__(self):
        self.tools: Dict[str, Dict] = {}
        self._load_tools()

    def _load_tools(self):
        for file in TOOL_DIR.glob("*.py"):
            spec = importlib.util.spec_from_file_location(file.stem, file)
            mod = importlib.util.module_from_spec(spec)
            spec.loader.exec_module(mod)  # type: ignore
            desc = getattr(mod, "description", "No description")
            run_fn = getattr(mod, "run", None)
            if callable(run_fn):
                self.tools[file.stem] = {
                    "module": mod,
                    "description": desc,
                    "run": run_fn,
                }

    def list_tools(self):
        return list(self.tools.keys())

    def get_tool(self, name: str) -> Callable:
        return self.tools[name]["run"]

    def get_description(self, name: str) -> str:
        return self.tools[name]["description"]

    def reload(self):
        """Hot‑reload tools at runtime."""
        self.tools.clear()
        self._load_tools()

Create a sample tool `tools/web_search.py`:

# tools/web_search.py
# Sample tool – version 1.0.0
description = "Search the web using DuckDuckGo and return top 3 URLs."

def run(query: str):
    import requests, bs4
    resp = requests.get(
        "https://duckduckgo.com/html/",
        params={"q": query},
        timeout=5,
    )
    soup = bs4.BeautifulSoup(resp.text, "html.parser")
    links = [
        a["href"]
        for a in soup.select(".result__a")[:3]
        if a.has_attr("href")
    ]
    return {"results": links}

The Event Loop: Bridging GUI Input with Claude’s API Streams

`execution_engine.py` runs the Claude client in a background `QThread`. It parses `tool_use` blocks on the fly, dispatches them to the registry, and streams back observable messages.

# execution_engine.py
# Execution Engine – version 1.2.0
import json
import queue
import time
from typing import Any, Dict

from PyQt6.QtCore import QThread, pyqtSignal, QObject
from anthropic import AsyncClient, APIError, RateLimitError

from tool_registry import ToolRegistry

class ClaudeWorker(QThread):
    # Signals to UI
    log = pyqtSignal(str)
    tool_start = pyqtSignal(str)
    tool_end = pyqtSignal(str, str)   # tool name, result preview
    tool_error = pyqtSignal(str, str) # tool name, error message

    def __init__(self, api_key: str, registry: ToolRegistry):
        super().__init__()
        self.api_key = api_key
        self.registry = registry
        self.input_queue = queue.Queue()
        self.client = AsyncClient(api_key=self.api_key)

    def enqueue_prompt(self, prompt: str):
        self.input_queue.put(prompt)

    def run(self):
        while True:
            try:
                prompt = self.input_queue.get(timeout=0.1)
            except queue.Empty:
                continue

            # Build system prompt with dynamic tool list
            sys_prompt = self._build_system_prompt()
            self.log.emit("🟢 Sending prompt to Claude…")
            try:
                # The SDK's async iterator yields MessageChunk objects
                async def stream():
                    async for chunk in self.client.messages.create(
                        model="claude-3-5-sonnet-202406",
                        max_tokens=1024,
                        temperature=0.0,
                        system=sys_prompt,
                        messages=[{"role": "user", "content": prompt}],
                        stream=True,
                    ):
                        yield chunk

                # Run the async generator in a sync context
                for chunk in self.client._sync(stream):
                    self._process_chunk(chunk)
            except RateLimitError as e:
                self.log.emit("⚠️ Rate limit hit, backing off…")
                self._backoff()
            except APIError as e:
                self.log.emit(f"❌ API error: {e}")
            except Exception as e:
                self.log.emit(f"❗ Unexpected error: {e}")

    def _build_system_prompt(self) -> str:
        """Inject tool descriptions dynamically."""
        tool_descs = "\n".join(
            f"- **{name}**: {self.registry.get_description(name)}"
            for name in self.registry.list_tools()
        )
        return (
            "You are Claude, an AI assistant capable of using tools. "
            "When you need to perform an action, output a JSON tool-use block "
            "matching one of the following tools:\n" + tool_descs
        )

    def _process_chunk(self, chunk: Any):
        # Detect tool-use blocks
        if hasattr(chunk, "tool_use"):
            tool_name = chunk.tool_use["name"]
            args = json.loads(chunk.tool_use["input"])
            self.log.emit(f"🔧 Tool requested: {tool_name} with {args}")
            self.tool_start.emit(tool_name)

            try:
                tool_fn = self.registry.get_tool(tool_name)
                result = tool_fn(**args)  # type: ignore
                preview = json.dumps(result)[:200]
                self.tool_end.emit(tool_name, preview)
                # Feed result back to Claude (simulated sync call)
                self.client.messages.create(
                    model="claude-3-5-sonnet-202406",
                    max_tokens=256,
                    temperature=0.0,
                    messages=[
                        {"role": "assistant", "content": f"Tool result: {json.dumps(result)}"}
                    ],
                )
            except Exception as exc:
                self.tool_error.emit(tool_name, str(exc))
                self.log.emit(f"❌ Tool {tool_name} failed: {exc}")
        else:
            # Normal assistant text – forward to UI log
            if hasattr(chunk, "content"):
                self.log.emit(chunk.content)

    def _backoff(self):
        """Simple exponential back‑off with jitter."""
        for attempt in range(1, 5):
            wait = (2 ** attempt) + (0.5 * time.random())
            self.log.emit(f"⏳ Waiting {wait:.1f}s before retry…")
            time.sleep(wait)

Wiring Everything Together – `main.py`

# main.py
# Launcher – version 1.0.0
import os
import sys
import json
from pathlib import Path

from PyQt6.QtWidgets import QApplication
from ui import MainWindow, GuiSignals
from tool_registry import ToolRegistry
from execution_engine import ClaudeWorker

API_KEY = os.getenv("ANTHROPIC_API_KEY")
if not API_KEY:
    sys.exit("❗ Set ANTHROPIC_API_KEY environment variable before running.")

app = QApplication(sys.argv)

# Shared objects
signals = Gui