Creating a Dedicated Agent GUI with Streamlit and Claude Desktop (2026)

When I first tried to cobble a Claude 3.5‑Sonnet “computer‑use” agent together with a simple CLI wrapper, the experience felt like driving a race car with the parking brake on. The model would spit out tool calls, the script would fire them, but there was no way to see what the agent was *thinking* in real time, pause it, or surface a helpful “are you sure?” before it clicked a button on my desktop. The result was a fragile loop that either hammered the API with retries or left me staring at a black console while the agent wandered off‑task.

What you really need is a lightweight GUI that keeps the conversation visible, streams the model’s chain‑of‑thought, and gives you a single place to confirm or abort tool actions. Streamlit 1.36+ gives us just that—an interactive, state‑ful front‑end that runs locally, plus a handful of tricks to stay responsive while the Claude 2026 SDK runs its asynchronous agent loop.

How to build a dedicated Claude‑AI GUI with Streamlit (2026 tutorial)

Learn how to create a dedicated desktop GUI for a Claude AI agent using Streamlit and the 2026 Anthropic SDK. This tutorial covers building an interactive interface, integrating the agent’s tool‑calling loop, managing state, and packaging the final app for distribution, moving beyond simple chat scripts.

System architecture & data flow

At a high level the app consists of three moving parts:

1. **Streamlit front‑end** – renders chat messages, tool buttons, and status widgets.
2. **Claude Agent core** – instantiated from `anthropic.Agent`, handling tool execution, streaming thoughts, and maintaining a rolling context window.
3. **Event bridge** – an `asyncio` task that shuttles user input from the UI to the agent and streams the agent’s partial responses back without blocking the UI thread.

flowchart LR
    UI[Streamlit UI] -->|user input| Bridge[Async Bridge]
    Bridge -->|invoke| Agent[Claude Agent (anthropic)]
    Agent -->|tool calls| Tools[Local Tools / Claude Desktop]
    Tools -->|results| Agent
    Agent -->|streamed tokens| Bridge
    Bridge -->|update| UI

The bridge keeps the UI responsive by yielding control after every token chunk. Streamlit’s `st.experimental_rerun` isn’t needed; we just update a `st.empty()` placeholder.

Step 1 – Architecting the Streamlit GUI

1.1 Session‑state container for agent memory

Streamlit re‑executes the script on every interaction, so we store the conversation history and the agent object in `st.session_state`. The first run creates them; subsequent runs reuse the same instance.

# file: app.py
# Requires: streamlit>=1.36, anthropic>=0.20, python>=3.11
import streamlit as st
from anthropic import Agent, AsyncAnthropic
from pathlib import Path
import json
import asyncio

# -------------------------------------------------
# Session‑state bootstrap
# -------------------------------------------------
if "agent" not in st.session_state:
    # Load persisted conversation if present
    conv_path = Path("conversation.json")
    if conv_path.exists():
        with conv_path.open() as f:
            messages = json.load(f)
    else:
        messages = []

    # Initialise the async client (uses the env var key)
    client = AsyncAnthropic()
    st.session_state.agent = Agent(
        client=client,
        model="claude-3-5-sonnet-202406",
        tools=["computer_use"],   # enables Claude Desktop tools
        system_prompt="You are a helpful desktop assistant.",
        temperature=0.0,
    )
    st.session_state.messages = messages
    st.session_state.pending = False   # UI lock flag

1.2 Layout – chat pane, tool panel, status bar

st.title("🖥️ Claude Desktop Agent")
chat_placeholder = st.empty()   # will hold the message stream
status_placeholder = st.empty() # for live status (thinking / waiting)

col1, col2 = st.columns([3, 1])
with col1:
    user_input = st.text_input("Your command", key="user_input")
    send_btn = st.button("Send", disabled=st.session_state.pending)

with col2:
    st.subheader("Controls")
    stop_btn = st.button("⛔ Stop", disabled=not st.session_state.pending)
    clear_btn = st.button("🗑️ Clear History")

The UI disables the send button while the agent is busy, preventing overlapping runs.

1.3 Real‑time message display

We render the conversation as a vertical list, appending new messages as they arrive. The streaming loop writes directly into the placeholder.

def render_chat():
    chat_md = ""
    for msg in st.session_state.messages:
        role = "🧑‍💻" if msg["role"] == "user" else "🤖"
        chat_md += f"**{role}**: {msg['content']}\n\n"
    chat_placeholder.markdown(chat_md, unsafe_allow_html=True)

render_chat()

Step 2 – Integrating the Claude 2026 SDK

2.1 Initialising the Agent with tool‑calling support

The 2026 SDK bundles a `Tool` abstraction; we only need to enable `computer_use`. If you plan to add custom tools later, define them with Pydantic models.

# Already done in session bootstrap – just for reference:
agent = st.session_state.agent

2.2 Building a robust agent loop

The loop must:

1. Append the user message to the context.
2. Call `agent.run_stream(messages)` which yields token chunks.
3. Detect tool call objects, execute them via Claude Desktop, and feed the results back.
4. Break on `assistant_message` completion or on user‑issued stop.

async def run_agent(user_text: str):
    st.session_state.pending = True
    # 1️⃣ add user turn
    st.session_state.messages.append({"role": "user", "content": user_text})
    render_chat()

    # 2️⃣ stream response
    async for event in agent.run_stream(st.session_state.messages):
        if event.type == "text":
            # Append token fragment to the last assistant message
            if not st.session_state.messages or st.session_state.messages[-1]["role"] != "assistant":
                st.session_state.messages.append({"role": "assistant", "content": ""})
            st.session_state.messages[-1]["content"] += event.text
            render_chat()
        elif event.type == "tool_use":
            # 3️⃣ handle tool call
            tool_name = event.name
            tool_input = event.input
            result = await execute_tool(tool_name, tool_input)
            # Feed result back into the conversation
            st.session_state.messages.append({
                "role": "assistant",
                "content": f"Tool `{tool_name}` returned: {result}"
            })
            render_chat()
        elif event.type == "error":
            st.error(f"Agent error: {event.error_msg}")
            break

    st.session_state.pending = False

2.3 Executing Computer‑Use tool calls

Claude Desktop ships a local gRPC endpoint (`http://localhost:8000/v1/compute`) that accepts JSON‑encoded commands. We wrap it with retry logic.

import aiohttp
import random
import time

MAX_RETRIES = 5

async def execute_tool(name: str, payload: dict) -> str:
    url = "http://localhost:8000/v1/compute"
    headers = {"Content-Type": "application/json"}
    for attempt in range(1, MAX_RETRIES + 1):
        try:
            async with aiohttp.ClientSession() as session:
                async with session.post(url, json={"tool": name, "input": payload}, headers=headers) as resp:
                    if resp.status == 200:
                        data = await resp.json()
                        return data.get("output", "no output")
                    elif resp.status == 429:
                        raise aiohttp.ClientResponseError(
                            request_info=resp.request_info,
                            history=resp.history,
                            status=429,
                            message="Rate limit",
                        )
                    else:
                        raise RuntimeError(f"Unexpected status {resp.status}")
        except (aiohttp.ClientError, RuntimeError) as e:
            if attempt == MAX_RETRIES:
                st.error(f"Tool `{name}` failed after {MAX_RETRIES} attempts: {e}")
                return f"Error: {e}"
            # exponential backoff with jitter
            backoff = (2 ** attempt) + random.uniform(0, 0.5)
            status_placeholder.info(f"Retrying tool `{name}` in {backoff:.1f}s…")
            await asyncio.sleep(backoff)

The `status_placeholder` gives the user visual feedback during retries.

2.4 Managing the context window

Claude 3.5‑Sonnet caps at ~200 k tokens. We trim the oldest messages once we approach the limit to avoid `InvalidRequestError`.

MAX_TOKENS = 180_000  # leave headroom

def prune_context():
    # Very naive token estimate: 4 chars ≈ 1 token
    total_chars = sum(len(m["content"]) for m in st.session_state.messages)
    if total_chars / 4 > MAX_TOKENS:
        # drop the oldest user‑assistant pair
        st.session_state.messages = st.session_state.messages[2:]

# Call before each run
prune_context()

Step 3 – Bridging the UI and the async agent loop

Streamlit runs synchronously, but we can launch the async `run_agent` coroutine in a background thread using `asyncio.run`.

if send_btn and user_input.strip():
    # Clear the input box after dispatch
    st.session_state.user_input = ""
    # Fire off the async task without blocking UI
    asyncio.run(run_agent(user_input.strip()))

if stop_btn:
    # Simple stop flag – the agent poll checks `pending`
    st.session_state.pending = False
    status_placeholder.warning("User requested stop – will halt after current chunk.")

The `pending` flag is checked inside the streaming loop. If the user hits **Stop**, the next token chunk will break out, and the UI updates instantly.

3.1 Persisting conversation history

When the user closes the window, we dump the message list to disk so a later launch resumes where they left off.

def persist_history():
    path = Path("conversation.json")
    with path.open("w") as f:
        json.dump(st.session_state.messages, f, indent=2)

if clear_btn:
    st.session_state.messages = []
    render_chat()
    persist_history()
    st.success("Chat history cleared.")

# Make sure we write on graceful shutdown
st.on_page_exit(persist_history)

Step 4 – Production‑grade error handling & state management

4.1 Rate‑limit and timeout recovery (already shown in `execute_tool`)

4.2 Global exception wrapper

Wrap the top‑level coroutine in a try/except to catch unexpected failures and surface them cleanly.

async def safe_run(user_text: str):
    try:
        await run_agent(user_text)
    except Exception as exc:
        st.error(f"Unexpected error: {exc}")
        # Reset pending flag so UI stays usable
        st.session_state.pending = False

Replace the earlier dispatch with `asyncio.run(safe_run(…))`.

4.3 Logging

We emit a thin JSON log to `agent.log` that can be inspected later.

import logging, sys

log = logging.getLogger("claude_gui")
log.setLevel(logging.INFO)
handler = logging.FileHandler("agent.log")
handler.setFormatter(logging.Formatter("%(asctime)s %(levelname)s %(message)s"))
log.addHandler(handler)

def log_event(event_type: str, payload: dict):
    log.info(json.dumps({"type": event_type, **payload}))

Insert `log_event(“tool_call”, {“name”: name, “input”: payload})` before executing a tool, and `”assistant_reply”` after each assistant message.

Step 5 – Packaging & distribution

5.1 Bundling with PyInstaller

Create a spec that includes the `streamlit` static files.

# In project root
pyinstaller --onefile --add-data "venv/Lib/site-packages/streamlit/static:streamlit/static" app.py -n ClaudeAgentGUI

After the build, `dist/ClaudeAgentGUI` is a self‑contained binary. Run it, and Streamlit automatically opens a local browser window; you can hide the browser by launching with `–server.headless true` and embedding a lightweight Chromium instance via `stlite` (see next).

5.2 Using Stlite for a true desktop feel

Stlite compiles Streamlit to WebAssembly and can be wrapped with Tauri or Electron. The minimal approach uses the `stlite-cli`:

npm i -g stlite-cli
stlite build ./ -o ./stlite-build
# Then package the folder with Tauri (Rust) or Electron (Node) for cross‑platform binaries.

The benefit: no separate Python interpreter is required on the target machine; the entire app runs inside the bundled WASM runtime.

5.3 Performance checklist

• **Token streaming** – confirm that `agent.run_stream` yields every 20‑30 tokens; adjust `max_output_tokens` if you see UI jitter.
• **CPU usage** – when the agent calls heavy local tools (e.g., image generation), isolate them in a subprocess to avoid blocking the async loop.
• **Memory** – prune the context as shown; also consider persisting embeddings for document retrieval rather than keeping raw text.

Advanced patterns – Multi‑agent coordination

If your workflow requires, for example, a research‑assistant agent that scrapes the web and a separate summarizer agent, spin up two `Agent` instances and orchestrate them via an in‑process message bus (e.g., `asyncio.Queue`). The UI can expose a tabbed pane (Streamlit `st.tabs`) that shows each agent’s stream side‑by‑side.

A practical demo is the **Claude AI GUI Agent: Drag‑Drop Workflow Build (2026)** – it shows how to wire a file‑upload widget into a “file‑analysis” agent that then hands off the result to a “report‑writer” agent. See the full walkthrough here: Claude AI GUI Agent: Drag‑Drop Workflow Build (2026).

—

Pro tips & warnings

My take

In practice I found the **session‑state‑first** pattern far less error‑prone than trying to store the agent in a global singleton. The extra line of code to re‑hydrate the agent from disk pays off the moment you want to add a “Load previous session” button. Also, prefer the built‑in `Agent` streaming API over manually constructing HTTP requests; it hides the token‑chunking logic and synchronises tool calls automatically.

Common Errors & Fixes

`ImportError: cannot import name ‘Agent’ from ‘anthropic’`

**Cause:** Using an older version