Execute Agent Tools in the Browser

Execute agent tools in the browser via WebSocket. Handle real-time tool requests from the agent and return results — for UI interactions, form manipulation, and client-side state access. What you’ll learn: WebSocket vs webhook tools, tool configuration, request handling, response format, and error handling.

Testing requires a valid API key. Generate a web integration token from your agent’s Web Integrations settings in the Dasha BlackBox dashboard before starting.

When to use WebSocket tools

WebSocket tool execution is designed for tools that need direct access to the user’s browser or application state: Use WebSocket tools for:

UI interactions (clicking buttons, navigating tabs, scrolling)
Form manipulation (filling inputs, selecting options)
Browser actions (opening windows, navigating URLs)
Client-side state (localStorage, sessionStorage, DOM access)
Real-time user feedback (animations, notifications)

Use webhook tools for:

Server-side API calls
Database queries
Email sending
Payment processing
Any operation requiring server-side secrets

WebSocket tools execute in the browser. Never use them for operations requiring server-side secrets or authentication credentials.

Tool execution flow

Step 1: Configure tools for WebSocket

Tools must be explicitly configured for WebSocket execution in your web integration.

Via dashboard

Go to Dashboard → Agents → [Your Agent] → Web Integrations
Select your integration
Navigate to Features tab
Under Tools, add tool names to execute via WebSocket

Via API

Include tool names in your web integration’s tools array:

curl -X PATCH https://blackbox.dasha.ai/api/v1/webIntegrations/{integrationId} \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "tools": ["scrollToElement", "fillFormField", "clickButton"]
  }'

Step 2: Handle tool requests

Listen for websocketToolRequest messages and respond with websocketToolResponse:

const ws = new WebSocket(
  `wss://blackbox.dasha.ai/api/v1/ws/webCall?token=${token}`
);

// Tool handler registry
const toolHandlers = {
  scrollToElement: async ({ selector }) => {
    const element = document.querySelector(selector);
    if (!element) {
      return { success: false, error: 'Element not found' };
    }
    element.scrollIntoView({ behavior: 'smooth' });
    return { success: true };
  },

  fillFormField: async ({ selector, value }) => {
    const element = document.querySelector(selector);
    if (!element) {
      return { success: false, error: 'Element not found' };
    }
    element.value = value;
    element.dispatchEvent(new Event('input', { bubbles: true }));
    return { success: true, value };
  },

  clickButton: async ({ selector }) => {
    const element = document.querySelector(selector);
    if (!element) {
      return { success: false, error: 'Element not found' };
    }
    element.click();
    return { success: true };
  }
};

ws.onmessage = async (event) => {
  const message = JSON.parse(event.data);

  switch (message.type) {
    case 'websocketToolRequest':
      await handleToolRequest(message);
      break;

    case 'text':
      if (message.content.source === 'assistant') {
        displayMessage(message.content.text);
      }
      break;

    case 'toolCall':
      console.log(`Agent calling tool: ${message.name}`);
      showToolIndicator(message.name);
      break;

    case 'toolCallResult':
      console.log(`Tool completed: ${message.name}`);
      hideToolIndicator(message.name);
      break;
  }
};

async function handleToolRequest(message) {
  const { id, toolName, args } = message.content;

  try {
    const handler = toolHandlers[toolName];
    if (!handler) {
      throw new Error(`Unknown tool: ${toolName}`);
    }

    const result = await handler(args);

    ws.send(JSON.stringify({
      type: 'websocketToolResponse',
      timestamp: new Date().toISOString(),
      content: { id, result }
    }));
  } catch (error) {
    ws.send(JSON.stringify({
      type: 'websocketToolResponse',
      timestamp: new Date().toISOString(),
      content: {
        id,
        result: {
          success: false,
          error: error.message
        }
      }
    }));
  }
}

Request message format

{
  "type": "websocketToolRequest",
  "timestamp": "2025-01-20T10:02:00Z",
  "channelId": null,
  "content": {
    "id": "wtr-abc123",
    "toolName": "fillFormField",
    "args": {
      "selector": "#email-input",
      "value": "user@example.com"
    }
  }
}

Response message format

{
  "type": "websocketToolResponse",
  "timestamp": "2025-01-20T10:02:01Z",
  "content": {
    "id": "wtr-abc123",
    "result": {
      "success": true,
      "value": "user@example.com"
    }
  }
}

Step 3: Tool implementation examples

Form automation tools

const formTools = {
  // Fill any form field by selector
  setFieldValue: async ({ selector, value }) => {
    const element = document.querySelector(selector);
    if (!element) {
      return { success: false, error: `Element not found: ${selector}` };
    }

    // Handle different input types
    if (element.tagName === 'SELECT') {
      element.value = value;
      element.dispatchEvent(new Event('change', { bubbles: true }));
    } else if (element.type === 'checkbox' || element.type === 'radio') {
      element.checked = Boolean(value);
      element.dispatchEvent(new Event('change', { bubbles: true }));
    } else {
      element.value = value;
      element.dispatchEvent(new Event('input', { bubbles: true }));
    }

    return { success: true, selector, value };
  },

  // Get current form values
  getFormData: async ({ formSelector }) => {
    const form = document.querySelector(formSelector);
    if (!form) {
      return { success: false, error: 'Form not found' };
    }

    const formData = new FormData(form);
    const data = Object.fromEntries(formData.entries());

    return { success: true, data };
  },

  // Submit a form
  submitForm: async ({ formSelector }) => {
    const form = document.querySelector(formSelector);
    if (!form) {
      return { success: false, error: 'Form not found' };
    }

    form.submit();
    return { success: true };
  }
};

Navigation tools

const navigationTools = {
  // Navigate to URL
  navigateTo: async ({ url, newTab = false }) => {
    if (newTab) {
      window.open(url, '_blank');
    } else {
      window.location.href = url;
    }
    return { success: true, url, newTab };
  },

  // Scroll to element
  scrollTo: async ({ selector, behavior = 'smooth' }) => {
    const element = selector
      ? document.querySelector(selector)
      : window;

    if (selector && !element) {
      return { success: false, error: 'Element not found' };
    }

    if (selector) {
      element.scrollIntoView({ behavior, block: 'center' });
    } else {
      window.scrollTo({ top: 0, behavior });
    }

    return { success: true };
  },

  // Switch tabs in a tabbed interface
  switchTab: async ({ tabId }) => {
    const tab = document.querySelector(`[data-tab="${tabId}"]`);
    if (!tab) {
      return { success: false, error: `Tab not found: ${tabId}` };
    }

    // Hide all tab panels
    document.querySelectorAll('.tab-panel').forEach(panel => {
      panel.classList.remove('active');
    });

    // Show selected panel
    const panel = document.querySelector(`#${tabId}`);
    if (panel) {
      panel.classList.add('active');
    }

    // Update tab buttons
    document.querySelectorAll('.tab-button').forEach(btn => {
      btn.classList.remove('active');
    });
    tab.classList.add('active');

    return { success: true, tabId };
  }
};

Data retrieval tools

const dataTools = {
  // Read displayed content
  getElementText: async ({ selector }) => {
    const element = document.querySelector(selector);
    if (!element) {
      return { success: false, error: 'Element not found' };
    }

    return {
      success: true,
      text: element.textContent?.trim() || '',
      html: element.innerHTML
    };
  },

  // Get cart items (e-commerce example)
  getCartItems: async () => {
    const items = Array.from(document.querySelectorAll('.cart-item')).map(item => ({
      name: item.querySelector('.item-name')?.textContent,
      price: item.querySelector('.item-price')?.textContent,
      quantity: item.querySelector('.item-quantity')?.value
    }));

    const total = document.querySelector('.cart-total')?.textContent;

    return {
      success: true,
      items,
      total,
      itemCount: items.length
    };
  },

  // Get page state
  getPageState: async () => {
    return {
      success: true,
      url: window.location.href,
      title: document.title,
      scrollPosition: {
        x: window.scrollX,
        y: window.scrollY
      }
    };
  }
};

Step 4: Complete implementation

Complete implementation examples

React
Vanilla JavaScript

import { useState, useEffect, useRef, useCallback } from 'react';

interface ToolExecution {
  id: string;
  name: string;
  status: 'pending' | 'executing' | 'completed' | 'error';
  args: unknown;
  result?: unknown;
}

interface ChatWithToolsProps {
  token: string;
  tools: Record<string, (args: unknown) => Promise<unknown>>;
}

export function ChatWithTools({ token, tools }: ChatWithToolsProps) {
  const [messages, setMessages] = useState<Array<{
    sender: 'user' | 'agent';
    text: string;
  }>>([]);
  const [executions, setExecutions] = useState<ToolExecution[]>([]);
  const [input, setInput] = useState('');
  const wsRef = useRef<WebSocket | null>(null);

  const handleToolRequest = useCallback(async (
    id: string,
    toolName: string,
    args: unknown
  ) => {
    // Update execution status
    setExecutions(prev => [
      ...prev,
      { id, name: toolName, status: 'executing', args }
    ]);

    try {
      const handler = tools[toolName];
      if (!handler) {
        throw new Error(`Unknown tool: ${toolName}`);
      }

      const result = await handler(args);

      setExecutions(prev => prev.map(e =>
        e.id === id ? { ...e, status: 'completed', result } : e
      ));

      wsRef.current?.send(JSON.stringify({
        type: 'websocketToolResponse',
        timestamp: new Date().toISOString(),
        content: { id, result }
      }));
    } catch (error) {
      const errorResult = {
        success: false,
        error: error instanceof Error ? error.message : 'Unknown error'
      };

      setExecutions(prev => prev.map(e =>
        e.id === id ? { ...e, status: 'error', result: errorResult } : e
      ));

      wsRef.current?.send(JSON.stringify({
        type: 'websocketToolResponse',
        timestamp: new Date().toISOString(),
        content: { id, result: errorResult }
      }));
    }
  }, [tools]);

  useEffect(() => {
    const ws = new WebSocket(
      `wss://blackbox.dasha.ai/api/v1/ws/webCall?token=${token}`
    );
    wsRef.current = ws;

    ws.onopen = () => {
      ws.send(JSON.stringify({
        type: 'initialize',
        timestamp: new Date().toISOString(),
        request: { callType: 'chat', additionalData: {} }
      }));
    };

    ws.onmessage = (event) => {
      const message = JSON.parse(event.data);

      switch (message.type) {
        case 'text':
          if (message.content.source === 'assistant' && message.content.text) {
            setMessages(prev => [...prev, {
              sender: 'agent',
              text: message.content.text
            }]);
          }
          break;

        case 'websocketToolRequest':
          handleToolRequest(
            message.content.id,
            message.content.toolName,
            message.content.args
          );
          break;

        case 'toolCall':
          setExecutions(prev => [
            ...prev,
            { id: message.callId, name: message.name, status: 'pending', args: message.args }
          ]);
          break;
      }
    };

    return () => ws.close();
  }, [token, handleToolRequest]);

  const sendMessage = () => {
    const text = input.trim();
    if (!text || !wsRef.current) return;

    setMessages(prev => [...prev, { sender: 'user', text }]);
    wsRef.current.send(JSON.stringify({
      type: 'incomingChatMessage',
      content: text,
      timestamp: new Date().toISOString()
    }));
    setInput('');
  };

  return (
    <div className="chat-with-tools">
      <div className="messages">
        {messages.map((msg, i) => (
          <div key={i} className={`message ${msg.sender}`}>
            {msg.text}
          </div>
        ))}
      </div>

      {executions.length > 0 && (
        <div className="tool-executions">
          <h4>Tool Executions</h4>
          {executions.map(exec => (
            <div key={exec.id} className={`execution ${exec.status}`}>
              <span className="tool-name">{exec.name}</span>
              <span className="tool-status">{exec.status}</span>
            </div>
          ))}
        </div>
      )}

      <div className="input-area">
        <input
          type="text"
          value={input}
          onChange={(e) => setInput(e.target.value)}
          onKeyPress={(e) => e.key === 'Enter' && sendMessage()}
          placeholder="Type a message..."
        />
        <button onClick={sendMessage}>Send</button>
      </div>
    </div>
  );
}

// Usage example
const myTools = {
  highlightElement: async ({ selector }: { selector: string }) => {
    const el = document.querySelector(selector);
    if (!el) return { success: false, error: 'Not found' };
    (el as HTMLElement).style.outline = '3px solid yellow';
    return { success: true };
  },

  showNotification: async ({ message }: { message: string }) => {
    alert(message);
    return { success: true };
  }
};

// <ChatWithTools token="YOUR_TOKEN" tools={myTools} />

<!DOCTYPE html>
<html>
<head>
  <title>WebSocket Tool Execution</title>
  <style>
    .container { max-width: 600px; margin: 20px auto; font-family: sans-serif; }
    .messages { height: 300px; overflow-y: auto; border: 1px solid #ddd; padding: 10px; margin-bottom: 10px; }
    .message { margin: 5px 0; padding: 8px; border-radius: 4px; }
    .message.user { background: #e3f2fd; text-align: right; }
    .message.agent { background: #f5f5f5; }
    .tool-indicator { background: #fff3e0; padding: 4px 8px; margin: 2px 0; font-size: 12px; }
    .input-area { display: flex; gap: 8px; }
    .input-area input { flex: 1; padding: 8px; }
  </style>
</head>
<body>
  <div class="container">
    <h2>Chat with Tool Execution</h2>
    <div class="messages" id="messages"></div>
    <div class="input-area">
      <input type="text" id="input" placeholder="Type a message...">
      <button id="send">Send</button>
    </div>
  </div>

  <script>
    const TOKEN = 'YOUR_WEB_INTEGRATION_TOKEN';
    const messagesEl = document.getElementById('messages');
    const inputEl = document.getElementById('input');
    const sendBtn = document.getElementById('send');

    // Tool implementations
    const tools = {
      scrollToTop: async () => {
        window.scrollTo({ top: 0, behavior: 'smooth' });
        return { success: true };
      },

      getWindowSize: async () => {
        return {
          success: true,
          width: window.innerWidth,
          height: window.innerHeight
        };
      },

      showAlert: async ({ message }) => {
        alert(message);
        return { success: true, message };
      }
    };

    const ws = new WebSocket(
      `wss://blackbox.dasha.ai/api/v1/ws/webCall?token=${TOKEN}`
    );

    ws.onopen = () => {
      ws.send(JSON.stringify({
        type: 'initialize',
        timestamp: new Date().toISOString(),
        request: { callType: 'chat', additionalData: {} }
      }));
      addMessage('system', 'Connected');
    };

    ws.onmessage = async (event) => {
      const msg = JSON.parse(event.data);

      switch (msg.type) {
        case 'text':
          if (msg.content.source === 'assistant' && msg.content.text) {
            addMessage('agent', msg.content.text);
          }
          break;

        case 'toolCall':
          addToolIndicator(`Calling: ${msg.name}`);
          break;

        case 'toolCallResult':
          addToolIndicator(`Completed: ${msg.name}`);
          break;

        case 'websocketToolRequest':
          await handleToolRequest(msg.content);
          break;
      }
    };

    async function handleToolRequest({ id, toolName, args }) {
      addToolIndicator(`Executing: ${toolName}`);

      try {
        const handler = tools[toolName];
        if (!handler) throw new Error(`Unknown tool: ${toolName}`);

        const result = await handler(args || {});

        ws.send(JSON.stringify({
          type: 'websocketToolResponse',
          timestamp: new Date().toISOString(),
          content: { id, result }
        }));

        addToolIndicator(`Done: ${toolName}`);
      } catch (error) {
        ws.send(JSON.stringify({
          type: 'websocketToolResponse',
          timestamp: new Date().toISOString(),
          content: {
            id,
            result: { success: false, error: error.message }
          }
        }));

        addToolIndicator(`Error: ${toolName} - ${error.message}`);
      }
    }

    function addMessage(sender, text) {
      const div = document.createElement('div');
      div.className = `message ${sender}`;
      div.textContent = text;
      messagesEl.appendChild(div);
      messagesEl.scrollTop = messagesEl.scrollHeight;
    }

    function addToolIndicator(text) {
      const div = document.createElement('div');
      div.className = 'tool-indicator';
      div.textContent = `🔧 ${text}`;
      messagesEl.appendChild(div);
      messagesEl.scrollTop = messagesEl.scrollHeight;
    }

    function sendMessage() {
      const text = inputEl.value.trim();
      if (!text) return;

      addMessage('user', text);
      ws.send(JSON.stringify({
        type: 'incomingChatMessage',
        content: text,
        timestamp: new Date().toISOString()
      }));
      inputEl.value = '';
    }

    sendBtn.addEventListener('click', sendMessage);
    inputEl.addEventListener('keypress', (e) => {
      if (e.key === 'Enter') sendMessage();
    });
  </script>
</body>
</html>

Error handling

Always return structured error responses:

async function handleToolRequest(message) {
  const { id, toolName, args } = message.content;

  try {
    // Validate tool exists
    const handler = toolHandlers[toolName];
    if (!handler) {
      throw new ToolError('UNKNOWN_TOOL', `Unknown tool: ${toolName}`);
    }

    // Validate arguments
    const validation = validateArgs(toolName, args);
    if (!validation.valid) {
      throw new ToolError('INVALID_ARGS', validation.errors.join(', '));
    }

    // Execute with timeout
    const result = await Promise.race([
      handler(args),
      new Promise((_, reject) =>
        setTimeout(() => reject(new ToolError('TIMEOUT', 'Tool execution timeout')), 30000)
      )
    ]);

    ws.send(JSON.stringify({
      type: 'websocketToolResponse',
      timestamp: new Date().toISOString(),
      content: { id, result }
    }));

  } catch (error) {
    ws.send(JSON.stringify({
      type: 'websocketToolResponse',
      timestamp: new Date().toISOString(),
      content: {
        id,
        result: {
          success: false,
          error: error.message,
          code: error.code || 'UNKNOWN_ERROR'
        }
      }
    }));
  }
}

class ToolError extends Error {
  constructor(code, message) {
    super(message);
    this.code = code;
  }
}

Best practices

Validate arguments before execution

function validateArgs(toolName, args) {
  const schemas = {
    fillFormField: { selector: 'string', value: 'string' },
    scrollToElement: { selector: 'string' },
    navigateTo: { url: 'string' }
  };

  const schema = schemas[toolName];
  if (!schema) return { valid: true };

  const errors = [];
  for (const [key, type] of Object.entries(schema)) {
    if (typeof args[key] !== type) {
      errors.push(`${key} must be a ${type}`);
    }
  }

  return { valid: errors.length === 0, errors };
}

Add logging for debugging

async function handleToolRequest(message) {
  const { id, toolName, args } = message.content;
  const startTime = performance.now();

  console.log(`[Tool] ${toolName} started`, { id, args });

  try {
    const result = await toolHandlers[toolName](args);
    const duration = Math.round(performance.now() - startTime);
    console.log(`[Tool] ${toolName} completed in ${duration}ms`, { result });

    // Send response...
  } catch (error) {
    const duration = Math.round(performance.now() - startTime);
    console.error(`[Tool] ${toolName} failed after ${duration}ms`, { error });

    // Send error response...
  }
}

Handle async operations safely

const asyncTools = {
  // Wait for element to appear
  waitForElement: async ({ selector, timeout = 5000 }) => {
    const startTime = Date.now();

    while (Date.now() - startTime < timeout) {
      const element = document.querySelector(selector);
      if (element) {
        return { success: true, found: true };
      }
      await new Promise(resolve => setTimeout(resolve, 100));
    }

    return { success: false, error: 'Element not found within timeout' };
  },

  // Retry operation
  retryOperation: async ({ toolName, args, maxRetries = 3 }) => {
    for (let attempt = 1; attempt <= maxRetries; attempt++) {
      try {
        const result = await toolHandlers[toolName](args);
        return { ...result, attempts: attempt };
      } catch (error) {
        if (attempt === maxRetries) {
          return { success: false, error: error.message, attempts: attempt };
        }
        await new Promise(resolve => setTimeout(resolve, 1000 * attempt));
      }
    }
  }
};

Troubleshooting

Tool not being called

Possible causes:

Tool not configured in web integration
Tool name mismatch between agent config and handler
Agent doesn’t understand when to use the tool

Solution: Verify tool name in dashboard matches exactly, check agent’s tool descriptions are clear.

Tool request times out

Possible causes:

Handler takes too long to execute
Handler never returns
DOM operation fails silently

Solution: Add timeouts, logging, and error boundaries:

const withTimeout = (fn, ms = 10000) => async (...args) => {
  const timeout = new Promise((_, reject) =>
    setTimeout(() => reject(new Error('Timeout')), ms)
  );
  return Promise.race([fn(...args), timeout]);
};

const safeTools = Object.fromEntries(
  Object.entries(tools).map(([name, fn]) => [name, withTimeout(fn)])
);

Agent doesn't use tool result

Possible causes:

Result format doesn’t match what agent expects
Error in result not clearly communicated
Agent prompt doesn’t instruct how to use results

Solution: Use consistent result structure, include descriptive messages:

// Good: Clear, structured result
return {
  success: true,
  data: { orderId: 'ORD-123', status: 'shipped' },
  message: 'Order found and is currently shipped'
};

// Bad: Ambiguous result
return { orderId: 'ORD-123' };

Introduction

Create

Test

Deploy

Monitor

Advanced Features

Troubleshooting

When to use WebSocket tools

Step 1: Configure tools for WebSocket

Via dashboard

Step 3: Tool implementation examples

Step 4: Complete implementation

Best practices

Troubleshooting

Tool not being called

Next steps

Message Reference

Error Handling

​When to use WebSocket tools

​Step 1: Configure tools for WebSocket

​Via dashboard

​Step 3: Tool implementation examples

​Step 4: Complete implementation

​Best practices

​Troubleshooting

​Tool not being called

​Next steps

Message Reference

Error Handling

When to use WebSocket tools

Step 1: Configure tools for WebSocket

Via dashboard

Step 3: Tool implementation examples

Step 4: Complete implementation

Best practices

Troubleshooting

Tool not being called

Next steps