Monitor your agents with signals

In W&B Weave’s Agents view, the Signals tab shows tags and ratings for your agent’s conversations. Signals surface quality and safety issues to flag problems, find patterns, and highlight the traces that need your attention. Use signals to automatically score the quality of your agent’s responses, notice when a user is frustrated, or flag NSFW content.

​

Get started

To view signals for your project:

1. Navigate to https://wandb.ai and select your project.
2. In the sidebar menu, select Agents to view all agent conversations saved for your project.
3. In the tab bar, select Signals.

​

Key terms

• Turn: One back-and-forth exchange between the user and the agent.
• Rating: A numeric score between 0.0 and 1.0 assigned to a matching span.
• Tags: Labels assigned to matching spans, such as “user-frustration” or “nsfw”.

​

Signals table

The Signals tab displays a table of scored turns from your agent. Each row represents the output of one of your signal monitors. The following columns appear by default. Use the time window selector and Filter bar to narrow results by scorer, agent, score range, or time period. The Score volume timeline shows counts of signals that Weave evaluated for ratings or tagged. It reflects the rows shown in the table and supports drag-to-filter on the timeline.

​

Create a new signal

To start scoring your agent’s turns, create one or more signals. Select + New signal to open the Create signals drawer. The drawer groups available signals into two categories:

• Tags: Apply a label automatically to matching spans, such as user-frustration or nsfw. Use tags to categorize spans or flag unwanted behavior. The signals UI only displays rows for spans that matched at least one tag, so a tag signal might run successfully even if you don’t see any output.
• Ratings: Assign a score from 0 to 1 to matching spans. Use ratings to rate agent performance and measure improvements over time.

​

Preset signals

Each category offers preset templates that you can select directly. Select any combination of presets across both categories, then select Create [N] signals to create them all at once with default settings.

​

Tags presets

​

Ratings presets

​

Custom signals

To create a custom tag signal from an empty Signals tab, see Create a custom signal.

​

Manage and edit signals

If you have at least one created signal, select the active signals button in the toolbar—labeled with the number of active signals, such as 1 active signals—to open a drawer that lists all active signals for the project. From there you can toggle signals on or off, delete them, or edit any signal. The editor shows the same configuration fields as a custom signal.

​

Troubleshoot signals

Signal activity appears under Traces in the project sidebar. If you don’t see the expected signal matches, debug using the Traces table. For example, check the scorer name and the Status column for error conditions. Scorer execution errors show a red indicator for Status and include error details.