Use AI Ecosystem

Early Access (EA) Use AI Ecosystem to check on your fleet, narrow a problem to a specific agent or conversation, and understand what drives your LLM spend. To learn how AI Ecosystem works, visit What is AI Ecosystem?.

​

Checking the health of your agent fleet

Check fleet health regularly to spot changes in failures, token usage, or responsiveness, and tell whether they affect one agent or many. To check the health of your agent fleet:

1. Select AI Ecosystem () from the navigation menu.
2. On the Performance & Cost tab, review the summary metrics for conversations, active agents, active tools, failure rate, tokens, average cost per conversation, and estimated total cost. By default, the tab shows the last 7 days of data. You can adjust the view to focus your investigation.
3. Review the Conversation Volume chart to understand how much activity your agents handle over time.
4. In the Performance section, review the time-series charts:
   • Failures over time: Failures across your agents.
   • Tokens over time: Token usage across your agents.
   • GenAI span latency: Latency of your GenAI spans.
   • Time to first token: How quickly your models start responding.
5. Investigate a spike by selecting a conversation ID in a table paired with a chart, such as Recent conversations with failures or Top conversations by token consumption. The conversation opens in the Agent Timeline.

​

Adjusting the view

Adjust the Performance & Cost tab to focus on a time period, agent, or model. To adjust the view:

1. Select AI Ecosystem () from the navigation menu.
2. On the Performance & Cost tab, adjust the view:
   • To choose how to group the data, select Agents or Models.
   • To focus on specific agents or models, select them in the Agents or Models filter.
   • To change the time period, select the time range menu, which shows Last 7 days by default, and choose a new range.

​

Finding the agent to investigate first

When your fleet shows a problem, narrow it to the agent that needs attention before you dig into individual conversations. To find the agent to investigate first:

1. Select AI Ecosystem () from the navigation menu.
2. On the Performance & Cost tab, locate the Directory section, which lists every agent in your fleet with its key health metrics.
3. Enter an agent or model name in the search field.

​

Finding what drives your LLM spend

Break down your estimated spend to find which agents, models, and conversations cost the most. To find what drives your LLM spend:

1. Select AI Ecosystem () from the navigation menu.
2. On the Performance & Cost tab, locate the Cost section.
3. Review the total and average cost per conversation. Use the average as a baseline to track over time.
4. Review the cost breakdowns by agent, by model, by agent and model together, and by token type.
5. Investigate a costly conversation by selecting its conversation ID in a cost table. The conversation opens in the Agent Timeline.

To learn about how Honeycomb calculates cost, visit How Honeycomb Calculates LLM Costs.

​

Investigating a specific conversation

When a teammate, an alert, or a query points you to a specific conversation, open it in the Agent Timeline to review every step. To investigate a specific conversation:

1. Select AI Ecosystem () from the navigation menu.
2. Select the Conversations tab.
3. Enter a conversation ID, or select a conversation from the list of recent conversations.

The conversation opens in the Agent Timeline. To learn what the Agent Timeline shows, visit Agent Timeline.

​

Troubleshooting

Running into issues? Here are some common problems and ways to fix them.

​

My data isn’t showing up in AI Ecosystem

AI Ecosystem uses the OpenTelemetry GenAI spans your agents already send to Honeycomb. If your conversations appear in the Agent Timeline, AI Ecosystem needs no additional instrumentation. Otherwise, instrument your agents to emit spans with GenAI attributes.