Dashboards vs. Questions

The view into my ETL system

Now that we have agents that can analyse our data for us, the question becomes whether to retain our dashboards or not. I think it's pretty clear that the standard answer is right: we need both these systems.

Dashboards are useful for awareness of current state and how it's evolved. An LLM-based query is useful for interrogating the data.

So, to see the general health of my system I use a dashboard, and to ask specific questions about something that's odd I would use an agent.

For instance, I run an ETL system for a client and every now and then I pop in to check the health of the system. I want to see how fast it's running, how many syncs have failed, and questions like that which are the same every time I look. I've found that the easiest way I have of getting a good idea of what's up is to make sure that my dashboard always has the same information in the same place and I specifically make sure that the page loads identically without any movement.

On the other hand, it's just as important to be able to ask ad-hoc whether some specific pipeline or subpopulation is performing differently or has a reason for change - and here an agent equipped with a read-only key and access to my Kubernetes cluster is quite powerful. One recent way I asked it was if a spike in failure rate that showed up on my dashboard was due to a new API change from Meta's marketing API. It was able to identify that it was a transient error from their servers rather than a persistent problem and provide me the evidence I needed to confirm.