I want to describe the experience of sitting in a trading meeting where everyone in the room is looking at the same dashboard and drawing the wrong conclusion from it, because I think it is the most common experience in senior ecommerce and the least discussed.
The revenue is down week on week. The dashboard shows traffic flat, conversion rate slightly down, average order value flat. The conclusion in the room is a traffic quality problem. The media agency is asked to review targeting. The creative is flagged for refresh. The SEO team is asked about algorithm changes.
Nobody in that room looked at the cohort behaviour underneath the aggregate conversion rate. Nobody asked whether the traffic flat was a mix shift between high-intent and low-intent sources that the aggregate number was averaging together. Nobody checked whether the conversion rate decline was site-wide or concentrated in a specific device, category, or customer segment that would change the diagnosis entirely.
The dashboard produced a conclusion before anyone asked a question. That is not analysis. It is pattern-matching on a surface read of a number that was not designed to carry the weight of the decision being made on it.
Related reading