The agency you are managing was chosen before you arrived. The brief they are working to was written before you arrived. The metrics they are being measured against were agreed before you arrived. And you are accountable for the results they produce.
This is the specific frustration that CMOs in established businesses carry and rarely name in public, because naming it sounds like excuse-making even when it is structural reality. You did not build the agency relationship. You inherited it, along with the contractual obligations, the internal dependencies, the political relationships between the agency and your predecessor, and the reporting framework that was designed to make the relationship look productive.
Unwinding that situation takes time and political capital you may not have in the tenure window you have been given. And so you work inside it, optimising at the edges, and you carry the knowledge that the architecture of the relationship is the constraint and the relationship itself is the thing you are not permitted to name as the problem.
Related reading