"it’s knowing when the agent is confidently wrong and how to fence it in with tests, architecture, and invariants."

Strongly suspect this is simply less efficient than doing it yourself if you have enough expertise.