In a previous role, we had audits to help us to refactor our data pipelines with confidence. While it may make sense to run audits in staging, staging data can very different vs production. This means that the changes get merged based on guardrails that allow staging to run successfully, but end up breaking in production.
To get around this, you run tests in production. Never test in production you say? What’s the use of testing if it doesn’t stop you from breaking things. The following quote is from Erik Bernhardsson.
Lets let the right workflows emerge from what makes teams the most productive, and lets let data workflows stand on their own feet.
The book of best practices for data (perhaps ML too) is still being written. Plus there’s probably a startup idea there somewhere.