← Back to R&D
Safety

The Trust Problem With AI-Generated Automations

Hacroo Technologies · May 20, 2025 · 4 min read

Early users who could describe exactly what they wanted, and got a correct workflow generated for it, still frequently hesitated before actually deploying it. That hesitation looked at first like a UX problem — maybe the interface didn't build enough confidence. It wasn't. It was a completely reasonable response to being asked to trust an opaque system with real customer emails, real payment records, real business communication.

Correctness isn't the same as trustworthiness

A workflow can be functionally correct and still not feel safe to run, because "correct" is something the user has to take on faith until they've watched it work. Trust isn't earned by a system being right — it's earned by a system being inspectable, reversible, and honest about what it doesn't know. Those are different design requirements than accuracy, and no amount of improving generation quality substitutes for them.

Correct, but opaque
  • Works, but users can't see why
  • Trust requires blind faith
  • Users hesitate before deploying
Correct and inspectable
  • Shows what it will do, in plain English
  • User confirms before anything runs
  • Trust earned by evidence, not faith

What actually built confidence

The single biggest shift in user willingness to deploy didn't come from a more capable model — it came from showing the generated workflow in plain language before deployment, letting someone read "this will email the customer, then log the order in your sheet" and confirm that matches their intent, rather than asking them to trust a black box or reverse-engineer a node diagram. Explainability turned out to matter more for trust than raw accuracy did.

Why this became a permanent design constraint

This is the direct ancestor of two things that later became formal parts of the product: schema-grounded generation (so the explanation is actually accurate to what will run) and a preview-and-confirm step before anything deploys. Trust, for a system that acts on someone's business, isn't a feeling to design for — it's a set of concrete guarantees: show your work, ask before anything irreversible, and never claim more certainty than you have.

More R&D

Want to see this working, not just described?