There's a structural problem in how AI-powered financial products are being designed, and it has almost nothing to do with the quality of the models. The models are getting better. The interfaces are getting smoother. The user experience, in the narrow sense of ease-of-use, improves with every iteration. And yet something fundamental is missing — something that will determine whether these products actually work in the only sense that matters: whether people trust them enough to act on them, and whether anyone is accountable when they're wrong.
I'm going to call this the trust and accountability layer — the design infrastructure that sits between a model's output and a user's decision, and that most fintech AI products have not built.
The usability assumption
Most AI-powered financial products — robo-advisors, credit-scoring tools, personal finance managers, trading signal platforms — are designed as if the primary barrier to adoption is usability. The assumption is: if we make the interface clean, the onboarding smooth, and the recommendations clear, people will use the product and trust the output.
This assumption is borrowed from consumer tech, where it's largely correct. Making it easier to order food, hail a ride, or send a message does increase adoption, because the downside of a bad recommendation in those contexts is small. You get a mediocre restaurant suggestion. You wait an extra three minutes for your ride. The stakes are low enough that usability is the binding constraint.
Financial decisions are categorically different. The stakes are high, the consequences are delayed, the feedback loops are noisy, and the user's ability to evaluate the quality of a recommendation is usually poor. In this context, usability is necessary but nowhere near sufficient. The binding constraint is trust — and trust, in financial contexts, has specific structural requirements that good UX alone cannot satisfy.
What trust actually requires in financial AI
Trust in a financial recommendation is not a feeling. It's a judgment — a cognitive assessment that the recommendation is likely to be reliable enough to act on, given what's at stake. That judgment requires three things that most fintech interfaces do not provide:
1. Legibility of reasoning
When a human financial advisor recommends a portfolio allocation, the client can ask "why?" and get an answer that connects the recommendation to the client's specific situation, goals, and constraints. The answer might be imperfect, but it's legible — the client can follow the reasoning chain and decide whether it makes sense.
Most AI-powered financial tools skip this step entirely. They present a recommendation — "rebalance your portfolio to 60/40" or "your credit score suggests refinancing" — without making the reasoning accessible. Some offer a confidence score. Some show a chart. But almost none provide the kind of structured, situation-specific reasoning that would let a user evaluate why this recommendation, for me, right now.
This isn't just a UX problem. It's an epistemological one. If a user can't evaluate the reasoning behind a recommendation, they're not making a decision — they're delegating one. And delegation without legibility is not trust. It's faith. Financial products built on faith rather than trust are fragile in exactly the way that matters: they collapse when the recommendation turns out to be wrong, because the user never had the tools to evaluate it in the first place.
2. A clear model of uncertainty
Financial outcomes are inherently uncertain, and any honest recommendation must communicate that uncertainty in a way the user can integrate into their decision. This is where most fintech AI fails most visibly.
The standard approach is to present recommendations with false precision — "expected return: 7.2%" — or to bury uncertainty in disclaimers that no one reads. Neither approach helps the user. What would help is a design language for uncertainty that is native to the interface: visual representations of probability distributions, scenario comparisons, explicit statements about what the model does and doesn't know, and clear markers for when a recommendation is high-confidence versus speculative.
I've managed a $50M+ credit portfolio and built Monte Carlo simulation models for default probability estimation. One thing that work teaches you quickly is that the way you communicate uncertainty matters as much as the way you calculate it. A model that accurately estimates a 12% probability of default is useless if the person acting on it interprets "12%" as either "basically zero" or "definitely going to happen" — both of which are common misinterpretations. The interface has to do work that the model alone cannot.
3. Accountability architecture
When a human financial advisor gives bad advice, there is a chain of accountability: regulatory obligations, fiduciary duty, professional licensing, and the reputational consequences of a visible failure. When an AI system gives bad advice, the accountability chain is — what, exactly?
This is not a hypothetical concern. It's a design question. And most fintech products answer it by not answering it: they disclaim liability, present recommendations as "informational only," and leave the user holding the risk of a decision that was shaped by a system they couldn't evaluate.
A genuine trust layer would include explicit accountability structures: who is responsible when this recommendation is wrong? What recourse does the user have? How is the system's track record measured and made visible? These are not legal add-ons. They are core design requirements for any product that asks users to make consequential decisions based on algorithmic output.
The design implications
If you take the trust and accountability layer seriously, it changes how you design a fintech AI product in fundamental ways.
The recommendation is not the product. The product is the decision-support system — the infrastructure that helps a user evaluate a recommendation, understand its uncertainty, and decide whether to act on it. The recommendation itself is an input to that system, not its output.
Explainability is not optional. It's not a feature to add in v2. It's the foundation on which everything else is built. If you can't explain why the model recommends what it recommends, in terms a non-technical user can evaluate, you don't have a product — you have an oracle. And oracles, historically, have a poor track record in financial markets.
Uncertainty must be a first-class design element. Not a footnote, not a disclaimer, not a tooltip. The visual language of the interface must make uncertainty as salient and legible as the recommendation itself. This is hard. It requires design innovation, not just model sophistication. But it's the only honest way to present financial predictions.
Accountability must be structural, not rhetorical. Saying "we're committed to responsible AI" is rhetoric. Building visible track records, user-accessible audit logs, and clear escalation paths when recommendations go wrong is structure. The difference between the two is the difference between marketing and engineering.
Why this matters beyond fintech
The trust and accountability layer isn't unique to financial products. It's relevant anywhere AI is being used to support high-stakes decisions — healthcare, legal, hiring, education. But finance is where the problem is most acute, for two reasons.
First, financial decisions have clear, measurable outcomes with real economic consequences. This means the cost of a bad recommendation is quantifiable, which makes the absence of accountability structures particularly visible.
Second, financial markets have a long institutional history of building trust through regulation, fiduciary duty, and professional norms. The AI products entering this space are, in many cases, building around or outside those institutional structures rather than engaging with them. That's a strategic choice with consequences, and those consequences are starting to become visible.
The companies that will win in fintech AI are not the ones with the best models. They're the ones that build the trust infrastructure that makes those models usable for real decisions by real people with real money at stake. That infrastructure doesn't exist yet in most products. Building it is the hard problem — and the important one.
My research at Rutgers will explore how organizations adapt to institutional uncertainty — including the institutional uncertainty created when AI systems enter regulated domains without the trust architecture those domains require. The fintech space is a natural laboratory for that question, and I expect it to become a more important one as adoption scales and the first wave of consequential failures arrives.