Smartnumbers Investigate
Real-time fraud investigation platform that empowers teams to stop fraudsters in their tracks with machine learning-powered case flagging
Opportunity
Fraud investigators were fighting blind—reactive when they needed to be predictive. We needed to flag high-risk calls in real-time and empower teams to stop fraudsters in their tracks.
The Team
Product Designer, Developer, Product Manager, Data Analyst
Wins
- 120% increase in weekly active users completing key actions
- ML accuracy improved via decision feedback loop from investigators
- Network effect growth—investigators started using us as their starting point
- Stickiness through accuracy—difficult to turn off because of our performance
The Silent Threat
Every day, fraudsters were slipping through the cracks. They'd call multiple times, testing defenses, probing for weaknesses. By the time investigators caught on, the damage was done—funds transferred, identities stolen, trust broken.
The Outcome
A real-time fraud investigation platform that empowers teams to stop fraudsters in their tracks with machine learning-powered case flagging.
Scroll to find out how we did it
The Discovery
Digging Into the Data
We started with a question: What does a valuable investigation actually look like? The uncomfortable truth? Neither we nor the business really knew.
So we went hunting.
Product usage data revealed patterns:
- High-risk callers weren't random—they were prolific repeat offenders, often calling multiple times per day
- Our tool was being used as an information gathering resource, but investigations started elsewhere
- Timing was everything: the closer to real-time, the better the outcome
User interviews validated our hypothesis: investigators needed cases flagged as close to real-time as possible to have any chance of protecting funds. The window of opportunity was minutes, not hours.
The Ideation
From Insights to Ideas
Armed with facts, I synthesized findings for the team. Context was everything—everyone needed to understand the investigator's reality before we could design for it.
After a show-and-tell session, we moved into effort definition. Standard prioritization matrices couldn't capture the complexity, so we developed a custom framework that tracked:
- Implementation effort
- Technical complexity
- Success metrics (this would prove crucial later)
Key Discussion Point: What does success look like? We decided to track decision outcomes—aiming for as many "fraud" or "suspicious" classifications as possible.
The twist? We wanted to feed these decisions back to our machine learning team as a factor for calculating risk. By collaborating with other product managers, this would improve accuracy across the entire platform.
The Design
Crafting the Investigation Journey
With our chosen direction, I designed a new user journey that transformed how investigators worked.
Real-time Flagging
Cases appear as calls happen, not hours later
Unified View
All investigation data in one contextual interface
Smart Decisions
One-click classification with ML feedback loop
Accuracy Loop
Investigator decisions improve future flagging
The Validation
User Testing Results
We put the prototype in front of real investigators. The response exceeded expectations:
User Love
Investigators embraced the journey immediately. The workflow felt natural, not forced.
Clarity
Case conditions were understood without training. The risk indicators resonated.
Future Vision
Users were vocal about next iterations—feedback that went straight into our opportunity solution tree.
The Impact
From Launch to Scale
After development and launch, we tracked metrics weekly, tuning as needed. The results told a clear story.
The Network Effect
Something unexpected happened. Based on our logic and accuracy, fraud investigators started using Smartnumbers as their starting point for investigations. We became the first tool they reached for, not the last.
We eventually ran out of cases to flag—a good problem to have. But we knew exactly what phase 2 looked like, thanks to our user testing insights.
The Ultimate Win: Stickiness through accuracy. It's literally difficult to turn us off because of our performance. Users who love the product become advocates, driving network effect growth.
What We Learned
Timing is Everything
Real-time beats perfect. Speed of flagging mattered more than perfect accuracy.
Feedback Loops
User decisions can train your ML. Design for data collection from day one.
Success Breeds Demand
When you solve the problem well, users want more. Plan for scale from launch.
From Reactive to Predictive
Smartnumbers Investigate transformed how fraud teams operate—from chasing yesterday's fraud to stopping today's. By combining real-time flagging with intelligent decision capture, we didn't just build a tool; we built a learning system that gets smarter with every investigation.
The investigators who once fought blind now had visibility. And that made all the difference.