Stripe's Transformer Payments Foundation Model

Stripe's Transformer-Based Payments Foundation Model

Key Insights:

• Problem: Traditional ML models for payments (fraud detection, authorization, etc.) rely on discrete features (BIN, zip, etc.) and task-specific training, limiting their effectiveness in capturing complex patterns.
• Solution: Stripe built a payments foundation model – a self-supervised transformer network that learns dense, general-purpose vectors for each transaction.
• Analogy: Similar to how language models embed words, this model embeds payments, capturing rich data and relationships in a high-dimensional vector space.
• Training Data: Trained on tens of billions of transactions.
• Benefits:
  • Improved card-testing attack detection: Detection rate for card-testing attacks on large users improved from 59% to 97% overnight.
  • Versatility: The same embeddings can be applied across other tasks like disputes or authorizations.
  • Captures Semantic Meaning: Reveals complex sequential dependencies and latent feature interactions in transactions that manual feature engineering misses.
• Impact of Traditional ML: Reduced card testing for users on Stripe by 80% over the past couple of years.
• Previous Improvements: +15% conversion, -30% fraud using traditional ML methods