Customer-Level Behavioural Risk Scoring with Gradient Boosting, Stratified Validation, and LOW/MEDIUM/HIGH Bucket Policies for Digital Banking

AF
Aldi Ferdiyan2026-08-044 min read

1. Introduction

Banks and fintechs must monitor not only fraudulent payments but also customers whose behaviour drifts from historical norms—large night transfers on new devices, sudden destination sprawl, declining cashflow, or elevated ATO signals. Transaction fraud models answer "is this event fraud?"; behavioural risk models answer "is this customer becoming high risk?" Both layers complement each other in an enterprise AI risk platform.

This paper documents the Nexius Risk Behaviour module: a supervised customer-scoring pipeline from master-data feature aggregation through training, evaluation, bucketing, and serving. The contribution is a transparent, bank-oriented research protocol that makes imbalance-aware and operational choices explicit.

2. Problem Formulation

Let each customer c be represented by an aggregated feature vector x_c. We estimate
s_c = P(high_risk | x_c) ∈ [0, 1].
A policy π maps scores to buckets {LOW, MEDIUM, HIGH} (default cutoffs ≈ 0.4 and 0.7). Research evaluation emphasises ranking metrics on a stratified held-out set; operational evaluation reports confusion-based metrics at the primary high-risk threshold and early-warning rule coverage.

3. Data and Feature Engineering

3.1 Data Sources

Features are aggregated from master tables:

  • customers
  • accounts
  • transactions
  • logins
  • devices
  • watchlist
  • fraud_labels
  • statement_lines

Institutions can substitute equivalent warehouse extracts.

3.2 Feature Groups

  • KYC Profile: PEP, monthly income, labeled risk, watchlisted
  • Accounts & Cashflow: account count, dormant accounts, average balance, cashflow slope from statement lines
  • Transaction Behaviour: counts & amount statistics, night ratio, unique destinations, transfer ratio, spend-versus-income
  • Access & Fraud History: login volume, failed logins, device count, rooted devices, prior fraud transaction count

3.3 Label Construction

  • Positive label is_high_risk is true when:
    • KYC risk level is HIGH, or
    • the customer is watchlisted, or
    • extreme behavioural heuristics fire (elevated night activity, many unique destinations, ATO-like login failures, rooted devices, prior fraud txs, sharply declining cashflow)
  • Remaining customers are negatives.
  • Heuristic positives are documented as research proxies—not substitutes for investigator-confirmed risk ratings in production studies.

4. Learning Algorithm

  • Train a binary classifier with LightGBM when available, otherwise HistGradientBoosting.
  • Class imbalance handled via class_weight=balanced.
  • Tree ensembles suit tabular banking aggregates, support fast research iteration, and enable permutation-importance attribution.
  • The current deployed artifact: LGBMClassifier.

5. Experimental Protocol

5.1 Stratified Split

  • Use stratified holdout with configurable test_size to preserve high-risk prevalence across train/test.
  • When class counts allow, stratified K-fold PR-AUC provides an extra stability check.
  • Split metadata is reported explicitly (e.g., time, n_train=8000, n_test=2000).

5.2 Metrics

  • Accuracy excluded as headline KPI under imbalance.
  • Prioritise PR-AUC and ROC-AUC.
  • At a threshold tuned towards F1 with a precision preference, report: Precision, Recall, and F1 for high-risk class.
  • Latest holdout snapshot: PR-AUC=1.000, ROC-AUC=1.000.

5.3 Bucket Policy

  • Continuous scores map to:
    • LOW (< medium cutoff)
    • MEDIUM
    • HIGH (≥ high cutoff; defaults near 0.4 / 0.7)
  • Separating scoring from buckets enables research on model quality independently from risk appetite.

5.4 Early Warning Overlays

  • Rule-based early warnings (declining cashflow, high night ratio, many unique destinations, repeated failed logins, prior fraud history) accompany scores
  • Analysts receive actionable narratives beyond a raw probability.

6. Research Use Cases

  • Imbalance-aware benchmarking: compare PR-AUC vs ROC-AUC as high-risk prevalence changes.
  • Feature-family ablation: remove KYC, velocity, or access groups to quantify lift for investigation hypotheses.
  • Bucket stress tests: sweep medium/high cutoffs against review-team capacity.
  • Label sensitivity: measure degradation when heuristic positives replace investigator ratings.
  • Cohort drift: retrain on rolling windows to study channel-mix and behaviour-shift effects.

7. Deployment Notes

  • Artifacts persist as model bundles with metrics metadata.
  • Batch scoring produces scored customers for dashboards.
  • Online serving exposes GET /ai/risk/scores and overview aggregates.
  • Dashboards should show full-population aggregates (server-side); detail tables remain paginated.
  • Human-in-the-loop review is always required for HIGH buckets in regulated settings.

8. Limitations

  • Results on synthetic master data are not external validity claims for any bank.
  • Heuristic labels, delayed KYC updates, and incomplete device/statement coverage can inflate research metrics.
  • Accuracy is deliberately omitted as a headline KPI under imbalance.

9. Conclusion

We formalised a customer-level behavioural risk research pipeline aligned with banking operations: aggregated tabular features, imbalance-aware boosting, stratified evaluation, PR-centric metrics, an explicit LOW/MEDIUM/HIGH bucket policy, and early-warning overlays. The protocol supports reproducible experimentation and incremental productionisation within an enterprise AI risk platform.

References

  • Davis, J., & Goadrich, M. (2006). The relationship between Precision-Recall and ROC curves. ICML.
  • Ke, G., et al. (2017). LightGBM: A highly efficient gradient boosting decision tree. NeurIPS.
  • Bolton, R. J., & Hand, D. J. (2002). Statistical fraud detection: A review. Statistical Science.
  • Chandola, V., Banerjee, A., & Kumar, V. (2009). Anomaly detection: A survey. ACM Computing Surveys.
  • Nexius AI Lab. (2026). Enterprise AI Risk Platform — Risk Behaviour Module Technical Note (internal).