| Metric | Value |
|---|---|
| Nodes | 1000 |
| Edges | 13079 |
| k (nearest neighbours) | 20 |
| Communities (Louvain) | 5 |
| Modularity | 0.679 |
| Degree-matched null modularity (Louvain, resolution 1) | 0.170 ± 0.004 |
| Same-procedure null (resolution 0.5) | 0.000 ± 0.000 |
| ARI vs planted truth | 0.417 |
SettleWise - intelligence report
A behavioural similarity network, tested statistics, one small model and an intervention-scenario sweep, on a synthetic collections book with planted structure
This is the same analysis the dashboard’s Intelligence page shows, rendered from the same tables, as a document. Everything here is evidence for a decision system, not the decision: no number in this report can change an amount, a floor or an account status - server/offer_engine.py does that, deterministically.
Data
1,000 synthetic historical borrowers and 26,008 interaction events, generated with five planted behavioural communities so that every method below can be scored against a known truth. That choice is deliberate and its cost is stated in the limitations: the effect sizes are properties of the generator, not of a real population.
Behavioural similarity network
Edges are cosine similarity on standardised [contact_success_rate, daytime_contact_rate, evening_contact_rate, evening_lift, objection_rate, refusal_rate, promise_rate, log_calls]; each node linked to its 20 nearest; undirected. The null model rewires the graph while preserving every node’s degree, then asks how much modularity Louvain finds in the result. Two versions are reported on purpose: the conventional one (Louvain at resolution 1) is the comparison, and the observed modularity sits 134 standard deviations above it. The same-procedure null - the resolution-0.5 detection this pipeline actually uses - collapses a random graph into one community, so it is ~0 with ~no variance; on its own it would look like overwhelming evidence and be nothing of the kind.
| Segment | n | Paid | 95% CI | Picks up | Best window | Median days to payment |
|---|---|---|---|---|---|---|
| Delayed but responsive | 268 | 93% | 90%-96% | 49% | evening (75%) | 11 |
| Prompt payers | 274 | 99% | 97%-100% | 71% | evening (70%) | 5 |
| Prompt payers (2) | 85 | 41% | 32%-51% | 80% | evening (81%) | 6 |
| Hardship | 158 | 78% | 72%-85% | 42% | morning (40%) | 19 |
| Hard to reach | 215 | 53% | 47%-60% | 19% | evening (19%) | 21 |
Statistical findings
Every finding carries its sample size and interval; p-values are Benjamini-Hochberg adjusted across the family, and confidence intervals come from a borrower-clustered bootstrap because call attempts within a borrower are not independent.
| Question | Segment | n | Effect | 95% CI | adj. p | Significant |
|---|---|---|---|---|---|---|
| Does calling in the evening (17:00-20:00) change the chance the borrower picks up? | all | 15902 | odds ratio (evening vs daytime) 1.84 | 1.73-1.97 | < 0.001 | yes |
| Within the “Delayed but responsive” segment, does evening contact change pick-up? | Delayed but responsive | 4319 | odds ratio (evening vs daytime) 7.05 | 6.14-8.09 | < 0.001 | yes |
| Within the “Hard to reach” segment, does evening contact change pick-up? | Hard to reach | 4478 | odds ratio (evening vs daytime) 1.13 | 0.96-1.32 | 0.203 | no |
| Within the “Hardship” segment, does evening contact change pick-up? | Hardship | 2814 | odds ratio (evening vs daytime) 1.01 | 0.86-1.19 | 0.945 | no |
| Within the “Prompt payers” segment, does evening contact change pick-up? | Prompt payers | 4018 | odds ratio (evening vs daytime) 1.03 | 0.89-1.17 | 0.926 | no |
| Within the “Prompt payers (2)” segment, does evening contact change pick-up? | Prompt payers (2) | 273 | odds ratio (evening vs daytime) 2.04 | 1.16-3.58 | 0.023 | yes |
| Does an SMS reminder the day before a promised payment make the payment more likely? | all | 2958 | odds ratio (reminded vs not) 2.27 | 1.94-2.65 | < 0.001 | yes |
| Within “Delayed but responsive”, does a reminder help promises get kept? | Delayed but responsive | 1155 | odds ratio (reminded vs not) 3.00 | 2.34-3.86 | < 0.001 | yes |
| Within “Hard to reach”, does a reminder help promises get kept? | Hard to reach | 284 | odds ratio (reminded vs not) 2.89 | 1.74-4.79 | < 0.001 | yes |
| Within “Hardship”, does a reminder help promises get kept? | Hardship | 387 | odds ratio (reminded vs not) 3.54 | 2.32-5.40 | < 0.001 | yes |
| Within “Prompt payers”, does a reminder help promises get kept? | Prompt payers | 1082 | odds ratio (reminded vs not) 1.88 | 1.38-2.57 | < 0.001 | yes |
| Within “Prompt payers (2)”, does a reminder help promises get kept? | Prompt payers (2) | 50 | odds ratio (reminded vs not) 1.04 | 0.34-3.19 | 0.945 | no |
| Do the behavioural communities found by the network differ in whether borrowers pay? | all | 1000 | Cramer’s V 0.51 | < 0.001 | yes | |
| Do the three contact strategies differ in whether borrowers pay? | all | 1000 | Cramer’s V 0.01 | 0.945 | no | |
| Adjusting for segment and balance, does the “evening_contact” strategy speed up payment vs standard? | all | 1000 | hazard ratio vs standard 1.13 | 0.95-1.34 | 0.212 | no |
| Adjusting for segment and balance, does the “reminder_first” strategy speed up payment vs standard? | all | 1000 | hazard ratio vs standard 1.15 | 0.97-1.36 | 0.192 | no |
| How long until a first payment lands, and does it differ by segment? | all | 1000 | log-rank chi-square 479.82 | < 0.001 | yes | |
| Does the day of the week a call is made change whether it is answered? | all | 15902 | Cramer’s V 0.01 | 0.945 | no |
The pooled evening-contact effect hides its own structure: the odds ratio is large in one segment and indistinguishable from 1 in three others, which is the case for targeting rather than a blanket policy. The nulls are reported too - the Cox hazard ratios for contact strategy on time to payment are not significant, and weekday does nothing.
Predictive model
Payment within seven days of a contact attempt, on features computed as of the attempt (nothing from the future leaks in; the scaler is fit on the training split only). Elastic-net logistic regression against gradient-boosted trees, champion chosen on validation PR-AUC, scored on a held-out test split of attempts.
| Model | Train | Test | ROC-AUC | PR-AUC | Brier | Champion |
|---|---|---|---|---|---|---|
| Gradient-boosted trees | 9713 | 2804 | 0.694 | 0.428 | 0.174 | yes |
| Elastic-net logistic regression | 9713 | 2804 | 0.682 | 0.416 | 0.176 | |
| Similar-borrower rate (baseline) | 9713 | 2804 | 0.591 | 0.335 | 0.279 |
The model is modest - a test ROC-AUC of 0.69 - and is shown as such. It is evidence for a recommendation the offer engine may decline, not a decision.
Epidemic-curve reframing
The survival data restated as counts over calendar time: every historical account moves susceptible → active → recovered (paid) or escalated. This is not an epidemic - nothing transmits between borrowers. R_eff = β × contact rate × duration is a load indicator on the collections process, not a forecast of growth.
| Segment | n | β (conversion per attempt) | contacts / active day | duration (RMST, days) | R_eff |
|---|---|---|---|---|---|
| All segments | 1000 | 0.115 | 0.71 | 11.3 | 0.92 (0.83-1.02) |
| Delayed but responsive | 268 | 0.081 | 0.90 | 11.4 | 0.83 (0.70-1.00) |
| Hard to reach | 215 | 0.025 | 0.44 | 15.3 | 0.17 (0.14-0.21) |
| Hardship | 158 | 0.040 | 0.56 | 14.6 | 0.33 (0.24-0.43) |
| Prompt payers | 274 | 0.303 | 2.07 | 6.3 | 3.95 (3.61-4.29) |
| Prompt payers (2) | 85 | 0.128 | 0.61 | 9.6 | 0.75 (0.45-0.99) |
Network robustness (percolation)
Both removal strategies leave the network fully robust up to about 65% of borrowers removed - a side effect of every borrower having at least 20 nearest-neighbour edges by construction. Past 65% removed, targeted removal of the highest-betweenness borrowers starts fragmenting the network measurably faster than an equally-sized random removal (more than 2 standard deviations below the random baseline). The gap is widest at 75% removed: targeted leaves 12% of the network connected vs 25% for random. That gap is evidence the bridge borrowers marked in the network chart are structurally load-bearing once enough of the graph is gone, even though no small removal set can fragment it on its own.
Intervention scenarios (impact-network style)
The question impact network analysis asks: give an intervention to k borrowers - which k? Where it starts is a targeting rule and budget; whether it takes hold is the borrower’s own historical contact rate; the effect and its uncertainty are the segment odds ratios above, drawn from their confidence intervals on each of 200 realizations; common random numbers make every rule a paired comparison against a random-k null.
| Intervention | Best rule (k = k_ref) | Uplift | Random | Gap (random SDs) | Meaningful |
|---|---|---|---|---|---|
| evening | at_risk | +11.9 ± 5.8 | +2.1 ± 1.5 | 6.4 | yes |
| evening_reminder | at_risk | +13.7 ± 10.1 | +3.7 ± 2.1 | 4.7 | yes |
| reminder | bridge | +3.9 ± 2.5 | +2.6 ± 1.9 | 0.6 | no |
Giving 100 borrowers evening calling: targeting the borrowers least likely to pay on their own yields +11.9 expected additional payers (SD 5.8, +68881 dollars) against +2.1 for a random 100 (SD 1.5). That gap of 9.8 payers is more than random targeting’s own noise - 6.4 random-targeting SDs, so where the intervention lands matters here. Targeting by network position (bridge borrowers) is indistinguishable from random here (-0.0 SD): betweenness on a similarity network says who resembles whom, not who will respond, because nothing spreads between borrowers - which is exactly the layer this analysis lacks compared with impact network analysis.
What this deliberately lacks is INA’s second layer: nothing spreads between borrowers, so there is no dispersal network - and that is exactly why targeting by network position equals random here. Position on a similarity network says who resembles whom, not who will respond.
Evaluation against the planted truth
| Check | Result |
|---|---|
| modularity_beats_null | pass |
| evening_effect_found_in_delayed | pass |
| reminder_effect_found | pass |
| segments_differ_in_payment | pass |
| weekday_null_stayed_null | pass |
| model_beats_baseline | pass |
Limitations
- Synthetic data. What transfers is the method and its checks, not the numbers.
- ARI 0.42 is moderate; the generator plants noise and recovery is partial by design.
- The model is modest, and the scenario baseline is the segment’s observed payment rate, not a per-borrower score: scoring the training borrowers would be in-sample.
- Single-layer network; no transmission, so no dispersal layer.
- Odds ratios estimated on pick-up and promise completion are applied to payment odds directly in the scenarios, which overstates the effect wherever a pick-up gain does not convert.