No Peeking Forward: Time-Conscious Graph Fraud Detection

In my final article [1], I threw out plenty of concepts centered round constructing structured graphs, primarily targeted on descriptive or unsupervised exploration of information by means of graph buildings. Nevertheless, after we use graph options to enhance our fashions, the temporal nature of the info have to be taken into account. If we need to keep away from undesired results, we have to be cautious to not leak future data into our coaching course of. This implies our graph (and the options derived from it) have to be constructed in a time-aware, incremental approach.

Knowledge leakage is such a paradoxical downside {that a} 2023 examine by Sayash Kapoor and Arvind Narayanan [2] discovered that, as much as that second, it had affected 294 analysis papers throughout 17 scientific fields. They classify the forms of knowledge leakages starting from textbook errors to open analysis issues. 

The problem is that in prototyping, outcomes usually appear very promising when they’re actually not. More often than not, individuals don’t understand this till fashions are deployed in manufacturing, losing the time and sources of a complete group. Then, efficiency often falls wanting expectations with out understanding why. This problem can develop into the Achilles’ heel that undermines all enterprise AI initiatives. 

…

ML-base leakage

Knowledge leakage happens when the coaching knowledge incorporates details about the output that received’t be out there throughout inference. This causes overly optimistic analysis metrics throughout improvement, creating deceptive expectations. Nevertheless, when deployed in real-time techniques with the right knowledge circulate, the mannequin predictions develop into untrustworthy as a result of it discovered from data not accessible.

Ethically, we should attempt to supply outcomes that really mirror the capabilities of our fashions, fairly than sensational or deceptive findings. When a mannequin strikes from prototyping to manufacturing, it mustn’t fail to generalize correctly; if it does, its sensible worth is undermined. Fashions that fail to generalize effectively can exhibit vital issues throughout inference or deployment, compromising their usefulness. 

That is particularly harmful in delicate contexts like fraud detection, which regularly contain imbalanced knowledge situations (with fewer fraud circumstances than non-fraud). In these conditions, the hurt attributable to knowledge leakage is extra pronounced as a result of the mannequin would possibly overfit to leaked knowledge associated to the minority class, producing seemingly good outcomes for the minority label, which is the toughest to foretell. This may result in missed fraud detections, leading to severe sensible penalties.

Knowledge leakage examples could be categorized into textbook errors and open analysis issues [2] as follows:

Textbook Errors:

• Imputing lacking values utilizing all the dataset as a substitute of solely the coaching set, inflicting details about the check knowledge to leak into coaching.
• Duplicated or very related cases showing each in coaching and check units, similar to pictures of the identical object taken from barely totally different angles.
• Lack of clear separation between coaching and check datasets, or no check set in any respect, resulting in fashions getting access to check data earlier than analysis.
• Utilizing proxies of consequence variables that not directly reveal the goal variable.
• Random knowledge splitting in situations the place a number of associated information belong to a single entity, similar to a number of declare standing occasions from the identical buyer.
• Artificial knowledge augmentation carried out over the entire dataset, as a substitute of solely on the coaching set.

Open issues for analysis:

• Temporal leakage happens when future knowledge unintentionally influences coaching. In such circumstances, strict separation is difficult as a result of timestamps could be noisy or incomplete.
• Updating database information with out lineage or audit path, for instance, altering fraud standing with out storing historical past, may cause fashions to coach on future or altered knowledge unintentionally.
• Complicated real-world knowledge integration and pipeline points that introduce leakage by means of misconfiguration or lack of controls.

These circumstances are a part of a broader taxonomy reported in machine studying analysis, highlighting knowledge leakage as a vital and sometimes an underinvestigated threat for dependable modeling [3]. Such points come up even with easy tabular knowledge, they usually can stay hidden when working with many options if every one will not be individually checked. 

Now, let’s take into account what occurs after we embody nodes and edges within the equation…

…

Graph-base leakage

Within the case of graph-based fashions, leakage could be sneakier than in conventional tabular settings. When options are derived from linked elements or topological buildings, utilizing future nodes or edges can silently alter the graph’s construction. For instance:

• methodologies similar to graph neural networks (GNNs) study the context not solely from particular person nodes but additionally from their neighbours, which may inadvertently introduce leakage if delicate or future data is propagated throughout the graph construction throughout coaching.
• when the graph construction is overwritten or up to date with out preserving the previous occasions means the mannequin loses worthwhile context wanted for correct temporal evaluation, and it could once more entry data within the incorrect time or lose traceability about attainable leakage or issues with the info that originate the graphs.
• Computing graph aggregations like diploma, triangles, or PageRank on all the graph with out accounting for the temporal dimension (time-agnostic aggregation) makes use of all edges: previous, current, and future. This causes knowledge leakage as a result of options embody data from future edges that wouldn’t be out there at prediction time.

Graph temporal leakage happens when options, edges, or node relationships from future time factors are included throughout coaching in a approach that violates the chronological order of occasions. This ends in edges or coaching options that incorporate knowledge from time steps that needs to be unknown.

…

How can this be fastened?

We will construct a single graph that captures all the historical past by assigning timestamps or time intervals to edges. To research the graph as much as a particular time limit (t), we “look again in time” by filtering any graph to incorporate solely the occasions that occurred earlier than or at that cutoff. This method is good for stopping knowledge leakage as a result of it ensures that solely previous and current data is used for modeling. Moreover, it affords flexibility in defining totally different time home windows for protected and correct temporal evaluation.

On this article, we construct a temporal graph of insurance coverage claims the place the nodes symbolize particular person claims, and temporal hyperlinks are created when two claims share an entity (e.g., telephone quantity, license plate, restore store, and so forth.) to make sure the proper occasion order. Graph-based options are then computed to feed fraud prediction fashions, fastidiously avoiding the usage of future data (no peeking).

The thought is easy: if two claims share a typical entity and one happens earlier than the opposite, we join them in the intervening time this connection turns into seen (determine 1). As we defined within the earlier part, the best way we mannequin the info is essential, not solely to seize what we’re actually on the lookout for, but additionally to allow the usage of superior strategies similar to Graph Neural Networks (GNNs).

In our graph mannequin, we save the timestamp when an entity is first seen, capturing the second it seems within the knowledge. Nevertheless, in lots of real-world situations, it’s also helpful to think about a time interval spanning the entity’s first and final appearances (for instance, generated with one other variable like plate or e-mail). This interval can present richer temporal context, reflecting the lifespan or energetic interval of nodes and edges, which is effective for dynamic temporal graph analyses and superior mannequin coaching.

Code

The code is obtainable on this repository: Hyperlink to the repository

To run the experiments, arrange a Python ≥3.11 atmosphere with the required libraries (e.g., torch, torch-geometric, networkx, and so forth.). It’s endorsed to make use of a digital atmosphere (through venv or conda) to maintain dependencies remoted. 

Code Pipeline

The diagram of Determine 2, reveals the end-to-end workflow for fraud detection with GraphSAGE. Step 1 hundreds the (simulated) uncooked claims knowledge. Step 2 builds a time-stamped directed graph (entity→declare and older-claim→newer-claim). Step 3 performs temporal slicing to create practice, validation, and check units, then indexes nodes, builds options, and at last trains and validates the mannequin.

Step 1: Simulated Fraud Dataset

We first simulate a dataset of insurance coverage claims. Every row within the dataset represents a declare and contains variables similar to:

• Entities: insurer_license_plate, insurer_phone_number, insurer_email, insurer_address, repair_shop, bank_account, claim_location, third_party_license_plate
• Core data: claim_id, claim_date, type_of_claim, insurer_id, insurer_name
• Goal: fraud (binary variable indicating whether or not the declare is fraudulent or not)

These entity attributes act as potential hyperlinks between claims, permitting us to deduce connections by means of shared values (e.g., two claims utilizing the identical restore store or telephone quantity). By modeling these implicit relationships as edges in a graph, we will construct highly effective topological representations that seize suspicious behavioral patterns and allow downstream duties similar to characteristic engineering or graph-based studying.

Step2: Graph Modeling

We use the NetworkX library to construct our graph mannequin. For small-scale examples, NetworkX is adequate and efficient. For extra superior graph processing, instruments like Memgraph or Neo4j might be used. To mannequin with NetworkX, we create nodes and edges representing entities and their relationships, enabling community evaluation and visualization inside Python.

So, we’ve: 

• one node per declare, with node key equal to the claim_id and attributes as node_type and claim_date
• one node per entity worth (telephone, plate, checking account, store, and so forth.). Node key: "{column_name}:{worth}" and attributes node_type = (e.g., "insurer_phone_number", "bank_account", "repair_shop") label = (simply the uncooked worth with out the prefix)

The graph contains these two forms of edges:

• claim_id(t-1)→ claim_id(t) : when two claims share an entity (with edge_type='claim-claim')
• entity_value →claim_id: direct hyperlink to the shared entity (with edge_type='entity-claim')

These edges are annotated with:

• edge_type: to differentiate the relation (declare→declare vs entity→declare)
• entity_type: the column from which the worth comes (like bank_account)
• shared_value: the precise worth (like a telephone quantity or license plate)
• timestamp: when the sting was added (based mostly on the present declare’s date)

To interpret our simulation, we applied a script that generates explanations for why a declare is flagged as fraud. In Determine 4, declare 20000695 is taken into account dangerous primarily as a result of it’s related to restore store SHOP_856, which acts as an energetic hub with a number of claims linked round related dates, a sample usually seen in fraud “bursts.” Moreover, this declare shares a license plate and deal with with a number of different claims, creating dense connections to different suspicious circumstances. 

This code saves the graph as a pickel file: temporal_graph_with_edge_attrs.gpickle. 

Step 3: Graph preparation & Coaching

Illustration studying transforms advanced, high-dimensional knowledge (like textual content, pictures, or sensor readings) into simplified, structured codecs (usually referred to as embeddings) that seize significant patterns and relationships. These discovered representations enhance mannequin efficiency, interpretability, and the power to switch studying throughout totally different duties.

We practice a neural community, to map every enter to a vector in ℝᵈ that encodes what issues. In our pipeline, GraphSAGE does illustration studying on the declare graph: it aggregates data from a node’s neighbours (shared telephones, outlets, plates, and so forth.) and mixes that with the node’s personal attributes to supply a node embedding. These embeddings are then fed to a small classifier head to foretell fraud.

3.1. Temporal slicing

From the only full graph we create in step 2, we extract three time-sliced subgraphs for practice, validation, and check. For every break up we select a cutoff date and hold solely (1) declare nodes with claim_date ≤ cutoff, and (2) edges whose timestamp ≤ cutoff. This produces a time-consistent subgraph for that break up: no data from the longer term leaks into the previous, matching how the mannequin would run in manufacturing with solely historic knowledge out there.

3.2 Node indexing

Give each node within the sliced graph an integer index 0…N-1. That is simply an ID mapping (like tokenization). We’ll use these indices to align options, labels, and edges in tensors.

3.3 Construct node options

Create one characteristic row per node:

• Kind one-hot (declare, telephone, e-mail, …).
• Diploma stats computed inside the sliced graph: normalized in-degree, out-degree, and undirected diploma inside the sliced graph.
• Prior fraud from older neighbors (claims solely): fraction of older linked claims (direct declare→declare predecessors) which can be labeled fraud, contemplating solely neighbors that existed earlier than the present declare’s time.
  We additionally set the label y (1/0) for claims and 0 for entities, and mark claims in claim_mask so loss/metrics are computed solely on claims.

3.4 Construct PyG Knowledge

Translate edges (u→v) right into a 2×E integer tensor edge_index utilizing the node indices and add self-loops so every node additionally retains its personal options at each layer. Pack all the pieces right into a PyG Knowledge(x, edge_index, y, claim_mask) object. Edges are directed, so message passing respects time (earlier→later).

3.5 GraphSage:

We implement a GraphSAGE structure in PyTorch Geometric with the SAGEConv layer. so, we run two GraphSAGE convolution layers (imply aggregation), ReLU, dropout, then a linear head to foretell fraud vs non-fraud. We practice full-batch (no neighbor sampling). The loss is weighted to deal with class imbalance and is computed solely on declare nodes through claim_mask. After every epoch we consider on the validation break up and select the choice threshold that maximizes F1; we hold one of the best mannequin by val-F1 (early stopping).

3.6 Inference outcomes.

Consider one of the best mannequin on the check break up utilizing the validation-chosen threshold. Report accuracy, precision, recall, F1, and the confusion matrix. Produce a carry desk/plot (how concentrated fraud is by rating decile), export a t-SNE plot of declare embeddings to visualise construction.

The carry chart evaluates how effectively the mannequin ranks fraud: bars present carry by rating decile and the road reveals cumulative fraud seize. Within the high 10–20% of claims (Deciles 1–2), the fraud fee is about 2–3× the typical, suggesting that reviewing the highest 20–30% of claims would seize a big share of fraud. The t-SNE plot reveals a number of clusters the place fraud concentrates, indicating the mannequin learns significant relational patterns, whereas overlap with non-fraud factors highlights remaining ambiguity and alternatives for characteristic or mannequin tuning.

…

Conclusion

Utilizing a graph that solely connects older claims to newer claims (previous to future) with out “leaking” future fraud data, the mannequin efficiently concentrates fraud circumstances within the high scoring teams, reaching about 2–3 occasions higher detection within the high 10–20%. This setup is dependable sufficient to deploy.

As a check, it’s attainable to strive a model the place the graph is two-way or undirected (connections each methods) and evaluate the spurious enchancment with the one-way model. If the two-way model will get considerably higher outcomes, it’s possible due to “temporal leakage,” which means future data is wrongly influencing the mannequin. It is a strategy to show why two-way connections shouldn’t be utilized in actual use circumstances.

To keep away from making the article too lengthy, we’ll cowl the experiments with and with out leakage in a separate article. On this article, we give attention to creating a mannequin that meets manufacturing readiness.

There’s nonetheless room to enhance with richer options, calibration, and small mannequin tweaks, however our focus right here is to elucidate a leak-safe temporal graph methodology that addresses knowledge leakage.

References

[1] Gomes-Gonçalves, E. (2025, January 23). Purposes and Alternatives of Graphs in Insurance coverage. Medium. Retrieved September 11, 2025, from https://medium.com/@erikapatg/applications-and-opportunities-of-graphs-in-insurance-0078564271ab

[2] Kapoor, S., & Narayanan, A. (2023). Leakage and the reproducibility disaster in machinelearning-based science. Patterns. 2023; 4 (9): 100804. Hyperlink.

[3] Guignard, F., Ginsbourger, D., Levy Häner, L., & Herrera, J. M. (2024). Some combinatorics of information leakage induced by clusters. Stochastic Environmental Analysis and Danger Evaluation, 38(7), 2815–2828.

[4] Huang, S., et. al. (2024). UTG: In the direction of a Unified View of Snapshot and Occasion Based mostly Fashions for Temporal Graphs. arXiv preprint arXiv:2407.12269. https://arxiv.org/abs/2407.12269

[5] Labonne, M. (2022). GraphSAGE: Scaling up Graph Neural Networks. In the direction of Knowledge Science. Retrieved from https://towardsdatascience.com/introduction-to-graphsage-in-python-a9e7f9ecf9d7/

[6] An Introduction to GraphSAGE. (2025). Weights & Biases. Retrieved from https://wandb.ai/graph-neural-networks/GraphSAGE/studies/An-Introduction-to-GraphSAGE–Vmlldzo1MTEwNzQ1