πŸ›‘οΈ Production-Ready Location Fraud Detection

Technology-Agnostic Blueprint for Engineering Teams

< 10ms API Response
≀ 30s Background Processing
< 1KB Per User State
7M+ Users @ 150MB
⚑ Ultra-Minimal Redis User-State Model
< 150MB

Complete fraud state for 7 million active users

Field Type / Size Purpose Implementation Notes
fs uint16 (0-1000) Current fraud score Single monotonic scale simplifies alert routing
fc uint8 (0-100) Confidence in fs Simple % value; API maps to LOW/MED/HIGH
f uint8 bitmap 8 instantaneous risk flags Fits in one byte; bitmap ops are O(1) in Redis
lh3 uint32 Last H3 index (β‰ˆ 20m @ res 10) 4 bytes store the "where"
lt uint32 Unix-seconds of lh3 2Γ— uint32 keeps epoch math trivial
ls uint8 enum Location source (GPS/WiFi/IP/Peer/Sensor) Avoids string parsing on hot path
ph uint8 (0-100) Rolling peer-trust score Used for quick gating of peer reports
lut uint32 Last time Tier 2 refreshed fs Separates data seen vs analysis done
lvt uint32 Last time any event seen For inactivity expiry / cold-start heuristics
hb_h3 uint32 (optional) Stable "home-base" cell Only written when Tier 3 has high certainty

πŸ“ Memory Calculation

1 hash Γ— 14 bytes raw β‰ˆ 20 bytes after Redis zip-list overhead

7 million active users = 140MB total memory footprint

🚩 8-Bit Fraud Detection Flags
Bit 0: Impossible Travel
Bit 1: GPS/IMU Mismatch
Bit 2: Known Bad IP/VPN
Bit 3: Peer Disagreement
Bit 4: Timezone Mismatch
Bit 5: High Risk Geofence
Bit 6: Location Instability
Bit 7: Behavioral Anomaly

Each flag uses a single bit, allowing for efficient bitwise operations and minimal memory usage. All 8 flags fit in a single uint8, enabling O(1) flag checks and updates.

βš™οΈ Three-Tier Analytical Framework
🚨 Tier 1: Immediate Triage
< 5s E2E
Step Logic State
Fetch hot state Pull usr:{id} hash (lh3, lt, f) READ
Velocity check v = haversine(lh3,newH3)/(t-lt); if v>Vmax set f.bit0 WRITE f
GPS Γ— IMU sanity Compare Ξ”v from GPS to IMU vector; flag f.bit1 on mismatch WRITE f
IP/VPN lookup Test client IP against vpn_ips_bloom; if hit set f.bit2 READ bloom
Update location lh3=newH3, lt=t, ls=source, lvt=t WRITE
Emit trigger Publish (user-id, changedBits, lh3) β†’ tier1_processed β€”

Pure stream processing: All math is stateless except 1-hop look-back, enabling horizontal scaling of Tier 1 micro-services.

πŸ” Tier 2: Context Analysis
≀ 30s SLA
πŸ“š Tier 3: Deep Learning
Hours/Days
Sub-task Cadence Algorithm
Spatial baseline mining 6-hour window DBSCAN on 30-day history β†’ home/work H3 clusters
Travel-corridor profiling nightly Frequent-itemset of (origin-dest) H3 pairs
Peer trust graph update hourly TrustGNN/PageRank on 7-day peer-edges
Bloom & map refresh as feeds arrive Re-build vpn_ips_bloom, bssid_loc_h3, sensor_loc_h3
πŸ“Š End-to-End Data Flow

1. SDK Event

Compressed location data β†’ Kafka topics

β†’

2. Tier 1 Process

Stateless checks, Redis update ≀ 5ms

β†’

3. API Response

Single Redis hash read < 10ms P99

β†’

4. Tier 2 Analysis

30s context analysis, score update

β†’

5. Tier 3 Learning

Baseline refresh, model training

POST /location-score-assessment < 10ms

Input:
{ "user-id": "12345", "location": { "lat": 37.7749, "lon": -122.4194, "acc": 5, "altitude": 52 }, "bssid": "aa:bb:cc:dd:ee:ff", "ssid": "HomeWiFi", "timezone": "PST", "systemclock": 1625097600, "uptime": 86400 }
Response:
{ "score": "MEDIUM", "confiability": 0.85, "reasons": [ "IMPOSSIBLE_TRAVEL", "GPS_IMU_MISMATCH" ] }
🎯 Key Algorithmic Challenges & Solutions

❄️ Cold-Start (New Users)

  1. Population priors – start fs=baseline_mean, fc=20
  2. Strong external signals dominate – IP-VPN hit, impossible velocity immediately drop fs
  3. Peer-assisted bootstrap – trusted peers (ph > 80) corroborate, raise fc quickly

πŸ“ˆ Dynamic Risk Adaptation

Risk Tier Tier 2 Frequency Data Horizon Extra Actions
HIGH (fs<300) every event 5 min Push blocking webhook
MED (300-700) 30s default 30 min Normal pipeline
LOW (>700) 2 min 2 hours Opportunistic caching

πŸ“ Evidence Package / Reason Codes

For every Redis write:

usr:{id}:evidence -> LIST capped 20 LPUSH evidence "2025-07-01T12:30Z|IMPOSSIBLE_TRAVEL|velocity=1123km/h" TRIM evidence 0 19

API can return last N reason codes alongside score. Flags map 1-to-1 to human-readable codes.

🀝 Peer Network Trust Dynamics

Trust score update (Tier 2):

if peer_report agrees_with_consensus: ph = min(100, ph + Ξ±) else: ph = max(0, ph – Ξ²*(1 – trust(peer)))
  • Decay: ph = ph * 0.98 each hour without confirmation
  • Sybil control: Tier 3 caps trust for high-mutual/low-external edge clusters
🎯 Performance Guarantees & Targets

πŸš€ API Performance

  • βœ… ≀ 10ms – Single Redis hash GET + JSON mapping
  • βœ… P99 < 15ms – Including network overhead
  • βœ… Horizontal scaling – Shard on user-id

⚑ Processing Latency

  • βœ… Tier 1 < 5s – Stream-only math, Redis O(0.1ms)
  • βœ… Tier 2 ≀ 30s – Bounded window, OLAP ~100ms
  • βœ… Tier 3 hours – Batch ML model training

πŸ’Ύ Memory Efficiency

  • βœ… β‰ˆ 20 bytes per user hot state
  • βœ… Bloom filters < 100MB for tens of millions IPs
  • βœ… No cross-partition joins except Tier 3 batch
πŸ› οΈ Technology Stack Mapping
Stream Processing
  • Apache Kafka (events)
  • Apache Flink (Tier 1)
  • LZ4/Snappy compression
Hot Cache
  • Redis Enterprise
  • Hash zip-list optimization
  • Bloom filters
Workflow Engine
  • Temporal/Prefect (Tier 2)
  • Durable execution state
  • Idempotent retries
Operational DW
  • Apache Doris/ClickHouse
  • Materialized views
  • Recent history (1-7 days)
Long-term Storage
  • Apache Druid
  • Historical events
  • ML model training
Spatial Systems
  • H3 Hexagonal Indexing
  • Haversine distance
  • Geohash alternatives
πŸ—ΊοΈ Implementation Roadmap

Phase 1: Foundation (Weeks 1-4)

  • βœ… Redis cluster setup with hash optimization
  • βœ… Kafka topics with compression
  • βœ… Basic Tier 1 stream processor
  • βœ… Simple API endpoint (GET Redis)
  • βœ… H3 spatial indexing integration

Phase 2: Core Logic (Weeks 5-8)

  • βœ… Physics-based impossibility detection
  • βœ… Bloom filter for IP blacklists
  • βœ… Workflow engine setup (Temporal/Prefect)
  • βœ… Tier 2 basic signal fusion
  • βœ… Evidence package system

Phase 3: Advanced Features (Weeks 9-12)

  • βœ… Peer verification network
  • βœ… Multi-signal consistency checks
  • βœ… Dynamic risk tier adaptation
  • βœ… Operational DW integration
  • βœ… Advanced scoring algorithms

Phase 4: ML & Optimization (Weeks 13-16)

  • βœ… Tier 3 behavioral baselining
  • βœ… Graph neural networks (TrustGNN)
  • βœ… LSTM trajectory anomaly detection
  • βœ… Performance optimization
  • βœ… Production monitoring & alerting
πŸ›‘οΈ Resilience & Failover Strategy

πŸ”„ Redis Cluster Resilience

  • Cluster-level replication keeps hot state read-available
  • Automatic failover with Redis Sentinel
  • Cross-AZ deployment for high availability
  • Tier 2 re-hydration from OLAP backlog on missing keys

βš™οΈ Workflow Engine Recovery

  • Idempotent by design via analysis-version checkpointing
  • Automatic retry from last checkpoint on crash
  • Durable execution state survives engine restarts
  • Cron fallback catches stragglers (lut < now-25s)

πŸ“Š Data Pipeline Resilience

  • Kafka replication across multiple brokers
  • Stream processor auto-scaling based on lag
  • Dead letter queues for failed events
  • Circuit breakers prevent cascade failures

🎯 Graceful Degradation

  • API falls back to last known score if Redis unavailable
  • Tier 1 continues with reduced accuracy if bloom filter fails
  • Tier 2 skips missing signals, adjusts confidence
  • Population priors used for completely new users
πŸ“ˆ Success Metrics & KPIs
99.9%
API Availability
Sub-10ms P99 response time
95%+
Fraud Detection Rate
< 1% false positive rate
30s
Max Processing Time
Tier 2 analysis SLA

πŸ“Š Operational Metrics

β€’ Memory usage: < 150MB @ 7M users
β€’ Kafka throughput: 100K+ events/sec
β€’ Redis ops: 1M+ reads/sec
β€’ Tier 1 latency: < 5s E2E
β€’ Tier 2 success: 99%+ completion
β€’ Data freshness: < 30s lag P95