AI Security URL & Log Analyzer
Local-first threat intelligence — no data leaves your machine.
Overview
Security teams are constantly under pressure to analyze suspicious URLs and log files quickly and accurately. Most cloud-based solutions require sending potentially sensitive data — internal IPs, usernames, API paths — to third-party servers. This project solves that by running a fine-tuned BERT-based phishing detection model entirely on-device, inside a Flask web application, with no external API calls for the core analysis.
The Problem
Cloud threat intelligence platforms expose sensitive log data to external servers. Small security teams lack the budget for enterprise tools, and manual URL/log triage is slow and error-prone. There was no lightweight, offline-capable solution that combined ML-based phishing detection with a visual dashboard.
The Solution
Built a self-contained Flask application that loads Hugging Face transformer models once at startup and caches them in memory for sub-second inference. The dashboard accepts raw URLs or pasted log content, runs them through the local model pipeline, and renders threat scores, IOC extractions, and a timeline heatmap — all without a single external API call for sensitive data.
Key Features
- Local BERT-based phishing detection with no external API dependency
- Real-time URL classification returning phishing/benign probability scores
- Log file upload and batch analysis with pattern and IOC extraction
- Threat timeline heatmap showing suspicious activity clusters
- TypeScript-powered frontend with live dashboard updates
- Model warm-start caching to eliminate repeated cold-start delays
- Input sanitization to prevent injection via malicious log payloads
Challenges & Learnings
The biggest challenge was cold-start latency — loading BERT-class transformer models takes 2–5 seconds the first time. Solved this by pre-loading the model at Flask startup using Python's module-level globals, so every subsequent request gets sub-100ms inference. Memory was another constraint: running a transformer on CPU requires ~1.5 GB RAM. Implemented lazy module loading and ensured only one model instance exists per server lifetime. On the frontend, synchronising TypeScript chart state with async WebSocket updates required careful promise chaining to avoid race conditions.
Tech Stack
Backend
AI / ML
Frontend
Security
More Projects