I’ve been building and debugging a custom AI-driven SOC platform from the ground up using Elasticsearch, FastAPI, Python detection pipelines, and a live investigation dashboard.
This started as a simple SOC dashboard experiment, but quickly evolved into a real security engineering project involving backend architecture, detection engineering, incident processing, API authentication, and live frontend synchronization.
What has been implemented so far:
- Installed and configured Elasticsearch on a VPS
- Built a custom Python detection engine querying Elasticsearch directly
- Created live brute-force and suspicious activity detections
- Built a FastAPI-based incident ingestion backend
- Added token-authenticated /incident APIs
- Implemented incident normalization pipelines
- Added MITRE ATT&CK mapping enrichment
- Created a dynamic Risk Scoring Engine
- Connected incidents to a live frontend investigation dashboard
- Replaced static/mock data with real-time incident ingestion
- Built live sync workflows between backend and frontend
- Fixed PM2 crashes and backend restart failures
- Solved Elasticsearch 8.x ↔ Python client incompatibilities
- Recovered broken Python virtual environments
- Built reproducible environments using pip freeze > requirements.txt
A big part of this project wasn’t just coding — it was solving real infrastructure and engineering problems:
- curl: (52) Empty reply from server
- FastAPI route failures
- PM2 process crashes
- Elasticsearch authentication issues
- API version mismatches
- Python dependency conflicts
- backend/frontend integration bugs
- incident rendering issues
- live sync failures
Current architecture:
Elasticsearch→ Custom Detection Engine→ Incident Normalization→ MITRE ATT&CK Mapping→ Risk Scoring Engine→ FastAPI Incident API→ Live Incident Investigation
Dashboard
The platform is now successfully ingesting and displaying real security incidents live from Elasticsearch into the investigation UI.
Some of the commands/workflows used during the build:
python3 -m venv venv source venv/bin/activate pip install fastapi uvicorn elasticsearch requests uvicorn main:app --host 0.0.0.0 --port 80.. pip freeze > requirements.txt curl http://localhost:80../api/incident
Brute-force incident testing:
curl -X POST http://localhost:80../incident \ -H "Authorization: Bearer Token.. \ -H "Content-Type: application/json" \ -d '{ "title":"RDP Bruteforce", "severity":"critical", "category":"unauthorized_access", "ip":"192.168.1.0" }'
Slowly turning this into a real AI-powered security operations platform, one layer at a time.

