BotSense
Botnet detection that explains every alert in terms an analyst can act on.
My part: the explainable AI module, the monitoring dashboard, and the team’s Git repository.
The problem
A detection model can flag traffic as malicious, but a bare score doesn’t tell a security analyst what is happening or what to do next. We wanted every alert to come with a reason.
How the system works
The team’s pipeline has two stages, trained on four public datasets: N-BaIoT, IoT-23, CTU-13 and CIC-IDS-2017.
- Stage 1: a Random Forest sorts traffic into IoT and non-IoT. Accuracy 96.70%, AUC-ROC 0.9989.
- Stage 2: a CNN-LSTM detector per branch flags botnet activity. On a held-out test set, the IoT detector reached 99.91% accuracy, 99.95% precision, 99.92% recall and 99.93% F1.
What I built
- Integrated Gradients, written from scratch with PyTorch autograd, to explain the deep learning detectors.
- SHAP to explain the Random Forest.
- A rule engine that turns each explanation into a threat pattern, a severity level and a recommended action.
- Explaining one flow takes about 85 to 95 ms, depending on the detector.
- A SOC dashboard in PyQt6: live monitoring, PCAP upload, detection history and the full explanation for each alert.
What surprised us
The slow part wasn’t where we expected. We assumed explanations were slowing the system down. Measuring showed it was the LSTM, because it processes a sequence one step at a time.
The datasets didn’t fit together. The four datasets differ a lot in format and features, so preprocessing took far longer than planned.
The most accurate model wasn’t the one we shipped. XGBoost scored 99.56% against Random Forest’s 96.70%, but its saved model crashed with a segfault on Apple Silicon. We shipped Random Forest: about 3 points less accurate, but it ran on every machine the team used.
What I learned
A high score on a public dataset is a starting point, not proof. The next real test is live network traffic.