Top 4 methods for Anomaly Detection in time series data

Anomaly detection is one of the most commonly asked interview questions. Below I summarize 4 top methods for detection in time series data. And corresponding code for using it.

1. Visualize the Data: Plotting the time series data is an important first step to identify any anomalies in the data. This will give us a better understanding of how the data behaves and help us identify any outliers.

Sometimes the data is too noisy that you may not be able to detect the anomalies by visualization and that may need further analysis.

2. Statistical Modeling: Using statistical models such as ARIMA, Holt-Winters or Exponential Smoothing can help detect anomalies by analyzing past values, trends and seasonality in your data points. If a point does not fit into this model, then it could be considered anomalous.

3. Machine Learning Algorithms: Using machine learning algorithms such as clustering or deep learning techniques can also be used for anomaly detection on timeseries data. Clustering algorithms can divide the data points into clusters and identify outliers that do not fit into any of the clusters. Deep learning models such as LSTMs can also be used to detect anomalies in timeseries data by learning patterns from past data points.

4. Data Mining: Data mining techniques such as association rule mining, outlier detection or change point analysis can also be used for anomaly detection on timeseries data. These methods look for changes in the distribution of your data and identify unusual patterns that could indicate an anomaly.

Now, its often tough to start right into coding the anomaly detection methods, so below are two approaches in deep learning, One using LSTM and another using Autoencoders.

1. Following is the code for LSTM in pytorch for 210 dimensions. You can extend it to as many dimensions as you want.

import torch 
from torch import nn, optim 
import numpy as np 

# Define the dimensions of our input and output data. 
input_dim = 210 # Number of features in our input dataset 
output_dim = 1 # Number of classes (binary classification) 
# Create an instance of a Long Short-Term Memory (LSTM) 
# network with 2 hidden layers and dropout rate 0.2. 

model = nn.LSTM(input_size=input_dim, hidden_size=2, num_layers=3, dropout=0.2) 

# Define the loss function as binary cross entropy 
criterion = nn.BCELoss() 

# Define the optimizer…