Detecting anomalous network patterns

One of the many challenges a SRE/Devops/Cloud Security Officer has to face in his or her job is to know at any time what is going on in the cloud egress network. They must perform continuous analyses and checks to determine which cloud systems communicate which each other , which cloud system are sending data outside and which protocols are they employed. What about the data that is transferred to third parties and the data entering the network from outside? All this information must be available for evaluation at any time, even if that particular period of time lies in the past.
Differenciate between organic & in-organic trafiic pattern is difficult. Anomaly algorithm identifies when a metric is behaving differently than it has in the past, taking into account trends, seasonal day-of-week, and time-of-day patterns. It is well-suited for metrics with strong trends and recurring patterns that are hard to monitor with threshold-based alerting.
We at dream11 used anomaly detection algorithm to detect anomaly data pattern in

1. Network Bytes IN / OUT, TCP connection attempt/ connection establish / connection close, TCP retransmission & packet drop
2. DNS successful lookups , #SERVFAIL, #NXDOMAIN