The Fifth Elephant 2016

• What is Security Analytics

• How Symantec discovers risks and weaknesses in Enterprises

• Data Collection and Storage

• Summarizing data with Hybrid OLAP
o Relational OLAP
o Multidimensional OLAP

• Application of Probabilistic Data Structures

• Exposing Apache Spark’s strenghts with REST layer on top

• HBase on steroid with Co-Processors

Outline

Symantec is the leading security solution provider on this planet. It has top of the line products like Norton Antivirus™ and Symantec Endpoint Protection under its portfolio, securing retail and enterprise endpoints. Much of its capability is derived from endpoint data intelligence. Given that the amount of log data generated in few months reaches petabytes, the long term correlation poses a tough challenge to detect severe network attacks like advanced persistent threats or targeted campaigns. To familiarize the audience we are going to give a brief overview of security analytics.

Our team is focused on crunching data at large scale. Data collection and processing happens in both streaming and batch mode. We are going to describe a Hybrid Online Analytics Processing(OLAP) framework built on top of Apache Spark, Hbase and Hive. We are going to offer some design tips and tricks to fetch queries in web scale on Big Data with Hbase co-processors and getting real time response from Apache Spark with customized REST layer on top.
Application of some the probabilistic data structures like Hyperloglog and Count Min Sketch in security domain.

Speaker bio

Pratim is currently working as an Architect in Symantec CTO team that is building Unified Security Platform. He has more than 13 years of experience building enterprise products and being part of a successful sillicon valley startup in the past. He holds a bachelors degree in CSE from IIT-Kharagpur.

Links

• https://in.linkedin.com/in/pratim-mukherjee-bb52103

Slides

http://www.slideshare.net/pratimmukherjee/security-analytics-at-web-scale