Delivering Security Insights with Data Analytics and Visualization
•Download as PPTX, PDF•
10 likes•3,716 views
It's an interesting exercise to look back to the year 2000 to see how we approached cyber security. We just started to realize that data might be a useful currency, but for the most part, security pursued preventative avenues, such as firewalls, intrusion prevention systems, and anti-virus. With the advent of log management and security incident and event management (SIEM) solutions we started to gather gigabytes of sensor data and correlate data from different sensors to improve on their weaknesses and accelerate their strengths. But fundamentally, such solutions didn't scale that well and struggled to deliver real security insight. Today, cybersecurity wouldn't work anymore without large scale data analytics and machine learning approaches, especially in the realm of malware classification and threat intelligence. Nonetheless, we are still just scratching the surface and learning where the real challenges are in data analytics for security. This talk will go on a journey of big data in cybersecurity, exploring where big data has been and where it must go to make a true difference. We will look at the potential of data mining, machine learning, and artificial intelligence, as well as the boundaries of these approaches. We will also look at both the shortcomings and potential of data visualization and the human computer interface. It is critical that today's systems take into account the human expert and, most importantly, provide the right data.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Delivering Security Insights with Data Analytics and Visualization
Similar to Delivering Security Insights with Data Analytics and Visualization (20)
More from Raffael Marty
More from Raffael Marty (20)
Recently uploaded
Recently uploaded (9)
Delivering Security Insights with Data Analytics and Visualization
- 1. Delivering Security Insights with Data Analytics and Visualization Raffael Marty VP Security Analytics ACSAC Orlando November 2017
- 2. Disclaimer © Raffael Marty 2 "This presentation was prepared solely by Raffael Marty in his personal capacity. The material, views, and opinions expressed in this presentation are the author's own and do not reflect the views of Sophos Ltd. or its affiliates."
- 3. Raffael Marty • Sophos • PixlCloud • Loggly • Splunk • ArcSight • IBM Research • SecViz • Logging • Big Data • ML & AI • SIEM • Leadership • Zen
- 4. 4 The master of Kennin temple was Mokurai. He had a little protégé named Toyo who was only twelve years old. Toyo saw how students entered the masters room each day and received instructions and guidance in Zen. The young boy wished to do zazen (meditation) as well. Upon convincing Mokuri, he went in front of the master who gave him the following koan to ponder: "You can hear the sound of two hands when they clap together," said Mokurai. "Now show me the sound of one hand."
- 5. Outline 5 • Big Data for Security • A Security (Big) Data Journey • Machine Learning and Artificial Intelligence • Data Visualization • Solving Security Problems with Data • A Glimpse Into the Future • My 5 Security Big Data Challenges
- 6. Big Data For Security 6
- 7. “memory has become the new hard disk, hard disks are the tapes of years ago.” -- unknown source 7
- 8. Security Data Data • infrastructure / network logs (flows, dns, dhcp, proxy, routing, IPS, DLP, …) • host logs (file access, process launch, socket activity, etc.) • HIPS, anti virus, file integrity • application logs (Web, SAP, HR, …) • metrics • configuration changes (host, network equipment, physical access, applications) • indicators of compromise (threat feeds) • physical access logs • cloud instrumentation data • change tickets • incident information Context • asset information and classification • identity context (roles, etc.) • information classification and location (tracking movement?) • HR / personnel information • vulnerability scans • configuration information for each machine, network device, and application
- 9. Big Data Systems – A Complex Ecosystem 9 Storing any kind of data o Schema-less but with schema on demand o Storing event data (time-series data, logs) o Storing metrics Data access o Fast random access o Ad-hoc analytical workloads o Search o Running models (data science) Data processing needs o Metric generation from raw logs o Real-time matching against high volume threat feeds o Anonymization o Building dynamic context from the data o Enrichment with entity information Use-cases • Situational awareness / dashboards • Alert triage • Forensic investigations • Incident management • Reports (e.g., for compliance) • Data sharing / collaboration • Hunting • Anomaly detection • Behavioral analysis • Pattern detection • Scoring requires
- 10. Are Today’s Systems Ready For Big Data Use Cases? 10 Data Sources • Haven’t been built with analysis in mind • Logs are incomplete • Log formats are not standardized Log mgmt | SIEM | “Big Data Lakes” • Don’t scale well to volumes, variety, and velocity • No standard data pipelines – results in point to point integrations that are imperfect • No standard storage concepts – results in data duplication • No standard use-cases – results in ‘spaghetti architectures’
- 11. Security (Big) Data Journey 11Image credit: http://journeyofhealth.org/
- 12. (Incomplete) Security Data History 12 “Big Data Is An Old Problem in Security” 1980 Firewalls, IPSs, OSs, Apps, Infra, etc. SecurityBigData syslogd(8) 1996 Log Management and first SIM “Big Data” in security RDBMS (way earlier already) 2004 CEF Standard (2007 CEE) 2006 2009 2014 2016 First logging as a service offering Security Data Lake Apache Metron (Open SOC) Apache Spot Distributed storage and processing (Hadoop 0.1.0) AWS (re-launch) Kafka Separation of query engines and data stores (Presto, Drill, parquet, etc.) Continued innovation on cloud platforms (Athena, S3, etc.) First RAID conference (ML / AD) ML is slow and missing training data First VizSec conference Device and user-context correlation First ”security analytics” solution Deep Learning in security (traffic and malware identification) ”Big Bang of Deep Learning” First unstructured data store and search engine (Solr) Columnar data stores become popular (MonetDB, etc.) R (previously S) Data Lake Data centralization Data insight
- 13. Security Data – The State Today 13 • “Security Data Lakes – an excuse to collect anything without having to think about schemas and access patterns.” • Data and infrastructure challenges to overcome o Data standardization (parsing, schemas) - Meaning of log entries and fields within - When is a log generated, when not? o Data infrastructure - One architecture for all use-cases - Self maintaining and healing o Building ‘content’ across customers? - Different policies - Different data sources and configurations o Data Privacy
- 14. 14http://theconversation.com/your-questions-answered-on-artificial-intelligence-49645 Data Science Data Mining Machine Learning Artificial Intelligence
- 15. ML and AI – What Is It? 15 • Machine learning – Algorithmic ways to “describe” data o Supervised - We are giving the system a lot of training data and it learns from that o Unsupervised - We give the system some kind of optimization to solve (clustering, dim reduction) • Deep learning – a ‘newer’ machine learning algorithm o Eliminates the feature engineering step o Verifiability issues • Data Mining – Methods to explore data – automatically and interactively • Artificial Intelligence – “Just calling something AI doesn’t make it AI.” ”A program that doesn't simply classify or compute model parameters, but comes up with novel knowledge that a security analyst finds insightful.”
- 16. Machine Learning in Security 16 • Supervised o Malware classification - Deep learning on millions of samples - 400k new malware samples a day - Has increased true positives and decreased false positives compared to traditional ML o Spam identification • Unsupervised o Tier 1 analyst automation (reducing workload from 600M events to 100 incidents)* o User and Entity Behavior Analytics (UEBA) - Uses mostly regular statistics and rule-based systems * See Respond Software Inc.
- 17. Application of Machine Learning - Anomaly Detection Objective : Find ‘security incidents’ in the data – deviations from the ‘norm’ • What’s “normal”? • Needs explainability for clusters • Observe clusters over time (requires stable ‘incremental’ clustering) • Even 0.01% of false positives are too high (1m log records -> 100 anomalies)
- 18. Limits of Machine Learning 18 “Everyone calls their stuff ‘machine learning’ or even better ‘artificial intelligence’ - It’s not cool to use statistics!” “Companies are throwing algorithms on the wall to see what sticks - see security analytics market” Machine Learning Challenges • An algorithm is not he answer. It’s the process around it (find the best fit algorithm for the data and use-case, feature engineering, supervision, drop outs, parameter choices, etc.) • Even in deep learning, it’s not just about using tensorflow. Features matter (e.g., independent bytes versus program flow) • The algorithms are only as good as the data and the knowledge of the data o Common data layers / common data models o Enriched data o Clean data (e.g, source/destination confusions) • How do we build systems that incorporate expert knowledge?
- 19. Illustration of Parameter Choices and Their Failures • t-SNE clustering of network traffic from two types of machines perplexity = 3 epsilon = 3 No clear separation perplexity = 3 epsilon = 19 3 clusters instead of 2 perplexity = 93 epsilon = 19 What a mess
- 20. Illustration of Parameter Choices and Their Failures • Dangerous clusters
- 21. Adversarial Machine Learning 21 • An example of an attack on deep learning
- 22. The Role of 22
- 23. S e c u r i t y . A n a l y t i c s . I n s i g h t . “How Can We See, Not To Confirm - But To Learn” - Edward Tufte
- 25. Visualization Overview 25 • Why? o Verify output of machine generated intelligence o Focus experts where they are most useful, rather than having them build tools / queries to understand the data o Enable exploration and hunting • What are the limitations? o Data is always a problem – we need clean, enriched data o Visualization of large data sets o Interpretation is hard - “And the single port with no traffic is port 0, which is reserved [24]” found in “Visualization of large scale Netflow data” by Nicolai H Eeg-Larsen - “… and the destinations are Internet Web Server or DNS server or both with the port 0.” - “.. so many TCP port scans are distributed in the whole day that most of them can be considered as false positives.” https://www.researchgate.net/publication/257686749_IDSRadar_A_real-time_visualization_framework_for_IDS_alerts
- 26. VAST Challenge 2013 Submission – Spot the Problems? 26 dest port! Port 70000? src ports! http://vis.pku.edu.cn/people/simingchen/docs/vastchallenge13-mc3.pdf
- 27. Visualization Challenges 27 • Backend o Super quick data access in any possible way (search, scan, summarize) o Ability to ingest any data source - intelligent parsing anyone? • User Interface o The right visualization paradigms o How to visualize 1m records? o The right data abstractions / summarizations / aggregations o Easy to use and still flexible enough • Data Science o Make the machine help us interpret the data • How to encode domain knowledge?
- 28. Visualization Challenges - Security Metrics 28 • How to quantify ‘security’? • Provide context
- 29. Solving Security Problems With Data 29
- 30. Solving Security Problems With Data Objective: Automatically detect “problems” / attacks with data Solution: Not ML or AI – the right process for the problem at hand • Any data science approach: o Encode domain knowledge – leverage trained experts (e.g., malware classification with n-grams, or URLs) o Involve the right ‘entities’ (e.g., push problems out to the end user) o Collect the right data for the given use-cases – don’t forget context and cleaning o Plan for expert feedback / validation loop o Build solutions for actual problems with real data that produce actionable insight o Share your insights with your peers – security is not your competitive advantage • Supervised: o Be selective on the problems that have good, large training data sets • Unsupervised: o We need good distance functions. Ones that encode domain knowledge!
- 31. Applications of Data in Security 31 • Prioritize event and entity data • Rule-based correlations • Behavior modeling • Risk / exposure / threat computation • Configuration assessments • Data classification • Data abstraction • Cross ‘boundary’ data sharing • Cross ‘customer’ analytics • Crowd intelligence • Enable free-form exploration • Identify and attribute attacks • Incident response • Improve prevention • Allocate / prioritize work / resources • Situational awareness • Understand exposure • Risk inventory • Spam, malware detection • Feedback loop on initiatives • Simplify security • Continuous attestation • Micro segmentation • Risk informed, dynamic enforcement (automation) Data Data Operations Applications Data is a core driver for many or most security use-cases
- 32. A Glimpse Into The Future 32http://www.aberdeenessentials.com/techpro-essentials/business-leaders-can-utilize-data-even-without-technology-background/
- 33. My Magic 8 Ball • Data is distributed across the edge and (a) central data store o We will have a (data lake)++ in every company with all security data (likely in the cloud) o Centralize data for correlation (could we get a decentralized correlation system?) o Keep raw sensor data at the edge and access through federated query system o Threat intelligence will be tailored to your organization and exchanged in real-time • APIs will be everywhere to let products integrate with each other • Security Analytics as a product category, as well as orchestration will merge with the data platforms (SIEM++) • Algorithms take a back seat – insights are key o Nobody cares whether you call something artificial intelligence or machine learning. It’s about actual results o Products will learn from users more and more • Startups will deliver innovation, but only large organizations will be able to deliver on the overall security promise • Detection is great. Protection is key. Closing the loop between insight and action. o Continuous attestation o Risk-based defense • No 3D visualizations
- 34. Thoughts on How We Get There 34 • Focus on three types of users o Data scientists and hunters – that now how to program, have security domain knowledge, and can find complex insights o Security analysts – that are using product interfaces to deal with security issues that the system couldn’t deal with automatically o Non security experts – that need insight into what is happening, but don’t know enough to intervene • AWS will productize the ’all encompassing data backend’ (others will contribute the technology) o Abstracting the data storage layer o Self-optimizing and monitoring query engine • Hire and train good UX people • Hire and train security domain experts o ”A course doesn’t make you a data scientist – not a good one at least”. It’s about the domain knowledge! • Use deep belief networks rather than deep learning • Build systems that help analysts and exports be more effective o Don’t try to replace them - let them do the interesting work o Don’t make up use-cases. Go into organizations and learn what the real problems are o Understand the user personas you are catering to o Stop building islands of products – SA is a feature – how do we build that on top of a common platform? o Move away from algorithm thinking into use-cases and workflows • Collect all your data (network and endpoint) in one data store
- 36. My 5 Challenges • Establish a pattern / algorithm / use-case sharing effort • Define a common data model everyone can buy into (CIM, CEF, CEE, Spot, etc.) o Including a semantic component for log records, not just syntax • Build a common entity store o Hooked up to a stream of data it automatically extracts entities and creates a state store o Allows for fast enrichment of data at ingest and query time o Respects and enforces privacy • Design a great CISO dashboard (framework) o Risk and “security efficiency” oriented, actionable views • Develop systems that ’absorb’ expert knowledge non intrusively
- 37. Questions? 37 http://slideshare.net/zrlram @raffaelmarty "You can hear the sound of two hands when they clap together," said Mokurai. "Now show me the sound of one hand."