CROSS REFERENCE TO RELATED APPS
This application is a branch continuation claiming priority of International Patent Application No. PCT/US22/30859 filed May 25, 2022, claiming priority of the US provisional patent application. No. 63/254,368 filed October 11, 2021, the entire contents of which are incorporated herein by reference.
CAMPO
This disclosure relates to the use of event streams to record, monitor, and investigate enterprise security.
FUND
As enterprise networks become more complex and security threats become more sophisticated, there continues to be a need for improved techniques for monitoring security events and for identifying and investigating potential security threats within the enterprise network.
SUMMARY
A platform for network threat research is supplemented by data from cloud resources, such as third-party cloud-based application platforms. The resulting blended data set can be incrementally updated and used to automatically launch investigations at appropriate times.
A platform for managing threat data integrates threat data from a variety of sources, including internal threat data from instrumented compute instances associated with an enterprise network and threat data from one or more independent external resources. Threat assessments are gradually reviewed as this threat data is received asynchronously from various sources, and a threat intervention container is automatically created and presented to an investigator when a composite threat score for one or more of compute instances reaches a predetermined threshold.
In one aspect, a system disclosed herein may include: a plurality of computing instances coupled to an enterprise network; and a data lake that stores a stream of events from various security data sources. The event stream may include: security events from one or more sensors in the plurality of computing instances attached to the business network; cloud resource data from a cloud service that supports the plurality of compute instances attached to the enterprise network; and contextual data for the activity of the plurality of computing instances of a third party service. The system may further include a threat management facility configured to perform the steps of: updating a composite threat score based on the number of security data sources as the event stream is received asynchronously from the number of security data sources; and the automatic creation of an investigation container for interactive investigation of security risks when the composite threat score reaches a predetermined threshold to initiate an investigation.
Implementations may include one or more of the following features. Security events can include asynchronous data from multiple compute instances. Security events may include batch data from one or more of the plurality of compute instances. The plurality of computing instances may include at least one network device for the enterprise network. The plurality of compute instances may include at least one virtual computing device hosted on a virtualization platform. Security events may include threat detection data from a local security agent running on one of the plurality of compute instances. Contextual data may include geolocation data from the third party service. The data lake may apply deduplication logic to eliminate one or more recurring detections from at least one of one or more sensors. The cloud service may include one or more of a web application, a cloud storage service, an email application, an authentication service, a zero-trust network access resource, a cloud computing service, the cloud and a virtualization platform. The cloud service may include a network monitor that runs on a third-party firewall and is securely coupled to the threat management facility. The number of security data sources may include a monitor for a query interface to the data lake. The threat management function can be configured to present a user interface associated with the research container on a screen to a user. The threat management function may be configured to convey a link to a user interface associated with the research container in a message to a user. The threat management function can be configured to display a user interface associated with the research container, the user interface including a control for requesting additional event data captured by a data logger from one of the plurality of compute instances before of a window of time before the creation of the research container. The threat management function can be configured to display a user interface associated with the research container, the user interface including a control for adjusting a filter applied by one of the plurality of compute instances to a local event stream by selecting local security events to communicate in the event stream to the data lake. The threat management function can be configured to display a user interface associated with the research container, including a data lake query tool user interface. The threat management function can be configured to calculate the composite threat score by assigning security events to an attack matrix that lists the malware strategies in a first dimension and the malware techniques for each of the malware strategies in a first dimension. second dimension, and calculating the composite threat score based on a pattern of traversal of the attack matrix through a timeline of security events. The threat management function can be configured to calculate the composite threat score by applying a machine learning algorithm to the attack matrix walk pattern to determine the threat probability. The composite threat score can include two or more scores based on two or more of the number of security data sources, where the default threshold includes a separate threshold for each of the two or more scores. The Threat Management feature can be configured to revise the Composite Threat Score downward when a false positive is identified and automatically close the investigation container when the Composite Threat Score reaches a predetermined second threshold to end the investigation.
The foregoing and other objects, features, and advantages of the devices, systems, and methods described herein will become apparent from the following description of particular embodiments thereof, as illustrated in the accompanying drawings. The drawings are not necessarily to scale, but rather are intended to illustrate the principles of the devices, systems, and methods described in this document.
DESCRIPTION
In the following, embodiments will be described with reference to the accompanying figures. However, the foregoing can be embodied in many different ways and should not be construed as limited to the illustrated embodiments set forth herein.
All documents mentioned in this document are incorporated herein by reference in their entirety. References to articles in the singular must be understood to include articles in the plural, and vice versa, unless otherwise explicitly stated or clarified in the text. Grammatical conjunctions are intended to express any and all disjunctive and conjunctive combinations of conjoined clauses, sentences, words, and the like, unless otherwise indicated or made clear by the context. Therefore, the term "or" should generally be understood as "and/or" and so on.
Listing of ranges of values in this document is not intended to be limiting, but refers individually to each and every value that falls within the range, unless otherwise stated in this document, and each separate value within of that range is incorporated into the specification as if it were listed individually. Whom. The words "about", "approximately" or the like, when accompanying a numerical value, are to be construed as indicating a deviation that would be appreciated by one of ordinary skill in the art to operate satisfactorily for an intended purpose. Similarly, words of approximation such as "approximately" or "substantially" when used in reference to physical characteristics, should be understood to contemplate a range of deviations that a person skilled in the art would appreciate in order to operate satisfactorily for a corresponding use. , function, purpose, or the like. Value ranges and/or numerical values are provided herein as examples only and do not constitute a limitation on the scope of the described embodiments. When ranges of values are provided, it is also intended to include each value within the range as if it were stated individually, unless expressly stated otherwise. The use of any and all examples or illustrative language ("eg", "such as", or the like) provided herein is intended merely to better illustrate the embodiments and is not intended to be a limitation in scope. of the achievements. No language in the specification should be construed as indicating anything not claimed to be essential to the practice of the embodiments.
In the following description, it is understood that terms such as "first", "second", "top", "bottom", "top", "bottom" and the like, are words of convenience and should not be construed as limiting terms.
It should also be understood that endpoints, devices, compute instances, or the like referred to as "inside" an enterprise network may also be "associated with" the enterprise network, for example, when such assets are outside a door. enterprise gateway but nonetheless managed by or in communication with a threat management facility or other centralized security platform for the enterprise network. Therefore, any description that refers to an asset within the enterprise network should be understood as contemplating a similar asset associated with the enterprise network regardless of location in a network environment, unless a different meaning is explicitly provided or left clear from context.
As described in this document, a threat management system can use a sensor, event, analysis, and response (SEAR) approach to protect businesses against cybersecurity threats.
As an example only, users of the threat management feature100You can define and enforce policies that control access to and use of compute instances, networks, and data. Administrators can update policies, for example, by designating authorized users and terms of use and access. The Threat Management Facility100You can update and enforce those policies at various levels of control that are available, such as directing compute instances to control network traffic that can pass through firewalls and wireless access points, applications and data available from servers, applications and data that can be accessed by endpoints, and data and network resources that can be executed and used by endpoints. The Threat Management Facility100may provide many different services, and policy management may be offered as one of the services.
Moving on to a description of certain capabilities and components of the threat management system101, an exemplary business facility102it may be or may include any networked computing infrastructure. For example, the enterprise installation102they can be corporate, commercial, organizational, educational, governmental or similar. As home networks become more complicated and include more home and cloud computing instances, an enterprise installation102it can also, or instead, include a personal network, such as a home or home group. Company facilities102The computer network may be distributed among a plurality of physical facilities such as buildings on a campus, and located in one or a plurality of geographic locations. The enterprise installation configuration shown is merely exemplary, and it will be understood that there may be any number of compute instances, fewer or more of each compute instance type, and other compute instance types. As shown, the example enterprise installation includes a firewall10, a wireless access point11, a full stop12, A server14, a mobile devicesixteen, an apparatus or IOT device18, a cloud computing instance19, and a server20. Once again, the compute instances10-20represented are exemplary, and there may be any number or type of computing instances10-20at a given business facility. For example, in addition to the items represented in the business installation102, there may be one or more gateways, bridges, wired networks, wireless networks, virtual private networks, other computing instances, etc.
The Threat Management Facility100may include certain facilities, such as a policy management facility112, security management facility122, update installation120, ease of definitions114, installation of network access rules124, ease of corrective action128, installation of detection techniques130, application protection installation150, asset classification service160, entity model installation162, event collection facility164, event log function166, analysis center168, dynamic policy service170, identity management facility172and market management facility174as well as other facilities. For example, there might be a test facility, a threat research facility, and other facilities. It should be understood that the threat management function100it can be implemented in whole or in part on several different compute instances, with some parts of the threat management function on different compute instances at different locations. For example, part or all of one or more of the various facilities100,112-174may be provided as part of an S security agent that is included in software running on a compute instance10-26within the company premises. Some or all of one or more of the facilities100,112-174can be provided on the same physical hardware or logical resource as a gateway, such as a firewall10the wireless access point11. Some or all of one or more of the facilities may be provided on one or more cloud servers operated by the company or by a security service provider, such as the cloud computing instance.109.
In embodiments, a marketplace provider199may make available one or more additional facilities to the company facility102through the threat management function100. The marketplace provider can communicate with the threat management facility.100through the installation of market interface174to provide additional functionality or capabilities to the threat management facility100and compute instances10-26. As non-limiting examples, the market provider199may be a third party information provider, such as a physical security event provider; the market provider199It may be a systems provider, such as an HR system provider or a fraud detection system provider; the marketplace provider may be a specialist analytics provider; etc. The market provider199, with the appropriate permissions and authorization, can receive and send events, observations, inferences, checks, convictions, policy violations, or other information to the threat management facility. For example, the market provider199you can subscribe to and receive certain events, and in response, based on the events received and other events available to the marketplace provider199, send inferences to the market interface and, in turn, to the analysis function168, which in turn can be used by the security management facility122.
The identity provider158can be any remote identity management system or similar configured to communicate with an identity management facility172, for example, to confirm the identity of a user, as well as to provide or receive other information about users that may be useful to protect against threats. In general, the identity provider can be any system or entity that creates, maintains, and manages identity information for principals while providing authentication services to third-party applications, for example, within a federation or distributed network. The identity provider may, for example, offer user authentication as a service, while other applications, such as web applications, outsource the user authentication step to a trusted identity provider.
In embodiments, the identity provider158You can provide user identity information, such as multi-factor authentication, to a SaaS application. Centralized identity providers, such as Microsoft Azure, can be used by an enterprise facility instead of maintaining separate identity information for each application or group of applications, and as a centralized point to integrate multi-factor authentication. In embodiments, the identity management facility172may communicate information about hygiene or security risks to the identity provider158. The identity management facility172You can determine a threat score for a user based on events, observations, and inferences about that user and the compute instances associated with the user. If a user is perceived as risky, the identity management facility172you can inform the identity provider158and the identity provider158You can take steps to address the potential risk, such as confirming the user's identity, confirming that the user has approved access to the SaaS application, remediating the user's system, or other steps that may be helpful.
In embodiments, the threat protection provided by the threat management facility100can extend beyond the boundaries of the company facility network102to include clients (or client facilities) as an endpoint22off company premises102, a mobile device26, a cloud computing instance109, or any other device, service or the like that uses network connectivity that is not directly associated with or controlled by the company facility102such as a mobile network, a public cloud network, or a wireless network in a hotel or cafe. While threats can come from a variety of sources, such as network threats, physical proximity threats, secondary location threats, computing instances10-26You can be protected against threats even when a compute instance10-26is not connected to the company facility102network, such as when compute instances22,26use a network that is outside the company premises102and separated from the company premises102, for example, by a gateway, a public network, etc.
In some implementations, compute instances10-26can communicate with cloud applications, such as a SaaS application156. The SaaS application156may be an application used but not operated by the company facility102. Examples of commercially available SaaS applications156include applications from Salesforce, Amazon Web Services (AWS), applications from Google Apps, Microsoft Office365apps and so on. A given SaaS application156you can contact an identity provider158to verify the identity of the user according to the requirements of the company facility102. The compute instances10-26can communicate with an unprotected server (not shown) such as a third-party website or application over a network154such as the Internet or any other public network, private network or a combination of these.
In embodiments, the installation aspects of threat management100can be provided as a standalone solution. In other embodiments, installation aspects of threat management100can be integrated into a third-party product. An application programming interface (for example, a source code interface) may be provided so that aspects of the threat management function100It can be integrated or used by or with other applications. For example, the threat management facility100can be standalone in the sense that it provides direct threat protection to an enterprise or computing resource, where the protection is underwritten directly100. Alternatively, the threat management facility may offer protection indirectly, through a third-party product, where a company may subscribe to services through the third-party product, and the threat management facility may provide threat protection to the company.100through the third-party product.
The security management facility122can provide protection against a variety of threats by providing, but not limited to, endpoint security and control, email security and control, web security and control, reputation-based filtering, machine learning classification, unauthorized user control, control of guests and unsupported computers, and more.
The security management facility122can provide protection from malicious code to a compute instance. The security management facility122It may include features to scan apps, files, and data for malicious code, delete or quarantine apps and files, prevent certain actions, take corrective actions, and other security measures. The scan may use any of a variety of techniques, including, but not limited to, signatures, identities, classifiers, and other suitable scanning techniques. In embodiments, scanning may include scanning some or all files periodically, scanning an application when the application is running, scanning data transmitted to or from a device, scanning in response to predetermined actions or combinations of actions, etc. Application, file, and data scanning can be performed to detect known or unknown malicious code or PUAs. Aspects of protection against malicious code can be provided, for example, in the security agent of an endpoint.12, on a wireless access point11or firewall10, as part of application protection150provided by the cloud, and so on.
In one embodiment, the security management facility122may provide security and control of email, for example, to detect spam, viruses, spyware and phishing, to control email content and the like. Email security and control can protect against inbound and outbound threats, protect email infrastructure, prevent data leakage, provide spam filtering, and more. Aspects of email security and control can be provided, for example, in the security agent of an endpoint.12, on a wireless access point11or firewall10, as part of application protection150provided by the cloud, and so on.
In one embodiment, the security management facility122can provide web security and control, for example, to detect or block viruses, spyware, malware, PUAs, help control web browsing and the like, which can provide comprehensive web access control enabling smooth web browsing safe and productive. Web security and control can provide Internet usage policies, reporting on suspicious computing instances, content security and filtering, active monitoring of network traffic, URI filtering, and the like. Web security and control aspects can be provided, for example, in the security agent of an endpoint12, on a wireless access point11or firewall10, as part of application protection150provided by the cloud, and so on.
In one embodiment, the security management facility122can provide network access control, which typically controls access to and use of network connections. Network control can prevent unauthorized, guest, or non-compliant systems from accessing networks, and can control network traffic that is otherwise not controlled at the client level. In addition, network access control can control access to virtual private networks (VPNs), where VPNs can, for example, include communication networks tunneled through other networks and establish logical connections that act as virtual networks. In embodiments, a VPN can be treated in the same way as a physical network. Aspects of network access control can be provided, for example, in the security agent of an endpoint12, on a wireless access point11or firewall10, as part of application protection150provided by the cloud, for example, from the threat management facility100or other network resources.
In one embodiment, the security management facility122can provide host intrusion prevention through behavior monitoring and/or runtime monitoring, which can provide protection against unknown threats by analyzing application behavior before or during application execution. This may include monitoring code behavior, application programming interface calls made to libraries or the operating system, or monitoring application activities. Monitored activities may include, for example, memory read and write, disk read and write, network communication, process interaction, etc. Behavior and runtime monitoring can intervene if code is deemed to be acting suspiciously or maliciously. Aspects of runtime and behavioral monitoring can be provided, for example, in the security agent of an endpoint12, on a wireless access point11or firewall10, as part of application protection150provided by the cloud, and so on.
In one embodiment, the security management facility122can provide reputation filtering, which can target or identify sources of known malware. For example, reputation filtering may include lists of URIs of known malware sources or known suspicious IP addresses, code authors, code signers, or domains that, when detected, may invoke action by the reputation management facility. threats.100. Based on reputation, potential threat sources can be blocked, quarantined, restricted, monitored, or some combination of these, before a data exchange can take place. Aspects of reputation filtering can be provided, for example, in the security agent of an endpoint12, on a wireless access point11or firewall10, as part of application protection150provided by the cloud, and so on. In embodiments, certain reputation information may be stored in a computing instance.10-26and other reputation data available through cloud lookups in an application protection search database, such as that application protection can provide.150.
In embodiments, the information may be sent from the company facility102to a third party, such as a security vendor or the like, which may lead to better performance of the threat management function100. In general, feedback can be useful for any aspect of threat detection. For example, the types, times, and number of virus interactions that a business facility102The experiences can provide useful information for the prevention of future virus threats. Comments may also be associated with behaviors of individuals within the company, such as being associated with the most common policy violations, network access, unauthorized application loading, unauthorized external device usage, and the like. In embodiments, the feedback may allow evaluation or profiling of customer actions that are policy violations that may provide a predictive model for business policy improvement.
An update management facility120can provide control over when updates are made. Updates can be transmitted automatically, transmitted manually, or some combination of these. Updates may include software, definitions, reputations, or other code or data that may be useful to various installations. For example, the update function120can manage receipt of updates from a vendor, distribution of updates to company facilities102networks and computing instances, or the like. In embodiments, updates can be provided to the enterprise installation102network, where one or more computing instances on company premises102the network can distribute updates to other compute instances.
The Threat Management Facility100may include a policy management function112that manages rules or policies for the enterprise installation102. Exemplary rules include access permissions associated with networks, applications, compute instances, users, content, data, and the like. The policy management function112you can use a database, a text file, another data store, or a combination to store policies. In one embodiment, a policy database may include a block list, black list, allow list, white list, and more. As some non-limiting examples, the policies may include a list of business facilities102external network locations/applications that compute instances can or cannot access, a list of types/classifications of network locations or applications that compute instances can or cannot access, and contextual rules to evaluate whether the lists apply . For example, there may be a rule that does not allow access to sports websites. When the client facility requests a website, a security management facility122You can access the rules within a policy facility to determine if the requested access is related to a sports website.
The policy management function112can include access rules and policies that are distributed to maintain control of access by compute instances10-26to network resources. Exemplary policies can be defined for an enterprise installation, application type, subset of application capabilities, organization hierarchy, compute instance type, user type, network location, time of day, connection type, or any other definition. adequate. Policies can be maintained through the threat management feature.100, in association with a third party, or the like. For example, a policy might restrict instant messaging (IM) activity by limiting such activity to support personnel when communicating with customers. More generally, this can allow departmental communication as needed or useful for departmental functions, but can otherwise preserve network bandwidth for other activities by restricting IM use to staff who need it. access for a specific purpose. In one embodiment, the policy management facility112can be a standalone application, can be part of the network server installation142, can be part of the company installation102network, it can be part of the customer installation, or any suitable combination of these.
The policy management function112it can include dynamic policies that use contextual or other information to make security decisions. As described here, the dynamic policy service170You can dynamically generate policies based on observations and inferences made by the analysis function. Dynamic policies generated by the dynamic policy service170can be provided by policy management facility112to the security management facility122for execution.
In embodiments, the threat management facility100may provide configuration management as an aspect of the policy management function112, security management facility122, or some combination. Configuration management can define acceptable or required configurations for compute instances10-26, applications, operating systems, hardware or other assets, and manage changes to these configurations. The evaluation of a configuration can be carried out against standard configuration policies, detection of configuration changes, correction of incorrect configurations, application of new configurations, etc. An enterprise facility may have a set of standard configuration rules and policies for particular compute instances that may represent a desired state of the compute instance. For example, in a given compute instance12,14,18, a version of a client firewall may be required to run and install. If the required version is installed but in a disabled state, the policy violation may prevent access to data or network resources. One solution may be to enable the firewall. In another example, a configuration policy may disallow the use of USB drives and policy management112may require a configuration that disables access to the USB drive through a compute instance registry key. Aspects of configuration management may be provided, for example, in the security agent of an endpoint.12, on a wireless access point11or firewall10, as part of application protection150provided by the cloud, or any combination of these.
In embodiments, the threat management facility100may also provide isolation or removal of certain applications that are unwanted or that may interfere with the operation of a compute instance10-26o Threat management facility100, even if said application is not malware per se. The operation of such products can be considered a violation of the configuration. Removal of such products may be initiated automatically whenever such products are detected, or access to data and network resources may be restricted when they are installed and running. In the event that such applications are services that are provided indirectly through a third-party product, the application or the corresponding processes may be suspended until steps are taken to remove or deactivate the third-party product.
The policy management function112may also require update management (for example, as provided by the update function120). Management of updates for the security installation.122and policy management service112can be provided directly by the threat management facility100, or, for example, by a hosted system. In embodiments, the threat management facility100It can also provide patch management, where a patch can be an update to an operating system, application, system tool, or the like, where one of the reasons for patching is to reduce vulnerability to threats.
In embodiments, the security facility122and policy management service112can send information to company premises102network and/or computing instances10-26, the installation of the company102network and/or computing instances10-26can extract information from security installation122and policy management service112, or there may be a combination of information push and pull. For example, the enterprise installation102network and/or computing instances10-26can extract up-to-date security installation information122and policy management service112via update function120, an update request can be based on a period of time, at a certain time, on a date, on demand or similar. In another example, the security facility122and policy management service112can push the information to the company facility102network and/or computing instances10-26providing notification that updates are available for downloading and/or transmitting the information. In one embodiment, the policy management facility112and security facility122can work in conjunction with update management installation120provide information to company facilities102network and/or computing instances10-26. In various embodiments, policy updates, security updates, and other updates may be provided by the same or different modules, which may be the same or independent of a security agent running on one of the compute instances.10-26.
As threats are identified and characterized, the definition function114Threat Management Facility100You can manage definitions used to detect and remediate threats. For example, identity definitions can be used to scan files, applications, data streams, etc. for the determination of malicious code. Identity definitions can include instructions and data that can be parsed and acted upon to recognize characteristics of known or potentially malicious code. Definitions can also include, for example, code or data to be used in a classifier, such as a neural network or other classifier that can be trained using machine learning. The classifier can use updated code or data to classify threats. In embodiments, the threat management facility100and compute instances10-26new definitions may be provided to you periodically to include the latest threats. The definition update can be managed by the update function.120, and can be done at the request of one of the computing instances10-26, in a push, or some combination. Updates can be done over a period of time, upon request of a device10-26, on the determination of a major new definition or series of definitions, and so on.
A threat research facility (not shown) can provide an ongoing effort to maintain the threat protection capabilities of the threat management facility.100in light of the continuous generation of new or evolved forms of malware. Threat research can be provided by researchers and analysts working on known threats, in the form of policies, definitions, corrective actions, etc.
The security management facility122you can scan an outgoing file and verify that the outgoing file can be transmitted according to the policies. When checking outgoing files, the security management function122you may be able to discover threats that were not detected on one of the compute instances10-26, or violation of policy, such transmission of information that must not be communicated in the clear.
The Threat Management Facility100can control access to company premises102networks A network access facility.124You can restrict access to certain applications, networks, files, printers, servers, databases, etc. In addition, the ease of access to the network124You can restrict user access under certain conditions, such as user location, usage history, need to know, job title, connection type, time of day, authentication method, system settings customer or the like. Network access policies may be provided by the policy management facility.112, and can be developed by the company installation102, or prepackaged by a supplier. Ease of network access124can determine if a given compute instance10-22access must be granted to a requested network location, for example, inside or outside the company premises102. Ease of network access124can determine if a compute instance22,26as a device off company premises102You can access the company facilities102. For example, in some cases, policies may require that when certain policy violations are detected, certain network access be denied. Ease of network access124You can communicate corrective actions that are necessary or helpful to bring a device back into policy compliance as described below regarding corrective action installation128. Aspects of ease of access to the network124can be provided, for example, in the endpoint security agent12, on a wireless access point11, in a firewall10, as part of application protection150provided by the cloud, and so on.
In one embodiment, the network access facility124You can access policies that include one or more of a block list, black list, allow list, white list, unacceptable website database, acceptable website database, of network site reputation data or similar to access locations that may or may not be accessed by the client installation. In addition, the network access function124You can use rule evaluation to analyze network access requests and apply policies. The network access rule function124You can have a generic set of policies for all compute instances, such as denying access to certain types of websites, controlling instant messaging access, or the like. Rule evaluation may include regular expression rule evaluation or other rule evaluation methods for interpreting the network access request and comparing the interpretation with established rules for network access. Classifiers can be used, such as neural network classifiers or other classifiers that can be trained by machine learning.
The Threat Management Facility100may include an asset classification service160. The asset classification facility will discover the assets present in the business facility102. A compute instance like any of the compute instances10-26described in this document can be characterized as a stack of assets. The one-tier asset is a physical hardware item. The compute instance may be, or may be, implemented on physical hardware, and may or may not have a hypervisor, or may be an asset managed by a hypervisor. The compute instance can have an operating system (for example, Windows, MacOS, Linux, Android, iOS). The compute instance can have one or more container layers. The compute instance may have one or more applications, which may be native applications, for example, to a physical asset or virtual machine, or running containerized within a compute environment on a physical asset or virtual machine, and those Applications can link libraries or other code or the like, for example, for a user interface, cryptography, communications, device drivers, mathematical or analytical functions, etc. The stack can also interact with the data. The stack can also or instead interact with the users, so the users can be considered active.
The threat management function may include entity models162. Entity models can be used, for example, to determine the events that assets generate. For example, some operating systems may provide useful information to detect or identify events. For example, operating systems may provide usage and process information accessed through an API. As another example, it may be possible to instrument certain containers to monitor the activity of applications running in them. As another example, entity models for users can define roles, groups, allowed activities, and other attributes.
The Event Collection Facility164can be used to collect events from any of a wide variety of sensors that can provide relevant events from an asset, such as sensors on any of the compute instances10-26, application protection installation150, a cloud computing instance109etc. The events that can be collected can be determined by the entity models. There can be a variety of events collected. Events may include, for example, events generated by the company installation102or compute instances10-26, such as when monitoring data transmission through a gateway such as a firewall10and wireless access point11, monitor compute instance activity, monitor files/data stored on compute instances10-26such as desktop computers, laptops, other mobile computing devices, and cloud computing instances19,109. Events can vary in granularity. An exemplary event may be the communication of a specific packet through the network. Another exemplary event may be the identification of an application that communicates over a network.
The event log function166can be used to store events collected by the event collection facility164. The event log function166can store collected events so that the analytics facility can access and analyze them168. Some events may be collected locally, and some events may be reported to an event store at a central location or cloud facility. Events can be logged in any suitable format.
Events collected by the event log function166can be used by analytics facility168make inferences and observations about facts. These observations and inferences can be used as part of the policies enforced by the security management facility. Observations or inferences about events may also be recorded by the event logging facility.166.
When the security management facility detects a threat or other policy violation122, the mechanism of corrective measures128can be used to remedy the threat. Corrective action can take a variety of forms, non-limiting examples include collecting additional threat data, terminating or modifying an ongoing process or interaction, sending a warning to a user or administrator, downloading a data file with commands, definitions, instructions, or the like to remediate the threat, request additional information from the requesting device, such as the application that initiated the activity of interest, run a program or application to remediate a threat or violation, increase telemetry o Log interactions for further evaluation, ( continue) Block requests to a particular network location(s), Scan a requesting application or device, Quarantine a requesting application or device, Isolate the requesting application or device, Deploy a sandbox, blocking access to resources, for example a USB port, or other corrective actions. In more general terms, the corrective action mechanism122may take any action or implement any appropriate action to address the detection of a threat, potential threat, policy violation, or other event, code, or activity that may compromise the security of a computing instance10-26or company facility102.
The Threat Management Facility308it can perform a variety of threat management functions, such as any of those described in this document. The Threat Management Facility308can usually include an application programming interface310to third party services320, a user interface312to access network administration and threat management features, and a number of threat detection tools314.
In general, the application programming interface310can support programmatic connections to third-party services320. The application programming interface310You can, for example, connect to Active Directory or other customer information about files, data storage, user identities and profiles, roles, access privileges, etc. More generally, the application programming interface310it can provide a programmatic interface to client or third-party context, information, management, and security tools, etc. The application programming interface310may also, or instead, provide a programmatic interface to hosted applications, identity provider integration tools or services, etc.
user interface312may include a website or other graphical or similar interface, and may generally provide an interface for user interaction with the threat management function308eg for threat detection, network management, auditing, configuration, etc. This user interface312can generally facilitate human curation of MIs as contemplated here, for example by presenting MIs along with other supplementary information, and providing controls for the user to remove such MIs as desired, eg by allowing execution or access , denying execution or access, or participating in corrective measures such as sandboxing, quarantine, vaccination, etc.
Threat detection tools314It can be any of the threat detection tools, algorithms, techniques, or the like described in this document, or any other tools or the like useful for detecting threats or potential threats within an enterprise network. This can, for example, include signature-based tools, behavioral tools, machine learning models, etc. In general, threat detection tools314may use event data provided by endpoints within the enterprise network, as well as any other available context such as network activity, heartbeats, etc., to detect malware or potentially unsafe conditions for a network or endpoints connected to the enterprise network. grid. In one aspect, threat detection tools314It can usefully integrate event data from multiple endpoints (including, for example, network components such as gateways, routers, and firewalls) to improve threat detection in the context of complex or distributed threats. Threat detection tools314may also or instead include tools to report to a separate modeling and analysis platform318For example, to support further investigation of security issues, creation or refinement of threat detection models or algorithms, review and analysis of security breaches, etc.
Threat management tools316can typically be used to manage or remediate threats to the enterprise network that have been identified by threat detection tools314in another way. Threat Management Tools316may, for example, include tools for isolating, quarantining, removing, or repairing or managing malicious code or malicious activity, for example, using any of the techniques described in this document.
the end point302it can be any of the endpoints or other computing or similar instances described in this document. This may, for example, include end-user computing devices, mobile devices, firewalls, gateways, servers, routers, and any other computing device or instance that can connect to a business network. As described above, the end point302can usually include a security agent306that locally supports threat management on the endpoint302such as by monitoring malicious activity, managing endpoint security components302, maintain policy compliance and communicate with the threat management facility308to support integrated security protection as contemplated here. the security agent306can, for example, coordinate endpoint instrumentation302to detect various types of events involving various computing objects on the endpoint302and monitor the event log on a data logger304. the security agent306it can also, or instead, scan computer objects, such as electronic communications or files, monitor the behavior of computer objects, such as executables, etc. the security agent306may, for example, apply behavioral or signature-based threat detection techniques, machine learning models (for example, models developed by the Modeling and Analytics Platform) or any other tools or the like suitable for detecting malware or potential malware in the end point302.
the data logger304You can log events that occur on or are related to the endpoint. This may, for example, include events associated with computing objects on the endpoint.302such as file manipulations, software installations, etc. This can also or instead include activities directed from the endpoint302such as requests for content from Uniform Resource Locators or other network activity involving remote resources. the data logger304can log data at any frequency and any level of granularity consistent with proper endpoint operation302in an intended or intended way.
the end point302can include a filter322to manage a flow of information from the data logger304to a remote resource such as threat detection tools314Threat Management Facility308. In this way, a detailed log of events can be maintained locally on each endpoint, while network resources can be conserved to report a filtered event stream containing the information deemed most relevant to threat detection. . The filter322it can also, or instead, be configured to report causal information that causally relates collections of events to each other. In general, the filter322can be configurable so that, for example, the threat management function308you can increase or decrease the level of reporting based on the current security status of the endpoint, a group of endpoints, the corporate network, and the like. The notification level may also be based on, or in place of, currently available computing and network resources, or in any other appropriate context.
In another aspect, the endpoint302can include a query interface324so that remote resources, such as the threat management facility308you can check the data logger304remotely for additional information. This may include a request for specific events, activity for specific computing objects, or events during a specific period of time, or some combination of these. So, for example, the threat management facility308you can request all system information registry changes in the last forty-eight hours, all files opened by system processes in the last day, all network connections or network communications in the last hour, or any other parameterized request of activities monitored by data recorder304. In another aspect, the entire data record, or the entire record during a predetermined time window, may be requested for further analysis at a remote resource.
It will be appreciated that communications between third party services320, a threat management facility308and one or more endpoints, such as the endpoint302it can be facilitated by using consistent naming conventions across products and machines. For example, the system300can usefully implement globally unique device identifiers, user identifiers, application identifiers, data identifiers, uniform resource locators, network streams, and files. The system may also, or instead, use tuples to uniquely identify communications or network connections based on, for example, source and destination addresses, etc.
Accordingly, a system disclosed herein includes an enterprise network and an endpoint attached to the enterprise network, and a threat management facility coupled in a communication relationship with the endpoint and a plurality of other endpoints via of the business network. The endpoint may have a data logger that stores an event stream of event data for compute objects, a filter for creating a filtered event stream with a subset of event data from the event stream, and a query interface for receiving queries to the data logger from a remote resource, the endpoint further includes a local security agent configured to detect malware on the endpoint based on event data stored by the data logger, and further configured to communicate the flow of events filtered through the corporate network. The threat management feature can be configured to receive the filtered event stream from the endpoint, detect malware on the endpoint based on the filtered event stream, and remediate the endpoint when malware is detected, the threat management feature will be It further configures to modify the security functions within the enterprise network based on a security state of the endpoint.
The threat management feature can be configured to adjust event data notification through the filter in response to a change in the filtered event stream received from the endpoint. The threat management feature can be configured to tune event data notification through the filter when the filtered event stream indicates a compromised security state of the endpoint. The threat management feature can be configured to adjust the reporting of event data from one or more endpoints in response to a change in the filtered event stream received from the endpoint. The threat management feature can be configured to tune event data notification through the filter when the filtered event stream indicates a compromised security state of the endpoint. The threat management feature can be configured to request additional data from the data logger when the filtered event stream indicates a compromised security state of the endpoint. The threat management feature can be configured to request additional data from the data recorder when an endpoint security agent reports a security compromise regardless of the filtered event stream. The threat management feature can be configured to adjust the handling of network traffic on a gateway to the enterprise network in response to a predetermined change in the filtered event stream. The threat management feature can include a machine learning model to identify potentially malicious activity on the endpoint based on the filtered event stream. The threat management function can be configured to detect potentially malicious activity based on a plurality of event streams filtered from a plurality of endpoints. The threat management feature can be configured to detect malware on the endpoint based on the filtered event stream and additional context for the endpoint.
The data logger can log one or more events from a kernel driver. The data logger may record at least one change to a system configuration record for the endpoint. Endpoints can include a server, a corporate network firewall, a corporate network gateway, or any combination of these. The endpoint can be attached to the corporate network through a virtual private network or a wireless network. The endpoint can be configured to periodically transmit a snapshot of raw aggregated data from the data logger to the threat management facility for remote storage. The data recorder can be configured to delete records in the data recorder corresponding to the snapshot to free up memory on the endpoint for additional recording.
The coloring system410It can be used to label or color software objects to improve tracking and detection of potentially harmful activity. The coloring system410You can, for example, tag files, executables, processes, network communications, data sources, etc. with any suitable information. A variety of techniques can be used to select static and/or dynamic labels for any of these various software objects, and to manage the mechanics of applying and propagating color information accordingly. For example, a process can inherit a color from an application that starts the process. Similarly, a file can inherit a color from a process when it is created or opened by a process, and/or a process can inherit a color from a file that the process has opened. More generally, any type of tagging, as well as rules for propagating, inheriting, changing or manipulating such tags, can be used by the coloring system.410as seen here.
The key management system412can support key management for the endpoint402to selectively allow or prevent access to content on the endpoint402on a file-specific basis, a process-specific basis, an application-specific basis, a user-specific basis, or any other appropriate basis to prevent data leakage and to support more granular and immediate control over access to content on the final point402when a security compromise is detected. So, for example, if a particular process running on the endpoint is compromised, or potentially compromised or suspected, that process's keys can be revoked to prevent, for example, data leakage or other malicious activity. .
The heartbeat system414can be used to provide periodic or aperiodic information from the endpoint402or other system components about health, security, system status, etc. A heartbeat can be encrypted or in plain text, or some combination of these, and can be communicated one way (for example, from the endpoint408to install threat management408) or bidirectional (for example, between the endpoint402and the server406, or any other pair of system components) at any useful time.
In general, these various control and management systems can cooperate to provide better threat detection and response. For example, the coloring system.410can be used to assess when a particular process is potentially opening inappropriate files based on an inconsistency or mismatch in colors, and can confirm a potential threat based on an interrupted heartbeat of the system414. The key management system412it can then be implemented to revoke the process keys so that no more files can be opened, deleted, or modified. More generally, the cooperation of these systems enables a wide variety of reactive measures that can improve detection and remediation of potential threats to an endpoint.
As an example, the event graph500represented in the figure starts with a computer object that is a USB device512, which may be connected to an endpoint. Where the USB device512includes a directory or file system, the USB device512it can be mounted or accessed by a file system on an endpoint to read the content. the usb device512can be detected513and content of the USB device512it can be opened514, for example, by an endpoint user or automatically by the endpoint in response to USB device detection512. the usb device512may include one or more files and applications, for example, a first file516, a second file518, and a first application520. the first file516may be associated with a first event522and the second file may be associated with a second event524. the first application520you can access one or more files on the endpoint, for example the third file526is shown in the figure. the first application520can also or instead perform one or more actions528, how to access a URL530. Accessing the URL530you can download or run a second app532at the endpoint, which in turn accesses one or more files (for example, the fourth file534shown in the figure) or is associated with other events (for example, the third event536shown in the figure).
In the example provided by the event graph500represented in the figure, the security event detected502can include action528associated with the first application520, for example, by accessing the URL530. As an example, the URL530it can be a known malicious URL or a URL or network address associated with malware. the url530it can also or instead blacklist a network address that, while not associated with malware, may be prohibited by a security policy of the endpoint or the enterprise network in which the endpoint is a participant. the url530it can have a certain reputation or an unknown reputation. Therefore, access the URL530can be detected by known computer security techniques.
In response to security event detection502, the event graph500can be traversed in reverse order from a compute object associated with the security event502based on the sequence of events included in the event graph500. For example, traversing backwards from the action528leads to at least the first application520and the USB device512. As part of a root cause analysis, one or more cause identification rules can be applied to one or more of the above computing objects that have a causal relationship to the detected security event.502, or to each computing object that has a causal relationship with another computing object in the sequence of events preceding the detected security event502. For example, other computing objects and events can be tangentially associated with causally related computing objects by traversing the event graph.500in reverse order, as the first file516, the second file518, the third file525, the first event522, and the second event524represented in the figure. In one aspect, the rules for identifying one or more causes are applied to computing objects that precede the detected security event.502until a cause of the security event502is identified.
In the example shown in the figure, the USB device512can be identified as the root cause504of the security event502. In other words, the USB device512was the source of the application (the first application520) that started the security event502(the action528from accessing the potentially malicious or unwanted URL530).
the event graph500may be similarly traversed in the future from one or more of the root causes504or the security event502to identify one or more computing objects affected by the root cause504or the security event502. For example, the first file516and the second518may potentially be damaged because the USB device512including malicious content. Similarly, any related actions taken after the security event502like any made by the second application532it may be corrupted. Further testing or remediation techniques can be applied to any of the compute objects affected by the root cause.504or the security event502.
the event graph500may include one or more computing objects or events that are not located in a path between the security event502and the root cause504. These computing objects or events can be filtered or 'cut' from the event graph500when performing a root cause analysis or an analysis to identify other computing objects affected by the root cause504or the security event502. For example, compute objects or events that can be removed from the event graph.500can include usb drive510and the USB device being detected513.
It will be appreciated that the graph of events500represented in
the event graph500they can be created or parsed using rules that define one or more relationships between events and computing objects. The C Language Integrated Production System (CLIPS) is a public domain software tool intended for building expert systems and can be suitably adapted for the analysis of a graph such as the event graph.500to identify patterns and otherwise apply rules for pattern analysis. While other tools and programming environments can also be used, CLIPS can support a suitable forward and reverse chaining inference engine for a large amount of input data with a relatively small set of inference rules. With CLIPS, a new data source can trigger a new inference, which can be suitable for dynamic solutions for root cause investigations.
An event graph like the event graph500shown in the figure can include any number of nodes and edges, where computing objects are represented by nodes and events are represented by edges marking causal or directional relationships between computing objects, such as data flows, control flows , network flows, etc. . While processes or files are common forms of nodes that can appear on such a graph, any other computing object such as an IP address, registry key, domain name, uniform resource locator, command line input or other object may also, or instead, be designated to be a node in an event graph as contemplated herein. Similarly, while an edge can be made up of an IP connection, a file read, a file write, a process invocation (parent, child, etc.), a process route, a thread injection, a record write, a domain name service query, a uniform resource locator access can be designated, etc., other edges. As described above, when a security event is detected, the source of the security event can serve as the starting point within the event graph.500, which can then be traversed back to identify a root cause using any number of suitable cause identification rules. the event graph500it can then be usefully traversed from that root cause to identify other computing objects that are potentially contaminated by the root cause so that a more thorough remediation can be performed.
A local event recorder650it may be part of the event log function. Some logged events may be stored locally and others may be communicated to another computing instance, such as the cloud. Some events will be sent in real time, some will only be stored locally (and should be retrievable). An event filter662can be used to parse events. local analyzes664in a compute instance it can be used to locally identify events of interest. a communication facility660will communicate events to a central event store, such as a threat management facility610, which can be a cloud installation. local compliance666can be used to take action in response to events, as determined by the policy management facility666. In embodiments, events may have attributes (eg, required, optional (eg, best effort), sensitive (eg, tokenize it in the local event store)). Some events will be sent in real time, some will only be stored locally (and should be retrievable).
One goal may be to discover as much as we can about company assets and reduce surprises, such as compute instances unknown to network administrators, unpatched compute instances, or valuable data leaving the company.
As a non-limiting example, static policies can be assigned to access files and data. Events involving files and data can be observed by sensors, for example in a file system filter, generating events. Events can be determined to be of interest based on policies.
In embodiments, events are continuously analyzed against a baseline. The baseline can be adjusted to account for normal behavior. Comparison with baselines can include looking for outliers and anomalies, as well as impossible events. For example, if a user logs in from Germany and then logs in from San Francisco, that may be considered impossible. Comparisons can be made at different levels. For example, the entity can be compared to itself, eg how this user on Monday compares to this same user at some point in the past. The entity can also or instead be compared to a peer group, for example, it is a member of the finance department that behaves similarly to other members of the finance department. The entity may also or instead be compared to other entities within the company. For example, the entity can be compared to other users at similar companies in the same industry or in the same location, as well as the universe of all users.
It can also include real-time and retrospective threat information, as well as vulnerability information and patch information.
With a sufficient level of confidence in the inferences, active and adaptive responses can be made. For example, dynamic policies771it can be updated to better match the security profile to the environment that has been discovered and observed, for example, by adjusting security settings within a security policy or security policy group. A policy compliance facility773you can enforce these updated dynamic policies771on compute instances, such as compute instances711-714.
In embodiments, high interaction interfaces allow an administrator to interact with the event store.764to better understand assets on company premises and for specific purposes, such as threat hunting.
As shown in step802, the method800may include providing a model such as a threat detection model to assess the probability that a sample of threats is at least safe or malicious based on a training set of samples of known threats. This may include any of the machine learning models or other threat detection models covered in this document. As shown in step804, the method800It may also include providing threat samples, such as code samples known to be safe and code samples known to be malicious. This may also include, or instead, known safe and unsafe samples of network activity, file content, file activity, behaviors, events, etc. The threat detection model may include a machine learning model trained on these threat samples, or any other suitable training set, or some combination thereof. Therefore, providing the model may include training a machine learning model to identify malicious code in a training set that includes threat samples that are known to be safe and malicious.
The model may include a model for evaluating the probability that a threat sample is at least safe or malicious based on a training set of known threat samples. The model may also include, or instead, an integrative model that assesses a potential threat using a threat sample based on a combination of a first model configured to identify malicious code based on behavior tags, a second model configured to identify malicious code malware based on an executable file. route and a third model configured to identify malicious code based on a uniform resource locator within the threat sample, or any of the other integrating models considered in this document.
As shown in step806, the method800may include identification of intermediate threats. For example, this may include identifying a new threat sample as an intermediate threat that does not have a predetermined probability of being malicious or safe based on the model, or using any of the other techniques described in this document.
As shown in step808, the method800It may include the identification of supplementary information relevant to the evaluation of the new threat sample, such as the relevant characteristics of the new threat sample that contribute to an inference of malicious code.
For example, the method800may include the identification of one or more features, as relevant features of the new threat sample associated with a malware inference, using a random forest of human-readable features associated with a malware inference in the sample training set of known threats (or any other suitable training set or similar). Random forests or random decision forests are a joint learning method for classification, regression, and other tasks, which operate by building a multitude of decision trees at training time and generating the class that is the mode of the classes (classification ) or mean prediction (regression) of the individual trees. As a significant advantage, the structure of decision trees can be organized around human-interpretable features, such as whether a threat sample is signed or whether the threat sample opens new files during execution. While the creation of a random forest is generally computationally expensive, and other, more efficient techniques for automated classification are known, the output of a random forest on human-interpretable features can provide very useful context for a human reviewer when evaluating intermediate threats. as seen here. , and thus provides particular advantages over other classification techniques in this context, even when used in conjunction with other (possibly more computationally efficient) models and classification techniques to assess the risk of samples of unknown threats.
Identification of supplemental information may also include, or instead, identification of similar threat samples that are known to be safe or malicious, including one or more safe threat samples similar to the new threat sample and one or more safe threat samples of malicious threats similar to the new threat sample. In this context, similarity can usefully be computed based on a k-nearest neighbor algorithm. Similar threat samples may, for example, include a list of safe threat samples ranked based on similarity to the new threat sample according to the k-nearest neighbor algorithm, which itself may be presented as a list classified in a user interface. The similar code may also include, or instead, a list of malicious threat samples ranked based on their similarity to the new threat sample according to the nearest neighbor algorithm. Using these ranked lists, a user can advantageously be presented with an ordered list of closest known safe threat samples and closest known unsafe samples. A k-nearest neighbor algorithm is a non-parametric method that assigns a new element to a particular class based on a nearest neighbor within a (usually multidimensional) feature space for training data.
While this approach provides a computationally efficient technique for assessing the similarity of certain types of data, it will be understood that other computational measures of similarity are known in the art, and can be usefully employed to assess the similarity of a new threat sample with Known safe and unsafe data. threat displays as contemplated in this document. For example, a nearest centroid classifier or a closest prototype classifier uses a classification model that assigns a classification based on a nearest centroid that can be used to assess similarity as contemplated herein. As another example, an n-gram analysis supports efficient fuzzy matching and can be used to perform fast, large-scale similarity analysis for a given file path against a large database of known malicious and benign file paths and URLs. .
While certain parts of this description emphasize the analysis of executables for the detection of suspected or intermediate threat identification, it should be understood that the term "threat sampler" is not so limited. Other threat samples based on, for example, files, caches, or other data sources may be used. The events, p. For example, in a filtered event stream, or can also be used instead, and the techniques described in this document for use with code samples are also generally applicable to other threat samples rather than explicit computer code, such as network activity, content, event streams that identify activities or behaviors, etc. So, for example, activities like visiting a particular URL, opening an attachment, sending an email, or other events can also be analyzed as threat samples using a built-in model or other threat detection tools to identify potential threats from malware on a website. endpoint or group of endpoints.
As shown in step810, the method800You can include the display of intermediate threats and supplemental information in a user interface for the user to see, or augment a description of the new threat display in a user interface with the supplemental information. This may, for example, include presenting a description of the new threat sample, one or more relevant features, and similar threat samples in a user interface. In one aspect, the method may include displaying a list of the similar threat samples ranked according to similarity to the new threat sample using, for example, a k-nearest neighbor algorithm or any other suitable technique for measuring similarity. This may, for example, include similarity of executable code, similarity of behaviors, similarity of file names, similarity of called URLs, or similarity of any other objective characteristic or combination of characteristics that may be correlated with risk (or lack of risk). In one aspect, several of the most similar secure samples and several of the most similar unsafe samples can be presented together and ranked, for example, based on relative threat or based on similarity. Threat samples can be displayed along with descriptive information, attributes, behavioral characteristics, metadata, etc., as well as any other information that can help a human user assess relative similarity when disposing of the current new threat sample.
More generally, any supplementary information that may be useful to a user in evaluating a new threat sample may be collected and displayed to the user. For example, this may include augmenting the description of the new threat sample with a reputation of the new threat sample, eg, based on reputation information available from a threat management facility. This may also include, or instead augment the description of the new threat sample with a suspicion score based on a genetic analysis of the characteristics of the new threat sample. In another aspect, this may include augmenting the description of the new threat sample with contextual information, such as users, related processes, associated data sources or files used by the threat sample, signature analysis, behavior analysis, update history software or endpoint status. , Etc.
As shown in step812, the method800may include removal of the intermediate threat(s), such as receiving user input via the user interface that classifies the new threat sample as safe, unsafe, or indeterminate. Thus, in one aspect, the user interface may be configured to receive user input that categorizes the new threat sample as secure, insecure, or indeterminate. When a disposition as unsafe does not automatically initiate corrective action, the user interface can also be configured to receive an express instruction for corrective action, such as any of the corrective actions described in this document, or any other suitable action to remove or otherwise manage. otherwise a new threat. In another aspect, the user interface can be configured to receive user input to adjust the filtering of an event stream from an endpoint that provided the new threat sample, which can allow for an increase or decrease in the number of reports. of events from the endpoint instead of, or in addition to, a specific characterization of the new threat sample.
In another aspect, a system as contemplated herein includes a memory that stores a first model for evaluating the probability that a sample threat is at least safe or malicious, a second model that characterizes a manner in which a number of interpretable features by humans contribute to an evaluation of the suspicion of a file and a third model to evaluate the similarity of threat samples. The system may include a threat management function that includes a processor configured to apply the first model to identify a new threat sample as an intermediate threat when the new threat sample does not have a predetermined probability of being malicious or safe based on the first model. model. The system may also include a web server configured to present a user interface that includes a description of the intermediate threat, augmented by one or more characteristics of the intermediate threat identified with the second model and one or more samples of similar threats identified with the second model. third model. the web server further configured to receive information from a user through the user interface eliminating the intermediate threat. Intermediate threat removal may include intermediate threat remediation. Intermediate threat removal may also include, or instead, characterizing the intermediate threat as secure, insecure, or indeterminate.
a variety of tools904for explicit provision of new threats, samples can be provided. For example, the user interface900can include tools904such as buttons or similar controls for a user to mark a particular event such as secure, insecure, low priority, unknown, or the like. user interface900You can also provide controls to query the enterprise network for additional information, to adjust the filtering of event streams from endpoint data loggers, to initiate scans or other analysis, and so on.
In one aspect, the user interface900can show a window906with more granular information about the characteristics that contribute to the suspicion. For example, a scan of a threat sample might be 90% suspicious of malicious code, while a file path scan might be 57% suspicious, and a URL scan might be 77% suspicious. While an integrative model can combine these various features into a single estimate of suspicion or potential risk, the individual values can be useful to a user trying to manually get rid of an intermediate threat. In addition, for any particular feature (for example, URL parsing in
In one aspect, the user interface1050can show a window1052list the human-interpretable features that contribute to an estimate of suspicion. For example, the user interface1050may present particular characteristics in the window1052such as whether a threat token is signed, whether the threat token calls crypto libraries, and whether the threat token inspects other processes. For each function of this type, the user interface1050It can also present the number of samples of known good and bad threats for that feature, with the features nested progressively according to the hierarchy of a random decision forest.
The features shown in this list may be a subset of features in a random forest of human-interpretable features that is selected based on relevance, for example, how strongly indicative those features are of security or suspicion. In one aspect, this can include features that have a higher percentage weight toward safety or suspiciousness. In another aspect, this may include features with the largest number of relevant samples (eg, higher up the decision tree). In another aspect, these and other factors may be collectively weighted or otherwise evaluated to select a subset of features to display to a user. This approach can usefully assist a human user when evaluating an intermediate threat for their manual disposition by providing a display of features that are the largest or most significant contributors to the potential risk associated with a sample threat.
In another aspect, the user interface may provide a display of the random output of the forest (eg, quantitative data on various human-interpretable features), or a display of most samples of similar safe and unsafe threats, or some combination of these. For example, the user interface may provide one or more user controls for the user to select between these different analytics and/or other analytics, contextual information, or other supplementary information.
The compute instances1102can be any of the computing instances described herein, including but not limited to any physical device such as a laptop, desktop, gateway, router, firewall, smartphone, tablet, or the like , as well as a virtualized instance of any of the foregoing or any other computer, user device, container or the like. the sensors1104and events1106it can also generally be any of the sensors and events described in this document. The local security agent.1108may be any of the security agents described in this document, or any other software or similar component running on or in association with one of the Compute Instances1102to locally manage the security of the compute instance and/or coordinate security services with the threat management facility1112and other remote resources.
The local security agent.1108can collect events1106of sensors1104in the computing instance1102, and form the collected events1106event vectors1110for communication with threat management facility1112. the sensors1104and/or local security agent1108can usefully process events1106in various ways to facilitate communication, computational efficiency, or post-processing. For example, events1106can be tokenized. That is, a process that causes or creates an event.1106it can be assigned a number or other identifier, which can be used locally by a computing instance or globally within the enterprise to identify a particular known process. An event1106you can also encode (tokenized or not) a relationship between different processes. For example, for a particular process that caused an event1106, a parent-child relationship or other dependency on other processes can be encoded by providing process ids or the like within the event1106, along with information characterizing the relationship between the processes. A Uniform Resource Locator or other information to identify resources or network locations may also be tokenized or otherwise processed to support efficiency, consistency, and the like. For example, a URL may be event-encoded1106as a hash of a URL, or as a part of a URL, or some combination of these (for example, a literal encoding of the top-level domain and a hash of some or all of the remaining path information). Other events1106such as registry changes, system calls, remote procedure calls, and the like can be literally coded into an event1106where they are relatively compact, or identified using any suitable tokenization, compression, or the like.
Other techniques may also be used or in their place. For example, user- or machine-specific information can be modified where appropriate to anonymize event vectors.1110and mitigate the exposure of sensitive information during network communications. A vector of events1110or individual events1106therein may also or instead be encrypted to secure the content against malicious interception. In another respect, the facts1106or vector events1110It can be compressed to conserve network resources. The event vectors1110can also or instead be prioritized, e.g. to increase sensitivity and decrease response times for event vectors1110associated with a high probability of malicious activity. In this last aspect, the local security agent1108can parse events locally1106and/or event vectors1110to enable proper prioritization, as well as to support local detection and response to malicious or potentially malicious activities.
It will also be appreciated that events1106and/or event vectors1110it can be usefully labeled in a variety of ways. While tagging with process identifiers is described above, this can also include, or instead, an identification of an entity associated with the event.1106the event vector1110. In this context, the entity can be any physical, logical, or conceptual entity useful for monitoring the activity of compute instances.1102as described in this document. For example, the entity may include a user, a physical device, a virtualized machine, an operating system, an application, a process, a hardware subsystem (for example, a network interface card, USB drive, camera, etc.). etc.), a network resource, a domain controller, a remote software service, etc. It should also be understood that the various types of entities may be simultaneously associated with a particular event.1106, detector1104, the event vector1110, or private events1106may be associated with multiple entities or event vectors1110. So, for example, storing a file can be an event1106associated with a particular user, a particular machine, a particular operating system, a particular physical storage device, etc.
In one aspect, event vectors1110They can be organized around entities. So, for example, a request for access to a network resource can be an event1106. When such a request is initiated by a user, an event vector1110for that user can be created and reported along with other temporally adjacent or otherwise related events1106associated with that user. When the network request involves an interaction with, for example, an authentication and identity management system, this may be represented as another entity or as an event.1106(the event group1106) in the event vector1110to the user. At the same time, a second vector of events1110for compute instance1102they can also be created and reported along with other temporally adjacent or otherwise related events1106associated with that compute instance1102. Alternatively, the event vectors1110They can be organized around chronology. That is, groups of events.1106within a time window can be reported as a vector of events1101. The event vectors1110can also, or instead, be organized around other aspects of the system1100, as particular sensors1104the sensor groups1104, causal relationships between events1106, particular triggers, types of activity (for example, network communications, operating system, processes, etc.), and so on. In general, the source of each event1106, as a particular sensor1104, or some entity, computer object or the like associated with the sensor1104, can be coded with the event1106to allow explicit identification by the threat management facility1112or other downstream processing resources. Although depicted in
The event vectors1110can be received by the threat management facility1112and stored as an event stream1114in a data repository1116, which can be any suitable datastore, memory, file, or the like for storing the event vectors1110. The event vectors1110may be timestamped or otherwise labeled by the threat management facility1112to record the chronology. In general, the flow of events1114it can be used for analysis and detection as described later in this document.
In general, an analysis module1118can analyze the flow of events1114to identify patterns of events1106inside the event stream1114useful for identifying unusual or suspicious behavior. In one aspect, this may include the creation of entity models1120that characterize the behavior of entities, such as any of the entities described here. Each entity model1120may, for example, include a multidimensional description of events1106for an event-based entity1106occurring over time for that entity. This can be, for example, a statistical model based on a history of events1106for the entity over time, for example, using a window or moving average of events1106.
entity models1120they can be, for example, vector or similar representations of different events1106expected for or associated with an entity, and may also include information about the frequency, magnitude, or pattern of occurrence of each event1106. In one aspect, the entity model1120may be based on an entity type (for example, a particular type of laptop or a particular application), which may have a related event schema that defines the event types1106that are associated with that entity type. This can provide a useful structural model for organizing events.1106and characterize an entity before any vector of events1110are collected, and/or to report what events1106monitor or associate with a particular entity.
Like a flow of events1114is collected, a statistical model or the like can be developed for each event1106represented within the entity model so that a baseline of expected activity can be created. In one aspect, an existing model may be used, for example, when the entity or entity type is already known and well characterized. The entity model can also be created by observing the activity of the entity (as recorded in the event stream).1114) over time. This may include, for example, tracking the entity for an hour, a day, a week, or any other suitable time interval to create a model with a sufficient probability of representing normal behavior to be useful as a baseline, such as seen at present. . In a practical example, certain software applications have been shown to produce a useful baseline in approximately two weeks. It will also be understood that once an entity model is created, the entity model may usefully be updated, which may occur at any suitable interval based on, for example, the length of time to obtain a stable baseline, the amount of activity by the entity, the importance of the entity (for example, for security, the operation of a computing instance1102, etc.), or any other factor.
These techniques can be used to create an entity model.1120for any of the entities described in this document, including, but not limited to, physical hardware items, virtualized items, software items, data and date stores, programming interfaces, communication interfaces, remote resources, etc., or any of the other entities, computer objects, assets or the like described in this document. In one aspect, entities can be organized around a conceptual stack for an endpoint in an enterprise network, such as providing entities for a domain controller, compute instance, user, operating system, library, application, a process and data. This can also include, or instead, any of a number of physical devices, such as a laptop, desktop, gateway, router, firewall, smartphone, tablet, personal computer, etc. a laptop, a server, a mobile device, an IoT device. The entity may also include, or instead of, hardware subsystems, such as a peripheral, keyboard, mouse, display, network interface card, USB drive, camera, disk drive, or other storage device. physical storage etc The entity may also include, or in lieu of, a virtualized instance of any of these physical devices or systems, or any other virtualized computing instance or other computing resource, such as a virtual machine, hypervisor, or the like. In another aspect, this may include computing objects or resources such as a container, operating system, library, application, process, file or other data, or the like. An entity may also include, or be in place of, remote resources, such as a cloud computing resource, cloud data resource, remote software service, or any other network or similar resource. An entity can also include other entities, such as a user or related identity, or more specific system resources, such as a kernel driver, system registry, process cache, and so on. More generally, any physical, virtual, logical, or other computing resource, asset, or the like that can usefully be instrumented and/or monitored to provide events for use as contemplated herein may be an entity as that term is used in this description.
As noted above, the entities of interest here may exist non-exclusively at various levels of hardware and software abstraction, and the entity models may have a similar and overlapping scope. By way of non-limiting example, an entity model for a laptop may include applications that run on the laptop. In one aspect, the entity model may incorporate all network activity on the laptop, while in another aspect, network activity may be associated with entity models for specific applications. Or network activity may be associated with both entities, eg, such that a single event is incorporated into multiple event vectors associated with multiple entities. In general, these design choices can affect the granularity of detections, the amount of processing and communications overhead, etc., and any variations consistent with implementation within an enterprise network as contemplated here are intended to fall within the scope of this disclosure.
Accordingly, in one aspect, an entity model may contain a schema or the like that describes events associated with an entity (or an entity type), together with information about the normal or expected behavior for each event.1106associated with the entity. In one aspect, an entity type (for example, a laptop or laptop from manufacturer X, or a virtual machine in environment Y) can be used to select a schema for an entity model, while activities of particular instances of that entity type. to generate the baseline for the entity model used in detections and the like. So, for example, if a user installs an office productivity suite, an entity model can be selected for that entity type based on event types.1106it is known to be associated with the use of the application or the capabilities of the application. However, different users may use the software differently, so the baseline of expected behavior can be assessed for a particular application installation by monitoring application activity over time. In another aspect, the schema of an entity model may itself be extensible. That is, the scheme of different events.1106can be created based on activity observations associated with the entity. When a new type of event1106is detected for that entity, the event1106can be added to the schema for a corresponding entity type.
Once an entity model1120a stable baseline has been created and established, the entity model1120can be implemented for use in tracking prospective activity. This monitoring can, for example, use the same event stream1114which was used to create the entity model1120, or a filtered or otherwise processed version of the event stream1114. It will be appreciated that entity models1120can generally be implemented as fixed, relatively static or discrete models, or one or more of the entity models1120can be continually updated to change over time as new information becomes available, for example in the event stream1114in another way.
The detection engine1122can compare new events1106generated by an entity, as recorded in the event stream1114, to entity model1120that characterizes a baseline of expected activity. Rendering the entity model1120and the event vectors1110in a common or related vector space, deviations from expected behavior can be usefully identified based on the vector distance between one or more event vectors1110and the entity model1120. This comparison can usefully employ a variety of similarity vectors or measures known in the art. For example, the comparison may use one or more vector distances such as a Euclidean distance, a Mahalanobis distance, a Minkowski distance, or any other suitable measure of difference within the corresponding vector space. In another aspect, a k-nearest neighbor classifier can be used to compute a distance between a point of interest and a training data set, or more generally to determine whether an event vector1110should be classified as within the reference activity characterized by the entity model.
It will be understood that while event vectors1110and entity models1120as described here provides a useful technique for observing deviations from a baseline of expected behavior by entities within an enterprise, the detection engine1122may also, or instead, employ other detection techniques based on the flow of events1114, for example, to support real-time detection of suspicious or malicious behavior. For example, certain events1106It can be a direct, independent indicator of malicious activity, such as initiating communications with a known command and control center for an Advanced Persistent Threat. Other events1106it can potentially be indicative of malicious activity, such as initiating full-disk encryption or transmitting sensitive information from an endpoint. While there are tools to detect this type of malicious activity, relevant events1106can be present in the event stream1114, and the answer facility1124can usefully trigger additional analysis, investigation, or other responses based on the flow of events1114instead of or in addition to monitoring deviations from the entity's baselines. In another aspect, concurrent deviations by different entities, or a pattern of deviations for a single entity or between entities, can also be usefully monitored. For example, a deviation in the behavior of a trusted application across multiple compute instances1102, either at the same time or in succession, may indicate a software update deployment rather than malicious behavior. Conversely, if a number of compute instances1102start communicating simultaneously with an unknown network address, this may be an indication that malware is spreading between devices on a corporate network. More generally, deviations between different entities, or between multiple instances of a particular entity, can provide useful information about the actual or potential causes of change, and can inform subsequent manual or automated investigations.
In general, where the flow of events1114deviates from a baseline of expected activity that is described in entity models1120for one or more entities, the response facility can initiate any number of responses1124Threat Management Facility1112. In one aspect, this may include implementing known remedies for malicious activity, such as quarantine, termination of network communications, termination of processes or applications, an increase in local monitoring activity on affected compute instances.1102, messages to a network administrator, filtering of network activity, virus scanning, patching or security fixes, etc. This can also happen on policy updates. For example, security policies for compute instances1102, users, applications, or the like may be upgraded to security settings that impose more stringent controls or limits on activity, including, for example, limits on network activity (bandwidth, data quotas, allowed network addresses , etc.), limits on system changes (for example, registry entries, certain system calls, etc.), limits on file activity (for example, file permission changes), higher levels supervision of local activity, etc.
As shown in step1202, the method1200it may include instrumenting the endpoint, eg with a local agent, to detect a plurality of types of changes to a plurality of computing objects. In general, the changes can be any of the events or other actions described in this document, and the compute objects can be any of the compute objects described in this document. For example, computer objects may include multiple files, multiple processes, and/or multiple executables. Computer objects may also include, or in place of, one or more of an electronic communication, a system settings register, a secure kernel cache, or any other data or data structure stored on or communicated to or from an endpoint. the end point. Similarly, the types of changes can be any types of changes that can be usefully monitored in a threat management context as contemplated here. For example, the endpoint can be instrumented to detect file reads and writes, but not file opens or closes. Or the endpoint may be instrumented to monitor incoming and outgoing email, but not outgoing email to other users within the company. As another example, the endpoint can be instrumented to monitor changes to operating system registry entries by non-system processes, or to monitor read/write activity that substantially increases file entropy. More generally, any type of change that may contribute to a suspicion or safety determination can be usefully monitored, with instrumentation of suitable corresponding computing objects, all as contemplated herein.
As shown in step1204, the method1200This may include creating an event stream from the local agent that includes each type of change to each of the compute objects detected on the endpoint.
As shown in step1206, the method1200may include storing the event stream in a data logger at the endpoint. Typically, this may be an unfiltered event stream that contains additional event data not included in a filtered event stream that is sent to a threat management facility, and may include some or all of the event data that the endpoint is instrumented to detect. For example, the unfiltered event stream may include other of the plurality of types of changes to the plurality of computing objects in a filtered event stream, or changes to additional ones of the plurality of computing objects not included in the event stream. filtered out.
As shown in step1208, the method1200You can include event stream processing with a filter at the endpoint to provide a filtered event stream that includes a subset of the types of changes to a subset of the compute objects. In one aspect, the subset of computer objects includes one or more of a file, an executable, a process, a database, and a message. In another aspect, the types of changes include at least one of file read, file write, file copy, file encrypt, file decrypt, network communication, registry update, software installation, permission change, and a query to a remote resource. It will be understood that while the filtered event stream is illustrated as coming from the event stream stored by the data logger, the filtered event stream may also be created directly by a security agent, or instead as it is created. the unfiltered event stream is captured and sent to the data logger for storage.
Event stream processing with the filter may also include local adjustment of the filter at the endpoint, eg in response to local changes detected in or by the endpoint. For example, the endpoint may locally adjust the level of filtering based on a reputation score for one or more processes, files, or the like on the endpoint. This filtering can be done for all detectable events on the endpoint or for specific processes. So, for example, when the reputation of a new process or other computing object is unknown, the endpoint can decrease filtering to provide a larger data report to the threat management facility for that particular process. So while I pass1216below contemplates control of the filter from a central threat management facility or similar, the filter can also be controlled locally on an endpoint in response to changes in security posture, policy compliance posture, or any other event, context , malware detections and so on.
In one aspect, the filtered event stream can be organized around anchor points such as a file, a domain name, or any other useful piece of data or metadata whose presence can be monitored on an endpoint. For example, a file hash can be created for a file and used to test the presence of that file on an enterprise's endpoints. Whenever this anchor point, eg the corresponding file hash, is detected on an endpoint, a collection of related events, metadata, context, etc. can be added. to the stream of filtered events to report to a central threat management facility.
In another aspect, the level of filtering can be controlled locally based on factors or requirements other than threat detection. For example, an event stream may be filtered to remove personally identifiable information, for example, to comply with data privacy regulations. As another example, filtering can be controlled based on network usage restrictions, for example, so that a particular endpoint does not exceed a predetermined hourly, daily, or weekly bandwidth quota for event reporting.
Furthermore, it will be understood that the filtered event stream may include synthetic events that characterize other collections of events into a single event or condensed group of events. This approach advantageously enables more compact communication of relevant information to a threat management facility, as well as more compact storage of information at the endpoint. In one aspect, synthetic events may be stored by the data logger instead of (eg, to reduce memory requirements) or in addition (eg, to reduce communications requirements while maintaining a record fuller or a related activity) a more detailed log of granular events on the endpoint. In another aspect, the data logger may store full details of the event, and the endpoint may (eg with the security agent) create synthetic events dynamically to facilitate more compact communication with the threat management facility.
As shown in step1210, the method1200it may include the transmission of the filtered event stream to a threat management facility. The filtered event stream may be transmitted at any suitable frequency, including periodic, aperiodic, or other scheduled transmission, as well as forced transmission (for example, at intervals determined by the endpoint) or dropped transmission (for example, at intervals determined by the endpoint). determined by the threat management facility). , or any combination of these. So, for example, the endpoint (or the security agent on the endpoint) can periodically report the filtered event stream at a predetermined time, with supplemental broadcasts provided when the security agent detects a potential threat, or requested when the threat is detected. threat management feature detects a potential threat.
As shown in step1212, the method1200may include receiving the filtered event stream at the threat management facility.
As shown in step1214, the method1200You can include processing of the filtered event stream in the threat management facility to assess a security state of the endpoint. This may include any suitable processing to parse the events within the filtered event stream. For example, processing of the filtered event stream may include searching for potential malicious activity on the endpoint, for example, based on a pattern of activities within the filtered event stream, or based on a specific activity, such as a unauthorized change to a registry entry. Processing of the filtered event stream may also include, or instead, scan for a security risk on the endpoint, such as a missing security patch, a firewall configuration change, a security scanner uninstall malware etc In another aspect, the processing of the filtered event stream may include safe checking of the endpoint's health, for example with a safe heartbeat or the like from the endpoint, to ensure that the endpoint has not been otherwise compromised. . In another aspect, the processing of the filtered event stream may include monitoring for changes that cause the endpoint to not comply with a security policy for an enterprise, or that otherwise present an actual or potential risk to the security of the enterprise. network for the company
As shown in step1216, the method1200may include conditional transmission of settings to filtering by the endpoint. For example, the method1200may include, in response to a predetermined security state detected by the threat management facility, transmitting a setting to the endpoint for at least one of the types of change or computing objects used by the filter to process the event stream. This may include transmitting a setting to a filter used by the endpoint to select which of the plurality of types of changes to the plurality of computing objects is reported by the data logger in the filtered event stream. Thus, for example, when the security state indicated by the filtered event stream is a potentially compromised state of a file, process, or the like, the threat management function can lower the filtering to receive more data about various changes in or by computer objects. at the end point. This can include general changes to the filtering level or specific changes that focus on specific computing objects or types of changes that could be related to a potential compromise. In one aspect, tuning to endpoint filtering may include a change to the subset of change types included in the filtered event stream, such as increasing the types of changes included in the filtered event stream when the endpoint is potentially compromised. , or decrease the types of changes included in the filtered event stream when a potential compromise was fixed. The tuning may also include, or instead, a change to the subset of computing objects included in the event stream, for example, by monitoring additional processes, directories, or the like when a potential compromise is detected.
Adjustments can also be made to filtering by other endpoints within an enterprise network. For example, when a compromise is detected on one endpoint, behaviors or other patterns detected in the event stream (filtering) for that endpoint can be used to tune filtering on other endpoints to make it easier to detect similar or related patterns. elsewhere within the company. grid. Similarly, endpoints or data resources known to contain assets of high business value may have fine-tuned filtering to facilitate more detailed and frequent monitoring of related assets.
In another aspect, the filtering can be adjusted independently of the current filtered event stream, eg, based on another context. For example, when an employee is about to leave a company, filtering can be reduced or removed from any associated compute instances so that computing or network activity can be more closely monitored until departure.
As shown in step1218, the method1200may include other processing based on the filtered event stream. For example, the method1200It can include correlating the filtered event stream with a malware event on the endpoint and searching for the malware event on one or more endpoints attached to the enterprise network based on a pattern of events in the filtered event stream. In another aspect, the method1200You can include the storage of the filtered event stream in the threat management facility. In another aspect, the method1200This may include, when the filtered event stream shows that the endpoint's security state is compromised, initiating corrective action, for example, using any of the remediation tools available to the threat management facility.
Accordingly, a system including an endpoint and a threat management facility is also described herein. The endpoint may run a data logger to store a stream of events that includes a plurality of types of changes to a plurality of computing objects detected on the endpoint, and the endpoint may run a local agent to process the stream of events. with a filter in a filtered event. flow including a subset of the plurality of types of changes to a subset of the plurality of computing objects. The home agent can also be configured to communicate the filtered event stream to a remote resource over a data network. The threat management function can be configured to receive the filtered event stream from the endpoint and process the filtered event stream to assess a security status of the endpoint. The threat management function can further be configured to respond to a predetermined change in security state by transmitting an adjustment to the endpoint for at least one of the types of changes or the computing objects used by the filter to process the event stream. In one aspect, the threat management function can be configured to initiate a remediation of the endpoint when the security state of the endpoint is compromised.
As shown in step1302, the method1300it may include instrumenting the endpoint as described herein, eg, with a local agent, to detect a plurality of types of changes to a plurality of computing objects. In general, the changes can be any of the events or other actions described in this document, and the compute objects can be any of the compute objects described in this document. For example, computer objects may include multiple files, multiple processes, and/or multiple executables. Computer objects may also include, or instead of, one or more of an electronic communication, a system configurations registry, and a secure kernel cache.
As shown in step1304, the method1300This may include creating an event stream from the local agent that includes, for example, each type of change to each of the compute objects detected on the endpoint.
As shown in step1306, the method1300may include storing the event stream in a data logger at the endpoint. As described above, this can typically be an unfiltered event stream that contains additional event data not included in a filtered event stream that is sent to a threat management facility, such as some or all of the event data that the endpoint is instrumented to detect. For example, the unfiltered event stream may include additional changes to the plurality of change types to the plurality of computing objects in a filtered event stream, or one or more of the plurality of change types to the additional ones of the plurality of computer objects. .
As shown in step1308, the method1300You can include event stream processing with a filter at the endpoint to provide a filtered event stream that includes a subset of the types of changes to a subset of the compute objects. In one aspect, the subset of computer objects includes one or more of a file, an executable, a process, a database, and a message. In another aspect, the types of changes include at least one of file read, file write, file copy, file encrypt, file decrypt, network communication, registry update, software installation, permission change, and a query to a remote resource.
As shown in step1310, the method1300it may include the transmission of the filtered event stream to a threat management facility, for example, as described above.
As shown in step1312, the method1300may include receiving the filtered event stream at the threat management facility.
As shown in step1314, the method1300You can include processing of the filtered event stream in the threat management facility to assess a security state of the endpoint. This can include any suitable processing for the events within the filtered event stream. For example, processing of the filtered event stream may include searching for potential malicious activity on the endpoint, for example, based on a pattern of activities within the filtered event stream, or based on a specific activity, such as a unauthorized change to a registry entry. Processing of the filtered event stream may also include, or instead, scan for a security risk on the endpoint, such as a missing security patch, a firewall configuration change, a security scanner uninstall malware etc In another aspect, the processing of the filtered event stream may include safe checking of the endpoint's health, for example with a safe heartbeat or the like from the endpoint, to ensure that the endpoint has not been otherwise compromised. . More generally, this may include any of the processing described in this document that could usefully be performed by a stream-of-event-based threat management installation of one or more endpoints associated with an enterprise network.
As shown in step1316, the method1300it may include conditionally passing on a request to the endpoint, or more specifically, to the data logger on the endpoint, for additional event data in the unfiltered event stream. For example, this may include, in response to a default security state detected by the threat management facility, requesting additional event data from the data logger for at least one of the other types of changes that the subset of the types of changes changes or other of the plurality of computing objects than the subset of the computing objects. The request may include a request for all event data in an unfiltered event stream stored by the data logger for a predetermined time window. The request may also include, or in lieu of, a request for a larger group of additional Computing object event or change types. The default change in security state can be any change that raises suspicions or indicates that additional information may be useful for manual review, automated review, forensic documentation, or some combination of these. For example, the default change in endpoint security state may include an increased probability of malicious activity associated with the endpoint. The change may also include, or instead, a change in policy enforcement, detection of known malware, suspicious network communications, access to high-value business assets, etc.
As shown in step1318, the method1300may include other processing based on the filtered event stream. For example, the method1300It can include correlating the filtered event stream with a malware event on the endpoint and searching for the malware event on one or more endpoints attached to the enterprise network based on a pattern of events in the filtered event stream. In another aspect, the method1300You can include the storage of the filtered event stream in the threat management facility. In another aspect, the method1300This may include, when the filtered event stream shows that the endpoint's security state is compromised, initiating corrective action, for example, using any of the remediation tools available to the threat management facility. More generally, any action necessary or useful to detect, investigate, dispose of, or manage threats based on the filtered event stream can usefully be performed in this step.
Accordingly, in one aspect, a system including an endpoint and a threat management facility is disclosed herein. The endpoint may execute a data logger to store an event stream of event data including a plurality of types of changes to a plurality of computing objects detected at the endpoint. The endpoint may also run a local agent configured to process the event stream with a filter on a filtered event stream that includes a subset of the plurality of types of changes to a subset of the plurality of computing objects. The home agent can also be configured to communicate the filtered event stream to a remote resource over a data network. The threat management function can be configured to receive the filtered event stream from the endpoint and to process the filtered event stream to assess a security state of the endpoint, the threat management function can further be configured to respond to a default change in security state by transmitting a request to the endpoint for additional event data stored by the data logger. In one aspect, the threat management function is further configured to initiate a remediation of the endpoint when the security state of the endpoint is compromised.
the business network1402may include any of the endpoints described in this document, such as laptops, desktops, mobile devices, or other computing instances for users, as well as firewalls, gateways, and any other participants, security infrastructure, network infrastructure, or similar that forms an enterprise network as described in this document. In general, the business network1402can produce a sequence of events like any of the events described in this document. This can include sensor events, local security agent events, network element events or points of presence (such as firewalls, gateways, WiFi routers, access points, etc.), etc. It will be appreciated that, in general, these events may be streaming events that are provided and incorporated into the streaming service.1404in real time, or batches of events that are delivered as collections of events in a single stream, for example, based on a local reporting schedule used within the enterprise network1402or according to the availability of the network.
the current service1404can ingest events from the enterprise network1402including any of the events and the like described in this document. In one aspect, the streaming service1404may receive events through an interface using pre-signed uniform resource locators or other techniques that may automatically add prefixes that identify a client, device, or other source of information for each event or collection of events. the current service1404you can also or instead receive data from any other event sources relevant to the security of the company or useful for managing the data platform1400as described in this document. For example, this may include receiving threat detection signature updates from third-party security resources, receiving software updates and patches from software vendors, etc. In general, the streaming service1404may include any suitable storage or event stream processing technology, or any similar hardware and/or software layer suitable for storing, managing, processing, and querying event streams as contemplated herein, or supporting event-based information. Some or all of the data on the streaming service1404they may also or instead be stored in a high-speed storage facility for queries or other data processing that has high performance requirements.
the transformer1406you can usually process events on the streaming service1404, for example, by organizing the data according to one or more applicable schemas from the schema registry1412and augmenting the data with any suitable metadata to provide augmented event data for use in threat detection, investigation, and management. For example, the transformer1406you can add a client identifier, firewall identifier, or other information to identify the source of an event. the transformer1406you can also, or instead, add a schema version that specifies a schema in the schema registry1412which can be used to organize the data provided to the streaming service1404or the data lake1408. the transformer1406may also, or instead, create a timestamp, file size, hash, file path, or other information useful for identifying or describing data associated with, or the source or interpretation of, an event, which may be added to the event before storing it in the data lake1408. Generally, the transformer1406can stream transformed event data back to the streaming service1404for short-term use (for example, one hour, one day, seven days, etc.) by listeners1410or high speed access by query engine1414. the transformer1406you can also or instead stream transformed event data to the data lake1408for long-term storage (for example, one week, one month, one year, etc.). It will be understood that the general limits for short-term and long-term storage may vary depending on, for example, storage capacity, processing speed, data volume, etc. when the transformer1406send messages with metadata to streaming service1404, the transformer1406you can use any suitable data format, and you can usefully compress the stream representation by including pointers to replace, for example, a schema, the underlying source data, etc.
While displayed as a single transformer1406, it will be understood that the platform1400may use any number of transformers, operating in sequence or in parallel, or some combination of these, suitable to process events in a timely manner and maintain flow service1404in a state suitable for, for example, real-time threat detection, remediation, and/or other security-related functions.
the data lake1408can receive messages from transformer1406and store the message data in a way that supports long-term storage and allows search and retrieval by the query engine1414. In general, the data lake1408may provide a single data store, including source data in a natural or originally provided raw data format, for example, as binary large objects ("blobs") or other files or the like, together with any metadata or transformed data that they be added. the data lake1408may contain structured data (for example, from relational databases), semi-structured data containing CSV, logs, XML, JSON, etc., and/or unstructured data such as emails, documents, PDF files, and binary data such as images, audio, video and any other data that can be received from the corporate network1402or other sources relevant to the network security system as described in this document. In one aspect, the source data in the streaming service1404can be filtered or otherwise processed by the transformer1406to improve the quantity and quality of data maintained in the data lake1408for the various uses described here. A variety of cloud-based and other data lake technologies are known in the art and are commercially available, and can be adapted for use with the data lake.1408described in this document.
listeners1410can be user-configurable or pre-configured listeners that monitor the streaming service using, for example, metadata provided by the transformer1406, for events of interest. every listener1410can monitor a stream of events supported by the streaming service1404and generate appropriate alerts, actions, or other responses by applying rules, application logic, filters, and so on, to events in the event stream.
Schema registration1412can store schematics for use of, for example, the transformer1406and/or listeners1410when writing data to streaming service1404, reading data from streaming service1404, or otherwise processing or interacting with data on the streaming service1404or the data lake1408. In general, schemas can be versionable or extensible, and each message in the broadcast service1404the use of a schema to structure the data may include a schema identifier in the message to facilitate interpretation and other uses by consumers of the streaming service1404. . . . users of the platform1400in general, and the streaming service1404and data lake1408in particular, they can inspect current schemas, update schemas (which they own or control), and otherwise access the schema registry1412to interact with the streaming service1404and data lake1408in a structured way, or support various functions of the platform1400described in this document. As new schemas are created, for example, to address new types of data or information, or as current schemas are updated, a history of schema identifiers and versions can be kept in the schema registry.1412for later reference, and/or a newer schema can be inserted into the data in the data lake1408and/or current1404.
The query engine1414can be any suitable search engine to query the data lake1408and other data sources. This may include automated queries executed according to a query database schedule.1420. This can also, or instead, include preconfigured queries that are run from the query database.1420by a user from one of the consoles1418. This can also include queries that contain preconfigured query customizations or fully customized queries that are initiated by users from the consoles.1418. It will be understood that while the data lake1408is a useful target for query engine queries1414, the query engine1414you can also or instead request data from other resources, such as the streaming service1404, terminals or security agents in the business network1402, or third-party data sources, such as threat libraries and the like.
query monitor1416you can usually monitor the query activity by the query engine1414as well as other activity of the user consoles1418. This may include monitoring query activity by console users.1418as well as automated or scheduled query activity managed through the query database1420. In one aspect, the query monitor1416can log specific queries launched by the query engine1414to track, for example, the popularity of existing queries, user modifications to existing queries, and the like. So, for example, a query that users change frequently can be republished in the query database.1420in its modified form for later use as a preconfigured query. In another aspect, the query engine1414you can monitor a context in which queries are started or adapted. For example, a pattern of queries or query modifications can be correlated with a concurrent development of a known threat and used to create query-based threat detection techniques or to identify query activity that can be associated with effective threat management. real. As another example, when launching specific (non-query) measures from one of the consoles1418following a query, including activity such as scans, remedial actions, or the like, this can be used to assess the effectiveness of the query and identify queries that appear to be most useful or informative to users. Therefore, by monitoring query activity initiated through the query engine1414and/or other contextual activity of users through the consoles1418, the query monitor1416You can correlate specific queries with threat identification, threat response, etc., or track the popularity of a query or sequence of queries. All this information can be stored in the query database.1420together with query logs, preconfigured queries, and the like for use in monitoring and evaluating query activity as described in this document.
the consoles1418, which may be administrative consoles for system administrators, or any other user console or the like, may be deployed from a server or other remote or hosted system using, for example, web technologies or the like to support a local interface to any suitable end user devices. In general, each console1418it can display query information, security information, user options, and the like, and it can provide user controls for entering text, selecting options, configuring queries, and so on. Thus, in one aspect, a host device for the platform1400can cause one of the consoles1418including a user interface to be presented on an end user device to an administrator or other end user. each console1418You can also include a local agent to track console user activity. While a query monitor1416on the data platform1400you can usually track query activity by a local query engine1414, an agent on each console can advantageously support tracking other user activity that does not involve direct interactions with the query database1420the query engine1414.
The database1420it can be any useful database for storing information related to queries as described in this document. This can, for example, include preconfigured queries for deployment from one of the consoles1418via query engine1414, as well as a log of queries made by the query engine1414along with metadata such as the time of the query, the user who initiated the query, and the structure of the query. This may also include, or instead, contextual information, such as activity on one of the consoles.1418before, during and/or after initiating a query, or any other information that may be useful in evaluating the effectiveness or diagnostic relevance of queries initiated via the query engine1414.
As shown in step1502, the method1500may include storage of a data lake1501, like any of the data lakes described in this document. This may include, for example, storing a data lake containing a first plurality of data objects representing security events and a plurality of descriptions for the first plurality of data objects. The first plurality of data objects may include security events from one or more data loggers at endpoints in a business network, which may be received in an event stream.1503as an event stream hosted by the streaming service described above, or in any other suitable data repository or service. The plurality of descriptions can be organized according to one or more schemas that characterize the structure of the data contained in the data objects. These schemas can, for example, be stored in a schema registry and used to transform or describe the data structure in the event stream.
In one aspect, the data lake1501you can use a flat schema that uses columnar storage organized by fields like username, time, device, and the like. Data objects in the data lake can also be organized for ease of use, for example by putting identifiers or other high-level metadata in a separate small file, putting commonly used data (for example, extracted or derived data for dashboards). analysis, real-time data). event listener etc) in a second small file and putting the remaining data in a larger data file to access if needed.
As shown in step1504, the method1500it may include receiving a second plurality of data objects. These data objects can be received in an asynchronous stream of security events from the enterprise network. In one aspect, the asynchronous stream of security events may include one or more batch transfers that include groups of security events. In another aspect, the asynchronous stream of security events may include stream transfers of individual security events. Asynchronous transmission can also include, or instead, a combination of batch transfers and streaming transfers, such as when some devices on the enterprise network transmit events in real time, other devices store and forward events and batches, and other devices send events in a connectivity. -In a dependent manner based on, for example, the availability, quality or bandwidth of an available connection. In general, the data objects in the data lake may include security events from one or more data loggers on endpoints in the enterprise network, or any other information from any other source or combination of sources useful for security analysis. and the like.
As shown in step1506, the method1500it may include filtering the received data objects, for example, using the transformer described above. This may include filtering the second plurality of data objects to remove duplicate data objects already included in the first plurality of data objects. With multiple sensors and endpoints generating events asynchronously, it is possible for a particular event to be reported more than once. To avoid polluting the data lake1501with duplicate data, the transformer can usefully remove the duplicate information. For example, filtering may include applying at least one bloom filter to identify one of the second plurality of data objects that might be in the data lake and selectively performing a deduplication search on the data lake for only one of the data objects. the second plurality of data objects. where the possibility of a duplicate exists, for example where the bloom filter indicates that the data object might already be present in the data lake.
A bloom filter is a space-efficient data structure that uses hashing techniques to test whether an element is a member of an array. In general, a bloom filter eliminates the possibility of false negative matches, but not false positive matches. While other filtering techniques are possible, such as a brute force search of existing records in the data lake1501, the bloom filter provides a compact and computationally efficient technique that is advantageously extensible with the addition of new elements to an array. Thus, a bloom filter can be created and used to advantage with a growing data lake to efficiently test whether a particular data object has already been stored in the data lake.1501and to reduce the number of queries to the data lake1501that might otherwise be required for deduplication. This can significantly increase the efficiency of the transformer, particularly when a query to the data lake1501it is substantially slower than applying the bloom filter. It will also be understood that a separate bloom filter may be created for each device in order to manage size. Therefore, when a new device appears on the enterprise network, a new bloom filter can be created and associated with a device identifier or another identifier for the new device so that the new bloom filter can be applied to events associated with the device identifier.
As shown in step1508, the method1500may include augmenting the second plurality of data objects, for example augmenting each of the second plurality of data objects with a corresponding description that is organized according to at least one of one or more schemes used by the transformer described above to structure data into an event stream and a data lake. For example, an event or message in the event stream1503can be processed into several different files, including, for example, a first metadata file with high-level metadata that identifies an event as a source device, an event time, and a target identifier such as a size, hash, file name, or similar for the object. This first metadata file can use a global schema (eg for identification) for all data objects placed in the event stream.1503and/or data lake1501.
A second metadata file can include tagging or parsing to support real-time listening. More generally, the second metadata file may include any relevant event descriptions or identification information, summaries, analytics, and the like to support high-speed processing of the event stream.1503. This can include any useful tagging or characterization for automated listeners to identify relevant data or events in the event stream.1503, and can be customized by a particular user according to the intended use. For example, the second metadata file can identify an entity type (eg firewall, gateway, mobile device, etc.), an event type (eg policy violation, configuration change , network event, etc.), a type of user (eg, , system, human, etc.), a type of traffic, a reputation (including quantitative reputation, such as a reputation score, or information from qualitative reputation, such as "good", "bad", or "unknown"), or any other attribute(s) or information that might be useful to listeners. Schemas for this information can be selected, for example, for particular users of the data lake1501, for particular devices providing security events, for particular network locations, etc. Thus, in one aspect, one of the schemes used to characterize data objects may include a device-dependent scheme selected for one of the data objects according to a source of one of the data objects when received in the asynchronous stream. . While device-dependent schemas can usefully be used to structure metadata differently for different source devices, schemas can also be specific to a user, network location, application, process, or any other source of information. network, physical or logical. of an event
In one aspect, one or more schemas may be columnar schemas to provide a flat, non-hierarchical structure for metadata to improve efficiency, for example, when processing real-time event data in the event stream.1503.
As shown in step1510, the method1500it may include storing the second plurality of data objects and a corresponding plurality of descriptions according to one or more schemas with the first plurality of data objects in the data lake. In addition to any metadata files (such as the two described above), this can include a raw data file that contains a complete data object as it appeared natively in the event stream.1503of the business network. After the above processing, the resulting collection of files can be stored in the data lake.1501in an augmented form that includes the raw data file together with the first and second metadata files, and/or any other descriptive data or analysis that may be useful to subsequent users. Data objects can be stored in the data lake.1501in any of a number of ways to optimize storage and usage. For example, data objects may use a flat schema and may be marked according to any suitable access or use restrictions. This may include the labeling of data such as sensitive, confidential, financial, technical, valuable, containing personally identifiable information, etc. As a transformer or other system processes data for storage, data objects can also be structured for optimal use in the event stream.1503and/or in subsequent queries to the data lake1501.
In another aspect, metadata files can be stored in the event stream.1503for real-time processing, while the (usually larger) raw data object is sent to the dataset1501. In this case, the metadata files may include a pointer or other location identifier to aid in the retrieval of the raw data from the data set.1501when requested by, for example, one of the listeners. In another aspect, the raw data object can never enter the event stream1503, and can instead be sent directly to a transformer or similar entity for processing and storage in the data lake1501. In this way, the flow of events1503can be used exclusively for high-speed processing of smaller metadata files, with raw data objects stored separately in the data lake1501to be accessed if/when needed by a listener detecting relevant information in the metadata, or by a user querying the data lake1501.
As shown in step1512, the method1500may include hearing objects1512. This may include monitoring the flow of events1503, for example, by monitoring metadata placed in the event stream1503by a transformer using one or more proprietary schemas, to identify any relevant attributes, events, actions, or the like in the event stream1503which may be relevant to a function of one of the listeners. When relevant metadata is detected, a corresponding listener can take any appropriate action, including creating a user alert or notification, initiating corrective action, requesting additional information from endpoints on an enterprise network (for example, requesting data stored in local data loggers), retrieving a corresponding raw data object from the data lake1501for analysis etc In general, this listening can occur when new items are placed in the event stream.1503(for example, in real time), or as raw data objects and/or metadata files stored in the data lake1501, or any combination of these.
As shown in step1514, the method1500can include data lake lookup1501for security events of interest. This may include searching for metadata in metadata files that augment raw data objects, searching directly in raw data objects, or some combination of these. It will be understood that security events of interest may include any enterprise network events that may be indicative of malicious activity, vulnerabilities, policy compliance, or otherwise relevant to threat detection and security management as described in this document.
As shown in step1516, the method1500may include making any additional inquiries. For example, when a sensitive file is sent via email from an endpoint, this may be an allowed communication when performed by a human user with the proper credentials, but an impermissible communication when no human user is present at the endpoint. When a local security agent monitors human presence, the corresponding information can be stored in a local data logger but not automatically sent to the event stream. In this case, in response to the data obtained during the search of the data lake, the method1500it may include directly querying at least one of the endpoints for additional information. It will be understood that this example is intended to be non-limiting, and any event or combination of events that suggests an additional query may be used as a trigger to request additional information from one or more endpoints or data loggers on the enterprise network as contemplated herein. .
Thus, more generally, when searching the data lake1501for security events of interest, an event that requires additional information from an endpoint can be identified, and the method1500It may include a variety of searches or other tools to support subsequent manual (eg, human) or automated (eg, machine) investigation. These additional inquiries can be made for a variety of reasons, such as when investigation of a developing threat is continuing, when performing historical analysis of a previous security breach, or when suspicious activity within the enterprise network arises. Any of these can cause an analyst to create new searches, change the parameters of existing searches, drill down on particular search results, etc., and all of these types of research can be usefully supported by the data lake.1501, including any augmented metadata contained therein.
Accordingly, this document describes a system that includes a data lake, a transmission service and a transformer service. The data lake may include a data storage medium that stores a first plurality of data objects representing security events within an enterprise network and a plurality of descriptions for the first plurality of data objects, each of the plurality of descriptions organized according to one or more schemes. The Stream service can be configured to receive a stream of asynchronous events from additional data objects that represent security events from the enterprise network. The transformer service can be configured to process the stream of asynchronous events by filtering additional data objects to remove duplicates of one or more data objects already stored in the data lake, thus providing filtered data objects, to augment each of filtered data objects with a corresponding description organized according to one or more schemas, thus providing augmented data objects, and for storing the augmented data objects in the data lake.
The asynchronous event stream may include at least one batch transfer including a group of security events, a streaming transfer of one or more individual security events, and a connectivity-dependent transfer. Filtering out the additional data objects may include applying a bloom filter to the asynchronous event stream to detect a first set of additional data objects that are definitely not in the data lake and a second set of additional data objects that could be in the data lake. . The transformer service can be further configured to perform a data lake deduplication lookup on each of the second set of additional data objects. In one aspect, the system may further include a query engine configured to search the data lake for one or more security events of interest and/or to request data from local data registrars operating at endpoints within the enterprise network. .
As shown in step1602, the method1600may include receiving an event stream that includes security events from an enterprise network, such as security events from one or more sensors on compute instances on the enterprise network. The event stream may be received, for example, in a stream service or other suitable resource. This can include any events described in this document, such as events captured by sensors at endpoints throughout the enterprise network.
As shown in step1604, the method1600may include storing the event stream in a data lake. This can include any of the data lakes described in this document, which can be augmented with metadata files for efficient search and analysis. It will be understood that events may also or instead be stored locally in endpoint data logs, or any other location in between. While the data lake can provide a useful means for the storage and retrieval of relevant information, local data logs can also be used to advantageously enable the download of more granular storage, or raw event information, to local devices in the entire business network. These distributed resources may be viewed as useful and/or necessary when investigating business threats.
As shown in step1606, the method1600it may include storing a plurality of queries for execution against the event stream and/or the data lake. In general, the plurality of queries can be configured to investigate security issues within the enterprise network based on the flow of events and/or other information available to a query engine and relevant to the investigation of security issues. This may, for example, include database queries or the like configured for use with the data lake.
As shown in step1608, the method1600It may include the use of multi-query monitoring, for example, with a monitoring agent or other automated monitoring system. This may include monitoring the use of plurality queries as users issue them from one or more administrative consoles to an enterprise network threat management facility. This may also include, or instead, monitoring automated queries that are issued, for example, on a scheduled basis from the query database or other automated tool.
In one aspect, this may include monitoring changes to one or more of the plurality of queries. In another aspect, this may include monitoring post-query remediation activity initiated in one or more administrative consoles. For example, where a user consistently initiates a particular type of remediation after receiving the results of a particular preconfigured query, then the query itself can be used as an indicator of a corresponding threat, for example, the threat being targeted. remedying. after the consultation. This allows for enhanced contextual recommendations for a user initiating the query and/or a variety of preemptive and/or automated responses based on an inference that the threat is present on the enterprise network. In another aspect, this may include monitoring queries for a plurality of enterprise networks, for example, so that activity from various enterprises can be aggregated and analyzed to identify best practices (and bad practices) and learn how query activity from administrators map to the development of threats and/or the success of responses to threats. Therefore, the query activity can be crowdsourced to allow individual enterprise administrators to benefit from the successes and failures identified in the activity by other administrators for other networks.
In one aspect, this may include monitoring the use of the plurality of queries by one or more experts, such as a professional security analyst or technician, to develop an expert system or the like for use in generating recommendations or guidance for others. when managing networks This system can make contextual recommendations, respond to queries and requests for help, etc.
As shown in step1610, the method1600may include determining a usage history based on the usage of the plurality of queries. In one aspect, this may include the use of the queries themselves, such as when and how often the queries are used, or if and how they are changed when users deploy them from the administrative consoles. For example, the usage history may include the popularity of one or more of the plurality of queries. This can help identify which queries are perceived to be useful so that these queries can be suggested preferentially to users, optimized for performance, or otherwise adapted for more widespread and frequent use.
In another aspect, the usage history may include a pattern of changes to one or more of the plurality of queries. A variety of useful inferences based on patterns of change are available. For example, a constantly requested change, modification, or customization to a particular stored query might suggest that the stored query is not optimal for many users. In this case, the stored query can be updated to reflect this user preference. In another example, different types of changes to a popular query may correspond to different threats or other security issues. In this case, a particular modification to a query to focus on, for example, events in a particular time window, on particular device types (mobile devices, USB drives, etc., devices with a particular operating system, etc.) , in particular locations (eg, path names, network locations, etc.), etc., can be used to draw useful inferences about a threat or to resolve an ongoing threat.
In another aspect, a pattern of post-query activities initiated from an administrative console (or other user interface or the like) can provide useful usage history for security analysis. For example, when a particular query is consistently followed by a particular security response, such as remediation, malware scanning, network isolation, and the like, then the query can be used as an indicator of a corresponding threat. Similarly, a consistent pattern of post-query activity may suggest that a particular query is valuable or useful, and the query may rank higher in lists of queries presented to a user, or be suggested more often in response to user queries. Other types of inferences can also be usefully drawn, such as the addition of a new user, a structural reorganization of a company, a change in network security policy, etc. By enabling a query monitor or similar tool to monitor activity on administrative consoles beyond data lake search activity, or by providing a local monitoring agent for each administrative console, additional context can be made available to help recognize patterns associated with malware threats, threat remediation, and so on beyond what could be discovered based solely on events reported by sensors within the enterprise network. As a significant advantage, this type of monitoring indirectly captures human levels of interest and response that would not otherwise be available when rules are applied to enterprise network event streams. A context in which queries are executed can also, or instead, be usefully used as part of usage history and can include information about an administrative console user, additional global threats, or security information from sources other than the business network, business value and reputation associated with security events, endpoint heartbeats providing reports, reputation associated with endpoints or endpoint users, etc.
Another context can also be used, or instead, such as if an existing workflow, process, connectivity, or the like was interrupted immediately before a new query. For example, when an administrative console draws attention to a particular endpoint or sequence of events, this can provide useful context about where a threat is taking place. Similarly, threat mitigation activity can provide useful context for the value of a particular query and the circumstances under which the query might be used. Relevant threat mitigation activity may, for example, include machine isolation, forensics, malware trap creation, etc., as well as additional queries that could be used to assess how a particular machine was infected or similar.
As shown in step1612, the method1600may include the initiation of an action by the threat management facility based on usage history. This may include any of a variety of automated or similar corrective actions, as well as alerts or notifications to administrators or other users who may be affected by a security breach. For example, initiating action may include identifying a pattern of queries associated with a known threat and generating a recommendation for one or more corrective response actions. Initiating the action may also include, or instead, evaluate the utility of one of the plurality of queries based on a pattern of post-query activity. In another aspect, initiating the action may include evaluating the usefulness of one of the plurality of queries based on a pattern of query modifications by users.
More generally, the system can initiate any useful action based on aggregate search behavior, the context of the search behavior, user actions before, during, and after searches, or available inferences from combinations of the above. former. For example, if a particular query is very popular, then data for this query can be periodically prefetched to improve response times. Similarly, when a particular query is very frequently followed by another particular query, the data for the second query may be prefetched when the first query is started. As another example, when a particular query is regularly followed by requests for additional information from local data loggers at endpoints within the enterprise network, the query monitor or some other suitable automated software agent may initiate corresponding data requests from security agents on the endpoints in to begin aggregating the data intended for use. This can be particularly advantageous in contexts where significant latency in responses from individual security agents is expected across the enterprise network.
In another aspect, a general log of events surrounding a threat including, e.g. actions and the like, can usefully characterize a particular threat in a way that allows for the subsequent identification of similar threats. For example, this data can be turned into training sets for machine learning, and a model can be trained to detect future threats or provide guidance to respond to threats, based on these observations.
Based on the above, this document describes a system that includes a data lake, an administrative console, a database, and a query monitoring agent. The data lake can store an event stream that includes security events from one or more sensors on compute instances in an enterprise network. The administrative console can receive queries from users and have the queries run on the event stream. The database can store a plurality of queries for execution against the flow of events in the administrative console, the plurality of queries configured to investigate security issues within the enterprise network based on the flow of events. The query monitor agent may provide a query monitor configured to monitor query plurality usage in the administrative console, to determine a usage history based on query plurality usage, and to initiate action by a threat management facility based on usage history.
As shown in step1702, the method1700It can include instrumenting an endpoint in an enterprise network with a sensor and a local security agent. The sensor can include any of the sensors described here and can be configured to generate an event log in response to an event. The local security agent can be configured to locally receive the event log from the sensor, or otherwise receive event data in response to an event occurring (or detected on) the endpoint.
As shown in step1704, the method1700may include receiving event data, such as event logs or the like from sensors at the endpoint.
As shown in step1706, the method1700may include generating a source identifier that identifies a source of the event in a context of the sensor. This may include determining the context of the sensor (eg, with the local security agent) by inspecting one or more network resources associated with the endpoint. In general, this contextual identifier can augment the source identification information using any information locally available to the local security agent and useful for determining the source of any corresponding event. The source identifier may combine any source identifier available in the local context of the security agent, such as one or more logical, physical, and/or virtual identifiers of an event source.
For example, the source identifier may include a physical address associated with the source, a network address associated with the source, and at least one temporary address assigned by the endpoint to the source of the event. As a more specific example, the physical address may include a media access control address associated with the source, or any other physical layer address or the like associated with a physical address of the source and/or useful for identifying or accessing the source. source, p. , at a physical layer of a network protocol stack. The network address may, for example, include an Internet Protocol address associated with the source, or any other network layer address or the like associated with a network address of the source and/or useful for identifying or accessing the source. source at a network layer of a network protocol stack. The temporary address may, for example, include the name of a process running on the endpoint that is associated with the source, a directory location or path name, or any other transient identifier created by the endpoint to refer to an object or location on the endpoint, or a network-accessible resource. The temporary address may also include, or instead, an identifier for at least one of the endpoint's users, a device associated with the endpoint, a path associated with a computing object on the endpoint, a process running at the endpoint, an application at the endpoint, or any other address created or assigned by the endpoint or the like. More generally, information about an endpoint can change. Directory structures can be modified. Files can be moved or renamed. Processes can be stopped and restarted with different process IDs. By capturing such transient information at the time an event occurs, temporary address information can provide enhanced contextual insight into the event, as well as the event's relationship to other events and other entities in an enterprise network.
Locally augmented source identifiers can also allow the context of an event to be explicitly associated with other source identifier information, which can simplify the identification of associations or correlations between events being processed remotely from the endpoint. For example, there might be half a dozen sensors on an endpoint recording at a particular Internet Protocol address, without specifying whether each is a source of traffic from that IP address, a destination for network traffic from that source, a user of a resource at that IP address, and so on. Similarly, an event can have multiple IP addresses associated with it, without specifying how each IP address is related to the event. For example, a firewall may receive three different IP addresses for a network request, such as an IP address for an endpoint, an IP address for a service requested from the endpoint, and an IP address for an authentication system used in connection with application. . Each of these IP addresses can be successfully associated with an endpoint network request, but can have different relationships with another endpoint event or with a different endpoint communicating through the firewall. By precomputing this type of information at the endpoint and establishing the relationship of these various identifiers to an event, for example, by explicitly identifying the relationship of the source to the IP address and/or by specifying other address information associated with the source. , a threat management facility can more quickly and efficiently analyze events in the right context. Similarly, this type of extended source identification allows for more precise query targeting, analysis, and remediation.
For example, if an attack is detected from an IP address, the threat management function can more quickly and directly query what else on the enterprise network is associated with the IP address, what other devices or resources have a relationship with the IP address, etc. This allows identification not only of an IP address, but also of a particular user, a particular process, and the like, as well as a more accurate picture of what other network resources (for example, other connectivity pathways), users and resources are associated. that IP address. Similarly, this allows for specific associations at particular points in time. Thus, for example, an IP address may be explicitly associated with a particular MAC address, a particular user and/or a particular user at the time an event is logged, even though these relationships are transient in nature.
The general approach described above advantageously supports the joining of relevant information to an event log based on local contextual information at the point of origin, rather than requiring inferences about relationships to be made in a remote threat management installation where multiple logs of events from multiple sensors and endpoints. they are received. This allows for an accurate local join of the entire state of an event, including information such as a known user, a known process, a known network adapter (or other hardware), etc. In particular, this allows a capture of temporary and transient information that could not otherwise be clearly associated with an event when analyzing data, and in particular values that may change over time, such as an IP address, a user identifier, a MAC address, an application ID, a process ID, a file hash, etc. In one aspect, these temporary tags can be combined or associated with non-temporal information (eg, a hardware device identifier for an endpoint) to facilitate useful and accurate mappings during post-processing. Similarly, this supports enhanced charting and chart navigation by a human investigator, where events, entities, and relationships can be more accurately identified to support switching to different views or analysis as they occur. an investigation progresses. It will also be appreciated that some or all of this contextual information may be stored in local data loggers unless/until requested or needed by a central threat management facility.
In addition to augmenting the source identification information using the local context available to the endpoint at the time an event is detected or logged, an event can be further augmented with causally related events that are also available in the local context of the endpoint. final point. For example, an event can be augmented with an event graph using, for example, the techniques described in this document. This sparkline of events may, for example, include a relatively small local collection of causally related events. In one aspect, the mini-event graph may include one or more (causally) immediately preceding events, or one or more (causally) immediately following events. In another aspect, the mini-event graph may be a highly filtered event graph that includes one or two additional events, such as an identified root cause for the event, a final action in an event graph that includes the event, a known malicious action that follows, and causally related to the event, and so on. This can also, or instead, include condensed or summarized events, such as condensing an autosave or copy to a USB drive as a single, summarized event rather than any number of related event literals associated with a user action. or particular computer activity. .
As shown in step1708, the method1700may include creating a modified event record by adding the source identifier to the event record. This modified event log may include the sensor event log, along with any relevant event details, as well as an augmented source identifier, as described above, based on information available to a local security agent for the endpoint. . This source identifier can usefully identify a source of the event in a context of the sensor and/or endpoint.
As shown in step1710, the method1700may include storing the modified event log in a data logger at the endpoint. This allows local logging with any desired duration or level of granularity regardless of storage requirements or restrictions in a threat management facility that might receive the modified event log. So, for example, if a remote threat management installation determines that a particular modified event record is not immediately relevant or useful, the local data logger can retain a copy in case a request for information is later received. Additional Threat Management facility. The method1700may also include, or instead store a plurality of change event records in the data logger, such as a stream of change event records from multiple sensors at the endpoint, for example, according to any logging policy, storage restrictions or other considerations. In general, this allows the data logger (eg, under the control of the local security agent) to respond to queries from resources external to the terminal for additional data stored in the data logger.
As shown in step1712, the method1700may include transmission of the modified event log to a threat management facility.
As shown in step1714, the method1700This may include the processing of change event logs, along with any other available information, at a remote threat management facility, such as any of the threat management facilities described in this document. In general, the relationship of one of the change event records to one or more change event records stored in the threat management facility can be determined based on source identifiers that have been augmented with contextual data as described in present before transmitting them to the threat management facility. . A variety of useful tools for eg forensics, threat investigation, policy enforcement, etc. can be performed on a threat management installation based on these modified event logs.
For example, the method1700may include providing a user interface from the threat management facility for navigating a chain of events based on relationships between a plurality of source identifiers in a plurality of change event logs received at the threat management facility. This may include a display of associations between source identifiers that allows a user to navigate from one source to another based on overlapping or related source identifiers in the augmented event log data. Similarly, the user interface can be configured to support navigation through a chain of events based on relationships between a plurality of source identifiers in a plurality of modified event logs, for example by displaying events on a graph, along with with associations between the nodes being identified. or inferred based on modified event logs. The method1700may also or instead include the creation of a graph causally associating two or more events based on a plurality of source identifiers in a plurality of modified event logs received at the threat management facility, which may be displayed on the user interface or otherwise used to process or parse collections of events based on changed event logs.
In another aspect, this may include processing a stream of modified event logs to assess security threats to an enterprise network, for example, using any of the various techniques described in this document, to detect threats and trace causal chains. of events to the root cause. Similarly, processing may include processing a stream of modified event logs in the threat management facility to assess a security state of the endpoint, for example by ensuring that particular events are associated with users or processes. suitable, and so on. In one aspect, the event sequences may be associated with one another, eg, by determining a relationship of a source identifier for an event to one or more source identifiers in modified event logs received at the threat management facility. In another aspect, the processing of the modified event log stream in the threat management facility may include deduplication of one or more event logs based on a reconciliation of source identifiers. Other techniques for processing modified event log streams may also be used, or instead of them.
In one aspect, a system is described herein that includes a local security agent running on an endpoint and configured to receive data characterizing an event from a sensor on the endpoint; to generate an event log in response to the event; to determine a sensor context that includes a physical address associated with an event source and a temporary address assigned by the endpoint to the event source; adding a source identifier to the event log that identifies the source of the event, including the sensor context, thus providing a modified event log that includes the event log and the source identifier; and transmitting the modified event log to a remote resource. The local security agent may also, or instead, process events and augment event logs to create modified event logs as described more generally in this document.
The system may also include a threat management function like any of the threat management functions described in this document. The threat management facility can, for example, be configured to receive a stream of change event logs from a plurality of endpoints in an enterprise network including the endpoint, and to assess security threats to the enterprise network based on of the modified event log stream. , or to otherwise process a stream of modified event records as described in this document.
user interface1802can be any of the user interfaces described in this document and can be hosted, for example, on a host1803for a threat management facility1806or other network resource for use in managing network security and investigating potential security threats, and presented on any suitable end-user device. user interface1802it may be a web-based interface or other proprietary or similar interface suitable for access and use by a technician responsible for network security matters for a managed domain such as an enterprise network or the like. As described here, the user interface1802can be presented by threat management installation1807, for example, on a screen for a user such as an administrator, technician, or other security personnel or user. user interface1802can generally provide interactive access to the threat management function1806, the data lake1807, and any associated sources of security data as described in this document. The Threat Management Facility1806can be configured to broadcast a link to the user interface1802in a message to the user, such as an email message, a text message, or a pop-up window in a threat management installation interface.
user interface1802It can be associated with an investigation container, such as any of the investigation containers described in this document, which can be configured to support augmented threat investigation as described in this document. For example, the user interface1802may include a control to request additional event data captured by a data logger from one of the plurality of computing instances for a time window prior to the creation of the research container. user interface1802it may also include, or instead, a control for adjusting a filter applied by one of the plurality of compute instances to a local event stream when local security events are selected to communicate in the event stream with the data lake. user interface1802you can also, or instead, include a tool to query the data lake.
an end point1804, which can be any of the endpoints or other compute instances described in this document, can include a local security agent like any of the local security agents described in this document. The local security agent can detect an event (or more specifically, receive a detection from a sensor on the endpoint1804) and send a local threat indication to the host1803, either as a single event or in an event vector containing other contemporaneous event data. This can, for example, include precomputed data at the endpoint1804such as a severity score, a threat rating (such as a MITRE rating or similar), or other contextual information related to the detected event, any of which may be used by the host1803to display within the user interface1802, or as a basis for further inquiry or investigation to identify other information potentially relevant to an investigation of a potential threat. Although not shown, it will be understood that the end point1804can be attached to an enterprise network that includes, or is associated with, the threat management facility1806and other components such as the host1803for user interface1802and the data lake1807.
Although it is represented as a single endpoint1804, it will be understood that the enterprise network may more generally include a plurality of computing instances. For example, the plurality of compute instances may include at least one network device, such as a switch, router, wireless access point, gateway, firewall, etc. In another aspect, the plurality of computing instances can include at least one virtual computing device hosted on a virtualization platform such as a virtual desktop infrastructure or other cloud computing resource or the like.
In general, security events provided by these compute instances may include asynchronous data that is collected locally and transmitted incrementally as data is acquired by the compute instances, for example, at unknown, unscheduled, and/or not synchronized from the various compute instances. . In another aspect, the security events may include batch data from one or more of the plurality of computing instances. In one aspect, batch data may be collected and transmitted at scheduled intervals. In another aspect, batch data may be reported asynchronously, for example, in response to local triggering events, such as detection of a local threat, a change in computing activity, or an accumulation of batch data reaching a threshold. default duration, size, or activity. guy. In one aspect, the security events may include threat detection data from a local security agent running on one of the plurality of compute instances and/or from multiple local security agents on multiple compute instances.
A threat management facility1806can provide relevant information for threat detection and investigation. For example, the threat management facility can receive event data from the endpoint1804and other endpoints within an enterprise network. In one aspect, some of this data may be used in raw form by the host.1803to display in the user interface1802. In another aspect, the event data may be processed by the threat management facility.1806to generate a contextual threat score for the event detected by the endpoint1804based on, for example, other events or context for the endpoint, other events within an enterprise network, or any other available data. As described in this document, in one aspect, the threat management facility1806can be configured by computer executable code or the like to perform the step of updating a composite threat score based on the number of security data sources, since the event stream is received asynchronously from the number of security data sources. security. In another aspect, the threat management facility1806can be configured via computer executable code or the like to perform the step of automatically creating an investigation container for interactive investigation of security risks when the composite threat score reaches a predetermined threshold for starting an investigation.
As described in this document, the threat management function1806It can be configured to support augmented threat management in a number of ways. For example, the threat management facility1806can be configured to calculate a composite threat score for a compute instance by mapping the security events in the event stream to an attack matrix that lists the malware strategies in a first dimension and the malware techniques for each of the strategies of malware in a second dimension. Based on this mapping, the threat management function1806You can calculate the composite threat score based on a pattern of walking the attack matrix through a timeline of security events. The Threat Management Facility1806it can also, or instead, be configured to calculate the composite threat score by applying a machine learning algorithm to the walk pattern of the attack matrix to determine the threat probability. In another aspect, the composite threat score may also include, or instead, two or more scores based on two or more of the number of security data sources. In this case, the default threshold for launching an investigation container (or creating an alert or the like) can be based on an aggregate threshold for the two or more scores, or a separate threshold for each of the two or more scores. In another aspect, the threat management facility1806It can be configured to review the Composite Threat Score downward when a false positive is identified and, in response, to automatically close the Investigation Container when the Composite Threat Score reaches a predetermined second threshold to end the investigation. That is, when objective evidence indicates a reduced risk posture, an automatically created research container can be removed or deleted without the need for additional administrative intervention.
a data lake1807, like any of the data lakes described herein, can generally receive an event stream that includes events from a plurality of security data sources, such as any of the events, event streams, and data sources described herein. . In one aspect, this may include data sources within the enterprise network, such as security events from one or more sensors in the plurality of computing instances attached to the enterprise network. It is useful to include other types of events from internal sources in the event stream for the enterprise network. For example, the threat management facility1806and/or host1803can be equipped with a monitor for a query interface to the data lake1807. As described here, the pattern of queries and response activities from an administrative console can provide useful information for detecting and remediating threats on the enterprise network. And as such, the query interface monitor can provide a useful source of security data for the event stream.
the data lake1807may also, or instead, receive data from data sources external to, but associated with, the enterprise network, such as data from cloud resources from a cloud service1808support the plurality of computing instances coupled to the business network. For example, the cloud service1808may include one or more of a web application, a cloud storage service, an email application and an authentication service, a zero-trust network access resource, a cloud computing service, and a platform of virtualization. In another aspect, the cloud service1808may include a network monitor that runs on a third-party firewall and is securely coupled to the threat management installation1806. This latter embodiment can be useful, for example, when a firewall or other third-party component can locally log useful information for threat analysis, but does not provide a secure interface to access those local logs. To address this security issue, a network monitor can be installed in the firewall to provide a secure communications interface for remote access to the local security event log. The network monitor can automatically transmit data at suitable intervals, or can be configured to respond to remote queries for log data, or some combination of these.
the data lake1807may also or instead receive data from external data sources independent of the enterprise network, such as contextual data for the activity of multiple compute instances of a third-party service1810. Useful contextual data may, for example, include geolocation data from a third-party service that provides a geolocation for a computing instance, resource, threat sample, or the like. The geolocation data may, for example, be based on active geolocation using Global Positioning System data, cellular network triangulation data, WiFi network signal strength analysis, or the like, or passive geolocation using, for example , Internet Protocol (IP) address lookups for location data. mapped to an IP address for a device. In another aspect, the useful contextual data may include threat detection data from third party threat management services. For example, this may include threat classification, threat identification, signature analysis, etc., provided by one or more remote resources based on data available in the event stream or data lake, e.g. example, of the plurality of computing instances.
As usually indicated by an arrow1806, the threat management function1806and the end point1806may be connected to a partner business network and may share data and control information for security management of the partner business network. It will be understood that, while illustrated as a separate entity, the threat management function1806can be the same as the host1803for research user interface1802, or placed with the host1803and the data lake1807, or otherwise engaged in a communication relationship with these components, as generally indicated by a schematic1805imprisoning these entities. In another aspect, the host1803can be operated by a third party independently of a threat management facility1806for a company network, so that a company technician can log in and use the user interface1802or a third-party security service can use the host1803and user interface1802to manage security on an outsourced basis for an enterprise network managed by the threat management facility1806.
In one aspect, the threat management facility1806You may use a data lake, such as any of the data lakes or similar event stream data repositories as described in this document, to store information related to events that occur within a managed estate, any of which may be useful for investigating potential threats as described in this document.
cloud service1808it can be any cloud service, application platform, data facility or the like that provides user access to data and cloud services. In one aspect, the cloud service1808it may be a zero-trust network access resource that provides secure access to applications and the like for users associated with a corporate network. In another aspect, the cloud service1808may include a cloud data warehouse or other remote data storage facility. cloud service1808It may also include, or instead of, a cloud computing platform, a software-as-a-service (SaaS) solution, or other cloud-based service or a combination of services. In one aspect, the cloud service1808it may include an authentication service, identity management platform, or the like that is used to identify and authenticate authorized users to various network resources. In another aspect, the cloud service1808may include a network monitor associated with a third party network infrastructure or the like. For example, a firewall or other network hardware may be equipped with a network monitor configured to obtain local activity logs and report them to other entities over a secure communication channel.
cloud service1808can provide a variety of information useful for investigating potential threats associated with the endpoint1804. For example, the cloud service1808may provide application usage statistics, file or data transfer activity, logins and login attempts, etc. cloud service1808it can also or instead report administrative activity such as new accounts, authentication histories, etc. In one aspect, the cloud service1808it can also, or instead, expose underlying data that can be useful in assessing threats. For example, when a user's email account is hosted by a cloud service1808, email traffic (incoming, outgoing, or both) can be analyzed to investigate, for example, potential sources and targets of malicious activity. Other platforms, such as e-commerce accounts, social media accounts, and the like, can also usefully be scanned for relationships to potential malicious activity in the absence of privacy or similar restrictions. While a single cloud service1808is shown, it will be understood that any number of cloud resources1808can be used by an endpoint1804, and can provide data to a host1803as generally described in this document.
The third party service1810it can be any third-party security service useful for identifying and/or investigating potential threats. For example, this may include a classification service such as MITRE ATT&CK™ that provides a framework for categorizing deployed security threats based on patterns of malicious tactics and techniques. As another example, this may include a dictionary of threat signatures or a real-time database of active threats. More generally, any third-party resource to identify, characterize, or respond to various threats may be used as a third-party resource as described in this document. In one aspect, the MITRE data can be used to provide categorizations and human-readable descriptions for potentially malicious activity detected on the endpoint.1804.
Accordingly, in one aspect described herein is a system that includes a plurality of computing instances coupled to an enterprise network, a data lake that stores an event stream from various security data sources, including the event stream , and a threat management facility. The event stream may, for example, include security events from one or more sensors in the plurality of coupled compute instances on the enterprise network, cloud resource data from a cloud service that supports the plurality of coupled compute instances to business network and contextual data. for the activity by the plurality of computing instances of a third-party service. The threat management function can be configured to perform the steps of updating a composite threat score based on the number of security data sources as the event stream is received asynchronously from the number of security data sources. security and automatically create an investigation container for the interaction. security risk investigation when the composite threat score reaches a predetermined threshold to initiate an investigation.
A compute instance1902, like any of the endpoints or other compute instances described in this document, can run a local security agent that is instrumented to store local events in a data logger and publish the data at any suitable time. The events can be any of the events, detections, and the like described in this document, and the data logger can include any of the data loggers described in this document. As described here, the compute instance1902may be associated with an enterprise network managed by a threat management facility.
An ingest service1904can run on compute instance1902or externally from the compute instance1902, and can be implemented, for example, using a microservices architecture to support flexibility and scalability. In general, the intake service1904can receive (or request/retrieve) events1906of the computing instance1902, format the events1906as needed, and publish the events1906to a stream of events1908. An "event" shall be understood to include any of the events described in this document, including events detected by a compute instance, events stored in a data store, threat detections, and other indications of compromise derived from such events, the events published in the event. stream, augmented events that are enriched or otherwise processed in the event stream, and events stored in the similar data. Although the format and content of such events can vary significantly, all such events are intended to be included within the scope of the term "event" unless a different meaning is explicitly provided or is clear from the context.
The flow of events1908it may include any infrastructure, including hardware and software, suitable for managing and processing a stream of events. This may include, for example, any of a number of commercially available event processing technologies, data flow management systems and the like, as well as proprietary event processing platforms with similar capabilities and/or combinations of these suitable for your application. use in an event. -Information system driven. In general, the flow of events1908It can support event display, event storage, event-based processing, complex event processing, etc.
The system1900may include one or more enrichment workers1910, which can monitor events1906in the flow of events1908, and enrich these events1906in any suitable way. This may include formatting the events1906according to one or more schemes for the flow of events1908, normalizing the facts1906for a coherent representation of events1906from different sources, filtering the events1906to remove duplicate or unnecessary data, increasing events1906to provide additional data, map event data to known threat types, process events1906to generate additional events1906for posting to the event stream1908, Etc.
While the enrichment workers1910can usefully facilitate data normalization, filtering, deduplication, augmentation, and the like based on internal rules for an enterprise network, system1900you can also employ one or more enrichment brokers1912for enriching events1906in the flow of events1908using external resources. This architecture advantageously provides a bridge to a wide range of external resources useful for detecting, identifying, and remediating threats to an enterprise network. the enrichment broker1912can usually monitor events1906in the flow of events1908, request data from one or more external resources based on event data in events1906and provide data and analytics responsive to external resources. The response data can be used to enrich the corresponding events.1906, for example, by writing directly to one or more fields in an existing event schema, or to create new events1906for posting to the event stream1908, or some combination of these.
For example, in one aspect, the enrichment corridor1912you can access a geolocation service1914remote from the company network and provided by a third party in a manner accessible to the enrichment broker1912through a data network. In one aspect, the geolocation service1914may respond to a request containing an IP address by providing any geolocation data available for that IP address, including, by way of example and without limitation, a city, state, country, country code, postal code, latitude, a longitude, and any other location information associated with the IP address. In another aspect, additional identifying information may be provided to the geolocation service.1914such as a MAC address or other machine identification information, where such information may be used by the geolocation service1914to improve the accuracy of a device's location.
As another example, the enrichment broker1912can access a security resource, such as a threat intelligence service1916. In this context, the enrichment broker1912may provide a hash, metadata, code sample, or the like for a computing object such as a file or the like contained in or associated with one of the events1906. The threat information service1916may respond with reputation data such as a reputation score associated with the computer object, a detection name such as a malware type and/or family for the computer object, and a reputation band that categorizes the computer object or the reputation score associated with the compute object using discrete bins or risk ranges. For example, reputation bands might include a category for malware, possibly an unwanted app, an unknown app, or a known good app. A threat information service1916may also or instead provide other information based on an analysis of a sample computer object. For example, the threat information service1916can provide remediation strategies, source identification, cryptographic authentication, or any other data useful for analyzing or evaluating risks associated with computer objects and related events1906in the flow of events1908.
Other services can also be used or instead to enrich the data in the event stream1908. For example, this may include third-party security services that identify or characterize risks, machine learning services configured to identify confidential or sensitive information (such as addresses, names, account numbers, etc.), key management systems, etc. . In another aspect, the service may include a search service that parses, for example, file hashes, URLs, IP addresses, or other contextual data or metadata associated with an event. In another aspect, services such as a sandbox or other analysis tool may be provided, for example, to run code samples in a secure environment and provide any resulting analysis.
The system1900may include a data lake ingestion service1918which is generally used to persist the enriched and augmented data in the event stream1908to the data lake1920in a way that is consistent and searchable. This may include storing the events1908using any suitable schema, which may be managed in a schema registry or the like. The data lake ingestion service1918you can also augment the data further, for example by adding a customer identifier so that different events1906associated with different customers can be individually tracked, managed and analyzed in the data lake1920. the data lake1920it can include any of the data lakes described in this document and can be accessed through a hosted user interface, for example, by an enterprise network threat management installation.
In general, a system2000can include events2002in a flow of events2004, like any of the events and event streams described in this document. The events2002can be processed by a data lake ingestion service2006, like any of the ingestion services described in this document, for persistent storage in a data lake2008, like any of the data lakes described in this document, for further search and analysis.
A compute instance2010associated with an enterprise network, like any of the compute instances described in this document, may use a cloud service2012that supports users associated with the enterprise network. For example, a user on the compute instance2010can login to a user account2016in the cloud service2012, for example, when authenticating to the cloud service2012with user credentials, authentication factors or other data that identifies the user to the cloud service2012.
cloud service2012may include any resource, remote computing service, or the like that is external to a business network and accessible to users associated with the business network through a data network or interconnection of networks, such as any combination of public or private data networks described in this document. For example, the cloud service2012can include an application platform such as Office365provided by Microsoft Corporation, or any other similar network-accessible suite of applications for, for example, email, document processing, presentations, spreadsheets, databases, etc. cloud service2012it can also or instead include a cloud storage service that includes file storage resources, databases, backup utilities, etc. In another aspect, the cloud service2012It can include a cloud computing platform that provides computing resources to deploy applications, virtual servers, etc. In another aspect, the cloud service2012may include a virtualization platform to instantiate virtual compute instances for the enterprise network. Thus, by way of non-limiting examples, the cloud service2012may include any combination of an email application, cloud storage service, cloud computing service, virtualization platform, authentication service, identity management platform, web application, resource of access to the zero-trust network, a social networking platform, a multi-user database, a virtual reality or augmented reality resource, etc.
In one aspect, a non-cloud resource can be configured to behave like a cloud resource for event monitoring purposes. For example, a corporate network might include a third-party firewall that logs traffic data locally. While it may be advantageous to publish firewall log data in the event stream2004, the firewall may not include capabilities for secure external data communications. In this case, a network monitor can be installed on the firewall to communicate securely with external resources on the one hand, and to query and report data from the firewall data log on the other. So, in one aspect, the cloud service2012It may include a network monitor running on a device, such as a third-party firewall that is configured to provide a remote and secure interface for events detected locally on the device.
Once a user has accessed the cloud service2012, the cloud service2012You can internally generate data from cloud resources of interest in threat investigation and management. For example, this may include authentication to the cloud service2012, or using an identity management platform to authenticate to some other service. This may also include, or instead, administrative events in the cloud service.2012such as user account changes, account permissions or preferences, notification rules, forwarding rules, etc. In another aspect, this may include application activity on the cloud service2012started from compute instance2010such as opening, editing, sending, creating, or retrieving files, or other activities such as file searches, file sharing, and the like. For other tools, such as an email server, this may include mail reading activity, as well as replies, deletions, attachment handling, etc. It shall be understood that when monitoring data from cloud resources directly in or from the cloud service2012, it is possible to make detections that are not possible by monitoring user activity directly. For example, if a user receives a large number of suspicious messages and/or attachments, or if a group of files is transmitted to an unknown address, this can be detected without the user accessing the corresponding email account.
To support the acquisition of data from cloud resources in this way, an administrator, for example, an administrator of a threat management facility for an enterprise network, can log in to the cloud service.2012from a second computing instance2014with an administrative or master account for the enterprise network. Use of administrative application programming interfaces (APIs) for the cloud service2012, admin can set up a streaming service2020to monitor user account activity and publish any corresponding events2002to event flow2004. It should be noted that the streaming service2020illustrated as a component of the cloud service2012. Some cloud services have built-in reporting or streaming services that can be usefully configured to publish events.2002to event flow2004as described in this document. In another aspect, the cloud service2012may provide a programming environment that enables an administrator or other security personnel to create and deploy a service, process, application, or the like within the computing environment of the cloud service2012to perform these functions. Although not illustrated in
By way of non-limiting example, in a Microsoft Office365environment, an administrator can create a master application and add the application to the cloud service resources2012. The credentials for this main application can be saved in the threat management facility to facilitate administrative access to the cloud environment through the administrative APIs.2018that support, for example, event graphs, identity management, authentication, application platform features and configurations, etc. Office auditing capabilities365can then be enabled via the administrative APIs2018and an app for a streaming service2020it can be deployed in the cloud environment using the credentials of the master application. When running on the cloud service2012the streaming service2020can ingest activity logs (for example, for all users associated with Office365client) of the Office365functions for auditing and processing these activity logs for publication in the event stream2004. For example, cloud resource data provided by the cloud service2012eg Office audit data365, can be filtered (for example, to remove data that is not relevant for threat detection and management), mapped to a schema used by the event stream2004and/or data lake2008and transformed so that event descriptions, threat detections, and the like provided from the cloud service2012the environment precisely matches the corresponding elements in the schema used by the data lake2008. The streaming service2020can provide a list of known users associated with a company. In one aspect, the streaming service2020You can add user information to events, for example, to identify a user (by an email address, user principal name, or other identifier) associated with each event. In another aspect, for example, where the cloud service2012and/or data lake2008are implemented as multi-tenant databases, a customer identifier can be added to events2002to allow identification of the customer and an associated business network within the data in the event stream2004and the data lake2008.
In this way, cloud resource data for a complex enterprise system, including data such as cloud computing statistics, network utilization, email server activities, identity management requests, and the like, can be monitored consistently. useful way in an event stream.2004independent of endpoint reporting. The events2002posted to event stream2004from the cloud service2012it can also be augmented or enriched using any of the techniques described in this document, such as the techniques described with reference to
In the following description, the event flow, security events, sensors, compute instances, enterprise network, compute instances, data lake, query interface, administrative consoles, threat management facilities and other components may be any of the corresponding components described herein.
As shown in step2102, the method2100You can start by storing an attack matrix like the attack matrix described with reference to
As shown in step2104, the method2100may include receiving an event stream that includes a plurality of security events from a plurality of sensors in a plurality of computing instances in an enterprise network.
As shown in step2106, the method2100it may include event stream storage, for example, in a data lake that provides a query interface to one or more administrative consoles of a threat management facility. The event stream may also, or instead, be stored on any suitable transient or persistent storage medium for the purposes described herein.
As shown in step2108, the method2100It may include the identification of a pattern crossing the attack matrix indicative of a malware threat on one of the compute instances based on two or more of the security events in the data lake. This may, for example, include two or more security events detected on a single compute instance, or two or more security events detected on multiple compute instances in the enterprise network, or some combination of these.
In general, a traversal pattern of the attack matrix may correspond to a chronological display of particular techniques detected in the computing instance. When this pattern reflects a sequence of techniques typically used to maliciously access or control a computing device, the pattern may indicate a corresponding malware threat on the device. For example, the pattern may be indicative of a malware element on the compute instance that performs a series of specific tasks to achieve a malicious goal. The pattern may also be indicative of a breach of the enterprise network, such as a breach that exposes control of a compute instance that could lead to, but has not yet resulted, a compromise of data and/or compute instances in the enterprise. grid. As a significant advantage, following a chronological sequence in which known strategies and tactics are implemented, more specifically by assigning security events to the attack matrix, can allow the identification of developing malware attacks based on security strategies. recognizable attack even in the absence of specific malware signatures. or code of behavior.
A variety of useful techniques can be employed to identify crossover patterns indicative of malicious activity. For example, a machine algorithm can be trained to identify malicious patterns based on a training data set that associates one or more traversal patterns with one or more known malware instances, or more generally, with a data set of training tagged with known benign and malicious. cross patterns. In this way, a machine learning detection model can be implemented that receives traversal patterns, for example, as ordered sequences of techniques in the attack matrix, and generates a detection result, such as probability of malware presence or probability of from each of them. or more specific types of attacks. In another aspect, pattern identification may include the application of one or more rules that specify a traversal order within the attack matrix associated with the malicious activity. More generally, any technique or combination of techniques useful for identifying malicious activity based on an ordered sequence of specific techniques can be used in combination with the attack matrix to support detection or risk scoring as described in this document.
It will be understood that, while a chronological walkthrough pattern of the attack matrix can be usefully employed in this context, the method2100may also, or instead, include identification of a coverage pattern from the attack matrix. That is, a pattern within the attack matrix based on two or more security events, for example data lake security events, may be indicative of a malware threat on one or more of the compute instances. This can include a scope of the pattern, for example, where security events appear in the attack matrix, as well as a frequency of the pattern, for example, how many occurrences of one or more security events appear in a cell of the attack matrix. attack (or multiple cells in the attack matrix). Therefore, it will be understood that while a chronological pattern of events is emphasized here as indicative of potential malware, a spatial pattern and/or numerical pattern of events may also be used within, or in their attack matrix. place, without departing from the scope of this document. disclosure, and all such patterns are intended to be included in a pattern as that term is used in this document, unless a more specific meaning is explicitly provided or is clear from the context.
As shown in step2110, the method2100This may include creating a first threat score for one of the compute instances based on the traversal pattern of the attack matrix. In one aspect, scoring can be performed directly, for example, by converting an output from a machine learning model to a score using any suitable scaling, weighting, quantification, categorization, or the like. In another aspect, the multiple outputs of a detection model, such as the machine learning model, can be weighted based on the corresponding risks of different types of potential threats. So, for example, the threat score may include a weighted sum of probabilities of different types of threats, as identified by a machine learning model. In another aspect, for example, when using a rule-based technique, the threat score can be based on a predicted malware type using one or more detection rules, or as a weighted sum of probabilities for different indicated threat types. by the rules. based technique.
In general, the pattern within the attack matrix used for threat qualification, whether it is a spatial pattern, a numerical frequency pattern, or a chronological pattern, can include events from one or more endpoints within the enterprise network. It will also be understood that a window for adding events within the pattern may be fixed, variable, or some combination thereof. For example, in one aspect, events can be simply grouped by date and used for pattern detection based on occurrence in a single day. In another aspect, a pattern for each individual endpoint may include two or more days, for example, when a pattern is related to detecting complex patterns or slowly and incrementally deployed malware. In another aspect, the window for pattern detection may be variable, for example, when one or more events are known to be potentially related to an attack with a long implementation time frame. In the latter case, a standard window of, say, one day can be used to detect patterns, while the window for accumulating events into a pattern can be extended when several initial events indicate that additional time may be useful for certain types. detection. .
As shown in step2112, the method2100may include monitoring other activity to aid in threat detection. For example, this may include monitoring a usage of the query interface to the data lake, eg, by tracking a plurality of queries to the data lake from one or more administrative consoles. Query activity can be used, for example, as described in this document, to detect query patterns indicative of a threat, or to create detection rules based on a correlation of query activity with a threat that is detected in patterns function within the attack matrix. It will be understood that another activity may also be used or in place of it to assist in detection as described herein. For example, in one aspect, pattern-based detection within the attack matrix can be combined with individual detections from an endpoint, eg, where individual detections are used to identify the location of a general threat detected with the attack matrix. attack, or where patterns within the attack matrix are used to further verify or investigate individual endpoint detections of uncertain conviction.
As shown in step2114, the method2100This may include creating a second threat score for one of the compute instances based on the use of the query interface. In general, this may include the use of any of the techniques described in this document to assess the probability of risk based on a pattern of queries to the data lake, together with any other available contextual information where such information helps to evaluate the activity. malicious within the corporate network. For example, this may include creating the second threat score based on a usage pattern of a series of queries stored for the query interface, or creating the second threat score based on a change to one of several queries. stored for the query interface. . For example, when a user requests data in a particular order, or makes a specific change to a pre-existing stored query, for example to expand a search or focus on a particular data type or source, this may indicate an expanding threat. . Research that suggests a higher potential risk or a higher likelihood of compromise within the business network.
As shown in step2116, the method2100may include displaying the first threat score and the second threat score on a screen of one of the administrative consoles when at least one of the first threat score and the second threat score reach a predetermined threshold. This may include, for example, displaying each of the threat scores separately, or displaying a composite threat score based on the first threat score, the second threat score, and/or one or more scores, such as any of the following. threat scores described in this document. . In one aspect, displaying the first threat score and the second threat score on the screen may include displaying an alert on the screen that provides a link to additional data related to the first threat score and the second threat score, as descriptions of the supporting query behavior. , attack matrix crossover and the like, as well as other objective risk data related to the IT instance and/or the business network.
In one aspect, the predetermined threshold for displaying risk data may include a threat score threshold such as a specific numerical value that is compared to the threat score(s). In another aspect, the predetermined threshold may be a percentile threshold for various compute instances on the enterprise network. For example, if the threat score(s) is in the highest risk percentile group, the threat score(s) may be displayed. The percentile threshold can be a fixed percentile, for example, the top ten percent, or a variable percentile, for example, the top five percent when there are several very high risks, or the top ten, fifteen, or twenty percent, when there are mainly or exclusively low objectively assessed risks throughout the business network.
As shown in step2118, the threat score(s) can be used to perform threat detection within the enterprise network. That is, instead of, or in addition to displaying a threat score, the method2100It may include detecting whether a threat is present on the enterprise network based on the threat score(s). If a threat (or potential threat) is detected in step2118For example, when the threat score(s) reach a predetermined second threshold for risk detection, the method2100you can proceed to step2120where a corrective action can be initiated, and/or a step taken2122where a threat detection rule is created. Otherwise, the method2100you can proceed to step2104where the process can continue to receive events, store events, score events, etc.
As shown in step2120, in response to the detection of a threat, the method2100may include initiating one or more corrective actions. This may include any of the corrective actions described in this document. By way of example and not limitation, this may include adjusting event filtering on one or more of the Compute Instances, updating security software, quarantining the Compute Instances, running malware detection software on one or more of the Compute Instances, compute instances, isolating one or more of the compute instances, isolating potentially malicious code, rebooting one or more of the compute instances, restricting network traffic for one or more of the compute instances, etc. .
As shown in step2122, in response to the detection of a threat, the method2100may include the creation of a threat detection rule associated with any corresponding activity. In general, scoring techniques based on the attack matrix and the use of queries can be used independently as risk assessment tools. However, these scoring techniques can also be used to advantage in combination to detect new zero-day and similar threats. For example, a malware threat can be detected based on the use of data lake queries, as described generally in this document. In response to such usage indicating a malware threat, the method2100It may include examining a crossover pattern in the attack matrix and creating a new rule for threat detection based on the crossover pattern associated with the query activity. This may include a traversal pattern concurrent with query activity, immediately prior to query activity, or prior to query activity for a predetermined period of time, which can range from seconds to days, depending on the nature of the threat correspondent. It will also be understood that in this context, the path pattern may be filtered or otherwise processed to improve detection quality. For example, when multiple query strings are associated with multiple instances of malware, the traversal pattern preceding each query string can be compared to eliminate extraneous events/detections.
It will be understood that while a particular order of steps is indicated in
In accordance with the foregoing, disclosed herein is a system that includes a data lake that stores an event stream that includes security events from one or more sensors on computing instances in an enterprise network, an administrative console configured to run queries against the data lake, a plurality of queries stored in a database for execution against the flow of events in the administrative console, the plurality of queries configured to investigate security issues within the enterprise network based on the flow of events, a query monitoring agent configured to monitor the use of multiple queries in the administrative console, an attack matrix stored in the database, the attack matrix listing malware strategies in a first dimension and malware techniques for each of the malware strategies in a second dimension, and a threat management function. The threat management function can be configured, for example, via computer executable code, to create a first threat score for a compute instance associated with the enterprise network based on a pattern of attack matrix traversal by a series of security events at the event. flow and to create a second threat score for the compute instance based on the use of the query interface, the threat management function was further configured to present the first threat score and the second threat score for the compute instance in the administrative console.
In one aspect, the query monitoring agent may be configured to determine a usage history based on the usage of the plurality of queries and to initiate an action by the threat management function based on the usage history. In another aspect, the threat management feature can be configured to monitor post-query activity from the administrative console and to detect malicious activity based on usage of the query interface and post-query activity. The threat management facility can also be configured to launch a research container, such as any of the research containers described in this document, when at least one of the first threat score and the second threat score reach a predetermined threshold.
By way of non-limiting example, the attack matrix can use categories for malware strategies and techniques such as those defined in the MITRE ATT&CK coverage map. This coverage map lists tactics such as initial access, execution, persistence, privilege escalation, defense evasion, credential access, discovery, lateral movement, collection, command and control, exfiltration, and impact. For each of these strategies, the MITRE coverage map describes techniques (and "sub-techniques" under the MITRE rubric) suitable for detection using various threat detection tools. For example, the initial access technique in this coverage map may include techniques such as unauthorized compromise, public application exploitation, external remote services, hardware additions, replication via removable media, file attachment of phishing, link phishing, phishing through the service, supply chain compromise, trust relationship, etc. More generally, any matrix, mapping, table, or other arrangement of these various techniques suitable for tracking a pattern of individual exploits that suggest malicious activity can be used as an attack matrix as described in this document. Using industry standard tactic categories, such as those in the MITRE coverage map, can advantageously facilitate integration with other pre-existing events, detection types, detection rules, and the like.
A pattern of travel2202illustrated within the attack matrix2200using a series of circular techniques coupled by arrows. This pattern of travel2202can be detected using any of the threat detection techniques described in this document, and each detected event can be assigned to a technique within the attack matrix2200, producing a run pattern that proceeds chronologically or sequentially from technique to technique within the attack matrix2200. As described later in this document, this traversal pattern2202can provide a useful foundation for threat detection using the techniques described in this document.
As shown in step2302, the method2300may include receiving an event stream, such as any of the event streams described in this document. In general, this may include a plurality of data objects received in an asynchronous stream of security events from the enterprise network. This may also include, or instead, a combination of batch uploads that include groups of security events being batched together for publication to the event stream, and continuous stream uploads of individual security events from one or more over the plurality of computing instances, eg when events are detected. In one aspect, the event stream may include one or more events received from data loggers in computing instances on the enterprise network.
As shown in step2304, the method2300may include the storage of a data lake based on the flow of events. This may, for example, include storing a plurality of data objects representing a plurality of security events, such as any of the events described in this document, received in the event stream, for example, from one or more loggers. of data in a plurality of computing instances in an enterprise network. This may include any of the data loggers, compute instances, and enterprise networks described in this document. In one aspect, this may include filtering each of the plurality of security events in the event stream with a deduplication search before adding them to the data lake to avoid a buildup of redundant data in the data lake. In another aspect, this may include storing the plurality of security events in the data lake as one or more data objects, each augmented with a description organized according to a schema, such as a schema used by the data to structure events. data received in the event stream. .
As shown in step2306, the method2300may include querying the data lake, for example, using any of the user interfaces and/or query interfaces described in this document.
As shown in step2308, the method2300may include threat detection based on data from the data lake. For example, this may include detection of a number of threats on the enterprise network in a threat management facility by querying the data lake, which may be initiated, for example, from an administrative console of the threat management facility . This may also include, or instead, a detection that uses any of the techniques described in this document, such as monitoring query activity, tracking cross-sectional patterns in an attack matrix, etc.
As shown in step2310, the method2300it may include the identification of one or more security event sequences related to a threat. In one aspect, this may include manual review of sequences of events that precede an identified threat. In another aspect, this may include automatically identifying a characteristic sequence of security events in the event stream preceding each detected threat, based on data lake queries, using one or more reconnaissance tools. of patterns to analyze the plurality of data objects in the data lake The characteristic sequence of security events may include a similar sequence of events from a plurality of compute instances in the enterprise network or a similar sequence of events from a single compute instance on the business network.
As shown in step2312, the method2300may include the creation of a folding rule in response to threat detection. For example, this may include, in response to detecting the number of threats and identifying the characteristic sequence of security events, automatically creating a folding rule to combine the characteristic sequence of security events into one event. added security in the data lake. The folding rule can also, or instead, specify one or more rules to augment the aggregated security event with metadata describing the characteristic sequence of events. For example, the folding rule can capture a first occurrence of a repeating event, a number of times a similar or identical event occurs, a last occurrence, a time interval in which multiple events occur, and so on. More generally, the metadata may characterize a series of events in the security event stream, a frequency of events in the security event stream, and a first and last security event in the security event stream.
By way of non-limiting example, when a compute instance transmits a large file to a remote storage facility as a number of individual packets, the folding rule may simply indicate the time the communication started, the destination, and the number of packets. packages (or the size of the file). As another example, when a user logs in multiple times, this can be contained in a single login event that identifies the time of a first login and the number of logins. Similarly, events can be folded into compute instances. For example, a particular type of detection may be observed on numerous endpoints within an enterprise network, and this can be compressed into a single data lake element that identifies the detection and the number of affected endpoints. The folding rule can also provide a filter, for example, that simply discards duplicate events that are literal duplicates of another event or that do not provide additional information useful for threat detection. In one aspect, folding rules can be created to save storage space, for example, when individual events are discarded after the folding rule creates an aggregate event. In another aspect, collapsible rules can be created to improve detection, in which case the aggregated event can provide useful data for efficient downstream threat detection, while the data lake can also retain each of the individual events underlying the event. aggregate.
More generally, there are events within an enterprise network that differ, if at all, only with respect to an accompanying timestamp, or accompanying machine or user identifier. In these cases, the events can be folded or otherwise aggregated into a single object of interest. This can include operating system detections, such as login errors, suspicious file detections, such as double file extensions (for example, in a [filename].xxx.yyy syntax, so that the user interface displays the file name with a misleading file name), etc. forward. In one aspect, detections can be categorized by, eg, risk, geographic region, client, classification rule (eg, by event type), attributes (filename, user, path, etc.), event type , etc. identify high-frequency events that could be usefully folded. When events with identifiable characteristics show a very high frequency within a historical interval, this can provide a basis for automatically or manually creating detection-oriented folding rules.
As shown in step2314, the method2300You can include the display of the folding rule in a user interface or other display for human review.
As shown in step2316, the method2300may include receiving a folding rule endorsement. This can be a manual approval provided by an administrator in a threat management installation console, or by some other system user or technician, and can be useful before the collapsible rule is deployed for use with the flow. of events.
As shown in step2318, the method2300may include implementation of the folding rule to filter and/or process a stream of events prior to storage in the data lake. It will be understood that although the emphasis in this description is on folding rules for events stored in the data lake, folding rules may also be usefully identified and deployed to compute instances for use in local monitoring. In general, the characteristic sequence of events may include a similar (or identical) sequence of events from two or more of the plurality of computing instances in the enterprise network, or a similar (or identical) sequence of events from only one of the plurality. of computing instances in the corporate network.
As shown in step2320, the method2300may include the application of the folding rule. This may include, for example, receiving a second plurality of security events and applying the collapsible rule to instantiate an aggregate security event from a second plurality of data objects in the second plurality of security events in the event stream. The second plurality of security events can be any of the events described in this document. For example, the events of the second plurality may be an asynchronous stream of security events from the enterprise network, the asynchronous stream includes a combination of batch transfers including groups of security events and streaming transfers of individual security events from one or more compute instances in the enterprise. grid.
As shown in step2322, the method2300may include processing of aggregate events created by the folding rule. This may include any use or review of an added event. For example, this may include storing the aggregate security event instance created by the folding rule in the data lake. This may also include, or instead, perform malware detection for the enterprise network with the threat management function using one or more instances of the aggregated security event stored in the data lake. As noted above, this can advantageously improve efficiency and reduce computational complexity of certain detections where a pattern of interest within a data stream can be accurately compressed into a representative aggregated event. In another aspect, this may include initiating a remediation of an enterprise network threat with the threat management function based on one or more instances of the aggregated security event stored in the data lake.
Collapsible rules such as those described in this document can operate more generally to compress data for communication to a data lake, or to distribute storage more effectively among endpoints within an enterprise network. That is, while in some cases a collapsible rule can be created to efficiently handle repetitive and low-value events with losses when data about individual events is lost, in other cases the data may be amenable to various forms of compression to the storage. or communicating within a lossless representation of the relevant sequence of detected events. For example, in one aspect, a sequence of related events can be compressed, for example by converting a sequence of detections to a list of timestamps for identical or substantially identical events, in a manner that allows for compact communication with the data lake as a single meta event, followed by decompression across the data lake into individual events used for detection. This can be useful, for example, where an analysis of individual events could potentially be useful for threat detection, but the nature of the sequence is amenable to more efficient representation during communication with the data lake. In another aspect, the collapsible rule can alleviate storage requirements for the data lake, for example, by storing certain information on a source compute instance (for example, a specific list of timestamps, machine identifiers, event identifiers, process names, users, or the like), for example, in a data recorder or other storage facility in the compute instance, while sending a folded representation with, for example, a timestamp for a first and last event, along with a count of the number of events in the folded meta. -event. This approach allows a query to the compute instance to retrieve the full representation of the underlying events, without the need for the corresponding data to be transmitted to the data lake and stored in the data lake in the first instance.
Accordingly, a system including a data lake, a stream service, and a transformer service is also described herein as any of the data lakes, stream services, and transformer services described herein. The data lake may include a data storage medium that stores a first plurality of data objects representing security events within an enterprise network and a plurality of descriptions for the first plurality of data objects, each of the plurality of descriptions organized according to one or more schemes. The Stream service can be configured to receive a stream of asynchronous events from additional data objects that represent security events from the enterprise network. The transformer service can be configured to process the stream of asynchronous events by filtering the additional data objects to remove one or more duplicate data objects that are already stored in the data lake, thus providing filtered data objects, to augment each of filtered data objects with a corresponding description organized according to one or more schemas, thus providing augmented data objects and for storing the augmented data objects in the data lake. The transformer service can also be configured to apply a folding rule to aggregate a sequence of similar events into an aggregate security event for storage in the data lake, the folding rule augments the aggregate security event with metadata characterizing one or more than one series of events in the security event sequence.
In one aspect, the collapsible rule may be generated automatically by correlating one or more previous instances of the similar sequence of events with behavior of interest in the enterprise network. In another aspect, the metadata may characterize one or more of a series of events in the security event stream, a frequency of events in the security event stream, and a first and last security event in the security event stream. .
As shown in step2402, the method2400it may include receiving a local threat indication from an endpoint. This can include data from a local security agent on an endpoint like any of the endpoints described in this document. The data may be received, for example, at the host described above, or any other resource or the like that supports a threat research platform as contemplated herein. The local threat indication may, for example, include an event at the endpoint indicative of malicious activity. The local threat indication may also include, or instead, a threat detection obtained by a local security agent by applying a detection rule to events detected on the endpoint. For example, threat detection rules, threat signatures, machine learning models, and other resources can be deployed to the endpoint and used by the local security agent for local threat detection. Any tools, models, rules and the like can be used locally by the endpoint to generate a local threat indication as described in this document. The local threat indication may also, or instead, include a classification indicating a category of malicious activity associated with events detected on the endpoint. This may be, for example, a classification provided by a third-party resource, such as the MITER ATT&CK classification system, or any other suitable taxonomy, classification resource, or the like.
As shown in step2404, the method2400may include receiving a contextual threat score calculated by a threat management facility based on event data received from the contextual information received at the threat management facility. The threat management facility can use any context, event information, or the like, as generally described in this document, and can assess a threat based on data from the endpoint, other endpoints, event context, or any combination of these. For example, contextual information may include one or more of the classification information for a suspected threat, a network location associated with a suspected threat, geolocation data for a suspected threat (which may be obtained using various associated geolocation resources, for example, the IP address space on the Internet with multiple geographic locations), one or more of a path, a file name, a process name, and a machine identifier for a suspected threat, and a threat score based on one or more plus machine learning rules and heuristic rules. It will be understood that the contextual information may also include, or instead, events or event vectors received from the endpoint and/or from other endpoints within a business network or the like.
In one aspect, contextual information includes any transient threat data, such as third-party data provided by contextual information sources independent of the enterprise's security infrastructure. This may, for example, include third-party threat identification or threat scoring tools, geolocation services, reputation databases, threat signature databases, etc. As a significant advantage, feeding contextual information of this type to the data lake provides a non-transient record that can be subsequently analyzed to assess a threat posture at the time the information was acquired and to detect changes in context over time. over time. For example, contextual information, such as third-party geolocation data or third-party threat scores, is based on external resources that may provide different information when viewed at different times. By way of non-limiting example, the geolocation of an IP address, or the IP address or URL of a command and control center for an advanced persistent threat, may be unknown at the time an event is logged, but may be discovered. or changed at a later time. These changes over time will generally not be available in a form that can be monitored by the business. To ensure that contextual information is correctly placed on a timeline of events when qualifying threats or performing forensics, contextual information can be stored in a way that allows investigation of the known context at a given time. It will be understood that to facilitate any corresponding time-based or time-sensitive analysis, contextual information, as described herein, may have a useful time stamp when stored in the data lake based on the time of acquisition. of the data of a third party. resource or other transitory data source. This approach can also facilitate the creation, refinement, or other use of machine learning models in which data samples, such as code segments, are tagged with contextual information that was available at a particular time of interest.
As shown in step2406, the method2400may include receiving data from cloud resources based on an action associated with the endpoint in a cloud service. In general, the cloud service can be any of the cloud services described in this document. By way of non-limiting examples, the cloud service may include one or more of an email application, a web application, a cloud storage service, a zero trust network access resource, a virtualization, a cloud computing service and an authentication system. service. Cloud resource data from such cloud services may include, for example, one or more authentication to the cloud service, administrative events to the cloud service, and application activity to the cloud service. started from the end point. The action associated with the endpoint that triggers the generation of cloud resource data may, for example, include an activity by an endpoint user, such as the user accessing or using the cloud service which started from the end point. The action may also include, or instead of, an activity performed by an administrator of an enterprise network associated with the threat management facility, where such activity could be relevant to the assessment of a potential threat.
The cloud service can also be any other remote service or device that can be accessed in the cloud through suitable hardware and software. For example, the cloud service may include a firewall, for example by adding a network monitor that runs on a third party firewall to transmit log data from the firewall over a secure communication channel. More generally, any network device or other hardware or software that can be usefully monitored for security purposes can be configured as a cloud service by providing a network interface to securely access the corresponding data and functions.
As shown in step2408, the method2400may include determining a composite threat score for the endpoint based on at least the local threat indication, the contextual threat score, and cloud resource data. It will be appreciated that data from the various possible sources may be combined in a number of different ways. For example, each source can independently score the risk (or present data that the host can score), and the composite threat score can reflect the highest individual score of each of these independently scored risks. In another aspect, the scores from the sources may be combined on a weighted or unweighted basis, or otherwise combined to obtain a composite score that represents the contributions of each separate data source.
It will be understood that the composite threat score may be based on other data that is processed as described herein. For example, this may include telemetry data from any of the sensors described in this document, which may be processed using one or more folding rules to reduce noise and increase useful detection signals in the telemetry data. It will also be understood that collapsible rules may include, for example, collapsible rules implemented in a compute instance to locally reduce noise in the telemetry data from sensors in the compute instance, or this may include collapsible rules implemented in the data lake. to compress or otherwise represent enterprise-wide telemetry data (or individual compute instances within the enterprise) in a way that eliminates repetitive data of little value to threat analysis.
As shown in step2410, it can be determined if the Composite Threat Score is above a predetermined threshold to automatically start the investigation. If the composite threat score is not above the default threshold, the method2400you can proceed to step2412where you can display the score. If the composite threat score is above the default threshold, then the method2400you can proceed to step2414where an investigation is created.
As shown in step2412, the method2400You can include the composite threat score display in a user interface, such as any of the threat research user interfaces discussed in this document. This may include the optional display of the composite threat score (along with accompanying summary information for the suspected threat), subject to filters set in the user interface. Therefore, the user interface can be configured to only display threats above a filter threshold. This may also include classifying or sorting suspected threats (above the filter threshold) using any appropriate criteria such as composite threat score value, user, threat type, time of occurrence, etc.
As shown in step2414, the method2400may include automatically launching an investigation for a suspected threat when the composite threat score is above a predetermined threshold. This may, for example, include creating an investigation container (including a data structure and/or user interface) to investigate activity associated with the composite threat score as described above, or otherwise creating a user interface and/or data structure to support investigation of the context and details of a particular suspected threat. The research container may, for example, display (and/or store) one or more of local threat indication, contextual threat score, and cloud resource data. The investigation container can also provide access to supporting data for one or more of the local threat indications, contextual threat score, and cloud resource data to facilitate further investigation and other follow-up by a technician. .
As shown in step2416, the method2400It can include updating threat data based on various sources, such as the endpoint, threat management facility, or cloud services. This may, for example, include incremental updating of the Composite Threat Score and/or any accompanying score, detection or contextual information, or the like, as such information becomes available from various resources. In one aspect, this may include incrementally updating the Composite Threat Score based on data from one or more local security agents, the threat management facility, and the cloud service. This may also include, or instead, incremental updating of the composite threat score based on endpoint status information from the threat management facility, or any other resource for generating threat scores, assessing status or the security posture of an endpoint, etc. This may also, or instead, include incremental updating of the composite threat score with data from one or more third-party security service providers. For example, this may include updating threat definitions, signatures, detection rules, events, event vectors, health scores, reputation data, geolocation data, etc. More generally, this may include incrementally updating data for a potential threat using any of the remote, local, or other resources described in this document.
Where an investigation container has been created, this may also or instead include incremental updating of the information in the investigation container as data from one or more sources becomes available. In another aspect, this may include augmenting the information in the research container based on a history of responses by other users to a potential threat associated with the research container. For example, as described in this document, successful and failed responses, or other investigation paths associated with good or bad results, can be logged and used to aid subsequent threat investigation and remediation activity. When there are answers, queries, or the like that are known (based on historical activity) to be associated with better results, the research platform may automatically initiate one or more of those steps, or recommend them to a technician, or some combination of these.
Accordingly, further described herein is a system including a plurality of computing instances associated with an enterprise network and a threat management function for the enterprise network, the threat management function configured to determine a composite threat score based on: a threat indication received from one of the compute instances; cloud resource data based on an action performed on a cloud service and associated with one of the compute instances; and a contextual score based on geolocation data received from a remote geolocation service for an assumed threat associated with the local threat indication. The system may include an administrative console configured to display the composite threat score in a user interface. Cloud resource data may, for example, include application activity launched on the cloud resource from the compute instance.
As shown in step2502, the method2500You can start by receiving an event stream like any of the event streams described in this document. For example, this may include receiving, at a threat management facility, a first stream of local threat data events from various local security agents running on various compute instances associated with an enterprise network, or more generally, receiving a first event. Event flow based on local threat data from multiple compute instances in an enterprise network. This may include any of the events or threat data described in this document that may be provided directly by local security officers. For example, local threat data may include a classification that indicates a category of malicious activity associated with events detected in one of several compute instances, such as a MITRE classification, or another classification that provides a reference for consistent classification of different types of threats. malware and attacks.
As shown in step2504, the method2500may include storing the event stream in a data lake, such as any of the data lakes described in this document. This may, for example, include storing threat detections or other data in the first stream of events according to a threat schema for the threat management facility. This can include any scheme that enforces a consistent structure in the event stream to facilitate finding, retrieving, and using threat data in the data lake for threat management functions. As an example, for threat events such as threat detections or indications of compromise, a threat schema might include fields for a date of the threat (or more generally, the indication of compromise), a time of a threat, a worker (e.g., software module, process, microservice, or similar) that detected the threat, an attack type (using any suitable classification scheme), a threat score (for example, on a scale of 0 to 10), a identifier for a detection method used to detect the threat, a third-party attack category (for example, a MITRE classification or similar), a human-readable description of the threat, a unique identifier for the threat, a unique registration identifier for detection etc.
As shown in step2506, the method may include authentication with a cloud service provider. For example, this may include authenticating a resource at a cloud service threat management facility to provide administrative access to management functions and programming interfaces to access data on user accounts hosted on the cloud service. cloud service. In general, the cloud service provider can be any service provider that provides cloud computing facilities to the users associated with the business network. For example, the cloud service provider may include a cloud computing service that hosts virtual computing instances for the enterprise network, an identity provider that provides identity management and/or authentication services for users of the enterprise network, a third-party network security service for enterprise network users, an application hosting provider for enterprise network users, or some combination of these. The cloud service provider may also host, or in its place, one or more of an email application, a cloud storage service, a cloud computing service, a virtualization platform, a web application , a zero-trust network access resource, a network monitor that runs on a third-party firewall, and an authentication service.
As shown in step2508, the method2500may include cloud service settings for streaming. This may, for example, include enabling any appropriate cloud service native logging, reporting, notification or auditing features that are provided for individual accounts or for the service as a whole, using an administrative account for the cloud service. . This may also include, or instead, installing or activating a streaming service on the cloud service and/or configuring an external service to receive a data stream from the cloud service and publish the data, in the appropriate format in accordance with applicable schematics, in the flow of events. or any suitable intermediate manipulator.
As shown in step2510, the method2500may include receiving security data from the cloud service provider. This may include any security data provided natively by the cloud service provider, such as authentication data for a cloud service provider, administrative events at the cloud service provider, and application activity on the cloud service provider. cloud service provider launched from one of the compute instances. It will also be understood that there may be any number of intermediaries between the cloud service and a data lake that stores threat data. Thus, receipt of security data may include receipt of data from the cloud service with an external service that processes it accordingly and posts it to the event stream, and/or this may include receipt of data in the event stream data lake. In either case, the data may be formatted, filtered, augmented, or otherwise processed for submission to the data lake.
As shown in step2512, the method2500It may include mapping the cloud service provider's security data to a threat scheme for delivery or storage in the data lake. For example, this may include mapping the cloud service provider's security data into a second event stream that conforms to the threat schema for the threat management facility.
It will be understood that the data schema or structures used by the cloud service may differ significantly from the threat schema used in the data lake and may include fields and data specific to the cloud service environment. For example, in a Windows office365environment, the data may include Active Directory data (such as access events, login (and failed login) or token service events, role change or group change events), Exchange data ( for example, administrative audit data, single message events, and mailbox audit events, malware/phishing detection events, antivirus events, mail forwarding events, and message auto-labeling events), SharePoint data ( for example, file operation events, file sharing events, user feedback events, list content and item events, auto-tagging policy events, and search events), Skype (for example, call events , locked user events) and Teams (admin events, device events, analytics events, user events, guest access, or team building events). More generally, a cloud service can provide auditing of sensitive data access, file sharing, policy compliance, file access and use, search/query activity, password changes, permission changes (for users , groups, files, folders, messages, etc.) login attempts, messaging services (forward, send, read, delete, move, login, etc.), threat intelligence, etc., any of which it can usefully be stored as threat data in the data lake. As such, the cloud service data can be processed in a number of ways to tailor the cloud service data to the data lake requirements. Cloud service provider data may also include, or instead, user identification information, such as a username, email address, user identifier, or the like. This information may be provided as a data structure containing all users associated with the company and/or as a label, tag, or the like for each event associated with a particular user.
For example, security data mapping may include scaling one or more quantitative threat scores in the security data to a range of threat scores for the threat schema, so that quantitative risk assessments of the service in the security data cloud are generally more comparable with other quantitative risk assessments in the data. lake from other sources. Security data mapping may also, or instead, include the conversion of one or more threat types in the security data to a threat category for the threat schema. As with quantitative scaling, using a similar or identical categorization taxonomy for data from the cloud service and other sources allows for accurate comparison and analysis in parallel. Therefore, the categories used by the cloud service can be mapped to analogous categories (or, in some cases, simply category names) used within the data lake. In another aspect, mapping the security data may include transforming risk metadata in the security data into one or more handles for the threat schema. For example, when a native cloud service environment reports a failed login with user credentials using a specific alphanumeric code, this can be mapped to a corresponding descriptor (eg "login failed") used within the data lake. Some cloud service providers provide a data blob of all available threat data formatted according to a cloud service schema. In this case, security data mapping may also include, or instead convert a data blob in the cloud service provider's security data into a plurality of risk elements identified in the threat schema. , for example, by parsing, recategorizing, scaling, transforming, or otherwise processing data in the data blob.
As shown in step2514, the method2500may include augmenting the second event stream. In general, the second stream of events can be augmented as described in this document with additional security data from one or more third party security data providers and/or with geolocation data and other supplementary data. The second event stream may also include, or instead, contextual data for an alleged threat from the source of an event or one or more external resources. By way of example and not limitation, the contextual data may include one or more of the classification information for the alleged threat, a network location associated with the alleged threat, a geographic location for the alleged threat, a route for the alleged threat, a filename for the alleged threat, a process name for the alleged threat, a machine identifier for the alleged threat, a user identifier associated with the alleged threat, and so on. In one aspect, augmenting the second event stream may include receiving this contextual data and/or receiving other contextual information for an alleged threat, such as data from a remote data source. For example, the remote data source may provide classification information for the purported threat, a network location associated with the purported threat, geolocation data for the purported threat, or any other useful information, which may be based on any of the contextual data or other data already present in the event stream.
As shown in step2516, the method2500may include storing the second event stream in the data lake. Threat data from the cloud service can be used in conjunction with other threat data in the data lake for various threat detection and management functions.
As shown in step2518, the method2500may include calculation of a threat score based on the second stream of events. For example, this may include calculating a threat score for one of the compute instances based on data from the first event stream and the second event stream stored in the data lake. As a significant benefit, this enables better threat assessment using on-premises threat data obtained directly from an on-premises security agent running on a compute instance, along with cloud service data from a cloud service used by the compute instance. In one aspect, this may include calculating multiple threat scores, such as a first threat score based on on-premises threat data, a second threat score based on cloud threat data, and a third threat score based on cloud threat data. on data from a remote third-party threat. service. This may also include, or instead, a calculation of a composite threat score, such as any of the composite threat scores described in this document, based on local threat data on the compute instance, threat data on cloud from a cloud service used by the compute instance and/or supplemental threat data from one or more third-party threat services.
As shown in step2520, the method2500it may include any additional processing usefully associated with the cloud service provider's second event stream. For example, this may include displaying one or more of a plurality of threat scores, such as any of the individual or composite threat scores described in this document, in a user interface such as an administrative console of a threat management installation to a business network. Threat scores can be presented in a ranked order according to threat severity, and can also, or instead, be color-coded or presented in a way that highlights the most serious risks. In another aspect, further processing may include remediating an associated threat (or initiating a remediation of the threat), for example, using any of the remediation techniques described in this document, such as initiating a remediation of one of the compute instances in the enterprise network when one of the threat scores meets a predetermined criteria or quantitative threshold.
It will be appreciated that detections based on the simultaneous monitoring of a compute instance and an associated cloud service can allow significantly greater flexibility in threat detection and remediation, particularly when user behavior on a compute instance could be compared to User activity on an account. in the cloud service. For example, when a phishing email is detected in an email service for a user, a remediation or notification may be initiated in a compute instance associated with the user before the user has an opportunity to view the communication. As another example, activity on a user-associated compute instance can be compared to login or other authentication activity at an identity provider to assess the likelihood of credential theft or misuse. More generally, any threat detection based on a combination of threat data from a cloud service and threat data from a local security agent can be usefully performed using the techniques described in this document.
In accordance with the foregoing, also disclosed herein is a system that includes a plurality of computing instances associated with an enterprise network, each of which runs a local security agent that provides event data to a first event stream and a function threat management for the enterprise network. The threat management facility can be configured, for example, via computer-executable code, to authenticate to a cloud service provider for the enterprise network, receive security data from cloud service providers in a second stream events, calculating a composite threat score indicative of a security risk of one of the compute instances based on data from the first event stream and the second event stream, and displaying the composite threat score in a user interface. The cloud service provider may, for example, include one or more of an application hosting platform, a communication platform, an identity management platform, and a remote security service platform.
A window of investigation2604inside the user interface2600for one of those threats can include information currently available for the threat to supplement the threat summary2602. For example, the research window2604may display information associated with the threat, such as a process, path, process owner, certificate information, hash (for example, SHA256 hash), and one or more threat scores, such as any of the threat scores described in this document. This may, for example, include machine learning scoring (eg based on event vectors associated with the threat), third party scoring (eg based on event identification from a threat resource), third-party security), a global reputation (eg, from the threat management facility), a local reputation (eg, from the originating endpoint), etc. Information about the affected endpoint such as device name, device type, IP address, geolocation information, operating system, user, etc. can also be provided.
In one aspect, a research container2606for a particular threat can be automatically started as a persistent programmatic object to handle a case for a particular threat and stored in a data repository2608such as a database for a threat management facility or other host for threat research. The research container2606it can be advantageously launched, for example, when the composite threat score for a threat reaches a predetermined threshold indicating a high probability of malicious activity. The research container2606may include, or may communicate with, a user interface that provides various controls for user investigation using any resources available to the host. This may include query access to the data lake, proofing tools, external resources, etc.
In one aspect, the research window2604(or a research container2606supporting the research window2604) can be gradually updated within the information from various available sources. For example, while an initial detection from an endpoint or threat management installation can be used immediately to create a detection, assign a composite threat score, and display the newly detected threat in the user interface2600if the score reaches a user-defined threshold, a wide variety of relevant information may not be immediately available. For example, many of the resources, such as third-party resources used to classify or score a threat or cloud services for the enterprise network, may have significant latencies, limited availability, or bandwidth limitations. At the same time, not all information related to a detection may be available at the time of detection. That is, information from other affected devices within an enterprise network, information from other processes on an affected endpoint, new threat identifications or signatures, and the like, may only be available after the initial event/detection. In the systems and methods described in this document, the data for an investigation can be updated gradually as new data becomes available and, where appropriate, the composite threat score can be updated based on the new information. As a significant advantage, this approach enables early reporting and classification of potential threats, while making it easy to augment a variety of data sources that have different latencies and availability.
As shown in step2702, the method2700may include receiving local threat data. This can, for example, include any of the local threat data described in this document. In one aspect, local threat data includes a local threat indication from a local security agent running on a compute instance or other threat data obtained locally from the compute instance, such as a local threat indication that identifies a category of malicious activity associated with one or more events detected on the compute instance. Local threat data may also include, or instead, a local threat identification for an event indicative of malicious activity on the compute instance, or a classification indicating a category of malicious activity associated with events detected on the compute instance. calculation. In another aspect, the local threat data may include a threat detection obtained by the local security agent by applying a detection rule based on at least one malware signature and behavior detected on the compute instance.
In general, receiving the threat data may include receiving the threat data local to a threat management facility directly from the compute instance over a data network, although it will be understood that the data may initially be published to an event stream. and/or stored in a data lake as described in this document, and the threat management facility can retrieve local threat data from such data sources proactively (by requesting the data) or reactively (after receiving a notification of new local threat data). This may also include calculating a local threat score on the compute instance for transmission to the threat management facility.
As shown in step2704, the method2700may include receipt of contextual threat data, such as any of the contextual threat data described in this document. For example, this may include geolocation data or other supplemental data retrieved from a third-party service for a suspected threat detected on the compute instance. In one aspect, this may include calculating a contextual threat score based, at least in part, on geolocation data retrieved from a third-party service for an assumed threat on a compute instance, and then transmitting this contextual threat score. to the threat management facility (or event stream). In another aspect, the threat management facility may calculate the contextual threat score, for example, by calculating composite threat scores in step2708. Receiving the threat data may include receiving the contextual threat data in a threat management facility directly from the third party service over a data network, although it will be understood that the data may initially be published to an event stream and/or be stored in a data lake as described in this document, and the threat management facility can retrieve contextual threat data from those data sources proactively (by requesting the data) or reactively (after receiving notification of new threats). contextual threat data).
As shown in step2706, the method2700may include receiving data from cloud resources. As described in this document, this can generally include any action-based cloud resource data associated with the compute instance, or a user of the compute instance, in a cloud service. In one aspect, the cloud service may calculate or otherwise provide a cloud threat score that is transmitted to the event stream, data lake, or threat management facility. In another aspect, the threat management facility can calculate the cloud threat score, for example, by calculating the composite threat scores in step2708. In general, the cloud service can be any of the cloud services described in this document, including, but not limited to, a web application, a cloud storage service, an email application, an authentication service, a zero-trust network access resource, a network monitor running on a third-party firewall, a cloud computing service, and a virtualization platform. Receiving data from cloud resources may include receiving data from cloud resources in a threat management facility directly from a cloud service over a data network, although it will be understood that the data may be published initially in an event stream and/or stored in a data file. lake as described in this document, and the threat management facility can retrieve cloud resource data from those data sources proactively (by requesting the data) or reactively (after receiving notification of new resource data from the cloud).
As shown in step2708, the method2700It may include calculating or otherwise determining a composite threat score based on on-premises threat data, contextual threat data, and cloud resource data. Each component of the composite threat score can be evaluated and scored independently, or the components can be scored together, or some combination of these. Therefore, the composite threat score can include a single score based on a combination of two or more of the local threat indication, contextual threat data, and cloud resource data. In another aspect, the composite threat score may include a number of scores, each individually based on a local threat indication, the contextual threat score, and cloud resource data. The composite threat score can also include a combination of these two types of scores.
As shown in step2710, the method2700may include evaluating whether to investigate a potential threat based on the composite threat score. When the composite threat score meets one or more predetermined criteria for the investigation, the method2700you can proceed to step2712where you can create a research container. When the composite threat score does not meet one or more predetermined criteria for investigation, the method2700you can go back to step2702and additional local, contextual, and cloud data can be collected.
As shown in step2712, the method2700may include the automatic creation of an Investigation Container to investigate activity associated with the Composite Threat Score in response to the Composite Threat Score reaching a predetermined threshold. In general, the research container may be a data object that includes data associated with a potential threat, along with links, pointers, or the like to external resources associated with the potential threat, or to threat research in general.
The investigation container may be associated with (and/or may programmatically include) a user interface, such as the user interface described with reference to
As shown in step2714, the method2700may include additional processing consistent with the use of the investigation container to investigate and remove threat detections. For example, this may include transmitting a notification with a link to the user interface associated with the investigation container to a device associated with a security technician so that the security technician can access the user interface and container. investigation to investigate the potential threat. A notification regarding a new research container can be communicated in other ways. For example, this may include displaying a pop-up window or other display element within a screen of a device the technician is currently using, or sending a research container availability notification through some other means. For example, the threat management facility may transmit a text message, email, or phone communication to a technician to notify them of the availability of a new research container, particularly when a threat score associated with the research container indicates a serious security risk and/or a potential compromise of highly valuable data.
Additional processing may also include, or instead, display research container information to a user. For example, this may include displaying the Composite Threat Score(s) in a user interface associated with the research container, or otherwise presenting related threat information in the user interface, such as one or more than local threat data (or indication of local threats). ), contextual threat data, and/or cloud resource data. In one aspect, this may include displaying the composite threat score in a notification to a security technician within an email, text message, or other communication.
In another aspect, further processing may include incremental updating of information in the research container. Due to the distributed nature of threat data sources, there may be different latencies, reporting frequencies, etc. Some data may be received in batches, such as remote resource lookups or data aggregated and grouped by an endpoint, such as a gateway or firewall, or any of the other computing instances described in this document. In this case, the research container may be updated incrementally as each new piece of information becomes available, for example, as the data becomes available independently from each compute instance, third-party service (for example, data from geolocation) and cloud service. . For example, additional processing may include incremental updating of the composite threat score based on an evaluation of the compute instance by the threat management facility, for example, when the threat management facility identifies a threat detection. threats or other potential risk. When a composite threat score or other metric indicates reduced risk severity based on gradually updated data, further processing may also include removal or closure or termination of the research container. For example, when a request to a third-party service provides a response indicating that the potential threat is signed by a trusted third-party, then the investigation container can be disposed of.
In this manner, a human investigator or computer scoring tool or the like can maintain a current view of a threat associated with the investigation container as the risk posture develops over time. At the same time, the research container may reflect new information received after the research container was created. In addition to incremental updating of the supporting data, additional processing may include incremental updating of the scores that are presented in or associated with the research container. For example, this may include incrementally updating the composite score(s) as data becomes available from one or more remote sources, such as the cloud service or a third-party data source.
Additional processing may include providing interactive access, in a user interface associated with the research container, to the supporting data for the composite threat score. In one aspect, incremental updating of the information in the research container may include updating this supporting data that is accessible through the research container. This allows tools within the UI to link to current data, for example, data in the data lake or other store that has been gradually updated with new data received after the research container was created but before the research container was created. a request for data from the user. Information displayed within the investigation container, or an interface accessed from the investigation container, can also be updated dynamically within the display to ensure that a user is responsive to the latest available security information.
In another aspect, further processing may include augmenting the information in the research container based on a history of responses by other users to a potential threat associated with the research container. Therefore, as described generally in this document, user query activity received in the context of previous investigation containers, including query patterns, query updates, and/or responses taken to queries in an administrative console, may used to increase the information in the research container. for example, by providing query-based threat detections or by suggesting useful remedial actions based on the current context.
Accordingly, this document describes a system that includes a plurality of computing instances associated with an enterprise network, a threat management facility for the enterprise network, and an administrative console. The threat management function can be configured, for example, by computer executable code, to determine a composite threat score based on a local threat indication received from one of the compute instances, based cloud resource data on an action performed on a cloud service and associated with that of compute instances and a contextual score based on geolocation data received from a remote geolocation service for an alleged threat associated with the local threat indication. The administrative console can be configured to automatically display a user interface associated with an investigation container when the composite threat score reaches a predetermined threshold.
As shown in step2802, the method2800may include asynchronous receipt of local threat data from a compute instance on an enterprise network. This may include a local threat indication from a local security agent running on the compute instance and may identify a category of malicious activity associated with one or more events detected on the compute instance, or provide threat information local to the instance computing. This may also include, or instead of, any of the other local threat data described in this document. Asynchronous local threat data, as described herein, shall be understood to be local threat data that is not synchronized in time with data from other data sources. Data can be received before other data, after other data, during the reception of other data, etc., and can be received in batches or as individual events, each and every one of which can be published to an event stream regardless of others. data sources on a non-predetermined schedule. In this context, a resource that receives or analyzes the data, such as a threat management facility, will typically not be able to determine in advance an order or schedule of data from each data source, except in those cases where a transmission is scheduled. particular data. in advance and transmitted through a reliable communication channel.
As shown in step2804, the method2800may include asynchronous receipt of contextual threat data, such as any of the contextual threat data described in this document, from a third-party service. This may, for example, include geolocation data retrieved from a third-party service for an alleged threat detected on the compute instance or any other contextual data. Asynchronous contextual threat data, as described herein, shall be understood to be contextual threat data that is not synchronized in time with data from other data sources. Data can be received before other data, after other data, during the reception of other data, etc., and can be received in batches or as individual events, each and every one of which can be published to an event stream regardless of others. data sources on a non-predetermined schedule. In this context, a resource that receives or analyzes the data, such as a threat management facility, will typically not be able to determine in advance an order or schedule of data from each data source, except in those cases where a transmission is scheduled. particular data. in advance and transmitted through a reliable communication channel.
As shown in step2806, the method2800may include asynchronous receipt of cloud resource data, such as any of the cloud resource data described in this document, from a cloud service. For example, this may include cloud resource data based on one or more actions associated with the compute instance in a cloud service that supports one or more cloud-based applications for enterprise network users, or any other cloud resource data. This may also include data from two or more cloud service providers, each of which provides data from cloud resources asynchronously to one another. Asynchronous cloud resource data, as described herein, shall be understood to be cloud resource data that is not time-synchronized with data from other data sources. Data can be received before other data, after other data, during the reception of other data, etc., and can be received in batches or as individual events, each and every one of which can be published to an event stream regardless of others. data sources on a non-predetermined schedule. In this context, a resource that receives or analyzes the data, such as a threat management facility, will typically not be able to determine in advance an order or schedule of data from each data source, except in those cases where a transmission is scheduled. particular data. in advance and transmitted through a reliable communication channel.
As shown in step2808, the method2800may include, in response to asynchronous data from one of the plurality of sources, incrementally calculating a composite threat score indicative of a threat risk to the computing instance based on the threat data. The composite threat score can include any of the composite threat scores described in this document. In this context, incremental computation of a composite threat score refers to a computation (or recalculation or update) of the composite threat score in response to one or more asynchronous data items from any of a plurality of sources. That is, if local threat data, such as threat detection or threat score, is received from a compute instance, the composite threat score will be calculated without waiting for other potentially relevant information, such as associated lookups for supplementary data (such as geolocation data). ). from third-party sources, or related information from a cloud resource. Instead, the composite threat score can be calculated immediately with available data and then gradually updated as new data items become available from multiple sources.
As shown in step2810, if an incrementally calculated composite threat score reaches a predetermined threshold, additional action can be taken, such as creating an investigation container as shown in step2814, or otherwise responding to the corresponding potential threat. Alternatively, if the incrementally calculated composite threat score does not reach a predetermined threshold, the method2800can proceed to2816where additional threat data may be received from one or more of the data sources.
As shown in step2814, the method2800This can include the automatic creation of a research container, like any of the research containers described in this document, when the composite threat score reaches a predetermined threshold. The investigation container may be associated with a user interface for interactively investigating the sources of the composite threat score.
Research container creation can include any number of additional and/or related steps to help a user investigate and dispose of an associated threat. For example, the method2800it may also include displaying the compound threat score to a user in the user interface associated with the investigation container, or otherwise facilitating the investigation and remediation of any related threats. The method2800It can also include creating an alert or notification to a user when the composite threat score reaches a predetermined threshold, such as a message containing a link to the research container (or a user interface displaying data from the research container).
As shown in step2816, the method may include updating threat data from one or more of a plurality of sources. This may include receiving additional asynchronous data from one of a plurality of sources and updating the composite threat score displayed in the user interface in response. In another aspect, updating threat data may include updating data internally, e.g. otherwise, augment the information for the investigation container based on other data available in a data lake or other resource for the enterprise network.
In accordance with the foregoing, a system including a plurality of computing instances associated with an enterprise network and a threat management facility for the enterprise network is also disclosed herein. The threat management facility can be configured, for example, via computer executable code, to receive threat data asynchronously from a plurality of sources, the threat data includes at least one local threat indication from a security agent location on a compute instance, local threat indication by identifying a category of malicious activity associated with one or more events detected on the compute instance, geolocation data retrieved from a third-party service for a suspected threat detected on the compute instance, and data from Action-based cloud resources associated with the compute instance in a cloud service that supports one or more cloud-based applications for users on the enterprise network. The threat management function can be further configured to respond to asynchronous data from one of a plurality of sources by performing the steps of incrementally calculating a composite threat score indicative of a threat risk to the compute instance, creating a threat container investigation when the composite threat score meets a predetermined threshold, displays the composite threat score in a UI associated with the investigation container, and updates the composite threat score in the UI in response to additional asynchronous data from a from multiple sources.
The above systems, devices, methods, processes, and the like may be implemented in hardware, software, or any combination of these suitable for a particular application. Hardware may include a general purpose computer and/or a dedicated computing device. This includes implementation on one or more microprocessors, microcontrollers, embedded microcontrollers, programmable digital signal processors, or other programmable devices or processing circuitry, together with internal and/or external memory. This may also, or instead, include one or more application-specific integrated circuits, programmable gate arrays, programmable array logic components, or any other device or devices that can be configured to process electronic signals. It will further be appreciated that an embodiment of the processes or devices described above may include computer executable code created using a structured programming language such as C, an object-oriented programming language such as C++, or any other high or low level language. programming language (including assembly languages, hardware description languages, and database programming languages and technologies) that can be stored, compiled, or interpreted to run on one of the above devices, as well as heterogeneous combinations of processors, processor architectures or combinations of different hardware and software. In another aspect, the methods may be incorporated into systems that perform the steps thereof, and may be distributed among devices in various ways. At the same time, processing can be distributed among devices, such as the various systems described above, or all functionality can be built into a dedicated stand-alone device or other hardware. In another aspect, the means for performing the steps associated with the processes described above may include any hardware and/or software described above. All of these permutations and combinations are intended to fall within the scope of the present disclosure.
Embodiments described herein may include computer program products that comprise computer executable code or computer usable code that, when executed on one or more computing devices, performs any and/or all steps thereof. Code can be stored non-transiently in a computer's memory, which can be a memory from which the program is executed (such as random access memory associated with a processor), or a storage device such as a disk drive. , flash memory or any other optical, electromagnetic, magnetic, infrared or other device or combination of devices. In another aspect, any of the systems and methods described above may be incorporated into any suitable transmission or propagation medium carrying computer executable code and/or any input or output thereof.
The method steps of the implementations described in this document are intended to include any suitable method for causing said method steps to be performed, in accordance with the patentability of the following claims, unless a different meaning is expressly provided or is clear from the context. . So, for example, performing the step of X includes any suitable method of causing another party, such as a remote user, remote processing resource (for example, a server or cloud computer), or a machine, to perform the step. of X. Similarly, performing steps X, Y, and Z may include any method of directing or controlling any combination of such other persons or resources to perform steps X, Y, and Z to obtain the benefit of those steps. Therefore, the method steps of the implementations described herein are intended to include any suitable method for causing one or more parties or entities to perform the steps, consistent with the patentability of the following claims, unless expressly provided. a different meaning or is otherwise clear. the context. Such parties or entities need not be under the direction or control of any other party or entity, and need not be located within any particular jurisdiction.
It will be appreciated that the methods and systems described above are given by way of example and not limitation. In the absence of an explicit indication to the contrary, the disclosed steps may be modified, supplemented, omitted and/or rearranged without departing from the scope of this disclosure. Numerous variations, additions, omissions, and other modifications will be apparent to one skilled in the art. Furthermore, the order or presentation of method steps in the above description and drawings is not intended to mandate this order of performing the listed steps unless a particular order is expressly required or is clear from the context. Therefore, while particular embodiments have been shown and described, it will be apparent to those skilled in the art that various changes and modifications in form and detail can be made without departing from the spirit and scope of this description and are intended for form a part of the invention as defined by the following claims, which are to be construed in the broadest sense permitted by law.
FAQs
US Patent Application for THREAT DATA MANAGEMENT PLATFORM Patent Application (Application No. 20230114719 Issued April 13, 2023)? ›
Searching for Abandoned Patents
When you have the patent number, you can search the USPTO Patent Application Information Retrieval website by patent number or application number. The listing in the PAIR database includes the patent's status.
Searching for Abandoned Patents
When you have the patent number, you can search the USPTO Patent Application Information Retrieval website by patent number or application number. The listing in the PAIR database includes the patent's status.
An abandoned patent occurs when the inventor doesn't finish the patent process or fails to pay any required fees. With an abandoned patent, you get to take advantage of someone else's hard work. More so, you get the opportunity to expand and improve on the work of others, which is key to fostering innovation.
How do you tell if a US patent application has been granted? ›Patent status is available through the Patent Application Information Retrieval (PAIR) system. PAIR gives access to: The status of issued patents. The status of patent applications.
How many patent applications are rejected? ›Did you know that about 86% of patent applications are rejected on the first try?