Part 11. Wazuh Events and MISP Automation

Automating Threat Intel with Graylog

Intro

SIEM Stacks can often feel as if you are trying to drink water from a fire hose. You deploy a few monitoring agents and you are able to tread water and get by just fine. However, once you start monitoring hundreds to thousands of servers, you quickly drown in log overflow. This makes it extremely difficult for our SOC team to correlate threat intel with every network connection, every process hash, every file uploaded that have occurred on our endpoints. The mundane task of manually inspecting every event sounds like a nightmare. No wonder SOC turnover is so high!

In Part Ten of our series we installed MISP, the open source threat sharing intelligence platform. We enriched MISP with events that contain IoCs such as malicious:

  • IPs
  • Domains
  • Hashes
  • Emails
  • Etc

MISP has a strong API that we will use via Graylog to enrich our logs on the fly with Threat Intel. Taking the need for manual analysis out of the mix!

Flow

The beauty with Graylog is that we are able to enrich logs on the fly, before they are written to the Wazuh Indexer for storage. We explored an example of this in further detail in Part Nine with our log normalization. Graylog is also able to make HTTP Posts to API endpoints and use the received response to populate the log. This is perfect for our use case because we want Graylog to use MISP’s API to inspect values contained within our selected fields and use the response to populate the log even further.

MISP Details

The flow of data is shown below:

Data Flow

Fluent-Bit that is running on the Wazuh Manager will foward the alerts.json file to Graylog. Graylog will strip out the IP address, Domain, Hash, etc. and build an API request to send to MISP. MISP will respond back to Graylog and include event details if the value included in the Graylog request exists within MISP. Graylog will take the MISP response, and new fields to the log and write that log into the Wazuh-Indexer .

MISP API

API DOCS

Let’s see how the MISP API works before including it within Graylog.

  1. Create API Key for Authentication

Administration -> List Auth Keys

MISP Auth Keys
Add Auth Key

2. Make Request via Curl (I am not using HTTPS in this demo)

curl -H "Accept: application/json" -H "Content-Type: application/json" -H "Authorization: **MISP_API_KEY*" http://misp.socfortress.co/attributes/restSearch/value:mb4z3nlfyrcjnoqf.onion
Response Back

Take it to an Online Json Parser to make it easier on the eyes:

Online JSON Parser

We now see how MISP will respond to our API request and will use these fields within the response to tell Graylog what to collect. I recommend first understanding how the API is going to respond to your request prior to configuring Graylog.

Graylog

Similarly to Part Eight, we will configure the below within Graylog:

  • Data Adapter (Configure API Endpoint and JSONPath)
  • Cache (Configure caching)
  • Lookup Tables (Merge Data Adapter and Cache together)
  • Pipeline Rule (When Graylog makes the request and what to do with the response)

Create Data Adapter

Within Graylog, we first need to create a Data Adapter . The Data Adapter is where we configure the API request that will be made, such as the URL, Auth keys, Headers, etc.

  1. Navigate to System -> Lookup Tables and select Data Adapters .
  2. Create an HTTP JSONPath Data Adapter.
Data Adapter Configuration

3. Verify a successful request by testing a lookup for mb4z3nlfyrcjnoqf.onion

Successful Request

Create Data Cache

Another benefit with using Graylog is the built in Data Caching. Caching allows for us to limit the load put on the MISP server. Graylog will only invoke the MISP API if the value that Graylog is asking for is not within the Cache.

MISP Cache

Create Lookup Table

The lookup table component ties together a data adapter instance and a cache instance. It is needed to actually enable the usage of the lookup table in extractors, converters, pipeline functions and decorators.

  1. Navigate to System -> Lookup Tables and select Lookup Tables .
MISP Lookup Table

Creating Pipeline Rule

With our Lookup Table configured, we need to tell Graylog when we want to invoke the MISP API. This is achieved by creating a Pipeline Rule . For this example I will create a pipeline rule to invoke the MISP API when a file is added to our endpoint. I will take the syscheck_sha256_after field and use that value to build the API request to MISP. This will tell us if the file that was added to the system has a known malicious hash or not.

RAW File Added Log with Hash
  1. Navigate to System -> Pipelines and select Manage rules.
Manage Rules

2. Create the MISP Lookup on SHA256 HashRule

rule "OSSEC SYSCHECK ADDED - MISP THREAT INTEL"
when
$message.rule_group1 == "ossec" AND $message.rule_group2 == "syscheck" AND $message.syscheck_event == "added"
then
let ldata = lookup(
lookup_table: "misp_query",
key: to_string($message.syscheck_sha256_after)
);
set_fields(
fields: ldata,
prefix: "misp_"
);
end

3. Add the rule to Stage 1 in the Pipeline

Stage 1 Rule

Testing

Download an EICAR (https://www.eicar.org/download-anti-malware-testfile/) file onto an endpoint to test out the flow.

File Downloaded to Endpoint

Since this event matches our Pipeline rule of $message.rule_group1 == “ossec” AND $message.rule_group2 == “syscheck” AND $message.syscheck_event == “added” , the MISP API will be invoked and Graylog will add the results to the log. We can then view the result within Grafana.

Now we can quickly see that this downloaded file is potentially malicious and all this work was done automagically :).

Watch the video to see how we can build a Hyperlink within Grafana to quickly view the results in MISP 😉

You can now start to build dashboards, such as we did in Part 6, and alerts to notify the SOC team when MISP reports a positive hit!

Conclusion

Throughout this post we demonstrated how we can automate our Threat Intel with MISP’s API. However, we only created a Pipeline rule to trigger the API on File Uploaded events. See if you can apply this same principal to IPs, domains, etc. and begin exploring on your own! Happy Defending 😄.

Need Help?

The functionality discussed in this post, and so much more, are available via SOCFortress’s Professional Services. Let SOCFortress help you and your team keep your infrastructure secure.

Website: https://www.socfortress.co/

Professional Services: https://www.socfortress.co/ps.html

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
SOCFortress

SOCFortress is a SaaS company that unifies Observability, Security Monitoring, Threat Intelligence and Security Orchestration, Automation, and Response (SOAR).