PART ONE: Backend Storage
PART TWO: Log Ingestion
PART THREE: Log Analysis
PART FOUR: Wazuh Agent Install
PART FIVE: Intelligent SIEM Logging
PART SIX: Best Open Source SIEM Dashboards
PART SEVEN: Firewall Log Collection Made Easy
PART EIGHT: Firewall Threat Intel With GreyNoise
PART NINE: Log Normalization
PART TEN: MISP Threat Intel
SIEM Stacks can often feel as if you are trying to drink water from a fire hose. You deploy a few monitoring agents and you are able to tread water and get by just fine. However, once you start monitoring hundreds to thousands of servers, you quickly drown in log overflow. This makes it extremely difficult for our SOC team to correlate threat intel with every network connection, every process hash, every file uploaded that have occurred on our endpoints. The mundane task of manually inspecting every event sounds like a nightmare. No wonder SOC turnover is so high!
In Part Ten of our series we installed MISP, the open source threat sharing intelligence platform. We enriched MISP with events that contain IoCs such as malicious:
MISP has a strong API that we will use via Graylog to enrich our logs on the fly with Threat Intel. Taking the need for manual analysis out of the mix!
The beauty with Graylog is that we are able to enrich logs on the fly, before they are written to the Wazuh Indexer for storage. We explored an example of this in further detail in Part Nine with our log normalization. Graylog is also able to make HTTP Posts to API endpoints and use the received response to populate the log. This is perfect for our use case because we want Graylog to use MISP’s API to inspect values contained within our selected fields and use the response to populate the log even further.
The flow of data is shown below:
Fluent-Bit that is running on the Wazuh Manager will foward the
alerts.json file to Graylog. Graylog will strip out the IP address, Domain, Hash, etc. and build an API request to send to MISP. MISP will respond back to Graylog and include event details if the value included in the Graylog request exists within MISP. Graylog will take the MISP response, and new fields to the log and write that log into the
Let’s see how the MISP API works before including it within Graylog.
- Create API Key for Authentication
Administration -> List Auth Keys
2. Make Request via Curl (I am not using HTTPS in this demo)
curl -H "Accept: application/json" -H "Content-Type: application/json" -H "Authorization: **MISP_API_KEY*" http://misp.socfortress.co/attributes/restSearch/value:mb4z3nlfyrcjnoqf.onion
Take it to an Online Json Parser to make it easier on the eyes:
We now see how MISP will respond to our API request and will use these fields within the response to tell Graylog what to collect. I recommend first understanding how the API is going to respond to your request prior to configuring Graylog.
Similarly to Part Eight, we will configure the below within Graylog:
- Data Adapter (Configure API Endpoint and JSONPath)
- Cache (Configure caching)
- Lookup Tables (Merge Data Adapter and Cache together)
- Pipeline Rule (When Graylog makes the request and what to do with the response)
Create Data Adapter
Within Graylog, we first need to create a
Data Adapter . The
Data Adapter is where we configure the API request that will be made, such as the URL, Auth keys, Headers, etc.
- Navigate to System -> Lookup Tables and select
- Create an
HTTP JSONPathData Adapter.
3. Verify a successful request by testing a lookup for
Create Data Cache
Another benefit with using Graylog is the built in
Data Caching. Caching allows for us to limit the load put on the MISP server. Graylog will only invoke the MISP API if the value that Graylog is asking for is not within the Cache.
Create Lookup Table
The lookup table component ties together a data adapter instance and a cache instance. It is needed to actually enable the usage of the lookup table in extractors, converters, pipeline functions and decorators.
- Navigate to System -> Lookup Tables and select
Creating Pipeline Rule
With our Lookup Table configured, we need to tell Graylog when we want to invoke the MISP API. This is achieved by creating a
Pipeline Rule . For this example I will create a pipeline rule to invoke the MISP API when a file is added to our endpoint. I will take the
syscheck_sha256_after field and use that value to build the API request to MISP. This will tell us if the file that was added to the system has a known malicious hash or not.
- Navigate to System -> Pipelines and select
2. Create the
MISP Lookup on SHA256 HashRule
rule "OSSEC SYSCHECK ADDED - MISP THREAT INTEL"
$message.rule_group1 == "ossec" AND $message.rule_group2 == "syscheck" AND $message.syscheck_event == "added"
let ldata = lookup(
3. Add the rule to
Stage 1 in the Pipeline
Download an EICAR (https://www.eicar.org/download-anti-malware-testfile/) file onto an endpoint to test out the flow.
Since this event matches our Pipeline rule of
$message.rule_group1 == “ossec” AND $message.rule_group2 == “syscheck” AND $message.syscheck_event == “added” , the MISP API will be invoked and Graylog will add the results to the log. We can then view the result within Grafana.
Now we can quickly see that this downloaded file is potentially malicious and all this work was done automagically :).
Watch the video to see how we can build a Hyperlink within Grafana to quickly view the results in MISP 😉
You can now start to build dashboards, such as we did in Part 6, and alerts to notify the SOC team when MISP reports a positive hit!
Throughout this post we demonstrated how we can automate our Threat Intel with MISP’s API. However, we only created a Pipeline rule to trigger the API on File Uploaded events. See if you can apply this same principal to IPs, domains, etc. and begin exploring on your own! Happy Defending 😄.