Part 7. Firewall Log Collection Made Easy

Learn how to use Graylog to ingest syslog events into your SIEM

Walkthrough Video

Intro

Throughout this series we have successfully deployed EDR Agents (FOLLOW PART FOUR) to collect relative security logs from our Windows, Linux, and MacOS, endpoints. While endpoint log collection is a must for any SIEM stack, storing network logs is equally as important. Malicious traffic has to get to your endpoint devices somehow, and lacking insight into what traffic is coming and going through your network devices (firewalls, switches, wireless access points, etc.) leaves your SOC team wondering how that malicious traffic made its way into the environment and who may all be impacted.

Throughout this post we will ingest, parse, and enrich network logs collected from a pfSense firewall.

Architecture

Just about all network devices can be configured to enable remote logging which forwards their logs to a listening syslog device. We will configure our pfSense device to send logs to our listening Graylog server. Graylog will then parse and write the logs to our Wazuh-Indexer for storage and searchability.

High Level Flow of Traffic

Configure Graylog

Graylog is going to be the listener that will receive the logs from our pfSense firewall.

FOLLOW PART TWO TO DEPLOY YOUR OWN GRAYLOG SERVER

Graylog Input

We first need to deploy an Input on Graylog that will assign a listening port for our ingested logs. Looking at the pfSense documentation, we see that pfSense will forward UDP traffic:

Syslog over UDP

View your network device documentation to understand what protocol is used to forward logs. We must match this within our Graylog Input.

  1. Navigate into System -> Inputs and launch a Raw/Plaintext UDP Input
Launch UDP Input

2. Configure the Input. I am selecting port 5514, but you can configure to your port of choice.

Configure Input
Running Input

3. Add a static field . This is important so that we can configure a Stream that will route only messages with the field and value of log_type: pfsense to our pfSense Stream (explained below).

Static Field
Static Field

Configure Network Device

Log into your pfSense device and navigate to Status -> System Logs -> Settings to configure our remote forwarding.

pfSense Remote Forwarfing Configuration
  1. Set Log Message Format to BSD
BSD Format

2. Scroll down to the Remote Logging Options section and enable forwarding to our Graylog server and listening port (5514 in our example).

Remote Logging Configuration

3. Save the configuration.

Our pfSense device is now forwarding logs to Graylog.

Graylog Extractor

We now see traffic hitting our Graylog Input.

Graylog Receiving Logs.

Let’s view the messages by selecting Show received messages to verify that our firewall logs are being received.

pfSense Messages

From the screenshot above, we see our firewall logs being ingested but they are not being parsed how we need them to be. Without parsing and beautifying these messages, we won’t be able to build dashboards or integrate threat intel with our ingested firewall logs.

To solve this problem, let’s build Graylog Extractors for this Input that Graylog will use to parse the received logs and store them within their correct fields.

  1. Head back over to your Input and select Manage Extractors .
Manage Extractors

2. Select Import extractors .

Import extractors

3. Paste in the below JSON.

{
"extractors": [
{
"title": "pfSense filterlog: IPv4 TCP",
"extractor_type": "regex",
"converters": [
{
"type": "csv",
"config": {
"trim_leading_whitespace": false,
"column_header": "RuleNumber,SubRuleNumber,Anchor,Tracker,Interface,Reason,Action,Direction,IPVersion,TOS,ECN,TTL,ID,Offset,Flags,ProtocolID,Protocol,Length,SourceIP,DestIP,SourcePort,DestPort,DataLength,TCPFlags,Sequence,ACK,Window,URG,Options",
"strict_quotes": false
}
}
],
"order": 0,
"cursor_strategy": "copy",
"source_field": "message",
"target_field": "FilterData",
"extractor_config": {
"regex_value": "^.*filterlog\\[.....\\]:(.*)$"
},
"condition_type": "regex",
"condition_value": "^.*filterlog\\[.....\\]:(.*),(in|out),4,.*,tcp,.*$"
},
{
"title": "pfSense filterlog: IPv4 UDP",
"extractor_type": "regex",
"converters": [
{
"type": "csv",
"config": {
"trim_leading_whitespace": false,
"column_header": "RuleNumber,SubRuleNumber,Anchor,Tracker,Interface,Reason,Action,Direction,IPVersion,TOS,ECN,TTL,ID,Offset,Flags,ProtocolID,Protocol,Length,SourceIP,DestIP,SourcePort,DestPort,DataLength",
"strict_quotes": false
}
}
],
"order": 0,
"cursor_strategy": "copy",
"source_field": "message",
"target_field": "FilterData",
"extractor_config": {
"regex_value": "^.*filterlog\\[.....\\]:(.*)$"
},
"condition_type": "regex",
"condition_value": "^.*filterlog\\[.....\\]:(.*),(in|out),4,.*,udp,.*$"
}
],
"version": "4.2.7"
}

4. Select Add extractors to input .

Head back to the ingested messages and view the new extractors in action! Here we see Graylog parsing and assigning field names appropriately.

Parsed Messages

Store Logs into our Wazuh-Indexer

Graylog is now receiving and parsing our logs, but we need to configure Graylog to store these logs within an Index on our Wazuh-Indexer.

FOLLOW PART ONE TO DEPLOY A WAZUH-INDEXER

Create an Index

We first need to create an Index that will hold our ingested firewall logs.

  1. Navigate to System -> Indices and select Create index setwithin the Graylog WebUI.
Create Index Set

2. Set your Index options.

FOLLOW PART FIVE TO LEARN MORE ABOUT CONFIGURING AN INDEX

Index Options
Index Options Continued.

3. Save the Index.

The pfsense_0 index has now been created within our Wazuh-Indexer.

pfsense_0 index

Create Stream

Let’s now configure a Stream within Graylog which will route our ingested pfSense logs to our newly created pfsense_0 index.

  1. Navigate to Streams within the Graylog WebUI and Create Stream.
Create Stream

2. Configure your Stream to use the newly created Index.

Configure Stream

3. Add a rule to the stream to only route logs with the log_type: pfsense field name to our pfSense Logs stream.

Manage Rules
Stream Rule

4. Save and Start your Stream.

5. Select the stream to verify messages are being routed appropriately.

Stream Routing

Notice the above screenshot shows our correct stream and index.

Conclusion

Throughout this post we configured our pfSense firewall to forward logs to our Graylog server. We then added Graylog extractors so that we could map our data to unique fields which we can then use for alerting and dashboard creation. Collecting network device logs is crucial for any security team looking to get full insight into traffic coming and going to and from their internal networks. So what are you waiting for? Go start ingesting your firewall logs with Graylog and achieve full visibility! Happy Defending 😄.

Need Help?

The functionality discussed in this post, and so much more, are available via SOCFortress’s Professional Services. Let SOCFortress help you and your team keep your infrastructure secure.

Website: https://www.socfortress.co/

Professional Services: https://www.socfortress.co/ps.html

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
SOCFortress

SOCFortress is a SaaS company that unifies Observability, Security Monitoring, Threat Intelligence and Security Orchestration, Automation, and Response (SOAR).