Part 7. Firewall Log Collection Made Easy

SOCFortress

6 min readNov 20, 2022

Learn how to use Graylog to ingest syslog events into your SIEM

PART ONE: Backend Storage

PART TWO: Log Ingestion

PART THREE: Log Analysis

PART FOUR: Wazuh Agent Install

PART FIVE: Intelligent SIEM Logging

PART SIX: Best Open Source SIEM Dashboards

Walkthrough Video

Intro

Throughout this series we have successfully deployed EDR Agents (FOLLOW PART FOUR) to collect relative security logs from our Windows, Linux, and MacOS, endpoints. While endpoint log collection is a must for any SIEM stack, storing network logs is equally as important. Malicious traffic has to get to your endpoint devices somehow, and lacking insight into what traffic is coming and going through your network devices (firewalls, switches, wireless access points, etc.) leaves your SOC team wondering how that malicious traffic made its way into the environment and who may all be impacted.

Throughout this post we will ingest, parse, and enrich network logs collected from a pfSense firewall.

Architecture

Just about all network devices can be configured to enable remote logging which forwards their logs to a listening syslog device. We will configure our pfSense device to send logs to our listening Graylog server. Graylog will then parse and write the logs to our Wazuh-Indexer for storage and searchability.

Configure Graylog

Graylog is going to be the listener that will receive the logs from our pfSense firewall.

FOLLOW PART TWO TO DEPLOY YOUR OWN GRAYLOG SERVER

Graylog Input

We first need to deploy an Input on Graylog that will assign a listening port for our ingested logs. Looking at the pfSense documentation, we see that pfSense will forward UDP traffic:

Syslog over UDP

View your network device documentation to understand what protocol is used to forward logs. We must match this within our Graylog Input.

Navigate into System -> Inputs and launch a Raw/Plaintext UDP Input

2. Configure the Input. I am selecting port 5514, but you can configure to your port of choice.

3. Add a static field . This is important so that we can configure a Stream that will route only messages with the field and value of log_type: pfsense to our pfSense Stream (explained below).

Configure Network Device

Log into your pfSense device and navigate to Status -> System Logs -> Settings to configure our remote forwarding.

Set Log Message Format to BSD

2. Scroll down to the Remote Logging Options section and enable forwarding to our Graylog server and listening port (5514 in our example).

3. Save the configuration.

Our pfSense device is now forwarding logs to Graylog.

Graylog Extractor

We now see traffic hitting our Graylog Input.

Let’s view the messages by selecting Show received messages to verify that our firewall logs are being received.

From the screenshot above, we see our firewall logs being ingested but they are not being parsed how we need them to be. Without parsing and beautifying these messages, we won’t be able to build dashboards or integrate threat intel with our ingested firewall logs.

To solve this problem, let’s build Graylog Extractors for this Input that Graylog will use to parse the received logs and store them within their correct fields.

Head back over to your Input and select Manage Extractors .

2. Select Import extractors .

3. Paste in the below JSON.

{
  "extractors": [
    {
      "title": "pfSense filterlog: IPv4 TCP",
      "extractor_type": "regex",
      "converters": [
        {
          "type": "csv",
          "config": {
            "trim_leading_whitespace": false,
            "column_header": "RuleNumber,SubRuleNumber,Anchor,Tracker,Interface,Reason,Action,Direction,IPVersion,TOS,ECN,TTL,ID,Offset,Flags,ProtocolID,Protocol,Length,SourceIP,DestIP,SourcePort,DestPort,DataLength,TCPFlags,Sequence,ACK,Window,URG,Options",
            "strict_quotes": false
          }
        }
      ],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "FilterData",
      "extractor_config": {
        "regex_value": "^.*filterlog\\[.....\\]:(.*)$"
      },
      "condition_type": "regex",
      "condition_value": "^.*filterlog\\[.....\\]:(.*),(in|out),4,.*,tcp,.*$"
    },
    {
      "title": "pfSense filterlog: IPv4 UDP",
      "extractor_type": "regex",
      "converters": [
        {
          "type": "csv",
          "config": {
            "trim_leading_whitespace": false,
            "column_header": "RuleNumber,SubRuleNumber,Anchor,Tracker,Interface,Reason,Action,Direction,IPVersion,TOS,ECN,TTL,ID,Offset,Flags,ProtocolID,Protocol,Length,SourceIP,DestIP,SourcePort,DestPort,DataLength",
            "strict_quotes": false
          }
        }
      ],
      "order": 0,
      "cursor_strategy": "copy",
      "source_field": "message",
      "target_field": "FilterData",
      "extractor_config": {
        "regex_value": "^.*filterlog\\[.....\\]:(.*)$"
      },
      "condition_type": "regex",
      "condition_value": "^.*filterlog\\[.....\\]:(.*),(in|out),4,.*,udp,.*$"
    }
  ],
  "version": "4.2.7"
}

4. Select Add extractors to input .

Head back to the ingested messages and view the new extractors in action! Here we see Graylog parsing and assigning field names appropriately.

Store Logs into our Wazuh-Indexer

Graylog is now receiving and parsing our logs, but we need to configure Graylog to store these logs within an Index on our Wazuh-Indexer.

FOLLOW PART ONE TO DEPLOY A WAZUH-INDEXER

Create an Index

We first need to create an Index that will hold our ingested firewall logs.

Navigate to System -> Indices and select Create index setwithin the Graylog WebUI.

2. Set your Index options.

FOLLOW PART FIVE TO LEARN MORE ABOUT CONFIGURING AN INDEX

3. Save the Index.

The pfsense_0 index has now been created within our Wazuh-Indexer.

Create Stream

Let’s now configure a Stream within Graylog which will route our ingested pfSense logs to our newly created pfsense_0 index.

Navigate to Streams within the Graylog WebUI and Create Stream.

2. Configure your Stream to use the newly created Index.

3. Add a rule to the stream to only route logs with the log_type: pfsense field name to our pfSense Logs stream.

Manage Rules

4. Save and Start your Stream.

5. Select the stream to verify messages are being routed appropriately.

Notice the above screenshot shows our correct stream and index.

Conclusion

Throughout this post we configured our pfSense firewall to forward logs to our Graylog server. We then added Graylog extractors so that we could map our data to unique fields which we can then use for alerting and dashboard creation. Collecting network device logs is crucial for any security team looking to get full insight into traffic coming and going to and from their internal networks. So what are you waiting for? Go start ingesting your firewall logs with Graylog and achieve full visibility! Happy Defending 😄.

Need Help?

The functionality discussed in this post, and so much more, are available via SOCFortress’s Professional Services. Let SOCFortress help you and your team keep your infrastructure secure.

Website: https://www.socfortress.co/

Professional Services: https://www.socfortress.co/ps.html