Part 6. Best Open Source SIEM Dashboards

SOCFortress

9 min readNov 14, 2022

Visualize your SIEM logs with Grafana

PART ONE: Backend Storage

PART TWO: Log Ingestion

PART THREE: Log Analysis

PART FOUR: Wazuh Agent Install

PART FIVE: Intelligent SIEM Logging

Walkthrough Video

Intro

In PART FIVE we configured Graylog to beautify our ingested Wazuh logs by extracting the JSON keys and their values. Without this step, we are not able to build precise dashboards or widgets that enable us to visualize our data in a manner that allows us to understand it and make quick decisions. Throughout this post, we will learn how to turn our data from:

Welcome Grafana

Grafana is the perfect visualization tool when it comes to visualizing our security events. Kibana (Wazuh-Dashboards) can also be used to visualize our data, but over the years I have not been impressed with Kibana’s visualizations, difficulty to customize, lack of unique data sources, and overall speed. In my opinion, Grafana is the best visualization tool for all SIEM stacks.

Visualization Panels

Visualization panels are the building blocks that make up our dashboards. Dashboards are simply a combination of visualization panels pieced together to form a concise and accurate representation of the intent of the dashboard. Grafana’s wide range of visualization panels provides more flexibility and ease of “understanding the big picture” than is offered in Kibana.

The Grafana community is strong and the ability to import prebuilt dashboards built by others in a matter of seconds allows us to start visualizing our data in no time!

Find your perfect dashboard here: https://grafana.com/grafana/dashboards/

Limitless Customization

Customization is limitless with Grafana’s provided ability for us to write our own HTML and CSS panels. Grafana makes it simple for us to add our own branding and unique feel to our dashboards!

Unique Data Sources

Often times, the data that we want to display within our dashboards does not reside in just one datasource. I could have my SIEM logs being written to Elasticsearch, a MySQL database storing my threat intel, and a CSV file storing CNs and their associated DCs. Rather than trying to funnel that data into Elasticsearch, we can configure multiple data sources within Grafana with ease!

I Want to go Fast

In the Security World, we do not have time to wait. Our visualizations and searches must load in a matter of seconds, no matter their complexity. Grafana’s speed to present the data back to us is unmatched. Although Grafana and Kibana are both simply making API queries to Elasticsearch to load the data, I have found that Grafana’s response time blows Kibana out of the water!

Installation

Let’s now install Grafana. This guide details installing the OSS version of Grafana on a Debian 11 machine.

sudo apt-get install -y apt-transport-https
sudo apt-get install -y software-properties-common wget
sudo wget -q -O /usr/share/keyrings/grafana.key https://apt.grafana.com/gpg.key

Add the repository:

echo "deb [signed-by=/usr/share/keyrings/grafana.key] https://apt.grafana.com stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

sudo apt-get update
sudo apt-get install grafana

Open the /etc/grafana/grafana.ini file to apply configuration settings to your Grafana instance. Here we can configure our authentication mechanism, HTTPS certificates, and much more!

Adding Certificates

#################################### Server ####################################
[server]
# Protocol (http, https, h2, socket)
protocol = https

# The ip address to bind to, empty will bind to all interfaces
;http_addr =

# The http port  to use
;http_port = 3000

# The public facing domain name used to access grafana from a browser
domain = grafana.yourdomain.com

# Redirect to correct domain if host header does not match domain
# Prevents DNS rebinding attacks
;enforce_domain = false

# The full public facing url you use in browser, used for redirects and emails
# If you use reverse proxy and sub path specify full url (with sub path)
;root_url = %(protocol)s://%(domain)s:%(http_port)s/
root_url = %(protocol)s://%(domain)s/

# Serve Grafana from subpath specified in `root_url` setting. By default it is set to `false` for compatibility reasons.
;serve_from_sub_path = false

# Log web requests
;router_logging = false

# the path relative working path
;static_root_path = public

# enable gzip
;enable_gzip = false

# https certs & key file
cert_file = /etc/ssl/certs/*grafanacert*.pem
cert_key = /etc/ssl/private/*grafanakey*.key

# Unix socket path
;socket =

# CDN Url
;cdn_url =

# Sets the maximum time using a duration format (5s/5m/5ms) before timing out read of an incoming request and closing idle connections.
# `0` means there is no timeout for reading the request.
;read_timeout = 0

#################################### Database ####################################

ENSURE GRAFANA HAS PERMISSIONS TO READ YOUR CERT AND KEY

Let’s now start up Grafana

systemctl start grafana-server

Configuring Our Wazuh Indexer Data Source

With Grafana installed and running, let’s now configure Grafana so that it can read our ingested SIEM logs stored within our Wazuh Indexer.

Select Configuration -> Data Sources

2. Select Elasticsearch — Remember that at the end of the day the Wazuh-Indexer is Elasticsearch 7.10.2

3. Set your Elasticsearch connection settings (see walkthrough video to configure grafana user within our Wazuh Indexer)

Above URL blocks:

https://nvd.nist.gov/vuln/detail/${__value.raw}

https://grafana.*yourdomain*.com/explore?left=%5B%22now-6h%22,%22now%22,%22WAZUH%22,%7B%22refId%22:%22A%22,%22query%22:%22_id:${__value.raw}%22,%22alias%22:%22%22,%22metrics%22:%5B%7B%22id%22:%221%22,%22type%22:%22logs%22,%22settings%22:%7B%22limit%22:%22500%22%7D%7D%5D,%22bucketAggs%22:%5B%5D,%22timeField%22:%22timestamp%22%7D%5D

4. Save and Test

5. Select Explore to ensure Grafana is able to load the data

Create Your First Dashboard

Now that we are able to use Grafana to load our data, let’s begin to visualize our data by creating our first dashboard.

For instance, I want to build a dashboard that details all network connections being made from my endpoints. I want to build a dashboard that details the source, destination, and connected to port of every network connection. There are many different panels we could choose from, but I have found that the Sankey Panel is easiest on the eyes.

Sankey Panel

Add a new dashabord and a new panel

2. Select the Sankey Panel

If not installed, install via the commandline: grafana-cli plugins install netsage-sankey-panel

3. Select the WAZUH data source

4. Build your search query — Our Wazuh Rule group sysmon_event3 pertain to all network connections made via our Windows endpoints. View the walkthrough video to see how we can parse our rule_groups to set the rule_group3 value.

rule_group3:sysmon_event3

5. Add the data fields we want to use to populate the panel

6. Use the Organize Fields Transform to clean up the field names

7. Set your panel title and Apply

That’s easy enough, but I want more on my dashboard.

Worldmap Panel (Follow Walkthrough video to enable the Geo IP Lookup Table)

Let’s add some geolocation of our destination IPs so we can spot network connections made to any countries of concern (Russia is always a popular choice)

Add GeoLite DB to your Graylog server:

cd /etc/graylog/server
wget https://github.com/socfortress/Wazuh-Rules/releases/download/1.0/GeoLite2-City.mmdb
wget https://github.com/socfortress/Wazuh-Rules/releases/download/1.0/GeoLite2-ASN.mmdb
systemctl restart graylog-server

Follow Walkthrough video to enable the Geo IP Lookup Table

Add a Worldmap Panel and select the Wazuh Data Source

2. Build our Search Query and Groupings

3. Set your Panel Title and Apply

Our dashboard is starting to come together!

Network Connections Histogram

Let’s build a panel that details a history of the number of network connections made per endpoint.

Select Graph and the Wazuh Data Source

2. Set the Panel Query and Grouping

3. Set Panel Options

4. Set your Title and Apply

Processes Invoking Network Connections

From a security perspective, it is benefical to know what process is invoking the network connection. For example, I expect Chrome to be making network connections, but other processes such as Powershell may be of concern.

Add a Bar Gauge panel

2. Build the search query

3. Set the panel options

4. Set Panel Title and Apply

Integrate with Talos Intelligence to View IP Reputation

We can build our panels with dynamic data links that allow us to plug the value of the field into a URL that will be populated with our data value. We will use a data link to build a URL that will open a tab to view the IP reputation within Talos Intelligence. This allows us to quickly spot known malicious IPs that our endpoints have connected to.

Add a Table panel

2. Build the search query

3. Add a Data Override

https://talosintelligence.com/reputation_center/lookup?search=${__value.text}

4. Add your Panel Title and Apply

Select a linked IP to view the Threat Intel presented by Talos

Notice that our URL is dynamically built for us!

Rearranging The Visualizations

With our panels now built, we can freely drag and drop our panels to arrange them in a manner that is logical and easy to follow.

Just the Beginning

With Grafana the possibilities are truly endless. We can build dashboards that detail DNS queries, running process, alert summaries, and so much more!

Check out the SOCFortress dashboards that come out of the box to see what all is possible!

VIEW SOCFORTRESS DASHBOARDS

Conclusion

Throughout this blog post we installed Grafana, connected Grafana to our Wazuh data source, and walked through how we can build panels to populate our Network Connections dashboard. This post only touches the surface of what Grafana has to offer. So what are you waiting for? Enable your SOC team to visualize your ingested SIEM logs with the speed, flexibility, and customization of Grafana! Happy Defending 😄.

Need Help?

The functionality discussed in this post, and so much more, are available via SOCFortress’s Professional Services. Let SOCFortress help you and your team keep your infrastructure secure.

Website: https://www.socfortress.co/

Professional Services: https://www.socfortress.co/ps.html