Splunk & CIS Top 20 Security Monitoring

Placeholder Image
The purpose of this blog is to provide a quick overview on one can exploit Splunk and the Centre for Internet Security Critical Security Controls for Effective Cyber Defence best practises, otherwise known as CIS top 20.  The best practises consist of 20 key security controls (CSC), that an organisation could use to block or mitigate cyber-attacks and improve their security posture overall. The CIS CSC are ranked in order of overall importance and application to a corporate security strategy. For example, the first two controls, surrounding known inventory, are at the top of the list and are foundational in nature, ranking “very high” for attack mitigation.

The published top 20 controls are as follows:
CSC 1: Inventory of Authorized and Unauthorized Devices
CSC 2: Inventory of Authorized and Unauthorized Software
CSC 3: Secure Configurations for Hardware and Software on Mobile Devices, Laptops, Workstations, and Servers
CSC 4: Continuous Vulnerability Assessment and Remediation
CSC 5: Controlled Use of Administrative Privileges
CSC 6: Maintenance, Monitoring, and Analysis of Audit Logs
CSC 7: Email and Web Browser Protections
CSC 8: Malware Defences
CSC 9: Limitation and Control of Network Ports, Protocols, and Services
CSC 10: Data Recovery Capability
CSC 11: Secure Configurations for Network Devices such as Firewalls, Routers, and Switches
CSC 12: Boundary Defence
CSC 13: Data Protection
CSC 14: Controlled Access Based on the Need to Know
CSC 15: Wireless Access Control
CSC 16: Account Monitoring and Control
CSC 17: Security Skills Assessment and Appropriate Training to Fill Gaps
CSC 18: Application Software Security
CSC 19: Incident Response and Management
CSC 20: Penetration Tests and Red Team Exercises

  • Splunk software can be used to build and operate security operations centres of any size
  • Support the full range of information security operations, including posture assessment, monitoring, alert and incident handling, CSIRT, breach analysis and response, and event correlation
  • Out-of-the-box support for SIEM and security use cases
  • Detect known and unknown threats, investigate threats, determine compliance and use advanced security analytics for detailed insight
  • Proven integrated, big data-based security intelligence platform
  • Use ad hoc searches for advanced breach analysis
  • On-premises, cloud, and hybrid on-premises and cloud deployment options
  • Improve operational efficiency with automated and human-assisted decisions by using Splunk as a security nerve centre
  • Indexes data from any machine data source
  • Searches through machine data from a centralized console
  • Allows the security professional to add tags, create event types and correlate the incoming data with business context
  • Proactively monitors and alerts on security incidents, with automatic remediation of security issues – for example, changing a firewall rule in response to Splunk search results
  • Allows for the creation of reports, dashboards and other forms of analytics to communicate security information throughout the organization

So How Can Splunk Help?
Splunk has a free App that one can use to help provide security compliance, this is based on the CIS top 20 security controls as published by the Centre for Internet Security (CIS).

https://splunkbase.splunk.com/app/3064/

cis1

The app provides a data-agnostic framework which exploits Splunk’s data analytics features and functions such as data model, otherwise known as the Common information model (CIM). This features allows Splunk to be data-agnostic as it normalises data for common fields, such as source IP, user etc.

The App has been developed for anyone in the IT security segment, and Splunk Administrators. It relies on data being ingested into Splunk, this would be as part of the Splunk design and assessment process, such as identifying which sources of data should be ingested, example being Windows Active Directory Logs, Firewall Logs and so forth.
Once this data has been ingested and its CIM compliant, the app runs searches and populates the dashboards for CIS compliancy, this will provide Security teams with insights as to their compliancy status and improve their overall security posture.
There are hundreds of free Splunk TA’s (Splunk Technical Add-ons), these provide field extractions, lookups, tags, and event-types, these all help the Splunk CIS app for presenting the data.

If any customisation is required, lookup files and TA’s can be developed for any application that is required for CIS compliancy.

Splunk supports the controls in four ways:

Verification:
As Splunk software ingests data, it can generate reports and dashboards that show compliance or non-compliance with controls. Incidents of non-compliance can generate alerts to SOC personnel.

Execution:
In the case of an attack or non-compliance, Splunk software can carry out recommended actions to meet controls. With version 6.0 of the CIS CSC, Splunk software becomes even more critical, since control 14 surrounding audit logs has been promoted to position

Verification & Execution:
Data from third-party sources can be correlated with data ingested in Splunk software to meet the control.

Support:
The Splunk platform provides flexible features that help security professionals with controls that are largely policy and process based.

Mapping Example – CSC 5

Controlled use of admin privileges can be accomplished with a number of toolsets that restrict the use of administrative accounts. The simplest methods are OS-level tools, like sudo, and controls that can be put in place with vendor-supplied tools like Active Directory, so with this in mind you want to comply with CSC 5: Controlled Use of Administrative Privileges to your IT environment.
Splunk can help by consuming authentication logs from across the technology environment that detail account activity, including how accounts are being accessed and from where. Authentication logs come from, but are not limited to: host devices, domain controllers, directory servers, network devices, application logs and many others. All of this data will be ingested into Splunk software for searching and correlation.
Any use of known administrative accounts like “Administrator” and “root” and “sa” can easily be searched across the entire environment and reported or alerted upon.
The below is an example of a dashboard showing Successful Logins from 10 Most Rare Users – Privileged Accounts

cis2

So, Get Splunking, and the CIS App if you need to implement the CIS CSC, Splunk makes all data in your organization security relevant, as data is indexed by Splunk Enterprise, it becomes instantly searchable and security professionals can easily correlate all of these seemingly disparate data sources. Furthermore, the different data types can be seen in the context of data locked in business systems, which is often the key factor in determining correct root causes. Security professionals can then build dashboards and reports on top of the data, and set up actions and alerts to be executed on specific thresholds. In addition, any analysis can be operationalized to proactively protect your organization from an emerging threat.

Check to see who’s copying DATA from your restricted Linux servers!

Placeholder Image

There’s a lot of insiders that do a lot of copying and this is one method that could help you observe who’s copying the data when they shouldn’t be init!.

 Install the https://splunkbase.splunk.com/app/833/ which is the TA for collecting Linux OS data onto your restricted Linux servers. Once this has been ingested into Splunk, check the sourcetype and ensure the data is correct and the parsing is good.

TA version used there was 5.2.4

SPL = index=linux sourcetype=bash_history

cp1

If the data looks good, create a table, run a simple SPL search to check for any copy or running sudo command for this sourcetype.

SPL = index=linux sourcetype=bash_history sudo OR cp  | table _time,  user_name,  host, bash_command

cp2

From this you could enhance the table with some colours and see which user has been a very naughty boy OR girl!!!!

cp3

You could do other tables or charts which show data being deleted, which could be a disgruntled employee wanting to do some damage before they leave.

Done!

 

 

 

Splunk Stream Is Cool

Placeholder Image

Splunk steam once configured can monitor many protocols over the wire, so I wanted to see what I could get into Splunk.

I configured the stream app https://splunkbase.splunk.com/app/1809/  which includes a binary that captures the packets onto a number of test servers running the universal forwarder. In the real world you may use a tap port or use the independent Stream Forwarder which uses HEC, so you could ingest network data straight to it.

My config was on some test servers to capture the packets via the streamfwd binary.

Follow the Stream documentation for the config: https://docs.splunk.com/Documentation/StreamApp/7.1.2/DeployStreamApp/InstallSplunkAppforStream

Also deploy the Stream App onto the search head which provides the dashboards / props /transforms and configuration of the Stream App

So I wanted a simple check on icmp traffic, so I enabled the icmp protocol in the config in the Stream app Configuration > Configure Streams

stream2

I ran some ping checks and could see the data via a basic SPL:

index=dc_stream sourcetype=”stream:icmp” | table src_ip, dest_ip,  protocol_stack, bytes, bytes_in, bytes_out

stream3

 

I created a simple chart to see the data and which destination has had most icmp packets

SPL: index=dc_stream sourcetype=”stream:icmp” | timechart sum(bytes) as total_bytes by dest_ip

stream4

So this demonstrates how one can capture wire data and then run some SPL to get stats on network traffic your interested in.

Here’s some other stream data dashboards examples that you get.

DNS is a good, you could see how active the DNS server is.stream5

 

Done.

This app is helpful in getting wiredata into Splunk – go check it out

https://splunkbase.splunk.com/app/4372/

 

Monitor Linux RPM Installs With Splunk

Placeholder Image

This Splunk config will help you monitor which software packages are being installed on your critical Linux servers.

Watch for RPM packages being installed on some critical Linux Centos/RHEL servers, it could be an indication of someone not following change control or you could use it to monitoring change control and many other use cases to monitor such an event.

Before configuring the below you will need to ensure you have setup Splunk, indexes, uf’s and have some test Linux servers. You also need to have Splunk admin level skills, or be an experienced Splunker.

This config was performed on Centos 7.x servers and Splunk 7.x

Config:

  • A few fields were created using regex, this was done after analysing the logs
  • New tags and event types are created for this config.

Fields: action/ software

Eventypes: yum_packages
Tags: Installed /installed

Configure an inputs / eventypes / tags to monitor the yum log file:

inputs.conf (Deploy to the UF/Linux Server)

[monitor:///var/log/yum.log]
whitelist = (yum.log)
sourcetype = linux_yum
index = syslog
disabled = 0

props.conf (Deploy to the Search Head / Indexers)

[linux_yum]
CHARSET=UTF-8
TIME_PREFIX=^
MAX_TIMESTAMP_LOOKAHEAD=17
TIME_FORMAT=%b %d %H:%M:%S
REPORT-syslog=syslog-extractions
SHOULD_LINEMERGE=false
LINE_BREAKER =([\r\n]+)
KV_MODE = auto
NO_BINARY_CHECK = true
TRUNCATE = 9999
TRANSFORMS=syslog-host
disabled=false

#Extract and action field which is Installed and the software field which is the RPM package installed.
EXTRACT-action = ^(?:[^ \n]* )\d+\s\d+:\d+:\d+\s(?P<action>\w+)
EXTRACT-software = ^(?:[^ \n]* )\d+\s\d+:\d+:\d+\sInstalled:\s\d+:(?P<software>.*)

#normalise the action field as status
FIELDALIAS-action = action as status

Add the below to add event types and tags for the linux_yum sourcetype, this will help with CIM model compliance.

eventtypes.conf (Deploy to the Search Head / Indexers)

[yum_packages]
search = sourcetype= sourcetype=linux_yum
#tags = installed Installed

tags.conf (Deploy to the Search Head / Indexers)

[eventtype=yum_packages]
Installed = enabled
installed = enabled

After the data has been ingested, install some test RPM packages and run the below search, you should get a similar output as in the screenshot.

index=syslog sourcetype=linux_yum action=”Installed”
| rename software as installed_software_rpm
| fields _time, host, action, installed_software_rpm
| eval date=strftime(_time, “%d/%m/%Y %H:%M:%S”)
| stats count by date, action, host, installed_software_rpm

software

Done

Splunk Ports Check Scanner

Placeholder Image

Here’s a simple Splunk port scanning script I put together – its helped me when the ports required have not been opened on clusters members (indexers/search heads) and I was getting connection failed errors – so I thought I’d share this for those that may need to quickly check the Splunk port status in a multiple Splunk server enviroment – you can change the ports for your enviroment, should they have been changed from the default.

https://github.com/iopsmon/splunk_port_scanner

 

Splunk And Windows Security Logs

Placeholder Image

There’s a plethora of Windows Security events you can monitor, so the task can become overwhelming right, which ones etc, so these are just a few of the key ones
to start with and then you expand them as you go on, of course you can enable and ingest all of them, but you’ll need to keep an eye on the volume and daily quota limit as Windows logs can be very noisy.

So look at the ones below, and monitor these as a quick start set.

 

Good Microsoft Article On Which Events:
https://docs.microsoft.com/en-gb/windows-server/identity/ad-ds/plan/security-best-practices/audit-policy-recommendations#events-to-monitor

4719 System audit policy was changed.
4964 Special groups have been assigned to a new logon.
1102 The audit log was cleared.
4706 A new trust was created to a domain.
4724 An attempt was made to reset an account’s password.
4739 Domain Policy was changed.
4608 Windows Is Starting Up.
4609 Windows Is Shutting Down.
4625 An account failed to log on.
4648 Privilege Escalation.
4697 Attempt To Install a Service.
4700 A scheduled task was enabled.
4720 A user account was created.
4722 A user account was enabled
4723 An attempt was made to change an account’s password.
4725 A user account was disabled.
4726 A user account was deleted.
4728 A member was added to a security-enabled global group.
4731 A security-enabled local group was created.
4732 A member was added to a security-enabled local group.
4740 A user account was locked out.
4743 A computer account was deleted.
4767 A user account was unlocked.

This is is simple Windows dashboard example you can produce from the security events

win

 

Good Splunk Links

Placeholder Image

Here are some useful Splunk links

[Splunk Apps and TA’s]
https://splunkbase.splunk.com/

[Splunk Disk Capacity Sizing]
https://splunk-sizing.appspot.com/

[Splunk Forum (Great place for questions and answers)]
https://answers.splunk.com/index.html

[Splunk data on boarding guide – good for understanding about data]
https://conf.splunk.com/files/2017/slides/data-onboarding-where-do-i-begin.pdf

[Splunk wiki]
http://wiki.splunk.com/Main_Page

[Splunk Reference]
https://www.splunk.com/pdfs/solution-guides/splunk-quick-reference-guide.pdf

[Splunk Quick Command Reference]
https://wiki.splunk.com/images/a/a3/Splunk_4.x_cheatsheet.pdf

[Splunk hot/cold/warm data process]
https://conf.splunk.com/files/2017/slides/splunk-data-life-cycle-determining-when-and-where-to-roll-data.pdf

[Data Onboarding Cheet Sheet]
https://www.aplura.com/wp-content/uploads/2016/09/data_onboarding_cheat_sheet_v2.pdf