Splunk / Python Script / Syslog Demo Data

Placeholder Image

Officially now a Splunk Certified User: (Splunker!)

bg1

With that in mind I thought I’d create a demo script to load some log data into Splunk, this is to show the data and some charts.

As the data is based on time series (Time Stamped) you’ll be able to see information from the log entries and a few charts.

The use case could be to show all syslog data from all pre-defined critical servers, then run searches against the data for particular log events / messages.

A typical search could be “Failed from ip_address”

I created a python script, which generates a log file called dc_security.log, the script has two options, one to quickly generate data or one with some delays so you can leave it for a while to build the data and show this over time in Splunk. The log data is based on syslog format and have put various messages in the log events.

Example Data:

Dec 2 02:51:31 LINUX_SRV3 user joker has tried to login to this server and failed from ip_address 70.70.70.21

After you download the script run the script – sudo python ./dc_security_v1.0.py and select either option.

Download the script from Github

https://github.com/iopsmon/log_data_generator

sp1

After the file has been created, copy the file: dc_security.log to a server from where you can access Splunk web gui and upload the data into an index called dcsecurity or an index of your choice.

Splunk > Add Data > Upload Data > drag the file into the target box > next

sp2

Set the source type – as the log file is like syslog entries, choose > Operating systems > linux_message_syslog

Press Next > Review > Submit > Start Searching

From the search type the below query and you should get a display of all the logs from that file

sourcetype=linux_messages_syslog

sp3.jpg

Here’s a few charts based on the data

Search 1: This searches for the message failed AND from ipaddress:

sourcetype=linux_messages_syslog failed AND from ip_address

sp4

 

Search 2 : This search query shows which server has had the most failed login attempts You can see there are 3500 events, this could be an IOC (Indicator Of  Compromise) like a brute force attack.

It’s also showing the source of the IP, based on this you would run your security process and actions.

sourcetype=linux_messages_syslog failed AND from ip_address | top host

sp5

Hopefully this shows you how to get some log data into Splunk, then run some quick searches, and create some charts for insights.

In a production environment you would use forwarders to collect and forward the syslog data on an ongoing basis.

 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s