Convert UTC time to EST

Example to convert UTC to EST time format

Use case: Convert AWS S3 logs to EST format

| eval time=strftime(strptime(utc_time_field,"%Y-%m-%dT%H:%M:%S.%N%Z"),"%Y-%m-%d %I:%M:%S.%N_%Z") | table utc_time_field, time

Deploy a heavy forwarder on Linux

Download Splunk Enterprise 64 bit
Download splunk-6.3.1-f3e41e4b37b2-Linux-x86_64 2.6+ kernel Linux distributions (64-bit)

Extract using the command  gzip -dc file.tgz | tar xf -
This will create a splunk directory

Start Splunk
bin/splunk start
Accept license
splunk will start and listen on port 8000

from the browser, hit splunk:8000
default login: admin/changeme
change the password
activate license
restart

Display search results in column format and sort

index="myindex" "searchstring" | eval fields=split(_raw," ") | eval field1=mvindex(fields, 7) | eval field2=mvindex(fields, 10) | table _time, host, field1, field2 | sort field2

Do not index - send data to null queue from specific IPs

There is a use case to exclude logs generated using white security scans. A list of scanner IPS have been provided.

Solution
Create props.conf
 [elasticbeanstalk-access_log-too_small]
TRANSFORMS-null-whitehat= wht-null-addr1, wht-null-addr2

Create transforms.conf
##This transform nullqueues event containing ip addresses in CIDR range 65.122.153.0/27
[wht-null-addr1]
REGEX = 65\.122\.153\.[012]\d?|65\.122\.153\.3[01]|65\.122\.153\.\d\s
DEST_KEY = queue
FORMAT = nullQueue

##This transform nullqueues event containing ip addresses
##23.56.192.166 or 10.11.254.21 or 10.11.153.45
[wht-null-addr2]
REGEX = 23\.56\.192\.166|10\.11\.254\.21|10\.11\.153\.45
DEST_KEY = queue
FORMAT = nullQueue

Place these files in /etc/apps/search directory in Splunk indexers.
Restart Splunk indexer