在嘗試調整 regular expression 無果後，決定從 NXLog 著手，在 NXLog 送出前，先將 (上午|下午) 調整成 (AM|PM)，再將 log 送出。
然後，又發現在 SIEM 查詢時，下午的 windns log 都沒有出現，檢查發現在 OSSIM 的 plugin - windns-nxlog.cfg 中，其 date 欄位有先經過 normalize_date() function 處理，而 2018/6/8 06:17:49 PM 這個格式不符其支援的格式，故須修改成 6/8/2018 06:17:49 PM 這種格式。
以下是中文版與英文版所產出之 DNS debugging log。
----------------------------------------------------------------------------------------------------------------------
Windows DNS log from Windows 2008 R2 CHT:
2018/4/16 上午 06:17:49 0B38 PACKET 0000000002134EF0 UDP Rcv 192.168.3.252 3780 Q [0001 D NOERROR] A (6)amazon(3)com(0)
2018/4/16 上午 06:17:49 0B38 PACKET 0000000002EA9D30 UDP Snd 168.95.1.1 0fc0 Q [0001 D NOERROR] A (6)amazon(3)com(0)
2018/4/16 上午 06:17:49 0B38 PACKET 000000000212DAB0 UDP Rcv 168.95.1.1 0fc0 R Q [8081 DR NOERROR] A (6)amazon(3)com(0)
2018/4/16 上午 06:17:49 0B38 PACKET 0000000002134EF0 UDP Snd 192.168.3.252 3780 R Q [8081 DR NOERROR] A (6)amazon(3)com(0)
-----------------------------------------------------------------------------------------------------------------------
Windows DNS log from Windows 2008 R2 Eng:
2/19/2015 10:03:57 PM 2AE8 PACKET 00000005CF374F80 UDP Rcv 192.168.0.42 fdd7 Q [0001 D NOERROR] A (9)imap-mail(7)outlook(3)com(0)
2/19/2015 10:03:57 PM 2AE8 PACKET 00000005CB426930 UDP Snd 10.255.176.137 0c4c Q [0001 D NOERROR] A (9)imap-mail(7)outlook(3)com(0)
2/19/2015 10:03:57 PM 2AE8 PACKET 00000005D03B4CE0 UDP Rcv 10.255.176.137 0c4c R Q [8081 DR NOERROR] A (9)imap-mail(7)outlook(3)com(0)
2/19/2015 10:03:57 PM 2AE8 PACKET 00000005D03B4CE0 UDP Snd 192.168.1.42 fdd7 R Q [8081 DR NOERROR] A (9)imap-mail(7)outlook(3)com(0)
-----------------------------------------------------------------------------------------------------------------------
調整 nxlog.conf 設定如下:
======================================================================
< extension json >
Module xm_json
< /extension >
## Custom CSV format for the windns-nxlog AlienVault plugin.
< extension transform_alienvault_csv_dns >
Module xm_csv
Fields $Hostname, $SourceName, $Message
FieldTypes string, string, string
Delimiter ,
< /extension >
< input dns_logs >
Module im_file
File "C:\\temp\\DNSLog\\debug.log"
SavePos TRUE
InputType LineBased
#根據 OSSIM 的 Date Format supported by normalize_date() function，調整 Date Format
#Convert the Chinese characters into English characters (上午 to AM、下午 to PM)
#Change the datetime format from "yyyy/mm/dd (上午|下午) hh:mm:ss" to "mm/dd/yyyy hh:mm:ss (AM|PM)"
#Convert the Chinese characters into English characters (上午 to AM、下午 to PM)
#Change the datetime format from "yyyy/mm/dd (上午|下午) hh:mm:ss" to "mm/dd/yyyy hh:mm:ss (AM|PM)"
Exec if $raw_event =~ /^(\d+)\/(\d+\/\d+)\s(上午\s)(\d+\:\d+\:\d+\s)(.+)/ $raw_event = $2 + '/' + $1 + ' ' + $4 + 'AM ' + $5;
Exec if $raw_event =~ /^(\d+)\/(\d+\/\d+)\s(下午\s)(\d+\:\d+\:\d+\s)(.+)/ $raw_event = $2 + '/' + $1 + ' ' + $4 + 'PM ' + $5;
Exec if $raw_event !~ /^\d/ drop();\
else\
{\
$Message = $raw_event;\
$SourceName = "DNS";\
$raw_event = to_json();\
}
< /input >
< output out_alienvault_dns_nxlog >
Module om_udp
Host 192.168.1.2
Port 514
Exec if not defined $Message { drop(); }
## Replace newlines, tabs and carriage returns with blanks:
Exec $Message = replace($Message, "\t", " "); $Message = replace($Message, "\n", " "); $Message = replace($Message, "\r", " ");
## Ensure that commonly undefined values are set:
Exec if not defined $AccountName { $AccountName = "-"; }
Exec if not defined $AccountType { $AccountType = "-"; }
Exec if not defined $Domain { $Domain = "-"; }
## Ensure we send in the proper format:
Exec $Hostname = hostname_fqdn();
Exec transform_alienvault_csv_dns->to_csv(); $raw_event = $Hostname + ' DNS-NXLOG: ' + $raw_event;
< /output >
## Route for dns nxlog logs:
< route route_dns_nxlog >
Path DNS_Logs => out_alienvault_dns_nxlog
< /route >
======================================================================
OSSIM中查看rawlog，原本輸出為
Jun 8 00:07:48 DNS.test.corp DNS-NXLOG: "DNS.test.corp","DNS","2018/6/8 ?W?? 12:09:02 0B38 PACKET 00000000029DDCE0 UDP Rcv 192.168.3.31 7f19 Q [0001 D NOERROR] A (9)resolver1(5)ofind(6)ctmail(3)com(0)"
Jun 8 00:07:48 DNS.test.corp DNS-NXLOG: "DNS.test.corp","DNS","2018/6/8 ?W?? 12:09:02 0B38 PACKET 00000000029DDCE0 UDP Snd 192.168.3.31 7f19 R Q [8081 DR NOERROR] A (9)resolver1(5)ofind(6)ctmail(3)com(0)"
進行 Pre-parsing 後變為
Jun 8 15:43:09 DNS.test.corp DNS-NXLOG: "DNS.test.corp","DNS","6/8/2018 03:42:58 PM 0B34 PACKET 00000000020B9D10 UDP Rcv 192.168.3.31 afdd Q [0001 D NOERROR] A (13)gmail-smtp-in(1)l(6)google(3)com(0)"
Jun 8 15:43:09 DNS.test.corp DNS-NXLOG: "DNS.test.corp","DNS","6/8/2018 03:42:58 PM 0B34 PACKET 00000000020B9D10 UDP Snd 192.168.3.31 afdd R Q [8081 DR NOERROR] A (13)gmail-smtp-in(1)l(6)google(3)com(0)"
再回到 OSSIM web介面，ANALYSIS \ SECURITY EVENTS (SIEM) 下，DATA SOURCE 下拉選項出現 windns，表示 DNS Log 已成功寫入資料庫了!!
備註：
OSSIM Date Format supported by normalize_date() function:
https://www.alienvault.com/documentation/usm-appliance/plugin-management/supported-formats-normalize_date.htm
技術參考來源: http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html
Rewriting and modifying messages
There are many ways to modify log messages. A simple method which does not always work is to modify the $raw_event field (in case of syslog) without parsing the message. This can be done with regular expressions using capturing, for example:
if $raw_event =~ /^(aaaa)(replaceME)(.+)/ $raw_event = $1 + 'replaceMENT' + $3;
The more complex method is to parse the message into fields, modify some fields and finally reconstruct the message from the fields. The conditional rewrite of the syslog facility example shows such a syslog message modification method.
備註：
OSSIM Date Format supported by normalize_date() function:
https://www.alienvault.com/documentation/usm-appliance/plugin-management/supported-formats-normalize_date.htm
技術參考來源: http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html
Rewriting and modifying messages
There are many ways to modify log messages. A simple method which does not always work is to modify the $raw_event field (in case of syslog) without parsing the message. This can be done with regular expressions using capturing, for example:
if $raw_event =~ /^(aaaa)(replaceME)(.+)/ $raw_event = $1 + 'replaceMENT' + $3;
The more complex method is to parse the message into fields, modify some fields and finally reconstruct the message from the fields. The conditional rewrite of the syslog facility example shows such a syslog message modification method.
沒有留言:
張貼留言