RackPan的學習備忘錄: 2018

2018年10月8日星期一

LibreNMS Fast up/down Check

LibreNMS是以 ICMP ping 來檢查裝置存活與否，而其檢查頻率是隨著poller的頻率，通常預設是5分鐘，而那就意味著我們可能需要5分鐘才能得知設備離線。

那麼，要如何加快這個頻率呢? LibreNMS提供了一支script : ping.php 來加快這個檢查頻率至1分鐘而不必增加 snmp 負載。

設定ping check頻率為1分鐘:

1. 變更 config.php 設定檔中的 ping_rrd_step 設定值，新增以下設定

$config[‘ping_rrd_step’] = 60;

2. 更新 rrd 檔的step

# cd /opt/librenms

# ./script/rrdstep.php -h all

3. 於 /etc/cron.d/librenms 加入排程

# vi /etc/cron.d/librenms

…

* * * * * librenms /opt/librenms/ping.php >> /dev/null 2>&1

另外，因為fping指令僅回應up/down，故需再加入以下設定於 config.php 以防因網路延遲而造成誤判。

$config['fping_options']['retries'] = 2;

$config['fping_options']['timeout'] = 500;

$config['fping_options']['interval'] = 500;

參考來源: LiberNMS Doc

2018年9月19日星期三

LibreNMS on CentOS 7 安裝與設定

LibreNMS on CentOS 7 安裝與設定

LibreNMS是一個基於PHP的開源網路監控系統，透過SNMP協定，收集網路設備資訊，如CPU、RAM、HDD、Network等。

LibreNMS的安裝可參考官網提供的文件進行安裝。

請注意，LibreNMS僅支援PHP 5.6.4以上版本。

l 安裝需要的套件

1. 先安裝 epel-release，再安裝所有需要套件

# yum install epel-release

# rpm –Uvh https://mirror.webtatic.com/yum/el7/webtatic-release.rpm

# yum install composer cronie fping git ImageMagick jwhois mariadb mariadb-server mtr MySQL-python net-snmp net-snmp-utils nginx nmap php72w php72w-cli php72w-common php72w-curl php72w-fpm php72w-gd php72w-mbstring php72w-mysqlnd php72w-process php72w-snmp php72w-xml php72w-zip python-memcached rrdtool

2. 新增 librenms 使用者帳號

# useradd librenms -d /opt/librenms -M -r

# usermod -a -G librenms nginx

3. 下載 LiberNMS

# cd /opt

# composer create-project --no-dev --keep-vcs librenms/librenms librenms dev-master

l DB Server設定

1. 設定 MySQL，password更改為自己的密碼

# systemctl start mariadb

# mysql –u root

CREATE DATABASE librenms CHARACTER SET utf8 COLLATE utf8_unicode_ci;

CREATE USER 'librenms'@'localhost' IDENTIFIED BY 'password';

GRANT ALL PRIVILEGES ON librenms.* TO 'librenms'@'localhost';

FLUSH PRIVILEGES;

exit

2. 關閉 MySQL strict mode

# vi /etc/my.cnf

Within the [mysqld] section please add:

########################

innodb_file_per_table=1

sql-mode=""

lower_case_table_names=0

########################

3. 設定開機啟動 mariadb

# systemctl enable mariadb

# systemctl restart mariadb

l Web Server設定

1. 設定 PHP-FPM

##### 設定時區為 Asia/Taipei #####

# vi /etc/php.ini

Data.timezone = “Asia/Taipei”

2. 設定站台

# vi /etc/php-fpm.d/www.conf

…….

;user = apache

user = nginx

group = apache ; keep group as apache

;listen = 127.0.0.1:9000

listen = /var/run/php-fpm/php7.2-fpm.sock

listen.owner = nginx

listen.group = nginx

listen.mode = 0660

啟動 PHP-FPM

# systemctl enable php-fpm

# systemctl restart php-fpm

3. 設定 Nginx，server_name修改為自己的網址

# vi /etc/nginx/conf.d/librenms.conf

### 加入以下內容 ###

server {

listen 80;

server_name librenms.example.com;

root /opt/librenms/html;

index index.php;

charset utf-8;

gzip on;

gzip_types text/css application/javascript text/javascript application/x-javascript image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon;

location / {

try_files $uri $uri/ /index.php?$query_string;

}

location /api/v0 {

try_files $uri $uri/ /api_v0.php?$query_string;

}

location ~ \.php {

include fastcgi.conf;

fastcgi_split_path_info ^(.+\.php)(/.+)$;

fastcgi_pass unix:/var/run/php-fpm/php7.2-fpm.sock;

}

location ~ /\.ht {

deny all;

}

設定開機啟動nginx

# systemctl enable nginx

# systemctl restart nginx

l SELinux設定

1. 關閉SELinux

# vi /etc/sysconfig/selinux

SELINUX=disabled

# reboot

2. 或安裝 policy tool for SELinux

# yum install policycoreutils-python

設定LibreNMS

# semanage fcontext -a -t httpd_sys_content_t '/opt/librenms/logs(/.*)?'

# semanage fcontext -a -t httpd_sys_rw_content_t '/opt/librenms/logs(/.*)?'

# restorecon -RFvv /opt/librenms/logs/

# semanage fcontext -a -t httpd_sys_content_t '/opt/librenms/rrd(/.*)?'

# semanage fcontext -a -t httpd_sys_rw_content_t '/opt/librenms/rrd(/.*)?'

# restorecon -RFvv /opt/librenms/rrd/

# semanage fcontext -a -t httpd_sys_content_t '/opt/librenms/storage(/.*)?'

# semanage fcontext -a -t httpd_sys_rw_content_t '/opt/librenms/storage(/.*)?'

# restorecon -RFvv /opt/librenms/storage/

# semanage fcontext -a -t httpd_sys_content_t '/opt/librenms/bootstrap/cache(/.*)?'

# semanage fcontext -a -t httpd_sys_rw_content_t '/opt/librenms/bootstrap/cache(/.*)?'

# restorecon -RFvv /opt/librenms/bootstrap/cache/

# setsebool -P httpd_can_sendmail=1

# setsebool -P httpd_execmem 1

Allow fping

建立一檔案 http_fping.tt，檔案內容如下---

module http_fping 1.0;

require {

type httpd_t;

class capability net_raw;

class rawip_socket { getopt create setopt write read };

}

#============= httpd_t ==============

allow httpd_t self:capability net_raw;

allow httpd_t self:rawip_socket { getopt create setopt write read };

建立完成後，執行以下指令

# checkmodule -M -m -o http_fping.mod http_fping.tt

# semodule_package -o http_fping.pp -m http_fping.mod

# semodule -i http_fping.pp

l Firewall設定

# firewall-cmd --zone public --add-service http

# firewall-cmd --permanent --zone public --add-service http

# firewall-cmd --zone public --add-service https

# firewall-cmd --permanent --zone public --add-service https

l snmpd設定

# cp /opt/librenms/snmpd.conf.example /etc/snmp/snmpd.conf

設定snmp community string

# vi /etc/snmp/snmpd.conf

### 將RANDOMSTRINGGOESHERE 改為自家環境使用的community string (ex: public)

# Change RANDOMSTRINGGOESHERE to your preferred SNMP community string

com2sec readonly default public

啟動 snmpd

# curl -o /usr/bin/distro https://raw.githubusercontent.com/librenms/librenms-agent/master/snmp/distro

# chmod +x /usr/bin/distro

# systemctl enable snmpd

# systemctl restart snmpd

l Cron job設定

# cp /opt/librenms/librenms.nonroot.cron /etc/cron.d/librenms

複製 logrotate 設定檔

LibreNMS log路徑在 /opt/libernms/logs，可使用以下rotate設定檔定期清除過期資料。

# cp /opt/librenms/misc/librenms.logrotate /etc/logrotate.d/librenms

l 權限設定

# chown -R librenms:librenms /opt/librenms

### 若沒有關閉SELinux，則需設定以下SELinux權限設定 ###

# setfacl -d -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/

# setfacl -R -m g::rwx /opt/librenms/rrd /opt/librenms/logs /opt/librenms/bootstrap/cache/ /opt/librenms/storage/

l Web installer

開啟瀏覽器，依照螢幕上指示進行LiberNMS系統設定。

並請注意，系統預設使用HPPT而非HTTPS，故請不要將LibreNMS暴露在公開網路上。

http://libernms.example.com/install.php

l 最後步驟

恭喜!! 安裝設定完成後，您要做的就是從 Device \ Add Device 頁面，將要監控的網路設備加入監控即可。

l Troubleshooting

# cd /opt/librenms

# ./validate.php

以上內容參考自 LibreNMS官方文件

2018年8月27日星期一

如何查詢由GPO套用的信任網站清單

通常在企業環境下，會因為企業環境佈署 Windows AD 設定群組原則 (GPO) 鎖定「信任網站」，但要如何查詢套用的網站清單，則會因為被鎖定而無法看到全部內容，如下圖!

試過找 registry，看不到...

後來看到另一種思路，查看套用的GPO，嗯，有效，命令如下---

C:\ gpresult /z

若要查看特定網站網址---

C:\ gpresult /z | find "localhost"

另外加碼，有先進提到可以在企業環境底下，自行透過 registry 加入信任網站，嗯.....
有需要的可以過去看看!!

S小魚仔S Windows 7 使用登錄編輯器 (Regedit) 修改信任網站。

2018年6月11日星期一

NXLog 如何調整Log message內容再送出給Log Server

利用 NXLog 將 Windows DNS Log 送至 OSSIM，卻發現 OSSIM 一直無法將 raw log 進行 parsing，經檢查 OSSIM plugins 設定檔 windns-nxlog.cfg 的 regex 後發現，應該是原本 log 中的中文 (上午|下午) 送到 OSSIM 後無法被辨識。

在嘗試調整 regular expression 無果後，決定從 NXLog 著手，在 NXLog 送出前，先將 (上午|下午) 調整成 (AM|PM)，再將 log 送出。

然後，又發現在 SIEM 查詢時，下午的 windns log 都沒有出現，檢查發現在 OSSIM 的 plugin - windns-nxlog.cfg 中，其 date 欄位有先經過 normalize_date() function 處理，而 2018/6/8 06:17:49 PM 這個格式不符其支援的格式，故須修改成 6/8/2018 06:17:49 PM 這種格式。

以下是中文版與英文版所產出之 DNS debugging log。

----------------------------------------------------------------------------------------------------------------------

Windows DNS log from Windows 2008 R2 CHT:

2018/4/16 上午 06:17:49 0B38 PACKET 0000000002134EF0 UDP Rcv 192.168.3.252 3780 Q [0001 D NOERROR] A (6)amazon(3)com(0)

2018/4/16 上午 06:17:49 0B38 PACKET 0000000002EA9D30 UDP Snd 168.95.1.1 0fc0 Q [0001 D NOERROR] A (6)amazon(3)com(0)

2018/4/16 上午 06:17:49 0B38 PACKET 000000000212DAB0 UDP Rcv 168.95.1.1 0fc0 R Q [8081 DR NOERROR] A (6)amazon(3)com(0)

2018/4/16 上午 06:17:49 0B38 PACKET 0000000002134EF0 UDP Snd 192.168.3.252 3780 R Q [8081 DR NOERROR] A (6)amazon(3)com(0)

-----------------------------------------------------------------------------------------------------------------------

Windows DNS log from Windows 2008 R2 Eng:

2/19/2015 10:03:57 PM 2AE8 PACKET 00000005CF374F80 UDP Rcv 192.168.0.42 fdd7 Q [0001 D NOERROR] A (9)imap-mail(7)outlook(3)com(0)

2/19/2015 10:03:57 PM 2AE8 PACKET 00000005CB426930 UDP Snd 10.255.176.137 0c4c Q [0001 D NOERROR] A (9)imap-mail(7)outlook(3)com(0)

2/19/2015 10:03:57 PM 2AE8 PACKET 00000005D03B4CE0 UDP Rcv 10.255.176.137 0c4c R Q [8081 DR NOERROR] A (9)imap-mail(7)outlook(3)com(0)

2/19/2015 10:03:57 PM 2AE8 PACKET 00000005D03B4CE0 UDP Snd 192.168.1.42 fdd7 R Q [8081 DR NOERROR] A (9)imap-mail(7)outlook(3)com(0)

-----------------------------------------------------------------------------------------------------------------------

調整 nxlog.conf 設定如下:

======================================================================

< extension json >

Module xm_json

< /extension >

## Custom CSV format for the windns-nxlog AlienVault plugin.

< extension transform_alienvault_csv_dns >

Module xm_csv

Fields $Hostname, $SourceName, $Message

FieldTypes string, string, string

Delimiter ,

< /extension >

< input dns_logs >

Module im_file

File "C:\\temp\\DNSLog\\debug.log"

SavePos TRUE

InputType LineBased

#根據 OSSIM 的 Date Format supported by normalize_date() function，調整 Date Format
#Convert the Chinese characters into English characters (上午 to AM、下午 to PM)
#Change the datetime format from "yyyy/mm/dd (上午|下午) hh:mm:ss" to "mm/dd/yyyy hh:mm:ss (AM|PM)"

Exec if $raw_event =~ /^(\d+)\/(\d+\/\d+)\s(上午\s)(\d+\:\d+\:\d+\s)(.+)/ $raw_event = $2 + '/' + $1 + ' ' + $4 + 'AM ' + $5;
Exec if $raw_event =~ /^(\d+)\/(\d+\/\d+)\s(下午\s)(\d+\:\d+\:\d+\s)(.+)/ $raw_event = $2 + '/' + $1 + ' ' + $4 + 'PM ' + $5;

Exec if $raw_event !~ /^\d/ drop();\

else\

$Message = $raw_event;\

$SourceName = "DNS";\

$raw_event = to_json();\

}

< /input >

< output out_alienvault_dns_nxlog >

Module om_udp

Host 192.168.1.2

Port 514

Exec if not defined $Message { drop(); }

## Replace newlines, tabs and carriage returns with blanks:

Exec $Message = replace($Message, "\t", " "); $Message = replace($Message, "\n", " "); $Message = replace($Message, "\r", " ");

## Ensure that commonly undefined values are set:

Exec if not defined $AccountName { $AccountName = "-"; }

Exec if not defined $AccountType { $AccountType = "-"; }

Exec if not defined $Domain { $Domain = "-"; }

## Ensure we send in the proper format:

Exec $Hostname = hostname_fqdn();

Exec transform_alienvault_csv_dns->to_csv(); $raw_event = $Hostname + ' DNS-NXLOG: ' + $raw_event;

< /output >

## Route for dns nxlog logs:

< route route_dns_nxlog >

Path DNS_Logs => out_alienvault_dns_nxlog

< /route >

======================================================================

OSSIM中查看rawlog，原本輸出為

Jun 8 00:07:48 DNS.test.corp DNS-NXLOG: "DNS.test.corp","DNS","2018/6/8 ?W?? 12:09:02 0B38 PACKET 00000000029DDCE0 UDP Rcv 192.168.3.31 7f19 Q [0001 D NOERROR] A (9)resolver1(5)ofind(6)ctmail(3)com(0)"

Jun 8 00:07:48 DNS.test.corp DNS-NXLOG: "DNS.test.corp","DNS","2018/6/8 ?W?? 12:09:02 0B38 PACKET 00000000029DDCE0 UDP Snd 192.168.3.31 7f19 R Q [8081 DR NOERROR] A (9)resolver1(5)ofind(6)ctmail(3)com(0)"

進行 Pre-parsing 後變為

Jun 8 15:43:09 DNS.test.corp DNS-NXLOG: "DNS.test.corp","DNS","6/8/2018 03:42:58 PM 0B34 PACKET 00000000020B9D10 UDP Rcv 192.168.3.31 afdd Q [0001 D NOERROR] A (13)gmail-smtp-in(1)l(6)google(3)com(0)"

Jun 8 15:43:09 DNS.test.corp DNS-NXLOG: "DNS.test.corp","DNS","6/8/2018 03:42:58 PM 0B34 PACKET 00000000020B9D10 UDP Snd 192.168.3.31 afdd R Q [8081 DR NOERROR] A (13)gmail-smtp-in(1)l(6)google(3)com(0)"

再回到 OSSIM web介面，ANALYSIS \ SECURITY EVENTS (SIEM) 下，DATA SOURCE 下拉選項出現 windns，表示 DNS Log 已成功寫入資料庫了!!

備註：

OSSIM Date Format supported by normalize_date() function:
https://www.alienvault.com/documentation/usm-appliance/plugin-management/supported-formats-normalize_date.htm

技術參考來源: http://nxlog-ce.sourceforge.net/nxlog-docs/en/nxlog-reference-manual.html

Rewriting and modifying messages
There are many ways to modify log messages. A simple method which does not always work is to modify the $raw_event field (in case of syslog) without parsing the message. This can be done with regular expressions using capturing, for example:

if $raw_event =~ /^(aaaa)(replaceME)(.+)/ $raw_event = $1 + 'replaceMENT' + $3;

The more complex method is to parse the message into fields, modify some fields and finally reconstruct the message from the fields. The conditional rewrite of the syslog facility example shows such a syslog message modification method.

2018年10月8日 星期一

LibreNMS Fast up/down Check

2018年9月19日 星期三