應用分析(之後遇到,慢慢再新增)
案例一
例如Log message為:
2016/2/4 �U�� 05:01:40 08F0 PACKET 0000000002F70FE0 UDP Rcv 120.101.0.1 ef80 Q [1000 NOERROR] A (3)es2(8)niucloud(3)niu(3)edu(2)tw(0)
因不屬於默認的格式,須自己解析欄位如下:
%{DATA:date} %{GREEDYDATA:A1} %{TIME:time} %{WORD:A2} %{WORD:A3} %{WORD:A4} %{WORD:protocal} %{WORD:A5} %{IP:client} %{WORD:A6} %{WORD:A7} %{SYSLOG5424SD:A8} %{WORD:A9} %{GREEDYDATA:A10}
補充:
1.DATA:date=>型態:欄位名稱
2.欄位名稱命名時要小心,有些為默認字不可取名,ex:type
這時候的結果為
{
"date": ["2016/2/4"],
"A1": ["�U��"],
"time": ["05:01:40"],
"A2": ["08F0"],
"A3": ["PACKET"],
"A4": ["0000000002F70FE0"],
"protocal": ["UDP"],
"A5": ["Rcv"],
"client": ["120.101.0.1"],
"A6": ["ef80"],
"A7": ["Q"],
"A8": ["[1000 NOERROR]"],
"A9": ["A"],
"A10": ["(3)es2(8)niucloud(3)niu(3)edu(2)tw(0)"]
}
而我們要針對A9做修改成 es2.niucloud.niu.edu.tw,程式碼為:
filter {
grok {
match=>%{DATA:date} %{GREEDYDATA:A1} %{TIME:time} %{WORD:A2} %{WORD:A3} %{WORD:A4} %{WORD:protocal} %{WORD:A5} %{IP:client} %{WORD:A6} %{WORD:A7} %{SYSLOG5424SD:A8} %{WORD:A9} %{GREEDYDATA:A10}
}
mutate {
gsub => ["A10","\(\d*\)","."]
#利用正規表示法將符合"\(\d*\)"取代為".",而A10結果為.es2.niucloud.niu.edu.tw.
gsub => ["A10""^\.",""]
gsub => ["A10""^.$",""]
#前後"."取代為空,A10結果為es2.niucloud.niu.edu.tw
}
}