如何向 Elasticsearch 提供我的 Snort 警報日誌？

我昨天從 ELK howto開始，並且相當容易地啟動和執行 ELK。接下來我想做的就是將我的 Snort 警報日誌插入其中。我使用過濾器和絕對討厭的 Grok 正則表達式配置了 Logstash（如下所示），以使用grokdebug拆分所有欄位來測試它。然後我打開了 snort，警報日誌開始填滿，然後是 logstash 重新啟動（--configtest當然是在這樣做之後）。我安裝了 ES “Head” 外掛，這樣我就可以四處逛逛了。似乎我的 snort 警報正在與我在 howto 中創建的系統日誌映射進行映射（下圖）。在 ES 中，我似乎無法使用 logstash 配置中定義的任何欄位（ids_proto、src_ip、dst_ip）進行搜尋。為什麼是這樣？我需要定義一個映射還是這裡有其他東西搞砸了？

input 
{
 file {
   path => "/var/log/snort/alert"
   type => "snort_tcp"  # a type to identify those logs (will need this later)
   start_position => beginning 
   ignore_older => 0      # Setting ignore_older to 0 disables file age checking so that the tutorial file is processed even though it’s older than a day. 
   sincedb_path => "/dev/null"
 }
}


filter {
if [type] == "snort_tcp" {
     grok {
       add_tag => [ "IDS" ]
       match => [ "message", "%{MONTHNUM:month}\/%{MONTHDAY:day}-%{HOUR:hour}:%{MINUTE:minute}:%{SECOND:second}\s+\[\*\*\]\s+\[%{INT:ids_gid}\:%{INT:ids_sid}\:%{INT:ids_rev}\]\s+%{DATA:ids_proto}\s+\[\*\*\]\s+\[Classification:\s+%{DATA:ids_classification}\]\s+\[Priority:\s+%{INT:priority}\]\s+\{%{WORD:ids_proto}\}\s+%{IP:src_ip}\:%{INT:src_port}\s+\-\>\s+%{IP:dst_ip}\:%{INT:dst_port}"]
     }
   }
   geoip {
     source => "[src_ip]"
     target => "SrcGeo"
   }
   geoip {
     source => "[dst_ip]"
     target => "DstGeo"
   }
       if [priority] == "1" {
     mutate {
       add_field => { "severity" => "High" }
     }
   }
   if [priority] == "2" {
     mutate {
       add_field => { "severity" => "Medium" }
     }
   }
   if [priority] == "3" {
     mutate {
       add_field => { "severity" => "Low" }
     }
   }
   if [ids_proto] {
     if [ids_proto] =~ /^GPL/ {
       mutate {
         add_tag => [ "Snort-ET-sig" ]
         add_field => [ "ids_rule_type", "Emerging Threats" ]
       }
     }
     if [ids_proto] =~ /^ET/ {
       mutate {
         add_tag => [ "Snort-ET-sig" ]
         add_field => [ "ids_rule_type", "Emerging Threats" ]
       }
     }
     if "Snort-ET-sig" not in [tags] {
       mutate {
         add_tag => [ "Snort-sig" ]
         add_field => [ "ids_rule_type", "Snort" ]
       }
     }
   }
if "Snort-sig" in [tags] {
     if [ids_gid] == "1" {
       mutate {
         add_field => [ "Signature_Info", "http://rootedyour/.com/snortsid?sid=%{ids_sid}" ]
       }
     }
     if [ids_gid] != "1" {
       mutate {
         add_field => [ "Signature_Info", "http://rootedyour.com/snortsid?sid=%{ids_gid}-%{ids_sid}" ]
       }
     }
   }
   if "Snort-ET-sig" in [tags] {
     mutate {
       add_field => [ "Signature_Info", "http://doc.emergingthreats.net/bin/view/Main/%{ids_sid}" ]
     }
   }
 }



output 
{
 elasticsearch 
  {
   hosts => ["localhost:9200"]
   manage_template => false
   index => "snort_tcp-%{+YYYY.MM.dd}"     
 }
}

這裡有幾件事：

• 預設映射logstash創建將所有字元串欄位設置為未分析，這往往對下游查看工具更友好。
• 根本不設置映射，就像您正在做的那樣，使用預設的 ElasticSearch動態映射，這不適合 Logstash。

為了進行測試，我推薦以下輸出部分：

output 
{
elasticsearch 
 {
   hosts => ["localhost:9200"]
   manage_template => true
   index => "logstash-%{+YYYY.MM.dd}"     
 }
}

以這種方式設置時，logstash 索引將獲得預設的 logstash 映射，其行為可能更接近您的預期。如果是這種情況，您可能必須定義一個映射文件。

output 
{
elasticsearch 
 {
   hosts => ["localhost:9200"]
   manage_template => true
   index => "snort_tcp-%{+YYYY.MM.dd}"
   template => "/etc/logstash/template.json"
   template_name => "snort_tcp"
 }
}

引用自：https://serverfault.com/questions/809156