Creating custom elasticsearch index with logstash

Creating custom elasticsearch index with logstash

I have to create custom index in elasticsearch using logstash. I have created new template in elasticsearch, and in logstash configuration i have specify template path,template_name and template_overwrite value,but still whenever I run logstash, new index is generated with logstash-dd-mm-yy regex,not with template_name specified in properties,
logstash -config file is
input {
file {
path => “/temp/file.txt”
type => “words”
start_position => “beginning”
}
}
filter {

mutate {
add_field => {“words” => “%{message}”}
}

}
output {
elasticsearch {
hosts => [“elasticserver:9200”]
template => “pathtotemplate.json”
template_name => “newIndexName-*”
template_overwrite => true
}
stdout{}
}

Index template file is
{
“template”: “dictinary-*”,
“settings” : {
“number_of_shards” : 1,
“number_of_replicas” : 0,
“index” : {
“query” : { “default_field” : “@words” },
“store” : { “compress” : { “stored” : true, “tv”: true } }
}
},
“mappings”: {
“_default_”: {
“_all”: { “enabled”: false },
“_source”: { “compress”: true },
“dynamic_templates”: [
{
“string_template” : {
“match” : “*”,
“mapping”: { “type”: “string”, “index”: “not_analyzed” },
“match_mapping_type” : “string”
}
}
],
“properties” : {
“@fields”: { “type”: “object”, “dynamic”: true, “path”: “full” },
“@words” : { “type” : “string”, “index” : “analyzed” },
“@source” : { “type” : “string”, “index” : “not_analyzed” },
“@source_host” : { “type” : “string”, “index” : “not_analyzed” },
“@source_path” : { “type” : “string”, “index” : “not_analyzed” },
“@tags”: { “type”: “string”, “index” : “not_analyzed” },
“@timestamp” : { “type” : “date”, “index” : “not_analyzed” },
“@type” : { “type” : “string”, “index” : “not_analyzed” }
}
}
}
}

Please help

Solutions/Answers:

Solution 1:

To do what you want, you have to set the index parameter in the Elasticsearch output block. Your output block will look like this:

output {
    elasticsearch {
     hosts => ["elasticserver:9200"]
     index => "newIndexName-%{+YYYY.MM.dd}"
     template => "pathtotemplate.json"
     template_name => "newIndexName-*"
     template_overwrite => true
    }
    stdout{}
}

References

Related:  Elasticsearch remove default fields from search's response body