katip-elasticsearch

ElasticSearch scribe for the Katip logging framework.

BSD-3-Clause licensed by Ozgun Ataman, Michael Xavier

Maintained by [email protected]

This version can be pinned in stack with:[email protected]:6b2b6357022fefdf4c57c89b04bb54ff3e871d63c1026919b9c79c1c40e4a4b2,2541

Katip Elasticsearch

katip-elasticsearch is a scribe for the Katip logging framework that sends structured logs to ElasticSearch.

Built in bounded buffering.
Configurable pool of logging workers to help with high write volume.
Optional field type annotation to avoid mistyping values.
Optional automatic date sharding, so logs can be filed into monthly, weekly, daily, hourly, every minute indices. You can even specify your own index routing logic. This pattern can be seen in the ELK stack as a way of keeping indexes reasonably sized and easy to optimize, rotate, and manage.
Customizable retry policy for temporary outages and errors.
Automatic index and mapping setup.

Drop direct dependency on random, upgrade uuid to >= 1.3.12 for safer id generation. Previously, UUID was using randomIO, which uses the system clock as a seed. So if multiple nodes happened to start at the same time, they would produce conflicting UUID sequences.

Default index sharding policy to daily. Previously it was no sharding. The reasoning here is that no sharding creates very large indices which become very difficult to manage in production. Rotating data out on a time basis is very slow compared to deleting date-based indices.

Upgrade note: if you were using the defaults before and switch to daily, rather than having the index name of my-index, you’ll start seeing my-index-2016-3-14. The good news is that whatever you’re using to use to search against your logs (such as kibana) will support index patterns, so just use the pattern of my-index* to get everything. Eventually if you have a retention period, you can manually delete the my-index index without disruption.

Depends on 18 packages(full list with versions):