Fluentd

Fluentd is a data collecting software that's designed to process data streams. It's an open source software, which means that it's completely free to use and constantly being refined by its user base. Fluentd lets you unify your methods of data collection and data consumption to give you a better overall understanding of and use for data. Fluentd has a unified logging layer to separate backend systems from important data sources. This unifies every aspect of data collection and processing, from collecting to filtering to buffering to outputting across multiple destinations. This unification helps to keep data clear, concise, and accurate. Fluentd also has more than five hundred plugins to stay connected to outputs and data sources. You can choose the plugins that suit your unique data collection needs. This keeps the core interface simple and easily navigable -- nothing will ever be there that you don't want there. Unlike other data collecting softwares, Fluentd retains a simplistic core, so you don't have to spend endless time customizing and hiding different features. Fluentd is written with a combination of Ruby and C programming languages, and it requires very few system resources. The vanilla instance can run on 40MB of memory while processing 13,000 events per second. There's also a lightweight alternative called Fluent Bit for people with stricter memory requirements. More than five thousand data-driven companies currently use Fluentd because it's a software tool that has been proven to work. The largest Fluentd user currently collects data from more than fifty thousand servers. Microsoft, Nintendo, and Amazon Web Services are just a few of the companies that use Fluentd. Fluentd is part of the Cloud Native Computing Foundation. The CNCF was created to build sustainable digital ecosystems and foster communities around a number of high quality open source projects. All of Fluentd's components are freely available with the Apache 2 License. Fluentd supports file-based and memory-based buffering, which adds a level of protection to your data. It also has support for robust failover, and it can be set up to accommodate high availability. Fluentd has a strong community, and it's easy to find community support for any issues that might crop up. There's a public issue tracker, a Twitter feed, a Facebook presence, and a Google Group. And of course the source code itself is freely available for perusal.