Logstash
Logstash is software that helps filter and store data. It's a product of Elastic, a company founded in 2012 in Amsterdam. Elastic was created by the programmers who made Apache Lucene and Elasticsearch: Shay Banon, Steven Schuurman, Simon Willnaur and Uri Boness. Today, the company employs people in 32 countries. The company's United States headquarters is in Mountain View, Calif. There are other American offices in Phoenix, New York and San Francisco. Elastic's other offices are in Amsterdam, Hong Kong, Switzerland, the United Kingdom, Berlin, Munich, Norway, Japan, Korea, Singapore, France and Australia. Other products offered by Elastic include Kibana, Beats, X-Pack, Elastic Cloud, ES-Hadoop and some other smaller offerings. Notable customers include Sprint, Indiana University, Travelport, Grab, Stormfish, SAP Concur, the CBC, Discover, Fermilab, eBay, Lyft, IEEE, Blizzard Entertainment, Citibank, Otto Motors, Cox Communications and Credit Suisse.
- Accepts data from virtually any source or event and processes it for output to a "stash" (or many storage locations, depending on filters)
- Data can be parsed via programming and transformed while in transit.
- For example, Logstash can be used to decipher geographic coordinates from an IP address input
- Compatible with Elastic Stack data analytics for quick insight
- Multiple visibility (Elastic X-Pack) tools to keep track of server and program performance
- Open source and completely free to download and use
- Actively developed. The latest update (version 6.2.3) was released on March 20, 2018
- Installation is just four simple steps and a command line: bin/logstash -f logstash.conf
- A comprehensive guide is available online to help new administrators get started (there's also a helpful video available here)
- Dependent on Java, which has its own problems with security vulnerabilities (the latest version of Logstash require Java version 8 or higher for installation)
Data management is a fundamental part of tech oriented businesses, and today, data can come from many sources at the same time. But how does a company organize a huge flow of data from every direction? This is where Logstash comes in handy. This server side program is specifically designed to ingest data from all "sizes, shapes and sources." During processing, the data can be sent through filters to determine its final destination, which can be one or a variety of "stashes," such as servers, clouds or other storage areas. Data can be inputted from Elastic Beats, Amazon Web Services CloudWatch API, Couch DB, Elasticsearch clusters, shell command output, events from files, Ganglia packets via UDP, GELF-format messages (as events) via Graylog2, Google Cloud PubSub, Graphite, HTTP or HTTPS events, events from an IRC server, JMS Broker events, Java application metrics via JMX, AWS Kinesis stream events, Lumberjack, Puppet server events, RabbitMQ exchange events, RELP events over a TCP socket, Salesforce SOQL queries, SNMP trap events, AWS Simple Queue Service, Twitter streaming API, UNIX socket events, XMPP/Jabber protocol events and more. There's even a plugin to generate random events for testing purposes. Logstash is perfect for automating data collection and processing, saving loads of productivity time for other tasks. The software is highly configurable, and over 200 plugins exist to make sure that Logstash can handle whatever data it is given.
Alternatives to Logstash
Logentries
Scribe
Sentry
SQL Diagnostic Manager for MySQL
Fluentd
Prometheus
Splunk
Papertrail
Raygun
Scalyr
Graylog
Sumo Logic
Logsniffer
Flume
Loggly
Stackify
IQLECT
Cloudingo
DemandTools
Informatica Data Quality
Melissa DATA
Email List Verify
