Monitoring API Performance with Elastic Stack

The Elastic Stack (also known as ELK) is an amazing group of open source projects, including Elasticsearch, Logstash, and Kibana, that work together to gather, manage, and visualize large amounts of data. DreamFactory includes a Logstash connector which allows you to send information about select API interactions to an Elasticsearch database where it can subsequently be analyzed. This data can then be visualized using a tool such as Kibana or Graphite.

Logstash Configuration

To integrate the Elastic Stack alongside DreamFactory, you'll begin by selecting the `Logstash` connector from the services select box:

Next you'll identify the location of your Logstash daemon. This could be running on the same server as DreamFactory, or on a dedicated environment such as AWS Elasticsearch. Then you'll identify both the APIs you'd like to monitor and specifically which data objects you'd like to capture:

Now that data is being forwarded from DreamFactory through Logstash and onwards to Elasticsearch, you're free to create a dashboard which visualizes any aspect of the captured data, including HTTP status codes, user-specific API usage, and much more: