How to get started with Kibana

Kibana is an open source analytics and visualization tool that makes it easy to search, visualize, and explore large volumes of data through a browser-based interface. Alongside Elasticsearch, Logstash, and Beats, Kibana is a core part of the Elastic Stack (formerly known as the ELK Stack). 

Elasticsearch, the search engine at the heart of the Elastic Stack, is one of the most popular open source projects for search and analytics. Elasticsearch is what searches, stores, and analyzes the data that you explore in Kibana—it’s really a search engine, a data store, and an analytical engine all in one.

Elasticsearch allows users to perform Google-style searches on their data, or ask questions like “What countries are the visitors to my website coming from?” It’s also extremely fast and distributed, which allows users to scale to larger data sets. Now take this power and combine it with the rich user interface that Kibana provides, and you have a real-time solution to explore your data.

With Elasticsearch and Kibana, you can explore practically any type of data, from text documents to machine logs, application metrics, ecommerce traffic, sensor telemetry, or your company’s business KPIs. Once the data is in Elasticsearch, you can explore and interact with it in Kibana; you can search through the data using the Kibana search bar, visualize the data using various chart types, and play around with the visualizations using real-time dashboards. You can also display dashboards on a large screen providing visibility across your entire company or office.

In this article, I’ll walk you through everything you need to know in order to start exploring your data in Kibana and create useful visualizations. We’ll look at how to get data into Kibana, how to use Kibana to explore your data, and how to use Kibana to create visualizations and dashboards.

Adding data to Kibana

The first thing you’ll need to do is get some data into Kibana to work with. Once you choose your deployment and you have Elasticsearch running, you can log into Kibana for the first time.

Elastic

To explore Kibana, you can use Kibana sample data or your own data. If you choose the latter, Kibana provides various ways to ingest data. For example, if you use Beats (Elastic’s family of single-purpose data shippers), simply choose which system Beats should collect the data from and let Beats continuously collect data for you. 

Elastic

Or, if you have JSON or CSV data, simply upload a file.

For this article, I will use the sample data that ships with Kibana in order to show you the core capabilities of Kibana.

Elastic

When you add sample data, Kibana creates an index pattern, sample visualizations, and a dashboard. If you’re adding your own data, you will need to create a Kibana index pattern yourself.

What are Kibana index patterns?

Elasticsearch stores data in indices—these are somewhat analogous to tables if you’re more familiar with relational databases. Index patterns tell Kibana which Elasticsearch indices you want to explore. You can create an index pattern for a specific index in Elasticsearch or you can query multiple indices at the same time by using a wildcard *. You can have multiple index patterns in Kibana (like you have numerous tables in a database). When creating visualizations or searching your data, you will need to choose which index pattern to run your search on.

Navigating in Kibana

You’ll see a number of applications in the left-hand menu in Kibana. In this article, we’ll go through the first three, which are focused on finding data insights: Discover, Visualize, and Dashboard.

Discover

Discover is where you can search and filter your raw documents. 

Elastic

Each record is represented as a line. You can expand the lines to see all of the fields in each record and their values.

On the left side, you’ll see a side menu that lists all of your fields. Discover is a good place to search for a specific record. There are several ways you can search your data. 

You can perform a free text search, like a Google search. With a free text search, Elasticsearch will search across your documents and will return all the documents that contain the keyword you searched for. For example, just type the word “error” into the search bar. Or you can search based on a specific field using the autocomplete.

Elastic

Discover can also show the data in a table format. By selecting fields from the menu on the left, you will see the same fields appear as the columns of the table. The histogram above the table is a quick way to see the distribution of documents over time; if you click on a specific time range, Discover will zoom in to that time range and the page will refresh to show only the documents that fall within that range. 

Elastic

Visualize 

They say that a picture is worth a thousand words, and this is often true when trying to convey complex ideas. 

Visualize is where you can create visualizations and explore your data using a number of out-of-the-box charts. 

Elastic

Kibana supports many chart types. Based on the questions you have in mind and how you want to explore your data, you’ll want to choose the appropriate type of chart—whether it’s for time series data, for prominent terms, or even a geographical map. All of these are real-time visualizations and can be explored with live data. 

If you need a specific visualization that you can’t find out of the box in Kibana, you can also use Vega, an open source library for visualizations.  

In general, when visualizing data in Kibana, there are two core definitions that are worth understanding. 

• Bucket aggregations: A bucket aggregation groups documents into buckets, each of which can contain multiple documents, a single document, or nothing at all. 
• Metrics aggregation: After you create buckets, a metrics aggregation will calculate a value for each bucket. 

For example, if we wanted to visualize the average number of bytes daily, we would create daily buckets on the x-axis, and then calculate the average bytes in each bucket, meaning each day.

Elastic

Now if we wanted to, we could add more metrics or even more buckets to show, for example, the average bytes based on the top three responses. 

Elastic

Now that we have created this visualization, you can save and add it to a dashboard. 

Dashboards

Why add something to a dashboard? Dashboards are an extremely powerful concept in Kibana. They are a live, real-time way to view your data from multiple perspectives and interact with the data all in the same view.

Dashboards are also extremely interactive: 

• Select an area of a chart to zoom into the specific time range.
• Click on a slice in a pie chart to filter on that value.

You’ll immediately see how all the panels in your dashboard will focus on the selection you made, quickly providing fresh new views based on your selection. 

And of course, you can always use the search bar to simply type your search term and view all your charts with the most relevant data.

Elastic

Now that we have covered the basics, you can create multiple visualizations, add them to your first dashboard, and start getting insights from your data. 

In the next article, we’ll cover more advanced ways you can leverage Kibana to create pixel-perfect infographics from your data and ways you can visualize your data on top of maps.

If you’re ready to try it out yourself, the easiest way to get started is to take advantage of the free 14-day trial of the Elasticsearch Service on Elastic Cloud—the official hosted Elasticsearch offering from Elastic, which includes Kibana. If you prefer, you can also download Elasticsearch and Kibana to run on your laptop or deploy in a data center. 

Alona Nadler is a senior product manager at Elastic focusing on Kibana. She has spent nearly a decade in the big data and security analytics space and previously helped ArcSight build out their next-generation security analytics solution. Alona is passionate about data analytics, design, and user experience.

—

New Tech Forum provides a venue to explore and discuss emerging enterprise technology in unprecedented depth and breadth. The selection is subjective, based on our pick of the technologies we believe to be important and of greatest interest to InfoWorld readers. InfoWorld does not accept marketing collateral for publication and reserves the right to edit all contributed content. Send all inquiries to newtechforum@infoworld.com.