A hidden Kibana Gem - Static Lookup

April 23, 2020

by Vinh Nguyên

• Elastic Stack

Since Kibana v6.4.0 there are significant improvements and enhancements. Under all these enhancements, a hidden gem and long-awaited feature is finally available: the Static Lookup Field Formatter.

About Static Lookup

What does it do? It translates document keys into human-friendly semantic values. For instance, think of currencies and countries. A currency or country is unique with its ISO-code. Since all humans are not living dictionaries, it is hard to remember all associations.

For those who are working with currencies, like the finance sector, this is always familiar:

756 ⇒ CHF ⇒ Swiss Franc

What about 392? I have to look that up.

392 ⇒ JPY ⇒ Japanese Yen

Create Data Index

First of all, let me give an example dataset. We have currencies and their usage of countries. First, I create an index with field mapping.

PUT currencies
{
  "settings": {
    "index": {
      "number_of_shards": 1,
      "number_of_replicas": 0
    }
  },
  "mappings": {
    "doc": {
      "properties": {
        "code": {
          "type": "keyword"
        },
        "country": {
          "type": "keyword"
        }
      }
    }
  }
}

• The code field contains the currency code and is of type keyword.
• The country field contains an array of countries of type keyword.
• The choice for the keyword type is mandatory since the Kibana Static Lookup works only for text fields.

Insert Documents

We create documents for CHF, EUR and USD.

PUT currencies/doc/756
{"code":"CHF","country":"CH"}

Euro:

PUT currencies/doc/978
{"code":"EUR","country":["AD","AT","BE","CY","EE","FI","FR","DE","GR","GP","IE","IT","LV","LT","LU","MT","MQ","YT","MC","ME","NL","PT","RE","BL","PM","SM","SK","SI","ES"]}

US-Dollar:

PUT currencies/doc/840
{"code":"USD","country":["US","AS","BB","BM","VG","EC","SV","GU","HT","MH","FM","MP","PW","PA","PR","TL","TC","VI","UM"]}

Kibana View

Now we add a Kibana Index Pattern for the currencies index. In the discover view, you see the raw result.

Configure Static Lookup

Go to the Kibana Index Pattern page and press edit on the field currency and country. Choose the Static Lookup from the list.

Enter some values for currencies.

Enter some values for countries. For this demonstration, it is not necessary to translate everything. This gap is intentional to show the differences.

Examine Lookup

The Kibana fields country should have the field formatter Static Lookup. Below picture shows it only for currency. You should see two entries if you reproduce my example.

View Results

If you open the Discover Page, you can see the translated values between the untranslated values.

Summary

One word of advice. Elasticsearch is not a relational database. It is about efficient search. Stop making it look like one.

On the User Interface or Kibana, you should present human-friendly values. With the Static Lookup now we have a standard for this. Thanks to the new feature, Static Lookup, we bridged an understanding gap for unfamiliar data.

I can think of multiple use cases. Think of universal customer id's that you can translate into semantic names if you don't have the customer master data in Elasticsearch.

In the long run, enriching your data with meaningful information is the better approach.

Currencies tend not to change that often, so using the Static Lookup is appropriate in this situation.