Webiny Docs home page
What you'll learn
  • how to customize Elasticsearch index for each application
This article requires advanced knowledge of the Elasticsearch.

Overview
anchor

Where Do We Use the Elasticsearch Index?
anchor

We use the Elasticsearch indexes for our File Manager, Form Builder, Headless CMS and Page Builder applications.

In File Manager, Form Builder and Page Builder applications we use one index per tenant and locale combination, if locale specific indexes are set.

In Headless CMS we use index for each tenant, locale and model combination because of the complexity of the data stored into Elasticsearch.

By the default, we have a basic setup for applications that use Elasticsearch:

Also, each of the applications has its own default plugin for the index:

Each of those plugins define how the Elasticsearch index for that particular application will look like when it is created. The plugins use the base mapping configuration external link.

We do not store all the data into the Elasticsearch because there is no need for it (eg. Headless CMS Models and Groups).
Elasticsearch index is created when adding new locale, which also includes the initial installation of the system.

When Would You Use the Possibility to Change the Elasticsearch Index?
anchor

Good example on when to change the default index configuration, or add a new one, is when you are using a locale which is not supported by our default template, for example: Japanese.

The Japanese Elasticsearch index configuration will look a lot different from our default one because of different text analyzers and mapping.

There is official Elasticsearch blog external link on how to implement Japanese full-text search.

The last added plugin is always used to create the index, assuming it is passing the `.canUse()` check on the plugin.

Enabling Locale in the Index Name
anchor

In the version 5.26.0 we introduced an environment variable WEBINY_ELASTICSEARCH_INDEX_LOCALE which, when set to true - default in 5.26.0, will add a locale code to the created index name. Prior to 5.26.0 we did not add the locale in the Page Builder, the Form Builder and the File Manager, and we will not change that for existing projects.

All new projects that are created with 5.26.0, or greater, will have the locale code in the index name.

To read on how to enable locale code in your existing project, created prior to 5.26.0, read this article.

Available Plugins
anchor

File Manager
anchor

For the File Manager Files we have the FileElasticsearchIndexPlugin external link.

Form Builder
anchor

For the Form Builder Forms and Submissions we have the FormElasticsearchIndexPlugin external link.

Headless CMS
anchor

For the Headless CMS Entries we have CmsEntryElasticsearchIndexPlugin external link.

Page Builder
anchor

For the Page Builder Pages we have the PageElasticsearchIndexPlugin external link.

How to Create Your Own Index Configuration?
anchor

Rules
anchor

There is a single rule that you will need to follow if you want to successfully create the index with your configuration:

The plugin you create will only get applied if the .canUse() method is passing the check.

We intended it to be used to check if the index configuration can be applied given the locale that is set at the moment index is going to be created.

If you want the index configuration to be always applied, no matter the locale, just return `true` in the `.canUse()` method in the plugin.

How Are the Index Names Created?
anchor

Overview
anchor

Index name can contain dashes (-), underscores (_), numbers (0-9) and english letters (a-z). It is crated by our configuration method, so you do not need to worry about that.

You can set the ELASTICSEARCH_SHARED_INDEXES environment variable to true to create a single index for all tenants for each application (or for each model in case of the Headless CMS), but we do not encourage it.

You can also set the ELASTIC_SEARCH_INDEX_PREFIX environment variable to, for example, my_prefix_ to add that prefix to every index created. Use case for this would be for when you have multiple environments on a single Elasticsearch instance. You can prefix each environment indexes with, for example, dev_, prod_, staging_, test_, etc…

File Manager
anchor

Index name for the File Manager Files is built out of the tenant id, locale code if enabled, and -file-manager suffix. If your tenant is a root tenant, the index will look like root-file-manager. If your tenant is a root tenant, and you have locale (en-US) for index name enabled, the index will look like root-en-us-file-manager.

You can see how the index name is created in this external link file.

Form Builder
anchor

Index name for the Form Builder Forms and Submissions is built out of the tenant id, locale code if enabled, and -form-builder suffix. If your tenant is a root tenant, the index will look like root-form-builder. If your tenant is a root tenant, and you have locale (jp) for index name enabled, the index will look like root-jp-form-builder.

You can see how the index name is created in this external link file.

Headless CMS
anchor

Index name for the Headless CMS Entries is built out of tenant id, -headless-cms, locale code and model id. If your tenant is a root tenant, locale is en-US and model is Articles, the index will look like root-headless-cms-en-us-articles.

You can see how the index name is created in this external link file.

Page Builder
anchor

Index name for the Page Builder Pages is built out of the tenant id, locale code if enabled, and -page-builder suffix. If your tenant is a root tenant, the index will look like root-page-builder. If your tenant is a root tenant, and you have locale (de-DE) for index name enabled, the index will look like root-de-de-page-builder.

You can see how the index name is created in this external link file.

Examples
anchor

Headless CMS index plugin CmsEntryElasticsearchIndexPlugin, which disables mappings on a largeText field in de locale:

import { base as baseConfiguration } from "@webiny/api-elasticsearch/indexConfiguration/base";
import { CmsEntryElasticsearchIndexPlugin } from "@webiny/api-headless-cms-ddb-es/plugins/CmsEntryElasticsearchIndexPlugin";
const plugin = new CmsEntryElasticsearchIndexPlugin({
  body: {
    ...baseConfiguration,
    mappings: {
      ...baseConfiguration.mappings,
      properties: {
        ...baseConfiguration.mappings.properties,
        // we do not want to map a field with name "largeText"
        largeText: {
          enabled: false
        }
      }
    }
  },
  locales: ["de"]
});

Page Builder index plugin PageElasticsearchIndexPlugin, which disables indexing on a largeText field on all locales.

import { base as baseConfiguration } from "@webiny/api-elasticsearch/indexConfiguration/base";
import { PageElasticsearchIndexPlugin } from "@webiny/api-page-builder-so-ddb-es/plugins/definitions/PageElasticsearchIndexPlugin";
const plugin = new PageElasticsearchIndexPlugin({
  order: 351,
  body: {
    ...baseConfiguration,
    mappings: {
      ...baseConfiguration.mappings,
      properties: {
        ...baseConfiguration.mappings.properties,
        // we do not want to index a field with name "largeText"
        largeText: {
          enabled: false
        }
      }
    }
  }
});

Useful Links
anchor