hadoop►elasticsearch►how to tune for search speed

elasticsearch how to tune for search spped

How to Tune for search speed

  1. Give memory to the filesystem cache

    1
    2
    Elasticsearch heavily relies on the filesystem cache in order to make search fast.
    In general, you should make sure that at least half the available memory goes to the filesystem cache so that elasticsearch can keep hot regions of the index in physical memory.

  2. Use faster hardware

    1
    2
    If your search is I/O bound, you should investigate giving more memory to the filesystem cache (see above) or buying faster drives.
    If your search is CPU-bound, you should investigate buying faster CPUs.

  3. Document modeling

    1
    Documents should be modeled so that search-time operations are as cheap as possible.

  4. Pre-index data

    1
    2
    You should leverage patterns in your queries to optimize the way data is indexed.
    For instance, if all your documents have a price field and most queries run range aggregations on a fixed list of ranges, you could make this aggregation faster by pre-indexing the ranges into the index and using a terms aggregations.

  5. Mappings

    1
    The fact that some data is numeric does not mean it should always be mapped as a numeric field.

  6. Avoid scripts

    1
    In general, scripts should be avoided

  7. Search rounded dates

    1
    Queries on date fields that use now are typically not cacheable since the range that is being matched changes all the time. However switching to a rounded date is often acceptable in terms of user experience, and has the benefit of making better use of the query cache.

  8. Force-merge read-only indices

    1
    Indices that are read-only would benefit from being merged down to a single segment. This is typically the case with time-based indices: only the index for the current time frame is getting new documents while older indices are read-only.

  9. Warm up global ordinals

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    Global ordinals are a data-structure that is used in order to run terms aggregations on keyword fields.
    You can tell elasticsearch to load global ordinals eagerly at refresh-time by configuring mappings as described below:
    curl -XPUT 'localhost:9200/index?pretty' -d'
    {
      "mappings": {
        "type": {
          "properties": {
            "foo": {
              "type": "keyword",
              "eager_global_ordinals": true
            }
          }
        }
      }
    }'

  10. Warm up the filesystem cache

    1
    If the machine running elasticsearch is restarted, the filesystem cache will be empty, so it will take some time before the operating system loads hot regions of the index into memory so that search operations are fast. You can explicitly tell the operating system which files should be loaded into memory eagerly depending on the file extension using the index.store.preload setting