If a replica shard is in an INITIALIZATION state and the primary shard is healthy, then the shard is being replicated from the primary to the replica. You can use the cat API to get this state:
|
1 2 3 4 5 |
$ curl '0:9200/_cat/shards' prod_article_index 0 p STARTED 13708596 55.2gb 192.168.27.181 es-a05.mycompany.com prod_article_index 0 r INITIALIZING 192.168.25.158 es-a04.mycompany.com prod_article_index 1 r INITIALIZING 192.168.25.158 es-a04.mycompany.com prod_article_index 1 p STARTED 13710169 55gb 192.168.11.214 es-m04.mycompany.com |
Now, we have no clue from this view what progress has been made, if any. On large shards, it may even look like things are frozen. How do we gain insight into what is happening? Well, thankfully, there’s a status page which is called like so:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 |
$ curl '0:9200/my_index/_recovery?pretty=true' ... "index" : { "files" : { "total" : 516, "reused" : 0, "recovered" : 468, "percent" : "90.7%" }, "bytes" : { "total" : 59067844341, "reused" : 0, "recovered" : 29729874095, "percent" : "50.3%" }, "total_time_in_millis" : 0 }, ... |
Queue size reached
|
1 2 3 4 5 6 |
$ curl '0:9200/_cat/thread_pool?v' host ip bulk.active bulk.queue bulk.rejected index.active index.queue index.rejected search.active search.queue search.rejected es-m05.mycompany.com 192.168.9.141 0 0 0 0 0 0 0 0 0 es-a04.mycompany.com 192.168.25.158 0 0 0 0 0 0 0 0 0 es-a05.mycompany.com 192.168.27.181 0 0 0 0 0 0 1 0 83425 es-m04.mycompany.com 192.168.11.214 0 0 0 0 0 0 1 0 77815 |
To increase your queue size, you can add the following to elasticsearch.yml. Replace “bulk” and “search” with the appropriate thread pool name, along with a reasonable value. You can find the list of those at http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-threadpool.html .
|
1 2 |
threadpool.bulk.queue_size: 3000 threadpool.search.queue_size: 3000 |
Leave a Reply