Batch size is too large, size must be less than or equal to: [10000] #156
Description
Hi,
Is there an example to make a request with scroll batch because when i try to retrieve all indexed data, i receive the error :
Exception in thread "main" org.scalatest.exceptions.TestFailedException: The future returned an exception of type: com.sumologic.elasticsearch.restlastic.RestlasticSearchClient$ReturnTypes$ElasticErrorResponse, with message: ElasticsearchError(status=500): JString({"error":{"root_cause":[{"type":"query_phase_execution_exception","reason":"Batch size is too large, size must be less than or equal to: [10000] but was [10029]. Scroll batch sizes cost as much memory as result windows so they are controlled by the [index.max_result_window] index level setting."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"materials","node":"zGodcjEaSrCXFO9_SJuvzg","reason":{"type":"query_phase_execution_exception","reason":"Batch size is too large, size must be less than or equal to: [10000] but was [10029]. Scroll batch sizes cost as much memory as result windows so they are controlled by the [index.max_result_window] index level setting."}}],"caused_by":{"type":"query_phase_execution_exception","reason":"Batch size is too large, size must be less than or equal to: [10000] but was [10029]. Scroll batch sizes cost as much memory as result windows so they are controlled by the [index.max_result_window] index level setting."}},"status":500}).
here my code
var resFutQuery = restClient.query(index, tpe, new QueryRoot(MatchAll))
var total = 10
whenReady(resFutQuery , timeout(Span(10, Minutes))) { res =>
total = res.rawSearchResponse.hits.total;
}
val resFutScroll = restClient.startScrollRequest(index, tpe, new QueryRoot(MatchAll, sizeOpt = Some(total)))
whenReady(resFutScroll, timeout(Span(10, Minutes))) { res =>
res._2.rawSearchResponse.hits.hits.foreach(
x => {
Process
}
)
}
Thank you in advance