Add local flag and master_timeout support to cluster pending tasks API#116
Open
augment-app-staging[bot] wants to merge 1649 commits intomainfrom
Open
Add local flag and master_timeout support to cluster pending tasks API#116augment-app-staging[bot] wants to merge 1649 commits intomainfrom
augment-app-staging[bot] wants to merge 1649 commits intomainfrom
Conversation
…and has_parent queries and filters. Closes elastic#4534
also double check that once a recycler is released, it can't be released again or used
also double check that once a recycler is released, it can't be released again or used
…eue. This prevents missing very short timeouts which fire before the calling thread had the chance to add the task to the queue and are therefore ignored. This is mostly of importance for testing where we explicitly want tasks to timeout and set it to a very low value.
Double-release protection added in 1c758b0 made QueueRecycler throw NPEs when trying to recycle existing instances.
Added copy of SloppyMath.java from lucene 4.6+ and setup GeoDistance for new haversin method closes elastic#3862
* adapt tests * introduced default GeoDistance function * Updated docs closes elastic#4498
Tests now ensure that the computed distance is correct within 1‰ instead of 1‱.
… directly to the codebase (no submodule) within rest-api-spec (temporarily disabled FileUtilsTests & REST tests as there's temporarily no rest-spec dir) Relates to elastic#4540 elastic#4376
Closes elastic#4540 Relates to elastic#4376
…ded back fixed rest-api-spec paths in TESTING docs Relates to elastic#4540 & elastic#4376
Currently we only test if readers are correctly released when exceptions occur during reopen or flush. This commit adds a test that randomly throws exceptions during the search execution ie. when Terms are pulled or if a docs enum is created.
We are mocking out some functionality to add assertions etc. or randomize store types. We should randomly run with our defaults to make sure we don't hide any potential problems.
When an analysis plugins provides default index settings using `PreBuiltAnalyzerProviderFactory`, `PreBuiltTokenFilterFactoryFactory` or `PreBuiltTokenizerFactoryFactory` it fails when upgrading it with elasticsearch superior or equal to 0.90.5. Related issue: elastic#4936 Fix is needed in core. But, in the meantime, analysis plugins developers can fix that issue by overloading default prebuilt factories. For example: ```java public class StempelAnalyzerProviderFactory extends PreBuiltAnalyzerProviderFactory { private final PreBuiltAnalyzerProvider analyzerProvider; public StempelAnalyzerProviderFactory(String name, AnalyzerScope scope, Analyzer analyzer) { super(name, scope, analyzer); analyzerProvider = new PreBuiltAnalyzerProvider(name, scope, analyzer); } @OverRide public AnalyzerProvider create(String name, Settings settings) { return analyzerProvider; } public Analyzer analyzer() { return analyzerProvider.get(); } } ``` And instead of: ```java @Inject public PolishIndicesAnalysis(Settings settings, IndicesAnalysisService indicesAnalysisService) { super(settings); indicesAnalysisService.analyzerProviderFactories().put("polish", new PreBuiltAnalyzerProviderFactory("polish", AnalyzerScope.INDICES, new PolishAnalyzer(Lucene.ANALYZER_VERSION))); } ``` do ```java @Inject public PolishIndicesAnalysis(Settings settings, IndicesAnalysisService indicesAnalysisService) { super(settings); indicesAnalysisService.analyzerProviderFactories().put("polish", new StempelAnalyzerProviderFactory("polish", AnalyzerScope.INDICES, new PolishAnalyzer(Lucene.ANALYZER_VERSION))); } ``` Closes elastic#5030
RefrenceManager#accquire() See LUCENE-5436
Multiple nodes are now started when running REST tests against the `TestCluster` (default randomized settings are now used instead of the hardcoded `1`) Added also randomized round-robin based on all available nodes, and ability to provide multiple addresses when running tests against an external cluster to have the same behaviour
… throwing an exception.
…ocessed events closes elastic#5061
delete/50_refresh tests per shard refresh using refresh:true in delete api. We might run into troubles though if we have a replica that gets initialized after a doc was indexed and deleted, without a refresh, as that doc won't be found when searching against that specific replica shard (as a refresh happens automatically before a replica gets exposed as started).
… multiple nodes Replaced also wait for yellow with wait for green when using no replicas. get_source/60_realtime_refresh tests per shard refresh using refresh:true and realtime:true in get api. We might run into troubles though if we have a replica that gets initialized after a doc was indexed without a refresh, as that doc will be found when searching against that specific replica shard (as a refresh happens automatically before a replica gets exposed as started).
REST tests get run against either 1 node or multiple nodes. Wait for yellow with replicas>0 is not enough when running against multiple nodes as replicas shard might get initialized during testing, which can cause timing issues.
AndDocIdSet#IteratorBasedIterator was potentially initialized with NO_MORE_DOCS which violates the initial state of DocIdSetIterator and could lead to undefined behavior when used in a search context. Closes elastic#5049
more assertAcked, removed restriction on number of nodes, doesn't seem needed.
DRF similarity -> DFR similarity
Add backwards compatibility support for the cluster pending tasks API (/_cluster/pending_tasks) by adding: - local flag: When set to true, returns pending tasks from the local node rather than the master node, consistent with other cluster APIs. - master_timeout parameter: Timeout for connecting to the master node. Changes: - PendingClusterTasksRequest: Added local field with serialization support - TransportPendingClusterTasksAction: Override localExecute() to respect local flag - RestPendingClusterTasksAction: Parse local and master_timeout from REST params - PendingClusterTasksRequestBuilder: Added setLocal() convenience method - Added REST API spec file for cluster.pending_tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
localflag andmaster_timeoutparameter support to the cluster pending tasks API (/_cluster/pending_tasks) for backwards compatibility with the 0.90 branch.Changes
PendingClusterTasksRequestlocalboolean field (default:false) with getter/setterreadFrom/writeTo) for thelocalfieldMasterNodeOperationRequestwhich providesmasterNodeTimeoutTransportPendingClusterTasksActionlocalExecute()to respect thelocalflag, allowing the request to be handled on the local node whenlocal=trueRestPendingClusterTasksActionmaster_timeoutparameter from REST request and set on the request objectlocalparameter from REST request and set on the request objectPendingClusterTasksRequestBuildersetLocal(boolean)convenience methodREST API Spec
rest-api-spec/api/cluster.pending_tasks.jsonwithlocalandmaster_timeoutparametersPattern
Follows the same pattern used by other cluster APIs such as
ClusterHealthAction,ClusterStateAction, etc.Triggered by: Slack-Bot-Evals
Slack thread: https://augment-wic8570.slack.com/archives/C0AG2MXKEQJ/p1772675349340899