ZestSync

TL;DR : How to switch elastic search hosts at runtime using spring-data-elasticsearch?

We have a spring boot application (v : 2.5.2) using spring data elasticsearch (v: 4.2.2).

I am using java configurations to initialize ES connection as mentioned in High Level Rest Client section of spring's documentation.

We have a requirement where we need to change the elasticsearch hosts at runtime and not manually restart our servers.

I tried restarting spring IOC on hostname config change using something similar to spring actuator's RestartEndpoint. But this seems to work only in spring's embedded tomcat and not with the external tomcat that we use.

Is there an alternate way to do this?

[This is required in case there is a disaster and we need to switch to backup]

2 Answers

Nothing out of the box.

One idea (I did not try it out): You'd need to create your own implementation of the ElasticsearchOperations interface, lets call it MyOperations and return that from the AbstractElasticsearchConfiguration#elasticsearchOperations() method which your configuration inherits. In this MyOperations implementation (which is a singleton Spring bean), you'd create a delegate ElasticsearchOperations with the same code like in the configuration class that is connected to normal cluster. Every method in your implementation then delegates to the corresponding method of the delegate.

In the failure case, your implementation would create a new delegate that now connects to the secondary cluster and replace the first one with it. Or create both and implement a toggle feature to switch between the two.

Expanding on the answer provided by P.J.Meisch,

Create a custom implementation for ElasticsearchOperations

public class CustomElasticsearchOperations implements ElasticsearchOperations { private ElasticsearchOperations delegateElasticSearchOp; /** * Main Constructor to create the instance using existing elasticsearchConverter bean * @param elasticsearchConverter : ElasticsearchConverter */ public CustomElasticsearchOperations(ElasticsearchConverter elasticsearchConverter) { updateElasticsearchOperationDelegate(elasticsearchConverter); } private ElasticsearchOperations createNewElasticsearchOperationDelegate(ElasticsearchConverter elasticsearchConverter){ return new ElasticsearchRestTemplate(ElasticsearchRestHighLevelClientProvider.INSTANCE.elasticsearchClient(),elasticsearchConverter); } /** * Method to update the delegate ElasticsearchRestTemplate to register elasticsearch configurations * @param elasticsearchConverter : Default spring lib provided ElasticsearchConverter instance */ public void updateElasticsearchOperationDelegate(ElasticsearchConverter elasticsearchConverter){ this.delegateElasticSearchOp = createNewElasticsearchOperationDelegate(elasticsearchConverter); } /* * ElasticsearchOperations method implementation using Delegate ElasticsearchRestTemplatea */ @Override public IndexOperations indexOps(Class<?> aClass) { return delegateElasticSearchOp.indexOps(aClass); } @Override public IndexOperations indexOps(IndexCoordinates indexCoordinates) { return delegateElasticSearchOp.indexOps(indexCoordinates); } @Override public ClusterOperations cluster() { return delegateElasticSearchOp.cluster(); } @Override public ElasticsearchConverter getElasticsearchConverter() { return delegateElasticSearchOp.getElasticsearchConverter(); } // And so on... delegate the impl of overriden method to existing ElasticsearchRestTemplate } 

Where ElasticsearchRestHighLevelClientProvider is a class that holds the logic to create object of RestHighLevelClient that holds the values for current cluster configurations

// This is a sample from spring documentation. Create your own rest client with your own configs public enum ElasticsearchRestHighLevelClientProvider{ INSTANCE; // below is a sample, change it according to your requirements. public RestHighLevelClient elasticsearchClient() { String hosts [] = getLatestConfigs().getHosts(); final ClientConfiguration clientConfiguration = ClientConfiguration.builder() .connectedTo(hosts) .build(); return RestClients.create(clientConfiguration).rest(); } // your code for fetching cluster configs.. } 

And finally, register the bean of CustomElasticsearchOperations

@Configuration public class ElasticSearchAutoConfigurations extends ElasticsearchConfigurationSupport { @Bean( name = {"elasticsearchOperations", "elasticsearchTemplate"} ) public ElasticsearchOperations elasticsearchOperations(ElasticsearchConverter elasticsearchConverter) { return new CustomElasticsearchOperations(elasticsearchConverter); } } 

Now when your configurations change you can Autowire the instances of ElasticsearchOperations and ElasticsearchConverter beans, and invoke ElasticsearchOperations#updateElasticsearchOperationDelegate(elasticsearchConverter)

This will replace the instance of ElasticsearchRestTemplate within CustomElasticsearchOperations singleton bean with the new configs, and new requests will go to the cluster you want to switch to.

Note that

• If you have autowired ElasticsearchRestTemplate to make non spring repository based calls to elasticsearch, change the autowired candidate to parent interface ElasticsearchOperations as ElasticsearchRestTemplate's instance will not be present in the application context

• ElasticsearchConverter is already created within ElasticsearchConfigurationSupport class. So no need to create this bean again

ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJobmluYW1%2FdYOOrKeroZ6ceqWt05pknqSRqMGqr9KemKubmGK%2FprXNoquimZyewKZ5xKWYrKyZmMCmrdGcn2agn6jBtHnCpaysrJWnerGt0qyuqKqUYr2z