[jira] [Commented] (NUTCH-2739) indexer-elastic: Upgrade ES and migrate to REST client

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (NUTCH-2739) indexer-elastic: Upgrade ES and migrate to REST client

Tim Allison (Jira)

    [ https://issues.apache.org/jira/browse/NUTCH-2739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16969578#comment-16969578 ]

ASF GitHub Bot commented on NUTCH-2739:
---------------------------------------

sebastian-nagel commented on pull request #484: NUTCH-2739 : Upgrade ES and migrate to REST client
URL: https://github.com/apache/nutch/pull/484#discussion_r343888178
 
 

 ##########
 File path: src/plugin/indexer-elastic/src/test/org/apache/nutch/indexwriter/elastic/TestElasticIndexWriter.java
 ##########
 @@ -20,40 +20,31 @@
 
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.mapreduce.Job;
+import org.apache.http.HttpHost;
 import org.apache.nutch.indexer.IndexWriterParams;
 import org.apache.nutch.indexer.NutchDocument;
 import org.apache.nutch.util.NutchConfiguration;
-import org.elasticsearch.action.Action;
-import org.elasticsearch.action.ActionListener;
-import org.elasticsearch.action.ActionRequest;
-import org.elasticsearch.action.ActionRequestBuilder;
-import org.elasticsearch.action.ActionResponse;
-import org.elasticsearch.action.DocWriteRequest.OpType;
 import org.elasticsearch.action.bulk.BulkProcessor;
 import org.elasticsearch.action.bulk.BulkRequest;
-import org.elasticsearch.action.bulk.BulkItemResponse;
 import org.elasticsearch.action.bulk.BulkResponse;
-import org.elasticsearch.action.index.IndexResponse;
-import org.elasticsearch.client.Client;
-import org.elasticsearch.client.support.AbstractClient;
-import org.elasticsearch.common.settings.Settings;
-import org.elasticsearch.common.util.concurrent.EsRejectedExecutionException;
-import org.elasticsearch.index.shard.ShardId;
-import org.elasticsearch.threadpool.ThreadPool;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestHighLevelClient;
 import org.junit.Assert;
 import org.junit.Before;
 import org.junit.Test;
 
 import java.util.HashMap;
 import java.util.Map;
-import java.util.UUID;
+
+/*Test works only when there is a elastic client setup is already there.
+ * Because Rest Client requires a elastic server connection for it to function unlike Transport client*/
 
 Review comment:
   But maybe there is a way to mock the client, see [CustomRestHighLevelClientTests](/elastic/elasticsearch/blob/master/client/rest-high-level/src/test/java/org/elasticsearch/client/CustomRestHighLevelClientTests.java).
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


> indexer-elastic: Upgrade ES and migrate to REST client
> ------------------------------------------------------
>
>                 Key: NUTCH-2739
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2739
>             Project: Nutch
>          Issue Type: Improvement
>          Components: indexer, plugin
>    Affects Versions: 1.15
>            Reporter: Sebastian Nagel
>            Priority: Major
>             Fix For: 1.17
>
>
> The indexer-elastic plugin is based on 5.3.0 and should be upgraded to the most recent Elasticsearch version (7.3.0 or upwards).
> [TransportClient|https://www.elastic.co/guide/en/elasticsearch/client/java-api/7.3/transport-client.html] has been deprecated in ES 7.x and will be removed in 8.x. We should migrate to using the [REST client|https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.3/java-rest-high.html] and also check whether this would obsolete the indexer-elastic-rest plugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)