[jira] [Commented] (NUTCH-2739) indexer-elastic: Upgrade ES and migrate to REST client

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (NUTCH-2739) indexer-elastic: Upgrade ES and migrate to REST client

David Pilato (Jira)

    [ https://issues.apache.org/jira/browse/NUTCH-2739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16976067#comment-16976067 ]

ASF GitHub Bot commented on NUTCH-2739:
---------------------------------------

balashashanka commented on pull request #484: NUTCH-2739 : Upgrade ES and migrate to REST client
URL: https://github.com/apache/nutch/pull/484#discussion_r347143933
 
 

 ##########
 File path: src/plugin/indexer-elastic/src/test/org/apache/nutch/indexwriter/elastic/TestElasticIndexWriter.java
 ##########
 @@ -20,40 +20,31 @@
 
 import org.apache.hadoop.conf.Configuration;
 import org.apache.hadoop.mapreduce.Job;
+import org.apache.http.HttpHost;
 import org.apache.nutch.indexer.IndexWriterParams;
 import org.apache.nutch.indexer.NutchDocument;
 import org.apache.nutch.util.NutchConfiguration;
-import org.elasticsearch.action.Action;
-import org.elasticsearch.action.ActionListener;
-import org.elasticsearch.action.ActionRequest;
-import org.elasticsearch.action.ActionRequestBuilder;
-import org.elasticsearch.action.ActionResponse;
-import org.elasticsearch.action.DocWriteRequest.OpType;
 import org.elasticsearch.action.bulk.BulkProcessor;
 import org.elasticsearch.action.bulk.BulkRequest;
-import org.elasticsearch.action.bulk.BulkItemResponse;
 import org.elasticsearch.action.bulk.BulkResponse;
-import org.elasticsearch.action.index.IndexResponse;
-import org.elasticsearch.client.Client;
-import org.elasticsearch.client.support.AbstractClient;
-import org.elasticsearch.common.settings.Settings;
-import org.elasticsearch.common.util.concurrent.EsRejectedExecutionException;
-import org.elasticsearch.index.shard.ShardId;
-import org.elasticsearch.threadpool.ThreadPool;
+import org.elasticsearch.client.RestClient;
+import org.elasticsearch.client.RestHighLevelClient;
 import org.junit.Assert;
 import org.junit.Before;
 import org.junit.Test;
 
 import java.util.HashMap;
 import java.util.Map;
-import java.util.UUID;
+
+/*Test works only when there is a elastic client setup is already there.
+ * Because Rest Client requires a elastic server connection for it to function unlike Transport client*/
 
 Review comment:
   Hi Sebastian, I went through the CustomRestHighLevelClientTests, and it seems it is to test Rest High Level client changes. This will mock the client itself. But we need to mock the server with requests and response. So can we go ahead and not do the tests at all?
 
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


> indexer-elastic: Upgrade ES and migrate to REST client
> ------------------------------------------------------
>
>                 Key: NUTCH-2739
>                 URL: https://issues.apache.org/jira/browse/NUTCH-2739
>             Project: Nutch
>          Issue Type: Improvement
>          Components: indexer, plugin
>    Affects Versions: 1.15
>            Reporter: Sebastian Nagel
>            Priority: Major
>             Fix For: 1.17
>
>
> The indexer-elastic plugin is based on 5.3.0 and should be upgraded to the most recent Elasticsearch version (7.3.0 or upwards).
> [TransportClient|https://www.elastic.co/guide/en/elasticsearch/client/java-api/7.3/transport-client.html] has been deprecated in ES 7.x and will be removed in 8.x. We should migrate to using the [REST client|https://www.elastic.co/guide/en/elasticsearch/client/java-rest/7.3/java-rest-high.html] and also check whether this would obsolete the indexer-elastic-rest plugin.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)