Tracing Hadoop using HTrace with Zipkin

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Tracing Hadoop using HTrace with Zipkin

Alexandru Calin

I am trying to use HTrace with Hadoop 2.6.0 on Ubuntu 14.04. I have followed Hadoop's HTrace integration tutorial here.

I have added the following configuration properties to core-site.xml on all nodes ( namenode & datanodes ), as specified in the tutorial:

    <property>
        <name>hadoop.htrace.sampler</name>
        <value>NeverSampler</value>
      </property>
    property>
    <name>hadoop.htrace.spanreceiver.classes</name>
    <value>ZipkinSpanReceiver</value>
  </property>  
  <property>
        <name>hadoop.htrace.zipkin.collector-hostname</name>
        <value>hadoop-master</value>
      </property>
      <property>
        <name>hadoop.htrace.zipkin.collector-port</name>
        <value>9410</value>
      </property>

I have followed the Setting up ZipkinSpanReceiver section, compiled, built & copied the libraries:

/usr/local/hadoop/share/hadoop/common/lib/htrace-core-3.0.4.jar 
/usr/local/hadoop/share/hadoop/common/lib/htrace-zipkin-3.0.4-jar-with-dependencies.jar

I have downloaded Zipkin from here and started the server running the jar.

I have compiled, the TracingFsShell.java, with some minor modifications to including dependencies:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FsShell;
import org.apache.hadoop.tracing.SpanReceiverHost;
import org.apache.hadoop.util.ToolRunner;
import org.htrace.Sampler;
import org.htrace.Trace;
import org.htrace.TraceScope;

public class TracingFsShell {
  public static void main(String argv[]) throws Exception {
    Configuration conf = new Configuration();
    FsShell shell = new FsShell();
    conf.setQuietMode(false);
    shell.setConf(conf);
    SpanReceiverHost.getInstance(conf);
    int res = 0;
    TraceScope ts = null;
    try {
      ts = Trace.startSpan("FsShell", Sampler.ALWAYS);
      res = ToolRunner.run(shell, argv);
    } finally {
      shell.close();
      if (ts != null) ts.close();
    }
    System.exit(res);
  }
}

The modifications came after I wasn't able to build the example and have looked through the classes of the two compiled jars(htrace-core & htrace-zipkin). When I run the example, I get nothing in zipkin.

java -cp .:`hadoop classpath` TracingFsShell -ls /
Found 1 items
drwxr-xr-x   - hduser supergroup          0 2016-07-02 08:28 /user

Am I missing something?

Reply | Threaded
Open this post in threaded view
|

Re: Tracing Hadoop using HTrace with Zipkin

Sandeep Khurana
Never used any of these. But reading documentation of the links you sent "NeverSampler: HTrace is OFF for all spans" . Should you not use either ProbabilitySampler or AlwaysSampler

On Sat, Aug 27, 2016 at 7:22 PM, Alexandru Calin <[hidden email]> wrote:

I am trying to use HTrace with Hadoop 2.6.0 on Ubuntu 14.04. I have followed Hadoop's HTrace integration tutorial here.

I have added the following configuration properties to core-site.xml on all nodes ( namenode & datanodes ), as specified in the tutorial:

    <property>
        <name>hadoop.htrace.sampler</name>
        <value>NeverSampler</value>
      </property>
    property>
    <name>hadoop.htrace.spanreceiver.classes</name>
    <value>ZipkinSpanReceiver</value>
  </property>  
  <property>
        <name>hadoop.htrace.zipkin.collector-hostname</name>
        <value>hadoop-master</value>
      </property>
      <property>
        <name>hadoop.htrace.zipkin.collector-port</name>
        <value>9410</value>
      </property>

I have followed the Setting up ZipkinSpanReceiver section, compiled, built & copied the libraries:

/usr/local/hadoop/share/hadoop/common/lib/htrace-core-3.0.4.jar 
/usr/local/hadoop/share/hadoop/common/lib/htrace-zipkin-3.0.4-jar-with-dependencies.jar

I have downloaded Zipkin from here and started the server running the jar.

I have compiled, the TracingFsShell.java, with some minor modifications to including dependencies:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FsShell;
import org.apache.hadoop.tracing.SpanReceiverHost;
import org.apache.hadoop.util.ToolRunner;
import org.htrace.Sampler;
import org.htrace.Trace;
import org.htrace.TraceScope;

public class TracingFsShell {
  public static void main(String argv[]) throws Exception {
    Configuration conf = new Configuration();
    FsShell shell = new FsShell();
    conf.setQuietMode(false);
    shell.setConf(conf);
    SpanReceiverHost.getInstance(conf);
    int res = 0;
    TraceScope ts = null;
    try {
      ts = Trace.startSpan("FsShell", Sampler.ALWAYS);
      res = ToolRunner.run(shell, argv);
    } finally {
      shell.close();
      if (ts != null) ts.close();
    }
    System.exit(res);
  }
}

The modifications came after I wasn't able to build the example and have looked through the classes of the two compiled jars(htrace-core & htrace-zipkin). When I run the example, I get nothing in zipkin.

java -cp .:`hadoop classpath` TracingFsShell -ls /
Found 1 items
drwxr-xr-x   - hduser supergroup          0 2016-07-02 08:28 /user

Am I missing something?




--
Thanks and regards
Sandeep Khurana
Reply | Threaded
Open this post in threaded view
|

Re: Tracing Hadoop using HTrace with Zipkin

Alexandru Calin
Yes, you are right, sorry, that was I copy-paste mistake form the website, haven't noticed it. In reality I am using AlwawsSampler, in my configuration, 3 slaves, 1 namenode, lxc containers.

On Sat, Aug 27, 2016 at 5:05 PM, Sandeep Khurana <[hidden email]> wrote:
Never used any of these. But reading documentation of the links you sent "NeverSampler: HTrace is OFF for all spans" . Should you not use either ProbabilitySampler or AlwaysSampler

On Sat, Aug 27, 2016 at 7:22 PM, Alexandru Calin <[hidden email]> wrote:

I am trying to use HTrace with Hadoop 2.6.0 on Ubuntu 14.04. I have followed Hadoop's HTrace integration tutorial here.

I have added the following configuration properties to core-site.xml on all nodes ( namenode & datanodes ), as specified in the tutorial:

    <property>
        <name>hadoop.htrace.sampler</name>
        <value>NeverSampler</value>
      </property>
    property>
    <name>hadoop.htrace.spanreceiver.classes</name>
    <value>ZipkinSpanReceiver</value>
  </property>  
  <property>
        <name>hadoop.htrace.zipkin.collector-hostname</name>
        <value>hadoop-master</value>
      </property>
      <property>
        <name>hadoop.htrace.zipkin.collector-port</name>
        <value>9410</value>
      </property>

I have followed the Setting up ZipkinSpanReceiver section, compiled, built & copied the libraries:

/usr/local/hadoop/share/hadoop/common/lib/htrace-core-3.0.4.jar 
/usr/local/hadoop/share/hadoop/common/lib/htrace-zipkin-3.0.4-jar-with-dependencies.jar

I have downloaded Zipkin from here and started the server running the jar.

I have compiled, the TracingFsShell.java, with some minor modifications to including dependencies:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FsShell;
import org.apache.hadoop.tracing.SpanReceiverHost;
import org.apache.hadoop.util.ToolRunner;
import org.htrace.Sampler;
import org.htrace.Trace;
import org.htrace.TraceScope;

public class TracingFsShell {
  public static void main(String argv[]) throws Exception {
    Configuration conf = new Configuration();
    FsShell shell = new FsShell();
    conf.setQuietMode(false);
    shell.setConf(conf);
    SpanReceiverHost.getInstance(conf);
    int res = 0;
    TraceScope ts = null;
    try {
      ts = Trace.startSpan("FsShell", Sampler.ALWAYS);
      res = ToolRunner.run(shell, argv);
    } finally {
      shell.close();
      if (ts != null) ts.close();
    }
    System.exit(res);
  }
}

The modifications came after I wasn't able to build the example and have looked through the classes of the two compiled jars(htrace-core & htrace-zipkin). When I run the example, I get nothing in zipkin.

java -cp .:`hadoop classpath` TracingFsShell -ls /
Found 1 items
drwxr-xr-x   - hduser supergroup          0 2016-07-02 08:28 /user

Am I missing something?




--
Thanks and regards
Sandeep Khurana

Reply | Threaded
Open this post in threaded view
|

Re: Tracing Hadoop using HTrace with Zipkin

Alexandru Calin
So, for everyone else out there, the thing is that there is a certain tutorial to be followed for 2.6.0 version of Hadoop, and it can be found here : 
That solves the problem.

On Sat, Aug 27, 2016 at 5:18 PM, Alexandru Calin <[hidden email]> wrote:
Yes, you are right, sorry, that was I copy-paste mistake form the website, haven't noticed it. In reality I am using AlwawsSampler, in my configuration, 3 slaves, 1 namenode, lxc containers.

On Sat, Aug 27, 2016 at 5:05 PM, Sandeep Khurana <[hidden email]> wrote:
Never used any of these. But reading documentation of the links you sent "NeverSampler: HTrace is OFF for all spans" . Should you not use either ProbabilitySampler or AlwaysSampler

On Sat, Aug 27, 2016 at 7:22 PM, Alexandru Calin <[hidden email]> wrote:

I am trying to use HTrace with Hadoop 2.6.0 on Ubuntu 14.04. I have followed Hadoop's HTrace integration tutorial here.

I have added the following configuration properties to core-site.xml on all nodes ( namenode & datanodes ), as specified in the tutorial:

    <property>
        <name>hadoop.htrace.sampler</name>
        <value>NeverSampler</value>
      </property>
    property>
    <name>hadoop.htrace.spanreceiver.classes</name>
    <value>ZipkinSpanReceiver</value>
  </property>  
  <property>
        <name>hadoop.htrace.zipkin.collector-hostname</name>
        <value>hadoop-master</value>
      </property>
      <property>
        <name>hadoop.htrace.zipkin.collector-port</name>
        <value>9410</value>
      </property>

I have followed the Setting up ZipkinSpanReceiver section, compiled, built & copied the libraries:

/usr/local/hadoop/share/hadoop/common/lib/htrace-core-3.0.4.jar 
/usr/local/hadoop/share/hadoop/common/lib/htrace-zipkin-3.0.4-jar-with-dependencies.jar

I have downloaded Zipkin from here and started the server running the jar.

I have compiled, the TracingFsShell.java, with some minor modifications to including dependencies:

import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FsShell;
import org.apache.hadoop.tracing.SpanReceiverHost;
import org.apache.hadoop.util.ToolRunner;
import org.htrace.Sampler;
import org.htrace.Trace;
import org.htrace.TraceScope;

public class TracingFsShell {
  public static void main(String argv[]) throws Exception {
    Configuration conf = new Configuration();
    FsShell shell = new FsShell();
    conf.setQuietMode(false);
    shell.setConf(conf);
    SpanReceiverHost.getInstance(conf);
    int res = 0;
    TraceScope ts = null;
    try {
      ts = Trace.startSpan("FsShell", Sampler.ALWAYS);
      res = ToolRunner.run(shell, argv);
    } finally {
      shell.close();
      if (ts != null) ts.close();
    }
    System.exit(res);
  }
}

The modifications came after I wasn't able to build the example and have looked through the classes of the two compiled jars(htrace-core & htrace-zipkin). When I run the example, I get nothing in zipkin.

java -cp .:`hadoop classpath` TracingFsShell -ls /
Found 1 items
drwxr-xr-x   - hduser supergroup          0 2016-07-02 08:28 /user

Am I missing something?




--
Thanks and regards
Sandeep Khurana