Spring Sleuth - Tracing Failures -

in microservice environment see 2 main benefits tracing requests through microservice instances on entire business process.

1. finding latency gaps between or in service instances
2. finding roots of failures, whether technical or regarding business case

with zipkin there tool, addresses first issue. how can tracing used unveil failures in microservice landscape? want trace error afflicted spans, not each request, nothing went wrong. mentioned here custom sampler used.

alternatively, may register own sampler bean definition , programmatically make decision requests should sampled. can make more intelligent choices things trace, example, ignoring successful requests, perhaps checking whether component in error state, or else.

so tried implement that, doesn't work or used wrong. so, blog post suggested registered own sampler:

    @bean     sampler customsampler() {     return new sampler() {         @override         public boolean issampled(span span) {              boolean iserrorspan = false;             for(string tagkey : span.tags().keyset()){                 if(tagkey.startswith("error_")){                     iserrorspan = true;                 }             }             return iserrorspan ;         }     }; } 

and in controller create new span, being tagged error if exception raises

private final tracer tracer;  @autowired public democontroller(tracer tracer) {     this.tracer = tracer; }  @requestmapping(value = "/calc/{i}") public string calc(@pathvariable string i){     span span = null;     try {         span = this.tracer.createspan("my_business_logic");         return "1 / " + + " = " + new float(1.0 / integer.parseint(i)).tostring();     }catch(exception ex){         log.error(ex.getmessage(), ex);          span.logevent("error: " + ex.getmessage());         this.tracer.addtag("error_" + ex.hashcode(), ex.getmessage());         throw ex;     }     finally{         this.tracer.close(span);     } } 

now, doesn't work. if request /calc/a method sampler.issampled(span) being called before controller method throws numberformatexception. means, when issampled() checks span, has no tags yet. , sampler method not being called again later in process. if open sampler , allow every span sampled, see tagged error-span later on in zipkin. in case sampler.issampled(span) called 1 time httpzipkinspanreporter.report(span) executed 3 times.

so use case like, transmit traces, have error spans ? correct way tag span arbitrary "error_" tag ?

the sampling decision taken trace. means when first request comes in , span created have take decision. don't have tags / baggage @ point must not depend on contents of tags take decision. that's wrong approach.

you taking custom approach. if want go way (which not recommended) can create custom implementation of spanreporter - https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/spring-cloud-sleuth-core/src/main/java/org/springframework/cloud/sleuth/spanreporter.java#l30 . spanreporter 1 sending spans zipkin. can create implementation wrap existing spanreporter implementation , delegate execution when values of tags match. perspective doesn't sound right.