Author Archive

Spring microservices monitoring: /metrics endpoint and ELK, Part II: Improvements

 Introduction

 As we saw in the first part of this series, it is essential necessary to monitor the health of our microservices, and  to improve the tool we developed which helps us get metrics and forward them to an Elasticsearch instance. In this chapter we will pursue the question of how can we improve the processing of the result of the metrics endpoint?

Continue Reading

QCON London 2017

I found this experience really nice – to be able to get some insights on how people have solved big issues and how they got pwned.  What is more, it was good to see that I am personally going in the right direction (or at least, let’s say, getting on the trends)

These are technologies and trends that got my attention as the most used:

  •  Technologies/languages/tools:
    • Go
    • ELK
    • Luigi/Airflow – Tools for pipelining jobs
    • Redis
    • Hadoop
    • Kafka
    • Java 9
  • Trends:
    • Data Science
      •  Machine learning for prediction and fraud detection
    • Microservices
      • NodeJS

I chose to follow two main paths, with some extra things from here and there: Machine Learning/Data science, Microservices, and let’s call it “miscellaneous”. Below, an overview of the most interesting things:

Continue Reading

Spring microservices monitoring: /metrics endpoint and ELK

Introduction

On the actual vibe of microservices using, it becomes more and more necessary to monitorize the health of our distributed systems. Here we will expose a few ways to do it, concentrating us in Spring boot applications. Hands on!

Current tools available

On the ELK community already exist two easy solutions to poll for the /metrics endpoint – the Logstash Http_poller input plugin and the community beat springbeat. Both work on a similar way (configure a URL and voilá! data is forwarded to Elasticsearch) but in different contexts, the former is a plugin part of Logstash, while the latter is an independent beat application. And both seem to be nice options, if we want to know the state of microservices with a defined URL or a single instance or if we use a service discovery with a gateway pattern, i.e. zuul+Eureka. And both share the same limitation: There is no way to configure service discovery to poll for all the instances of a service (As of now…). Whether or not this limitation is a problem, depends on what we want and what we have. Do we have a single instance of each microservice? or if we have multiple instances, do we want to know the general aggregated state of them? (I would put an alarm here, because some data is tightly coupled to the instance or it’s JVM and aggregating it might be a problem). An option to solve this limitation is to write a little client, a Spring boot application that will use the service discovery provided by Spring Cloud Eureka to get all the instances, poll them, process data, and forward it to the ELK stack.

Continue Reading