THE OPEN SOURCE INTEGRATOR FOR ENTERPRISE

Spring microservices monitoring: /metrics endpoint and ELK

Introduction

On the actual vibe of microservices using, it becomes more and more necessary to monitorize the health of our distributed systems. Here we will expose a few ways to do it, concentrating us in Spring boot applications. Hands on!

Current tools available

On the ELK community already exist two easy solutions to poll for the /metrics endpoint – the Logstash Http_poller input plugin and the community beat springbeat. Both work on a similar way (configure a URL and voilá! data is forwarded to Elasticsearch) but in different contexts, the former is a plugin part of Logstash, while the latter is an independent beat application. And both seem to be nice options, if we want to know the state of microservices with a defined URL or a single instance or if we use a service discovery with a gateway pattern, i.e. zuul+Eureka. And both share the same limitation: There is no way to configure service discovery to poll for all the instances of a service (As of now…). Whether or not this limitation is a problem, depends on what we want and what we have. Do we have a single instance of each microservice? or if we have multiple instances, do we want to know the general aggregated state of them? (I would put an alarm here, because some data is tightly coupled to the instance or it’s JVM and aggregating it might be a problem). An option to solve this limitation is to write a little client, a Spring boot application that will use the service discovery provided by Spring Cloud Eureka to get all the instances, poll them, process data, and forward it to the ELK stack.

Continue Reading

Performance considerations when using Apache Kafka with SSL/TLS

Running a cluster of a distributed software such as Apache Kafka in a production environment will lead to operational concerns. One of these concerns is certainly security. Companies have varying guidelines and policies when it comes to security. This depends on various factors for example the type of application data that the system is processing or in which network infrastructure the system is running and what security concerns this zone implies on the servers that are running within.

A probable scenario is that such a cluster is run in an environment which dictates restrictiveness in regards to access control and protocol security. Access control means that identified clients communicate with the cluster only. When looking at protocol security the company or project might have a guideline that clients must use a secure protocol like SSL/TLS. Furthermore, communication among the cluster servers itself might also have to be secured.

Continue Reading

Import Relational Database Data to Elasticsearch with Spring Batch (Part 1)

Recently I had a customer project that required transferring large amounts of data from a relational database to the NoSQL database that is Elasticsearch in order to take advantage of its famous fast searching capabilities. Elasticsearch is part of the ELK stack that is released and maintained by Elastic.co. The abbreviation ELK stand for Elasticsearch, Logstash, and Kibana. The easiest way to transfer data from a traditional relational database into Elasticsearch is by using the “L” in the “ELK” stack: Logstash. Unfortunately, Logstash has some limitations, and one of those limitations is directly related to reading records from a relational database because, although database entry and Elasticsearch entry may seem very similar, it’s not possible to match single database entry to a single Elasticsearch document. This difference originates from the fact that Elasticsearch doesn’t use the notion of “relations” between its “records”, but instead it uses flat documents structure to store its data and flat documents have no relations between each other. In addition, the customer database have several millions entries which made the situation even more complicated. So, instead of using Logstash, a decision was made to write our own importer that was going to use batch processing and bulk writing into Elasticsearch.

Continue Reading

Testing an Angular 2 Service with a Mocked HTTP Interface

Angular 2 is the successor of the famous and broadly used AngularJS Javascript Framework, for building dynamic web applications. Due to the huge success and popularity of its predecessor, some client projects started using Angular 2 before its official release.This resulted in Angular 2 becoming one of the most famous platforms for web and hybrid application development. Although this framework is relatively new there are already a lot of tutorials and examples explaining how to develop an Angular 2 application. However, there is one aspect of Angular 2 that is not widely covered, but is still very important for every project: Writing tests. The command used to generate the default test cases is “ng generate” and it’s contained in the Angular 2 CLI. However, it only generates the boilerplate code that is needed for each test case. The actual tests have to be created separately and the API is still experimental, so examples are often outdated. In example, most of the available tutorials use the “beforeEachProviders” method, which was removed in an earlier Angular 2 pre-release and was replaced by “TestBed.configureTestingModule({providers:…})” with a completely different signature. In the following example we will test a simple “ItemService” which executes one HTTP GET request against a REST endpoint.

Continue Reading

Developing plugins for Liferay DXP with Maven

A new major version of Liferay has been release past May with a lot of new features. One of the most important is related to plug-ins architecture. Basically Liferay is a web application running in a Java EE application container where plug-ins were built as separate web applications. But DXP introduces a module framework based on the OSGi architecture.

But… What it is OSGi? As defined by OSGi Alliance (the organization in charge of this specification): The OSGi technology is a set of specifications that define a dynamic component system for Java. These specifications enable a development model where applications are (dynamically) composed of many different (reusable) components. As the aim of this article is not the OSGi architecture but how it is implemented within Liferay, we are not going to deepen in OSGi specification.  In fact, you do not need to be very familiar with all the OSGI architecture for develop plug-ins for Liferay DXP. You only need to understand this 2 concepts: modules and components.

Continue Reading