transentis consulting

Implementation of a Large Scale Monitoring System

Imagine you are setting up a backend system that is providing services to over one million autonomous devices that are in constant, real-time communication with the backend – Internet of Things at scale.
Data from the devices is passed through a chain of systems and even routed to 3rd party systems for further computation. Results are then passed back to the devices in near real time so that they can make autonomous decisions.

The Challenge

The challenge is to keep an overview of what is going in in this huge, distributed system. There is no lack of data about the backend system and the devices connected to it: on the contrary, you need to analyse huge amounts of it, presented aggregated facts to decision makers and even make autonomous control decisions, all in near real time.

The Solution: Elasticsearch To The Rescue

We designed and implemented a monitoring system based on the Elastic stack: using Logstash to filter and extract the relevant data, Elasticsearch to store it and provide fast search capabilities and Kibana to display it.
The system is running world wide and consists of dozens of Elasticsearch clusters, the largest of which has over 100 nodes. Over 8 terrabytes of data are ingested daily.

Our Expertise

  • Data strategies and architectures

Target Group

Recent Blogposts

Meetup: AI for Enterprise Decision Making

Oliver Grasl - 2021-02-10

Ten Key Principles of Digital Transformation

Max Erdrich - 2021-01-24

Agile Transformation Processes

Ognjen Manojlovic - 2021-01-19

How To Deal With Requirements in Flawed Projects

Igor Tunjic - 2020-08-02

Training Artificial Intelligence to Play the Beer Game

Oliver Grasl - 2020-05-31

Email newsletter

Subscribe to our newsletter and stay up to date about the latest trends and news.

Name
Email
Subscribe to our newsletter and stay up to date about the latest trends and news.
  • transentis
Contact us

transentis consulting

Geisbergstraße 9

10777 Berlin


+49 30 9203833320

info@transentis.com