Introduction
Observability and monitoring rely on three fundamental pillars: logging, metrics, and traces. To implement these effectively in a microservices architecture, we must generate and manage relevant data, enabling us to understand the internal state of our applications and monitor them efficiently.
Understanding Logs
Logs are records of events occurring within a software application over time. They typically include:
- Timestamp: Indicates when the event happened.
- Event Information: Describes what occurred.
- Contextual Data: Provides additional details such as the thread processing the event or the user/tenant involved.
Logs serve as crucial tools for debugging and troubleshooting, helping reconstruct scenarios that occurred at specific times in an application instance.
Log Severity Levels
Logs are categorized by severity to control verbosity and ensure efficient debugging:
- TRACE: Most detailed level, often used for troubleshooting.
- DEBUG: Provides detailed information for development and debugging.
- INFO: General information about the application’s operations.
- WARN: Indicates potential issues that may require attention.
- ERROR: Highlights serious problems that require immediate action.
In production environments, excessive logging can degrade performance. Only severe events such as exceptions and critical errors should be logged. In development and testing environments, more detailed logs (e.g., DEBUG and INFO) can be enabled to facilitate troubleshooting.
If you are unfamiliar with implementing logging in Spring Boot, refer to my dedicated course on Spring Boot logging best practices, where I discuss how to activate specific log levels based on the environment.
Challenges of Logging in Microservices
In monolithic applications, logging is straightforward as all logs reside in a single location. Developers can easily search logs and troubleshoot issues.
However, in microservices architectures, each service generates its own logs, often stored across multiple servers or containers. This makes debugging more complex as developers must check multiple locations to track down an issue.
Centralized Logging in Microservices
Centralized logging addresses the challenge of scattered logs by collecting logs from all microservices and storing them in a single location. This simplifies troubleshooting, as developers can access all logs from one place instead of searching through numerous microservices.
Without centralized logging, developers would need to manually inspect logs across hundreds of microservices, which is highly inefficient and impractical.
Implementing Log Aggregation
A common question arises: Who is responsible for handling log aggregation?
- One basic approach is for developers to write custom logic inside microservices to save or stream logs to a centralized system.
- However, this approach has several disadvantages:
- Log aggregation is not related to business logic.
- It distracts developers from focusing on core business requirements.
- It introduces additional complexity and maintenance overhead.
A Better Approach: External Log Aggregation Tools
Instead of writing custom log aggregation logic, we can leverage dedicated log aggregation tools that seamlessly collect, store, and manage logs without modifying microservices. In the next section, we will explore such tools and how they can help streamline observability and monitoring in microservices.