Grafana Mimir, developed by Grafana Labs, is an open-source project designed to provide scalable, highly available, and multi-tenant long-term storage for Prometheus metrics. It is engineered to efficiently manage extensive time series data, enhancing monitoring capabilities across multiple instances.
The last 30 days have seen a concerted effort towards performance enhancements and stability improvements within the Grafana Mimir project. Key activities include significant updates to the query engine, dependency management, and bug fixes. The development team has been actively engaged in both feature development and maintenance tasks, with a strong emphasis on improving system performance and scalability.
Recent issues and pull requests (PRs) reflect a focus on performance optimization, particularly around Kafka integration and query processing efficiency. For instance, PR #9204 introduces Kafka configuration options, while PR #9211 addresses goroutine leaks in Alertmanager. These activities indicate a strategic shift towards optimizing resource utilization and reducing latency in data handling processes.
Summary
Quantified Reports
Detailed Reportsmimir-prometheus module.| Timespan | Opened | Closed | Comments | Labeled | Milestones |
|---|---|---|---|---|---|
| 7 Days | 6 | 1 | 6 | 2 | 1 |
| 30 Days | 38 | 18 | 46 | 19 | 1 |
| 90 Days | 108 | 64 | 130 | 46 | 1 |
| 1 Year | 374 | 199 | 750 | 147 | 1 |
| All Time | 1845 | 1300 | - | - | - |
Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.
| Developer | Avatar | Branches | PRs | Commits | Files | Changes |
|---|---|---|---|---|---|---|
| George Krajcsovits |  |
1 | 9/9/0 | 7 | 261 | 31325 |
| renovate[bot] |  |
4 | 23/16/4 | 29 | 437 | 13323 |
| Charles Korn |  |
4 | 24/24/0 | 39 | 131 | 10935 |
| Franco Posa |  |
4 | 5/2/1 | 23 | 35 | 7082 |
| Marco Pracucci |  |
2 | 14/13/1 | 15 | 85 | 4403 |
| Vladimir Varankin |  |
4 | 13/11/0 | 20 | 118 | 4296 |
| Dimitar Dimitrov |  |
7 | 18/12/3 | 32 | 46 | 3483 |
| chencs |  |
2 | 9/8/0 | 15 | 30 | 2883 |
| Joshua Hesketh |  |
2 | 11/9/0 | 14 | 100 | 2292 |
| Mauro Stettler |  |
3 | 1/0/0 | 10 | 22 | 1632 |
| Peter Štibraný |  |
3 | 6/4/1 | 14 | 53 | 1279 |
| Jonathan Halterman |  |
1 | 3/1/1 | 2 | 33 | 1125 |
| Julien Duchesne (julienduchesne) | 2 | 2/0/0 | 4 | 9 | 1089 | |
| Vernon Miller |  |
1 | 1/1/0 | 3 | 15 | 966 |
| Arve Knudsen |  |
2 | 7/6/0 | 7 | 36 | 936 |
| gotjosh (gotjosh) | 2 | 3/1/0 | 6 | 5 | 922 | |
| Ganesh Vernekar |  |
3 | 3/2/1 | 4 | 9 | 891 |
| zenador |  |
2 | 1/0/0 | 9 | 16 | 806 |
| Đurica Yuri Nikolić |  |
1 | 3/3/0 | 4 | 14 | 533 |
| Nick Pillitteri |  |
2 | 2/1/0 | 3 | 14 | 433 |
| Oleg Zaytsev |  |
1 | 1/1/0 | 2 | 13 | 408 |
| Santiago (santihernandezc) | 2 | 2/0/0 | 2 | 9 | 311 | |
| Jesus Vazquez (jesusvazquez) | 1 | 2/1/0 | 3 | 12 | 308 | |
| Jack Baldry (jdbaldry) | 1 | 1/0/0 | 2 | 3 | 240 | |
| Yuri Tseretyan |  |
1 | 0/0/0 | 1 | 11 | 210 |
| Nicholas Kress |  |
1 | 1/1/0 | 1 | 5 | 163 |
| George Robinson |  |
4 | 6/3/0 | 8 | 10 | 163 |
| Alexander Weaver |  |
2 | 3/2/0 | 7 | 21 | 159 |
| AvivGuiser |  |
1 | 2/1/1 | 2 | 37 | 139 |
| Sephia Laureencia |  |
1 | 0/0/0 | 1 | 9 | 97 |
| Fayzal Ghantiwala |  |
1 | 1/1/0 | 1 | 8 | 96 |
| Miguel Ángel Ortuño |  |
1 | 0/0/0 | 1 | 3 | 91 |
| Grot (@grafanabot) |  |
2 | 13/13/0 | 5 | 11 | 61 |
| David Grant |  |
1 | 0/0/0 | 1 | 4 | 19 |
| Taylor C |  |
1 | 2/2/0 | 1 | 1 | 14 |
| Tito Lins |  |
1 | 1/1/0 | 1 | 3 | 8 |
| Elias Abacioglu (Raboo) | 0 | 1/0/0 | 0 | 0 | 0 | |
| William Travis Holton (heytrav) | 0 | 1/0/0 | 0 | 0 | 0 | |
| None (DeadNews) | 0 | 1/0/1 | 0 | 0 | 0 | |
| Willem Gillis (Polygens) | 0 | 1/0/0 | 0 | 0 | 0 | |
| Bruno FERNANDO (rasta-rocket) | 0 | 1/0/0 | 0 | 0 | 0 | |
| Wilfried ROSET (wilfriedroset) | 0 | 1/0/0 | 0 | 0 | 0 | |
| None (github-actions[bot]) | 0 | 1/0/0 | 0 | 0 | 0 |
PRs: created by that dev and opened/merged/closed-unmerged during the period
The Grafana Mimir project currently has 545 open issues, indicating active development and user engagement. Recent activity shows a mix of enhancements, bugs, and feature requests, with several issues being created or updated in the past few days. Notably, there are discussions around performance improvements, such as optimizing the ingestion process and addressing memory usage concerns.
Several issues highlight recurring themes, including the need for better handling of out-of-order writes, improving error messaging for users, and enhancing the overall stability of components like the ingester and compactor. The presence of multiple issues related to configuration options suggests that users are seeking more flexibility and control over their deployments.
Issue #9206: BlockBuilder: Consume at startup only when lagging
Issue #9200: Mimir images built with profile-guided optimization
Issue #9188: Support Openstack application credentials in helm chart for authenticating to use Swift storage backend
Issue #9177: "The endpoint must not prefixed with the bucket name" error when bucket name matches beginning of S3 endpoint
Issue #9141: Include position information in annotations returned from sharded queries
Issue #9177: "The endpoint must not prefixed with the bucket name" error when bucket name matches beginning of S3 endpoint
Issue #9119: Make mimirtool not use color when pointed to non-tty output
Issue #9085: Config is not imported Prometheus/Alertmanager
Issue #9067: Add possibility to add common labels in helm chart
Issue #8990: Bug: downstream replied with Unprocessable Entity
Several issues stand out due to their implications for the project's functionality:
The enhancement requests (#9200 and #9188) indicate a focus on performance optimization and flexibility in deployment configurations, which could significantly impact user experience and system efficiency.
The bug reported in issue #9177 highlights a critical configuration issue that could lead to significant user frustration if not addressed promptly. This issue's resolution may require careful documentation updates to prevent future occurrences.
The ongoing discussions about improving error messages (#5898) suggest that user experience is a priority for the development team, aiming to reduce confusion during operational failures.
Overall, the recent activity reflects a proactive approach to addressing both user-reported issues and internal optimizations, which is essential for maintaining a robust observability tool like Grafana Mimir.
The analysis of the pull requests (PRs) for the Grafana Mimir project reveals a total of 115 open PRs, with a significant focus on improving functionality, fixing bugs, and enhancing performance. The recent activity indicates a robust development cycle, with various contributors addressing both immediate issues and long-term enhancements.
PR #9217: Cherry pick created timestamp fix - Introduces updates from related PRs to fix timestamp issues in the weekly-r304 branch.
PR #9215: Release 5.3.1 - A Helm release PR that requires another PR (#9216) to be merged first.
PR #9211: Fix goroutine leak in Alertmanager - Addresses a leak issue when applying bad configurations in multi-tenant setups.
PR #9210: Remove legacy TreeQueue implementation - Cleans up old code by removing the TreeQueue and related flags.
PR #9209: Enable querier worker queue priority - Draft PR focusing on enabling priority for query scheduling.
PR #9208: Update ingester downscaling docs - Documentation update to reflect new procedures for downscaling ingesters.
PR #9204: Introduce Kafka flavours - Adds a configuration flag to handle differences in Kafka implementations.
PR #9203: Use fork of franz-go - Switches to a forked version of the Franz Go client for better compatibility with Mimir's requirements.
PR #9199: Blockbuilder consume and build blocks - Continues work on integrating Kafka with TSDB components.
PR #9185: Validate label value - Adds UTF-8 validation for label values in the Distributor component.
PR #9181: Add application credential arguments for OpenStack Swift - Enhances Helm chart to support OpenStack authentication.
PR #9179: Allow ignoring out-of-order exemplars - Introduces a feature flag to manage out-of-order data handling.
PR #9178: Convert start timestamps to Prometheus timestamps - Adjusts how timestamps are handled in OTLP requests.
PR #9177: Update dependencies and fix bugs related to Prometheus integration.
PR #9176: Update prometheus no client golang at grafana/mimir-prometheus - Updates dependencies related to Prometheus client interactions.
PR #9175: Improve logging during migrations for UTF-8 support.
PR #9174: Add feature flag for logging during UTF-8 migration processes.
PR #9173: Improve error handling in Alertmanager configurations.
PR #9172: Backport changes from main branch to r303 regarding consistency checks in queries.
PR #9171: Release Helm chart 5.5.0-weekly for Mimir distributed deployment.
PR #9169: Improve ingestion storage queries in mixin dashboards.
PR #9168 & PR #9167: Minor improvements and documentation updates related to UTF-8 handling and other features.
Older PRs (up to PR #7000) show ongoing efforts to enhance performance, add features, and improve documentation across various components of the Mimir project, including storage management, query handling, and user experience improvements.
The pull requests indicate several key themes and patterns within the ongoing development of Grafana Mimir:
The project is experiencing active development with a consistent influx of new features, bug fixes, and performance improvements being proposed by various contributors. This is evident from the large number of open PRs (115), many of which are focused on critical areas such as memory management, query optimization, and integration with external systems like Kafka and OpenTelemetry.
Several recent PRs (e.g., PRs #9204, #9203, and others) emphasize performance enhancements, particularly around Kafka integration and query processing efficiency. The introduction of concurrent processing capabilities (as seen in PRs like #8711) suggests a strategic shift towards optimizing resource utilization and reducing latency in data handling processes.
Many PRs also focus on enhancing documentation (e.g., PRs #9208, #9181). This reflects an understanding that clear documentation is crucial for user adoption and effective usage of new features, especially as the project evolves rapidly with new functionalities being added regularly.
A significant number of PRs address bugs or stability issues (e.g., PRs like #9211, which fixes goroutine leaks). This highlights a commitment to maintaining software quality alongside feature development, ensuring that users can rely on stable releases even as new features are introduced.
The variety of contributors involved in these pull requests indicates a healthy community engagement around the project. Discussions around proposed changes often include feedback from multiple team members, suggesting an inclusive approach to development that values diverse input and collaboration.
Some older PRs remain open or have been marked as drafts for extended periods (e.g., PRs like #8887), indicating potential challenges in finalizing certain features or addressing complex issues that require more extensive testing or validation before they can be merged into the main codebase.
While there is substantial activity in terms of new pull requests being created, some older pull requests have not seen recent merge activity or resolution (e.g., several dated back over 60 days). This could indicate resource constraints or prioritization challenges within the team as they navigate numerous ongoing initiatives simultaneously.
Overall, the analysis reveals that Grafana Mimir is undergoing significant enhancements driven by community contributions focused on performance optimization, stability improvements, and comprehensive documentation efforts. However, attention should be given to managing older pull requests effectively to ensure timely resolutions and maintain momentum within the project's development cycle.
renovate[bot]
github.com/shirou/gopsutil/v4 and github.com/thanos-io/objstore.titolins (Tito Lins)
krajorama (György Krajcsovits)
mimir-prometheus module, including performance enhancements and bug fixes.jhesketh (Joshua Hesketh)
charleskorn (Charles Korn)
chencs (Casie Chen)
alexweav (Alexander Weaver)
grobinson-grafana (George Robinson)
KyriosGN0 (Aviv Guiser)
narqo (Vladimir Varankin)
dimitarvdimitrov (Dimitar Dimitrov)
seizethedave (David Grant)
francoposa (Franco Posa)
pracucci (Marco Pracucci)
Additional contributors include:
pstibrany, aldernero, colega, 56quarters, kressnick25, jhalterman, tacole02, codesome, fayzal-g, yuri-tceretian, replay, zenador, etc., each contributing with varying degrees of commits and changes primarily focused on bug fixes, enhancements, and documentation improvements.The recent activities indicate a highly collaborative environment with a clear focus on enhancing Grafana Mimir's capabilities while maintaining robust performance standards. The team's diverse contributions reflect a commitment to quality and continuous improvement within the project.