Quantify issues

Recent GitHub Issues Activity

_{Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.}

Rate pull requests

PR#11477 - Readme Update: Added IRIS under UI sectionopen

2_/5

Dhruv Anand (Dhruvanand24)Created: 2025-01-29

The pull request is minor, adding a single line to the README.md file to include a new UI application. While it provides useful information about an Android app related to the project, it is not a significant change. Additionally, the whitespace changes were unnecessary and could introduce merge conflicts, as pointed out by a reviewer. The author acknowledged this and intended to fix it, but the initial submission was flawed. Overall, the PR lacks significance and introduces potential issues, warranting a rating of 2.

[+] Read More

PR#11398 - ci : allow creating artifacts on PRs on demandopen

3_/5

Diego Devesa (slaren)Created: 2025-01-24

The pull request introduces a useful feature by allowing artifact creation on demand through a label, which enhances flexibility in the CI process. However, the changes are relatively minor, involving a simple conditional logic update in the workflow file. While this is a practical improvement, it lacks significant complexity or impact to warrant a higher rating. The PR is straightforward and functional but not particularly innovative or substantial, thus fitting the criteria for an average rating.

[+] Read More

PR#11397 - llama : add option to override model tensor buffersopen

3_/5

Diego Devesa (slaren)Created: 2025-01-24

This pull request introduces a new command line parameter to override model tensor buffers, which can potentially optimize performance by allowing more efficient offloading schemes. However, it is still in the demo phase and requires further testing and feedback from users to validate its utility and effectiveness. The changes are not particularly complex or groundbreaking, but they do offer a useful feature for specific use cases. Overall, it is an average contribution with potential for improvement based on user feedback.

[+] Read More

PR#11410 - Llama-bench: allow benchmarking lora impactopen

3_/5

uvos (IMbackK)Created: 2025-01-25

The pull request introduces a feature to benchmark the impact of lora on models, which is a useful addition for performance analysis. However, it is still in draft form and acknowledges that improvements are needed, as suggested by a reviewer. The changes are significant but not yet complete or exemplary, warranting an average rating.

[+] Read More

PR#11446 - Optimized DeepSeek V2/V3 implementation (MLA)open

3_/5

fairydreamingCreated: 2025-01-27

The pull request offers optimizations for the DeepSeek V2/V3 implementation, which could improve performance by caching latent representations and refining attention mechanisms. However, it remains a draft with several pending tasks, including removing unused tensors and addressing performance regressions. The changes are moderately significant but not yet complete or exemplary, warranting an average rating.

[+] Read More

PR#11484 - server : update auto gen files comments [no ci]open

3_/5

Daniel Bevenius (danbev)Created: 2025-01-29

This pull request makes minor documentation updates by moving comments from server.cpp to README.md and removing outdated references to deps.sh. While it improves clarity by centralizing instructions, the changes are not significant or complex. The update is useful but not critical, aligning with typical maintenance tasks. The PR is well-executed but lacks substantial impact, making it an average contribution.

[+] Read More

PR#11436 - vulkan: Catch pipeline creation failure and print an error messageopen

4_/5

Jeff Bolz (jeffbolznv)Created: 2025-01-26

The pull request addresses a critical issue by catching pipeline creation failures in Vulkan and logging appropriate error messages, enhancing the robustness of the application. Additionally, it fixes warnings related to on-demand compile changes. The code changes are concise and improve error handling without introducing significant complexity. However, the impact is moderate as it primarily focuses on error logging rather than introducing new features or major improvements.

[+] Read More

PR#11453 - ggml : x2 speed for WASM by optimizing SIMDopen

4_/5

Xuan-Son Nguyen (ngxson)Created: 2025-01-27

The pull request significantly improves the performance of the WASM implementation by optimizing SIMD instructions, achieving up to 2.82 times speed increase. The PR is well-documented, includes comprehensive testing, and demonstrates the potential of AI-generated code for low-level optimizations. However, it includes some unrelated style changes and minor formatting issues that could have been separated into a different PR. Overall, it's a substantial contribution with clear performance benefits.

[+] Read More

PR#11452 - llama: Add support for RWKV v7 architectureopen

4_/5

Molly Sophia (MollySophia)Created: 2025-01-27

This pull request introduces significant enhancements by adding support for the RWKV v7 architecture in the llama.cpp project. It includes new operations like L2 normalization and the core RWKV v7 kernel, implemented across multiple backends (CPU, CUDA, SYCL, Vulkan, Metal). The PR is well-documented with references to model benchmarks and provides extensive code changes across various files, indicating a thorough and comprehensive update. However, it lacks the implementation of chunkwise wkv7, which is noted as a TODO. Overall, it's a substantial contribution but not without room for further improvement.

[+] Read More

PR#11489 - server : add /apply-template endpoint for additional use cases of Minja functionalityopen

4_/5

Nigel Bosch (pnb)Created: 2025-01-29

The pull request introduces a new endpoint `/apply-template` to enhance the server's functionality by allowing users to format chat prompts without running inference. This addition is well-documented and includes a corresponding CI test, ensuring reliability and coverage. The changes are significant as they improve usability for users who need to modify prompts before generating responses. However, while the implementation is solid, it is not groundbreaking or exceptionally innovative, which is why it does not merit a perfect score.

[+] Read More

Quantify commits

Quantified Commit Activity Over 14 Days

Developer	Avatar	Branches	PRs	Commits	Files	Changes
Georgi Gerganov		4	14/13/1	43	30	8761
Olivier Chafik		1	2/2/0	3	24	3915
Xuan-Son Nguyen		2	10/8/0	9	18	2308
Johannes Gäßler		1	4/5/0	5	21	2269
Eric Curtin		1	10/9/1	9	7	2044
Jeff Bolz		1	9/11/0	11	13	852
Eve		1	0/1/0	1	6	840
bandoti		1	1/1/0	1	11	517
Nicolò Scipione		1	1/1/0	1	3	240
Akarshan Biswas		1	1/2/0	2	7	231
uvos		1	5/4/0	4	7	207
stduhpf		1	1/1/0	1	3	192
Diego Devesa		3	6/4/0	8	13	186
Daniel Bevenius		1	2/2/0	2	3	167
Radoslav Gerganov		1	2/2/0	2	11	129
tc-mb		1	1/1/0	1	6	115
LostRuins Concedo		2	1/1/0	3	3	103
amd-dwang		1	1/1/0	1	1	94
Junil Kim		1	0/0/0	1	5	92
Haus1		1	1/1/0	1	2	87
fj-y-saito		1	0/1/0	1	1	83
RunningLeon		1	0/1/0	1	1	60
Kyle Bruene		1	1/1/0	1	1	44
jiahao su		1	0/0/0	1	1	34
Nikita Sarychev		1	0/1/0	1	1	22
Nuno		1	3/3/0	3	2	21
Michael Engel		1	1/1/0	1	1	18
Jafar Uruç		1	1/1/0	1	3	16
Frank Mai		1	1/1/0	1	2	14
codezjx		1	1/1/0	1	2	11
lexasub		1	1/1/0	1	2	11
issixx		1	0/0/0	1	1	10
Molly Sophia		1	2/1/0	1	1	10
Ihar Hrachyshka		1	1/1/0	1	1	8
Jiří Podivín		1	1/1/0	1	1	5
peidaqi		1	1/1/0	1	1	4
William Tambellini		1	0/0/0	1	1	4
Bernhard M. Wiedemann		1	1/1/0	1	1	3
Michael Podvitskiy		1	2/1/0	1	1	2
David Renshaw		1	1/1/0	1	1	2
Emreerdog		1	1/1/0	1	1	2
someone13574		1	1/1/0	1	1	2
Christopher Nielsen		1	1/1/0	1	1	1
musoles		1	0/1/0	1	1	1
Nigel Bosch (pnb)		0	1/0/0	0	0	0
Charles Xu (chaxu01)		0	1/0/0	0	0	0
Tei Home (teihome)		0	1/0/0	0	0	0
Steve Grubb (stevegrubb)		0	1/0/1	0	0	0
Herman Semenoff (GermanAizek)		0	3/0/0	0	0	0
Jordan Nanos (JordanNanos)		0	1/0/1	0	0	0
Gabe Goodhart (gabe-l-hart)		0	1/0/1	0	0	0
Dhruv Anand (Dhruvanand24)		0	1/0/0	0	0	0
None (savesanketsw)		0	1/0/0	0	0	0
None (fairydreaming)		0	1/0/0	0	0	0
Rémy Oudompheng (remyoudompheng)		0	1/0/0	0	0	0
Aleksei Nikiforov (AlekseiNikiforovIBM)		0	1/0/0	0	0	0

_{PRs: created by that dev and opened/merged/closed-unmerged during the period}

Quantify risks

Project Risk Ratings

Report On: Fetch issues

Recent Activity Analysis

Overview

The llama.cpp project has seen significant recent activity, with a focus on expanding model support, enhancing performance, and addressing various bugs. The project continues to evolve rapidly, reflecting its active development and community engagement.

Notable Issues and Themes

1. Model Support and Enhancement Requests: A recurring theme is the request for support of new models like Qwen2-VL (#9246), Pixtral by Mistral (#9440), and others. This indicates a strong demand for the project to keep up with emerging models in the AI landscape.

2. Performance Optimization: Several issues highlight performance concerns, such as slow inference times (#11114) and degradation with specific configurations (#10435). These suggest ongoing efforts to optimize the library for different hardware setups.

3. Backend and Compilation Challenges: Issues related to specific backends (e.g., Vulkan, ROCm) and compilation problems on various platforms (e.g., Windows, ARM64) are prevalent. This underscores the complexity of maintaining cross-platform compatibility.

4. Bug Fixes and Anomalies: There are numerous bug reports ranging from segmentation faults (#9949) to unexpected behavior in model outputs (#10681). These highlight the challenges of ensuring stability across diverse use cases.

5. Community Contributions: The project benefits from a vibrant community contributing enhancements, such as new sampling methods (#11057) and chat template support (#11056). This reflects strong user engagement and collaboration.

Issue Details

Most Recently Created Issues

• #11490: Feature Request for Deepseek Janus-Pro-7B & Janus-1.3B support.

  • Priority: Enhancement
  • Status: Open
  • Created: 0 days ago

• #11488: Failed attempt to quantize model under Android termux proot.

  • Priority: Bug
  • Status: Open
  • Created: 0 days ago

Most Recently Updated Issues

• #11469: Misc. bug regarding missing amd64 CPU docker images.

  • Priority: Bug
  • Status: Closed
  • Updated: 1 day ago

• #11447: Compile bug related to CUDA Visual Studio on Windows.

  • Priority: Bug
  • Status: Closed
  • Updated: 1 day ago

Conclusion

The llama.cpp project is actively addressing a wide array of issues, from model support requests to backend-specific bugs. The community's involvement is crucial in driving improvements and ensuring the project's adaptability to new challenges in the AI domain.

Report On: Fetch pull requests

Analysis of Pull Requests for llama.cpp Project

Open Pull Requests

1. #11489: server : add /apply-template endpoint for additional use cases of Minja functionality

   • State: Open
   • Created: 0 days ago
   • Summary: Introduces a new /apply-template endpoint to the server, allowing users to apply chat templates to messages without running inference. This PR also includes a CI test.
   • Notable Aspects: The PR is very recent and seems well-documented with multiple commits enhancing the functionality and documentation. It addresses a specific use case for prompt formatting, which could be beneficial for users needing custom prompt templates.

2. #11484: server : update auto gen files comments [no ci]

   • State: Open
   • Created: 0 days ago
   • Summary: Updates comments related to auto-generated files in server.cpp, removing references to deps.sh which was removed in a previous commit.
   • Notable Aspects: This PR is focused on code maintenance and clarity, ensuring that documentation within the codebase is up-to-date with recent changes.

3. #11477: Readme Update: Added IRIS under UI section

   • State: Open
   • Created: 0 days ago
   • Summary: Adds IRIS, an Android app based on llama.cpp, to the README under the UI section.
   • Notable Aspects: This PR enhances the documentation by highlighting an application of llama.cpp, potentially increasing its visibility and adoption.

4. #11453: ggml : x2 speed for WASM by optimizing SIMD

   • State: Open
   • Created: 2 days ago
   • Summary: Proposes significant performance improvements for WASM by leveraging SIMD instructions.
   • Notable Aspects: This PR highlights the capability of LLMs to optimize low-level code, which is an innovative approach. It has generated considerable discussion among contributors about AI-driven code optimization.

5. #11452: llama: Add support for RWKV v7 architecture

   • State: Open
   • Created: 2 days ago
   • Summary: Adds support for the RWKV v7 architecture, including new operations and backend implementations.
   • Notable Aspects: This PR is significant as it expands the model compatibility of llama.cpp, potentially attracting users interested in RWKV models.

Recently Closed Pull Requests

1. #11480: Parse https://ollama.com/library/ syntax

   • State: Closed
   • Closed By: Eric Curtin (ericcurtin)
   • Summary: Allows llama-run to parse URLs from the Ollama library, improving usability for users searching for models via the web UI.
   • Notable Aspects: This enhancement improves user experience by allowing seamless integration with model repositories.

2. #11475: embedding : enable --no-warmup option

   • State: Closed
   • Closed By: Georgi Gerganov (ggerganov)
   • Summary: Enables a --no-warmup option for embeddings, allowing users to disable warmup runs.
   • Notable Aspects: This feature provides more control over execution parameters, which can be useful for performance tuning.

3. #11473: llamacpp-server: Fixed wrong function name in llamacpp server unit test

   • State: Closed
   • Closed By: Xuan-Son Nguyen (ngxson)
   • Summary: Corrects function names in unit tests for the server component.
   • Notable Aspects: Ensures accuracy and reliability of unit tests, contributing to overall code quality.

4. #11471: Hip: Supress transformation warning in softmax.cu

   • State: Closed
   • Closed By: uvos (IMbackK)
   • Summary: Suppresses transformation warnings in HIP builds related to loop unrolling.
   • Notable Aspects: Addresses build warnings, improving developer experience and reducing noise during compilation.

5. #11466 & #11465 & #11457 & #11449 & #11448 & #11445 & #11441 & #11438 & #11437 & #11434 & #11427 & #11424 & #11423 & #11422 & #11420 & #11419 & #11418 & #11409 & #11407 & #11399 & #11396 & #11392 & #11386 & #11381 & #11380 & #11377 & #11375 & #11373 & #11372 & #11369 & #11368 & #11366 & #11364 & #11362 & #11356 & #11355

Notable Patterns and Issues

• The project is actively maintained with frequent updates and contributions from various developers.
• There is a strong focus on performance optimization, as seen in several PRs targeting speed improvements across different platforms (e.g., WASM, CUDA).
• Documentation updates are regularly made to ensure clarity and accuracy, reflecting changes in functionality or usage.
• Some PRs address specific user needs or enhance usability, such as adding new command-line options or improving integration with external resources.

Overall, the llama.cpp project demonstrates a dynamic development environment with continuous enhancements and active community engagement. The open PRs indicate ongoing efforts to expand model support and optimize performance, while closed PRs reflect successful resolutions of issues and implementation of new features.

Report On: Fetch Files For Assessment

Source Code Assessment

File: examples/run/run.cpp

Structure and Quality Analysis

• Includes and Preprocessor Directives: The file begins with platform-specific includes and conditional compilation directives, ensuring compatibility across different operating systems. This is a good practice for maintaining cross-platform support.

• Namespace and Libraries: The use of standard libraries like <iostream>, <vector>, and <string> is appropriate for handling I/O operations, collections, and string manipulations. The inclusion of third-party libraries like curl and json.hpp indicates the file's functionality related to network operations and JSON parsing.

• Signal Handling: The file includes a signal handler for SIGINT, which is a good practice for gracefully terminating the program on user interruption.

• Utility Functions: Functions like fmt and printe are well-defined for formatted output, enhancing code readability and maintainability.

• Class Design: The Opt class encapsulates command-line argument parsing logic. It uses private member variables to store default values and public methods to initialize parameters. This design promotes encapsulation and separation of concerns.

• Error Handling: Error messages are printed using the printe function, which is consistent throughout the file. However, the use of exit codes could be standardized across different error scenarios for clarity.

• HTTP Client Implementation: The HttpClient class demonstrates a robust implementation for handling HTTP requests using libcurl. It includes methods for setting headers, handling progress updates, and managing file locks, which are crucial for reliable network communication.

• Code Organization: The file is organized into sections with clear responsibilities, such as argument parsing, HTTP client setup, and model initialization. This modular approach aids in understanding and maintaining the code.

• Documentation: Inline comments are sparse but present in critical sections. Additional comments explaining complex logic or assumptions would improve code comprehensibility.

Recommendations

1. Error Handling Consistency: Standardize exit codes across different error scenarios to improve debugging.
2. Documentation: Enhance inline comments to explain complex logic or assumptions.
3. Code Modularity: Consider breaking down large functions into smaller ones to improve readability and maintainability.

File: scripts/sync-ggml.last

Structure and Quality Analysis

• Content: This file contains a single line representing a commit hash.

• Purpose: It likely serves as a marker or reference point for syncing with a specific state of the ggml library.

Recommendations

1. Documentation: Add a comment or README entry explaining the purpose of this file to aid future developers in understanding its role in the project.

File: ggml/src/ggml.c

Structure and Quality Analysis

• Preprocessor Directives: The file includes several preprocessor directives for platform-specific configurations, ensuring compatibility across different environments.

• Functionality: This file appears to handle core functionalities related to memory management, logging, and mathematical operations within the ggml library.

• Error Handling: The use of custom abort functions (ggml_abort) with backtrace capabilities is a robust approach for debugging critical errors.

• Logging System: A structured logging system is implemented, allowing for customizable log levels and user-defined callbacks. This enhances the flexibility of logging outputs.

• Memory Management: Functions like ggml_aligned_malloc demonstrate careful consideration of memory alignment requirements, which is crucial for performance optimization on modern architectures.

Recommendations

1. Code Comments: Increase the use of comments to describe complex algorithms or platform-specific code sections.
2. Modularization: Consider refactoring large functions into smaller units where possible to enhance readability.
3. Testing: Ensure comprehensive testing across different platforms due to extensive use of platform-specific code paths.

File: ggml/src/ggml-cpu/ggml-cpu.c

Structure and Quality Analysis

• Platform-Specific Code: Similar to other files in this project, it includes platform-specific optimizations, particularly for SIMD operations on various architectures (e.g., ARM NEON, AVX).

• SIMD Operations: The file defines macros for SIMD operations that abstract architecture-specific intrinsics, promoting portability while leveraging hardware capabilities.

• Atomic Operations: Implements atomic operations using platform-specific APIs (e.g., Windows Interlocked functions), ensuring thread safety in concurrent environments.

• Threading Support: Includes threading constructs compatible with both Windows and POSIX systems, enhancing cross-platform multithreading support.

Recommendations

1. Inline Documentation: Provide detailed comments on SIMD macro definitions to clarify their purpose and usage.
2. Performance Testing: Conduct performance benchmarks on supported architectures to validate SIMD optimizations.
3. Code Refactoring: Evaluate opportunities for refactoring repetitive patterns in SIMD operation implementations to reduce redundancy.

File: common/arg.cpp

Structure and Quality Analysis

• Command-Line Parsing: Implements comprehensive command-line argument parsing with support for environment variable overrides. This flexibility enhances user experience by allowing configuration through multiple channels.

• Modular Design: Uses classes (common_arg) to encapsulate argument properties and behaviors, promoting reusability and maintainability.

• Error Reporting: Provides detailed error messages during argument parsing failures, aiding users in correcting input errors promptly.

Recommendations

1. Usage Documentation: Ensure that all command-line options are well-documented both in code comments and external documentation.
2. Validation Logic: Consider adding more validation checks for interdependent arguments to prevent invalid configurations.
3. Unit Tests: Implement unit tests covering various argument combinations to ensure robustness against edge cases.

File: src/llama.cpp

Structure and Quality Analysis

• Model Loading Logic: Contains functions related to loading model parameters from files, indicating its role as part of the model initialization process within the llama library.

• Error Handling Strategy: Utilizes try-catch blocks extensively around critical operations like model loading, providing resilience against runtime exceptions.

• Modular Architecture Support: Supports various model architectures through conditional logic, demonstrating flexibility in handling different model types within a single framework.

Recommendations

1. Code Comments: Enhance inline documentation around complex model loading logic to aid future maintenance efforts.
2. Refactoring Opportunities: Identify sections with repeated patterns or lengthy functions that could benefit from refactoring into helper functions.
3. Testing Coverage: Ensure comprehensive test coverage across different model architectures supported by this file to validate compatibility and performance expectations.

Report On: Fetch commits

Development Team and Recent Activity

Team Members and Their Recent Activities

1. Eric Curtin (ericcurtin)

   • Recent activities include parsing URLs for Ollama models, adding GitHub protocol pulling, updating README for llama-run, and enhancing user input handling in llama-run.
   • Collaborated with Xuan-Son Nguyen on some commits.

2. Georgi Gerganov (ggerganov)

   • Engaged in multiple activities such as syncing ggml, fixing line breaks on Windows builds, updating the README, and refactoring context-related code.
   • Worked on various branches including gg/llama-kv-cache and gg/build-pack-lib-include.

3. William Tambellini (WilliamTambellini)

   • Added an option to not print the stack on abort in ggml.

4. Issixx (issixx)

   • Fixed a termination issue in ggml-cpu threads during abort.

5. Daniel Bevenius (danbev)

   • Enabled --no-warmup option for llama-embeddings and made minor fixes to improve model load speed.

6. Molly Sophia (MollySophia)

   • Fixed missing k_cache store for rwkv6qwen2 and added support for QRWKV6 model architecture.

7. Emreerdog

   • Added hints for locating ggml on Windows using CMake.

8. Peidaqi

   • Fixed a wrong function name in llamacpp server unit test.

9. Xuan-Son Nguyen (ngxson)

   • Fixed build issues for CPU arm64, improved speculative decoding with context shift, and added support for Deepseek-R1-Qwen distill model.
   • Collaborated with Eric Curtin and Georgi Gerganov on several commits.

10. Uvos (IMbackK)

    • Suppressed transformation warning in HIP softmax.cu and added hipGraph support to ROCM.

11. Nuno (rare-magma)

    • Allowed installing pip packages system-wide in Docker and added perplexity and bench commands to the full image.

12. Akarshan Biswas (qnixsynapse)

    • Implemented SOFTMAX F16 mask support in SYCL and refactored SYCL compute forward functions.

13. Michael Engel (engelmi)

    • Handled missing model in CLI parameters for llama-run to prevent application crashes.

14. Johannes Gäßler (JohannesGaessler)

    • Refactored llama_decode_impl and fixed FP16 cuBLAS GEMM in CUDA.

15. Diego Devesa (slaren)

    • Added guide tokens support to TTS and fixed ARM build issues in Docker.

Patterns, Themes, and Conclusions

• The team is actively engaged in enhancing the functionality of the llama.cpp project across various aspects such as performance optimization, bug fixes, feature additions, and documentation updates.
• There is significant collaboration among team members, particularly between Georgi Gerganov, Xuan-Son Nguyen, and Eric Curtin.
• The project is under continuous development with frequent commits addressing both minor tweaks and major feature implementations.
• The focus is on improving compatibility across different platforms (e.g., Windows, Linux), enhancing user experience through better input handling and documentation updates, and expanding model support.
• The team is responsive to community feedback as seen by the rapid addressing of issues like build failures or feature requests related to specific models or platforms.