Quantify Issues

Recent GitHub Issues Activity

_{Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.}

Quantify commits

Quantified Commit Activity Over 30 Days

Developer	Avatar	Branches	PRs	Commits	Files	Changes
Ruihang Lai		1	23/23/1	23	84	5605
Shushi Hong		1	4/4/0	4	6	897
Gunjan Dhanuka		1	0/1/0	1	14	748
Mengshiun Yu		1	5/5/0	5	10	541
Charlie Ruan		1	5/5/0	5	13	268
Yaxing Cai		1	6/6/0	6	16	188
lizhuo		1	2/2/0	2	6	167
Wuwei Lin		1	3/3/0	3	9	156
mlc-gh-actions-bot		1	0/0/0	40	12	146
krishnaraj36		1	2/2/0	2	3	38
Molly Sophia		1	2/2/0	2	4	30
Yiyan Zhai		1	2/2/0	2	3	25
Git bot		1	0/0/0	3	1	6
sunzj		1	2/2/0	2	2	3
Ikko Eltociear Ashimine		1	1/1/0	1	1	2
BlindDeveloper		1	2/1/1	1	1	2
Chanhee Lee (chanijjani)		0	1/0/1	0	0	0

_{PRs: created by that dev and opened/merged/closed-unmerged during the period}

Report On: Fetch issues

Recent Activity Analysis

The MLC LLM project has seen significant recent activity, with 172 open issues on GitHub, indicating ongoing user engagement and potential areas for improvement. Notable themes include a variety of bug reports related to model compatibility and performance issues across different platforms, particularly concerning Android and iOS devices. There are also several feature requests aimed at expanding model support and enhancing functionality, such as speculative decoding and multi-GPU utilization.

Several critical bugs have been reported, including crashes when initializing models like Gemma and issues with speculative decoding that could hinder user experience. The presence of multiple issues related to specific models (e.g., Phi-3 mini and Qwen2) suggests that certain models may require additional attention for stability and performance optimization.

Issue Details

Most Recently Created Issues

1. Issue #2876: [Bug] Qwen2-1.5B Q4F16_0 - libc++abi: terminating due to uncaught exception of type std::length_error: vector

   • Priority: High
   • Status: Open
   • Created: 0 days ago
   • Updated: N/A

2. Issue #2875: [Bug] Android package Error: subprocess.CalledProcessError: Command returned non-zero exit status 2

   • Priority: High
   • Status: Open
   • Created: 0 days ago
   • Updated: N/A

3. Issue #2873: [Bug] RWKV v6 models fail to compile with latest mlc_llm

   • Priority: High
   • Status: Open
   • Created: 1 day ago
   • Updated: N/A

4. Issue #2871: [Question] How can I use the parameter logits_processors to modify the current logit?

   • Priority: Medium
   • Status: Open
   • Created: 1 day ago
   • Updated: N/A

5. Issue #2870: [Bug] TVM installation fails on Windows machines

   • Priority: High
   • Status: Open
   • Created: 1 day ago
   • Updated: N/A

6. Issue #2869: [Bug] Phi 3.5 mini crashes mobile app

   • Priority: High
   • Status: Open
   • Created: 2 days ago
   • Updated: N/A

7. Issue #2868: [Bug] When I enable "

Report On: Fetch pull requests

Overview

The analysis of the pull requests (PRs) for the MLC LLM repository reveals a total of five open PRs and a significant number of closed PRs, indicating ongoing development and maintenance efforts. The open PRs focus on enhancing model performance, implementing new features, and addressing issues related to memory management and benchmarking.

Summary of Pull Requests

Open Pull Requests

• PR #2663: [Serving] PagedKVCache Quantization
  Created 49 days ago. This PR introduces quantization schemes for the KV cache, significantly reducing memory requirements. It aims to optimize memory usage for models like Llama-3, which is crucial for deployment in resource-constrained environments.

• PR #868: Implement Whisper in new concise nn.Module API
  Created 363 days ago. This PR implements the Whisper model using a new API but has faced issues during testing, as indicated by user comments about errors encountered. It highlights the challenges in integrating new models into existing frameworks.

• PR #2585: [Bench] Add bench for GSM8K eval
  Created 78 days ago. This PR adds benchmarking capabilities for evaluating the GSM8K dataset, which is essential for assessing model performance on specific tasks.

• PR #2584: [Bench] Add bench for MMLU eval
  Created 78 days ago. Similar to PR #2585, this PR focuses on benchmarking for the MMLU dataset but notes issues with chat mode that need resolution.

• PR #1271: Add docker container support
  Created 292 days ago. This PR addresses community requests for Docker support, enabling easier deployment of models in various environments. It has seen active discussions regarding performance implications.

Closed Pull Requests

• PR #2874: [Fix] Fix RWKV v6 weights loading for 7B/14B models
  Recently merged. This PR resolves tensor dimension issues affecting model loading, showcasing ongoing efforts to ensure compatibility with various model sizes.

• PR #2872: [Conv] Fix Qwen2 conv template
  Recently merged. This minor fix improves the conversation template for Qwen2, reflecting attention to detail in user-facing features.

• PR #2867: [Engine] Preparation for switching between spec-decode mode and normal mode
  Recently merged. This PR enhances functionality by allowing more flexible decoding modes, which is critical for improving user experience during inference.

• PR #2860: [Fix] Update seq len info after prefix cache operation
  Recently merged. This fix ensures that sequence length information is accurately updated during operations involving prefix caching, which is vital for maintaining model performance.

Analysis of Pull Requests

The analysis of the pull requests reveals several key themes and areas of focus within the MLC LLM project:

1. Performance Optimization: A significant number of open and closed PRs are dedicated to optimizing memory usage and computational efficiency. For instance, PR #2663 introduces KV Cache quantization, which can drastically reduce memory consumption—an essential feature as models grow larger and more complex. The emphasis on quantization techniques indicates a proactive approach to resource management, particularly important in production environments where hardware limitations are common.

2. Benchmarking Enhancements: The introduction of benchmarking tools through PRs like #2585 and #2584 shows a commitment to ensuring that models are not only functional but also performant across various datasets. Benchmarking is crucial for validating improvements and guiding future development efforts.

3. Community Engagement and Responsiveness: The discussions around PR #868 highlight the challenges faced when integrating new features into existing frameworks. User feedback is actively considered, demonstrating a responsive development culture that values community input. Additionally, the Docker support introduced in PR #1271 reflects an understanding of user needs for easier deployment options.

4. Bug Fixes and Maintenance: Many recent PRs focus on fixing bugs or improving existing functionalities (e.g., PRs #2874, #2872). This indicates a healthy maintenance cycle where developers are attentive to both new feature implementation and existing codebase stability.

5. Diversity in Contributions: The variety of contributors involved in different aspects of the project—from model implementation to documentation updates—suggests a collaborative environment that encourages contributions from various stakeholders within the community.

6. Long-Term Vision: The repository's activity level (with over 1,500 commits) and its substantial star count indicate strong community interest and potential longevity in development efforts. The focus on cross-platform support further positions MLC LLM as a versatile tool suitable for diverse applications across different hardware architectures.

In conclusion, the pull request activity within the MLC LLM repository reflects a dynamic project landscape characterized by continuous improvement efforts, community engagement, and a clear focus on performance optimization and usability enhancements. These factors contribute to its potential success as a leading framework in the deployment of large language models.

Report On: Fetch commits

Repo Commits Analysis

Development Team and Recent Activity

Team Members and Recent Contributions

1. Ruihang Lai (MasterJH5574)

   • Recent Activity:
   • Contributed to multiple PRs focusing on engine actions, model support, and performance improvements.
   • Key contributions include preparation for switching decoding modes (#2867), fixes for various model loading issues (#2874, #2872), and enhancements to KV cache structuring (#2857).
   • Collaborated with Mengshiun Yu on model support (#2827) and co-authored several other PRs.
   • Ongoing work includes improving prefill policies and handling of batch actions.

2. Molly Sophia (MollySophia)

   • Recent Activity:
   • Fixed tensor dimension definitions for RWKV v6 models (#2874) and addressed issues in config parsing for RNN models (#2750).
   • No ongoing work reported beyond recent commits.

3. Mengshiun Yu (mengshyu)

   • Recent Activity:
   • Worked on fixes related to model attributes and updates for Android APKs (#2842, #2839).
   • Contributed to the integration of new models like Phi-3 vision (#2658).
   • No ongoing work reported beyond recent commits.

4. Yaxing Cai (cyx-6)

   • Recent Activity:
   • Focused on fixing prefix cache issues (#2798) and improving policies for prefix cache reuse (#2859).
   • Active in multiple bug fixes related to sequence length updates and handling of cache operations.
   • Ongoing work includes enhancements to batch prefill policies.

5. Charlie Ruan (CharlieFRuan)

   • Recent Activity:
   • Contributed to adding new presets for models, including Phi-3.5-mini (#2845), and fixed conversation templates (#2872).
   • Engaged in documentation updates regarding multi-GPU support.
   • No ongoing work reported beyond recent commits.

6. Yiyan Zhai (YiyanZhai)

   • Recent Activity:
   • Made minor contributions related to model adjustments and documentation updates.
   • No ongoing work reported beyond recent commits.

7. Shushi Hong (tlopex)

   • Recent Activity:
   • Focused on multi-GPU support for various models, including MiniCPM and InternLM2 (#2815, #2630).
   • Active in fixing bugs related to model compatibility.
   • No ongoing work reported beyond recent commits.

8. Sunzj

   • Recent Activity:
   • Minor contributions focused on fixing specific bugs related to weight storage types.
   • No ongoing work reported beyond recent commits.

9. Huanglizhuo (huanglizhuo)

   • Recent Activity:
   • Worked on Android-related updates, including markdown rendering features.
   • No ongoing work reported beyond recent commits.

10. Wuwei Lin (vinx13)

    • Recent Activity:
    • Involved in enhancing serving capabilities and fixing bugs in sampling processes.
    • No ongoing work reported beyond recent commits.

11. BlindDeveloper

    • Recent Activity:
    • Made minor changes related to Android SDK updates.
    • No ongoing work reported beyond recent commits.

12. Krishnaraj36

    • Recent Activity:
    • Contributed fixes related to compilation issues and enhancements for Windows CI.
    • No ongoing work reported beyond recent commits.

13. Gunjan Dhanuka (GunjanDhanuka)

    • Recent Activity:
    • Added support for the Aya-23 8B model with significant changes across multiple files.
    • No ongoing work reported beyond recent commits.

14. Andrey Malyshev (elvin-n)

    • Recent Activity:
    • Minor contributions focused on fixing compilation issues.
    • No ongoing work reported beyond recent commits.

Patterns, Themes, and Conclusions

• The development team is actively engaged in enhancing the functionality of the MLC LLM project with a focus on model support, performance optimization, and bug fixes.
• Collaboration among team members is evident, especially in complex PRs that involve multiple contributors working together on shared features or fixes.
• Recent activities indicate a strong emphasis on improving the handling of different model architectures, particularly with respect to multi-GPU support and efficient caching mechanisms.
• There is a consistent effort towards maintaining documentation alongside code changes, which is crucial for community engagement and usability.
• Overall, the team demonstrates a proactive approach to addressing both feature requests and technical debt through regular updates and fixes across various components of the project.