Quantify commits

Quantified Commit Activity Over 30 Days

Developer	Avatar	Branches	PRs	Commits	Files	Changes
hoshi-hiyouga		1	0/0/0	33	52	864
이루리		1	1/1/0	1	2	452
Иван		1	1/1/0	1	4	132
moontidef		1	2/2/0	3	9	64
Richard Wen		1	2/1/0	2	4	26
codingma		1	3/3/0	9	7	21
khazzz1c		1	1/1/1	1	2	5
liudan		1	0/0/0	1	2	4
Liuww		1	1/1/0	1	1	2
None (noiji)		0	1/0/0	0	0	0
piamo (piamo)		0	0/1/0	0	0	0
None (Zxilly)		0	1/0/0	0	0	0
Sangchun Ha (Patrick) (upskyy)		0	1/0/1	0	0	0
zzc (zzc0430)		0	1/0/0	0	0	0
kang sheng (techkang)		0	1/0/1	0	0	0
Huiyu Chen (chenhuiyu)		0	1/0/0	0	0	0
Ikko Eltociear Ashimine (eltociear)		0	1/0/0	0	0	0
Uminosachi (Uminosachi)		0	1/0/0	0	0	0
None (liu-zichen)		0	1/0/0	0	0	0
Ricardo (Ricardo-L-C)		0	1/0/0	0	0	0
Coding Steven (Truecodeman)		0	1/0/0	0	0	0
WeepingDogel (WeepingDogel)		0	1/0/1	0	0	0
None (LDLINGLINGLING)		0	1/1/0	0	0	0
None (huang-yu-sheng)		0	1/0/1	0	0	0

_{PRs: created by that dev and opened/merged/closed-unmerged during the period}

Quantify Issues

Recent GitHub Issues Activity

_{Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.}

Report On: Fetch issues

Recent Activity Analysis

The LLaMA-Factory project currently has 126 open issues, with recent activity indicating a surge in user inquiries and bug reports. Notably, many issues are related to model fine-tuning, inference errors, and memory management challenges, particularly with large models. A recurring theme is the struggle with out-of-memory (OOM) errors during training and inference, especially when using advanced configurations like LoRA and DeepSpeed.

Several issues exhibit similar characteristics, such as users encountering problems with specific model configurations or training parameters that lead to unexpected behavior. For instance, issues related to the integration of new models or methods often highlight the need for better documentation or examples.

Issue Details

Here are some of the most recently created and updated issues:

1. Issue #5206: 部署的时候能不能支持 vllm 部署 且 base 和 lora 不合并，即能访问base模型也能访问lora模型

   • Priority: Pending
   • Status: Open
   • Created: 1 day ago

2. Issue #5205: 请问webui后出现以下情况是什么原因，之前成功过一次，后来就不行了

   • Priority: Pending
   • Status: Open
   • Created: 1 day ago

3. Issue #5204: deepseekcoder-33b-instruct 微调输出不停止，微调的时候使用的是deepseekcoder的指令模版

   • Priority: Pending
   • Status: Open
   • Created: 2 days ago

4. Issue #5203: llava多模态模型使用dpo + lora微调报错

   • Priority: Pending
   • Status: Open
   • Created: 2 days ago

5. Issue #5202: llava多模态模型使用kto + lora微调报错

   • Priority: Pending
   • Status: Open
   • Created: 2 days ago

6. Issue #5199: Can't do inference on NPU, using the untuned Qwen2-7B-Instruct model

   • Priority: Pending, NPU
   • Status: Open
   • Created: 3 days ago

7. Issue #5195: save_steps is always 5

   • Priority: Pending
   • Status: Open
   • Created: 3 days ago

8. Issue #5194: What is the use of the config train on prompt, can anyone give me the docs of this extra config?

   • Priority: Pending
   • Status: Open
   • Created: 4 days ago

9. Issue #5190: 能不能把可选的依赖也直接放在包里呢

   • Priority: Pending
   • Status: Open
   • Created: 4 days ago

10. Issue #5189: LLaMA Factory, version 0.8.4.dev0 on Ascend910B failed for missing Op OnesLike

    • Priority: Pending, NPU
    • Status: Open
    • Created: 4 days ago

These issues reflect a mix of user requests for feature enhancements and reports of technical difficulties encountered during model training and deployment.

Important Observations

• There is a notable trend of users experiencing OOM errors when attempting to fine-tune large models or run inference with complex configurations.
• Many issues are labeled as "pending," indicating that they await further investigation or resolution from maintainers.
• The presence of multiple similar issues suggests a potential need for improved documentation or clearer guidelines on configuring models for specific tasks.
• Users frequently request support for new models and features, indicating an active interest in expanding the capabilities of the LLaMA-Factory framework.

This analysis highlights both the challenges faced by users and the ongoing engagement within the community to address these concerns through collaborative problem-solving and feature requests.

Report On: Fetch pull requests

Overview

The LLaMA-Factory project has a total of 18 open pull requests (PRs) currently under review. These PRs focus on enhancing model compatibility, improving performance, and adding new features to the framework.

Summary of Pull Requests

1. PR #5193: _is_bf16_available judgment supports npu

   • State: Open
   • Created: 3 days ago
   • Significance: This PR extends the functionality of the is_bf16_available check to include NPU devices, which is crucial for users working with Ascend hardware. It ensures that the framework correctly identifies device capabilities for bf16 support.

2. PR #5188: fix: report correct device count for intel xpu

   • State: Open
   • Created: 4 days ago
   • Significance: This PR corrects the device count reporting for Intel OneAPI devices, addressing potential misconfigurations in multi-device setups.

3. PR #5185: Add SailorLLM template

   • State: Open
   • Created: 4 days ago
   • Significance: Introduces a new template specifically for SailorLLM, enhancing support for Southeast Asian languages and improving user interaction with these models.

4. PR #5170: Avoid casting model params to float32 with unsloth

   • State: Open
   • Created: 5 days ago
   • Significance: Aims to reduce VRAM usage during LoRA training by avoiding unnecessary type casting, which can lead to out-of-memory errors on GPUs.

5. PR #5163: fix lr not change

   • State: Open
   • Created: 6 days ago
   • Significance: Fixes an issue where the learning rate scheduler was not properly integrated into the trainer, potentially impacting training performance.

6. PR #5156: fix Llama-template's system prompt bug

   • State: Open
   • Created: 6 days ago
   • Significance: Addresses a bug in the Jinja template that caused user-defined system prompts to be overwritten, enhancing customization options for users.

7. PR #5118: Support MistralV2 Format

   • State: Open
   • Created: 11 days ago
   • Significance: Adds support for MistralV2 format, which is essential for users utilizing this specific model architecture.

8. PR #5068: fix #3998

   • State: Open
   • Created: 14 days ago
   • Significance: Addresses an unspecified issue related to previous functionality, indicating ongoing maintenance and bug fixing.

9. PR #4733: merge easycontext

   • State: Open
   • Created: 41 days ago
   • Significance: Integrates easycontext into the workflow, which could enhance parallel processing capabilities during training.

10. PR #5019: overwrite training_step for CustomDPOTrainer to clear cuda cache every train step

    • State: Open
    • Created: 20 days ago
    • Significance: Implements a mechanism to clear CUDA cache after each training step, potentially improving memory management during training sessions.

11. PR #4957: docs: add Japanese README

    • State: Open
    • Created: 25 days ago
    • Significance: Enhances accessibility by adding a Japanese version of the README file, catering to a broader audience.

12. Additional PRs (#4877 to #1624) focus on various enhancements including metric updates, documentation improvements, and feature additions that support multiple models and functionalities within the LLaMA-Factory ecosystem.

Analysis of Pull Requests

The current set of open pull requests reflects several key themes in the ongoing development of the LLaMA-Factory project:

Device Compatibility and Performance Optimization

A significant number of PRs are focused on improving compatibility with various hardware configurations (e.g., NPU and Intel XPU). For example, PR #5193 enhances bf16 support for NPU devices, while PR #5188 ensures accurate device counting for Intel's OneAPI. These changes are crucial as they allow users to leverage different hardware effectively, which is particularly important in environments where resources may vary significantly.

User Experience Enhancements

Several PRs aim to improve user experience through better templates and documentation (e.g., PR #5185 introduces a SailorLLM template). The addition of language-specific templates and translations (like the Japanese README in PR #4957) indicates a strong commitment to making the framework accessible to a diverse user base. This is particularly relevant as LLaMA-Factory aims to cater to global audiences with varying linguistic backgrounds.

Bug Fixes and Maintenance

There is a clear emphasis on addressing existing issues and bugs within the codebase (e.g., PR #5163 fixes learning rate issues). This proactive approach not only improves stability but also builds trust within the community by demonstrating that contributors are attentive to user feedback and willing to resolve problems promptly.

New Feature Development

The introduction of new features such as support for additional model formats (e.g., MistralV2 in PR #5118) showcases an ongoing effort to expand the capabilities of LLaMA-Factory. This aligns with the project's goal of supporting a wide range of models and methodologies in fine-tuning large language models.

Community Engagement

The active discussion around many PRs indicates robust community engagement. Contributors are not only submitting code but also interacting with each other through comments and suggestions, fostering a collaborative environment that can lead to more innovative solutions and improvements over time.

In conclusion, the current pull requests reflect a dynamic development process characterized by enhancements in compatibility, user experience, bug fixes, and feature expansions—all critical elements that contribute to the overall robustness and usability of LLaMA-Factory as a leading framework for fine-tuning large language models.

Report On: Fetch commits

Repo Commits Analysis

Development Team and Recent Activity

Team Members

1. codemayq (codingma)

   • Recent Activity:
   • 9 commits with 21 changes across 7 files.
   • Collaborated on fixing examples and updating README files.
   • Merged a pull request to fix the evaluation dataset in examples.

2. hiyouga (hoshi-hiyouga)

   • Recent Activity:
   • 33 commits with 864 changes across 52 files.
   • Significant contributions include adding features, fixing bugs, and updating documentation.
   • Collaborated on multiple merges related to optimizers, training scripts, and dataset support.
   • Notably involved in merging pull requests for various fixes and enhancements.

3. YeQiuO (Richard Wen)

   • Recent Activity:
   • 2 commits with 26 changes across 4 files.
   • Worked on fixing bugs related to mask history and data loaders.
   • Merged a pull request addressing a truncation bug.

4. relic-yuexi (moontidef)

   • Recent Activity:
   • 3 commits with 64 changes across 9 files.
   • Contributed to adding support for AdamMini and renaming optimizer-related files.
   • Merged pull requests for bug fixes.

5. Eruly (이루리)

   • Recent Activity:
   • 1 commit with 452 changes across 2 files.
   • Added Korean web UI components, significantly impacting localization efforts.

6. liudan

   • Recent Activity:
   • 1 commit with 4 changes across 2 files.
   • No significant collaborative activity reported.

7. HardAndHeavy (Иван)

   • Recent Activity:
   • 1 commit with 132 changes across 4 files.
   • Added ROCm support, contributing to hardware compatibility.

8. khazic

   • Recent Activity:
   • 1 commit with 5 changes across 2 files.
   • Merged a pull request for adding reference addresses in documentation.

9. liuwwang

   • Recent Activity:
   • 1 commit with 2 changes across 1 file.
   • Merged a pull request addressing quantization issues.

Summary of Recent Activities

• The team has been actively working on enhancing the LLaMA-Factory project through numerous commits focused on bug fixes, feature additions, and documentation updates.
• Hoshi-hiyouga has been the most active contributor, significantly impacting the codebase by merging multiple pull requests and implementing new features.
• Collaboration is evident among team members, particularly in merging pull requests that address bugs and add functionalities, indicating a cohesive development effort.
• The recent activities reflect a strong focus on improving usability and performance of the software, as well as expanding its capabilities through localization and hardware support.

Patterns and Conclusions

• There is a clear division of labor with specific members focusing on distinct areas such as bug fixes, feature development, and documentation improvements.
• The frequency of merges suggests an agile workflow where contributions are rapidly integrated into the main branch, facilitating continuous improvement of the project.
• The engagement from multiple contributors indicates a healthy collaborative environment conducive to innovation and problem-solving within the LLaMA-Factory project.