The Open-Sora Plan, an open-source initiative by PKU-YuanGroup to replicate OpenAI's Sora model for video generation, has experienced a significant rise in user-reported issues, particularly concerning memory management and training configurations. The project, which emphasizes scalability on Huawei Ascend AI systems, continues to attract community contributions and maintains a robust development pace.
Recent issues highlight persistent challenges with memory management during training and inference, as well as confusion over model parameters. For instance, #442 questions the use of latent models, while #441 reports failures in frame addition during VAE training. These issues suggest potential gaps in documentation or implementation clarity.
Chestnut (qqingzheng)
Summary
Quantified Reports
Detailed Reportsanyres branch.Guangyi Liu (guangyliu)
gradio_web_server.py, addressed cache inconsistencies.Yunyang Ge (yunyangge)
LinB203 (lb203)
train_inpaint.sh, README.md.Apprivoiser
The team shows a high level of collaboration, with LinB203 leading in activity and working closely with Chestnut and Yunyang Ge on various enhancements.
| Timespan | Opened | Closed | Comments | Labeled | Milestones |
|---|---|---|---|---|---|
| 7 Days | 5 | 2 | 7 | 5 | 1 |
| 30 Days | 40 | 16 | 69 | 40 | 1 |
| 90 Days | 106 | 40 | 220 | 106 | 1 |
| All Time | 291 | 88 | - | - | - |
Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.
| Developer | Avatar | Branches | PRs | Commits | Files | Changes |
|---|---|---|---|---|---|---|
| lb203 |  |
3 | 0/0/0 | 33 | 164 | 35406 |
| yunyang Ge (yunyangge) | 1 | 1/1/0 | 1 | 48 | 4477 | |
| Chestnut (qqingzheng) | 1 | 5/5/0 | 5 | 30 | 4192 | |
| apprivoiser |  |
2 | 0/0/0 | 3 | 11 | 691 |
| Guangyi Liu |  |
1 | 1/1/0 | 1 | 4 | 10 |
| YJ (0YJ) | 0 | 0/0/1 | 0 | 0 | 0 | |
| Gyeongbo Sim (asdf2kr) | 0 | 1/0/0 | 0 | 0 | 0 | |
| Xinhua Cheng (cxh0519) | 0 | 1/0/1 | 0 | 0 | 0 | |
| Joel (wuxibin89) | 0 | 1/0/0 | 0 | 0 | 0 | |
| foreverpiano (foreverpiano) | 0 | 1/0/0 | 0 | 0 | 0 |
PRs: created by that dev and opened/merged/closed-unmerged during the period
The Open-Sora-Plan repository has seen a surge in activity, with 203 open issues currently being tracked. Notably, several recent issues have surfaced regarding bugs, training configurations, and model performance, indicating ongoing challenges faced by users. Common themes include memory management during training and inference, as well as the need for clarification on model parameters and configurations.
Several issues exhibit anomalies, such as the frequent occurrence of CUDA out-of-memory (OOM) errors despite varying GPU configurations and the confusion surrounding the use of different model weights for video generation tasks. Additionally, there are reports of discrepancies between expected and actual outputs when using specific configurations, suggesting potential gaps in documentation or implementation.
Issue #442: [Bug]? Not CFG latent_model_input Instead of latents?
Issue #441: Training the vae fails to add the number of frames to 29
Issue #438: sp加速训练策略和非sp策略的疑问
Issue #437: 关于训练方法和数据集切分问题？
Issue #434: Train Iterations for I2V
Issue #441: Training the vae fails to add the number of frames to 29 (updated)
Issue #438: sp加速训练策略和非sp策略的疑问 (updated)
Issue #437: 关于训练方法和数据集切分问题？ (updated)
Issue #430: Best practices for images to video generation? (updated)
Issue #429: OSError related to missing files in model path (updated)
The recent activity indicates a robust engagement from users who are actively testing and utilizing the Open-Sora-Plan framework. However, the volume of issues related to memory errors and configuration problems suggests that further optimization and clearer documentation may be necessary to enhance user experience and model performance.
The presence of multiple discussions around training strategies and model parameters highlights a potential need for improved onboarding materials or tutorials that can guide new users through common pitfalls encountered during setup and execution.
Overall, while the project is progressing well with community support, addressing these recurring issues will be crucial for maintaining user satisfaction and encouraging further contributions.
The Open-Sora-Plan repository has a total of 10 open pull requests (PRs) and 124 closed PRs. The recent activity indicates a mix of bug fixes, feature enhancements, and documentation updates, reflecting ongoing development and community engagement.
PR #440: A minor fix to correct a typo in sample_inpaint.sh from "PDNM" to "PNDM". This PR highlights the attention to detail in code quality.
PR #435: Addresses a bug related to reshaping tensor dimensions when global_bs = 1. This fix is significant for ensuring proper data handling during model training.
PR #416: Introduces functionality to resume dataloader training by skipping already consumed batches, which can significantly improve efficiency during long training sessions.
PR #325: Adds type checking to functions for sampling images and videos, enhancing code robustness and maintainability.
PR #322: Fixes a wrong file name in the README, ensuring that users have accurate documentation for command line usage.
PR #321: Adds support for "navit", indicating an expansion of features or models within the project.
PR #279: Introduces support for snr_gamma in rebalancing loss, which may enhance model performance based on recent research findings.
PR #247: Fixes a bug in cal_fvd.py, showcasing ongoing improvements in evaluation metrics.
PR #227: Refactors sample_t2v.py for easier deployment and adds type hints, improving code clarity.
PR #208: Refactors path handling for clarity using os.path.dirname, indicating a focus on code readability.
PR #443: Merged changes related to "Anyres", including trilinear lossless upsampling features. This indicates progress in enhancing model capabilities.
PR #409: A minor update to utils.py that was not merged but indicates ongoing maintenance efforts.
PR #422: Merged changes for lossless VAE chunking, reflecting improvements in model efficiency.
PR #421: Merged changes related to "Anyres", showing continued development in this area.
PR #405: Fixed a path typo in gradio_web_server.py, demonstrating attention to user experience.
The current state of pull requests in the Open-Sora-Plan repository reflects a healthy mix of maintenance, feature enhancement, and community engagement. The open pull requests indicate active contributions from various developers, with notable focus areas including bug fixes (e.g., PRs #435 and #440), feature additions (e.g., PRs #416 and #321), and documentation improvements (e.g., PRs #322 and #325). This diversity suggests that contributors are not only addressing immediate issues but are also looking to enhance the overall functionality and usability of the project.
A significant theme across the recent pull requests is the emphasis on robustness and efficiency. For instance, PR #416's implementation of dataloader resumption can greatly reduce training times, particularly beneficial for large datasets. Similarly, PR #279's introduction of snr_gamma reflects an alignment with contemporary research trends aimed at improving model performance through advanced loss functions.
The presence of multiple documentation-related pull requests indicates an awareness of the importance of clear instructions for users. Accurate documentation is crucial for fostering community engagement and ensuring that new users can effectively utilize the project without confusion. This is particularly important given the complexity often associated with AI models and their training processes.
However, there are some anomalies worth noting. For example, several older pull requests remain open without any recent activity or merging attempts. This could suggest potential bottlenecks in the review process or indicate that some contributions may not align with current project priorities. It would be beneficial for project maintainers to periodically review these older PRs to either merge them or provide feedback to contributors.
Moreover, while there is a strong focus on technical enhancements, it might be useful for the team to consider more strategic planning around feature releases and community involvement. Engaging contributors through regular updates or discussions about upcoming features could help streamline contributions and ensure alignment with project goals.
In conclusion, the Open-Sora Plan repository exhibits a vibrant development environment characterized by active contributions focused on improving functionality and user experience. Continued attention to both technical advancements and community engagement will be key to sustaining this momentum as the project evolves.
Guangyi Liu (guangyliu)
gradio_web_server.py and addressed cache inconsistencies for pretrained weights.LinB203 (lb203)
train_inpaint.sh, README.md, and several scripts related to training and inference.Chestnut (qqingzheng)
anyres branch.Yunyang Ge (yunyangge)
Apprivoiser
High Activity Level: LinB203 is the most active member with 33 commits, indicating a strong focus on enhancements and bug fixes. This suggests a proactive approach to maintaining code quality and feature development.
Collaborative Efforts: There is a notable pattern of collaboration among team members, particularly between LinB203, Chestnut, and Yunyang Ge. This indicates a cohesive team dynamic where members frequently work together on overlapping tasks.
Focus on Bug Fixes and Enhancements: The recent commits predominantly involve fixing bugs, updating documentation, and enhancing existing functionalities. This reflects an ongoing commitment to improving the robustness of the project.
Diverse Contributions Across Branches: The team is actively working across multiple branches, including main, anyres, suv, and others, which allows for parallel development of features without disrupting the main codebase.
The development team is actively engaged in improving the Open-Sora Plan project through collaborative efforts focused on bug fixes, documentation updates, and feature enhancements. The high level of activity from key contributors suggests a well-functioning team dedicated to advancing the project's capabilities while maintaining quality standards.