Open-Sora Plan, an open-source initiative by PKU-YuanGroup, aims to replicate OpenAI's Sora model for text-to-video generation. The project supports Huawei Ascend AI systems and is designed to produce industry-standard video quality.
In the past 30 days, the project has seen significant activity, highlighted by the release of version 1.2.0 on August 13, 2024. This update introduced an image-to-video generation model and marked a transition to a true 3D video diffusion model. However, the project also faces challenges with technical issues such as CUDA errors and GPU resource management, as evidenced by numerous open issues.
Recent issues and pull requests reveal a focus on expanding features while addressing technical difficulties. Key issues include CUDA-related errors (#386) and inquiries about model parameters (#390), indicating ongoing struggles with GPU configurations and documentation clarity.
| Developer | Avatar | Branches | PRs | Commits | Files | Changes |
|---|---|---|---|---|---|---|
| lb203 |  |
3 | 0/0/0 | 45 | 287 | 99241 |
| yunyang Ge |  |
1 | 2/1/1 | 1 | 22 | 5130 |
| apprivoiser |  |
1 | 0/0/0 | 3 | 32 | 1489 |
| Xinhua Cheng |  |
1 | 1/1/0 | 1 | 1 | 6 |
PRs: created by that dev and opened/merged/closed-unmerged during the period
| Timespan | Opened | Closed | Comments | Labeled | Milestones |
|---|---|---|---|---|---|
| 7 Days | 8 | 6 | 4 | 8 | 1 |
| 30 Days | 45 | 20 | 81 | 45 | 1 |
| 90 Days | 98 | 28 | 200 | 98 | 1 |
| All Time | 252 | 72 | - | - | - |
Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.
The Open-Sora-Plan repository has recently experienced a surge in activity, with 180 open issues currently documented. Notably, there are several urgent requests for assistance and clarification regarding model training and inference, particularly concerning CUDA errors and memory management. A recurring theme among the issues is the challenge of managing GPU resources effectively, especially when using multiple GPUs or specific configurations like A100 or H100 cards. Additionally, there are multiple inquiries about missing files and modules, indicating potential gaps in documentation or setup instructions.
Several issues highlight critical errors that users face during training and inference, such as out-of-memory (OOM) errors and problems related to the integration of various components of the model. The community appears engaged, with many users seeking support for technical challenges, suggesting a robust interaction between developers and users.
Issue #393: 求开源29x480p的ckpt
Issue #392: inpainting train
Issue #390: Question: Why the compression ratio for VAE is set to be 4X8X8
Issue #386: NCCL Error 1: unhandled cuda error
Issue #384: killed after the model is saved
Issue #383: A request
Issue #382: Issues with Training CausalVideoVAE v1.2.0
Issue #380: Can you tell me the typical inference time? Would it take 19 hours for 17 frame video after stage 1 training?
Issue #375: get_3d_sincos_pos_embed
Issue #366: about sp seed config. Why seed in each device is different?
This analysis underscores the importance of addressing both technical challenges and user experience improvements to enhance the overall effectiveness of the Open-Sora-Plan project.
The repository PKU-YuanGroup/Open-Sora-Plan has a total of 8 open pull requests and 115 closed pull requests, reflecting ongoing development and community engagement in enhancing the project's capabilities. The recent pull requests focus on various aspects such as feature enhancements, bug fixes, documentation updates, and refactoring.
PR #325: [refactor]: add type checking to sample image and video functions
This PR introduces type checking for functions related to sampling images and videos, improving code reliability. It was created 53 days ago and edited recently.
PR #322: [fix]: Fix wrong file name in readme
A minor fix that corrects a command line example in the README file by replacing a non-existent script name with an existing one. Created 57 days ago.
PR #321: [feat]: support navit
This substantial PR adds support for the NaViT model, including numerous files and significant code changes. It was created 57 days ago.
PR #279: add support snr_gamma for rebalancing loss
Introduces a new parameter for rebalancing loss in the model, referencing an academic paper for context. Created 88 days ago.
PR #247: [docs]: fix the bug in cal_fvd.py
A documentation-related fix that addresses issues in the evaluation script. Created 123 days ago.
PR #241: Fix grammatical errors in documentation
This PR improves the readability of documentation by fixing grammatical issues. It was created 124 days ago.
PR #227: Changes for sample T2V
Refactors the sample_t2v.py script to simplify deployment and adds type hints. Created 127 days ago.
PR #208: :recycle: [Refactor] path handling for clarity using os.path.dirname
Refactors path handling in the codebase to improve clarity. Created 129 days ago.
PR #389: release Open-Sora Plan v1.2.0 i2v
This PR marks the release of version 1.2.0 of Open-Sora Plan, merging multiple updates and improvements. Closed recently.
PR #388: release Open-Sora Plan v1.2.0 i2v (not merged)
A duplicate attempt at releasing version 1.2.0 that was not merged.
PR #352: Update Report-v1.2.0.md
Minor updates to the release report document, correcting typos and enhancing clarity. Closed recently.
PR #334: [feat]: Inpaint model and video ip adapter
Introduces new models for inpainting and video processing, showcasing significant additions to the project’s capabilities. Closed recently.
PR #290: fix the bug
A straightforward bug fix addressing issues in super-resolution scripts, merged successfully.
PR #289: Open-Sora-Plan v1.1.0 NPU
Support for training and inference on NPU, enhancing hardware compatibility with specific implementations closed successfully.
The pull requests submitted to the Open-Sora Plan repository reveal several key themes and trends that reflect both ongoing development efforts and community engagement:
A significant number of recent pull requests are dedicated to adding new features or improving existing functionalities within the project. For instance, PR #321 introduces support for the NaViT model, which is a substantial addition that likely enhances the project's capabilities in video generation tasks. This trend indicates a proactive approach by contributors to expand the project's scope beyond its initial design, potentially attracting more users and contributors interested in advanced modeling techniques.
Several pull requests focus on enhancing documentation quality (e.g., PR #322, PR #241). These updates are crucial as they help new users understand how to use the software effectively while also ensuring that existing users can leverage new features without confusion. The emphasis on correcting errors and improving clarity suggests a commitment to maintaining high-quality resources that facilitate user engagement with the project.
The presence of multiple bug fixes (e.g., PR #290) alongside refactoring efforts (e.g., PR #325) highlights an ongoing commitment to code quality and reliability within the repository. By addressing bugs promptly and refactoring code for better readability or performance, contributors demonstrate their dedication to creating a robust software environment that minimizes user frustration and maximizes functionality.
The variety of contributors involved in these pull requests reflects an active community surrounding Open-Sora Plan. The diversity of contributions—from feature additions to minor fixes—indicates a collaborative spirit where users feel empowered to participate in the development process actively. This engagement is essential for sustaining long-term project growth and ensuring that it remains relevant in a rapidly evolving field like AI-driven video generation.
One notable anomaly is PR #388, which attempted to release version 1.2.0 but was not merged despite being similar to PR #389, which successfully released it shortly after. This raises questions about workflow management within the repository—specifically how duplicate efforts are handled and communicated among contributors.
In conclusion, the pull request activity within PKU-YuanGroup/Open-Sora-Plan showcases a vibrant development environment characterized by feature enhancements, diligent documentation practices, proactive bug fixing, and strong community involvement—all of which are vital components for fostering a successful open-source project in today's competitive landscape.
LinB203
yunyangge
cxh0519
apprivoiser
The development team is actively engaged in enhancing the Open-Sora Plan project through collaborative efforts, significant feature releases, and ongoing documentation improvements. The focus on community contributions and robust feature sets positions the project as a notable player in AI-driven video generation within the open-source landscape.
Summary
Quantified Reports
Detailed Reports