The recently opened issues for the software project mainly revolve around technical difficulties and inquiries about the software's functionality. Issue #80 discusses the unavailability of certain features on Jetson ORIN, which might indicate compatibility issues with specific hardware. Issue #79 and #78 are inquiries about the software's functionality, specifically about the use of window attention technology during the training phase and how to process batch input in the software. Issue #77 reports a repeated build failure, which could indicate a recurring problem in the software's build process. Issue #74 requests the addition of .safetensors files in the model card, suggesting a need for more file format support.
The older open issues are a mix of technical difficulties, inquiries, and suggestions. Issue #72 reports an inability to load the software to a GPU with 24 GB vRAM with quantization, indicating possible memory management issues. Issue #67 asks about the best format for fine-tuning the model, suggesting a need for clearer instructions or documentation. Issue #66 reports the model giving answers in Russian, which could indicate an issue with language settings or training data. Issue #62 discusses the lack of an embedding model and engine for Mistral, suggesting a need for more features or functionality. Issue #61 asks about plans for supporting more languages, indicating user interest in multilingual support. The recently closed issues are not provided, so they cannot be summarized or analyzed. The common theme among all open issues is a need for clearer instructions, more features or functionality, and better compatibility with different hardware and file formats.
There are 7 open pull requests. The most recent ones are #81 and #75, both dealing with Dockerfile updates.
PR #81 is a fix for the Dockerfile to pin the PyTorch version to the latest stable release built with CUDA 11.8. This is crucial for the project's compatibility with specific CUDA versions.
PR #75 is also a Dockerfile update, this time to fix a system prompt bug in the FastChat dependency. This bug could potentially affect the user experience and functionality of the software, making this PR important.
PR #63 and PR #11 are the most discussed open PRs. PR #63 is a proposed correction of grammatical errors in the README.md file, but there is disagreement about whether the changes are necessary or correct. PR #11 is an extensive documentation update adhering to PEP257 and PEP8, but it seems to be awaiting review for a long time.
The oldest open PRs are #11, #26, #56, #63, and #70. PR #11 and #63 are already mentioned above. PR #26 proposes to add top_k text decoding to main.py, PR #56 suggests a minor correction in the README.md file, and PR #70 updates the README.md with information about the Mistral-7B model hosted on the Clarifai cloud.
There are 14 closed pull requests. The most recent ones are #49, #46, #44, #43, and #24.
PR #49 proposed a refactoring of the model.py file into smaller modules, but it was closed without merging as it was deemed unnecessary.
PR #46 proposed renaming the main.py file and was also closed without merging due to the removal of some parentheses and the proposed renaming of the main file.
PR #44 updated the README.md file and was merged.
PR #43 updated the PyTorch version in the Dockerfile to support NVIDIA H100 PCIe GPUs and was merged.
PR #24 aimed to fix a broken link in the README.md file, but it was closed without merging as the issue was already fixed in another PR.
Dockerfile updates are a recurring theme in the open PRs, highlighting the importance of maintaining up-to-date dependencies and compatibility with different systems.
Documentation updates, both in terms of correcting errors and adding new information, are also a common theme.
There seems to be a delay in reviewing some PRs, particularly #11, which has been open for over two months. This could potentially slow down the project's progress.
There are disagreements over certain proposed changes, such as the grammatical corrections in PR #63. This could lead to confusion and miscommunication among contributors.
The mistralai/mistral-src project is a reference implementation of the Mistral AI 7B v0.1 model. Created by the organization mistralai, the software is written in Jupyter Notebook and licensed under Apache License 2.0. The project is designed to run the 7B model, with the repository containing minimal code for this purpose. The project is actively developed, with the latest push made on 2023-12-11.
The repository is quite popular and active, with 4899 stars, 61 watchers, and 332 forks. The project has a size of 247kB and has 38 total commits made on a single branch. There are 51 open issues, indicating ongoing development and user engagement. The project uses a Docker-based deployment system and requires the installation of specific dependencies, as detailed in the README. The software stack includes Python and Jupyter Notebook, with the transformers library used in the deployment image.
The project employs a sliding window attention mechanism to speed up inference and reduce memory pressure. This is a notable aspect as it deviates from the traditional full attention mechanism used in vanilla transformers. The project also implements a rolling buffer cache and uses chunking for large prompts. The README provides a detailed explanation of these mechanisms, complete with illustrations. The project also includes links to various integrations and related projects, indicating its wide applicability. The README does not mention any explicit future plans or roadmaps for the project.
Summary
Detailed Reports