‹ Reports
The Dispatch

GitHub Repo Analysis: Generic


Software Project Analysis: ChatGLM3

ChatGLM3, a bilingual chatbot language model, has seen enhancements in power, function, and training data diversity. Currently, the project's repo does not bear noticeable anomalies, but highlights certain TODOs regarding tests and runtime issue-handling. The project has 25 open issues, some of which involve requests for added features or instructions, alongside reports of bugs and errors.

Key Concerns

  1. Unclear Documentation: Many open issues request further details on deployment and usage.
  2. Runtime Errors: Some users encounter errors or inconsistencies after inputting certain commands or phrases.
  3. Fine-Tuning Release: There's recurring anticipation for the release of fine-tuning code.

ChatGLM3, despite having attracted attention and involvement, needs focused issue resolution and better documentation, pertaining to particular features and deployment steps.

Spotlight Issues:

Closed Issues:

27 issues have been successfully closed.

Pull Requests:

No open pull requests. Three recent pull requests have been closed, of which only one was successfully merged.

Prognosis: This project shows potential but requires issue resolution and detailed documentation for optimal usage.

Detailed Reports

Report on issues



Active Issues:

  • #2: Users are awaiting release of fine-tuning code for the project. Some users have offered alternatives.
  • #55: User has modified openai_api.py originally for ChatGLM2 to support ChatGLM3.
  • #54: Request for details on how to deploy the model on a mobile device.
  • #53: Tool does not prompt for required inputs and sometimes generates unrelated output.
  • #52: Among other issues, there is a complaint about scrambled response content.
  • #51: Error encountered after inputting phrase "你好啊".
  • #50: Request for details on how to set a personalized system prompt.
  • #49: Query regarding the Python version compatibility.
  • #48: After some successful model deployment calls, a CUDA error emerges.
  • #46: Request for official support for ChatGPT-Next-Web.
  • #44: The example methods in composite_demo don't get called properly.
  • #41: Request for sample code to guide usage of ChatGLM3.
  • #37: Mistakes in generated text.
  • #35: Error occurs irrelevant of the content output.
  • #34: Large score discrepancies found when trying to reproduce benchmark results.
  • #30: Query about the availability of an int4 quantized weight for ChatGLM3-6b.
  • #28: Request for chat format score details for Chat model.
  • #26: Issue with function calling format used by OpenAI API.
  • #20: Request for updated openai api script.
  • #19: Query about support for online and offline quantization.
  • #18: Query about whether generate method can support embedding input.
  • #10: Query about plans for 10+ billion parameter models.
  • #9: Cant access web_demo.py in the browser despite having port open.
  • #8: ModuleNotFoundError encountered on running webdemo2.
  • #7: Issues with tokenizer and special character handling.
  • #2: Anticipation for the release of fine-tuning codes.

Closed Issue count: 27

Report on pull requests



Open Pull Requests:

No open pull requests found.

Recently Closed Pull Requests:

  1. #47 Add openai api support: This PR aimed to add support for the OpenAI API. It was closed without merging; possibly due to the incompatibility or lack of necessity. Requires additional context for the exact reason.

  2. #32 Add composite demo: This PR intended to add a composite demo to the code. Closed without merging. This might suggest that the demo was not required, had bugs, lacked documentation, or other issues impeding integration.

  3. #13 Improve web_demo2.py code and operation(agent prompt support): Improvement of the web_demo2.py code and functionality was proposed here. It was closed successfully, indicating the suggested improvements have been implemented.

Older Closed Pull Requests:

None provided.

Report on README



Summary

Project Name: THUDM/ChatGLM3

It's a bilingual chatbot language model, jointly released by ZhipuAI and the KEG laboratory at Tsinghua University. This model, while retaining the excellent features of the previous two generations of models, such as smooth dialogue and low deployment threshold, introduces the following features:

  • More powerful basic model with diverse training data.
  • More complete functionality support with a newly designed Prompt format.
  • More comprehensive open-source series including models for different text lengths.

The model is designed to be open-source and free to use for academic purposes after completing a survey for registration. Commercial use is also permitted without charge. The project team has not developed any apps based on this open-source model, including web, Android, iOS, or Windows apps. Due to the randomness of model responses and susceptibility to user input, the team is not responsible for any data safety issues or misuse of the model and its output.

The repo also includes instructions on how to deploy and use the model, including way to call up the chat model in code, load model locally from Hugging Face's model repository or ModelScope, web-based demos, and command-line interactive dialogue.

Potential Command additions are specified in the form of TODOs, like detailed instructions for running Tests and Deployments, highlighting potential enhancements, and dealing with potential code breaks during runtime.

However, the project has 25 open issues, the resolution of these issues might be important for better usage of the project.

Language: Python License: Other

Anomalies

No obvious anomalies were identified in this project.

Uncertainties

The code includes sentences in foreign languages, the accuracy and appropriateness of which cannot be verified.

There are no obvious indications of problems with the project beyond the stated open issues, but the 'trust_remote_code' in the release might need further assessments.

TODOs

Better explanations for tests and additional code handling for potential issues during runtime are needed. Also, resolution of the existing 25 open issues needs to be worked upon.