‹ Reports
The Dispatch

Development Stagnates as Critical Bugs Persist in LLM Graph Builder

The LLM Graph Builder, a tool for transforming unstructured data into structured knowledge graphs using Neo4j and Large Language Models, faces development stagnation due to unresolved critical bugs impacting functionality.

Recent Activity

Recent issues highlight persistent problems with model integration and graph generation. Notably, #749 reports errors with schema extraction using local models, while #713 describes chatbot failures in document retrieval. These issues suggest instability that may deter user adoption.

Development Team Activity

  1. Kartik Persistent

    • Latest Commit: UI enhancements and bug fixes.
    • PRs: #743 (Chat mode menu descriptions).
  2. Prakriti Solankey

    • Latest Commit: Schema population improvements.
    • PRs: #748 (Graph communities), #746 (Local search integration).
  3. Vasanthasaikalluri

    • Latest Commit: Document handling updates.
  4. Aashi Pandya

    • Latest Commit: Documentation updates.
  5. Pravesh Kumar

    • Latest Commit: Bug fixes and testing enhancements.
  6. Abhishek Kumar

    • Latest Commit: Backend improvements.
    • PRs: #738 (Dev to Staging).
  7. Ajay Meena

    • Latest Commit: UI changes.
  8. Morgan Senechal

    • Role: Project management and coordination.

Of Note

  1. Critical Bugs Persist: Issues like #749 and #713 are unresolved, affecting core functionalities.

  2. Security Concerns: PR #738 raises security issues about sensitive information logging.

  3. Community Engagement: High number of open issues indicates active user feedback but also highlights unresolved challenges.

  4. Local Model Integration: Demand for offline capabilities is evident but problematic (#730).

  5. UI Enhancements: Continuous efforts to improve user experience through UI updates and documentation.

Quantified Reports

Quantify Issues



Recent GitHub Issues Activity

Timespan Opened Closed Comments Labeled Milestones
7 Days 3 9 7 1 1
30 Days 23 22 45 8 1
90 Days 165 147 212 73 1
All Time 379 310 - - -

Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.

Quantify commits



Quantified Commit Activity Over 30 Days

Developer Avatar Branches PRs Commits Files Changes
Prakriti Solankey (prakriti-solankey) 6 10/14/0 31 88 12896
aashipandya 6 4/4/0 11 163 12151
kartikpersistent 11 5/5/1 57 77 4232
Pravesh Kumar (praveshkumar1988) 6 3/3/0 22 60 3074
None (vasanthasaikalluri) 9 2/3/0 26 24 2135
None (abhishekkumar-27) 4 1/0/0 8 3 1048
None (edenbuaa) 1 1/1/0 1 1 34
None (buerbaumer) 0 0/0/1 0 0 0

PRs: created by that dev and opened/merged/closed-unmerged during the period

Detailed Reports

Report On: Fetch issues



Recent Activity Analysis

The recent activity on the GitHub repository for the LLM Graph Builder indicates a vibrant development environment, with 69 open issues and a steady stream of contributions. Notably, several critical bugs have been reported, particularly around functionality related to local models and graph generation, which may impact user experience and adoption.

There are recurring themes in the issues, including problems with model integration (especially local models), errors during graph generation from various data sources, and challenges related to Docker deployments. The presence of multiple unresolved bugs suggests potential instability in the current version of the application.

Issue Details

Most Recently Created Issues

  1. Issue #749: A bug in the request for populate_graphic_schema

    • Priority: High
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
    • Description: Error encountered while using the local ollama Qwen2:72b model, indicating unsupported arguments in schema extraction.
  2. Issue #740: Incorrect calculation of the number of relationships

    • Priority: Medium
    • Status: Open
    • Created: 10 days ago
    • Updated: 7 days ago
    • Description: Relationships calculated without deduplication, only counting the last chunk.
  3. Issue #730: Add Ollama local models

    • Priority: Low
    • Status: Open
    • Created: 15 days ago
    • Updated: 1 day ago
    • Description: User unable to connect local Ollama model; requests configuration assistance.
  4. Issue #723: Add Communities to the application

    • Priority: Enhancement
    • Status: Open
    • Created: 16 days ago
    • Updated: 4 days ago
    • Description: Suggestion to enhance community features within the application.
  5. Issue #713: Chat bot cannot work

    • Priority: High
    • Status: Open
    • Created: 22 days ago
    • Updated: 16 days ago
    • Description: Chatbot fails to find relevant documents despite successful graph generation.

Most Recently Updated Issues

  1. Issue #740: Incorrect calculation of the number of relationships

    • Last updated by user xcd008, indicating ongoing engagement with this issue.
  2. Issue #730: Add Ollama local models

    • Last updated by user Ignacio, showing attempts to resolve configuration issues.
  3. Issue #713: Chat bot cannot work

    • Last updated by user edenbuaa, reflecting continued troubleshooting efforts.
  4. Issue #704: Text2Cypher Graph Generation doesn't answer despite correct query and results

    • Last updated by Michael Hunger, indicating ongoing discussions about integration issues.
  5. Issue #691: 'str' object has no attribute 'content'

    • Last updated by lemencolo, highlighting a common error encountered during document processing.

Analysis Implications

The high volume of open issues, particularly those related to bugs and model integration, suggests that users are facing significant challenges with the current functionalities of the LLM Graph Builder. The presence of critical bugs like those affecting schema extraction and relationship calculations may hinder users' ability to effectively utilize the tool for their needs.

Moreover, the focus on local model integration points to a growing demand for offline capabilities among users who may have privacy concerns or limitations regarding cloud services. Addressing these issues promptly could enhance user satisfaction and foster greater adoption of the tool.

In summary, while there is active engagement within the community, resolving these pressing issues will be crucial for maintaining momentum and ensuring that users can leverage the full potential of the LLM Graph Builder effectively.

Report On: Fetch pull requests



Overview

The analysis of the pull requests (PRs) for the LLM Graph Builder project reveals a dynamic and active development environment. The project has seen a variety of enhancements, bug fixes, and feature additions over time, reflecting its growth and the community's engagement.

Summary of Pull Requests

Recent Open Pull Requests

  • PR #738: Dev to Staging
    • Status: Open
    • Created by: abhishekkumar-27
    • Significance: This PR aims to merge changes from the development branch to staging. It includes multiple commits related to integration testing, bug fixes, and feature enhancements.
    • Notable Issues: Security concerns regarding clear text storage and logging of sensitive information have been raised by GitHub's advanced security bot.

Closed Pull Requests

  • PR #748: Graph communities

    • Status: Closed
    • Merged by: Prakriti Solankey
    • Significance: This PR adds functionality related to graph communities, enhancing the project's capabilities in handling graph data.
  • PR #746: Integrate local search to chat details

    • Status: Closed
    • Merged by: Prakriti Solankey
    • Significance: This PR integrates local search functionality into chat details, improving user interaction with the system.
  • PR #743: Added Description to chat mode menu

    • Status: Closed
    • Merged by: kartikpersistent
    • Significance: This PR adds descriptions to chat mode menus, enhancing user understanding of different modes available in the application.
  • PR #739: Add communities Checkbox to graph viz

    • Status: Closed
    • Merged by: Prakriti Solankey
    • Significance: This PR adds a checkbox for communities in graph visualization, allowing users to customize their view based on community data.

Other Notable Pull Requests

  • PR #737: Retry processing - node and rels count update condition for start from beginning

    • This PR addresses issues related to retry processing, ensuring that node and relationship counts are correctly updated when starting from the beginning after a retry.
  • PR #736: youtube transcript issue

    • This PR implements functionality for retrieving YouTube transcripts, expanding the project's capabilities in handling diverse data sources.

Analysis of Pull Requests

The LLM Graph Builder project exhibits several key themes and patterns in its pull request activity:

  1. Active Development and Community Engagement: The project has a high level of activity with numerous pull requests being opened, reviewed, and merged regularly. This indicates strong community involvement and active maintenance by the core team.

  2. Continuous Improvement and Feature Expansion: Many pull requests focus on enhancing existing features or adding new ones. For instance, the integration of local search into chat details (PR #746) and the addition of community checkboxes in graph visualization (PR #739) demonstrate ongoing efforts to improve user experience and expand functionality.

  3. Attention to Security and Quality Assurance: The presence of security-related comments in open pull requests (e.g., PR #738) highlights an awareness of security best practices. Additionally, integration testing and bug fixes are common across multiple pull requests, indicating a commitment to maintaining high software quality.

  4. Diverse Contributions: Contributions come from various developers with different focuses, such as UI enhancements (e.g., PR #743), backend improvements (e.g., PR #737), and new feature implementations (e.g., PR #736). This diversity enriches the project's development and helps address various aspects of the software.

  5. Documentation and Configuration Updates: Several pull requests include updates to documentation (e.g., README.md) and configuration files (e.g., docker-compose.yml), ensuring that users have up-to-date information on setup and usage.

In conclusion, the LLM Graph Builder project is characterized by active development, continuous improvement efforts, strong community engagement, attention to security and quality assurance, diverse contributions, and regular updates to documentation and configuration. These factors contribute to its growth and relevance in the field of knowledge graph construction from unstructured data using AI technologies.

Report On: Fetch commits



Repo Commits Analysis

Development Team and Recent Activity

Team Members

  • Kartik Persistent: Active in multiple features, including integration of various LLMs, bug fixes, and UI enhancements.
  • Prakriti Solankey: Focused on schema population, GCS integration, and UI improvements.
  • Vasanthasaikalluri: Worked on LLM integration, document handling, and UI adjustments.
  • Aashi Pandya: Contributed to documentation updates and feature enhancements.
  • Pravesh Kumar: Involved in bug fixes, testing, and feature development.
  • Abhishek Kumar: Participated in backend improvements and testing.
  • Ajay Meena: Assisted with various integrations and UI changes.
  • Morgan Senechal: Engaged in overall project management and coordination.

Recent Activities

  1. Integration of LLMs:

    • Multiple commits focused on integrating new LLM models (e.g., Bedrock, Ollama) into the application.
    • Enhanced token handling for LLMs to improve performance.
  2. Feature Development:

    • Implemented a chat mode for interacting with the knowledge graph.
    • Added support for uploading unstructured files to GCS.
    • Developed a settings modal for user-defined schema generation.
  3. Bug Fixes:

    • Resolved issues related to status messages in GCS uploads.
    • Fixed retrieval bugs affecting the application’s performance.
    • Addressed layout issues in the UI to enhance user experience.
  4. Documentation Updates:

    • Continuous updates to README files and frontend documentation to reflect new features and usage instructions.
  5. Testing Enhancements:

    • Integration QA tests were added to ensure stability across deployments.
    • Updated test cases to align with new features and bug fixes.
  6. UI Improvements:

    • Significant changes made to the frontend components for better usability, including tooltips and modal adjustments.
    • Enhanced visual elements for graph representation and user interactions.

Patterns and Themes

  • Collaboration: Frequent co-authorship indicates strong collaboration among team members on various features and bug fixes.
  • Focus on Usability: A consistent theme of improving user experience through UI enhancements and detailed documentation updates.
  • Rapid Iteration: The team is actively iterating on features based on user feedback, as evidenced by the number of bug fixes and enhancements being deployed regularly.

Conclusions

The development team is actively engaged in enhancing the LLM Graph Builder project through collaborative efforts. They are focusing on integrating advanced AI models, improving usability, fixing bugs, and maintaining comprehensive documentation. The project demonstrates a proactive approach to development with a clear emphasis on user experience and functionality.