‹ Reports
The Dispatch

OSS Report: janhq/jan


Jan Project Faces User Frustration Amidst Model Compatibility Issues

Jan, an open-source offline alternative to ChatGPT, has been experiencing significant user frustration due to model compatibility and performance issues following updates. The project, developed in TypeScript, aims to provide a self-hosted AI experience across various hardware configurations without relying on cloud services.

The most notable activity within the past 30 days includes a surge in user-reported issues, particularly concerning model loading failures and UI inconsistencies. The transition to the Cortex backend appears to have introduced instability, leading to numerous bug reports and feature requests aimed at improving user experience. Despite these challenges, the development team remains active, with ongoing efforts to enhance functionality through UI improvements and bug fixes.

Recent Activity

Recent issues predominantly revolve around bugs related to model handling and user interface functionality. For instance, #3410 addresses missing placeholders for empty search results, while #3409 highlights import errors on Windows systems. These issues indicate a trend of technical difficulties affecting user satisfaction.

The development team has been actively working on resolving these issues and enhancing the application. Key contributors include:

  1. Faisal Amir (urmauur)

    • Fixed remote model settings based on API key setup.
    • Removed model selection from the assistant tab.
    • Enabled copy instruction feature.
  2. Louis (louis-jan)

    • Addressed error messages related to settings retrieval.
    • Improved performance by cleaning up console logs.
  3. Van Pham (Van-QA)

    • Added a simple bug report template.
    • Participated in minor UI updates.
  4. NamH (namchuai)

    • Worked on bug fixes related to model visibility and engine initialization.
  5. hiento09

    • Migrated workflows from Cloudflare R2 to AWS S3 for nightly builds.
  6. Gabrielle Ong (gabrielle-ong)

    • Made minor copy edits across various files.
  7. Mark Nguyen (marknguyen1302)

    • Added tests for migration processes and isolated end-to-end tests.

Of Note

Quantified Reports

Quantify commits



Quantified Commit Activity Over 30 Days

Developer Avatar Branches PRs Commits Files Changes
Louis 6 24/22/2 34 355 82808
NamH 4 27/26/2 23 398 54938
Faisal Amir 7 46/39/5 46 96 3358
hiento09 2 3/2/1 5 15 2473
Mark Nguyen (marknguyen1302) 1 1/0/0 2 12 1214
Hien To 2 0/0/0 2 9 341
Van Pham 2 4/4/0 4 3 78
Nicole Zhu 2 1/1/0 3 6 74
Van QA 1 0/0/0 2 3 6
Gabrielle Ong 2 1/1/0 2 1 4
Luke Niesink 1 1/1/0 1 1 2
netwjx (netwjx) 0 0/0/2 0 0 0
None (Realmbird) 0 0/0/1 0 0 0
Quan Pham (Doko-Demo-Doa) 0 1/0/1 0 0 0
Marco Di Francesco (MarcoDiFrancesco) 0 1/0/1 0 0 0

PRs: created by that dev and opened/merged/closed-unmerged during the period

Quantify Issues



Recent GitHub Issues Activity

Timespan Opened Closed Comments Labeled Milestones
7 Days 35 18 1 1 5
30 Days 120 87 119 1 5
90 Days 243 149 318 4 7
All Time 1644 1451 - - -

Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.

Detailed Reports

Report On: Fetch issues



Recent Activity Analysis

The recent activity on the Jan repository indicates a significant influx of issues, with 193 open issues currently documented. The majority of these issues are categorized as bugs, particularly related to model loading and user interface functionality. Notably, many users report problems with model compatibility and performance, especially after updates. There is a clear trend of users expressing frustration over the handling of local models and API integrations, suggesting that the transition to the Cortex backend has introduced instability in existing functionalities.

Several issues highlight critical bugs, such as #3410 regarding missing placeholders for empty search results and #3409 about import errors on Windows. Additionally, there are numerous feature requests aimed at improving user experience, including enhancements to model management and UI responsiveness.

Issue Details

Most Recently Created Issues:

  1. Issue #3410:

    • Title: bug: Missing placeholder for empty search result for Hub and My models
    • Priority: P2
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
  2. Issue #3409:

    • Title: bug: Import with folder in Jan app is throwing error with Invalid path
    • Priority: P2
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
  3. Issue #3408:

    • Title: [v0.5.2-590] Automatically naming threads doesn't work
    • Priority: P2
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
  4. Issue #3407:

    • Title: bug: Remote model should NOT be visible in system monitor
    • Priority: P2
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
  5. Issue #3406:

    • Title: feat: Jan app use codesign from Homebrew LTD
    • Priority: P3
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
  6. Issue #3405:

    • Title: feat: Indicator light for model status (API)
    • Priority: P3
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
  7. Issue #3404:

    • Title: feat: Multiple network profiles for models (API)
    • Priority: P3
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
  8. Issue #3403:

    • Title: feat: Option to regenerate on failure (API)
    • Priority: P3
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
  9. Issue #3396:

    • Title: feat: Adding chunk ids for grounded RAG
    • Priority: P3
    • Status: Open
    • Created: 2 days ago
    • Updated: Edited recently
        1. 10. Additional recent issues include feature requests related to enhancing user experience and improving model management.

Important Observations

  • There is a notable concentration of bugs related to model loading failures and UI inconsistencies, particularly after updates.
  • Users are increasingly requesting features that would enhance usability, such as improved model management tools and clearer error messaging.
  • The transition to the Cortex backend appears to be causing significant disruptions in existing functionalities, leading to user frustration.
  • The presence of many feature requests indicates an engaged community looking to improve the software's capabilities.

This analysis underscores the importance of addressing both critical bugs and user-requested features to stabilize the platform and enhance user satisfaction moving forward.

Report On: Fetch pull requests



Overview

The dataset contains a comprehensive list of pull requests (PRs) from the Jan repository, showcasing a mix of feature additions, bug fixes, and UI improvements. The PRs indicate active development and community engagement, with notable contributions aimed at enhancing functionality and user experience.

Summary of Pull Requests

Open Pull Requests

  1. PR #3414: chore: small ui improvement

    • State: Open
    • Significance: Minor UI adjustments including removal of queued message indicators and color model list enhancements.
    • Notable: Created by Faisal Amir, indicating ongoing attention to UI details.
  2. PR #3413: feat: add blank state component for hub and my model search

    • State: Open
    • Significance: Introduces a blank state component to enhance user experience when no search results are found.
    • Notable: Visual aids included to illustrate changes.
  3. PR #3412: fix: avoid show remote model on system monitor

    • State: Open
    • Significance: Fixes an issue where remote models were incorrectly displayed in the system monitor.
    • Notable: Review comments suggest active collaboration among team members.
  4. PR #3402: fix: save state toggle input box setting

    • State: Open
    • Significance: Enhances user settings by saving the state of toggle inputs in chat features.
    • Notable: Involves UX updates that improve usability.
  5. PR #3354: feat: add tests for migrations

    • State: Open (Draft)
    • Significance: Focuses on adding tests for migration processes, crucial for maintaining code integrity during updates.
    • Notable: Indicates a proactive approach to testing within the development cycle.
  6. PR #3122: feat: Integrate Devbox shell for virtualized local development

    • State: Open
    • Significance: Proposes integration of Devbox shell to streamline local development environments.
    • Notable: Highlights an effort towards improving developer experience.

Closed Pull Requests

  1. PR #3415: fix: disable remote model when user did not setup apiKey

    • State: Closed (Merged)
    • Significance: Prevents users from accessing remote models without proper API key setup, enhancing security.
  2. PR #3411: chore: remove model selection in assistant tab model

    • State: Closed (Merged)
    • Significance: Simplifies the UI by removing redundant model selection options based on team discussions.
  3. PR #3401: feat: enable copy instruction

    • State: Closed (Merged)
    • Significance: Adds functionality to toggle copy instructions in new threads, improving user guidance.
  4. PR #3400: Chore/simple template report

    • State: Closed (Merged)
    • Significance: Introduces a simple template for reporting issues, streamlining the process for users.
  5. PR #3399: chore: require yarn 4.3.1

    • State: Closed (Draft)
    • Significance: Updates project dependencies to require the latest Yarn version, though this was deprioritized due to build issues.
  6. PR #3395: chore: clean up console modal component

    • State: Closed (Merged)
    • Significance: Refines the console modal component for better performance and usability.

Analysis of Pull Requests

The analysis of the pull requests reveals several key themes and trends in the ongoing development of the Jan project:

Active Development and Collaboration

The sheer volume of open and closed pull requests indicates a highly active development environment within the Jan project. The recent PRs focus on both minor UI improvements and significant feature enhancements, suggesting that developers are responsive to user feedback while also prioritizing stability and performance.

Emphasis on User Experience

A notable number of PRs are dedicated to enhancing user experience (UX). For instance, PRs like #3413 and #3401 introduce new components and functionalities aimed at making interactions more intuitive for users. This focus on UX is critical for a project that aims to provide an offline AI solution, as it directly impacts user satisfaction and engagement.

Continuous Improvement through Testing

The introduction of testing-focused PRs such as #3354 highlights a commitment to maintaining code quality through rigorous testing practices. This is particularly important given the project's complexity and its reliance on various hardware configurations and backends.

Community Engagement

The presence of review comments and discussions within PRs illustrates a collaborative culture among contributors. This engagement is vital for fostering a sense of community around the project, which can lead to more robust contributions and quicker resolution of issues.

Technical Challenges

Several PRs address technical challenges faced during development, such as fixing bugs related to API paths (#3316) or enhancing migration processes (#3354). These challenges reflect the complexities involved in developing an offline AI tool that supports multiple hardware configurations.

Future Directions

The ongoing discussions about integrating tools like Devbox (#3122) suggest that the team is not only focused on current features but is also considering future enhancements that could improve developer workflows and ease contributions from external developers.

In conclusion, the pull requests from the Jan repository reflect a dynamic project with active contributions aimed at enhancing both functionality and user experience while addressing technical challenges head-on. The collaborative spirit evident in the discussions around these PRs bodes well for the project's future growth and stability as it continues to evolve into a robust self-hosted AI solution.

Report On: Fetch commits



Repo Commits Analysis

Development Team and Recent Activity

Team Members and Recent Activities

  1. Faisal Amir (urmauur)

    • Recent Commits:
    • Fixed remote model settings based on API key setup.
    • Removed model selection from the assistant tab.
    • Enabled copy instruction feature.
    • Cleaned up console modal component.
    • Minor UI improvements across various components.
    • Collaborations: Worked closely with Louis on several features and fixes, including UI adjustments and bug resolutions.
  2. Louis (louis-jan)

    • Recent Commits:
    • Addressed issues related to error messages when retrieving settings.
    • Improved performance by cleaning up console logs.
    • Merged multiple pull requests, including adding a model assistant tab and fixing LaTex rendering issues.
    • Collaborations: Frequently collaborated with Faisal on UI improvements and bug fixes. Engaged in merging contributions from other team members.
  3. Van Pham (Van-QA)

    • Recent Commits:
    • Added a simple bug report template to the project.
    • Participated in minor UI updates and documentation improvements.
    • Collaborations: Collaborated with Louis on various tasks, including bug reporting templates.
  4. NamH (namchuai)

    • Recent Commits:
    • Worked on multiple bug fixes, including issues with model visibility and engine initialization.
    • Contributed to migration-related features and enhancements.
    • Collaborations: Engaged with Faisal and Louis on bug fixes and feature implementations.
  5. hiento09

    • Recent Commits:
    • Migrated workflows from Cloudflare R2 to AWS S3 for nightly builds.
    • Collaborations: Limited collaboration noted; primarily focused on infrastructure changes.
  6. Gabrielle Ong (gabrielle-ong)

    • Recent Commits:
    • Minor copy edits across various files.
    • Collaborations: Minimal activity; focused on documentation improvements.
  7. Mark Nguyen (marknguyen1302)

    • Recent Commits:
    • Added tests for migration processes and isolated end-to-end tests.
    • Collaborations: Worked independently but contributed significantly to testing efforts.

Patterns, Themes, and Conclusions

  • The development team exhibits a strong collaborative spirit, particularly between Faisal Amir and Louis, who frequently work together on UI enhancements and bug fixes.
  • Recent activities indicate a focus on improving user experience through UI adjustments, fixing bugs related to model handling, and enhancing the overall functionality of the application.
  • The introduction of new features like the copy instruction functionality and model assistant tab reflects ongoing efforts to expand the capabilities of the Jan application.
  • The team is actively addressing technical debt through cleanups and optimizations, as seen in the removal of unused code and enhancements to existing components.
  • The presence of multiple branches indicates a structured approach to feature development, with ongoing work in parallel tracks such as testing, migration, and infrastructure improvements.

Overall, the development team is engaged in a dynamic workflow characterized by collaboration, responsiveness to user needs, and a commitment to continuous improvement of the Jan application.