‹ Reports
The Dispatch

OSS Report: sourcegraph/cody


Cody Project Faces User Frustration Over Pro Plan Limitations Amidst Active Development

Cody, an AI coding assistant by Sourcegraph, is designed to enhance developer productivity through advanced search and codebase context integration within IDEs like Visual Studio Code. It offers features such as semantic search, autocomplete suggestions, and unit test generation.

The project has seen significant activity with 180 open issues, many of which highlight user frustrations regarding the limitations of the Pro plan, particularly concerning rate limits and functionality discrepancies. Recent issues also focus on bugs affecting user experience, such as context retrieval problems and UI stability issues. The development team is actively addressing these concerns with numerous commits aimed at improving both frontend and backend functionalities.

Recent Activity

Recent issues and pull requests indicate a focus on resolving bugs and enhancing user experience. Key issues include syntax highlighting errors (#5300), discrepancies in commit message features (#5298), and UI problems like the chat window displaying only a gray background (#5296). These issues suggest a need for improved stability and clarity in feature offerings.

Development Team and Recent Activities

  1. Quinn Slack (sqs)
    • 98 commits: Focused on UI enhancements and bug fixes.
  2. Tom Ross (umpox)
    • 33 commits: Enhanced editing functionalities and chat features.
  3. Beyang Liu (beyang)
    • 29 commits: Improved context management and chat functionalities.
  4. Vova Kulikov (vovakulikov)
    • 16 commits: Improved Cody Web initialization and context fetching.
  5. Kevin Chen (chenkc805)
    • 11 commits: Updated model selection logic.
  6. Naman Kumar (thenamankumar)
    • 11 commits: Implemented directory mention support.
  7. Valery Bugakov (valerybugakov)
    • 10 commits: Enhanced autocomplete features.
  8. Mikołaj Kondratek (mkondratek)
    • 7 commits: Fixed context API bugs.
  9. Hitesh Sagtani (hitesh-1997)
    • 7 commits: Backend improvements.
  10. Aditya Kalia (akalia25)
    • 5 commits: Updated telemetry features.

Of Note

The Cody project is actively evolving, with a strong emphasis on addressing user feedback to enhance usability and reliability across various coding tasks within integrated development environments.

Quantified Reports

Quantify Issues



Recent GitHub Issues Activity

Timespan Opened Closed Comments Labeled Milestones
7 Days 13 24 4 0 1
30 Days 49 88 62 1 1
90 Days 159 170 283 7 1
All Time 1651 1471 - - -

Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.

Quantify commits



Quantified Commit Activity Over 30 Days

Developer Avatar Branches PRs Commits Files Changes
Rik 6 10/4/1 59 93 29437
Quinn Slack 8 40/35/1 98 379 27708
Beatrix 12 27/24/1 45 265 8975
Tom Ross 4 22/19/1 33 90 8826
Beyang Liu 6 14/12/1 29 55 4797
Ólafur Páll Geirsson 3 10/8/1 16 60 4159
Valery Bugakov 2 6/6/0 10 65 2257
Vova Kulikov 2 12/12/0 16 60 2067
James McNamara 7 5/4/0 19 58 2000
Piotr Kukiełka 1 4/4/0 6 55 1858
Naman Kumar 2 5/4/0 11 51 1396
Dominic Cooney 2 0/0/0 8 31 1363
Ara (arafatkatze) 4 3/0/2 14 19 905
Keegan Carruthers-Smith 2 2/2/0 9 34 598
None (renovate[bot]) 1 1/0/0 1 1 566
Tim Lucas 2 0/0/0 2 20 488
Hitesh Sagtani 2 10/8/1 7 8 390
Mikołaj Kondratek 4 7/3/3 7 14 367
Aditya Kalia 1 4/4/0 5 14 364
Justin Milner 2 1/0/1 2 11 330
Jean-Hadrien Chabran 2 3/2/0 12 6 278
Daniel Marques 1 0/0/0 7 5 263
Kevin Chen 5 4/1/1 11 9 217
Camden Cheek 1 1/1/0 1 7 182
Jan Hartman 2 2/1/0 8 11 153
Kalan 2 2/2/0 3 2 30
Rafał Gajdulewicz 1 1/1/0 1 3 20
Taras Yemets (taras-yemets) 1 1/0/0 1 1 9
YK 1 1/1/0 1 1 2
M.F.M Fazrin (nirzaf) 0 2/0/1 0 0 0

PRs: created by that dev and opened/merged/closed-unmerged during the period

Detailed Reports

Report On: Fetch issues



Recent Activity Analysis

The Cody project on GitHub has seen a significant uptick in activity, with 180 open issues currently logged. Recent issues predominantly report bugs related to functionality and user experience, particularly concerning the integration of AI models and the handling of context in coding tasks. A noticeable trend is the frustration expressed by users regarding the limitations of the Pro plan, particularly concerning rate limits and functionality that seems to contradict advertised capabilities.

Several issues highlight recurring themes: problems with context retrieval, difficulties in model switching, and challenges with the user interface that disrupt workflow. The community appears engaged yet frustrated, signaling a critical need for improvements in both performance and clarity of features.

Issue Details

Most Recently Created Issues

  1. Issue #5300: bug: Julia has not syntax highlighted

    • Priority: High
    • Status: Open
    • Created: 0 days ago
  2. Issue #5298: bug: Experimental Commit Messages Feature Doesn't Match Actual Code Changes

    • Priority: High
    • Status: Open
    • Created: 0 days ago
  3. Issue #5296: bug: The Cody chat window panel shows nothing but a gray background.

    • Priority: High
    • Status: Open
    • Created: 0 days ago
  4. Issue #5295: bug: Not loading historical chat

    • Priority: High
    • Status: Open
    • Created: 0 days ago
  5. Issue #5274: bug: Cody not able to view or access files/context that I attach in the chat.

    • Priority: High
    • Status: Open
    • Created: 1 day ago

Most Recently Updated Issues

  1. Issue #5227: bug: commands don't work anymore (Edited 3 days ago)

    • Priority: High
    • Status: Open
  2. Issue #5207: bug: Cody panel prevents maximizing terminal panel (Edited 7 days ago)

    • Priority: Medium
    • Status: Open
  3. Issue #5206: bug: 应用重新产生的注释内容,会移除代码,仅展示注释信息 (Edited 7 days ago)

    • Priority: Medium
    • Status: Open
  4. Issue #5204: bug: Unable to login using GitHub (Edited 7 days ago)

    • Priority: High
    • Status: Open
  5. Issue #5200: feedback: Ability to see history for edit code prompts (Edited 1 day ago)

    • Priority: Medium
    • Status: Open

Analysis of Notable Issues

  • A significant number of recent issues relate to bugs in the user interface and functionality, particularly around how context is managed when interacting with code files. Users frequently report that Cody fails to recognize or utilize provided context effectively, leading to erroneous suggestions or incomplete responses.

  • The issue regarding commit messages (#5298) indicates a potential disconnect between expected and actual functionality, which could undermine user trust in the tool's reliability for version control tasks.

  • The reoccurring problem of the chat window displaying only a gray background (#5296) suggests possible stability issues within the extension that could deter users from engaging with its features.

  • User feedback indicates frustration with rate limits on the Pro plan (#4199), highlighting a need for clearer communication about usage limits and expectations for paid users.

  • The lack of proper syntax highlighting for languages like Julia (#5300) reflects a broader concern about language support within the tool, which may limit its usability for developers working in diverse coding environments.

In summary, while there is active engagement from users reporting issues and providing feedback, there are clear pain points that need addressing to enhance usability and reliability across various coding tasks within Cody.

Report On: Fetch pull requests



Report on Pull Requests

Overview

The Cody project has a total of 49 open pull requests (PRs) with a diverse range of changes aimed at improving functionality, fixing bugs, and enhancing user experience. The recent activity reflects ongoing development efforts, with several PRs addressing specific issues and implementing new features.

Summary of Pull Requests

Recent Pull Requests

  • PR #5299: Chat: Improve Cody Web initialization and view handling
    This PR optimizes the initialization process for the Cody Web interface by reducing duplicated messages and ensuring proper view handling based on authentication status. It addresses a regression that caused the login view to display incorrectly.

  • PR #5292: PLG: simplify model selection
    Simplifies the model selection process by removing the "Balanced" group and refining the available models for users. This change aims to streamline user choices and enhance clarity.

  • PR #5290: Smart Apply: Use system prompt to encourage code blocks being created
    Enhances the smart apply feature by utilizing the system prompt to guide users in creating code blocks, improving overall usability.

  • PR #5289: Edit: Support multiple models on enterprise
    Introduces support for selecting multiple models in enterprise settings, expanding customization options for users.

  • PR #5287: Fix scrollbar for LLM Dropdown in Cody plugin
    Adds a scrollbar to the LLM dropdown in the VS Code plugin, improving accessibility when many options are available.

  • PR #5281: Codegen: add C# support for Agent bindings
    Adds support for generating C# bindings in the code generation process, facilitating integration with Visual Studio.

  • PR #5278: E2Ev2: Baseline Framework
    Introduces a new end-to-end testing framework aimed at improving test reliability and speed, addressing issues with flaky tests in previous iterations.

  • PR #5276: added(telemetry): separate field for IDE name and server identification
    Updates telemetry to better identify IDEs and server interactions, enhancing data collection accuracy.

  • PR #5272: DO NOT MERGE: demo deepseek setup
    A draft PR intended for demonstration purposes related to deepseek setup.

  • PR #5264: reverting back to accuracy instead of power to not break sg/sg backend
    Reverts changes made in a previous PR to restore functionality related to model accuracy.

Older Pull Requests

Several older PRs focus on refining existing features, optimizing performance, and cleaning up code. Notably:

  • PR #5026 (E2Ev2 migration) and PR #5141 (reindex embeddings if stale) indicate ongoing efforts to enhance testing frameworks and maintain data integrity.
  • The presence of numerous drafts (e.g., PRs #5233, #5234) suggests active experimentation and feature development that may soon be ready for review.
  • The project has seen significant engagement with over 3,300 commits, indicating robust development activity.

Analysis of Pull Requests

The current set of open pull requests reflects several key themes:

  1. User Experience Improvements: Many recent PRs focus on enhancing user interaction with the Cody interface. For instance, PRs like #5299 and #5287 address UI responsiveness and clarity, which are crucial for maintaining user engagement. The emphasis on fixing regressions (e.g., improper view handling) indicates a commitment to providing a stable user experience.

  2. Feature Expansion: There is a clear trend towards expanding features, particularly in terms of model selection (#5292) and multi-model support (#5289). This aligns with the project's goal of offering diverse AI capabilities tailored to different user needs.

  3. Testing and Reliability Enhancements: The introduction of new testing frameworks (#5278) alongside improvements in telemetry (#5276) shows a proactive approach towards maintaining software quality. The focus on reducing flakiness in tests indicates an understanding of the importance of reliable software delivery.

  4. Community Engagement: The high number of open issues and pull requests suggests an active community contributing feedback and requesting features. This is further supported by discussions around PRs where contributors engage in meaningful dialogue about implementation details and potential improvements.

  5. Technical Debt Management: Several PRs aim to clean up code or remove unused features (e.g., PRs #5246, #5268). This is essential for maintaining code quality as the project scales, ensuring that it remains manageable and efficient over time.

  6. Diversity of Contributions: The variety of contributors working on different aspects—from UI enhancements to backend improvements—demonstrates a collaborative environment that encourages contributions across various skill sets.

In conclusion, the Cody project is actively evolving with a strong focus on user experience, feature expansion, testing reliability, community engagement, technical debt management, and diverse contributions. These factors collectively contribute to its potential success as an AI coding assistant within integrated development environments.

Report On: Fetch commits



Repo Commits Analysis

Development Team and Recent Activity

Team Members and Recent Activities

  1. Vova Kulikov (vovakulikov)

    • Recent Commits: 16 commits
    • Key Contributions:
    • Improved Cody Web initialization and view handling to reduce duplicated messages.
    • Fixed context fetching issues in Cody Web.
    • Enhanced the functionality of the chat sidebar.
    • Collaborations: Worked closely with other team members on UI improvements and context management.
  2. Mikołaj Kondratek (mkondratek)

    • Recent Commits: 7 commits
    • Key Contributions:
    • Focused on fixing bugs related to the Cody context API and improving file mention handling.
    • Addressed issues with copying code in the webview.
    • Collaborations: Engaged with the team on fixing context-related bugs.
  3. Naman Kumar (thenamankumar)

    • Recent Commits: 11 commits
    • Key Contributions:
    • Implemented directory mention support in Cody, enhancing user experience for navigating codebases.
    • Worked on fixing the context fetching logic.
    • Collaborations: Collaborated with Vova on context-related features.
  4. Hitesh Sagtani (hitesh-1997)

    • Recent Commits: 7 commits
    • Key Contributions:
    • Focused on backend improvements, including telemetry updates and model configurations.
    • Collaborations: Worked with the team on integrating new models into the system.
  5. Valery Bugakov (valerybugakov)

    • Recent Commits: 10 commits
    • Key Contributions:
    • Enhanced autocomplete features and improved analytics for completion events.
    • Worked on integrating new models into the autocomplete system.
    • Collaborations: Coordinated with others to ensure smooth integration of features.
  6. Aditya Kalia (akalia25)

    • Recent Commits: 5 commits
    • Key Contributions:
    • Updated telemetry features and ensured proper logging for user interactions.
    • Collaborations: Collaborated with the team to refine telemetry data collection.
  7. Quinn Slack (sqs)

    • Recent Commits: 98 commits
    • Key Contributions:
    • Major contributions to UI enhancements, including tab management in Cody Web and various bug fixes.
    • Worked extensively on improving user experience across different IDEs.
    • Collaborations: Actively collaborated with multiple team members on UI/UX improvements.
  8. Tom Ross (umpox)

    • Recent Commits: 33 commits
    • Key Contributions:
    • Focused on editing functionalities, including smart apply features and inline editing improvements.
    • Worked on enhancing chat functionalities within the application.
    • Collaborations: Collaborated with Vova and others on chat-related features.
  9. Kevin Chen (chenkc805)

    • Recent Commits: 11 commits
    • Key Contributions:
    • Made updates to model selection logic and fixed various bugs related to model usage in different contexts.
    • Collaborations: Worked with Valery on autocomplete improvements.
  10. Beyang Liu (beyang)

    • Recent Commits: 29 commits
    • Key Contributions:
    • Contributed to context management and improved chat functionalities, ensuring better integration of context items.
    • Collaborations: Engaged with multiple team members for context-related enhancements.

Patterns, Themes, and Conclusions

  • The development team is actively collaborating across multiple branches, focusing on both frontend UI enhancements and backend functionality improvements.
  • There is a strong emphasis on improving user experience in Cody Web, particularly around chat functionalities, autocomplete features, and context management.
  • The recent activities indicate a robust response to user feedback, as evidenced by numerous bug fixes and feature enhancements aimed at improving usability across different IDEs.
  • The high number of commits from key contributors like Quinn Slack suggests a concentrated effort towards refining the product's interface and functionality, which aligns with ongoing community engagement efforts reflected in open issues and pull requests.
  • Overall, the team's recent activities demonstrate a commitment to iterative improvement, responsiveness to user needs, and a collaborative approach to software development within the Cody project.