‹ Reports
The Dispatch

OSS Report: sourcegraph/cody


Cody Development Faces Stability Challenges Amid Active Enhancements

Cody, an open-source AI code assistant, is experiencing significant development activity focused on improving functionality and user experience, though stability issues persist, particularly with chat and autocomplete features.

Recent Activity

Recent issues and pull requests (PRs) reveal a focus on bug fixes related to chat interface stability and context awareness. Issues like #5633 and #5626 highlight recurring problems with disappearing chat windows and non-functional autocomplete. This indicates ongoing challenges in maintaining a stable user interface.

Development Team Activities

Of Note

  1. High Volume of Open Issues/PRs: With 249 open issues/PRs, managing contributions effectively remains a challenge.

  2. Branching Strategy: The presence of 594 branches suggests extensive experimentation but may complicate project management.

  3. Community Feedback Integration: Active incorporation of user feedback into development efforts is evident.

  4. Focus on Testing and Reliability: Numerous commits address test reliability and performance optimization.

  5. Interface Stability Concerns: Persistent issues with the chat interface indicate a need for focused stability improvements.

Quantified Reports

Quantify Issues



Recent GitHub Issues Activity

Timespan Opened Closed Comments Labeled Milestones
7 Days 24 4 9 1 1
30 Days 108 37 88 12 1
90 Days 202 148 262 15 1
All Time 1755 1565 - - -

Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.

Quantify commits



Quantified Commit Activity Over 30 Days

Developer Avatar Branches PRs Commits Files Changes
Quinn Slack 7 41/35/2 53 401 26721
Rik 3 2/0/0 23 118 23194
Beatrix 11 27/22/2 58 333 16666
Valery Bugakov 3 16/14/1 22 104 10395
Tom Ross 1 0/0/0 3 28 8004
Vova Kulikov 2 15/14/0 30 112 7997
Kalan 2 4/3/1 4 24 6285
Ólafur Páll Geirsson 5 12/8/2 20 97 3768
Hitesh Sagtani 3 16/13/2 24 57 3035
Beyang Liu 3 6/4/1 13 41 2677
None (renovate[bot]) 2 2/0/0 2 1 1728
Naman Kumar 2 6/5/0 8 46 1321
Ara 5 5/1/1 26 30 984
James McNamara 2 5/4/0 9 42 850
Mikołaj Kondratek 2 10/8/1 13 12 559
Piotr Kukiełka 2 14/13/1 10 21 434
Dan Adler 1 1/1/0 1 38 419
Peter Guy (peterguy) 2 1/0/0 3 5 409
Filip Haftek 1 0/0/0 3 3 340
Rob Rhyne (rrhyne) 1 1/0/1 5 4 317
Jan Hartman 1 1/1/0 2 6 170
Grace Shih 1 0/0/0 2 5 158
Taras Yemets 2 2/1/0 5 5 137
YK 2 4/3/0 4 9 93
Taiyab Raja (taiyab) 1 1/0/0 1 3 63
Daniel Marques 2 1/1/0 2 3 40
Dominic Cooney 2 1/0/0 3 11 29
Aditya Kalia 1 3/2/0 2 2 17
Simon Waterer 1 1/1/0 1 1 2
Martin Amps (martinamps) 0 1/0/1 0 0 0
ShalokShalom (ShalokShalom) 0 1/0/1 0 0 0

PRs: created by that dev and opened/merged/closed-unmerged during the period

Detailed Reports

Report On: Fetch issues



Recent Activity Analysis

Recent activity in the Cody GitHub repository indicates a high volume of issues, with 190 currently open. The predominant theme is bugs related to functionality and integration, particularly concerning the chat interface and autocomplete features. Notably, many users report issues with context awareness, where Cody fails to utilize relevant code files or provides incorrect suggestions. This suggests a potential gap in the model's ability to accurately parse and understand project structures.

Several issues highlight user frustration with the interface, including problems with command execution and unexpected behavior during interactions. The frequency of reports regarding the chat window turning gray or disappearing entirely raises concerns about stability and usability.

Issue Details

Most Recently Created Issues

  1. Issue #5653: bug: nie chce się uruchomić

    • Priority: Bug
    • Status: Open
    • Created: 0 days ago
    • Updated: N/A
  2. Issue #5643: bug:

    • Priority: Bug
    • Status: Open
    • Created: 2 days ago
    • Updated: N/A
  3. Issue #5642: bug:

    • Priority: Bug
    • Status: Open
    • Created: 2 days ago
    • Updated: N/A
  4. Issue #5634: bug: consequent chat with Alt + L doesn't add the selected lines to the context

    • Priority: Bug
    • Status: Open
    • Created: 2 days ago
    • Updated: N/A
  5. Issue #5633: bug: in a few cases in the last 2 days the chat window became black and I had to reload VS code to restart it.

    • Priority: Bug
    • Status: Open
    • Created: 2 days ago
    • Updated: N/A

Most Recently Updated Issues

  1. Issue #5626: VS Code: Autocomplete is not working.

    • Priority: Bug
    • Status: Open
    • Created: 3 days ago
    • Updated: 2 days ago
  2. Issue #5619: bug:

    • Priority: Bug
    • Status: Open
    • Created: 3 days ago
    • Updated: N/A
  3. Issue #5611: bug: Ollama yields tiny context and pure hallucination.

    • Priority: Bug
    • Status: Open
    • Created: 4 days ago
    • Updated: 2 days ago
  4. Issue #5609: bug: Cody UI crashes occasionally when showing output, however results still displayed in the chat history.

    • Priority: Bug
    • Status: Open
    • Created: 4 days ago
    • Updated: N/A
  5. Issue #5606: Cannot Submit Bug Report

    • Priority: Bug
    • Status: Open
    • Created: 4 days ago
    • Updated: N/A

Analysis of Notable Anomalies and Themes

  • A significant number of issues (over 40%) relate to bugs in the chat functionality, including disappearing windows, incorrect context usage, and failures in command execution.
  • Users frequently report that Cody does not recognize or utilize relevant files within their projects, leading to irrelevant suggestions or complete failures in providing assistance.
  • The chat interface's instability is a recurring theme, with multiple users experiencing crashes or gray screens during use.
  • Feedback indicates a strong desire for improvements in usability, particularly regarding how Cody interacts with existing codebases and handles user commands.

This analysis reveals critical areas for improvement in both functionality and user experience within the Cody extension, particularly around context awareness and interface stability.

Report On: Fetch pull requests



Overview

The analysis of the pull requests (PRs) for the Cody project reveals a dynamic and active development environment. The PRs cover a wide range of enhancements, bug fixes, and feature additions that reflect ongoing efforts to improve the functionality, performance, and user experience of Cody as an AI code assistant. Notable areas of focus include integration with various IDEs, support for multiple language models, and enhancements to interactive features like chat functionality and autocomplete suggestions.

Summary of Pull Requests

Recent Merges

  • PR #5652: Introduced single-flighting for ClientConfigSingleton, reducing duplicate requests during state changes.
  • PR #5651: Prevented instantiation of LocalEmbeddingsController for non-dotcom instances, eliminating unnecessary error messages for enterprise users.
  • PR #5649: Made readValuesFrom readonly to prevent misuse in testing code.
  • PR #5648: Used individual imports for lodash to improve tree-shaking and reduce bundle size.
  • PR #5647: Deflaked Cody Context Filters test by addressing timing issues related to background fetches.

Notable Features and Fixes

  • PR #5628: Added a prompt to prevent the Claude 3.5 Sonnet model from constantly apologizing, improving user experience.
  • PR #5624: Fixed cody internal bench and added an evaluation strategy for chat context, enhancing benchmarking capabilities.
  • PR #5623: Generated source maps for the agent CLI, improving stack trace readability for debugging.

Community Engagement

The Cody project actively engages with its community through GitHub issues and pull requests. Recent PRs address community-reported issues such as:

  • PR #5608: Addressed direct routing and prompt caching for PLG users, enhancing performance and reliability.
  • PR #5604: Exposed and tested request parameters for autocomplete providers, ensuring consistent behavior across different configurations.

Anomalies

  • A high number of open issues and PRs (249) suggests active development but also indicates potential challenges in managing contributions effectively.
  • The repository's extensive branching (594 branches) may reflect ongoing experimentation or feature development efforts.

Analysis of Pull Requests

The Cody project's pull requests demonstrate a robust development process characterized by: 1. Continuous Improvement: Regular updates and enhancements are made based on community feedback and internal testing. For instance, PRs addressing UI improvements, such as preventing constant apologies from certain models (#5628), highlight responsiveness to user experience concerns.

  1. Focus on Performance and Reliability: Several PRs aim to enhance performance metrics and reliability. For example, PR #5647 deflaked tests by addressing timing issues, while PR #5608 improved routing efficiency for specific user groups.

  2. Community-Driven Development: The active engagement with the community is evident through PRs that directly address reported issues or suggestions. This collaborative approach not only helps in identifying areas of improvement but also fosters a sense of community ownership over the project.

  3. Experimental Features and Flexibility: The high number of branches and experimental features suggest a willingness to explore new ideas and approaches. This is crucial for innovation but requires careful management to avoid fragmentation or confusion within the codebase.

  4. Technical Challenges and Solutions: The project faces technical challenges such as ensuring consistent behavior across different configurations (#5604) or improving debugging capabilities (#5623). The team's ability to address these challenges effectively is indicative of strong technical expertise.

In conclusion, the Cody project is marked by active development efforts focused on enhancing functionality, performance, and user experience. The team's responsiveness to community feedback, coupled with a willingness to experiment and innovate, positions Cody as a leading tool in integrating AI into software development processes. However, the high number of open issues/PRs and extensive branching also highlight areas where improved management could further streamline development efforts.

Report On: Fetch commits



Recent Activity Analysis of the Development Team

Team Members and Recent Activities

Quinn Slack (sqs)

  • 1 day ago: Implemented single-flighting and unit tests for ClientConfigSingleton (#5652). This reduces duplicate requests during state changes.
  • 1 day ago: Modified LocalEmbeddingsController to prevent instantiation for non-dotcom environments (#5651).
  • 1 day ago: Addressed flakiness in Cody Context Filters test (#5647).
  • 1 day ago: Optimized lodash imports for better tree-shaking (#5648).
  • 1 day ago: Made readValuesFrom readonly to prevent misuse (#5649).
  • 1 day ago: Refactored ModelsService into its own file with interface changes (#5646).
  • 2 days ago: Updated openctx for quieter provider errors (#5644).
  • 2 days ago: Prevented the Claude 3.5 Sonnet model from excessive apologizing (#5628).
  • 2 days ago: Updated access tokens for integration tests (#5629).
  • 2 days ago: Fixed auth issues related to observable emissions and model syncing (#5638).

Beatrix (abeatrix)

  • 2 days ago: Added a prompt to prevent excessive apologies from the Claude model (#5628).
  • 3 days ago: Updated chat telemetry to restore prompt and response text recording (#5553).

Kalan (kalanchan)

  • 2 days ago: Updated agent recordings and tokens for testing purposes (#5629).

Beyang Liu (beyang)

  • 3 days ago: Fixed cody internal bench and added evaluation strategy for chat context (#5624).

Hitesh Sagtani (hitesh-1997)

  • 4 days ago: Implemented direct routing and prompt caching for PLG users (#5608).

Valery Bugakov (valerybugakov)

  • 5 days ago: Enhanced autocomplete request parameters to improve testing coverage (#5604).

Ara (arafatkatze)

  • 5 days ago: Removed artificial latency behind a feature flag to improve user experience (#5480).

Patterns and Themes

  1. Focus on Testing and Reliability:

    • Multiple commits focus on improving test reliability, reducing flakiness, and enhancing test coverage, indicating a strong emphasis on maintaining code quality.
  2. Performance Improvements:

    • Several changes aim at optimizing performance, such as reducing duplicate requests, improving loading times, and enhancing the efficiency of context fetching.
  3. Refactoring for Clarity and Maintainability:

    • A significant number of commits involve refactoring code for better organization, clarity, and maintainability, suggesting an ongoing effort to streamline the codebase.
  4. Collaboration Across Features:

    • Members frequently collaborate on features that span multiple areas of the codebase, such as authentication, context handling, and model management.
  5. Community Engagement and Feedback Integration:

    • The team actively incorporates community feedback into their development process, as evidenced by numerous adjustments based on user interactions and reported issues.

Conclusion

The development team is actively engaged in enhancing the Cody project through focused efforts on testing reliability, performance optimization, code refactoring, and community feedback integration. The recent activities reflect a collaborative environment aimed at delivering high-quality software while addressing both technical debt and user needs.