‹ Reports
The Dispatch

OSS Report: Skyvern-AI/skyvern


Skyvern Faces Challenges with Environment-Specific Errors as Development Progresses

Skyvern, a project designed to automate browser-based workflows using Large Language Models and computer vision, has encountered several environment-specific issues, particularly with Docker and Windows setups, over the past month. This indicates potential architectural or configuration challenges that need addressing to improve user experience.

Recent Activity

Recent issues highlight recurring problems with browser context creation (#733), network errors (#688), and task execution failures (#704). These issues suggest a need for enhanced compatibility and documentation across different platforms. The development team has been actively working on resolving these issues, with Kerem Yilmaz leading efforts in bug fixes and feature enhancements.

Team Members and Recent Activities

  1. Kerem Yilmaz (ykeremy)

    • Fixed email node saving issue (#832).
    • Resolved angular select bug (#831).
    • Supported revival selection (#829).
  2. LawyZheng

    • Refactored custom-select/auto-complete context (#830).
    • Fixed multiple selection history (#813).
  3. Shuchang Zheng (wintonzheng)

    • Made TOTP code fetching more reliable (#809).
    • Improved verification code prompt (#798).
  4. Salih Altun (msalihaltun)

    • Updated text copy to use fallback mechanism (#811).
  5. VittorioYan

    • Added openai environment example and updated README (#764).
  6. Suchintan

    • Updated meeting link in documentation (#780).
  7. Kati3-design

    • Multiple documentation updates.
  8. Mihail Eric (mihail911)

    • Fixed support bot link (#770).

Of Note

Quantified Reports

Quantify Issues



Recent GitHub Issues Activity

Timespan Opened Closed Comments Labeled Milestones
7 Days 0 2 0 0 0
30 Days 6 8 18 6 1
90 Days 23 36 70 11 1
All Time 88 79 - - -

Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.

Quantify commits



Quantified Commit Activity Over 30 Days

Developer Avatar Branches PRs Commits Files Changes
Kerem Yilmaz 1 52/52/0 52 116 9809
LawyZheng 1 31/31/0 31 26 2824
Suchintan 1 1/1/0 10 10 354
Shuchang Zheng 3 22/20/1 22 18 189
kati3-design 1 3/3/0 3 2 168
Salih Altun 1 1/1/0 1 5 69
VittorioYan 1 2/2/0 2 3 33
Mihail Eric 1 2/2/0 2 1 4
Ron Chan (ngalongc) 0 2/0/1 0 0 0

PRs: created by that dev and opened/merged/closed-unmerged during the period

Detailed Reports

Report On: Fetch issues



Recent Activity Analysis

The Skyvern project currently has 9 open issues, indicating ongoing user engagement and potential challenges in the software's functionality. Notably, several issues have been raised regarding errors encountered during task execution, particularly related to browser context creation and network connectivity. A recurring theme is the difficulties users face when running the application in different environments (e.g., Windows with WSL, Docker setups), suggesting a need for clearer documentation or improved compatibility across platforms.

Issue Details

Most Recently Created Issues:

  1. Issue #733: Unknown error while creating browser

    • Priority: High
    • Status: Open
    • Created: 21 days ago
    • Updated: 6 days ago
    • Description: Users report an UnknownErrorWhileCreatingBrowserContext when attempting to create tasks through Docker on Windows. The issue involves permissions and potential missing dependencies within the Docker image.
  2. Issue #704: Skyvern doesn't load past initial page

    • Priority: Medium
    • Status: Open
    • Created: 34 days ago
    • Updated: 3 days ago
    • Description: The application continuously reloads the initial page without progressing, indicating a possible infinite loop or failure in task execution.
  3. Issue #694: Webhook failed

    • Priority: Medium
    • Status: Open
    • Created: 39 days ago
    • Updated: 8 days ago
    • Description: Users experience failures when Skyvern attempts to send webhooks, likely related to browser state cleanup issues.
  4. Issue #688: Network error

    • Priority: Medium
    • Status: Open
    • Created: 40 days ago
    • Updated: 1 day ago
    • Description: Users report a "Network error" when performing actions, with specific connection refusals noted in logs.
  5. Issue #678: Issue with database

    • Priority: Medium
    • Status: Open
    • Created: 41 days ago
    • Updated: 4 days ago
    • Description: Users encounter errors related to database connections during initial runs, suggesting configuration issues.

Notable Trends:

  • Many of the recent issues revolve around environment-specific problems, particularly with Docker and Windows setups.
  • Errors related to network connectivity and browser context creation are prevalent, indicating potential architectural weaknesses or configuration oversights.
  • Stale issues indicate that some problems may not be receiving timely attention or resolution, which could hinder user experience and adoption.

Summary of Important Issues

  • The most critical issue currently is #733, which affects users' ability to create tasks due to an unknown error in browser context creation.
  • Other significant issues include persistent loading problems (#704) and webhook failures (#694), both of which could severely impact usability.
  • The project has a mix of enhancement requests and bug reports, highlighting both ongoing development needs and immediate technical challenges that must be addressed for improved functionality.

Report On: Fetch pull requests



Overview

The analysis of the pull requests (PRs) for the Skyvern project reveals a mix of ongoing feature enhancements, bug fixes, and improvements to existing functionalities. The repository currently has two open PRs and a significant number of closed PRs, indicating active development and maintenance.

Summary of Pull Requests

Open Pull Requests

  • PR #788: improve need_verification_code prompt

    • State: Open
    • Created by: Shuchang Zheng
    • Summary: This PR clarifies the conditions under which a verification code is needed and adds a timeout warning log. It aims to enhance user understanding and debugging capabilities.
  • PR #729: Add endpoint to stream HAR file for tasks

    • State: Open
    • Created by: Ron Chan
    • Summary: Introduces a new endpoint to retrieve HAR files associated with tasks, catering to developers needing raw network traffic data.

Closed Pull Requests

  • PR #832: Fix email node not saved correctly

    • State: Closed
    • Merged by: Salih Altun
    • Summary: Corrects issues with saving email nodes by adding AWS parameters and updating default data.
  • PR #831: fix angular select bug

    • State: Closed
    • Merged by: Shuchang Zheng
    • Summary: Addresses an Angular select bug by modifying DOM utility functions to handle span elements correctly.
  • PR #830: refactor custom-select/auto-complete context

    • State: Closed
    • Merged by: LawyZheng
    • Summary: Refactors context handling for custom select and auto-complete features, improving the overall structure and usability.
  • PR #829: support revival selection

    • State: Closed
    • Merged by: Shuchang Zheng
    • Summary: Enhances dropdown handling with Angular detection and improves DOM listener management.
  • PR #828: Auto update output parameter names after renaming a block

    • State: Closed
    • Merged by: Salih Altun
    • Summary: Automatically updates output parameter names when a block's label changes, enhancing workflow consistency.
  • Additional closed PRs include various bug fixes, feature enhancements, and refactoring efforts aimed at improving the overall functionality and performance of the Skyvern application.

Analysis of Pull Requests

The pull requests in this dataset reflect a robust development cycle characterized by several key themes:

  1. Feature Enhancements: Many PRs focus on adding new features or enhancing existing ones. For instance, PR #729 introduces an endpoint for streaming HAR files, which is crucial for developers needing detailed network traffic analysis. Similarly, PR #788 aims to improve user prompts related to verification codes, indicating an emphasis on user experience and clarity.

  2. Bug Fixes: A significant number of closed PRs address bugs that affect functionality. For example, PR #831 resolves issues with Angular select elements, while PR #832 fixes problems related to email node saving. These fixes are essential for maintaining the integrity of the application as it evolves.

  3. Refactoring and Code Quality: Several PRs focus on refactoring code for better maintainability and readability. PR #830 exemplifies this effort by updating context handling for auto-complete features. Such refactoring is vital in large projects like Skyvern to ensure that the codebase remains manageable as new features are added.

  4. Community Engagement: The presence of multiple contributors indicates an active community around the project. Contributions from various developers suggest that Skyvern is fostering collaboration, which is critical for open-source projects. Comments from peers also highlight a culture of code review and quality assurance.

  5. Stale or Inactive PRs: The existence of open PRs that have not seen recent activity raises concerns about potential bottlenecks in the development process. For instance, PR #788 has been open for over a week without further updates, which could indicate resource constraints or prioritization issues within the team.

  6. Recent Trends: The recent focus on TOTP (Time-based One-Time Password) functionalities across multiple PRs suggests that there is an ongoing effort to enhance security features within the application. This aligns with broader industry trends emphasizing secure authentication methods.

  7. Documentation and User Guidance: Several PRs aim to improve user guidance through better logging and messaging (e.g., PR #820 adds a 'View' button in success messages). This reflects an understanding that user experience extends beyond functionality to include how users interact with feedback from the system.

In conclusion, the Skyvern project demonstrates a proactive approach to development through its active management of pull requests focused on feature enhancement, bug fixing, code quality improvement, and community engagement. However, attention should be given to managing open PRs effectively to prevent stagnation in development progress.

Report On: Fetch commits



Repo Commits Analysis

Development Team and Recent Activity

Team Members and Recent Activities

  1. Kerem Yilmaz (ykeremy)

    • Recent Commits:
    • Fixed email node saving issue (#832).
    • Resolved angular select bug (#831).
    • Supported revival selection (#829).
    • Refactored task node parameter display (#827).
    • Implemented auto-update for output parameter names after renaming a block (#828).
    • Collaboration: Frequently collaborates with Muhammed Salih Altun on various features and fixes.
    • In Progress: Continues to work on enhancing workflow editor functionalities, including node management and parameter handling.
  2. LawyZheng

    • Recent Commits:
    • Refactored custom-select/auto-complete context (#830).
    • Fixed multiple selection history (#813).
    • Improved context for select and auto-complete (#816).
    • Collaboration: Works closely with Kerem Yilmaz on UI components and bug fixes.
    • In Progress: Ongoing improvements to selection components and user interface elements.
  3. Shuchang Zheng (wintonzheng)

    • Recent Commits:
    • Made TOTP code fetching more reliable (#809).
    • Improved verification code prompt (#798).
    • Configured polling timeout for verification codes (#789).
    • Collaboration: Engages in collaborative efforts with Kerem Yilmaz on backend enhancements.
    • In Progress: Focused on improving security features related to verification codes.
  4. Salih Altun (msalihaltun)

    • Recent Commits:
    • Updated text copy to use fallback mechanism (#811).
    • Collaboration: Collaborated with Kerem Yilmaz on several features.
    • In Progress: Minimal recent activity; primarily involved in documentation and minor updates.
  5. VittorioYan

    • Recent Commits:
    • Added openai environment example and updated README (#764).
    • Collaboration: Limited collaboration noted; mainly focused on documentation.
    • In Progress: No significant ongoing work reported.
  6. Suchintan

    • Recent Commits:
    • Updated meeting link in documentation (#780).
    • Various documentation edits.
    • Collaboration: Engaged in documentation improvements but minimal technical contributions.
    • In Progress: Ongoing documentation updates.
  7. Kati3-design

    • Recent Commits:
    • Multiple documentation updates.
    • Collaboration: Primarily focused on documentation without significant technical contributions.
    • In Progress: Continues to refine project documentation.
  8. Mihail Eric (mihail911)

    • Recent Commits:
    • Fixed support bot link (#770).
    • Collaboration: Limited contributions; primarily focused on minor fixes.
    • In Progress: No ongoing work reported.

Patterns, Themes, and Conclusions

  • The majority of recent activity is driven by Kerem Yilmaz, who is heavily involved in both feature development and bug fixing, indicating a central role in the project's progress.
  • LawyZheng's contributions focus on UI/UX improvements, suggesting a strong emphasis on enhancing user interaction within the application.
  • Shuchang Zheng is actively working on security-related features, particularly around verification processes, highlighting a commitment to improving system integrity.
  • Collaboration among team members is evident, particularly between Kerem Yilmaz and Muhammed Salih Altun, indicating a cohesive development environment where members frequently support each other’s work.
  • Documentation efforts are being maintained by Suchintan and Kati3-design, which is crucial for onboarding new contributors and ensuring project clarity.
  • Overall, the team displays a balanced focus on both feature enhancements and bug resolution, with an ongoing commitment to improving the user experience and system reliability.