‹ Reports
The Dispatch

GitHub Repo Analysis: DS4SD/docling

Executive Summary

Docling, developed by DS4SD, is a Python-based tool for converting various document formats into HTML, Markdown, and JSON. It excels in PDF parsing and integrates with AI frameworks like LlamaIndex and LangChain. The project is popular with over 15,000 GitHub stars and is actively maintained. It is currently focused on expanding functionality and improving OCR capabilities.

Recent Activity

Recent Commits and PRs (Reverse Chronological)

  1. #618: Test improvement using temporary directories for CLI file generation.
  2. #606: Backend creation for USPTO patent parsing.
  3. #557: PubMed XML transformation support.
  4. #530: Layout processing updates for forms and key-value areas.
  5. #495: HTML parsing fix to skip NavigableString.
  6. #474: Addition of PPTX notes slide extraction.

Recent activities show a strong focus on enhancing document parsing capabilities and addressing user-reported issues promptly.

Risks

Of Note

Quantified Reports

Quantify issues



Recent GitHub Issues Activity

Timespan Opened Closed Comments Labeled Milestones
7 Days 44 24 61 0 1
30 Days 136 81 265 2 1
90 Days 245 146 614 36 1
All Time 260 157 - - -

Like all software activity quantification, these numbers are imperfect but sometimes useful. Comments, Labels, and Milestones refer to those issues opened in the timespan in question.

Rate pull requests



3/5
The pull request introduces a significant update to the HTML parser, adding functionality to detect and handle 'h1' elements, as well as various other improvements like handling SVGs and improving list item processing. However, it is still in draft form, lacks documentation updates, and has unresolved checklist items, indicating it's incomplete. The changes are moderately significant but not yet polished or finalized, warranting an average rating.
[+] Read More
3/5
This pull request introduces a feature to use vision models for annotating pictures in documents, which is a potentially useful addition. However, it is still in draft status and lacks completion in several areas such as documentation, tests, and commit message formatting. The changes are significant with multiple new files and modifications, but the PR is not yet ready for final review or merging. It resolves an existing issue but requires further work to meet quality standards.
[+] Read More
3/5
The pull request introduces a new notebook for using Weaviate with Docling for RAG workflows, which is a useful addition to the documentation. However, it has several issues that prevent it from being rated higher. The PR includes some technical errors, such as a CI build failure due to `metadata.widgets`, and strict version pinning that could be relaxed. Additionally, there are concerns about the reliance on OpenAI's API without alternatives, which could limit accessibility. While these issues have been addressed by the author, the PR remains average due to its limited impact and the need for further improvements.
[+] Read More
3/5
The pull request introduces a feature to extract presenter notes from PowerPoint slides, which is a useful addition. However, it lacks updated documentation and examples, which are crucial for understanding and utilizing the new feature effectively. The code changes are moderate in size and complexity, with tests added, but the PR is still pending review approval, indicating potential unresolved issues. Overall, it represents an average contribution that could be improved with more comprehensive documentation and peer reviews.
[+] Read More
3/5
The pull request addresses a specific issue by skipping NavigableString elements during HTML parsing, which is a necessary fix to avoid errors. The change is minor, involving only a few lines of code, and does not introduce new features or significant improvements. While it resolves an issue, the PR lacks documentation updates, examples, and tests, which are marked as necessary in the checklist but not provided. The contribution follows conventional commit guidelines and has been updated per reviewer feedback. Overall, it's a straightforward but unremarkable fix.
[+] Read More
3/5
The pull request introduces a new backend for transforming PubMed XML files to DoclingDocument, which is a significant feature addition. It includes end-to-end tests and updates documentation, showing thoroughness. However, it has notable limitations such as missing sections compared to original PDFs and issues with table conversion. The PR also partially resolves an issue, indicating it's not fully complete. Additionally, there are several review comments pointing out areas for improvement, such as licensing concerns and code structure suggestions. These factors make it an average PR with room for improvement.
[+] Read More
4/5
The pull request introduces significant improvements to the layout processing capabilities of the Docling project. It adds support for hierarchical layout components, enhances the processing of forms and key-value areas, and includes various bug fixes and performance optimizations. The changes are substantial, involving a large number of files and lines of code, indicating a thorough and comprehensive update. However, the PR lacks updated documentation and examples, which are crucial for understanding and utilizing the new features effectively. Additionally, it requires two reviewers for test updates, which is not yet fulfilled. These aspects prevent it from achieving an exemplary rating.
[+] Read More
4/5
This pull request introduces a significant feature by adding a backend to parse USPTO patents, which is a valuable addition to the project. The implementation appears thorough, with substantial code changes and additions, including tests and documentation updates. However, there are some limitations noted that will need to be addressed in future PRs, such as refactoring certain functions and adding more documentation. These limitations prevent it from being exemplary, but the PR is quite good overall.
[+] Read More
4/5
This pull request refactors a test to use a temporary directory for file generation, enhancing test isolation and cleanliness. The change is well-implemented with minimal lines of code, improving the test's robustness by ensuring no leftover files in the source directory. The PR includes necessary documentation and tests, adhering to conventional commit standards. While not a significant feature addition, it represents a good practice in test management and code quality.
[+] Read More

Quantify commits



Quantified Commit Activity Over 14 Days

Developer Avatar Branches PRs Commits Files Changes
**** 1 0/0/0 1 103 131446
Cesar Berrospi Ramis (ceberam) 2 2/0/0 6 35 54515
Christoph Auer 3 8/7/2 40 76 13736
Michele Dolfi 2 6/7/0 7 7 1799
Panos Vagenas 2 3/5/0 6 30 1388
Nikos Livathinos 3 4/4/0 12 38 1040
Peter W. J. Staar 1 1/1/0 1 7 639
github-actions[bot] 2 0/0/0 9 2 128
Maxim Lysak 1 3/3/0 3 1 70
Abhishek Kumar 2 1/1/1 2 3 62
Gaspard Petit 1 1/1/1 1 1 16
Aini 2 2/1/1 2 2 4
guglie 1 0/1/0 1 1 3
Sander Maijers 1 1/1/0 1 1 1
Ben Rood (bash99) 0 1/0/1 0 0 0
Simonas Jakubonis (simjak) 0 1/0/1 0 0 0
Lucas Morin (lucas-morin) 0 2/0/1 0 0 0

PRs: created by that dev and opened/merged/closed-unmerged during the period

Quantify risks



Project Risk Ratings

Risk Level (1-5) Rationale
Delivery 4 The project faces significant delivery risks due to a backlog of unresolved issues and delays in pull request reviews. The net increase in unresolved issues over the past 90 days indicates potential resource constraints or prioritization challenges. Additionally, several pull requests remain open for extended periods, suggesting bottlenecks in the review process. The lack of thorough documentation and testing updates further exacerbates these risks, as seen in PRs like #240 and #259, which are incomplete and lack necessary tests.
Velocity 4 The project's velocity is at risk due to the slow rate of issue resolution compared to the rate of new issues being opened. The backlog of unresolved issues is growing, indicating that the team's capacity may be insufficient to keep up with demand. Additionally, draft pull requests remaining open for long periods suggest delays in development progress. The concentration of changes among a few developers also poses a risk if these key contributors become unavailable.
Dependency 3 The project relies on a wide range of external libraries, as indicated by the poetry.lock file. While this provides flexibility, it also increases the risk of compatibility issues or failures if these dependencies are not regularly updated. Specific dependency issues, such as those with 'torchvision' on Python 3.13 (#596), highlight potential risks that could impact delivery timelines.
Team 3 The team faces risks related to coordination and communication, as evidenced by unmet review requirements for several pull requests (#606, #557, #530). This suggests possible bottlenecks in the review process and potential challenges in team collaboration. The reliance on key individuals for significant code contributions also poses a risk if these contributors become unavailable.
Code Quality 4 The code quality is at risk due to the high volume of changes being made by a few developers without sufficient peer review or documentation updates. Pull requests like #240 and #259 lack thorough documentation and tests, which could lead to maintainability issues and technical debt accumulation. The presence of parsing errors and unexpected behavior in critical functionalities further underscores these risks.
Technical Debt 4 The project is accumulating technical debt due to incomplete documentation, insufficient testing, and unresolved checklist items across multiple pull requests. The backlog of unresolved issues also contributes to this risk by indicating potential prioritization challenges or resource constraints that prevent timely resolution.
Test Coverage 4 Test coverage is insufficient to catch bugs and regressions effectively. Many pull requests lack comprehensive tests, such as PR#259 and PR#495, which could lead to undetected issues in production environments. The absence of detailed documentation further complicates efforts to ensure robust test coverage.
Error Handling 4 Error handling is inadequate across several areas of the project. Issues such as parsing errors (#351, #435) and dependency resolution problems (#596) highlight gaps in error handling mechanisms. The lack of comprehensive error handling in examples like 'rag_haystack.ipynb' further underscores this risk.

Detailed Reports

Report On: Fetch issues



Recent Activity Analysis

Recent GitHub issue activity for the Docling project has been robust, with a significant number of issues being opened and closed in the past few days. The issues range from bug reports and feature requests to questions about usage and enhancements. Notably, there have been several issues related to PDF parsing, OCR functionality, and integration with other tools like LlamaIndex and LangChain.

Several issues highlight anomalies or complications, such as:

  • Issue #607: A PermissionError when trying to convert documents from a directory, indicating potential file access or permission issues.
  • Issue #596: A dependency resolution problem with torchvision on Python 3.13, suggesting compatibility challenges with newer Python versions.
  • Issue #567: A missing model file in the latest version, which could disrupt users relying on specific model versions.
  • Issue #564: Memory issues when processing PDFs with images on Kubernetes, pointing to resource allocation concerns in containerized environments.

Common themes include:

  • OCR Challenges: Multiple issues relate to OCR performance and language support (e.g., Issues #426, #253).
  • PDF Parsing: Several reports of parsing errors or unexpected behavior with specific PDF files (e.g., Issues #351, #435).
  • Integration Requests: Users are interested in integrating Docling with other tools and frameworks (e.g., Issues #453, #465).

Issue Details

Most Recently Created Issues

  1. #607: PermissionError when converting documents from a directory.

    • Priority: Not specified
    • Status: Open
    • Created: 1 day ago

  2. #602: Enhancement request for EasyOCR to use the recog_network parameter.

    • Priority: Not specified
    • Status: Open
    • Created: 1 day ago

  3. #596: Dependency resolution issue with torchvision on Python 3.13.

    • Priority: Not specified
    • Status: Open
    • Created: 4 days ago

Most Recently Updated Issues

  1. #607: PermissionError when converting documents from a directory.

    • Priority: Not specified
    • Status: Open
    • Updated: 1 day ago

  2. #602: Enhancement request for EasyOCR to use the recog_network parameter.

    • Priority: Not specified
    • Status: Open
    • Updated: 1 day ago

  3. #596: Dependency resolution issue with torchvision on Python 3.13.

    • Priority: Not specified
    • Status: Open
    • Updated: 3 days ago

The recent activity indicates active engagement from both users and maintainers in addressing issues and enhancing the tool's functionality. The focus on OCR improvements and integration capabilities suggests ongoing efforts to expand Docling's utility in diverse document processing scenarios.

Report On: Fetch pull requests



Analysis of Pull Requests for DS4SD/docling

Open Pull Requests

  1. #618: test: generate file from CLI in a temporary directory

    • State: Open
    • Created: 0 days ago
    • Summary: This PR refactors a regression test to use a temporary directory for file generation, ensuring cleanup post-test.
    • Notable Points:
    • The PR is recent and addresses a testing improvement.
    • It has passed the conventional commit check.

  2. #606: feat: create a backend to parse USPTO patents into DoclingDocument

    • State: Open
    • Created: 1 day ago
    • Summary: Implements a backend for parsing USPTO patent documents and introduces handling for multiple InputFormat instances with the same MIME type.
    • Notable Points:
    • Requires two reviewers due to test updates, which is currently failing.
    • Introduces significant new functionality and refactoring.

  3. #557: feat: Create a backend to transform PubMed XML files to DoclingDocument

    • State: Open
    • Created: 7 days ago
    • Summary: Adds support for converting PubMed XML files, preserving document hierarchy and adding end-to-end tests.
    • Notable Points:
    • Known limitations include missing sections compared to original PDFs.
    • Requires two reviewers due to test updates, which is currently failing.

  4. #530: feat: Updated Layout processing with forms and key-value areas

    • State: Open
    • Created: 11 days ago
    • Summary: Updates layout processing to handle forms and key-value areas.
    • Notable Points:
    • Requires two reviewers due to test updates, which is currently failing.

  5. #495: fix: Skip NavigableString in HTML parsing

    • State: Open
    • Created: 15 days ago
    • Summary: Skips NavigableString elements during HTML parsing to avoid errors.
    • Notable Points:
    • First contribution by the author; requires sign-off commits.

  6. #474: feat: Add PPTX notes slides

    • State: Open
    • Created: 18 days ago
    • Summary: Extracts presenter notes from PowerPoint presentations.
    • Notable Points:
    • Awaiting integration with another PR for tagging invisible text.

  7. #451: docs: add Weaviate RAG recipe notebook

    • State: Open
    • Created: 20 days ago
    • Summary: Adds a notebook demonstrating RAG workflows using Weaviate and Docling.
    • Notable Points:
    • Issues with metadata causing CI failures.

  8. #259 & #240 (Drafts): feat & dev/update html parser with h1

    • Both are drafts and have been open for over a month, indicating they might be lower priority or in early stages of development.

Closed Pull Requests

  1. #616 (Closed without merge): feat: New layout processing with nested forms and key-value areas

    • Closed on the same day it was created, indicating it might have been superseded by another PR or was not ready for merging.

  2. #615 & #613 (Merged): docs & feat related to Haystack RAG example and EasyOCR parameter addition

    • Successfully merged, indicating enhancements in documentation and OCR capabilities.

  3. #608 (Merged): docs fix for accelerator example path

    • A minor documentation fix that was quickly merged.

  4. Other notable closed PRs include enhancements in AI runtime configuration (#514), handling of unsupported formats (#429), and various bug fixes (#496, #558).

Notable Issues

  • Several open PRs require two reviewers due to test updates but are currently failing this requirement (#606, #557, #530).
  • Some PRs have been closed without merging, which could indicate issues during review or changes in project direction (#616).
  • Draft PRs like #259 and #240 have been open for extended periods, suggesting they might need attention or prioritization.

Conclusion

The DS4SD/docling project is actively maintained with several ongoing developments aimed at enhancing functionality and fixing bugs. The project appears to be focusing on expanding its parsing capabilities (e.g., USPTO patents, PubMed XML) while also improving existing features like layout processing and OCR support. However, some process improvements could be made in terms of managing review requirements and addressing long-standing draft PRs.

Report On: Fetch Files For Assessment



Source Code Assessment

File: docs/examples/rag_haystack.ipynb

Structure and Quality Analysis

  • Notebook Structure: The notebook is well-structured with clear markdown cells explaining the purpose and setup of the example. It includes sections for overview, setup, indexing pipeline, and RAG pipeline.
  • Code Quality: The code is organized into logical blocks with comments explaining key steps. It uses modern Python features and libraries effectively.
  • Dependencies: The notebook relies on several external libraries such as docling-haystack, haystack-ai, and sentence-transformers. These are installed using %pip install which is suitable for Jupyter environments.
  • Parameterization: The use of environment variables and parameters like EXPORT_TYPE allows for flexibility in execution.
  • Error Handling: There is minimal error handling, primarily through warnings. More robust error handling could improve reliability.
  • Output Handling: Outputs are printed in a user-friendly manner, making it easy to interpret results.

File: docling/datamodel/pipeline_options.py

Structure and Quality Analysis

  • Code Organization: The file is well-organized with classes representing different configuration options for the pipeline. Each class has a clear purpose.
  • Use of Enums: Enums are used effectively to define constants like AcceleratorDevice and TableFormerMode, enhancing readability and maintainability.
  • Type Annotations: The code uses type annotations extensively, which improves clarity and helps with static analysis.
  • Pydantic Models: Pydantic is used for data validation, ensuring that configuration options are correctly set. This is a robust choice for managing configurations.
  • Error Handling: There is some error handling, particularly when dealing with environment variables. However, logging could be more comprehensive.

File: docling/models/easyocr_model.py

Structure and Quality Analysis

  • Class Design: The EasyOcrModel class inherits from BaseOcrModel, following good object-oriented design principles.
  • Dependency Management: The class checks for the presence of EasyOCR at runtime, providing a clear error message if it's missing. This is a good practice for optional dependencies.
  • GPU Utilization: The code attempts to use GPU if available, which is crucial for performance in OCR tasks. However, the logic could be simplified or made more explicit.
  • Error Handling: There is limited error handling within the OCR process. Adding try-except blocks around critical operations could improve robustness.
  • Performance Considerations: The use of numpy arrays and batch processing indicates attention to performance.

File: mkdocs.yml

Structure and Quality Analysis

  • Configuration Clarity: The configuration file is well-organized with sections clearly delineating site settings, theme options, navigation structure, markdown extensions, and plugins.
  • Customization: Custom themes and features are specified, indicating a tailored documentation experience.
  • Navigation Structure: The navigation section is detailed, providing a clear structure for the documentation site. This enhances user experience by making content easily accessible.

File: CHANGELOG.md

Structure and Quality Analysis

  • Versioning: The changelog follows semantic versioning conventions, which is essential for tracking changes over time.
  • Detail Level: Entries provide a good level of detail about features, fixes, documentation updates, and breaking changes. This transparency aids users in understanding updates.
  • Format Consistency: The format is consistent throughout the document, making it easy to read and parse.

File: poetry.lock

Structure and Quality Analysis

  • Dependency Management: The lock file contains detailed information about all dependencies, including versions and hashes. This ensures reproducibility of the environment.
  • File Size: At 7611 lines long, the file is quite large but typical for complex projects with many dependencies. Regular updates indicate active maintenance.
  • Security Considerations: Using a lock file helps prevent dependency-related security issues by locking versions.

Overall, the source files demonstrate good coding practices with clear organization, effective use of modern Python features, and attention to detail in configuration management. Improvements could be made in error handling across some files to enhance robustness.

Report On: Fetch commits



Repo Commits Analysis

Development Team and Recent Activity

Team Members and Activities

  • Panos Vagenas (vagenas)

    • Added a new example for Haystack RAG.
    • Contributed to the documentation and code styling.

  • Aini (itsainii)

  • Nikos Livathinos (nikos-livathinos)

    • Fixed path issues in documentation.
    • Introduced support for GPU accelerators, including options for controlling threads and devices.
    • Improved the handling of OCR devices in EasyOCR and RapidOCR models.

  • Christoph Auer (cau-git)

    • Worked on layout processing improvements.
    • Updated test ground-truth data.
    • Made several fixes related to layout postprocessing and table box snapping.
    • Collaborated with Nikos Livathinos on GPU accelerator support.

  • Abhishek Kumar (ab-shrek)

    • Added a timeout limit to document parsing jobs.

  • Michele Dolfi (dolfim-ibm)

    • Made enums serializable with human-readable values.
    • Contributed to various bug fixes and code improvements.

  • Cesar Berrospi Ramis (ceberam)

    • Added USPTO backend parser for patent applications.
    • Refactored XML backend parsers.

Patterns, Themes, and Conclusions

  1. Active Development: The team is actively developing new features, fixing bugs, and improving existing functionalities. This includes significant contributions to both the core functionality and documentation.

  2. Collaboration: There is evidence of collaboration among team members, especially in the development of new features like GPU accelerator support, where multiple contributors are involved.

  3. Focus on Performance: Recent commits indicate a focus on enhancing performance through GPU support and optimizing document parsing processes.

  4. Documentation Updates: Continuous updates to documentation suggest an emphasis on maintaining clarity and usability for end-users.

  5. Testing and Validation: Regular updates to test cases and ground-truth data reflect a commitment to ensuring code reliability and correctness.

  6. Feature Expansion: The addition of new features like the USPTO backend parser indicates ongoing efforts to expand the capabilities of the software to handle more document types and use cases.

Overall, the development team is actively engaged in enhancing the Docling project through feature development, performance improvements, and robust testing practices.