‹ Reports
The Dispatch

OSS Watchlist: posthog/posthog


GitHub Logo GitHub Logo

Performance Optimization in Replay Events Marks Significant Progress

Recent development efforts have focused on optimizing replay event queries, significantly improving performance and user experience.

Recent Activity

Team Contributions (Reverse Chronological Order)

Paul D'Ambra (pauldambra)

Tomás Farías Santana (tomasfarias)

Julian Bez (webjunkie)

Raquel Smith (raquelmsmith)

Zach Waterfield (zlwaterfield)

Dylan Martin (dmarticus)

Tiina Turban (tiina303)

Bianca Yang (xrdt)

David Newell (daibhin)

Conclusions

The PostHog development team has been actively working on various features and fixes across the platform. Recent activities show a strong focus on enhancing the user interface, improving backend functionalities, and adding new features such as dynamic CNAME targets, improved query performance, and better handling of async queries. The team has also been addressing bugs and optimizing existing functionalities to ensure a robust and user-friendly experience. The ongoing efforts reflect a proactive approach to maintaining and improving the platform's capabilities.

Risks

  1. Performance Issues in Replay Event Queries (#22823):

    • The optimization of replay event queries by using subqueries instead of joins is a significant change aimed at improving performance. However, this introduces the risk of unforeseen issues or bugs that may arise from the new query structure.
  2. Billing Calculations (#22821):

    • Fixes related to billing month-to-date and tier computations are critical for accurate billing. Any errors in these calculations could lead to overcharging or undercharging users, impacting user trust and financial accuracy.
  3. Deployment Issues (#22819):

    • A bug preventing the deployment of the self-hosted hobby version due to Postgres issues is critical. This affects users attempting to deploy self-hosted versions, potentially leading to loss of adoption or dissatisfaction among users.

Of Note

  1. Error Tracking Implementation (#22814):

    • The introduction of basic error tracking functionality is a significant enhancement that will help in monitoring and debugging system errors more effectively.
  2. Async Calculation for Saved Insights/Dashboards (#22823):

    • The addition of async calculation for saved insights and dashboards is a notable improvement that enhances performance and user experience by allowing concurrent processing.
  3. Billing Limits Configuration During Onboarding (#22822):

    • Including billing limits configuration during onboarding is an enhancement aimed at improving user experience by allowing users to set billing limits upfront, ensuring better control over their usage and costs.

Overall, the recent activity indicates a robust development cycle with significant improvements in performance, user experience, and system reliability. The team's proactive approach in addressing bugs and introducing new features reflects a strong commitment to continuous improvement and user satisfaction.

Quantified Commit Activity Over 7 Days

Developer Avatar Branches PRs Commits Files Changes
Ben White 5 6/2/3 56 71 17533
vs. last report +2 -1/-6/+3 +29 -56 +11913
Marius Andra 3 7/6/0 38 159 14116
vs. last report +1 +4/+3/= +30 +62 -23503
timgl 2 6/5/0 8 44 4804
vs. last report -1 -2/-2/= -6 -20 -5675
Tom Owers 2 9/8/1 10 60 2933
vs. last report -1 +4/+5/= +5 +31 +1280
Thomas Obermüller 3 12/8/0 22 67 2630
vs. last report = +5/+2/-1 +6 +51 +629
Michael Matloka 4 8/7/0 11 60 2602
vs. last report +3 +2/-2/= +1 -37 +1964
Eric Duong 6 12/8/1 18 84 2357
vs. last report +3 +10/+7/= +10 +60 +1270
Sandy Spicer 7 8/4/2 27 68 1667
vs. last report +4 +4/=/+2 +12 +29 +883
Zach Waterfield 2 10/10/0 15 74 1599
vs. last report = +7/+5/= +8 +64 +1302
David Newell 2 6/4/2 10 64 1561
vs. last report -1 +2/+2/+2 -3 +23 -10
github-actions 29 0/0/0 198 103 1460
vs. last report +11 =/=/= +75 +40 +488
Julian Bez 3 4/4/1 9 47 1290
vs. last report = -4/-2/+1 -4 -8 +636
Tomás Farías Santana 2 4/4/0 9 21 1062
vs. last report +1 +3/+4/= +8 +20 +1060
Tiina Turban 3 6/3/1 6 20 924
vs. last report -4 -1/+2/= -3 -5 -70
Dylan Martin 2 4/3/0 4 28 877
vs. last report = =/=/= -6 +1 +168
Raquel Smith 4 10/7/0 14 51 818
vs. last report +3 +8/+4/= +11 +20 +101
Nikita Vorobev 1 1/0/0 1 23 716
vs. last report +1 =/=/= +1 +23 +716
Paul D'Ambra 3 16/14/0 17 19 664
vs. last report +1 +7/+6/= +6 -42 -3185
Brett Hoerner 2 3/1/0 3 14 585
vs. last report +1 +2/=/= +2 +10 +521
Bianca Yang 3 7/6/0 11 9 364
vs. last report -1 +2/+1/= +1 -98 -1637
Robbie 3 2/1/0 6 14 274
vs. last report +2 +1/=/= +5 +13 +272
Frank Hamand 2 6/5/0 7 12 205
vs. last report +1 +4/+3/= +5 +9 +175
Neil Kakkar 1 1/1/1 1 8 172
vs. last report = =/+1/+1 -1 +6 -10
Juraj Majerik 1 4/4/0 4 15 113
PostHog Bot 5 8/4/4 8 2 96
vs. last report = +3/+3/-4 +3 = +46
ted kaemming 2 2/1/0 3 3 93
vs. last report = -2/-2/= -6 -12 -1204
Lior539 1 1/1/0 1 6 51
Phani Raj 1 1/1/0 1 5 46
vs. last report = =/+1/= -3 -3 -5
Cory Watilo (corywatilo) 2 2/0/0 2 4 40
feedanal 1 0/0/0 3 2 11
vs. last report +1 -1/=/= +3 +2 +11
Ian Vanagas 1 1/1/0 1 1 2
None (camerondeleone) 1 1/0/0 1 1 1
Edward Hartwell Goose (edhgoose) 0 1/0/0 0 0 0
Shawn Lockhart (shawnlikescode) 0 1/0/0 0 0 0

PRs: created by that dev and opened/merged/closed-unmerged during the period

Detailed Reports

Report On: Fetch commits



Project Overview

The PostHog development team is working on an open-source analytics platform that provides product analytics, session recording, feature flagging, and A/B testing capabilities. It is designed for self-hosting, allowing users to maintain control over their data while leveraging the platform's extensive analytics and optimization tools. The project is hosted on GitHub and continues to see active development with a balanced focus on both front-end improvements and back-end stability.

Recent Development Activities

Team Contributions (Reverse Chronological Order)

Paul D'Ambra (pauldambra)

  • 0 days ago: Fixed heatmap intro layout (frontend/src/toolbar/bar/toolbarLogic.ts, frontend/src/scenes/heatmaps/HeatmapsBrowser.tsx, frontend/src/scenes/heatmaps/heatmapsBrowserLogic.ts).
  • 0 days ago: Merged branch master into fix/undefined-thingies and fixed keyboard navigation in toolbar (frontend/src/toolbar/bar/toolbarLogic.ts).
  • 1 day ago: Fixed safely checking flags in replay API (posthog/session_recordings/session_recording_api.py).
  • 1 day ago: Added iframe protection for in-app heatmap browser (frontend/src/scenes/heatmaps/HeatmapsBrowser.tsx, frontend/src/scenes/heatmaps/heatmapsBrowserLogic.ts).
  • 1 day ago: Considered soft deleted when syncing caching state (posthog/caching/insight_caching_state.py, posthog/caching/test/test_insight_caching_state.py).
  • 1 day ago: Updated cache ages counting (posthog/caching/insight_cache.py, posthog/caching/insight_caching_state.py).

Tomás Farías Santana (tomasfarias)

  • 0 days ago: Implemented timestamp-based backfills for batch exports (posthog/batch_exports/http.py, posthog/batch_exports/models.py, posthog/batch_exports/service.py, posthog/temporal/batch_exports/backfill_batch_export.py, posthog/temporal/batch_exports/batch_exports.py, posthog/temporal/batch_exports/bigquery_batch_export.py, posthog/temporal/batch_exports/http_batch_export.py, posthog/temporal/batch_exports/noop.py, posthog/temporal/batch_exports/postgres_batch_export.py, posthog/temporal/batch_exports/redshift_batch_export.py, posthog/temporal/batch_exports/s3_batch_export.py, posthog/temporal/batch_exports/snowflake_batch_export.py, posthog/temporal/tests/batch_exports/test_backfill_batch_export.py, posthog/temporal/tests/test_encryption_codec.py).
  • 1 day ago: Refactored heartbeat to use heartbeater constantly (posthog/temporal/batch_exports/bigquery_batch_export.py, posthog/temporal/batch_exports/s3_batch_export.py, posthog/temporal/batch_exports/squash_person_overrides.py, posthog/temporal/common/heartbeat.py, posthog/temporal/common/utils.py, posthog/temporal/tests/batch_exports/test_bigquery_batch_export_workflow.py).

Julian Bez (webjunkie)

  • 0 days ago: Added async calculation for saved insights/dashboards (frontend/src/lib/api.ts, frontend/src/queries/query.ts, frontend/src/queries/schema.json, frontend/src/queries/schema.ts, frontend/src/scenes/dashboard/dashboardLogic.tsx, frontend/src/scenes/saved-insights/activityDescriptions.tsx, frontend/src/types.ts, posthog/api/query.py, posthog/api/services/query.py, posthog/api/test/__snapshots__/test_api_docs.ambr, posthog/api/test/dashboards/test_dashboard.py, posthog/api/test/test_insight.py, posthog/api/test/test_query.py, posthog/caching/calculate_results.py, posthog/caching/fetch_from_cache.py, posthog/caching/insight_cache.py, posthog/clickhouse/client/test/test_execute_async.py).
  • 1 day ago: Added concurrent refresh of insights on dashboard (frontend/src/scenes/dashboard/dashboardLogic.tsx).

Raquel Smith (raquelmsmith)

  • 0 days ago: Removed negation operator from check for disabledReason (frontend/src/scenes/settings/organization/OrgEmailPreferences.tsx).
  • 1 day ago: Added disabled reason to new member join setting (frontend/__snapshots__/scenes-app-insights--trends-line-edit--light.png,frontend/src/scenes/settings/organization/OrgEmailPreferences.tsx).

Zach Waterfield (zlwaterfield)

  • 0 days ago: Updated activation endpoint to be activate (ee/api/test/test_billing.py,frontend/__snapshots__/scenes-app-insights--trends-line-edit--light.png,frontend/src/scenes/billing/PlanComparison.tsx,frontend/src/scenes/billing/billing-utils.ts,frontend/src/scenes/billing/billingProductLogic.ts).
  • 1 day ago: Improved billing limits UI (frontend/__snapshots__/scenes-app-insights--trends-line-edit--dark.png,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2--dark.png,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2--light.png,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2-with-discount--dark.png,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2-with-discount--light.png,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2-with-limit-and-100-percent-discount--dark.png,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2-with-limit-and-100-percent-discount--light.png,frontend/src/scenes/billing/BillingLimit.tsx,frontend/src/scenes/billing/BillingLimitInput.tsx,frontend/src/scenes/billing/BillingProduct.tsx,frontend/src/scenes/billing/billingProductLogic.ts).

Dylan Martin (dmarticus)

  • 1 day ago: Fixed broken NPS survey results and added migration (frontend/__snapshots__/scenes-other-billing-v2--billing-v-2--dark.png,frontend/src/scenes/surveys/surveyLogic.tsx,latest_migrations.manifest,plugin-server/src/worker/ingestion/property-definitions-auto-discovery.ts,plugin-server/tests/worker/ingestion/property-definitions-auto-discovery.test.ts,posthog/hogql_queries/query_runner.py,posthog/migrations/0421_set_all_survey_responses_to_be_strings.py,posthog/test/base.py,posthog/test/test_migration_0421.py).

Tiina Turban (tiina303)

  • 1 day ago: Moved set usage tracking earlier (plugin-server/src/utils/event.ts,plugin-server/src/worker/ingestion/event-pipeline/runner.ts).
  • 1 day ago: Tracked usage outside of known events (plugin-server/src/main/ingestion-queues/metrics.ts,plugin-server/src/workers/event-pipeline-runner.js).

Bianca Yang (xrdt)

  • 1 day ago: Moved some print statements in usage_reports to logger (posthog/tasks/report_usage.js).
  • 2 days ago: Fixed /billing page usage calculations (ee/api/test/test_billing.js).

David Newell (daibhin)

  • 3 days ago: Added ability to sort path filter rules (frontend/__snapshots__/components-property-key-info--property-key-info--dark.png).
  • 3 days ago: Fixed properties invalid filters issue.

Conclusions

The PostHog development team has been actively working on various features and fixes across the platform. Recent activities show a strong focus on enhancing the user interface, improving backend functionalities, and adding new features such as dynamic CNAME targets, improved query performance, and better handling of async queries. The team has also been addressing bugs and optimizing existing functionalities to ensure a robust and user-friendly experience. The ongoing efforts reflect a proactive approach to maintaining and improving the platform's capabilities.

Report On: Fetch issues



Analysis of Progress Since Last Report

Overview

Since the last report 7 days ago, there has been significant activity in the repository. Numerous new issues have been created, and several issues have been closed. This indicates ongoing development and maintenance efforts.

Notable New Issues

  1. Issue #22823: Fix replay events should use subquery on person not join

    • Description: Optimization to avoid slow joins in replay events.
    • Significance: Improves performance by eliminating unnecessary joins.
  2. Issue #22822: Including billing limits in onboarding

    • Description: Work in progress to include billing limits configuration during onboarding.
    • Significance: Enhances user experience by allowing billing limits to be set during onboarding.
  3. Issue #22821: Fix billing month-to-date and tiers computations

    • Description: Ensures accurate billing calculations constrained by limits.
    • Significance: Prevents overcharging and ensures transparency in billing.
  4. Issue #22820: Allow SSH tunnel username for SSH key - Data Warehouse

    • Description: Adds a field to specify the username for SSH tunnels.
    • Significance: Enhances security and flexibility for data warehouse connections.
  5. Issue #22819: Unable to deploy self-hosted hobby version due to Postgres issues

    • Description: Bug preventing deployment of self-hosted hobby version.
    • Significance: Critical for users attempting to deploy self-hosted versions.
  6. Issue #22818: Fix typo in Nuxt install instructions

    • Description: Corrects a typo in installation instructions.
    • Significance: Minor but improves documentation accuracy.
  7. Issue #22817: Fix over setting heatmap actions

    • Description: Prevents redundant calls when processing URLs for heatmaps.
    • Significance: Optimizes performance and reduces unnecessary actions.
  8. Issue #22815: Refactor insights to use queries instead of filters for insight activity log

    • Description: Transition from filters to queries for insight activity logs.
    • Significance: Standardizes data handling and potentially improves performance.
  9. Issue #22814: Error tracking (simple start)

    • Description: Initial implementation of error tracking within the system.
    • Significance: Enhances debugging capabilities and system reliability.
  10. Issue #22812: Flag the persons-on-events project setting

    • Description: Flags person properties on events as default for new users.
    • Significance: Streamlines user experience and reduces dependency on complex joins.

Recently Closed Issues

  1. Issue #22816: Fix debug Clickhouse queries on localhost

    • Description: Ensures insights appear in "debug Clickhouse queries".
    • Significance: Improves debugging capabilities for local development.
  2. Issue #22813: Validate join key is a field node in data warehouse

    • Description: Ensures join keys are valid fields.
    • Significance: Prevents potential errors in data warehouse operations.
  3. Issue #22809: Use next_allowed_client_refresh for dashboard reload button

    • Description: Synchronizes dashboard reload with allowed refresh intervals.
    • Significance: Enhances user experience by providing timely data refreshes.
  4. Issue #22805: Allow x-highlight-request tracing header

    • Description: Adds tracing header to CORS allowlist.
    • Significance: Prevents CORS issues related to tracing headers.
  5. Issue #22801: Collapsible setting panel for in-app heatmaps

    • Description: Makes settings panel collapsible and adds borders.
    • Significance: Improves UI/UX for heatmap settings.
  6. Issue #22799: Change refresh modes for dashboard to blocking

    • Description: Reverts dashboard refresh mode to blocking.
    • Significance: Ensures consistency in data display on dashboards.
  7. Issue #22798: Export Hog interfaces

    • Description: Exports necessary interfaces for Hog functions.
    • Significance: Facilitates integration and usage of Hog functions.
  8. Issue #22797: New npm release for HogVM

    • Description: Bumps version of HogVM package.
    • Significance: Ensures latest features and fixes are available.
  9. Issue #22795: Heartbeating for all exports and other fixes

    • * * Description * * : Adds heartbeating to all batch exports and fixes related issues .
      • Significance * * : Improves reliability of batch export processes .

10 . * * Issue  # 22793 * * : Revert " feat ( insights ) : Async calculation for saved insights / dashboards " -    * * Description * * : Reverts async calculation feature due to concurrency concerns . -    * * Significance * * : Maintains system stability while addressing concurrency issues .

General Trends

Backend Improvements

  • Several optimizations and bug fixes have been made, particularly around query handling (# 22823 ,  # 22786 ) .
  • There is a focus on improving error handling , logging , and performance (# 22795 ,  # 22761 ) .

Frontend Enhancements

  • Multiple UI / UX improvements have been introduced , such as collapsible panels (# 22801 ), better loading states (# 22784 ), and enhanced insight displays (# 22752 ) .
  • New features like error tracking (# 22814 ) and improved billing UI (# 22726 ) have been added .

Bug Fixes

  • Numerous bugs have been addressed , including deployment issues (# 22819 ), incorrect funnel aggregations (# 22731 ), and session recording filtering errors (# 22716 ) .

Performance and Usability

  • Efforts to improve performance are evident with changes like optimizing replay events (# 22823 ) and reducing excessive logging that could lead to disk space issues (# 22737 ) .

Conclusion

The recent activity in the repository reflects a robust development cycle focused on both expanding features and maintaining system integrity . The team's responsiveness to issues , both in terms of introducing enhancements and resolving bugs , indicates a strong commitment to user satisfaction and continuous improvement . The introduction of new features alongside critical bug fixes suggests a balanced approach to development , ensuring both innovation and stability .

Overall , the project appears to be progressing well with significant improvements made over the past week .

Report On: Fetch PR 22823 For Assessment



PR #22823

Summary of Changes

This pull request addresses a significant performance issue related to querying replay events by optimizing the query structure. The main change involves replacing joins with subqueries when filtering replay events by person properties. Specifically, the changes include:

  1. Modification in property_to_expr function:

    • Added a new scope "replay_pdi" to handle person properties in replay queries.
  2. Changes in session_recording_list_from_filters.py:

    • Introduced two new classes PersonsPropertiesSubQuery and PersonsIdSubQuery to handle subqueries for person properties and person IDs respectively.
    • Updated the _where_predicates method to use these subqueries instead of joins.
    • Added a method _strip_person_properties to separate person properties from other properties.
  3. Updates in test files:

Code Quality Assessment

Strengths

  • Performance Optimization: The primary goal of this PR is well-addressed. By using subqueries instead of joins, the performance of querying replay events should see a notable improvement.
  • Modular Approach: The introduction of PersonsPropertiesSubQuery and PersonsIdSubQuery classes encapsulates the logic for handling person-related subqueries, making the code more modular and easier to maintain.
  • Comprehensive Testing: The changes are well-supported by updates in the test suite, ensuring that the new query structure is thoroughly tested.

Potential Improvements

  • Code Duplication: There is some duplication between the PersonsPropertiesSubQuery and PersonsIdSubQuery classes. Consider refactoring common logic into a base class or utility functions.
  • Documentation: While the code changes are clear, additional inline comments explaining the rationale behind certain decisions (e.g., why specific optimizations were chosen) would be beneficial for future maintainers.
  • Error Handling: Ensure that any potential errors introduced by the new subquery logic (e.g., empty results, null values) are adequately handled and tested.

Detailed Diff Analysis

Core Changes

# posthog/hogql/property.py
def property_to_expr(
    property: Union[BaseModel, PropertyGroup, Property, dict, list, ast.Expr],
    team: Team,
    scope: Literal["event", "person", "session", "replay", "replay_entity", "replay_pdi"] = "event",
) -> ast.Expr:
    if isinstance(property, dict):
        try:

Subquery Classes

# posthog/session_recordings/queries/session_recording_list_from_filters.py
class PersonsPropertiesSubQuery:
    _team: Team
    _filter: SessionRecordingsFilter
    _ttl_days: int

    def __init__(self, team: Team, filter: SessionRecordingsFilter, ttl_days: int):
        self._team = team
        self._filter = filter
        self._ttl_days = ttl_days

    def get_query(self) -> ast.SelectQuery | ast.SelectUnionQuery | None:
        if self.person_properties:
            return parse_select(
                """
                SELECT distinct_id
                FROM person_distinct_ids
                WHERE {where_predicates}
                """,
                {
                    "where_predicates": self._where_predicates,
                },
            )
        else:
            return None

    @cached_property
    def person_properties(self) -> PropertyGroup | None:
        person_property_groups = [g for g in self._filter.property_groups.flat if g.type == "person" in g.type]
        return (
            PropertyGroup(
                type=PropertyOperatorType.AND,
                values=person_property_groups,
            )
            if person_property_groups
            else None
        )

    @cached_property
    def _where_predicates(self) -> ast.Expr:
        return (
            property_to_expr(self.person_properties, team=self._team, scope="replay_pdi")
            if self.person_properties
            else ast.Constant(value=True)
        )

Test Updates

# posthog/session_recordings/queries/test/test_session_recording_list_from_filters.py
def test_filter_with_person_properties_exact(self):
    session_id_one, session_id_two = self._two_sessions_two_persons(
        "test_filter_with_person_properties_exact",
        session_one_person_properties={"email": "bla@gmail.com"},
        session_two_person_properties={"email": "bla2@hotmail.com"},
    )

    query_results: SessionRecordingQueryResult = self._filter_recordings_by(
        {
            "properties": [
                {
                    "key": "email",
                    "value": ["bla@gmail.com"],
                    "operator": "exact",
                    "type": "person",
                }
            ]
        }
    )

    assert [x["session_id"] for x in query_results.results] == [session_id_one]

Conclusion

Overall, this PR effectively addresses a critical performance issue by optimizing how replay events are queried. The changes are well-structured and supported by comprehensive tests. Minor improvements in code duplication and documentation could further enhance code quality.

Report On: Fetch pull requests



Analysis of Progress Since Last Report

Summary

Since the last analysis 7 days ago, there has been significant activity in the PostHog/posthog repository. Numerous pull requests (PRs) have been opened and closed, with various enhancements, bug fixes, and new features being introduced. Below is a detailed report of the changes:

Open Pull Requests

  1. PR #22823: fix: replay events should use subquery on person not join

    • State: Open
    • Created: 1 day ago
    • Details: This PR addresses performance issues by using subqueries instead of joins for replay events. It includes extensive testing and snapshot updates.
  2. PR #22822: wip: including billing limits in onboarding

    • State: Open
    • Created: 1 day ago
    • Details: This work-in-progress PR aims to include billing limits configuration during onboarding.
  3. PR #22821: fix: Billing month-to-date and tiers computations

    • State: Open
    • Created: 1 day ago
    • Details: Fixes issues with billing computations, ensuring accurate month-to-date values and tier data.
  4. PR #22818: fix: typo in nuxt install instructions

    • State: Open
    • Created: 1 day ago
    • Details: Corrects a typo in the Nuxt.js installation instructions.
  5. PR #22817: fix: over setting heatmap actions

    • State: Open
    • Created: 1 day ago
    • Details: Fixes an issue where heatmap actions were being set unnecessarily, improving performance.
  6. PR #22815: refactor(insights): use queries instead of filters for insight activity log

    • State: Open
    • Created: 1 day ago
    • Details: Refactors insight activity logs to use queries instead of filters.
  7. PR #22814: feat: Error tracking (simple start)

    • State: Open
    • Created: 1 day ago
    • Details: Introduces basic error tracking functionality.
  8. PR #22812: chore(settings): Flag the persons-on-events project setting

    • State: Open
    • Created: 1 day ago
    • Details: Flags the persons-on-events project setting for better management.
  9. PR #22811: chore: Person IDs on events by default

    • State: Open
    • Created: 1 day ago
    • Details: Makes person IDs on events the default setting.
  10. Other notable open PRs include improvements to batch export metrics (#22810), iframe error detection for heatmaps (#22808), support form enhancements (#22807), posthog-js update (#22804), and various other fixes and enhancements.

Notable Closed Pull Requests

  1. PR #22816: fix(product-analytics): Fix debug clickhouse queries on localhost

    • Merged by Robbie (robbie-c).
    • Fixes an issue where most insights did not appear in "debug clickhouse queries".
  2. PR #22813: chore(data-warehouse): Validate join key is a field node

    • Merged by Tom Owers (Gilbert09).
    • Ensures that join keys are valid field nodes in data warehouse views.
  3. Other notable closed PRs include improvements to dashboard reload button (#22809), tracing header allowance (#22805), collapsible settings panel for heatmaps (#22801), and various other fixes and enhancements.

Significant Issues and Resolutions

  • The introduction of replay event optimizations (PR #22823) is expected to significantly improve performance.
  • The resolution of billing computation issues (PR #22821) ensures accurate billing data.
  • The addition of error tracking (PR #22814) introduces new functionalities for monitoring errors.
  • Several performance improvements, such as optimizing heatmap actions (PR #22817) and fixing iframe errors for heatmaps (PR #22808).

Observations

  • There has been a notable focus on refining existing functionalities, improving error handling, and enhancing user experience across various components of PostHog.
  • The project continues to evolve with significant contributions aimed at both expanding capabilities and ensuring reliability.
  • The team is actively addressing bugs and issues reported by users, ensuring a robust platform.

Overall, the recent activity indicates an active development cycle aimed at both expanding capabilities and ensuring the reliability of the platform.

Report On: Fetch Files For Assessment



Source Code Assessment

File: frontend/src/toolbar/bar/toolbarLogic.ts

Structure and Quality Analysis

  1. Imports and Dependencies:

    • The file imports several modules from kea, which is a state management library.
    • It also imports various utility functions and types from other parts of the codebase.
    • The dependencies are well-organized and clearly indicate the modular structure of the application.
  2. Logic Definition:

    • The core logic is encapsulated within kea<toolbarLogicType>, which is a type-safe way to define logic in kea.
    • The use of path, connect, actions, reducers, selectors, and listeners demonstrates a comprehensive use of kea's capabilities.
  3. Actions:

    • Actions are defined for various toolbar functionalities such as toggling themes, minimizing, setting drag positions, and handling mouse events.
    • The actions are well-named and their purposes are clear.
  4. Reducers:

    • Reducers manage the state changes in response to actions.
    • The state properties like element, menu, visibleMenu, minimized, etc., are well-defined.
    • Persistence is used for some state properties, ensuring they are retained across sessions.
  5. Selectors:

    • Selectors compute derived state based on the current state.
    • They are used to calculate properties like dragPosition and menuProperties.
  6. Listeners:

    • Listeners handle side effects in response to actions.
    • They manage interactions such as enabling/disabling heatmaps, handling mouse movements, and synchronizing with external elements (like the hedgehog actor).
  7. Lifecycle Methods:

    • afterMount and beforeUnmount are used to set up and clean up event listeners, ensuring proper resource management.
  8. Code Quality:

    • The code is well-structured with clear separation of concerns.
    • Comments are sparse but the code is self-explanatory due to good naming conventions.
    • TypeScript types are used effectively, enhancing type safety and readability.

File: ee/session_recordings/persistence_tasks.py

Structure and Quality Analysis

  1. Imports and Dependencies:

    • Imports necessary modules for task scheduling (celery), logging (structlog), time manipulation (datetime), and database interactions (django).
  2. Task Definitions:

    • Two tasks are defined using Celery decorators: persist_single_recording and persist_finished_recordings.
    • These tasks handle the persistence of session recordings.
  3. Logging and Metrics:

    • Structured logging is set up using structlog.
    • Prometheus metrics are used to count session recordings that need persistence.
  4. Task Logic:

    • persist_single_recording: Calls a function to persist a single recording based on its ID and team ID.
    • persist_finished_recordings: Identifies recordings older than a day that need persistence and schedules them for persistence.
  5. Code Quality:

    • The code is concise and focused on its purpose.
    • Use of constants (one_day_old) improves readability.
    • Logging provides useful information for monitoring task execution.

File: posthog/batch_exports/http.py

Structure and Quality Analysis

  1. Imports and Dependencies:

    • Extensive imports covering Django REST framework components, database transactions, logging, date-time utilities, serializers, viewsets, etc.
  2. Helper Functions:

    • Functions like validate_date_input provide utility for date validation.
  3. Serializers:

    • Several serializers (BatchExportRunSerializer, BatchExportDestinationSerializer, etc.) define how models are serialized/deserialized for API interactions.
  4. ViewSets:

    • ViewSets like BatchExportRunViewSet, BatchExportViewSet, etc., define the API endpoints for batch export operations.
  5. Custom Actions:

    • Custom actions (e.g., backfill, pause, unpause) add specific functionalities to the ViewSets.
  6. Code Quality:

    • The code is modular with clear separation between different components (serializers, viewsets).
    • Comprehensive use of Django REST framework features enhances maintainability.

File: frontend/src/lib/api.ts

Structure and Quality Analysis

  1. File Too Long for Detailed Analysis:
    • The file exceeds the context window size limit for detailed analysis.

File: posthog/api/insight.py

Structure and Quality Analysis

  1. File Too Long for Detailed Analysis:
    • The file exceeds the context window size limit for detailed analysis.

File: frontend/src/scenes/surveys/SurveyAppearancePreview.tsx

Structure and Quality Analysis

  1. Imports and Dependencies:

    • Imports React hooks (useEffect, useRef) and custom hooks from kea (useValues).
    • Imports functions from an external module (posthog-js) for rendering survey previews.
  2. Component Definition:

    • Defines a functional component (SurveyAppearancePreview) that takes props related to survey data.
  3. Effect Hook:

    • Uses an effect hook to render survey previews whenever dependencies change (survey data or HTML availability).
  4. Code Quality:

    • The component is concise with clear responsibilities.
    • Use of hooks aligns with modern React practices.
    • TypeScript types enhance clarity regarding expected props.

File: plugin-server/src/utils/event.ts

Structure and Quality Analysis

  1. Imports and Dependencies:

    • Imports various types related to events from different modules within the project.
  2. Event Conversion Functions:

    • Functions like convertToOnEventPayload, convertToHookPayload, etc., convert raw event data into different formats required by various parts of the system.
  3. Parsing Functions:

    • Functions like parseRawClickHouseEvent parse raw event data from ClickHouse into more usable forms.
  4. Normalization Functions:

    • Functions like normalizeEvent ensure event data conforms to expected structures before processing.
  5. Code Quality:

    • The code is modular with each function having a clear responsibility.
    • Good use of TypeScript enhances type safety.
    • Comments explaining complex logic would be beneficial but overall readability is good due to descriptive function names.

File: frontend/src/scenes/billing/BillingLimit.tsx

Structure and Quality Analysis

  1. Imports and Dependencies:

    • Imports components from a UI library (LemonButton, LemonInput) and hooks from kea (useActions, useValues).
  2. Component Definition:

    • Defines a functional component (BillingLimit) that renders billing limit settings for a product.
  3. Conditional Rendering:

    • Uses conditional rendering to display different UI elements based on whether the user is editing billing limits or not.
  4. Form Handling:

    • Uses kea forms for managing form state and submission.
  5. Code Quality:

    • The component is well-structured with clear separation between different UI states (viewing vs editing).
    • Good use of hooks aligns with modern React practices. – TypeScript types enhance clarity regarding expected props.

File: posthog/api/plugin.py

Structure and Quality Analysis

  1. Imports and Dependencies: – Extensive imports covering Django REST framework components, serializers, viewsets, logging, file handling, etc.

  2. Helper Functions & Permissions Classes – Helper functions handle plugin attachment updates, logging activities, etc. – Custom permission classes enforce access control based on organization settings.

  3. Serializers – Serializers like PluginSerializer define how plugin models are serialized/deserialized for API interactions.

  4. ViewSets – ViewSets like PluginViewSet define API endpoints for managing plugins with custom actions (e.g., upgrade, update_source).

  5. Code Quality – The code is modular with clear separation between different components (serializers, viewsets). – Comprehensive use of Django REST framework features enhances maintainability.

File: frontend/src/scenes/dashboard/dashboardLogic.tsx

Structure and Quality Analysis

  1. File Too Long for Detailed Analysis
  2. The file exceeds the context window size limit for detailed analysis.

Summary

The analyzed files demonstrate good coding practices with modular structure, clear separation of concerns, effective use of TypeScript types, modern React practices (hooks), comprehensive use of Django REST framework features, structured logging, task scheduling using Celery, etc., contributing to maintainability & readability across different parts of PostHog's codebase.

Aggregate for risks