Recent development efforts have focused on optimizing replay event queries, significantly improving performance and user experience.
frontend/src/toolbar/bar/toolbarLogic.ts
, frontend/src/scenes/heatmaps/HeatmapsBrowser.tsx
, frontend/src/scenes/heatmaps/heatmapsBrowserLogic.ts
).master
into fix/undefined-thingies
and fixed keyboard navigation in toolbar (frontend/src/toolbar/bar/toolbarLogic.ts
).posthog/session_recordings/session_recording_api.py
).posthog/batch_exports/http.py
, posthog/batch_exports/models.py
, posthog/batch_exports/service.py
).posthog/temporal/batch_exports/bigquery_batch_export.py
, posthog/temporal/common/heartbeat.py
).frontend/src/lib/api.ts
, frontend/src/scenes/dashboard/dashboardLogic.tsx
).frontend/src/scenes/dashboard/dashboardLogic.tsx
).frontend/src/scenes/settings/organization/OrgEmailPreferences.tsx
).frontend/__snapshots__/scenes-app-insights--trends-line-edit--light.png
, frontend/src/scenes/settings/organization/OrgEmailPreferences.tsx
).ee/api/test/test_billing.py
, frontend/__snapshots__/scenes-app-insights--trends-line-edit--light.png
, frontend/src/scenes/billing/PlanComparison.tsx
).frontend/__snapshots__/scenes-app-insights--trends-line-edit--dark.png
, frontend/__snapshots__/scenes-other-billing-v2--billing-v-2--dark.png
).frontend/__snapshots__/scenes-other-billing-v2--billing-v-2--dark.png
, frontend/src/scenes/surveys/surveyLogic.tsx
).plugin-server/src/utils/event.ts
, plugin-server/src/worker/ingestion/event-pipeline/runner.ts
).plugin-server/src/main/ingestion-queues/metrics.ts
).posthog/tasks/report_usage.js
). ee/api/test/test_billing.js
). frontend/__snapshots__/components-property-key-info--property-key-info--dark.png
). The PostHog development team has been actively working on various features and fixes across the platform. Recent activities show a strong focus on enhancing the user interface, improving backend functionalities, and adding new features such as dynamic CNAME targets, improved query performance, and better handling of async queries. The team has also been addressing bugs and optimizing existing functionalities to ensure a robust and user-friendly experience. The ongoing efforts reflect a proactive approach to maintaining and improving the platform's capabilities.
Performance Issues in Replay Event Queries (#22823):
Billing Calculations (#22821):
Deployment Issues (#22819):
Error Tracking Implementation (#22814):
Async Calculation for Saved Insights/Dashboards (#22823):
Billing Limits Configuration During Onboarding (#22822):
Overall, the recent activity indicates a robust development cycle with significant improvements in performance, user experience, and system reliability. The team's proactive approach in addressing bugs and introducing new features reflects a strong commitment to continuous improvement and user satisfaction.
Developer | Avatar | Branches | PRs | Commits | Files | Changes |
---|---|---|---|---|---|---|
Ben White | ![]() |
5 | 6/2/3 | 56 | 71 | 17533 |
vs. last report | +2 | -1/-6/+3 | +29 | -56 | +11913 | |
Marius Andra | ![]() |
3 | 7/6/0 | 38 | 159 | 14116 |
vs. last report | +1 | +4/+3/= | +30 | +62 | -23503 | |
timgl | ![]() |
2 | 6/5/0 | 8 | 44 | 4804 |
vs. last report | -1 | -2/-2/= | -6 | -20 | -5675 | |
Tom Owers | ![]() |
2 | 9/8/1 | 10 | 60 | 2933 |
vs. last report | -1 | +4/+5/= | +5 | +31 | +1280 | |
Thomas Obermüller | ![]() |
3 | 12/8/0 | 22 | 67 | 2630 |
vs. last report | = | +5/+2/-1 | +6 | +51 | +629 | |
Michael Matloka | ![]() |
4 | 8/7/0 | 11 | 60 | 2602 |
vs. last report | +3 | +2/-2/= | +1 | -37 | +1964 | |
Eric Duong | ![]() |
6 | 12/8/1 | 18 | 84 | 2357 |
vs. last report | +3 | +10/+7/= | +10 | +60 | +1270 | |
Sandy Spicer | ![]() |
7 | 8/4/2 | 27 | 68 | 1667 |
vs. last report | +4 | +4/=/+2 | +12 | +29 | +883 | |
Zach Waterfield | ![]() |
2 | 10/10/0 | 15 | 74 | 1599 |
vs. last report | = | +7/+5/= | +8 | +64 | +1302 | |
David Newell | ![]() |
2 | 6/4/2 | 10 | 64 | 1561 |
vs. last report | -1 | +2/+2/+2 | -3 | +23 | -10 | |
github-actions | ![]() |
29 | 0/0/0 | 198 | 103 | 1460 |
vs. last report | +11 | =/=/= | +75 | +40 | +488 | |
Julian Bez | ![]() |
3 | 4/4/1 | 9 | 47 | 1290 |
vs. last report | = | -4/-2/+1 | -4 | -8 | +636 | |
Tomás Farías Santana | ![]() |
2 | 4/4/0 | 9 | 21 | 1062 |
vs. last report | +1 | +3/+4/= | +8 | +20 | +1060 | |
Tiina Turban | ![]() |
3 | 6/3/1 | 6 | 20 | 924 |
vs. last report | -4 | -1/+2/= | -3 | -5 | -70 | |
Dylan Martin | ![]() |
2 | 4/3/0 | 4 | 28 | 877 |
vs. last report | = | =/=/= | -6 | +1 | +168 | |
Raquel Smith | ![]() |
4 | 10/7/0 | 14 | 51 | 818 |
vs. last report | +3 | +8/+4/= | +11 | +20 | +101 | |
Nikita Vorobev | ![]() |
1 | 1/0/0 | 1 | 23 | 716 |
vs. last report | +1 | =/=/= | +1 | +23 | +716 | |
Paul D'Ambra | ![]() |
3 | 16/14/0 | 17 | 19 | 664 |
vs. last report | +1 | +7/+6/= | +6 | -42 | -3185 | |
Brett Hoerner | ![]() |
2 | 3/1/0 | 3 | 14 | 585 |
vs. last report | +1 | +2/=/= | +2 | +10 | +521 | |
Bianca Yang | ![]() |
3 | 7/6/0 | 11 | 9 | 364 |
vs. last report | -1 | +2/+1/= | +1 | -98 | -1637 | |
Robbie | ![]() |
3 | 2/1/0 | 6 | 14 | 274 |
vs. last report | +2 | +1/=/= | +5 | +13 | +272 | |
Frank Hamand | ![]() |
2 | 6/5/0 | 7 | 12 | 205 |
vs. last report | +1 | +4/+3/= | +5 | +9 | +175 | |
Neil Kakkar | ![]() |
1 | 1/1/1 | 1 | 8 | 172 |
vs. last report | = | =/+1/+1 | -1 | +6 | -10 | |
Juraj Majerik | ![]() |
1 | 4/4/0 | 4 | 15 | 113 |
PostHog Bot | ![]() |
5 | 8/4/4 | 8 | 2 | 96 |
vs. last report | = | +3/+3/-4 | +3 | = | +46 | |
ted kaemming | ![]() |
2 | 2/1/0 | 3 | 3 | 93 |
vs. last report | = | -2/-2/= | -6 | -12 | -1204 | |
Lior539 | ![]() |
1 | 1/1/0 | 1 | 6 | 51 |
Phani Raj | ![]() |
1 | 1/1/0 | 1 | 5 | 46 |
vs. last report | = | =/+1/= | -3 | -3 | -5 | |
Cory Watilo (corywatilo) | 2 | 2/0/0 | 2 | 4 | 40 | |
feedanal | ![]() |
1 | 0/0/0 | 3 | 2 | 11 |
vs. last report | +1 | -1/=/= | +3 | +2 | +11 | |
Ian Vanagas | ![]() |
1 | 1/1/0 | 1 | 1 | 2 |
None (camerondeleone) | 1 | 1/0/0 | 1 | 1 | 1 | |
Edward Hartwell Goose (edhgoose) | 0 | 1/0/0 | 0 | 0 | 0 | |
Shawn Lockhart (shawnlikescode) | 0 | 1/0/0 | 0 | 0 | 0 |
PRs: created by that dev and opened/merged/closed-unmerged during the period
The PostHog development team is working on an open-source analytics platform that provides product analytics, session recording, feature flagging, and A/B testing capabilities. It is designed for self-hosting, allowing users to maintain control over their data while leveraging the platform's extensive analytics and optimization tools. The project is hosted on GitHub and continues to see active development with a balanced focus on both front-end improvements and back-end stability.
frontend/src/toolbar/bar/toolbarLogic.ts
, frontend/src/scenes/heatmaps/HeatmapsBrowser.tsx
, frontend/src/scenes/heatmaps/heatmapsBrowserLogic.ts
).master
into fix/undefined-thingies
and fixed keyboard navigation in toolbar (frontend/src/toolbar/bar/toolbarLogic.ts
).posthog/session_recordings/session_recording_api.py
).frontend/src/scenes/heatmaps/HeatmapsBrowser.tsx
, frontend/src/scenes/heatmaps/heatmapsBrowserLogic.ts
).posthog/caching/insight_caching_state.py
, posthog/caching/test/test_insight_caching_state.py
).posthog/caching/insight_cache.py
, posthog/caching/insight_caching_state.py
).posthog/batch_exports/http.py
, posthog/batch_exports/models.py
, posthog/batch_exports/service.py
, posthog/temporal/batch_exports/backfill_batch_export.py
, posthog/temporal/batch_exports/batch_exports.py
, posthog/temporal/batch_exports/bigquery_batch_export.py
, posthog/temporal/batch_exports/http_batch_export.py
, posthog/temporal/batch_exports/noop.py
, posthog/temporal/batch_exports/postgres_batch_export.py
, posthog/temporal/batch_exports/redshift_batch_export.py
, posthog/temporal/batch_exports/s3_batch_export.py
, posthog/temporal/batch_exports/snowflake_batch_export.py
, posthog/temporal/tests/batch_exports/test_backfill_batch_export.py
, posthog/temporal/tests/test_encryption_codec.py
).posthog/temporal/batch_exports/bigquery_batch_export.py
, posthog/temporal/batch_exports/s3_batch_export.py
, posthog/temporal/batch_exports/squash_person_overrides.py
, posthog/temporal/common/heartbeat.py
, posthog/temporal/common/utils.py
, posthog/temporal/tests/batch_exports/test_bigquery_batch_export_workflow.py
).frontend/src/lib/api.ts
, frontend/src/queries/query.ts
, frontend/src/queries/schema.json
, frontend/src/queries/schema.ts
, frontend/src/scenes/dashboard/dashboardLogic.tsx
, frontend/src/scenes/saved-insights/activityDescriptions.tsx
, frontend/src/types.ts
, posthog/api/query.py
, posthog/api/services/query.py
, posthog/api/test/__snapshots__/test_api_docs.ambr
, posthog/api/test/dashboards/test_dashboard.py
, posthog/api/test/test_insight.py
, posthog/api/test/test_query.py
, posthog/caching/calculate_results.py
, posthog/caching/fetch_from_cache.py
, posthog/caching/insight_cache.py
, posthog/clickhouse/client/test/test_execute_async.py
).frontend/src/scenes/dashboard/dashboardLogic.tsx
).frontend/src/scenes/settings/organization/OrgEmailPreferences.tsx
).frontend/__snapshots__/scenes-app-insights--trends-line-edit--light.png
,frontend/src/scenes/settings/organization/OrgEmailPreferences.tsx
).ee/api/test/test_billing.py
,frontend/__snapshots__/scenes-app-insights--trends-line-edit--light.png
,frontend/src/scenes/billing/PlanComparison.tsx
,frontend/src/scenes/billing/billing-utils.ts
,frontend/src/scenes/billing/billingProductLogic.ts
).frontend/__snapshots__/scenes-app-insights--trends-line-edit--dark.png
,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2--dark.png
,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2--light.png
,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2-with-discount--dark.png
,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2-with-discount--light.png
,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2-with-limit-and-100-percent-discount--dark.png
,frontend/__snapshots__/scenes-other-billing-v2--billing-v-2-with-limit-and-100-percent-discount--light.png
,frontend/src/scenes/billing/BillingLimit.tsx
,frontend/src/scenes/billing/BillingLimitInput.tsx
,frontend/src/scenes/billing/BillingProduct.tsx
,frontend/src/scenes/billing/billingProductLogic.ts
).frontend/__snapshots__/scenes-other-billing-v2--billing-v-2--dark.png
,frontend/src/scenes/surveys/surveyLogic.tsx
,latest_migrations.manifest
,plugin-server/src/worker/ingestion/property-definitions-auto-discovery.ts
,plugin-server/tests/worker/ingestion/property-definitions-auto-discovery.test.ts
,posthog/hogql_queries/query_runner.py
,posthog/migrations/0421_set_all_survey_responses_to_be_strings.py
,posthog/test/base.py
,posthog/test/test_migration_0421.py
).plugin-server/src/utils/event.ts
,plugin-server/src/worker/ingestion/event-pipeline/runner.ts
).plugin-server/src/main/ingestion-queues/metrics.ts
,plugin-server/src/workers/event-pipeline-runner.js
).posthog/tasks/report_usage.js
). ee/api/test/test_billing.js
). frontend/__snapshots__/components-property-key-info--property-key-info--dark.png
). The PostHog development team has been actively working on various features and fixes across the platform. Recent activities show a strong focus on enhancing the user interface, improving backend functionalities, and adding new features such as dynamic CNAME targets, improved query performance, and better handling of async queries. The team has also been addressing bugs and optimizing existing functionalities to ensure a robust and user-friendly experience. The ongoing efforts reflect a proactive approach to maintaining and improving the platform's capabilities.
Since the last report 7 days ago, there has been significant activity in the repository. Numerous new issues have been created, and several issues have been closed. This indicates ongoing development and maintenance efforts.
Issue #22823: Fix replay events should use subquery on person not join
Issue #22822: Including billing limits in onboarding
Issue #22821: Fix billing month-to-date and tiers computations
Issue #22820: Allow SSH tunnel username for SSH key - Data Warehouse
Issue #22819: Unable to deploy self-hosted hobby version due to Postgres issues
Issue #22818: Fix typo in Nuxt install instructions
Issue #22817: Fix over setting heatmap actions
Issue #22815: Refactor insights to use queries instead of filters for insight activity log
Issue #22814: Error tracking (simple start)
Issue #22812: Flag the persons-on-events project setting
Issue #22816: Fix debug Clickhouse queries on localhost
Issue #22813: Validate join key is a field node in data warehouse
Issue #22809: Use next_allowed_client_refresh for dashboard reload button
Issue #22805: Allow x-highlight-request tracing header
Issue #22801: Collapsible setting panel for in-app heatmaps
Issue #22799: Change refresh modes for dashboard to blocking
Issue #22798: Export Hog interfaces
Issue #22797: New npm release for HogVM
Issue #22795: Heartbeating for all exports and other fixes
10 . * * Issue # 22793 * * : Revert " feat ( insights ) : Async calculation for saved insights / dashboards " - * * Description * * : Reverts async calculation feature due to concurrency concerns . - * * Significance * * : Maintains system stability while addressing concurrency issues .
The recent activity in the repository reflects a robust development cycle focused on both expanding features and maintaining system integrity . The team's responsiveness to issues , both in terms of introducing enhancements and resolving bugs , indicates a strong commitment to user satisfaction and continuous improvement . The introduction of new features alongside critical bug fixes suggests a balanced approach to development , ensuring both innovation and stability .
Overall , the project appears to be progressing well with significant improvements made over the past week .
This pull request addresses a significant performance issue related to querying replay events by optimizing the query structure. The main change involves replacing joins with subqueries when filtering replay events by person properties. Specifically, the changes include:
Modification in property_to_expr
function:
"replay_pdi"
to handle person properties in replay queries.Changes in session_recording_list_from_filters.py
:
PersonsPropertiesSubQuery
and PersonsIdSubQuery
to handle subqueries for person properties and person IDs respectively._where_predicates
method to use these subqueries instead of joins._strip_person_properties
to separate person properties from other properties.Updates in test files:
test_session_recording_list_from_filters.py
and test_session_recording_list_from_session_replay.py
to reflect the new query structure.PersonsPropertiesSubQuery
and PersonsIdSubQuery
classes encapsulates the logic for handling person-related subqueries, making the code more modular and easier to maintain.PersonsPropertiesSubQuery
and PersonsIdSubQuery
classes. Consider refactoring common logic into a base class or utility functions.# posthog/hogql/property.py
def property_to_expr(
property: Union[BaseModel, PropertyGroup, Property, dict, list, ast.Expr],
team: Team,
scope: Literal["event", "person", "session", "replay", "replay_entity", "replay_pdi"] = "event",
) -> ast.Expr:
if isinstance(property, dict):
try:
# posthog/session_recordings/queries/session_recording_list_from_filters.py
class PersonsPropertiesSubQuery:
_team: Team
_filter: SessionRecordingsFilter
_ttl_days: int
def __init__(self, team: Team, filter: SessionRecordingsFilter, ttl_days: int):
self._team = team
self._filter = filter
self._ttl_days = ttl_days
def get_query(self) -> ast.SelectQuery | ast.SelectUnionQuery | None:
if self.person_properties:
return parse_select(
"""
SELECT distinct_id
FROM person_distinct_ids
WHERE {where_predicates}
""",
{
"where_predicates": self._where_predicates,
},
)
else:
return None
@cached_property
def person_properties(self) -> PropertyGroup | None:
person_property_groups = [g for g in self._filter.property_groups.flat if g.type == "person" in g.type]
return (
PropertyGroup(
type=PropertyOperatorType.AND,
values=person_property_groups,
)
if person_property_groups
else None
)
@cached_property
def _where_predicates(self) -> ast.Expr:
return (
property_to_expr(self.person_properties, team=self._team, scope="replay_pdi")
if self.person_properties
else ast.Constant(value=True)
)
# posthog/session_recordings/queries/test/test_session_recording_list_from_filters.py
def test_filter_with_person_properties_exact(self):
session_id_one, session_id_two = self._two_sessions_two_persons(
"test_filter_with_person_properties_exact",
session_one_person_properties={"email": "bla@gmail.com"},
session_two_person_properties={"email": "bla2@hotmail.com"},
)
query_results: SessionRecordingQueryResult = self._filter_recordings_by(
{
"properties": [
{
"key": "email",
"value": ["bla@gmail.com"],
"operator": "exact",
"type": "person",
}
]
}
)
assert [x["session_id"] for x in query_results.results] == [session_id_one]
Overall, this PR effectively addresses a critical performance issue by optimizing how replay events are queried. The changes are well-structured and supported by comprehensive tests. Minor improvements in code duplication and documentation could further enhance code quality.
Since the last analysis 7 days ago, there has been significant activity in the PostHog/posthog repository. Numerous pull requests (PRs) have been opened and closed, with various enhancements, bug fixes, and new features being introduced. Below is a detailed report of the changes:
PR #22823: fix: replay events should use subquery on person not join
PR #22822: wip: including billing limits in onboarding
PR #22821: fix: Billing month-to-date and tiers computations
PR #22818: fix: typo in nuxt install instructions
PR #22817: fix: over setting heatmap actions
PR #22815: refactor(insights): use queries instead of filters for insight activity log
PR #22814: feat: Error tracking (simple start)
PR #22812: chore(settings): Flag the persons-on-events project setting
PR #22811: chore: Person IDs on events by default
Other notable open PRs include improvements to batch export metrics (#22810), iframe error detection for heatmaps (#22808), support form enhancements (#22807), posthog-js update (#22804), and various other fixes and enhancements.
PR #22816: fix(product-analytics): Fix debug clickhouse queries on localhost
PR #22813: chore(data-warehouse): Validate join key is a field node
Other notable closed PRs include improvements to dashboard reload button (#22809), tracing header allowance (#22805), collapsible settings panel for heatmaps (#22801), and various other fixes and enhancements.
Overall, the recent activity indicates an active development cycle aimed at both expanding capabilities and ensuring the reliability of the platform.
frontend/src/toolbar/bar/toolbarLogic.ts
Imports and Dependencies:
kea
, which is a state management library.Logic Definition:
kea<toolbarLogicType>
, which is a type-safe way to define logic in kea
.path
, connect
, actions
, reducers
, selectors
, and listeners
demonstrates a comprehensive use of kea
's capabilities.Actions:
Reducers:
element
, menu
, visibleMenu
, minimized
, etc., are well-defined.Selectors:
dragPosition
and menuProperties
.Listeners:
Lifecycle Methods:
afterMount
and beforeUnmount
are used to set up and clean up event listeners, ensuring proper resource management.Code Quality:
ee/session_recordings/persistence_tasks.py
Imports and Dependencies:
celery
), logging (structlog
), time manipulation (datetime
), and database interactions (django
).Task Definitions:
persist_single_recording
and persist_finished_recordings
.Logging and Metrics:
structlog
.Task Logic:
persist_single_recording
: Calls a function to persist a single recording based on its ID and team ID.persist_finished_recordings
: Identifies recordings older than a day that need persistence and schedules them for persistence.Code Quality:
one_day_old
) improves readability.posthog/batch_exports/http.py
Imports and Dependencies:
Helper Functions:
validate_date_input
provide utility for date validation.Serializers:
BatchExportRunSerializer
, BatchExportDestinationSerializer
, etc.) define how models are serialized/deserialized for API interactions.ViewSets:
BatchExportRunViewSet
, BatchExportViewSet
, etc., define the API endpoints for batch export operations.Custom Actions:
Code Quality:
frontend/src/lib/api.ts
posthog/api/insight.py
frontend/src/scenes/surveys/SurveyAppearancePreview.tsx
Imports and Dependencies:
useEffect
, useRef
) and custom hooks from kea (useValues
).posthog-js
) for rendering survey previews.Component Definition:
SurveyAppearancePreview
) that takes props related to survey data.Effect Hook:
Code Quality:
plugin-server/src/utils/event.ts
Imports and Dependencies:
Event Conversion Functions:
convertToOnEventPayload
, convertToHookPayload
, etc., convert raw event data into different formats required by various parts of the system.Parsing Functions:
parseRawClickHouseEvent
parse raw event data from ClickHouse into more usable forms.Normalization Functions:
normalizeEvent
ensure event data conforms to expected structures before processing.Code Quality:
frontend/src/scenes/billing/BillingLimit.tsx
Imports and Dependencies:
LemonButton
, LemonInput
) and hooks from kea (useActions
, useValues
).Component Definition:
BillingLimit
) that renders billing limit settings for a product.Conditional Rendering:
Form Handling:
Code Quality:
posthog/api/plugin.py
Imports and Dependencies: – Extensive imports covering Django REST framework components, serializers, viewsets, logging, file handling, etc.
Helper Functions & Permissions Classes – Helper functions handle plugin attachment updates, logging activities, etc. – Custom permission classes enforce access control based on organization settings.
Serializers – Serializers like PluginSerializer define how plugin models are serialized/deserialized for API interactions.
ViewSets – ViewSets like PluginViewSet define API endpoints for managing plugins with custom actions (e.g., upgrade, update_source).
Code Quality – The code is modular with clear separation between different components (serializers, viewsets). – Comprehensive use of Django REST framework features enhances maintainability.
The analyzed files demonstrate good coding practices with modular structure, clear separation of concerns, effective use of TypeScript types, modern React practices (hooks), comprehensive use of Django REST framework features, structured logging, task scheduling using Celery, etc., contributing to maintainability & readability across different parts of PostHog's codebase.