-
Notifications
You must be signed in to change notification settings - Fork 124
chore(shredder): propagate subplat deletes to cirrus #8401
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
this is blocked on mozilla/probe-scraper#963 creating the targeted tables |
|
as I understand it, this is not blocked on https://github.com/mozilla-services/cloudops-infra/pull/6696, because the table has already been created |
mikewilli
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typo fix but otherwise LGTM
...data-shared-prod/subscription_platform_backend_cirrus_derived/delete_events_v1/metadata.yaml
Outdated
Show resolved
Hide resolved
...-fx-data-shared-prod/subscription_platform_backend_cirrus_derived/delete_events_v1/query.sql
Show resolved
Hide resolved
sean-rose
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I belatedly thought of a couple more things.
Also, I'm curious whether you've been able to test this ETL code yourself to verify it works end to end. If not, we'll probably want to be careful to deploy this at a time when you're available to quick-fix any issues that may arise.
...-fx-data-shared-prod/subscription_platform_backend_cirrus_derived/delete_events_v1/query.sql
Show resolved
Hide resolved
| CROSS JOIN | ||
| subplat_namespaces | ||
| WHERE | ||
| DATE(delete_events.submission_timestamp) = @submission_date |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
question (blocking): Is it OK for this table to only have records from whenever it's deployed going forward?
If not, you'll probably want to output a different submission_timestamp condition for the table initialization query using Jinja and the is_init() function, including the appropriate historical range of dates.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
going forward is fine, we can truncate the impacted tables to avoid complicating this, esp knowing this is supposed to be "temporary" until some dev time is available after the EoY code freeze
I lack permissions to verify that getting secret values works as intended, I have manually confirmed the uid translation works as intended, as long as it's after 9am PST i'm available to fix |
|
The However, one easy way to unblock deployment of |
|
split out #8548 |
Description
This PR adds shredder config for using fxa delete events to delete subplat cirrus pings,
and adds experiment monitoring config for experimenter_cirrus and subplat_cirrus
Related Tickets & Documents
Reviewer, please follow this checklist