[O2B-1563] Create GAQ summary invalidation mechanism#2174
Open
isaachilly wants to merge 28 commits into
Open
Conversation
Add GaqSummaryRepository and GaqSummaryInvalidationRepository and export them from the repositories index. Update migration to replace the previous invalidated_at field with created_at and add updated_at to mirror default Sequelize tables.
Rename and change mcReproducible to be the coverage float not the boolean.
Forgot to remove.
Add GAQ summary invalidation whenever underlying data affecting GAQ changes. These changes ensure GAQ summary caches/records are marked for recomputation whenever detectors, QC flags, or run QC times that influence GAQ summaries are modified.
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
Validate GAQ summary invalidation behaviour on QC flag create/verify/delete, deleteAllForDataPass, explicit/default GAQ detector changes, and run QC time updates.
Add a calculation_failed boolean column to gaq_summaries. This allows us to know whether a summary has been attempted to be calculated but unsuccessful due to limited data etc.
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
Remove not-null constraints from several GAQ summary columns in the migration to allow NULL when values are unavailable.
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
Store invalidation timestamp directly on gaq_summaries instead of a separate gaq_summary_invalidations table.
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
Replace usage of GaqSummaryInvalidationRepository with GaqSummaryRepository across services and tests. Upsert calls now set an invalidatedAt timestamp to mark GAQ summaries as invalidated.
Create an index on invalidated_at column of gaq_summaries table to improve query performance, as it is commonly used to fetch the invalidated summary queue.
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
…already soft deleted The simplest way of invalidating GAQ summaries affected by the deletion of a dataPass' Flags is to just take the list of deleted flags. But this is inefficient if some flags were already deleted and thus had already had their summaries at the time.
Sequelize model omitted the primaryKey that migration file defines.
…e/O2B-1563/Create-GAQ-summary-invalidation-mechanism
Extract invalidation into helpers. Guard GAQ updateRun reload so don't run reload unnecessarily. Add to notComputable field in model allowNull default params. Add more test coverage for invalidation on run patch.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2174 +/- ##
==========================================
- Coverage 45.82% 41.94% -3.89%
==========================================
Files 1039 1043 +4
Lines 17189 17246 +57
Branches 3138 3144 +6
==========================================
- Hits 7877 7233 -644
- Misses 9312 10013 +701 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| this.home.bubbleTo(this); | ||
|
|
||
| this.lhcPeriods = new LhcPeriodsModel(this.router); | ||
| this.lhcPeriods = new LhcPeriodsModel(this); |
| this.lhcPeriods.bubbleTo(this); | ||
|
|
||
| this.dataPasses = new DataPassesModel(this.router); | ||
| this.dataPasses = new DataPassesModel(this); |
| this.qcFlags.bubbleTo(this); | ||
|
|
||
| this.simulationPasses = new SimulationPassesModel(this.router); | ||
| this.simulationPasses = new SimulationPassesModel(this); |
|
|
||
| try { | ||
| this._value = hasDateAndTime ? extractTimestampFromDateTimeInput(raw) : null; | ||
| this._value = raw.date && raw.time ? extractTimestampFromDateTimeInput(raw, { seconds: this._seconds }) : null; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I have a JIRA ticket
Notable changes for users:
Notable changes for developers:
Changes made to the database:
Will invalidate a GAQ summary row, if present or otherwise, if an identified trigger occurs (see JIRA ticket).
Simply calls a GAQ summary repository helper to create the invalidation from each triggers code.