Integrity Check Catalogue¶
52 active integrity checks run after every Numera load. This page documents what each check tests, what failure means, and suggested investigation steps.
Jump to a category: Data Load Completeness · Dimensional Integrity · SCD2 Integrity · PnL Integrity · Simulation Results · Risk Metrics · Schedule & Volume · Settlement Integrity · Pipeline Health · Power Volumes (15-minute)
Severity key¶
| Severity | Meaning |
|---|---|
| Error | Load is flagged as failed; requires investigation before the next load. |
| Warning | Load continues; advisory signal that should be monitored. |
Three checks emit Warning-level signals: two are Warning-only (check_bav_dim_tran_info_coverage_scd_endur_deal_enhanced, check_cardinality_dim_tran_info); check_fct_pnl_deal_vs_pnl_detail is dual-severity — it warns at >0 rows and escalates to Error at >5,000 rows. All others are Error-only.
Data Load Completeness¶
These checks verify that row counts in the staging and history layers match the raw source. Failures indicate that data has been dropped or not fully propagated during the ingestion pipeline.
check_all_index_rate_loaded¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: For each (sim_date, reval_type_id, sim_run_id, sim_version) in the past week, the row count in sat_endur_sim_result_index_rate must match the row count of the corresponding gen_results_index_rate rows in the raw sim_result source table.
What failure means: Index rate rows from one or more sim runs have not been fully loaded into the staging layer — some rate data from those sims is missing.
Suggested steps:
- Query
sat_endur_sim_result_index_rateand the rawsim_resultsource for the failing(sim_date, sim_run_id, sim_version)to confirm the row count gap. - Check whether the ingestion pipeline logged any errors for the affected sim run.
- If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and the affected
sim_dateandsim_run_id.
check_all_pnl_detail_loaded¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: For each (sim_date, reval_type_id, sim_run_id, sim_version) in the past week, the row count must agree across three layers: the raw sim_result source, sat_endur_sim_result_pnl_detail, and core_endur_metric_pnl_deal_bav.
What failure means: P&L detail rows are present in the raw source but have not fully propagated through the staging or the BAV metric layer, meaning some deal-level P&L is missing downstream.
Suggested steps:
- Identify the failing
(sim_date, sim_run_id, sim_version)by checking which combination shows a count mismatch betweenraw,hist_data, andpit_datain the check's output. - Confirm whether the gap is between raw and staging (
sat_endur_sim_result_pnl_detail) or between staging and the BAV layer (core_endur_metric_pnl_deal_bav). - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
sim_date.
check_all_tran_gpt_delta_loaded¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: For each (sim_date, reval_type_id, sim_run_id, sim_version) in the past week, the row count in sat_endur_sim_result_tran_gpt_delta and core_endur_metric_delta_profile must match the count of gen_results_tran_gpt_delta_by_leg rows in the raw sim_result source.
What failure means: Transaction-level delta rows from one or more sim runs have not been fully loaded into the staging or delta metric layer, meaning some Greeks data is missing.
Suggested steps:
- Identify the failing
(sim_date, sim_run_id, sim_version)from the check output. - Determine whether the gap is in
sat_endur_sim_result_tran_gpt_deltaorcore_endur_metric_delta_profile. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
sim_date.
check_all_var¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: For each (sim_date, reval_type_id, sim_run_id, sim_version) in the past week, the row count in sat_endur_sim_result_var must match the count of gen_results_value_at_risk rows in the raw sim_result source.
What failure means: Value-at-risk rows from one or more sim runs have not been fully loaded into the staging layer, meaning some VaR data is missing.
Suggested steps:
- Identify the failing
(sim_date, sim_run_id, sim_version)from the check output. - Confirm the raw source contains the expected rows and query
sat_endur_sim_result_varfor the same keys. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
sim_date.
check_raw_deal_split_row_counts¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: For each (tran_id, tran_version, _numera_row_type) in the raw deal table that arrived in the past week, the actual number of rows loaded must match the _source_num_rows metadata field embedded in the source file. Excludes alt_fmt_parameter row types.
What failure means: A raw deal file was split or partially ingested — the source recorded a different row count than what was actually loaded into Snowflake.
Suggested steps:
- Query the raw
dealsource for the failing(tran_id, tran_version, _numera_row_type)to compare_source_num_rowsagainst the actual count. - Check whether this is a one-off file delivery anomaly or a recurring pattern for the same deal.
- If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
tran_id.
Remediation steps need verification
The steps above are best-effort based on the SQL logic. They have not yet been validated against a real failure case. If you encounter this check firing, please share the context with the data engineering team so this entry can be improved.
check_sat_endur_parameter¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days (source side); 1-day grace on the upper bound |
What it checks: For every (tran_id, tran_version) pair with a parameter row type in the raw deal source that arrived between 7 and 1 days ago, the row count in hist_endur_side must match.
What failure means: Deal parameter rows from one or more transactions are present in the raw source but have not been fully loaded into hist_endur_side, meaning parameter-level data (rates, settlement types, notionals) may be incomplete.
Suggested steps:
- Query
hist_endur_sideand the rawdealsource for the failing(tran_id, tran_version)to confirm the count gap. - Check whether
hist_endur_sidehas any rows at all for the affectedtran_id. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
tran_id.
check_clearing_bank_statements¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 2 days |
What it checks: Scans twelve ABN and BNP clearing statement models (fct_raw_abn_* and fct_raw_bnp_*) for duplicate rows — any (_file_row_number, _filename) pair that appears more than once within the past 2 days.
What failure means: One or more clearing bank statement files have been ingested more than once, producing duplicate entries in the raw clearing statement tables.
Suggested steps:
- Identify which clearing statement model and which
_filenamecontains duplicates from the check output. - Confirm whether the file was delivered twice by the bank or whether the ingestion pipeline processed it twice.
- If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and the affected
_filename.
Dimensional Integrity¶
These checks verify that P&L and delta records can be linked to their dimensional attributes. Failures indicate deals that exist in the metric layer but have no corresponding entry in the dimension tables.
check_deals_not_linked_to_dims¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Most recent sim date |
What it checks: Identifies (metric_date, deal_id) pairs in fct_pnl_deal where dim_deal_attribute_id = 0 — the sentinel value indicating that no dimensional data was resolved — as of the most recent sim date.
What failure means: One or more deals in the current P&L have no dimensional attributes. They cannot be attributed to a portfolio, instrument type, or counterparty in reports.
Suggested steps:
- Identify the affected
deal_idvalues from the check output. - Query
dim_deal_attributeto confirm whether these deals have entries there at all. - If rows are missing from
dim_deal_attribute, the deals need to be processed by the dimensional model refresh. Wait one load cycle and re-check. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
deal_idvalues.
check_for_missing_dim_data_in_pnl¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: Rows in fct_pnl_deal within the past 7 days where dim_deal_attribute_id = 0 or dim_deal_sub_attribute_id = 0. Returns one row per (deal_id, missing_from, missing_to) group.
What failure means: One or more deals have appeared in the P&L without valid dimensional data across one or more metric dates. P&L totals for these deals cannot be attributed dimensionally.
Suggested steps:
- Query
fct_pnl_dealfor rows matching the failure condition (see Understanding a Failing Check for the exact SQL). - Check
dim_deal_attributefor the affecteddeal_idvalues. If they are absent, the deal is new and awaiting a dimensional model refresh. - If the deal is present in
dim_deal_attributebut still showsdim_deal_attribute_id = 0infct_pnl_deal, this is a pipeline logic issue — escalate with deal IDs and metric dates. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and latest
metric_date.
check_for_missing_dim_data_in_delta¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: Rows in fct_delta_profile within the past 7 days where dim_deal_attribute_id = 0 or dim_deal_sub_attribute_id = 0. Returns one row per (deal_id, missing_from, missing_to) group.
What failure means: One or more deals in the delta/Greeks profile have no dimensional attributes, meaning Greeks calculations for those deals cannot be attributed to a portfolio or instrument type.
Suggested steps:
- Query
fct_delta_profilefilteringdim_deal_attribute_id = 0 OR dim_deal_sub_attribute_id = 0andmetric_date >= DATEADD('day', -7, CURRENT_DATE()). - Check
dim_deal_attributefor the affecteddeal_idvalues. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and latest
metric_date.
check_dim_tran_info_is_missing¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days with a 2-day grace window |
What it checks: Rows in scd_endur_deal_enhanced where dim_tran_info_key IS NULL and _source_at falls in the window 2–7 days before today (excluding the most recent 2 days as a grace period for new-deal lag). Further excludes ins_class_id IN (0, 1, 2) — instrument classes for which Endur never sends tran_info — and deals where any prior version has tran_info in map_endur_dim_tran_info (indicating a transient pipeline lag, not a structural gap).
What failure means: Deals that are 2–7 days old genuinely have no tran_info attributes from Endur, meaning they cannot be fully linked to their transaction information dimension. This is a structural feed gap, not a timing issue.
Suggested steps:
- Query
scd_endur_deal_enhancedfor the failingdeal_idandtran_idvalues. - Confirm whether
bav_dim_tran_info_keyis also NULL — if the BAV fallback is populated, downstream analytics are unaffected and the null is in the direct key only. - If
bav_dim_tran_info_keyis also NULL, the deal type may not supply tran_info from Endur at all. Check theins_class_idof the affected deals. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, affected
deal_idvalues, and theirins_class_id.
check_bav_dim_tran_info_coverage_scd_endur_deal_enhanced¶
| Severity | Warning |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: Compares the count of rows in scd_endur_deal_enhanced that have a non-null dim_tran_info_key against the count that have a non-null bav_dim_tran_info_key. The BAV (best-available-version) fallback should always provide at least as many resolved keys as the direct version lookup.
What failure means: The BAV tran_info lookup is resolving fewer keys than the direct version lookup, which should not be possible. This indicates a problem with the BAV logic in scd_endur_deal_enhanced.
Suggested steps:
- Query
scd_endur_deal_enhancedto identify deals wherebav_dim_tran_info_key IS NULLbutdim_tran_info_key IS NOT NULL. - Review the BAV join logic in
core_endur_deal_enhancedfor the affected deals. - Escalate to the data engineering team with the check name and the sample of affected deals.
Remediation steps need verification
The steps above are best-effort based on the SQL logic. They have not yet been validated against a real failure case. If you encounter this check firing, please share the context with the data engineering team so this entry can be improved.
check_cardinality_dim_tran_info¶
| Severity | Warning |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: Returns one row if the total row count of dim_tran_info exceeds 15,000,000. High cardinality in this dimension degrades query performance for any report that joins through it.
What failure means: dim_tran_info has grown beyond its expected size. This is advisory — the data is not incorrect, but query performance for downstream analytical models may degrade.
Suggested steps:
- Run
SELECT COUNT(*) FROM dim_tran_infoto confirm the current row count. - Investigate whether recent deal activity or a dimensional model rebuild has caused an unexpected expansion.
- Escalate to the data engineering team with the current row count for architectural review.
check_discrete_bu_le_countries¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: Verifies that every row in dim_party_external has bunit_country = lentity_country. If no such rows exist (i.e. all rows are consistent), the check unexpectedly returns a row with an error message — this check fires if there are zero rows with differing countries (a known quirk of the check's logic), meaning failure is actually a sign that the data is unexpectedly homogeneous rather than inconsistent.
What failure means: All rows in dim_party_external have matching bunit_country and lentity_country — which may indicate that distinct business unit and legal entity country data is not being loaded separately. This check guards against a regression in the dimensional model that collapses these two fields to the same value.
Suggested steps:
- Query
SELECT DISTINCT bunit_country, lentity_country FROM dim_party_externalto confirm whether any rows have differing countries. - If all rows have identical
bunit_countryandlentity_country, investigate whether the source model that populates these fields has been changed. - Escalate to the data engineering team with the check name and the row count from
dim_party_external.
SCD2 Integrity¶
These checks verify the structural correctness of the SCD2 (Slowly Changing Dimension Type 2) history for scd_endur_deal_enhanced. Failures indicate corruption in the deal history that could cause point-in-time queries to return incorrect results.
check_scd2_current_for_scd_endur_deal_enhanced¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: For each deal_id in scd_endur_deal_enhanced, exactly one row must have _current = TRUE. Returns any deal_id with more than one current row.
What failure means: Multiple "current" versions exist for the same deal, which breaks all queries that use WHERE _current = TRUE to retrieve the latest deal state. Any model that filters on _current will return duplicates for affected deals.
Suggested steps:
- Query
scd_endur_deal_enhancedfiltering_current = TRUEand grouping bydeal_id HAVING COUNT(*) > 1to identify the affected deals. - Examine the
_valid_from,_valid_to, andtran_versionvalues for the duplicate current rows to understand how the corruption arose. - Escalate immediately to the data engineering team — this is a structural corruption requiring a model rebuild.
check_scd2_temporal_coverage_scd_endur_deal_enhanced¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 3 months |
What it checks: For deals traded in the past 3 months, verifies that there are no temporal gaps between SCD2 versions — each non-current row's _valid_to must equal the next row's _valid_from for the same deal_id. Instantaneous records (_valid_from = _valid_to) are excluded from the gap-detection window.
What failure means: One or more deals have a break in their SCD2 history — a period of time where no version is active. Point-in-time queries against those deals for the gap period will return no rows, causing silent data loss in any model that filters by timestamp.
Suggested steps:
- Query
scd_endur_deal_enhancedfor the affecteddeal_idvalues and examine their full version history (ORDER BY _valid_from) to visualise the gap. - Determine the
gap_secondsmagnitude — gaps of a few seconds may be benign timing artefacts, while gaps of hours or days indicate genuine coverage holes. - Escalate to the data engineering team with the check name, row count, affected
deal_idvalues, and gap timestamps.
check_scd2_valid_from_uniqueness_scd_endur_deal_enhanced¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 3 months |
What it checks: For deals traded in the past 3 months, verifies that each (deal_id, _valid_from) combination is unique in scd_endur_deal_enhanced. Instantaneous records are excluded. Duplicate _valid_from values indicate that two SCD2 rows were written for the same deal at the same timestamp.
What failure means: Multiple SCD2 rows exist for the same deal at the same timestamp. Point-in-time queries may return multiple rows or non-deterministic results for the affected deal at that timestamp.
Suggested steps:
- Query
scd_endur_deal_enhancedgrouping bydeal_id, _valid_from HAVING COUNT(*) > 1to identify the duplicates. - Check whether the duplicate rows have identical or different
tran_versionvalues — identical versions indicate a duplicate insert; different versions indicate two versions were assigned the same_source_attimestamp. - Escalate to the data engineering team — this requires a model rebuild.
check_match_sat_endur_deal_history_to_pit_endur_deal_history¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: Full outer join between sat_endur_deal_history (excluding tran_status_id values 10, 17, and 19) and pit_endur_deal_history on (tran_id, tran_version). Returns rows that appear in one table but not the other.
What failure means: Deal versions that exist in the staging history (sat_endur_deal_history) are absent from the PIT history layer (pit_endur_deal_history), or vice versa. This indicates a propagation failure or filtering discrepancy between the two layers.
Suggested steps:
- Examine the
table_missing_datacolumn in the check output to determine whether rows are missing fromsat_endur_deal_historyorpit_endur_deal_history. - Query both tables for the affected
(tran_id, tran_version)to confirm presence. - Escalate to the data engineering team with the check name, row count, and sample
tran_idvalues.
Remediation steps need verification
The steps above are best-effort based on the SQL logic. They have not yet been validated against a real failure case. If you encounter this check firing, please share the context with the data engineering team so this entry can be improved.
check_for_incorrect_scd_endur_deal_enhanced_tran_status¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: Identifies deals where the current version (_current = TRUE) has a tran_status_id that maps to a "New" status in ref_tran_status (excluding tran_status_id = 17). Returns the full SCD2 history for any deal_id found in a current "New" state.
What failure means: One or more deals are showing as currently active with a "New" status, which indicates that deal booking has not been confirmed in Endur. Deals in a "New" state should not be active in the PIT layer and may indicate an SCD2 state management issue.
Suggested steps:
- Identify the
deal_idvalues from the check output and check their currenttran_statusin Endur. - Confirm whether these are genuinely unconfirmed bookings or whether the status mapping is incorrect.
- Escalate to the data engineering team with the check name, row count, and the affected
deal_idvalues.
PnL Integrity¶
These checks verify the correctness and completeness of deal-level P&L data.
check_for_pnl_duplicates¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 1 month |
What it checks: Identifies (sim_date, portfolio, sim_def_id, sim_run_id, sim_version, _source_row) combinations in core_endur_sim_result_pnl_detail that appear more than once in the past month.
What failure means: Duplicate P&L detail rows exist for one or more sim runs and portfolios. Downstream aggregations (total P&L, attribution) will be double-counted for affected portfolios and dates.
Suggested steps:
- Identify the affected
(sim_date, portfolio)from the check output. - Query
core_endur_sim_result_pnl_detailfor the failing combination to confirm the duplicate_source_rowvalues. - Escalate to the data engineering team — duplicate P&L rows require investigation of the incremental load logic.
check_fct_pnl_deal_vs_pnl_detail¶
| Severity | Warning (>0 rows) / Error (>5,000 rows) |
| Source | numera-core/tests/ |
| Lookback | Most recent sim date, excluding dates within 3 days of today |
What it checks: For the most recent sim date (beyond a 3-day settling grace period), compares the sum of pnl_ltd_base per deal_id in sat_endur_sim_result_pnl_detail against the same sum in fct_pnl_deal. Returns rows where either deal is absent from one side or where the absolute difference exceeds £1,000.
What failure means: Deal-level P&L in fct_pnl_deal diverges from the raw sim result detail for that date. Discrepancies up to the warning threshold are normal during the settling period after a sim run; discrepancies at Error scale indicate a structural problem in the P&L pipeline.
Suggested steps:
- Identify the affected
deal_idvalues from the check output and note whetherpnl_detail_ltdorfct_pnl_deal_ltdis NULL (deal present in only one side) or whether both are present with a large discrepancy. - Query both
sat_endur_sim_result_pnl_detailandfct_pnl_dealfor the affectedmetric_dateanddeal_id. - If
num_errorsis between 1 and 5,000, allow one more load cycle — small discrepancies often self-resolve as sims complete. - If
num_errorsexceeds 5,000, escalate to the data engineering team with the check name, row count, and latestmetric_date.
check_pnl_aggregation¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Second-most-recent metric date |
What it checks: Verifies that aggregating core_endur_metric_pnl_deal_bav by (metric_date, dim_deal_attribute_key, dim_tran_info_key) produces the same totals for volume and pnl_ltd_base as core_endur_metric_pnl for the same grain. Uses the second-most-recent metric_date (not the latest) to avoid checking data still being settled.
What failure means: The deal-level PnL detail and the portfolio-level PnL summary are inconsistent — the rollup does not reproduce the summary for one or more dimension combinations. This can indicate a duplicate-row bug in either model.
Suggested steps:
- Query both
core_endur_metric_pnl_deal_bavandcore_endur_metric_pnlfor the failing(metric_date, dim_deal_attribute_key, dim_tran_info_key). - Compare the
volumeandpnl_ltd_basetotals between the two models. - Escalate to the data engineering team with the check name, row count, and the failing
metric_date.
check_for_missing_delivery_dates_in_pnl¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: Rows in core_endur_metric_pnl_deal_bav within the past 7 days where deal_delivery_start_date IS NULL.
What failure means: Deal-level P&L records exist in the BAV metric layer without a delivery start date populated. This field is required for delivery-period attribution and maturity calculations.
Suggested steps:
- Query
core_endur_metric_pnl_deal_bavfilteringdeal_delivery_start_date IS NULL AND metric_date >= DATEADD('day', -7, CURRENT_DATE()). - Identify the affected
deal_idvalues and check whetherscd_endur_deal_enhancedcontains a delivery date for those deals. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
deal_idvalues.
check_missing_bav_pnl_deals¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 3 days |
What it checks: Rows in core_endur_metric_deal_bav within the past 3 days where the (sim_run_id, sim_version, deal_id) combination does not appear in sat_endur_sim_result_deal_from_metadata.
What failure means: Deal-level metric rows exist in the BAV metric layer for sim runs in which those deals do not appear in the sim metadata. This indicates a potential orphan row — a deal metric entry with no corresponding sim source.
Suggested steps:
- Query
core_endur_metric_deal_bavandsat_endur_sim_result_deal_from_metadatafor the affected(sim_run_id, sim_version, deal_id)combinations. - Confirm whether the deals appear in any sim result for the affected
sim_date. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
sim_run_idvalues.
check_deals_both_fresh_and_stale¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 1 month |
What it checks: Identifies (metric_date, portfolio) combinations in core_endur_metric_pnl_deal_bav in the past month where the same deal_id appears with more than one distinct days_stale value — meaning the same deal is simultaneously classified as both fresh and stale for the same date and portfolio.
What failure means: The fresh/stale classification logic has produced a contradictory result for one or more deals. Fresh deals represent the current sim result; stale deals are carried forward from a prior date. A deal appearing in both categories for the same date indicates a pipeline ordering issue.
Suggested steps:
- Query
core_endur_metric_pnl_deal_bavfor the failing(metric_date, portfolio, deal_id)to see the differentdays_stalevalues present. - Check whether a sim re-run occurred for the affected date, which could have produced both a fresh and a stale entry.
- Escalate to the data engineering team with the check name, row count, and affected
metric_dateandportfolio.
check_multiple_scenario_pnl_results_for_fanout¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: For each (sim_date, sim_def_id, reval_type_id, scenario_id) in core_endur_sim_result_pnl_detail in the past 7 days, checks that there is only one distinct dim_metric_scenario_key. Multiple keys for the same scenario_id indicate a fan-out in the scenario dimension join.
What failure means: Stress sim results with multiple scenarios are producing a fan-out in the dim_metric_scenario_key column — multiple distinct keys are being generated for the same scenario_id, causing duplicate rows in scenario-level PnL aggregations.
Suggested steps:
- Query
core_endur_sim_result_pnl_detailfor the affected(sim_date, sim_def_id, scenario_id)to see the distinctdim_metric_scenario_keyvalues. - Check whether the scenario dimension has multiple rows for the same
scenario_id. - Escalate to the data engineering team with the check name, row count, and affected
sim_date.
Remediation steps need verification
The steps above are best-effort based on the SQL logic. They have not yet been validated against a real failure case. If you encounter this check firing, please share the context with the data engineering team so this entry can be improved.
Simulation Results¶
These checks verify the completeness and integrity of Endur simulation results used to produce PnL and credit exposure metrics.
check_ensure_only_one_sim_result_per_pfolio_per_day¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 7 days |
What it checks: For each (portfolio_id, metric_date, reval_type_id) in core_endur_metric_pnl_deal_bav in the past 7 days where days_stale = 0 (i.e. fresh results), verifies that there is only one distinct (sim_run_id, sim_version) combination. Multiple sim run/version pairs for the same portfolio and date indicate that the BAV selection has produced ambiguous results.
What failure means: The best-available-value selection for fresh sim results is returning multiple distinct sim runs for the same portfolio on the same date, meaning the canonical sim result is ambiguous. Downstream P&L totals may be inconsistent.
Suggested steps:
- Query
core_endur_metric_pnl_deal_bavfor the failing(portfolio_id, metric_date, reval_type_id)to identify the multiple(sim_run_id, sim_version)pairs. - Check
sat_endur_sim_result_deal_from_metadatato understand how many sim runs exist for that date and portfolio. - Escalate to the data engineering team with the check name, row count, and affected
metric_dateandportfolio_id.
check_endur_sim_rerun_drops_validated_deal¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | 14 days (configurable via integrity.sim_rerun_lookback_days) |
What it checks: Detects deals that had tran_status = 'Validated' on a recent business day but do not appear in core_endur_sim_result_deal_from_metadata for that day and reval type. Only checks (date, reval_type) combinations where canonical PnL data exists (confirming a sim did run). For each missing deal, classifies the failure mode:
RERUN_OVERWROTE_OVERNIGHT_SIM: the deal appeared in some sim for that date (visible insat_endur_sim_result_deal_from_metadata) but the canonical best-available selection chose a later re-run that excluded it after maturation.OVERNIGHT_BATCH_AFTER_MATURATION: the deal never appeared in any sim for that date because the overnight batch ran after the deal had already been matured.
What failure means: A validated deal's P&L contribution has silently disappeared from the canonical metric for a recent business date. The failure_mode column indicates which pattern applies. This check is self-healing: when Endur resolves the issue (un-matures and re-simulates the deal), it reappears in core_endur_sim_result_deal_from_metadata and this check automatically clears.
Suggested steps:
- Query
scd_endur_deal_enhancedfor the affecteddeal_idandsim_dateto confirm the deal's lifecycle (Validated → Matured timestamps). - Check
sat_endur_sim_result_deal_from_metadatafor all sim runs on the affectedsim_dateand reval type — confirm whether the deal appears in an earlier sim but not the canonical one. - If
failure_mode = RERUN_OVERWROTE_OVERNIGHT_SIM: the canonical selection logic chose a post-maturation re-run. The deal's P&L for that date is missing. Contact the data engineering team — a fix to the PIT model QUALIFY logic may be required. - If
failure_mode = OVERNIGHT_BATCH_AFTER_MATURATION: the overnight batch ran after maturation and the deal was excluded. In some cases Endur can un-mature and re-simulate; coordinate with the trading operations team. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, affected
deal_idvalues,sim_date, andfailure_mode.
check_credit_latest_metric_date¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Latest metric date per dim_metric_date |
What it checks: For every metric_date marked is_latest = TRUE in dim_metric_date, verifies that corresponding rows exist in all three credit fact tables: fct_credit_exposure, fct_credit_exposure_month, and fct_credit_pex_factor. Only fires after 08:00 Berlin time to allow the overnight pipeline to complete before the check runs.
What failure means: One or more credit fact tables are missing data for the latest metric date. Credit sim results run on a T+1 schedule relative to PnL sims, so the most recent is_latest date in dim_metric_date (derived from PnL sim availability) may be one day ahead of available credit data. A single failing metric date that self-resolves on the following day's load is the expected pattern; persistent failures or multiple missing dates indicate a structural credit pipeline issue.
Suggested steps:
- Query
fct_credit_exposure,fct_credit_exposure_month, andfct_credit_pex_factorforMAX(metric_date)in each to confirm the latest available date. - Check
dim_metric_datefor theis_latest = TRUEdate to understand the expected date. - If the gap is one day and self-resolves on the next load, this is the known T+1 lag pattern — no action required.
- If the gap is two or more days or does not self-resolve, escalate to the data engineering team with the check name, row count, and the expected and actual latest
metric_date.
Risk Metrics¶
These checks verify that risk metric data (delta and vega) is linked to its profile schedule.
check_delta_has_profile¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 2 days |
What it checks: Rows in core_endur_metric_delta_profile within the past 2 days where has_profile_data = FALSE.
What failure means: Delta metric rows exist for recent dates but are not linked to profile data. Delta values without profiles cannot be attributed to specific delivery periods.
Suggested steps:
- Query
core_endur_metric_delta_profilefilteringhas_profile_data = FALSE AND metric_date >= DATEADD('day', -2, CURRENT_DATE()). - Identify the affected
deal_idvalues and check whether profile data exists for those deals inpit_endur_profile_expanded. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
metric_dateanddeal_idvalues.
check_vega_has_profile¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 2 days |
What it checks: Rows in core_endur_metric_vega_profile within the past 2 days where has_profile_data = FALSE.
What failure means: Vega metric rows exist for recent dates but are not linked to profile data. Vega values without profiles cannot be attributed to specific delivery periods.
Suggested steps:
- Query
core_endur_metric_vega_profilefilteringhas_profile_data = FALSE AND metric_date >= DATEADD('day', -2, CURRENT_DATE()). - Identify the affected
deal_idvalues and check whether profile data exists for those deals. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
metric_dateanddeal_idvalues.
Schedule & Volume¶
These checks verify the integrity of deal schedule and volume data.
check_for_missing_schedule_commodity_versions¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 3 months (raw side); bounded by model watermark |
What it checks: For every commodity flat_schedule deal version in the raw source that was deduped at or before the model's own watermark, verifies that the latest tran_version in sat_endur_schedule_commodity matches the latest version in the raw source. Uses the model's own _dedup_finished_at watermark as the cutoff (not the pipeline run timestamp) to avoid false positives from raw data arriving between the load and the check.
What failure means: One or more deals have a newer tran_version in the raw commodity schedule source than in sat_endur_schedule_commodity. The schedule model has not yet processed the latest version of these deals.
History of false positives
This check previously fired as a false positive (BUG-2026-05-003) when the cutoff was derived from the pipeline run log rather than the model watermark, causing newly-deduped raw data to be in scope before the load had processed it. The cutoff logic was fixed in v1.4.172.
Suggested steps:
- Query
sat_endur_schedule_commodityand the rawdealsource for the affecteddeal_idvalues to confirm the version gap. - Check whether the gap is a single version (likely a timing artefact from the next dedup batch) or multiple versions (indicating persistent lag).
- If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
deal_idvalues.
check_null_day_start_time_in_pit_endur_schedule¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 3 days |
What it checks: Joins scd_endur_deal_enhanced (current versions, _source_at within the past 3 days) to pit_endur_schedule on (bav_tran_id, bav_tran_version) and returns rows where pit_endur_schedule.day_start_time_secs IS NULL.
What failure means: Recently-updated deals linked to schedule records are missing the day_start_time_secs field. This field is required for the 15-minute period unrolling pipeline to correctly assign delivery periods.
Suggested steps:
- Query
pit_endur_schedulefor the affected(tran_id, tran_version)to confirm the nullday_start_time_secs. - Check whether the raw schedule source contains this field for the affected deals.
- If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
deal_idvalues.
check_core_volume_daily_has_nulls¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 1 day |
What it checks: Rows in core_endur_volume_deal_daily that were ingested in the past day (_ingested_at >= DATEADD(DAY, -1, CURRENT_TIMESTAMP())) where any of dim_deal_attribute_key, volume_factor, or volume_sign is NULL.
What failure means: Recently-loaded volume records are missing critical fields required for the 15-minute volume pipeline. Rows with null dim_deal_attribute_key cannot be joined to dimensional data; null volume_factor or volume_sign breaks volume calculations.
Suggested steps:
- Query
core_endur_volume_deal_dailyfiltering for recent rows with any of the three null fields. - Identify the affected
deal_idvalues and check whether the source schedule data is complete. - Escalate to the data engineering team with the check name, row count, and affected
deal_idvalues — null dimension keys in this model indicate a pipeline issue upstream.
check_for_missing_parameters¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: Left-joins sat_endur_deal to hist_endur_side on (tran_id, tran_version) and returns rows where no corresponding parameter record exists in hist_endur_side.
What failure means: Deal versions in sat_endur_deal have no parameter data in the deal history. Parameter data (rates, settlement types, notionals) is required for volume and PnL calculations.
Suggested steps:
- Identify the affected
(tran_id, tran_version)pairs from the check output. - Confirm whether
hist_endur_sidehas been populated for these deals — queryhist_endur_sidedirectly. - If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
tran_idvalues.
Settlement Integrity¶
These checks verify the completeness and consistency of settlement event data.
check_for_missing_settle_deals¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: Identifies deals that have a current SCD2 record in scd_endur_deal_enhanced and at least one settlement event in hist_endur_event_settle, but are completely absent from pit_endur_deal_event_settle. Returns up to 100 affected deal_id values.
What failure means: Settlement events exist in the staging layer for active deals, but those deals have not been populated into the PIT settlement layer. This means pit_endur_settlement_event and fct_settle_deal are missing settlement data for these deals. The root cause is typically an incremental watermark gap where scd_endur_deal_enhanced was not yet populated for the deal when the PIT layer was built (BUG-2026-02-005).
Self-healing behaviour
The pit_endur_deal_event_settle model includes a look-back window that re-checks recent hist rows on each incremental run. Once scd_endur_deal_enhanced is populated for the affected deal, the settlement data will self-heal on the next load without requiring a full rebuild.
Suggested steps:
- Query
hist_endur_event_settleandscd_endur_deal_enhancedfor the affecteddeal_idto confirm the data exists in both upstream models. - Query
pit_endur_deal_event_settleto confirm the deal is absent. - If the deal is confirmed absent, allow one full load cycle — the look-back mechanism should populate it automatically.
- If not resolved after the next load cycle, escalate to the data engineering team with the check name, row count, and affected
deal_idvalues.
check_for_mixed_settlement_data¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: Returns (deal_id, tran_id, tran_version) groups in pit_endur_settlement_event where more than one distinct tran_status_id exists for the same combination. A single deal/version should have a single settlement status.
What failure means: The settlement event layer contains contradictory status data for the same deal version — for example, both active and matured status codes appearing together. This indicates a data quality issue in the settlement pipeline.
Suggested steps:
- Query
pit_endur_settlement_eventfor the affected(deal_id, tran_id, tran_version)to see the distincttran_status_idvalues. - Check
hist_endur_event_settleto understand the source status data for the affected deal. - Escalate to the data engineering team with the check name, row count, and affected
deal_idvalues.
check_for_multiple_deal_versions_in_settlement_data¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | Full history (unwindowed) |
What it checks: Returns deal_id values in pit_endur_settlement_event where more than one distinct tran_version exists. The PIT settlement layer should hold exactly one version per deal.
What failure means: Multiple deal versions are present in the settlement PIT layer for the same deal, which means settlement calculations may be using stale or superseded deal data.
Suggested steps:
- Query
pit_endur_settlement_eventfor the affecteddeal_idto see the distincttran_versionvalues present. - Check
scd_endur_deal_enhancedto confirm which version is the current BAV version. - Escalate to the data engineering team with the check name, row count, and affected
deal_idvalues.
check_for_multiple_document_ids_or_versions_per_event_id¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 6 months |
What it checks: For settlement events with a pymt_due_date in the past 6 months, joins fct_settle_deal to sat_endur_stldoc_header and checks whether any (deal_id, event_id) pair is linked to more than one distinct (document_id, document_version) combination.
What failure means: A single settlement event is linked to multiple settlement documents or document versions, indicating either a data integrity issue in the settlement document pipeline or a genuine multiple-document scenario that the model does not handle correctly.
Suggested steps:
- Query
fct_settle_dealandsat_endur_stldoc_headerfor the affected(deal_id, event_id)to see the distinct(document_id, document_version)pairs. - Check the
doc_typefromref_stldoc_doc_typeto understand what type of settlement document is involved. - Escalate to the data engineering team with the check name, row count, and affected
deal_idandevent_idvalues.
Pipeline Health¶
These checks monitor the operational health of the Snowflake pipeline infrastructure.
check_dedup_task_status¶
| Severity | Error |
| Source | numera-core/tests/ |
| Lookback | 2 days |
What it checks: Queries Snowflake's INFORMATION_SCHEMA.TASK_HISTORY for the most recent scheduled execution of each Snowflake task in the raw Endur database and schema. Returns tasks whose most recent execution completed with state = 'FAILED'.
What failure means: One or more Snowflake deduplication tasks that feed the raw Endur layer have failed in the past 2 days. Failed dedup tasks mean raw data from Endur has not been deduplicated and merged into the staging tables, so downstream models may be working from stale or incomplete source data.
Suggested steps:
- Note the task name from the check output.
- Query Snowflake's
INFORMATION_SCHEMA.TASK_HISTORYfor the failing task name to see the error message and failure timestamp. - Escalate immediately to the data engineering team with the check name, the failing task name, and the error message — dedup task failures block all downstream processing.
Power Volumes (15-minute)¶
These checks verify the structural correctness and consistency of the 15-minute power volume pipeline. All twelve checks are EWE-specific and live in dbt/tests/.
check_15m_fct_volume_deal_one_to_one_with_source¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | None (global structural invariant) |
What it checks: Compares the total row count of fct_volume_15m_deal against the row count of core_endur_volume_deal_daily_union joined to the current version of scd_endur_deal_enhanced. The counts must be equal. Returns one row (with actual and expected counts) if they differ; returns zero rows if they match.
What failure means: The fact table has more or fewer rows than its source after BAV filtering. A higher count indicates a fan-out in a join (e.g. a duplicate key in the shape dimension); a lower count indicates deals are being silently dropped. This check was introduced to prevent recurrence of the shape-dimension duplicate issue (BUG-2026-05-001).
Suggested steps:
- Note the
fact_rows,expected_rows, androw_count_diffvalues from the check output. - If
fact_rows > expected_rows: checkcore_endur_volume_15m_shapefor duplicatedim_volume_shape_keyvalues — a fan-out in the shape join is the most likely cause. - If
fact_rows < expected_rows: check whether recent deals incore_endur_volume_deal_daily_unionare missing fromfct_volume_15m_deal. - Escalate to the data engineering team with the check name and the three count values.
check_15m_floating_fixed_mutual_exclusion¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | Configurable window (default: current day ±1) |
What it checks: Verifies that no deal appears with non-zero volume in both fct_ewe_volume_pwr_by_business_case_deal (fixed price) and fct_ewe_volume_pwr_floating_by_business_case_deal (floating/index price) on the same delivery_date_utc. Comparison is at day grain to avoid false positives from gap-fill zero-volume rows.
What failure means: A deal is simultaneously classified as fixed-price and floating/index-price for the same delivery date. This indicates either a genuine business invariant violation (a PWR Index deal routed to a fixed-price business unit) or a filter in one of the wrapper views that has become too broad.
Suggested steps:
- Identify the affected
deal_idanddelivery_date_utcfrom the check output. - Query
scd_endur_deal_enhancedfor the deal to confirm itsinstrument_sub_type— a deal withinstrument_sub_type = 'PWR Index'should only appear in the floating wrapper. - Check the filter logic in
fct_ewe_volume_pwr_by_business_case_dealandfct_ewe_volume_pwr_floating_by_business_case_dealfor the deal's business unit. - Escalate to the data engineering team with the check name, row count, and affected
deal_idvalues.
check_15m_gap_fill_most_active_combo¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | Configurable window (default: current day ±7) |
What it checks: Dynamically selects the most active (business_case, grid) combination in the test window and verifies that every delivery_date_utc in the window has exactly 96 fifteen-minute rows for that combination in fct_ewe_volume_pwr_by_business_case. Returns rows for any date where the actual count differs from 96.
What failure means: The gap-fill mechanism in the EWE power volume wrapper is not producing exactly 96 periods per day for the most active combination. This could indicate a regression in the cross-join between dim_period and dim_combos, a partial population of dim_period for some dates, or a filter inadvertently dropping periods.
Suggested steps:
- Note the failing
(business_case, grid, delivery_date_utc)andactual_row_countfrom the check output. - Query
fct_ewe_volume_pwr_by_business_casefor the affected combination and date to count the rows directly. - If dates near the edge of the window (today ± 7) are affected, this may be a gap-fill boundary issue. Check whether
dim_periodcontains rows for the affected dates. - Escalate to the data engineering team with the check name, row count, and the affected combination and date.
check_15m_profile_schedule_reconciliation¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | Previous full calendar month (deals by trade date) |
What it checks: For every BAV-current power-toolset deal (toolset_id = 33) with a trade_date in the previous full calendar month, verifies that SUM(|notional|) across the deal's delivery profile legs (from pit_endur_profile_expanded) equals SUM(|total_volume|) across the deal's schedule rows in fct_volume_15m_deal. Profile legs are scoped to delivery legs only (those with a corresponding schedule row in core_endur_volume_deal_daily_union) to exclude pricing legs that have profile entries but no schedule. Uses a load watermark cap to prevent false positives from staging data that has not yet propagated to downstream models.
What failure means: The volume committed in a deal's delivery profile does not match the total volume in the 15-minute schedule for that deal. A 50% discrepancy typically indicates a multi-leg deal where the profile includes both delivery and pricing legs while the schedule covers only the delivery leg. Any discrepancy beyond the 0.001 tolerance indicates an inconsistency in the underlying fact tables.
Known false positives on test stack
This check previously failed on the test stack due to multi-leg power deals where both legs share the same settlement_type_id (BUG-2026-05-004). The fix scopes profile legs via core_endur_volume_deal_daily_union rather than settlement_type_id.
Suggested steps:
- Identify the failing
deal_id,profile_abs_total,schedule_abs_total, anddiscrepancyfrom the check output. - Check whether
discrepancy / profile_abs_total ≈ 50%— this indicates a two-leg deal and is typically a test design boundary case. Escalate to confirm. - For discrepancies that are not multiples of the profile total, check whether the deal's schedule rows in
fct_volume_15m_dealare complete. - Escalate to the data engineering team with the check name, row count, the failing
deal_idvalues, and the discrepancy magnitudes.
check_15m_quarter_deal_vs_agg¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | Configurable window (default: current day ±1) |
What it checks: Verifies that aggregating fct_volume_pwr_quarter_fixed_price_deal by (business_case, grid, delivery_date_utc, dim_deal_attribute_id) produces the same total volume as fct_volume_pwr_quarter_fixed_price at the same grain, within tolerance (default: 0.001). Comparison is at day grain (not period grain) for performance.
What failure means: The deal-level and aggregated versions of the quarter fixed-price wrapper report different total volumes for the same combination and date. This indicates an inconsistency between the two underlying numera-core fact tables (fct_volume_15m_deal and fct_volume_15m).
Now protected against cancelled-deal residue
This check previously failed when a cancelled deal left stale volume in the deal-grain base table (BUG-2026-06-016). The 15-minute pipeline now removes a deal's volume structurally the moment it leaves Validated/Matured status, so this reconciliation is no longer exposed to cancelled-deal residue. See Cancelled and withdrawn deals.
Suggested steps:
- Identify the failing
(business_case, grid, delivery_date_utc, dim_deal_attribute_id)anddiscrepancyfrom the check output. - Query both
fct_volume_pwr_quarter_fixed_price_dealandfct_volume_pwr_quarter_fixed_pricefor the affected combination to confirm the volume difference. - Escalate to the data engineering team with the check name, row count, and the failing combination details.
check_15m_short_term_desk_agg_wrapper_vs_upstream¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | Configurable window (default: current day ±1) |
What it checks: Verifies that the non-deal-grain short-term desk wrapper fct_ewe_volume_pwr_by_business_case reports the same total volume per (business_case, grid, delivery_date_utc) as fct_volume_pwr_15m_agg filtered to the same business unit and status criteria. Replicates the wrapper's full BU filter (ECC INTD BU, NORD POOL AS BU, and EWE TR SHORT TERM BU with PWR_ST_PF_Netting portfolio).
What failure means: The EWE short-term desk non-deal wrapper is reporting volumes that differ from the upstream aggregate for the same filter criteria and date. This may indicate a deal outside the allowed business units contributing volume to the wrapper, or a filter in the wrapper that has changed without the test being updated.
Suggested steps:
- Identify the failing
(business_case, grid, delivery_date_utc)anddiscrepancyfrom the check output. - Compare
wrapper_total_volume_absagainstupstream_total_volume_abs— which is larger? If the wrapper exceeds upstream, a deal outside the BU filter may be contributing. - Escalate to the data engineering team with the check name, row count, and the failing combination details.
check_15m_short_term_desk_wrapper_vs_upstream¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | Configurable window (default: current day ±1) |
What it checks: Verifies that the deal-grain short-term desk wrapper fct_ewe_volume_pwr_by_business_case_deal reports the same total volume per (deal_id, dim_deal_attribute_id, business_case, grid, delivery_date_utc) as fct_volume_15m_deal filtered to tran_status IN ('Validated', 'Matured') and external_business_unit IN ('ECC INTD BU', 'NORD POOL AS BU').
What failure means: The EWE short-term desk deal-grain wrapper is reporting volumes that differ from the upstream deal fact for the same filter criteria and date. This indicates an inconsistency in the filter logic or an aggregation error in the wrapper.
Suggested steps:
- Identify the failing
(deal_id, business_case, grid, delivery_date_utc)anddiscrepancyfrom the check output. - Query
fct_ewe_volume_pwr_by_business_case_dealandfct_volume_15m_dealfor the affected deal and date. - Escalate to the data engineering team with the check name, row count, and the failing deal and combination details.
check_15m_unfiltered_deal_vs_agg¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | Configurable window (default: current day ±1) |
What it checks: Verifies that aggregating fct_ewe_volume_pwr_by_business_case_deal_unfiltered over (deal_id, dim_deal_attribute_id) produces the same total volume as fct_ewe_volume_pwr_by_business_case_unfiltered at grain (business_case, grid, delivery_date_utc), within tolerance. Both wrappers apply the same filter (tran_status IN ('Validated', 'Matured')).
What failure means: The deal-level and aggregated versions of the unfiltered controlling wrapper report different volumes for the same combination and date, indicating an inconsistency between the underlying fct_volume_15m_deal and fct_volume_15m fact tables.
Now protected against cancelled-deal residue
This check previously failed when a cancelled deal left stale volume in the deal-grain base table (BUG-2026-06-016). The 15-minute pipeline now removes a deal's volume structurally the moment it leaves Validated/Matured status, so this reconciliation is no longer exposed to cancelled-deal residue. See Cancelled and withdrawn deals.
Suggested steps:
- Identify the failing
(business_case, grid, delivery_date_utc)anddiscrepancyfrom the check output. - Query both
fct_ewe_volume_pwr_by_business_case_deal_unfilteredandfct_ewe_volume_pwr_by_business_case_unfilteredfor the affected combination. - Escalate to the data engineering team with the check name, row count, and the failing combination details.
check_15m_no_live_rows_for_cancelled_deals¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | None (global structural invariant) |
What it checks: For every deal that has left Validated/Matured status, verifies that no live (in-scope) rows remain for that deal in any of the nine 15-minute power-volume base tables. A deal that is cancelled or withdrawn must contribute zero volume.
What failure means: A deal that is no longer in scope still has volume in the 15-minute pipeline — exactly the stale-data condition that previously inflated customer totals (BUG-2026-06-016). Daily totals for the affected (business_case, grid, delivery_date_utc) would be overstated.
Suggested steps:
- Identify the affected
deal_idvalues from the check output. - Query
scd_endur_deal_enhancedfor the deal's currenttran_statusto confirm it has leftValidated/Matured. - Escalate to the data engineering team with the check name, row count, and affected
deal_idvalues — a deal's volume has not been cleared after it left scope.
check_15m_tombstone_exclusivity¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | None (global structural invariant) |
What it checks: Verifies that within each 15-minute power-volume base table, no key is simultaneously live and marked-for-removal. A key contributes either real volume or nothing — never both at once.
What failure means: The internal mechanism that clears a removed deal's volume has produced a contradictory state for a key, which could cause that key's volume to be either double-counted or unpredictably dropped in customer views.
Suggested steps:
- Note the affected key columns from the check output.
- Escalate to the data engineering team with the check name, row count, and the affected keys — this indicates an inconsistency in the volume-removal logic.
check_15m_no_sentinel_in_customer_views¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | None (global structural invariant) |
What it checks: Verifies that no internal placeholder value (for example, an out-of-range delivery_date_utc such as 1970-01-01) ever appears in a customer-facing fct_volume_pwr_* or fct_ewe_volume_pwr_* view. Internal bookkeeping rows used to clear removed deals must be filtered out before any customer view.
What failure means: An internal placeholder row has leaked into a customer view, meaning a query could return a row with an obviously-invalid delivery date and zero volume. This indicates a customer view is missing its standard exclusion filter.
Suggested steps:
- Identify which customer view and which placeholder value appears from the check output.
- Escalate to the data engineering team with the check name, row count, and the affected view — the view's filter that hides internal rows needs to be restored.
check_15m_tombstone_zero_measure¶
| Severity | Error |
| Source | dbt/tests/ |
| Lookback | None (global structural invariant) |
What it checks: Verifies that every internal volume-removal placeholder row carries only null or zero volume measures. A placeholder row exists solely to clear a removed deal's prior volume; it must never carry real volume of its own.
What failure means: A placeholder row carries a non-zero volume, which would add phantom volume to totals if it were ever counted. This indicates a defect in how removal rows are constructed.
Suggested steps:
- Note the affected key and measure values from the check output.
- Escalate to the data engineering team with the check name, row count, and the affected rows.