feat: field rename and skip, plus four API-level fixes#70
Merged
Conversation
- DataModel._build_model: table-name validation iterated over self.model_config (which has no "tables" key after _validate_config) instead of self.tables_config, so the guard was silently dead and unknown table names in model_config were never caught - XMLConverter.parse_xml: skip_validation defaulted to False (validate) while Document.parse_xml and DataModel.parse_xml both default to True (skip); aligns the low-level default with the documented behaviour and with how the method is always called by its callers - DatabaseDialect.db_identifier: remove the dead temp_prefix parameter which was never passed by any caller and whose implementation (max_len += 14) was the inverse of what its docstring described - DataModelColumn: rename the model_config constructor argument and attribute to config to reflect that it holds the table-level config dict, not the top-level model config Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Allows overriding the database column name independently of the internal logical name, so XSD element names that are awkward or conflict with SQL reserved words can be mapped to a cleaner identifier without touching the schema or any other part of the pipeline. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Renames orderid → order_id in the shiporder table of the version 0 sample config to exercise the rename feature through the full DB roundtrip test, and regenerates the three DDL snapshots accordingly. ERD and tree snapshots are unaffected (they display logical names). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Renames product_name → prd_name in the item table of version 1, where item.reuse=False and product is elevated by default (no field-level transform config). Exercises rename on a post-elevation prefixed column name through the full DB roundtrip test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Columns and relations marked with {"transform": "skip"} are entirely
removed from the table schema (no DB column created), omitted from
fields_transforms (XML converter ignores them naturally), and absent
from parsed records. Skipped relations are also removed from
relations_1/relations_n so no FK columns or relation tables are built;
child tables with no other parents are pruned from the model.
Three tests cover a skipped scalar column, a skipped rel1 (including
child-table pruning), and a DB insert with a skipped field.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds optional 'comment' field to itemtype in orders.xsd, skips it in all three versions' item config, and adds <comment> to order1a.xml. The equivalent_xml test now verifies that a file WITH the skipped field and one WITHOUT produce identical flat data — the core contract of skip. Also removes test_field_skip_column and test_field_skip_db (superseded by the DDL snapshot and roundtrip tests), keeping only test_field_skip_relation which tests the unique relation-skip and child-table-pruning behaviour. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
API fixes (breaking — minor bump)
Four bugs in the public API that warranted a minor version increment:
db_identifierdead parameter removed —temp_prefix: bool = FalseonDatabaseDialect.db_identifierwas never passed by any caller; the branch itguarded was also backwards (it added length instead of subtracting it). The parameter is removed.
Table config validation now works —
DataModel._build_modelwas validating user-provided table names againstself.model_config.get("tables", {}),which is always empty because
_validate_configstrips the"tables"key. The fix usesself.tables_configinstead. Before this fix, typos in table namessilently passed without raising
DataModelConfigError.DataModelColumn.model_configrenamed toconfig— the attribute held a table-level config dict, not the top-level model config, making the nameactively misleading. All internal references updated.
XMLConverter.parse_xmldefault corrected —skip_validationdefaulted toFalse(validate every document) despite every caller passingTrue. Thedefault is now
True, matching the documented and expected behaviour.New features
rename— a new field-level config option that sets the physical database column name while keeping the original XSD name as the internal logical key:All internal machinery (data dicts, FK lookups, bulk insert, merge statements) continues to use the original logical name; only the visible DB column name changes. Works on plain columns and on elevated fields (use the post-elevation prefixed name as the config key).
transform: "skip"— completely excludes a field from the data model:A skipped column is absent from the DB schema, from fields_transforms (so the XML converter ignores it naturally via the existing not in fields_transforms guard), and from parsed records. A skipped relation is also removed from relations_1/relations_n so no FK columns or relation tables are created; if the child table has no other parents it is pruned from the model entirely. Note: skip is intentionally incompatible with round-trip XML reconstruction.
Documentation
docs/configuring.mdgains two new subsections under Fields configuration: Renaming columns and Skipping fields, with examples including the elevated-field case and a warning about skip's round-trip incompatibility.