Skip to content

Fix dotnet test --filter for FullyQualifiedName and DisplayName#548

Closed
niclasahlqvist wants to merge 1 commit into
machine:masterfrom
niclasahlqvist:fix/test-case-filter-support
Closed

Fix dotnet test --filter for FullyQualifiedName and DisplayName#548
niclasahlqvist wants to merge 1 commit into
machine:masterfrom
niclasahlqvist:fix/test-case-filter-support

Conversation

@niclasahlqvist

Copy link
Copy Markdown
Contributor

Summary

dotnet test --filter has been broken for FullyQualifiedName and DisplayName properties. Tests show up in --list-tests but --filter "FullyQualifiedName~MySpec" returns "No test matches". This fixes both.

Relates to #349, #327

Root Cause

Two bugs in SpecificationFilterProvider:

1. Dictionary keyed by TestProperty.Id instead of TestProperty.Label

The supportedProperties array and property lookup dictionary used TestCaseProperties.FullyQualifiedName.Id ("TestCase.FullyQualifiedName") as keys, but the vstest filter framework passes the Label ("FullyQualifiedName") to the property value provider callback. The lookup silently returned null for every test, filtering everything out.

Fix: Added both .Id and .Label as dictionary keys for backwards compatibility.

2. TestObject.GetPropertyValue() returns empty for well-known properties

testCase.GetPropertyValue(TestCaseProperties.DisplayName) returns empty string even though testCase.DisplayName has the correct value. This appears to be a vstest TestObject property bag issue across ObjectModel assembly versions — the property is set via the DisplayName setter but isn't reliably retrievable through the generic property bag.

Fix: Replaced the TestObject property bag lookup with direct TestCase property access for FullyQualifiedName and DisplayName. Trait-based properties (Tag, Subject, ClassName) continue to use the existing Traits collection lookup, which was already working.

Verified Filters

All tested against a real project with 2400+ MSpec tests:

Filter Example Status
FullyQualifiedName --filter "FullyQualifiedName~MySpec" ✅ Fixed
DisplayName --filter "DisplayName~my spec" ✅ Fixed
ClassName --filter "ClassName~MyClass" ✅ Already worked
Subject --filter "Subject~My Subject" ✅ Already worked
Tag --filter "Tag~mytag" ✅ Already worked

Changes

  • SpecificationFilterProvider.cs — rewrote GetPropertyValue to read well-known properties directly from TestCase, added Label keys to dictionary
  • No other files changed
  • Backwards compatible — existing filter behavior (Tag, Subject) unchanged

Two issues prevented test filtering from working:

1. The supportedProperties array and property lookup dictionary were keyed
   by TestProperty.Id (e.g. "TestCase.FullyQualifiedName") but the vstest
   filter framework passes the Label (e.g. "FullyQualifiedName"). Added
   both Id and Label as dictionary keys.

2. TestObject.GetPropertyValue() returns empty string for well-known
   properties like DisplayName across ObjectModel assembly versions.
   Replaced the property bag lookup with direct TestCase property access
   for FullyQualifiedName and DisplayName.

Trait-based filters (Tag, Subject, ClassName) were already working.
@niclasahlqvist

Copy link
Copy Markdown
Contributor Author

CI note: The build step succeeds, all net8.0 tests pass (423 specs, 0 failures). The test target fails because netcoreapp3.1 and net472 test projects abort — the CI runner doesn't have those runtimes installed. This is a pre-existing issue unrelated to this change.

@niclasahlqvist

Copy link
Copy Markdown
Contributor Author

Not sure how active this repo is anymore so I'll throw in a ping @robertcoltheart

@robertcoltheart

Copy link
Copy Markdown
Member

Closing in favor of machine/machine.specifications.runner.visualstudio#175

@niclasahlqvist

Copy link
Copy Markdown
Contributor Author

@robertcoltheart Alright - will there be a nuget release aswell?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants