Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MSTest SDK document limitations #43717

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

MSTest SDK document limitations #43717

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
d52620e
MSTest SDK document limitations
Evangelink Nov 22, 2024
c9f58f4
Update docs/core/testing/unit-testing-mstest-sdk.md
Evangelink Nov 24, 2024
98e952f
Apply suggestions from code review
Evangelink Nov 25, 2024
0874837
Update docs/core/testing/unit-testing-mstest-sdk.md
Evangelink Nov 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/core/testing/unit-testing-mstest-sdk.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,14 @@ Here's an example update when using the `DotNetCoreCLI` task in Azure DevOps:
+ arguments: '--configuration Release -p:TestingPlatformCommandLineArguments="--report-trx --results-directory $(Agent.TempDirectory) --coverage"'
```

## Known limitations

The MSTest SDK has limited integration with NuGet, meaning that the usual NuGet update and Visual Studio UI for managing NuGet packages **DOES NOT** work as expected. See this issue for more details: [NuGet#13127](https://github.com/NuGet/Home/issues/13127).
Copy link
Contributor

@adegeo adegeo Nov 27, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If someone is here viewing this section, which is Known limitations, it's not really beneficial to try and use CAPS to emphasize something that is a known limitation. Someone taking the time to read this paragraph will understand the context. It's not hidden in a sea of paragraphs and tables, it's the first thing mentioned. If anything, I would use this instead:

Suggested change
The MSTest SDK has limited integration with NuGet, meaning that the usual NuGet update and Visual Studio UI for managing NuGet packages **DOES NOT** work as expected. See this issue for more details: [NuGet#13127](https://github.com/NuGet/Home/issues/13127).
The MSTest SDK has limited integration with NuGet, meaning that the usual NuGet update and Visual Studio UI for managing NuGet packages **doesn't** work as expected. See this issue for more details: [NuGet#13127](https://github.com/NuGet/Home/issues/13127).

But honestly this isn't very clear about the limitation. As a user, I don't want to have to go click on a link and read through comment-after-comment to understand the limitation. Just tell me the limitation and what I need to do to work around it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adegeo Could you please suggest a wording. Basically, NuGet integration is busted so you cannot use VS/VSCode or NuGet CLIs to manage the update of the package and will instead need to do it manually or rely on dependabot.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Evangelink Sorry for the delay, I was out sick. If you understand the problem/limitation, can you reiterate it here in a response to myself? I'll take that info and provide a suggestion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No worries. As I was saying in the previous comment, the limitation is that you cannot use existing tooling to install or update the SDK package (VS nuget package manager won't work, dotnet nuget won't work...). You have to update manually or rely on dependabot.

This problem is not limited to MSTest but is true for any custom MSBuild sdk

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dependabot suggestion seems confusing. I don't know how someone would use dependabot, I thought that was just a github thing? Like if I send a copy of my project to a friend through email, what does he do to restore the packages and run the project? How does he actually manually install the required SDK files?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok let me list scenario:

1/ project is configured to be using MSTest SDK, you open it in VS/VS code or build it in cli then all is good. Restore is working properly, build is working you are all set.

2/ there is a new version of the SDK or some packages referenced by the SDK. You open VS nuget package manager and see only updates for the referenced packages not the SDK. When trying to update those packages they are added to your project but the SDK itself isn't updated and user can only update manually. The story is the same for VS code or cli.

3/ project isn't using MSTest SDK and you want to use it. Then you have to manually update your project files.

For the part about dependabot, yes this is GH only feature but it doesn't understand custom SDK where the version is declared into global.json so if you are using the SDK with version defined there and your code is on GH with dependabot configured, you would get bot PR bumping the version.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes this is GH only feature but it doesn't understand custom SDK where the version is declared into global.json

@Evangelink I think you meant it does understand custom SDK?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Evangelink @adegeo What do you think of this wording? Does it make it clearer?

Suggested change
The MSTest SDK has limited integration with NuGet, meaning that the usual NuGet update and Visual Studio UI for managing NuGet packages **DOES NOT** work as expected. See this issue for more details: [NuGet#13127](https://github.com/NuGet/Home/issues/13127).
The NuGet-provided MSBuild Sdks (including MSTest.Sdk) have limited tooling support when it comes to updating its version, meaning that the usual NuGet update and Visual Studio UI for managing NuGet packages does not work as expected. See this issue for more details: [NuGet#13127](https://github.com/NuGet/Home/issues/13127).


> [!NOTE]
> This limitation is not specific to MSTest SDK but to any NuGet-provided MSBuild SDK.
> Dependabot will handle updating the version in the `global.json` file, but you will need to [manually update the version in the project file](https://github.com/dependabot/dependabot-core/issues/8615).

## See also

- [Test project–related properties](../project-sdk/msbuild-props.md#test-projectrelated-properties)
Loading