You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Feb 25, 2026. It is now read-only.
Repository navigation
This repository was archived by the owner on Feb 25, 2026. It is now read-only.
[Feature] Smoke Test Pipeline Should Report Metrics on Application Sizes #3600
Besides just checking the final output of the NuGet package, one reason for the Smoke Tests are to help us understand the impact the toolkit packages have on the final size of an application.
While the Smoke Tests currently run in the CI and generate these independent apps. The size isn't currently reported back in a nicely consumable way either at the end of that job or with some sort of bot back to the issue itself.
Describe the solution
MVP: When smoke test is completed, it lists a table at the end of each package and the resulting app size to the console. This way we can scroll to the end of the job and get that data when needed.
Nice to haves:
Compares against some well-known cache from the last build on the main branch so we can get comparison +/- with the current code base
Have a bot report that table back to the PR in a comment.
Additional context & Screenshots
Somewhat related to #3560 as well. And to address being able to check and help with #3145 more explicitly.
Alright so the SmokeTests drop to SmokeTests\AppPackages and the final msixupload packages have a good breakdown of all the dependent dlls.
Each msixupload package has an inner msixbundle package and then an inner msix package, if we unzip all 3 of these, we can get a flat list of all the dlls required for the app. (Seems like there's a lot from the system, so need to check the new baseline version.) Then we can investigate what the extra dependencies are for each package besides the baseline and spit that out.
I think the strategy here I'm going to follow is to index all the files/sizes in the UWP Baseline project. Then for each other project, find the outliers to that list and report what those are and their sizes above the baseline.
This should give us not only the overall footprint each package adds to the toolkit, but it should also help us understand other dependencies each of those packages bring and how they contribute to that total.
I'll then try and output that to the console. Luckily this is something I can easily test locally. I'm going to make it a separate PowerShell script in the SmokeTests project, and then we can just call it from the DevOps YAML file after the Smoke Tests run.
@azchohfi do you know if there's a known location where we can store a file/metadata associated the last main build in DevOps or would we have to check it in somewhere to some repo? Curious if in the future we could have the script fetch that from somewhere and compare against it to know how things are changing for a given PR?
Not sure what the best way to do this would be. We could upload an xml or json file with these results somewhere and use that, but I don't know if we could just use the same repo for that. Not sure if we could use the ADO artifacts for that either, since there is a retention policy for builds. Maybe a release pipeline could push this to an Azure storage, and then the PRs pipelines could compare it with a known URL. I'm not sure if there is a more standard way of doing this. Lets brainstorm.
@azchohfi Hmm, was just messing with the DownloadPipelineArtifact wondering if we published it as an Artifact if we could use the specific flag to somehow grab the latest pipeline artifact from the main branch? Not sure if we can get that specific with the project guid? runVersionlatestFromBranch maybe? 🤔 That looks promising. I can try it out later once I get the base script running.
Need to test in the CI. Some dependencies I didn't quite realize were so big and rooted lower in the Toolkit like System.Text.Json is large and everywhere and it pulls in another one. Also have no idea what's pulling in the Microsoft.Bcl.AsyncInterfaces dependency...?
Activity
michael-hawker commentedon Dec 10, 2020
Alright so the SmokeTests drop to
SmokeTests\AppPackagesand the final msixupload packages have a good breakdown of all the dependent dlls.Each
msixuploadpackage has an innermsixbundlepackage and then an innermsixpackage, if we unzip all 3 of these, we can get a flat list of all the dlls required for the app. (Seems like there's a lot from the system, so need to check the new baseline version.) Then we can investigate what the extra dependencies are for each package besides the baseline and spit that out.Add a UWP Baseline project to the Smoke Tests for size comparison Com…
michael-hawker commentedon Dec 11, 2020
I think the strategy here I'm going to follow is to index all the files/sizes in the UWP Baseline project. Then for each other project, find the outliers to that list and report what those are and their sizes above the baseline.
This should give us not only the overall footprint each package adds to the toolkit, but it should also help us understand other dependencies each of those packages bring and how they contribute to that total.
I'll then try and output that to the console. Luckily this is something I can easily test locally. I'm going to make it a separate PowerShell script in the SmokeTests project, and then we can just call it from the DevOps YAML file after the Smoke Tests run.
@azchohfi do you know if there's a known location where we can store a file/metadata associated the last main build in DevOps or would we have to check it in somewhere to some repo? Curious if in the future we could have the script fetch that from somewhere and compare against it to know how things are changing for a given PR?
azchohfi commentedon Dec 11, 2020
Not sure what the best way to do this would be. We could upload an xml or json file with these results somewhere and use that, but I don't know if we could just use the same repo for that. Not sure if we could use the ADO artifacts for that either, since there is a retention policy for builds. Maybe a release pipeline could push this to an Azure storage, and then the PRs pipelines could compare it with a known URL. I'm not sure if there is a more standard way of doing this. Lets brainstorm.
michael-hawker commentedon Dec 11, 2020
@azchohfi Hmm, was just messing with the
DownloadPipelineArtifactwondering if we published it as an Artifact if we could use thespecificflag to somehow grab the latest pipeline artifact from the main branch? Not sure if we can get that specific with theprojectguid?runVersionlatestFromBranchmaybe? 🤔 That looks promising. I can try it out later once I get the base script running.Ref: https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/download-pipeline-artifact?view=azure-devops
azchohfi commentedon Dec 11, 2020
Not sure, but it seems like it might work.
michael-hawker commentedon Dec 11, 2020
Looking good so far:
Need to test in the CI. Some dependencies I didn't quite realize were so big and rooted lower in the Toolkit like
System.Text.Jsonis large and everywhere and it pulls in another one. Also have no idea what's pulling in theMicrosoft.Bcl.AsyncInterfacesdependency...?michael-hawker commentedon Dec 12, 2020
Right now it just outputs to the host, need to add more to actually spit out a file that we could compare against.
When we do that, we can also output this cleaner as a table and have an extra column for the comparison.
And we could colorize it fancier too! https://stackoverflow.com/a/49038815/8798708
Add initial script and step to SmokeTest job to analyze dependencies …
Add initial script and step to SmokeTest job to analyze dependencies …
Split out Smoke Tests and Unit Tests into a separate Jobs (#3617)