Add Yarn to the VS2017 Hosted agent

[amitse]

Please consider adding yarn to the VS2017 Hosted agent.

Currently we've following alternatives

• https://marketplace.visualstudio.com/items?itemName=geeklearningio.gl-vsts-tasks-yarn (Still in preview, can't use it for production)
• npm install yarn is also not a good idea as the package won't be signed, and the only integrity check performed is a basic SHA1 hash, which is a security risk

Thanks

[bryanmacfarlane]

You can also do curl download. Only npm one isn't signed

https://yarnpkg.com/lang/en/docs/install/#alternatives-tab

We won't be adding all the misc tools to the hosted convenience image. That image is VS2017 full install.

The other alternative is a private agent on your own VM

[amitse]

@bryanmacfarlane : Do we have curl installed in Hosted VS2017?

If i check list of software installed in Hosted 2017,
I don't see curl in that list.

The reason we are moving to yarn is because bower will soon be discontinued and, as an alternate bower's developers are recommending switching to yarn. bower readme.md

Hosted agent 2017 has bower installed out of the box, that's where we were hoping to get yarn installed before bower is discontinued.

[jawn]

@amitse
Hosted VS2017 has PowerShell 5. In Powershell 5.0, curl is an alias for Invoke-WebRequest.

[kachkaev]

I tried installing yarn on a VSTS Hosted agent, but it did not work. When I pick a Ubuntu 16.04 machine, npm install --global yarn fails because I don't have write access to global node_modules dir:

Error: EACCES: permission denied, access '/usr/local/lib/node_modules'

Installing Yarn on a standard Hosted agent does not work too because it does not come with choco:

'choco' is not recognized as an internal or external command, operable program or batch file.
##[error]Cmd.exe exited with code '1'.

On a Hosted VS2017 agent, choco.exe install yarn --ignore-dependencies kind of works

choco install log

2018-08-09T09:22:06.2369403Z Chocolatey v0.10.11
2018-08-09T09:22:19.7648542Z Installing the following packages:
2018-08-09T09:22:19.7656731Z yarn
2018-08-09T09:22:19.7661972Z By installing you accept licenses for the packages.
2018-08-09T09:22:24.5428846Z 
2018-08-09T09:22:24.5430437Z Progress: Downloading yarn 1.9.4... 14%
2018-08-09T09:22:24.5431474Z Progress: Downloading yarn 1.9.4... 43%
2018-08-09T09:22:24.5433136Z Progress: Downloading yarn 1.9.4... 72%
2018-08-09T09:22:24.5434411Z Progress: Downloading yarn 1.9.4... 100%
2018-08-09T09:22:25.1291110Z 
2018-08-09T09:22:25.1291762Z yarn v1.9.4 [Approved]
2018-08-09T09:22:25.1611908Z yarn package files install completed. Performing other installation steps.
2018-08-09T09:22:35.9362170Z Downloading yarn 
2018-08-09T09:22:35.9363272Z   from 'https://yarnpkg.com/downloads/1.9.4/yarn-1.9.4.msi'
2018-08-09T09:22:36.9289981Z 
2018-08-09T09:22:37.2199968Z Progress: 6% - Saving 84.58 KB of 1.28 MB
2018-08-09T09:22:37.3996112Z Progress: 12% - Saving 169.58 KB of 1.28 MB
2018-08-09T09:22:37.4898590Z Progress: 19% - Saving 254.58 KB of 1.28 MB
2018-08-09T09:22:37.6694176Z Progress: 25% - Saving 339.58 KB of 1.28 MB
2018-08-09T09:22:37.7596466Z Progress: 32% - Saving 424.58 KB of 1.28 MB
2018-08-09T09:22:37.8499126Z Progress: 38% - Saving 509.58 KB of 1.28 MB
2018-08-09T09:22:37.9399497Z Progress: 45% - Saving 594.58 KB of 1.28 MB
2018-08-09T09:22:38.0295770Z Progress: 51% - Saving 679.58 KB of 1.28 MB
2018-08-09T09:22:38.1195806Z Progress: 58% - Saving 764.58 KB of 1.28 MB
2018-08-09T09:22:38.2093417Z Progress: 64% - Saving 849.58 KB of 1.28 MB
2018-08-09T09:22:38.2532668Z Progress: 71% - Saving 934.58 KB of 1.28 MB
2018-08-09T09:22:38.3012373Z Progress: 77% - Saving 1019.58 KB of 1.28 MB
2018-08-09T09:22:38.3901439Z Progress: 84% - Saving 1.08 MB of 1.28 MB
2018-08-09T09:22:38.4329244Z Progress: 90% - Saving 1.16 MB of 1.28 MB
2018-08-09T09:22:38.4812753Z Progress: 97% - Saving 1.24 MB of 1.28 MB
2018-08-09T09:22:38.4816849Z Progress: 100% - Completed download of C:\Users\VssAdministrator\AppData\Local\Temp\chocolatey\yarn\1.9.4\yarn-1.9.4.msi (1.28 MB).
2018-08-09T09:22:38.4914841Z Download of yarn-1.9.4.msi (1.28 MB) completed.
2018-08-09T09:22:41.6368975Z Hashes match.
2018-08-09T09:22:41.6826615Z Installing yarn...
2018-08-09T09:22:48.4040212Z yarn has been installed.
2018-08-09T09:22:55.5131823Z #< CLIXML
2018-08-09T09:22:55.5257848Z <Objs Version="1.1.0.1" xmlns="http://schemas.microsoft.com/powershell/2004/04"><Obj S="progress" RefId="0"><TN RefId="0"><T>System.Management.Automation.PSCustomObject</T><T>System.Object</T></TN><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><Obj S="progress" RefId="1"><TNRef RefId="0" /><MS><I64 N="SourceId">1</I64><PR N="Record"><AV>Preparing modules for first use.</AV><AI>0</AI><Nil /><PI>-1</PI><PC>-1</PC><T>Completed</T><SR>-1</SR><SD> </SD></PR></MS></Obj><S S="debug">Host version is 5.1.14393.2248, PowerShell Version is '5.1.14393.2248' and CLR Version is '4.0.30319.42000'.</S><S S="verbose">Exporting function 'Format-FileSize'.</S><S S="verbose">Exporting function 'Get-ChecksumValid'.</S><S S="verbose">Exporting function 'Get-ChocolateyUnzip'.</S><S S="verbose">Exporting function 'Get-ChocolateyWebFile'.</S><S S="verbose">Exporting function 'Get-EnvironmentVariable'.</S><S S="verbose">Exporting function 'Get-EnvironmentVariableNames'.</S><S S="verbose">Exporting function 'Get-FtpFile'.</S><S S="verbose">Exporting function 'Get-OSArchitectureWidth'.</S><S S="verbose">Exporting function 'Get-PackageParameters'.</S><S S="verbose">Exporting function 'Get-PackageParametersBuiltIn'.</S><S S="verbose">Exporting function 'Get-ToolsLocation'.</S><S S="verbose">Exporting function 'Get-UACEnabled'.</S><S S="verbose">Exporting function 'Get-UninstallRegistryKey'.</S><S S="verbose">Exporting function 'Get-VirusCheckValid'.</S><S S="verbose">Exporting function 'Get-WebFile'.</S><S S="verbose">Exporting function 'Get-WebFileName'.</S><S S="verbose">Exporting function 'Get-WebHeaders'.</S><S S="verbose">Exporting function 'Install-BinFile'.</S><S S="verbose">Exporting function 'Install-ChocolateyDesktopLink'.</S><S S="verbose">Exporting function 'Install-ChocolateyEnvironmentVariable'.</S><S S="verbose">Exporting function 'Install-ChocolateyExplorerMenuItem'.</S><S S="verbose">Exporting function 'Install-ChocolateyFileAssociation'.</S><S S="verbose">Exporting function 'Install-ChocolateyInstallPackage'.</S><S S="verbose">Exporting function 'Install-ChocolateyPackage'.</S><S S="verbose">Exporting function 'Install-ChocolateyPath'.</S><S S="verbose">Exporting function 'Install-ChocolateyPinnedTaskBarItem'.</S><S S="verbose">Exporting function 'Install-ChocolateyPowershellCommand'.</S><S S="verbose">Exporting function 'Install-ChocolateyShortcut'.</S><S S="verbose">Exporting function 'Install-ChocolateyVsixPackage'.</S><S S="verbose">Exporting function 'Install-ChocolateyZipPackage'.</S><S S="verbose">Exporting function 'Install-Vsix'.</S><S S="verbose">Exporting function 'Set-EnvironmentVariable'.</S><S S="verbose">Exporting function 'Set-PowerShellExitCode'.</S><S S="verbose">Exporting function 'Start-ChocolateyProcessAsAdmin'.</S><S S="verbose">Exporting function 'Test-ProcessAdminRights'.</S><S S="verbose">Exporting function 'Uninstall-BinFile'.</S><S S="verbose">Exporting function 'Uninstall-ChocolateyEnvironmentVariable'.</S><S S="verbose">Exporting function 'Uninstall-ChocolateyPackage'.</S><S S="verbose">Exporting function 'Uninstall-ChocolateyZipPackage'.</S><S S="verbose">Exporting function 'Update-SessionEnvironment'.</S><S S="verbose">Exporting function 'Write-ChocolateyFailure'.</S><S S="verbose">Exporting function 'Write-ChocolateySuccess'.</S><S S="verbose">Exporting function 'Write-FileUpdateLog'.</S><S S="verbose">Exporting function 'Write-FunctionCallLogMessage'.</S><S S="verbose">Exporting alias 'Get-ProcessorBits'.</S><S S="verbose">Exporting alias 'Get-OSBitness'.</S><S S="verbose">Exporting alias 'Get-InstallRegistryKey'.</S><S S="verbose">Exporting alias 'Generate-BinFile'.</S><S S="verbose">Exporting alias 'Add-BinFile'.</S><S S="verbose">Exporting alias 'Start-ChocolateyProcess'.</S><S S="verbose">Exporting alias 'Invoke-ChocolateyProcess'.</S><S S="verbose">Exporting alias 'Remove-BinFile'.</S><S S="verbose">Exporting alias 'refreshenv'.</S><S S="debug">Loading community extensions</S><S S="debug">Importing 'C:\ProgramData\chocolatey\extensions\chocolatey-core\chocolatey-core.psm1'</S><S S="verbose">Loading module from path 'C:\ProgramData\chocolatey\extensions\chocolatey-core\chocolatey-core.psm1'.</S><S S="verbose">Exporting function 'Get-UninstallRegistryKey'.</S><S S="verbose">Exporting function 'Get-AppInstallLocation'.</S><S S="verbose">Exporting function 'Get-AvailableDriveLetter'.</S><S S="verbose">Exporting function 'Get-EffectiveProxy'.</S><S S="verbose">Exporting function 'Get-PackageCacheLocation'.</S><S S="verbose">Exporting function 'Get-PackageParameters'.</S><S S="verbose">Exporting function 'Get-WebContent'.</S><S S="verbose">Exporting function 'Register-Application'.</S><S S="verbose">Importing function 'Get-AppInstallLocation'.</S><S S="verbose">Importing function 'Get-AvailableDriveLetter'.</S><S S="verbose">Importing function 'Get-EffectiveProxy'.</S><S S="verbose">Importing function 'Get-PackageCacheLocation'.</S><S S="verbose">Importing function 'Get-PackageParameters'.</S><S S="verbose">Importing function 'Get-UninstallRegistryKey'.</S><S S="verbose">Importing function 'Get-WebContent'.</S><S S="verbose">Importing function 'Register-Application'.</S><S S="verbose">Exporting function 'Format-FileSize'.</S><S S="verbose">Exporting function 'Get-ChecksumValid'.</S><S S="verbose">Exporting function 'Get-ChocolateyUnzip'.</S><S S="verbose">Exporting function 'Get-ChocolateyWebFile'.</S><S S="verbose">Exporting function 'Get-EnvironmentVariable'.</S><S S="verbose">Exporting function 'Get-EnvironmentVariableNames'.</S><S S="verbose">Exporting function 'Get-FtpFile'.</S><S S="verbose">Exporting function 'Get-OSArchitectureWidth'.</S><S S="verbose">Exporting function 'Get-PackageParameters'.</S><S S="verbose">Exporting function 'Get-PackageParametersBuiltIn'.</S><S S="verbose">Exporting function 'Get-ToolsLocation'.</S><S S="verbose">Exporting function 'Get-UACEnabled'.</S><S S="verbose">Exporting function 'Get-UninstallRegistryKey'.</S><S S="verbose">Exporting function 'Get-VirusCheckValid'.</S><S S="verbose">Exporting function 'Get-WebFile'.</S><S S="verbose">Exporting function 'Get-WebFileName'.</S><S S="verbose">Exporting function 'Get-WebHeaders'.</S><S S="verbose">Exporting function 'Install-BinFile'.</S><S S="verbose">Exporting function 'Install-ChocolateyDesktopLink'.</S><S S="verbose">Exporting function 'Install-ChocolateyEnvironmentVariable'.</S><S S="verbose">Exporting function 'Install-ChocolateyExplorerMenuItem'.</S><S S="verbose">Exporting function 'Install-ChocolateyFileAssociation'.</S><S S="verbose">Exporting function 'Install-ChocolateyInstallPackage'.</S><S S="verbose">Exporting function 'Install-ChocolateyPackage'.</S><S S="verbose">Exporting function 'Install-ChocolateyPath'.</S><S S="verbose">Exporting function 'Install-ChocolateyPinnedTaskBarItem'.</S><S S="verbose">Exporting function 'Install-ChocolateyPowershellCommand'.</S><S S="verbose">Exporting function 'Install-ChocolateyShortcut'.</S><S S="verbose">Exporting function 'Install-ChocolateyVsixPackage'.</S><S S="verbose">Exporting function 'Install-ChocolateyZipPackage'.</S><S S="verbose">Exporting function 'Install-Vsix'.</S><S S="verbose">Exporting function 'Set-EnvironmentVariable'.</S><S S="verbose">Exporting function 'Set-PowerShellExitCode'.</S><S S="verbose">Exporting function 'Start-ChocolateyProcessAsAdmin'.</S><S S="verbose">Exporting function 'Test-ProcessAdminRights'.</S><S S="verbose">Exporting function 'Uninstall-BinFile'.</S><S S="verbose">Exporting function 'Uninstall-ChocolateyEnvironmentVariable'.</S><S S="verbose">Exporting function 'Uninstall-ChocolateyPackage'.</S><S S="verbose">Exporting function 'Uninstall-ChocolateyZipPackage'.</S><S S="verbose">Exporting function 'Update-SessionEnvironment'.</S><S S="verbose">Exporting function 'Write-ChocolateyFailure'.</S><S S="verbose">Exporting function 'Write-ChocolateySuccess'.</S><S S="verbose">Exporting function 'Write-FileUpdateLog'.</S><S S="verbose">Exporting function 'Write-FunctionCallLogMessage'.</S><S S="verbose">Exporting function 'Get-AppInstallLocation'.</S><S S="verbose">Exporting function 'Get-AvailableDriveLetter'.</S><S S="verbose">Exporting function 'Get-EffectiveProxy'.</S><S S="verbose">Exporting function 'Get-PackageCacheLocation'.</S><S S="verbose">Exporting function 'Get-WebContent'.</S><S S="verbose">Exporting function 'Register-Application'.</S><S S="verbose">Exporting alias 'Get-ProcessorBits'.</S><S S="verbose">Exporting alias 'Get-OSBitness'.</S><S S="verbose">Exporting alias 'Get-InstallRegistryKey'.</S><S S="verbose">Exporting alias 'Generate-BinFile'.</S><S S="verbose">Exporting alias 'Add-BinFile'.</S><S S="verbose">Exporting alias 'Start-ChocolateyProcess'.</S><S S="verbose">Exporting alias 'Invoke-ChocolateyProcess'.</S><S S="verbose">Exporting alias 'Remove-BinFile'.</S><S S="verbose">Exporting alias 'refreshenv'.</S></Objs>
2018-08-09T09:22:55.5289463Z 0
2018-08-09T09:22:55.6070314Z Only an exit code of non-zero will fail the package by default. Set 
2018-08-09T09:22:55.6070839Z  `--failonstderr` if you want error messages to also fail a script. See 
2018-08-09T09:22:55.6071168Z  `choco -h` for details.
2018-08-09T09:22:55.8768763Z   yarn may be able to be automatically uninstalled.
2018-08-09T09:22:55.8818509Z Environment Vars (like PATH) have changed. Close/reopen your shell to
2018-08-09T09:22:55.8818796Z  see the changes (or in powershell/cmd.exe just type `refreshenv`).
2018-08-09T09:22:56.0960242Z  The install of yarn was successful.
2018-08-09T09:22:56.0964652Z   Software installed as 'msi', install location is likely default.
2018-08-09T09:22:56.2056223Z 
2018-08-09T09:22:56.2056964Z Chocolatey installed 1/1 packages. 
2018-08-09T09:22:56.2057296Z  See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

  
However, running yarn install afterwards still fails (using https://marketplace.visualstudio.com/items?itemName=geeklearningio.gl-vsts-tasks-yarn):

##[error]Error: couldn't locate Yarn

How do you folks run yarn on Hosted agents (both Windows and Ubuntu)?

---

UPD: Replacing the custom yarn task with a simple cmd call to yarn install does not help find it:

'yarn' is not recognized as an internal or external command, operable program or batch file.

[kachkaev]

I was finally able to get yarn on the Hosted by using a custom task:

Still not sure how to install it manually (via bash / cmd) – that'd be useful to know.

[thomasdc]

@kachkaev You're correct, I had the exact same issue. I assume the environment variables aren't refreshed correctly after installing yarn through chocolatey. I switched to npm because this was taking way to long to get it working...

[kachkaev]

We can't switch to npm as we're relying on yarn.lock files and also use yarn workspaces. We also pay Microsoft for several Hosted agents, so going for a self-hosted one just because of yarn feels mad 😅

Yarn these days seems to be as popular as npm, so it'd be great to see it in VSTS Hosted agents by default. Docker node.js images used to only have node and npm originally, but they added yarn too about a year ago to satisfy the demand (docker-library/official-images#2705).