Programmatically connecting to Azure DevOps with a Service Principal (Management Group)


Continuing from my last post of programmatically connecting to Azure DevOps with a Service Principal at subscription level, I also wanted to show how you can create a DevOps service connection programmatically at a Management Group level.

Unlike the subscription level, you cannot just uses Az DevOps command with a management group parameter. This does not appear to be available, the answer is to pass in a json template.

You will need a few things already configured:

* See code in the powershell folder from the post on https://cann0nf0dder.wordpress.com/2020/09/14/app-only-auth-connect-to-sharepoint-online-with-msal-and-azure-keyvault/ to see how you can create this programatically.

Management Groups Enabled

  • Click on Start using management groups. This will create your “Tenant Root Group” and apply your subscriptions to the management group.

The above will set you up to walk through this demo, however, please ensure you understand what Management groups are, and how to use. https://docs.microsoft.com/en-us/azure/governance/management-groups/overview

‘User Access Administrator’ access.

On the above picture, you can see next to the words Tenant Root Group a link (details). You probably do not have the details link clickable at this time. This is because although you have been able to create the initial Tenant Root Group – Management Group, you need to promote your account access to it.

Note: You can only do this as a Global Administrator.

Manually

  • Go to your Azure Portal https://portal.azure.com
  • Go to Azure Active Directory
  • In the left hand navigation under Manage, click Properties
  • Under Access management for Azure resources switch the button to Yes.

Now if you go back to the Tenant Root Group – Management Group, you will be able to click the details link and have access to the Management group, see deployments made at that level, modify access for others etc.

Switching the button to No will then remove your access.

Programatically

Using Az Cli, log in with you account first az login. The below snippet, on line 2 shows how to give yourself access. Where line 5 & 6 would remove the account.

#Give access
az rest –method post –uri 'https://management.azure.com/providers/Microsoft.Authorization/elevateAccess?api-version=2015-07-01'
#Remove access
$account = username@example.com
az role assignment delete –assignee $account –role 'User Access Administrator' –scope '/'

The Code

Running of this code, will create a DevOps project if it doesn’t exist, and then create a Management Group level Service Connection to the Tenant Root Management Group. To apply to a different level management group would require modification to the code to grab the name and ID of the management group you wish to use and pass into the JSON template.

Your account is now set up to run, you will need to first be logged into AZ Cli.

Note: This can be a Service Principal, as long as the account being used is able to list App Registrations, and has ‘User Access Administrator’ RBAC on the Tenant Root Group – Management Group.

You will need to create a ‘management-group.json’ file which is used as a template, and key tokens will be replaced within the script.

{
"administratorsGroup": null,
"authorization": {
"scheme": "ServicePrincipal",
"parameters": {
"serviceprincipalid": "##ServicePrincipalId##",
"authenticationType": "spnKey",
"serviceprincipalkey": "##ServicePrincipalKey##",
"tenantid": "##TenantId##"
}
},
"createdBy": null,
"data": {
"environment": "AzureCloud",
"scopeLevel": "ManagementGroup",
"managementGroupId": "##ManagementGroupId##",
"managementGroupName": "##ManagementGroupName##"
},
"description": "Management Group Service Connection",
"groupScopeId": null,
"name": "##Name##",
"operationStatus": null,
"readersGroup": null,
"serviceEndpointProjectReferences": [
{
"description": "Management Group Service Connection",
"name": "##Name##",
"projectReference": {
"id": "##ProjectId##",
"name": "##ProjectName##"
}
}
],
"type": "azurerm",
"url": "https://management.azure.com/",
"isShared": false,
"owner": "library"
}

In the code below important parts to note:

(Line 32) – How the authentication works with DevOps. The Personal Access Token is added to the $Env: variable “AZURE_DEVOPS_EXT_PAT”

(Line 61 – 74) – Updating the json template, saving the file as a temp file, and then creating the Service Connection passing in the json template. The json template is the same template used by Azure DevOps when you set up the Management Group Service connection manually, you can see this by watching the network traffic.

<#
.SYNOPSIS
Creates a service connection for a ManagementGroup
Please ensure you are already logged to azure using az login
#>
param(
# Azure DevOps Personal Access Token (PAT) for the 'https://dev.azure.com/%5BORG%5D&#39; Azure DevOps tenancy
[Parameter(Mandatory)]
[string]
$PersonalAccessToken,
# The Azure DevOps organisation to create the service connection in, available from System.TeamFoundationCollectionUri if running from pipeline.
[string]
$TeamFoundationCollectionUri = $($Env:System_TeamFoundationCollectionUri -replace '%20', ' '),
# The name of the project to which this build or release belongs, available from $(System.TeamProject) if running from pipeline
[string]
$TeamProject = $Env:System_TeamProject,
[string]
$AppRegistrationName,
[securestring]
$AppPassword
)
$ErrorActionPreference = 'Stop'
$InformationPreference = 'Continue'
#Clearing default.
az configure –defaults group=
$account = az account show | ConvertFrom-Json
$Env:AZURE_DEVOPS_EXT_PAT = $PersonalAccessToken
Write-Information -MessageData:"Adding Azure DevOps Extension…"
az extension add –name azure-devops
Write-Information -MessageData "Configure defaults Organization:$TeamFoundationCollectionUri…"
az devops configure –defaults organization="$TeamFoundationCollectionUri"
Write-Information -MessageData "Getting App Registration: $AppRegistrationName…"
$AppReg = az ad app list –all –query "[?displayName == '$AppRegistrationName']" | ConvertFrom-Json
Write-Information -MessageData "Give App Registration access to Management Group Root…"
az role assignment create –role "Owner" –assignee $($AppReg.appId) –scope "/"
Write-Information -MessageData "Checking if $TeamProject project exists…"
$ProjectDetails = az devops project list –query "value[?name == '$TeamProject']" | Select-Object -First 1 | ConvertFrom-Json
if(-not $ProjectDetails){
Write-Information -MessageData "Creating $TeamProject project…"
$ProjectDetails = az devops project create –name $TeamProject
}
Write-Information -MessageData "Checking if service endpoint already exists…"
$ServiceEndpoint = az devops service-endpoint list –project "$TeamProject" –query "[?name == '$($AppReg.DisplayName)-Mg']" | Select-Object -First 1 | ConvertFrom-Json
if (-not $ServiceEndpoint) {
Write-Information -MessageData "Getting Json file for Management Group…"
$managementGroupJson = Get-Content -Raw -Path "$PSScriptRoot/management-group.json"
$configFilePath = "$PSScriptRoot/temp-managementGroup.json"
$managementGroupJson = $managementGroupJson -replace '##TenantId##', $($Account.homeTenantId) `
-replace '##ManagementGroupId##', $($Account.homeTenantId) `
-replace '##ManagementGroupName##', "Tenant Root Group" `
-replace '##ServicePrincipalId##', $($AppReg.appId) `
-replace '##ServicePrincipalKey##', $(ConvertFrom-SecureString -SecureString:$AppPassword -AsPlainText) `
-replace '##Name##', "$($AppReg.DisplayName)-Mg" `
-replace '##ProjectId##', $($ProjectDetails.id) `
-replace '##ProjectName##', $($ProjectDetails.name)
Write-Information -MessageData "Saving management json file…"
Set-Content -Value:$managementGroupJson -Path:$configFilePath
Write-Information -MessageData "Creating Service Connection name:$($AppReg.DisplayName)-Mg for project $TeamProject…"
$ServiceEndpoint = az devops service-endpoint create –project "$TeamProject" –service-endpoint-configuration "$configFilePath" | ConvertFrom-Json
Write-Information -MessageData "Clean up temp files"
Remove-Item -Path $configFilePath
}
Write-Information -MessageData "Updating Service Connection to be enabled for all pipelines…"
az devops service-endpoint update –project "$TeamProject" –id "$($ServiceEndpoint.id)" –enable-for-all true | Out-Null

To run the above code, you will need to put in your parameters. Replace with your values then run the below script, this will call the script above.

$PersonalAccessToken = "<PAT TOKEN>"
$TeamProject = '<PROJECT NAME>'
$TeamFoundationCollectionUri = 'https://dev.azure.com/<organizationName >'
$AppRegistrationName = '<Service Principal Name>'
$AppPassword = '<Service Principal Secret>'
$AppSecurePassword = ConvertTo-SecureString -String:$AppPassword -AsPlainText -Force
.\Install-ServiceConnectionManagementGroup.ps1 -PersonalAccessToken $PersonalAccessToken `
-TeamFoundationCollectionUri:$TeamFoundationCollectionUri `
-TeamProject:$TeamProject `
-AppRegistrationName:$AppRegistrationName `
-AppPassword:$AppSecurePassword

My team project is called AutomateDevOpsMG, and I used an App Registration called DevOps.

Running Script

Service Principal with Owner access on Management Group Level

Project ‘AutomateDevOpsMG’ and Service connection ‘DevOps-Mg’

Programmatically connecting to Azure Devops with a Service Principal (Subscription)


A previous post of mine Connecting to Azure Devops with a Service Principal has been popular since I have written it. Therefore, I’ve decided to extend on the topic and show how you can do it programatically with AZ DevOps.

You will need a few things already configured:

* See code in the powershell folder from the post on https://cann0nf0dder.wordpress.com/2020/09/14/app-only-auth-connect-to-sharepoint-online-with-msal-and-azure-keyvault/ to see how you can create this programatically.

Create a DevOps PAT token

  • Go to your Azure devops https://dev.azure.com
  • Sign in and click on User settings -> Personal access tokens
  • Click New Token
    • Give it a meaningful name so you know what the PAT token is for in the future. (E.g, Devops Service Connection)
    • Select your Organization
    • Select the Expiration date for as long as you need. Maximum 1 Year
    • Select Scopes at Full access (You might want to tighten your permission in a production environment, for this demo Full access is fine).
    • Click Create
  • Once you have clicked Create this is the only chance to grab a copy of the token. Please take a copy of this token as you will require it later.

The Code

You will need to first be logged into Az Cli. You can sign in using a service principal as you might with a pipeline, as long as the account being used is able to list App Registrations, and ‘User Access Administrator’ RBAC role to be able to apply contribute access to the DevOps service principal on the subscription (Line 43) .

The important part to note in the code is how the authentication works with Devops. The Personal Access Token is added to the $Env: variable “AZURE_DEVOPS_EXT_PAT”. (Line 32)

<#
.SYNOPSIS
Creates a service connection for a subscription
Please ensure you are already logged to azure using az login
#>
param(
# Azure DevOps Personal Access Token (PAT) for the 'https://dev.azure.com/%5BORG%5D&#39; Azure DevOps tenancy
[Parameter(Mandatory)]
[string]
$PersonalAccessToken,
# The Azure DevOps organisation to create the service connection in, available from System.TeamFoundationCollectionUri if running from pipeline.
[string]
$TeamFoundationCollectionUri = $($Env:System_TeamFoundationCollectionUri -replace '%20', ' '),
# The name of the project to which this build or release belongs, available from $(System.TeamProject) if running from pipeline
[string]
$TeamProject = $Env:System_TeamProject,
[string]
$AppRegistrationName,
[securestring]
$AppPassword
)
$ErrorActionPreference = 'Stop'
$InformationPreference = 'Continue'
$account = az account show | ConvertFrom-Json
#Clearing default.
az configure –defaults group=
$Env:AZURE_DEVOPS_EXT_PAT = $PersonalAccessToken
Write-Information -MessageData:"Adding Azure DevOps Extension…"
az extension add –name azure-devops
Write-Information -MessageData "Configure defaults Organization:$TeamFoundationCollectionUri …"
az devops configure –defaults organization="$TeamFoundationCollectionUri"
Write-Information -MessageData "Getting App Registration: $AppRegistrationName…"
$AppReg = az ad app list –all –query "[?displayName == '$AppRegistrationName']" | ConvertFrom-Json
Write-Information -MessageData "Give App Registration Contributor access to Subscription…"
az role assignment create –role 'Contributor' –assignee $($AppReg.appId)
Write-Information -MessageData "Checking if $TeamProject project exists…"
$ProjectDetails = az devops project list –query "value[?name == '$TeamProject']" | ConvertFrom-Json
if(-not $ProjectDetails){
Write-Information -MessageData "Creating $TeamProject project…"
$ProjectDetails = az devops project create –name $TeamProject
}
Write-Information -MessageData "Checking if service endpoint already exists…"
$ServiceEndpoint = az devops service-endpoint list –project "$TeamProject" –query "[?name == '$($AppReg.DisplayName)-Subscription']" | Select-Object -First 1 | ConvertFrom-Json
if(-not $ServiceEndpoint){
Write-Information -MessageData "Creating Service Connection name:$($AppReg.DisplayName)-Subscription for project $TeamProject…"
$Env:AZURE_DEVOPS_EXT_AZURE_RM_SERVICE_PRINCIPAL_KEY = $(ConvertFrom-SecureString -SecureString:$AppPassword -AsPlainText)
$ServiceEndpoint = az devops service-endpoint azurerm create –project "$TeamProject" –name "$($AppReg.DisplayName)-Subscription" –azure-rm-service-principal-id "$($AppReg.appId)" –azure-rm-subscription-id "$($Account.id)" –azure-rm-subscription-name "$($Account.name)" –azure-rm-tenant-id "$($Account.tenantId)" | ConvertFrom-Json
}
Write-Information -MessageData "Updating Service Connection to be enabled for all pipelines…"
az devops service-endpoint update –project "$TeamProject" –id "$($ServiceEndpoint.id)" –enable-for-all true | Out-Null

To run the above code, you will need to put in your parameters. Replace with your values then run the script, this will call the script above.

$PersonalAccessToken = '<Put your PAT Token>'
$TeamProject = '<Project Name>'
$TeamFoundationCollectionUri = 'https://dev.azure.com/<OrganizationName >'
$AppRegistrationName = '<Service Principal Name>'
$AppPassword = '<Service Principal Secret>'
$AppSecurePassword = ConvertTo-SecureString -String:$AppPassword -AsPlainText -Force
.\Install-ServiceConnectionSubscription.ps1 -PersonalAccessToken $PersonalAccessToken `
-TeamFoundationCollectionUri:$TeamFoundationCollectionUri `
-TeamProject:$TeamProject `
-AppRegistrationName:$AppRegistrationName `
-AppPassword:$AppSecurePassword

My team project is called AutomateDevOps, and I used an App Registration called DevOps.

Running Script

Service Principal with Contribute on Subscription

Project ‘AutomateDevOps’ and Service connection ‘DevOps-Subscription’

My next blog post explains how do make a Management Group Service Connection instead of a Subscription level. ‘Programmatically connecting to Azure Devops with a Service Principal (Management Group)

Connecting to Azure Devops with a Service Principal


Alternative posts:

I see a lot of blogs and examples on the internet that shows you how to connect to environments using a username and password. This is all well and good for testing, but I believe it is bad for real world scenarios.

I’m a contractor, and my time at the job is defined, after I leave the contract and move onto the next one, the company should disable my account. What happens next? Everything I have built using a username/password, stops working. Yes, I could argue, it ensures I get a call back, but most contracts I’ve been involved in have a clause that any bugs found will be fixed for free up to 3-6 months after the job is done. Also, I like to leave a place with them thinking “That guy is awesome, lets get him back for the next project!”.

This post is going to show you how to set up a Service Principal for your Azure Devops CI/CD. At the end, I will give a very basic deployment that creates a Resource Group in Azure. Please note, this example is to show how to set up when your Azure Devops is not part of the same Directory as your Azure Resource Tenant. When it is part of the same tenant.

First, we need to start in Azure and create a Service Principal. The Service Principal will need to be a contributor on the Subscription or the Resource group that your Devops project is going to manage.

Create Service Principal

  • Open Azure Portal
  • Navigate to Azure Active Directory
  • Click App registration
  • Click New Registration
    • Name: Devops-<Company>-<ProjectName> (E.g, Devops-CFCode-OperationsDemo)
    • Supported account types: Accounts in this organizational directory only (Single Tenant)
    • Redirect URI: (Leave blank)
    • Click Register
  • Make a note of Application (client) ID and your Directory (tenant) ID.

Create a Secret for the Service Principal

  • In the App Registration for the above app, click Certificates & secrets.
  • Under Client secrets, click New client secret
    • Description: DEVOPS
    • Expires: Never
    • Click Add
  • Make note of the secret

Assign Service Principal permission to Subscription

  • Open Azure Portal
  • Navigate to Subscriptions and select your subscription
  • Click Access control (IAM)
  • Click Add -> Add role assignment
    • Role: Contributor
    • Assign access to: Azure AD user, group, or service principal
    • Select: <Name of service Principal>
    • Click Save
  • From the Overview blade, grab the Subscription ID and Subscription Name.

You can also add API permissions, such as Graph, and then make direct calls to Graph API using PowerShell using this service principal within the pipeline. Now this side has all been set up, we can head over to our Devops.

Create Service Connection in Devops

  • Go into your project.
  • At the bottom left of your screen click Project Settings
  • Within Project settings, underneath Pipelines click Service connections*. If you have a star next to the Service connections word, it means that you are viewing the preview version. I’m going to show the following screens using a preview version.
  • Click Create service connection
  • Select Azure Resource Manager, click Next
  • Select Service principal (manual), click Next
  • On the New Azure service connection blade, (replace values with your values you grabbed earlier)
    • Environment: Azure Cloud
    • Scope Level: Subscription
    • Subscription Id: <SubscriptionID>
    • Subscription Name: <Subscription Name>
    • Service Principal Id: <Application (client) ID>
    • Credential: Service principal key
    • Service Principal Key: <Secret>
    • Tenant ID: <Directory (tenant) ID>
    • Details (This section is your choice)
      • Service connection name: <Name of Tenant>-<SubscriptionName>
      • Description:
    • Security: Tick – Grant access permission to all pipelines.
  • Click Verify, a Verification Succeeded should show if all the details are correct, and the service account has permission.
  • Click Verify and save

The Service Principle is now connected

Proving the Service Principal connections works

Within your Dev Ops project, click on Pipelines and select releases. We are going to create a Resource Group within our subscription.

  • Add an Azure CLI task.
    • Task verison: 2*
    • Azure Resource Manager connection: Pick the subscription you have created in the previous section.
    • Script Type: PowerShell
    • Inline script: Inline Script
    • Inline script: az group create –name “Demo-rg” –location uksouth
  • Click Save and create a release. After the release has run, and you have received success, an empty resource group should have been created within the subscription.