App-Only Auth Connect to SharePoint Online with MSAL and Azure KeyVault


Now that SharePoint Online CSOM is now works with .NET Framework, I thought I would put together a demo using Visual Studio code that connects to SharePoint.

A previous colleague and still good friend of mine Vardhaman Deshpande in June wrote a blog post showing how to connect to SharePoint Online using MSAL. It is so well written that really writing another blog about it seems a little pointless, so I have taken his blog a little further by connecting to a KeyVault in Azure and grabbing the certificate directly from there.

.NET Standard CSOM of SharePoint Online now uses OAuth for authentication. This means an Access Token needs to be grabbed and passed to every call that is made to SharePoint Online. We will do this by grabbing the AppID and Certificate from the KeyVault and then get the Access Token through ConfidentialClientApplicationBuilder. The Access Token will then be passed into the ClientContext so all calls are made with the Access Token to SharePoint Online.

Walk-through Demo

The demo I have put together can be found at my GitHub repository. By using a Azure AD App Registration and a client certificate, I will walk-through the steps here to set up the following:

  • Create a Resource Group in Azure
  • Create a KeyVault
  • Create a Certificate and Store it in the KeyVault
  • Create an Azure AD App registration
  • Store the ClientID in the KeyVault Secrets
  • Grant Azure AD Application permission for SharePoint – Sites.FullControl.All
  • Console code that connects to SharePoint Online.

Setup

To perform all the steps above as a manual walk-through would take a lot of time to go through. Also, where I can automate things I do. Therefore in the GitHub project under the PowerShell Folder there is a PowerShell file called Install-AzureEnvironment.ps1.

This uses AZ Cli and running the below script will create the above for you in your Azure environment. Replace “Contso” with the name of your tenant.

az login
$tenantName = "contso"
# Defaults to UK South
.\Install-AzureEnvironment.ps1 Environment $tenantName Name "SharePointMSAL"
# If wish to change location
#.\Install-AzureEnvironment.ps1 -Environment $tenantName -Name "SharePointMSAL" -Location:'<Location>'

The above will create the following Azure Resources (using the example of Consto as tenant name)

  • Resource Group: Contso-SharePointMSAL
  • App Registration: Contso-SharePointMSAL (Granted with SharePoint > Sites.FullControl.All)
  • Key Vault: Contso-SharePointMSAL (Will be truncated to 24 characters if longer)
  • CertificateName stored in KeyVault Certificates: Contso-SharePointMSAL
  • ClientId stored in KeyVault Secret: ConstoSharePointMSAL

Console Application

Using Visual Studio Code, I’ve create a .NET Core 3.1 console application and added the following nuget packages. Please see my previous blog post “Basic dotnet commands to create C# project in Visual Studio Code” on how to create a Console application and add nuget packages.

  • Microsoft.SharePointOnline.CSOM
    • Used for the SharePoint CSOM calls
  • Microsoft.Identity.Client
    • Used for OAuth authentication
  • Azure.Identity
    • Used for KeyVault authentication
  • Azure.Security.KeyVault.Secrets
    • Used for getting the Secret and Certificate from the vault
  • Microsoft.Extensions.Configuration
    • Used for collecting app.config values
  • Microsoft.Externsions.Configuration.FileExtensions
    • Used for collecting app.config values
  • Microsoft.Extensions.Configuration.Json
    • Used for collecting app.config values

Next you will need to create (or update if cloned the github project) the appsettings.json file. Replace the environment and site values for your environment.

{
"environment": "<tenantName>",
"name": "SharePointMSAL",
"site": "<relative URL e.g, /sites/teamsite>"
}
view raw appsettings.json hosted with ❤ by GitHub

Then update the program.cs file with the following code. The code has been written assuming your Azure resources using the .\Install-AzureEnvironment.ps1.

using System;
using Microsoft.Identity.Client;
using Microsoft.SharePoint.Client;
using System.Security.Cryptography.X509Certificates;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Azure.Identity;
using Azure.Security.KeyVault.Secrets;
namespace SharePointMSAL
{
class Program
{
static async Task Main(string[] args)
{
IConfiguration config = new ConfigurationBuilder()
.AddJsonFile("appsettings.json", true, true)
.Build();
string siteUrl = $"https://{config["environment"]}.sharepoint.com{config["site"]}";
string identity = $"{config["environment"]}-{config["name"]}";
string keyVaultName = GetKeyVaultName(identity);
string certificateName = identity;
string tenantId = $"{config["environment"]}.onmicrosoft.com";
string clientIDSecret = identity.Replace("_","").Replace("","");
string clientId = GetSecretFromKeyVault(keyVaultName, clientIDSecret);
//For SharePoint app only auth, the scope will be the Sharepoint tenant name followed by /.default
var scopes = new string[] { $"https://{config["environment"]}.sharepoint.com/.default" };
var accessToken = await GetApplicationAuthenticatedClient(clientId, keyVaultName, certificateName, scopes, tenantId);
var ctx = GetClientContextWithAccessToken(siteUrl, accessToken);
Web web = ctx.Web;
ctx.Load(web);
await ctx.ExecuteQueryAsync();
Console.WriteLine(web.Title);
}
private static string GetKeyVaultName(string identity)
{
var keyVaultName = identity;
if (keyVaultName.Length > 24)
{
keyVaultName = keyVaultName.Substring(0, 24);
}
return keyVaultName;
}
private static async Task<string> GetApplicationAuthenticatedClient(string clientId, string keyVaultName, string certificateName, string[] scopes, string tenantId)
{
var certificate = GetAppOnlyCertificate(keyVaultName, certificateName);
IConfidentialClientApplication clientApp = ConfidentialClientApplicationBuilder
.Create(clientId)
.WithCertificate(certificate)
.WithTenantId(tenantId)
.Build();
AuthenticationResult authResult = await clientApp.AcquireTokenForClient(scopes).ExecuteAsync();
string accessToken = authResult.AccessToken;
return accessToken;
}
public static ClientContext GetClientContextWithAccessToken(string targetUrl, string accessToken)
{
ClientContext clientContext = new ClientContext(targetUrl);
clientContext.ExecutingWebRequest += delegate (object oSender, WebRequestEventArgs webRequestEventArgs)
{
webRequestEventArgs.WebRequestExecutor.RequestHeaders["Authorization"] = "Bearer " + accessToken;
};
return clientContext;
}
public static X509Certificate2 GetAppOnlyCertificate(string keyVaultName, string certificateName)
{
var keyVaultUrl = $"https://{keyVaultName}.vault.azure.net";
var client = new SecretClient(new Uri(keyVaultUrl), new DefaultAzureCredential());
KeyVaultSecret keyVaultSecret = client.GetSecret(certificateName);
X509Certificate2 certificate = new X509Certificate2(Convert.FromBase64String(keyVaultSecret.Value), string.Empty,
X509KeyStorageFlags.MachineKeySet |
X509KeyStorageFlags.PersistKeySet |
X509KeyStorageFlags.Exportable);
return certificate;
}
public static string GetSecretFromKeyVault(string keyVaultName, string secretName){
var keyVaultUrl = $"https://{keyVaultName}.vault.azure.net";
var client = new SecretClient(new Uri(keyVaultUrl), new DefaultAzureCredential());
KeyVaultSecret keyVaultSecret = client.GetSecret(secretName);
return keyVaultSecret.Value;
}
}
}

After the code runs it will display the site Title. In my case ‘TestAPISite’.

The important piece of code to get the certificate from Azure Key Vault is GetAppOnlyCertificate function on line 78. This is using the new Azure.Identity and Azure.Security.KeyVault.Secrets libraries.

The Azure.Identity information can be found here: https://docs.microsoft.com/en-us/dotnet/api/overview/azure/identity-readme?view=azure-dotnet

The key to authentication to the KeyVault is on line 82 using DefaultAzureCredential, as this attempts to connect using different authentication methods. Once connected, it retrieves the certificate value and creates a X509Certificate2 certificate in memory. The only confusing part of the code is using Azure.Security.KeyVault.Secrets to get the value not Azure.Security.KeyVault.Certificates.

The image below taken from the Microsoft documentation, shows how the DefaultAzureCredential will attempt to authenticate via the following mechanisms in order.

https://docs.microsoft.com/en-us/dotnet/api/overview/azure/identity-readme?view=azure-dotnet#defaultazurecredential
  • Environment – The DefaultAzureCredential will read account information specified via environment variables and use it to authenticate.
  • Managed Identity – If the application is deployed to an Azure host with Managed Identity enabled, the DefaultAzureCredential will authenticate with that account.
  • Visual Studio – If the developer has authenticated via Visual Studio, the DefaultAzureCredential will authenticate with that account.
  • Visual Studio Code – If the developer has authenticated via the Visual Studio Code Azure Account plugin, the DefaultAzureCredential will authenticate with that account.
  • Azure CLI – If the developer has authenticated an account via the Azure CLI az login command, the DefaultAzureCredential will authenticate with that account.
  • Interactive – If enabled the DefaultAzureCredential will interactively authenticate the developer via the current system’s default browser.

Note: When I first ran my code on Visual Studio Code, I kept getting an authentication issue, it was because I had Visual Studio Enterprise installed on my machine and it was picking up the authentication method selected there which was pointing to a different tenant. You can see this in Visual Studio Enterprise by going into Tools > Options

The great thing about DefaultAzureCredential, is that if this code was within a Azure Function, I could run it first on my computer, then deploy it to Azure Functions with a Managed Identity, and it would still work without any changes to the code.

I hope you find this blog post useful.

Basic dotnet commands to create a C# Project in Visual Studio Code


Since SharePoint Online CSOM is now .NET Core compatible, it means I can start doing all my C# code work in Visual Studio Code and drop the heavy weight application of Visual Studio (Community/Enterprise). Yes, I might need to wait a little longer until SharePointPnPCoreOnline become .NET Core compatible too, but I’m sure that’s not far behind.

Although I have been a developer for many years, and used VS Code for a while for PowerShell and SPFX, I haven’t done any C# projects. This made me realise that I didn’t know how to start a C# project, how to add a reference to another project, add NuGet packet etc., inside VS Code. Does this mean I’m not very good at my job because I don’t know this simple information? No. It just means I’ve never done it before.

This blog post is going to just walkthrough some of the simple steps to get yourself up and running with a C# project using Visual Studio Code. For someone who spent most of his working life inside Visual Studio Community/Enterprise, the steps inside VS Code it is all command line calls and not using GUI’s and wizards. Which sometimes feels frustrating.

The dotnet https://docs.microsoft.com/en-us/dotnet/core/tools/ commands are what we will be using.

You can jump to the relevant point in the blog post from the bookmark links below:

Preparing a Folder to work in
Creating a Solution
Creating a Project
Adding Projects to Solutions
Adding a Project Reference
Getting Intellisense to work
Writing some code
Build the Solution
Run the Project
Publishing the Project
Debugging the Project
Adding a NuGet Package
Listing the NuGet Packages
Remove a NuGet Package

Preparing a Folder to work in

  • Open Visual Studio Code inside a blank folder.
    • File -> Open Folder
    • Navigate to where you want the project to be.
    • Click New Folder and name the folder
    • Select the folder.
  • Open the Terminal
    • Terminal -> New Terminal
    • You can use Bash / Windows PowerShell / PowerShell 6 / PowerShell 7

Creating a Solution

  • In the terminal type the following to create a solution. The name of the solution is the name of the folder it sits in, you can use –name to give it a different name.
    • dotnet new sln

Creating a project

There are loads of project types you can create, and just typing dotnet new –help
shows you all the different types. Using the short name creates that type of project.

Let us create a C# Console project, —output is the folder location where the project is going to be put.

  • dotnet new console –output CSharpConsole

Let us also create a C# Library class project.

  • dotnet new classlib –output CSharpLibrary

Adding Projects to the Solution

With Visual Studio when you added a project, you automatically added these projects to the solution. This is not the case with VS Code. You must do an additional step to add the projects to the solution.

  • dotnet sln add CSharpConsole/CSharpConsole.csproj
  • dotnet sln add CSharpLibrary/CSharpLibrary.csproj

Adding a Project Reference

To add the CSharpLibrary reference to the CSharpConsole we need to call the add the reference

  • dotnet add CSharpConsole/CSharpConsole.csproj reference CSharpLibrary/CSharpLibrary.csproj

Getting Intellisense to work

When I was walking through the steps to ensure this blog post is accurate, I didn’t need to do these following steps to get the intellisense to work. They just worked. However, I’m going to still add this here in case you find the intellisense is not working for you.

  • Ctrl + Shift + p
  • Type “OmniSharp: Select Project”
    and press enter
  • Choose the solution workspace entry

Writing some code

To continue this blog post that you can follow along with , you will require some code.

  • In the CSharpLibrary project, rename the class.cs to IntLibrary.cs
  • Paste in the following code, this is a simple method that checks if the string value is greater than 5.
using System;
namespace CSharpLibrary
{
public static class IntLibrary
{
public static bool NumberGreaterThan5(this string value)
{
Int32 intValue = 0;
bool canConvert = Int32.TryParse(value, out intValue);
if (canConvert)
{
if (intValue > 5)
{
return true;
}
}
return false;
}
}
}
view raw IntLibrary.cs hosted with ❤ by GitHub
  • In the CSharpConsole project, paste the following code. This code asks the user for an input and returns true or false if the value is above 5 or not. With a console check for too many rows or empty value to quit.
using System;
using CSharpLibrary;
namespace CSharpConsole
{
class Program
{
static void Main(string[] args)
{
LibraryProject();
}
public static void LibraryProject()
{
int row = 0;
do
{
if (row == 0 || row >= 25)
ResetConsole();
string input = Console.ReadLine();
if (String.IsNullOrEmpty(input)) break;
Console.WriteLine($"Input: {input} {"Greater than 5? ",30}: " +
$"{(input.NumberGreaterThan5() ? "Yes" : "No")}\n");
row += 3;
} while (true);
return;
void ResetConsole()
{
if (row > 0)
{
Console.WriteLine("Press any key to continue…");
Console.ReadKey();
}
Console.Clear();
Console.WriteLine("\nPress <Enter> only to exit; otherwise, enter a string and press <Enter>:\n");
row = 3;
}
}
}
}
view raw program.cs hosted with ❤ by GitHub

Build the solution

The command to build the solution is:

  • dotnet build

Run the Project

To run the project you need to pass in the Project file.

  • dotnet run –project CSharpConsole/CSharpConsole.csproj

Or if you are already in the project’s directory

  • dotnet run

Publishing the Project

To publish the project to run it as an executable in this console case

  • dotnet publish –configuration release

Debugging the Project

No matter how good you are, you will need to debug code at somepoint.

  • Put a breakpoint in your code. Press F5.
  • First time you will be asked the project type. Select .NET Core. This will create a launch.json file. As this project is a console project, we need to modify the configuration within the launch.json file to make the console point to Integrated terminal, otherwise the line to clear the console will fail.
    • Change “console”: “internalConsole”
      on line 15 and change to “console”: “integratedTerminal”
{
// Use IntelliSense to learn about possible attributes.
// Hover to view descriptions of existing attributes.
// For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
"version": "0.2.0",
"configurations": [
{
"name": ".NET Core Launch (console)",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build",
"program": "${workspaceFolder}/CSharpConsole/bin/Debug/netcoreapp3.1/CSharpConsole.dll",
"args": [],
"cwd": "${workspaceFolder}/CSharpConsole",
"console": "integratedTerminal",
"stopAtEntry": false
},
{
"name": ".NET Core Attach",
"type": "coreclr",
"request": "attach",
"processId": "${command:pickProcess}"
}
]
}
view raw launch.js hosted with ❤ by GitHub
  • Press F5 to debug the code.

Adding a NuGet Package

Nearly all projects now will involve adding a NuGet package of some sort. Again with the Command line to add, which I must admit I miss using a GUI. The GUI allowed you to search, now you need to know the name of the package. I’m going to add the SharePoint CSOM package to the console application

  • dotnet add CSharpConsole/CSharpConsole.csproj package Microsoft.SharePointOnline.CSOM

Optionally you can add the version parameter to get a specific version of a package.

  • dotnet add CSharpConsole/CSharpConsole.csproj package Newtonsoft.json –version 12.0.1

Listing the NuGet Packages

To find out which project has which NuGet packages already installed,

  • dotnet list package

Remove a NuGet Package

To remove a given NuGet Package you can type the following

  • dotnet remove CSharpConsole/CSharpConsole.csproj package Newtonsoft.json

Hopefully, that is enough to get yourself going, of course all commands have –help that explains how to use the command.

Grant Application and Delegate Permissions using an App Registration


This blog post came about because I wanted a way to create new Application Registrations and grant consent for the tenant, all programmatically. This is so I can use Devops pipelines to create and deploy my code without any human interaction or using a person account.

The AZ cli can grants permissions, but it does not seem to work for Admin consented permissions. I found the following post by Sam Coganhttps://samcogan.com/provide-admin-consent-fora-azure-ad-applications-programmatically/ saying that it was possible if you use REST API calls. This did work for me; however, it was just the Delegated permissions.

By reading through Sam’s post it helped me understand the connection between Application Registrations, Service Principals and Oauth2permission, and helped me on the quest of understand how to grant the Application permissions through appRoleAssignments.

I also want to credit Sahil Malik as I found his post https://winsmarts.com/how-to-grant-admin-consent-to-an-api-programmatically-e32f4a100e9d after I worked it all out myself, and was able to confirm that what I was doing was right.

At my Github project https://github.com/pmatthews05/CFAppOnlyGrantPermissions the README.md will walk you through how to set up and run the code. At the end of the README.md file you should have 2 Application Registration, where the Azure API Registration app would have created the second app (in my case CFCodeApp) for you. This code is idempotent. You can change the permissions for an existing Application Registration by providing it a different Permission.json file.

As the README.md file gives the instruction on how to run the code, I will not replicate it here. I will use the rest of this post to explain how the code works.

Permissions required for ‘Azure API Registration’

To allow the Azure API Registration to create new Application Registrations using AZ cli it requires to use both the legacy Azure Active Directory Graph and Microsoft Graph permissions. It seems that some of the commands in the az cli still points to https://graph.windows.net when it makes calls, according to some issue notes in the az cli git hub repository, it looks like this is in the process of being changed.

With Azure Active Directory Graph we need 2 permissions

  • Application.ReadWrite.All – This allows us to read and write the Application Registrations.
  • Directory.ReadWrite.All – This allows us to read the application registration permission list, and service principal information.

With Microsoft Graph we also need permission.

  • AppRoleAssignment.ReadWrite.All – This allows us to call the REST API to grant permissions and assign Role assignment permissions.

Steps in the code

  • Set-AppRegistration
  • Set-AppCredentials
  • Set-ServicePrincipalForAppId
  • Remove-CurrentAppPermissions
  • Set-DelegatePermissions
    • Remove-CurrentOauth2PermissionGrants
  • Set-ApplicationPermissions
    • Remove-CurrentServicePrincipalGrants

Please note, the snippets of code I am showing here in the blog, are showing the command(s) that are performing the main action, not the full function.

Set-AppRegistration

We need an App Registration to be created first. If the name already exists it just returns the existing App Registration

#Creates or updates an existing App Registration
az ad app create --display-name '$ApplicationName'
view raw appReg.ps1 hosted with ❤ by GitHub

Set-AppCredentials

#Creates a secret with a random password
az ad app credential reset --id $AppId --credential-description 'Registration' --end-date 2299-12-31
view raw appcredential.ps1 hosted with ❤ by GitHub

This will create a secret for the App Registration with a random secret with the description set to Registration. There are a couple of override parameters that I am not using, where you can give it your own Description, and provide your own SecureString secret. This returns the appCredentials that supply the appId, name, password and tenantId. In the script it outputs this to screen at the end, however, if using in production environment, you would probably want to put the secret value in a keyvault, without displaying to the user what the value is.

Set-ServicePrincipalForAppId

#Creates a Service Principal for a given Application Registration Id
az ad sp create --id $AppId
view raw servicePrincipal.ps1 hosted with ❤ by GitHub

All App Registrations require a Service Principal behind them. When you manually create an App Registration and assign permissions, it automatically creates a service principal for you. When you create an App Registration programmatically, it is your responsibility to also create the Service Principal. It is the Service Principal that defines the access policy and permissions for the user/application in the Azure AD tenant. A multi-tenant App Registration would have the same app Id in all tenants, but all have a different Service Principal which allows them access within that tenant. For example, in all tenants the AppId for the Microsoft Graph API is ‘00000003-0000-0000-c000-000000000000’ and in your tenant it has an associated Service Principal, which is a different object Id in your tenant compared to mine. When I finally understood this, it made more sense how this all ties together.

Reference: https://docs.microsoft.com/en-us/azure/active-directory/develop/app-objects-and-service-principals

The above az command is not idempotent, and therefore a check to see if it already exists is required.

Remove-CurrentAppPermissions

To allow idempotency of my script, I wanted to ensure that it removes all existing permissions before adding them back in. This piece of code does not remove the permission from the service principal, and if you stop the code after this command, you will see that your API Permissions in the GUI would look like this.

Seeing the permissions separated at the bottom of the screen, now understanding the relationship between Application Registrations and Service Principal, it makes a lot more sense to me now. The service principal still has access at this point and calls to these API’s will still work.

#Get all permissions for the Application Registration
$currentPermissionCollection = @(az ad app permission list --id $AppId | ConvertFrom-Json)
#Remove the permissions (resourceAppId) for the Application Registration
$currentPermissionCollection | ForEach-Object {
$permission = $PSItem
if ($permission.Count -eq 0) { return }
$permission.resourceAppId | ForEach-Object{
$resourceAppId = $PSItem
az ad app permission delete --id $AppId --api $resourceAppId
}
}

The code gets a list of all the permissions assigned to the Application Registration, then loops through each resourceAppId (the objectId value of the API permission service principal e.g, Microsoft Graph, SharePoint)*2 and deletes the permission.

Set-DelegatePermissions

To ensure the code is idempotent the first thing I am doing is removing the delegate permission from the Service Principal. See the next section on how this works.

Now we need to assign the Application Registration permissions for the delegate permissions. We do this by providing the AppID of our Application Registration, the API Permission AppID (the appID of the API Permission service principal e.g, Microsoft Graph)*2 and the oauth2Permissions scope Id*4

#Get Graph APIServicePrinicpal information
$APIServicePrincipal = az ad sp list --query "[?appDisplayName=='Microsoft Graph'].{appId:appId,objectId:objectId}" --all | ConvertFrom-Json
#Get Directory.ReadWrite.All oauth2Permission
$delegatePermInfo = az ad sp show --id $($APIServicePrincipal.appId) --query "oauth2Permissions[?value=='Directory.ReadWrite.All']" | ConvertFrom-Json
#Add Permission (Scope means Delegate)
az ad app permission add --id $appId --api $($APIServicePrincipal.appId) --api-permissions $($delegatePermInfo.id)=Scope

Next, we need to assign the Application Registration associated Service Principal oauth2Permissions to grant these permissions to the tenant.

Using Graph API explorer, you can view all the delegate permissions in your tenant using the following URL:

https://graph.microsoft.com/v1.0/oauth2permissiongrants

To find all the permissions grants for your Application Registration you will need the Service Principal Object ID*1 and then use the following URL:

https://graph.microsoft.com/v1.0/oauth2permissiongrants?$filter=clientId eq ‘<servicePrincipalObjectId>’ and consentType eq ‘AllPrincipals’

{
"@odata.context": "https://graph.microsoft.com/v1.0/$metadata#oauth2PermissionGrants",
"value": [
{
"clientId": "8xxxxxx-xxxx-xxxx-xxxx-xxxxxxxx8299",
"consentType": "AllPrincipals",
"id": "AxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxWry4",
"principalId": null,
"resourceId": "a2adb674-25ec-4bba-ba12-f6fc2f96af2e",
"scope": "User.Read Group.ReadWrite.All"
}
]
}
  • clientId: This is the Service Principal Object ID that is tied to your App Registration
  • consentType: Set to AllPrincipals when granted to the entire tenant, or Principal when granted to an individual user
  • id: The ID of the oauth2permissiongrants
  • principalId: This is set to null if using AllPrincipals, otherwise it will contain the objectID of the User that has been granted the permission
  • resourceId: This is the Serivce Principal Object ID value of the API Permission*2
  • scope: This is a string array of granted scope values for the given ResourceId. (e.g User.Read Directory.Read.All etc)

If the oauth2permissiongrants with the App Registration Service Principal Object ID and API Permission Service Principal Object ID (clientId and resourceId) doesn’t exist in your tenant, then you will need to POST a new oauth2permissiongrants, otherwise you will require to PATCH an existing oauth2permissiongrants/<id> with the new string array of scope values.

#Get a access Token
$tokenResponse = az account get-access-token --resource-type ms-graph | convertFrom-Json
$body = @{
clientId = $($servicePrincipal.objectId)
consentType = "AllPrincipals"
principalId = $null
resourceId = $($APIServicePrincipal.objectId)
scope = "User.Read Directory.ReadWrite.All"
startTime = "0001-01-01T00:00:00Z"
expiryTime = "2299-12-31T00:00:00Z"
}
$apiUrl = "https://graph.microsoft.com/v1.0/oauth2Permissiongrants"
Invoke-RestMethod -Uri $apiUrl -Headers @{Authorization = "Bearer $($tokenResponse.accessToken)" } -Method $method -Body $($body | ConvertTo-Json) -ContentType "application/json"

You must add a startTime and expiryTime, it does not matter what the datetime is, as long as expiryTime is later than the startTime.

Remove-CurrentOauth2PermissionGrants

To remove the permissions from the Service Principal for the Delegate Permissions, we need to remove the Oauth2PermissionGrants.

Unfortunately, with App Only permissions you cannot delete an oauth2permissiongrants. You require to access the directory as a person to delete. I found that by setting the scope to empty string, gives the same desired effect as removing them.

#Get all Permission for the Service prinipal ObjectId.
$exisitingCollection = az ad app permission list-grants --filter "clientId eq '$($ServicePrincipalObjectId)' and consentType eq 'AllPrincipals'" | ConvertFrom-Json
#Get a access Token
$tokenResponse = az account get-access-token --resource-type ms-graph | convertFrom-Json
$existingCollection | ForEach-Object {
$existing = $PSItem
#Get the PermissionGrant
$apiUrlPatch = "https://graph.microsoft.com/v1.0/oauth2Permissiongrants/$($existing.objectId)"
$body = @{
scope = ""
}
#Patch with an empty scope.
Invoke-RestMethod -uri $apiUrlPatch -Headers @{Authorization = "Bearer $(tokenResponse.accessToken)"} -Method $PATCH -Body $($body | ConvertTo-Json) -ContentType "application/json"
}

Please Note: I am using Invoke-RestMethod instead of az rest because I have not been able to get it to work without an error message.

Set-ApplicationPermissions

To ensure the code is idempotent the first thing I am doing is removing the application permission grants from the Service Principal. See the next section on how this works.

Now we need to assign the Application Registration permissions for the application permissions. We do this by providing the AppID of our Application Registration, the API Permission AppID (the appID of the API Permission service principal e.g, Microsoft Graph)*2 and the AppRoles scope Id*3

#Get Graph APIServicePrinicpal information
$APIServicePrincipal = az ad sp list --query "[?appDisplayName=='Microsoft Graph'].{appId:appId,objectId:objectId}" --all | ConvertFrom-Json
#Get Directory.ReadWrite.All appRolesPermission
$appRolePermInfo = az ad sp show --id $($APIServicePrincipal.appId) --query "appRoles[?value=='Directory.ReadWrite.All']" | ConvertFrom-Json
#Add Permission (Role means Application)
az ad app permission add --id $appId --api $($APIServicePrincipal.appId) --api-permissions $($appRolePermInfo.id)=Role

Next, we need to assign the Application Registration associated Service Principal AppRoleAssignments to grant these permissions to the tenant.

Using Graph API explorer, you can view all the Application Role Grants for your Application Registration. You will need the Service Principal Object ID*1 and then use the following URL:

https://graph.microsoft.com/v1.0/servicePrincipals//appRoleAssignments

"@odata.context": "https://graph.microsoft.com/v1.0/$metadata#servicePrincipals('862b7a01-7bb3-4540-ae4e-a2a84c7d8299&#39;)/appRoleAssignments",
"value": [
{
"id": "AxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxCw",
"deletedDateTime": null,
"appRoleId": "57xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx04",
"createdDateTime": "2020-06-18T10:45:50.700178Z",
"principalDisplayName": "CFCodeApp",
"principalId": "86xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx99",
"principalType": "ServicePrincipal",
"resourceDisplayName": "Windows Azure Active Directory",
"resourceId": "5cxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx27"
},
{
"id": "AxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxA8",
"deletedDateTime": null,
"appRoleId": "33xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx0d",
"createdDateTime": "2020-06-18T09:50:01.3746638Z",
"principalDisplayName": "CFCodeApp",
"principalId": "86xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx99",
"principalType": "ServicePrincipal",
"resourceDisplayName": "Microsoft Graph",
"resourceId": "a2xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx2e"
},
{
"id": "AxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxHI",
"deletedDateTime": null,
"appRoleId": "7axxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx61",
"createdDateTime": "2020-06-18T09:49:51.3226179Z",
"principalDisplayName": "CFCodeApp",
"principalId": "86xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx99",
"principalType": "ServicePrincipal",
"resourceDisplayName": "Microsoft Graph",
"resourceId": "a2xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxx2e"
}
]
}

Unlike the Oauth2PermissionGrants, where there is only one entry per clientid and resourceId which contains all the scopes, with AppRoleAssignments there is an entry for each scope, and it uses the appRoleId scope Id*3 instead of the scope string value.

  • id: The ID of the appRoleAssignment
  • principalId: The Service Principal Object ID that is tied to your App Registration
  • resourceId: This is the Serivce Principal Object ID value of the API Permission*2
  • appRoleId: This is the scope Id*3
#Get a access Token
$tokenResponse = az account get-access-token --resource-type ms-graph | convertFrom-Json
$body = @{
principalId = $ServicePrincipal.objectId
resourceId = $APIServicePrincipal.objectId
appRoleId = $appPermInfo.id
}
$appRoleAssignmentUrl = "https://graph.microsoft.com/v1.0/servicePrincipals/$($ServicePrincipal.objectId)/appRoleAssignments"
Invoke-RestMethod -Uri $appRoleAssignmentUrl -Headers @{Authorization = "Bearer $($tokenResponse.accessToken)" } -Method POST -Body $($body | ConvertTo-Json) -ContentType "application/json"

Remove-CurrentServicePrincipalGrants

To remove the permission from the Service Principal for the Application Role permissions, we need to remove the AppRoleAssignments ID’s for the service principal

#Get a access Token
$tokenResponse = az account get-access-token --resource-type ms-graph | convertFrom-Json
#Get all App Role Assignment Permission for the Service prinipal ObjectId.
$apiUrl = "https://graph.microsoft.com/v1.0/servicePrincipals/$ServicePrincipalObjectId/appRoleAssignments"
$appRoleAssignmentCollection = @(Invoke-RestMethod -Uri $apiUrl -Headers @{Authorization = "Bearer $($tokenResponse.accessToken)" } -Method GET -ContentType "application/json").value
$appRoleAssignmentCollection | ForEach-Object {
$appRoleAssignment = $PSItem
$deleteApiUrl = "$apiUrl/$($appRoleAssignment.id)"
Invoke-RestMethod -Uri $deleteApiUrl -Headers @{Authorization = "bearer $($tokenResponse.accessToken)" } -Method Delete -ContentType "application/json"
}

Please Note: I am using Invoke-RestMethod instead of az rest because I have not been able to get it to work without an error message.

Conclusion

That is it. In the Data folder of the github project there is an examplePermission.json file. As you can see that the JSON format is very flexible to add more or remove permissions. The name can be the appDisplayName or the AppId of the API Permission.

Run the Add-RegistrationAndGrantPermissions.ps1 script passing in the name of your App Registration you wish to create / update and your custom permission file. The script will run fine with user logged in, or an App Only with the correct permissions.

Please feel free to use/enhance the github project.

Footnotes

*1 How to find your Service Principal Object ID of your Application Registration

Using the GUI, the quickest way to find your Service Principal Object ID is to first go to the overview of the Application Registration. Then on the right-hand side of the screen, click the name of your Application Registration where Manage application in local directory is.

This will take you to the Service Principal information in your tenant. It is here you can get the ObjectID.

*2 How to find the Service Principal Object Id for the Permission API.

You may already know that Microsoft Graph API appId is 00000003-0000-0000-c000-000000000000, this is the same on all tenants, but the service principal object ID is different in each tenant. This is the resourceId. To find out what the objectID is in your tenant run the following script.

az ad sp list --query "[?appId=='00000003-0000-0000-c000-000000000000'].{appDisplayName:appDisplayName,appId:appId,objectId:objectId}" --all --output table
AppDisplayName AppId ObjectId
---------------- ------------------------------------ ------------------------------------
Microsoft Graph 00000003-0000-0000-c000-000000000000 axxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxe
....

Remove “?appId== ‘00000003-0000-0000-c000-000000000000′” between the [ ] and it will list all for your tenant.

*3 How to find the Application Role Id of a Permission API Scope.

For each Permission API such as Microsoft Graph API, there are Application Role. No matter what tenant you are in, the id of them is always the same. The below example gets the Application Scope of Directory.Read.All from the Permission API Microsoft Graph.

az ad sp show --id '00000003-0000-0000-c000-000000000000' --query "appRoles[?value=='Directory.Read.All']"

Output:

[
{
"allowedMemberTypes": [
"Application"
],
"description": "Allows the app to read data in your organization's directory, such as users, groups and apps, without a signed-in user.",
"displayName": "Read directory data",
"id": "7ab1d382-f21e-4acd-a863-ba3e13f7da61",
"isEnabled": true,
"value": "Directory.Read.All"
}
]

*4 How to find the oAuth2Permission Id of a Permission API Scope.

For each Permission API such as Microsoft Graph API, there are oauth2Permissions. No matter what tenant you are in, the id of them is always the same. The below example gets the Delegation Scope of Directory.Read.All from the Permission API Microsoft Graph.

az ad sp show --id '00000003-0000-0000-c000-000000000000' --query "oauth2Permissions[?value=='Directory.Read.All']"

Output:

[
{
"adminConsentDescription": "Allows the app to read data in your organization's directory, such as users, groups and apps.",
"adminConsentDisplayName": "Read directory data",
"id": "06da0dbc-49e2-44d2-8312-53f166ab848a",
"isEnabled": true,
"type": "Admin",
"userConsentDescription": "Allows the app to read data in your organization's directory.",
"userConsentDisplayName": "Read directory data",
"value": "Directory.Read.All"
}
]

Setting MS Teams Policies for Users using PowerShell


In my past two blogs I have shown you how to obtain all the users policies and output to a csv file, and how to create a new policy. In this blog post, I’m going to show you a couple of ways of setting users to new policies.

  • Change policy for an individual user
  • Change policy for a group of users
  • Change policies for group of users using a csv file

Change policy for an individual user

In this example I’m going to stick to just the messaging policy.

.\Set-IndivdualUserMessagingPolicy.ps1 -UserNameToSetPolicy:”Jeff.Hay@mytenant.onmicrosoft.com” -PolicyName:”NoGiphyOrStickersMessagePolicy”

param(
[Parameter(Mandatory)]
[string]
$UserNameToSetPolicy,
[Parameter(Mandatory)]
[string]
$PolicyName
)
Import-Module "C:\\Program Files\\Common Files\\Skype for Business Online\\Modules\\SkypeOnlineConnector\\SkypeOnlineConnector.psd1"
$Session = New-CsOnlineSession
Import-PSSession Session:$Session AllowClobber
if (-not $(Get-CsTeamsMessagingPolicy Identity:$PolicyName ErrorAction:SilentlyContinue)){
Write-warning "Unable to find Policy $PolicyName"
return
}
else{
Write-Information "Granting Message Policy $PolicyName for user $UserNameToSetPolicy"
Grant-CsTeamsMessagingPolicy PolicyName $PolicyName Identity $UserNameToSetPolicy
}
Remove-PSSession Session:$session
write-Information "Done"

It can take a while before the change is reflected in the Teams Administration.

Change policy for a group of users

There are a couple of ways you can do this. If you have your people data filled in correctly, such as Department, Office, City etc, you could assign all the people from one of these areas to a given policy. For example the following script grabs everyone from the sales department and assign them the sales policy for messaging:

Get-CsOnlineUser Filter {Department -eq 'sales'} | Grant-CsTeamsMessagingPolicy PolicyName "SalesPolicy"

Alternatively you could do it based on a AD Group, you first need to connect to grab the members of the given group, I would connect with AZ cli.

$GroupName = "SalesUsers"
$PolicyName = "SalesPolicy"
az login
az ad group member list group $GroupName query "[?userType == 'Member']" `
| ConvertFrom-Json `
| % { Grant-CsTeamsMessagingPolicy PolicyName $PolicyName Identity $_.userPrincipalName }

Change policies for a group of users using a csv file

In a previous blog post, I showed you how to obtain all the valid users from the tenant with their policies. We are going to use the csv it produces to change users policies.

The screen shot below shows my users with licenses, originally when I ran my script I only had DisplayName, UserPrincipalName and SipAddress showing, everything else was blank because my users were all in the global policies.

I have now filled in the CSV file with either SalesPolicy, HRPolicy, or NoGiphyOrStickersMessagePolicy. Left my account as global. These policy have already been created in my environment, help with doing that can be found in this blog post.

The following script requires your MS Teams Administrator username, and the path to the csv file. It loops through each item and then sets the polices for each user.

.\set-UserTeamPolicies.ps1 -UserName:admin@mytenant.onmicrosoft.com -Path:.\teamsuserpolicies.csv

param(
#Teams Administrator UserName
[Parameter(Mandatory)]
[string]
$Username,
#CSV File Path
[Parameter(Mandatory)]
[string]
$Path
)
$InformationPreference = 'Continue'
Write-Information MessageData "Obtaining Module, please connect when prompted…"
Import-Module "C:\\Program Files\\Common Files\\Skype for Business Online\\Modules\\SkypeOnlineConnector\\SkypeOnlineConnector.psd1"
$Session = New-CsOnlineSession UserName:$Username
Import-PSSession Session:$Session AllowClobber
@($(Import-csv Path:"$PSScriptRoot\$Path")).ForEach( {
$csv = $PSItem
$userPrincipalName = $csv.userPrincipalName
Write-Information MessageData:"Applying $($csv.DisplayName) Policies…"
#"TeamsMeetingPolicy",
Grant-CsTeamsMeetingPolicy PolicyName $($csv.TeamsMeetingPolicy) Identity $userPrincipalName
#"TeamsMessgingPolicy",
Grant-CsTeamsMessagingPolicy PolicyName $($csv.TeamsMessagingPolicy) Identity $userPrincipalName
#"TeamsMeetingBroadcastPolicy",
Grant-CsTeamsMeetingBroadcastPolicy PolicyName $($csv.TeamsMeetingBroadcastPolicy) Identity $userPrincipalName
#"TeamsAppPermissionPolicy",
Grant-CsTeamsAppPermissionPolicy PolicyName $($csv.TeamsAppPermissionPolicy) Identity $userPrincipalName
#"TeamsAppSetupPolicy",
Grant-CsTeamsAppSetupPolicy PolicyName $($csv.TeamsAppSetupPolicy) Identity $userPrincipalName
#"TeamsCallParkPolicy",
Grant-CsTeamsCallParkPolicy PolicyName $($csv.TeamsCallParkPolicy) Identity $userPrincipalName
#"TeamsCallingPolicy",
Grant-CsTeamsCallingPolicy PolicyName $($csv.TeamsCallingPolicy) Identity $userPrincipalName
#"CallerIDPolicy
Grant-CsCallingLineIdentity PolicyName $($csv.CallerIdPolicy) Identity $userPrincipalName
#"TeamsChannelsPolicy",
Grant-CsTeamsChannelsPolicy PolicyName $($csv.TeamsChannelsPolicy) Identity $userPrincipalName
#"TeamsEmergencyCallingPolicy",
Grant-CsTeamsEmergencyCallingPolicy PolicyName $($csv.TeamsEmergencyCallingPolicy) Identity $userPrincipalName
#"TeamsEmergencyCallRoutingPolicy",
Grant-CsTeamsEmergencyCallRoutingPolicy PolicyName $($csv.TeamsEmergencyCallRoutingPolicy) Identity $userPrincipalName
#"TenantDialPlan",
Grant-CsTenantDialPlan PolicyName $($csv.TenantDialPlan) Identity $userPrincipalName
#"TeamsUpgradePolicy"
Grant-CsTeamsUpgradePolicy PolicyName $($csv.TeamsUpgradePolicy) Identity $userPrincipalName
if ($Session.State -ne "Opened") {
Write-Warning "Session state closed, please reauthenticate"
Remove-PSSession Session:$Session
$Session = New-CsOnlineSession UserName:$Username
Import-PSSession Session:$Session AllowClobber
}
})
Write-Information "Complete"
Remove-PSSession Session:$Session

If you have a lot of users to update, the session might timeout. On line 52 – 57 there is a check to see if the session has timed out and then gets you to reauthenticate. If anyone else knows a better way to do this, please add a comment below, or get in touch.

Creating a new MS Teams Policy using PowerShell


In my previous blog I showed you how to obtain all the policies for users. This blog post, I’m going to show an example of how to create a new Policy for Meeting.

The script I’m providing below is a simple script that will:

  • Name the policy – Line 1
  • Import the SkypeOnlineConnector module – Line 4-7
  • Create the policy if it doesn’t exist – Lines 9 – 12
  • Set the properties of the Policy – Lines 15 – 31

This policy that is being created will prevent users of this policy creating Giphy, Memes or Stickers in their chats. (Basically, taking the fun out of teams. [insert wicked laugh])

$PolicyName = "NoGiphyOrStickersMessagePolicy"
$InformationPreference = 'Continue'
Import-Module "C:\\Program Files\\Common Files\\Skype for Business Online\\Modules\\SkypeOnlineConnector\\SkypeOnlineConnector.psd1"
$Session = New-CsOnlineSession
Import-PSSession Session:$Session AllowClobber
if (-not $(Get-CsTeamsMessagingPolicy Identity:$PolicyName ErrorAction:SilentlyContinue)) {
Write-Information MessageData:"New-CsTeamsMessagingPolicy -Identity:'$PolicyName'"
New-CsTeamsMessagingPolicy Identity:$PolicyName Description:'A policy for no Giphy Or Stickers for Messaging.'
}
Write-Information MessageData:"Set-CsTeamsMessagingPolicy -Identity:'$PolicyName'"
Set-CsTeamsMessagingPolicy Identity:$PolicyName `
AllowUrlPreviews:$true `
AllowOwnerDeleteMessage: $true `
AllowUserEditMessage: $true `
AllowUserDeleteMessage: $true `
AllowUserChat: $true `
AllowRemoveUser: $true `
AllowGiphy: $false `
GiphyRatingType: Strict `
AllowMemes: $false `
AllowImmersiveReader: $true `
AllowStickers: $false `
AllowUserTranslation: $true `
ReadReceiptsEnabledType: UserPreference `
AllowPriorityMessages: $true `
ChannelsInChatListEnabledType: DisabledUserOverride `
AudioMessageEnabledType: ChatsAndChannels `
Remove-PSSession Session:$session

As you can see from the above screen shot, I now have a policy called NoGiphyOrStickersMessagePolicy, and below is a screen shot of the settings within Teams Administration.

My previous blog showed you how to get 13 different policies for each user. Below are the links to those PowerShell Set policy commands with the different parameters.

Getting all MS Teams User Policies using PowerShell


With everyone working at home at the moment, you might need to grab a report of the User Policies for MS Teams. These few steps will show you how to grab all the users and display in a csv file.

To work with policies you first need to obtain the PowerShell module – SkypeOnlineConnector. Which you can download from this link https://www.microsoft.com/en-us/download/details.aspx?id=39366

Installing the SkypeOnlineConnector and creating a session

Once you have installed the PowerShell module you will need to import the module and create a PowerShell session.

Import-Module "C:\\Program Files\\Common Files\\Skype for Business Online\\Modules\\SkypeOnlineConnector\\SkypeOnlineConnector.psd1"
$Session = New-CsOnlineSession
Import-PSSession -Session:$Session -AllowClobber

You will be prompted for MS Teams administrator username and password, you can pass your credentials in with the -credential parameter at the end of New-CsOnlineSession, however this doesn’t work with MFA.

Once connected you will be able to grab all users using the Get-CsOnlineUser cmdlet, or grab one user by providing the users Identity.

Policies

If you call the above for a single user you will see that there are loads of policies that can be set. Not all are MS Teams related. The ones I will be focusing on are the same 12 policies you see when you view Assigned policies for a user in Teams Administration.

These policies are named slightly different in the results compared to the display name shown above.

Display Name Policy Name
Meeting policy TeamsMeetingPolicy
Messaging policy TeamsMessagingPoliy
Live events policy TeamsMeetingBroadcastPolicy
App permission policy TeamsAppPermissionPolicy
App setup policy TeamsAppSetupPolicy
Call park policy TeamsCallParkPolicy
Calling policy TeamsCallingPolicy
Caller ID policy CallerIdPolicy
Teams policy TeamsChannelsPolicy
Emergency calling policy TeamsEmergencyCallingPolicy
Emergency call routing policy TeamsEmergencyCallRoutingPolicy
Dial plan TenantDialPlan
Teams Upgrade TeamsUpgradePolicy

The following script will grab all users and their current policy for the above polices, with the provided path it will output to csv file.

Please note: Anything that is set to Global policy will be blank.

.\Get-UserTeamPolicies.ps1 -Path:.\AllTeamUsersPolicies.csv

param(
#OutPut CSV File Path
[Parameter(Mandatory)]
[string]
$Path
)
$InformationPreference = 'Continue'
Write-Information -MessageData "Obtaining Module, please connect when prompted..."
Import-Module "C:\\Program Files\\Common Files\\Skype for Business Online\\Modules\\SkypeOnlineConnector\\SkypeOnlineConnector.psd1"
$Session = New-CsOnlineSession
Import-PSSession -Session:$Session -AllowClobber
Write-Information -MessageData "Getting all enabled users"
$Users = Get-CsOnlineUser | Select-Object DisplayName, `
UserPrincipalName, `
SipAddress, `
TeamsMeetingPolicy, `
TeamsMessagingPolicy, `
TeamsMeetingBroadcastPolicy, `
TeamsAppPermissionPolicy, `
TeamsAppSetupPolicy, `
TeamsCallParkPolicy, `
TeamsCallingPolicy, `
CallerIdPolicy, `
TeamsChannelsPolicy, `
TeamsEmergencyCallingPolicy, `
TeamsEmergencyCallRoutingPolicy, `
TenantDialPlan, `
TeamsUpgradePolicy
$Users | Export-Csv -Path:$Path -NoTypeInformation
Write-Information "Complete"
Remove-PSSession -Session:$session

In a later blog, I will be using the csv file to update user policies.

Bonus: If you want to get all users for just a single policy you can perform a filter on the Get-CsOnlineUser

Write-host "Teams Meeting Policy"
Get-CsOnlineUser -Filter {TeamsMeetingPolicy -eq 'GivenPolicyNameBlankForGlobal'} | Select UserPrincipalName

Removing External Users fully from a SharePoint Tenancy using PowerShell


This blog post has all come about as the client I was working for was having problems sharing documents in SharePoint with some external users. It turned out that the user was already in Azure AD as a Contact which is part of Exchange. This meant when an internal person attempted to share/Invite into SharePoint/MSTeams it all appeared to work correctly for the external user, but sometimes it didn’t. When looking at external users through the Admin portal, this external user was showing, but their email address was blank. After speaking with Microsoft, it turns out, because the email address was already found within the tenancy, it creates a unique violation when adding the external user to the Active Directory.

I have been working with Microsoft support regarding this, and the resolution was that this is as design!!??! Only by feeding back on the Office 365 uservoice this issue “might” looked at and fixed. See resolution notes below:

Symptom:
When you invite external users who exist as contacts in your environment, their email does not get populated in their guest user ID which results in them not being able to login to your environment and access the shared data.
Cause:
The issue is coming from a conflict caused by the email address which is already populated for the mail contact.
Resolution:
This is behavior by design as all objects in Azure AD have to be unique.
You cannot have 2 objects with the same email address.

When you invite one of your contacts to your content in O365, it actually creates a completely new guest user object in your environment and since the email address which is supposed to be populated in the email attribute is already in use by the contact, the email address does not get populated.

The only way to resolve this issue at the moment is to eliminate any conflicts that are in place, by removing the conflicting email contact and re-invite the user to your content.
More information:
The best thing I can offer to you is the following:

Please go to our UserVoice portal where other people are facing the same behavior and up-vote it, comment and have the whole IT department do the same as well.

Allow a “Guest User” to be converted to a different account type
https://office365.uservoice.com/forums/273493-office-365-admin/suggestions/19966537-allow-a-guest-user-to-be-converted-to-a-differen

This led me to working on a process and script that would remove the users from everywhere.

Locations to remove the External User from:

  • Contacts
  • Azure AD Guest Users
  • Azure AD Deleted Users
  • All SharePoint Sites
  • All SharePoint Hidden User lists
  • SharePoint User Profile

Contacts

To remove the External User from the contacts you will need to use the MSOL PowerShell module.

$UserEmail = "<ExternalUserEmailAddress>"
Connect-MsolService
Get-MsolContact | ? EmailAddress -eq $UserEmail | Remove-MsolContact -Force

Or you can manually do this by going to admin.microsoft.com and under Users -> Contacts select the user and click Delete contacts.

Azure AD

To remove the External User from Azure AD you will still require using the MSOL PowerShell module. In fact, this script and the above script could be merged.

$Environment = "<TenantName>"
$UserEmail = "<ExternalUserEmailAddress>"
Connect-MsolService
$externalConversionEmail = ($UserEmail -replace '@', '_') + "#EXT#@" + $Environment + ".onmicrosoft.com"
$FoundUser = Get-MsolUser | ? UserPrincipalName -eq $externalConversionEmail
if($FoundUser){
Remove-MsolUser -UserPrincipalName $($FoundUser.UserPrincipalName) -Force
#To see All Deleted User Get-MsolUser -ReturnDeletedUsers
Remove-MsolUser -UserPrincipalName $($FoundUser.UserPrincipalName) -RemoveFromRecycelBin -Force
#To Remove All Deleted Users Get-MsolUsers -ReturnDeletedUsers | Remove-MsolUser -RemoveFromRecycleBin -Force
}

To do this manually, in admin.microsoft.com under Users -> Guest Users, select the user and click delete.

Then go into Users -> Deleted users and remove them from there.

Remove from SharePoint

To remove from SharePoint, if you have a large tenancy and you don’t know all the places where the external user could have been shared with, then you will have to use the following script. This script will remove the external user from the SharePoint Site, ensure that they are removed from the User Information list, and then lastly it will clear the person from the SharePoint User Profile.

I discovered that if I didn’t remove them from the User Profile, when attempted to reshare a document with that user, the people picker would grab the internal userprincipalname (<ExternalUserEmail>#EXT#@<Tenant>.onmicrosoft.com) as the email address and then prevent me clicking the Sharing button. This is because the people picker uses Graph API /Me/People and grabs the value from there. Once removed from everywhere, including the User Profile this no longer happens.

The following script uses SPO PowerShell Module and you will need to connect first using Connect-SPOService. The account that you use, needs to be a SharePoint Global Administrator.

The script checks if it can find the ExternalUser, and if it can remove the user using Remove-SPOExternalUser.

Then it loops through every site collection and looks for the user using Get-SPOUser with the internal userprincipalname. If found it removes the user using Remove-SPOUser. Once it has looped through all SharePoint sites, it then checks the SharePoint User Profile and removes the user from UserProfile Remove-SPOUserProfile. This command will remove a user from the UserProfile if they in the “Active Profiles” or the “Profiles Missing from Import”

<#
.SYNOPSIS
Loops through the SharePoint sites of the tenant, looking for the external user and removing them.
You need to have already connected to the Tenant as a SharePoint Global Adminstrator using Connect-SPOService -url:https://<tenant>-admin.sharepoint.com
.EXAMPLE
.\Remove-ExternalUserFromTenant.ps1 -Environment:<tenant> -UserEmail:<externalEmailAddres>
#For Tenant called Dev34223 and external email address fred.bloggs@outlookdomain.com
.\Remove-ExternalUserFromTenant.ps1 -Environment:Dev34223 -UserEmail:fred.bloggs@outlookdomain.com
#>
[CmdletBinding(SupportsShouldProcess)]
param(
[Parameter(Mandatory)]
[string]
$Environment,
[Parameter(Mandatory)]
[string]
$UserEmail
)
Clear-Host
$sites = Get-SPOSite -Limit ALL
$externalConversionEmail = ($UserEmail -replace '@', '_') + "#EXT#@" + $Environment + ".onmicrosoft.com"
$ErrorActionPreference = 'Stop'
$InformationPreference = 'Continue'
Write-Information -MessageData "Get $UserEmail External User within SharePoint"
$ExtUser = Get-SPOExternalUser -Filter $UserEmail
if ($null -ne $ExtUser) {
Write-Information -MessageData "Remove $UserEmail within SharePoint"
Remove-SPOExternalUser -UniqueIDs @($ExtUser.UniqueId) -Confirm:$false
}
$found = $false
$Sites | ForEach-Object {
$site = $PSItem
$i = $i + 1
try {
Get-SPOUser -site:$($site.Url) -LoginName:$externalConversionEmail
write-Information "Found user $UserEmail in site $($site.Title) Url:$($site.Url)"
Remove-SPOUser -site:$($site.Url) -LoginName:$externalConversionEmail
$found = $true;
}
catch {
#User not found.
}
Write-Progress -Activity "Removing User - $UserEmail" -Status "Progress:$($site.Url)" -PercentComplete ($i / $Sites.count * 100)
}
if ($found) {
Write-Information "User $UserEmail removed from SharePoint Sites"
}
else {
Write-Information "User $UserEmail wasn't found within SharePoint Sites"
}
Write-Information -MessageData "Remove $externalConversionEmail from SharePoint User profile"
try {
Remove-SPOUserProfile -LoginName $externalConversionEmail
}
catch {
Write-Information "Unable to find $externalConversionEmail in the user profiles."
}

If the plan is to add the external person back into your tenant, once the script has run, you will need to wait at least a few hours (maybe leave it for a day to be sure) to ensure all back end processes of Microsoft have completed.

When you share a document/folder with the external user they will get the invited link and enter a code experience, this way they do not turn up inside you Azure AD. However, if you share a site with them, or add them to a MS Teams, they will appear in your Azure AD correctly.

How does the privacy in O365 Groups/Teams affect the SharePoint Site


When you create a O365 Group or Teams you get given the option for Privacy. Either Public or Private. When an O365 group is created a Modern SharePoint Team site is created, but how is the site affected depending on the privacy?

Private Privacy

In a “Private” group, in the SharePoint site, the members of the O365 Group get added to the SharePoint Members Group.

And the owners of the O365 Group get added to the Site Collection Administrators.

Public Privacy

In a public site, the Members and Owners of the group get added to the SharePoint Members group and Site Collection Administrators respectively. The extra difference I have noticed, is that the “Everyone except external users” group is also added to the SharePoint members group.

What does this mean?

A private site is what it says. It is private. If it’s joined to a Teams anyone who is not a member of the group, cannot see MS Teams, search for site or documents inside it, or have access to the SharePoint site.

A public site is different. If it’s joined to Teams, a person who isn’t a member to the group will not see the MS Teams, meaning they cannot see channel chats, but can search to join it.

The documents however are available to them. They can search for these documents, access the SharePoint Site contribute on the documents too if they like.

Testing this

I have created two MS Teams. The first one is called Public OTB Permissions and set the privacy to Public. I am the only member of this site. The second site is called Private OTB and set the privacy to Private. Again, I am the only member of this site. In both “General” channels I have uploaded a Word document.

Public Privacy

Private Privacy


I have a second user called Alan Brewer, and in SharePoint I’m going to search for the word “Place” as this word is in both test documents.

Alan can only see the document from the public site, and when clicked through can access the document, contribute towards it. He can even view the whole SharePoint site. This is because he is being let into the site via the “Everyone except external users” claim that has been added to the SharePoint Members group for the site.

Just to complete this, I have added another user called Adam Barr to the Private site.

If I now sign in as Adam and perform the search “Place” like before, Adam can see both documents.

What happens when I switch the privacy for the O365 group.

When you decided you want to make a “Private site” public or a “Public site” Private, there is a process that happens in the back end which will either add/remove the “Everyone Except External users” to the SharePoint members group. I’ve noticed this can take up to 15 minutes or so before it is reflected in the SharePoint site.

Should I remove/change the “Everyone Except External users” from a public SharePoint site?

You can, but I don’t recommend it. If like me, personally it might be better to have the Everyone Except External Users in the Visitors group. Allowing the site to still be public, but you must be a member to contribute or change anything. The trouble with this, the site would be set up they way you want it, but then if an Owner decides to change the O365 Groups Privacy to Private, the backend process does not remove the “Everyone Except External Users” from the SharePoint Visitors Group, leaving the documents open to all. Then if they decided to switch it back or change an O365 Group from Private to Public Privacy, the backend process would add the “Everyone Except External Users” to the SharePoint members group. Making the site now open and everyone can contribute again. You could train all owners to ensure they remove/add this group if they switch the privacy, but who will remember to do that!

You could, if you are a developer, have a webhook using MS Graph that is attached to monitor O365 Group changes. https://docs.microsoft.com/en-us/graph/webhooks. Although possible, you would need to ensure that there is a delay from capturing the O365 Group webhook privacy change, and updating SharePoint, as you need to run after the backend process has been and flipped the “Everyone Except External Users”. If switching from Public to Private policy, any process you develop, if runs too late would leave the site open for a given amount of time, or if it fails could leave the site open where the users believe it is all locked down.

Viewing, Restoring and Removing Items from the SharePoint Recycle Bin – The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator.


I’ve had a script for a while that allows you to view all the items in the Recycle Bin for a Site Collection and prints out to a CSV file. Recently the environment I’ve been running this in has been throwing an error saying;

The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator“.

Getting all items out of the recycle bin.

Originally, I used the PNP Powershell command Get-PnPRecycleBinItem and it was only when I did a Google search for this issue, I found that other people were also having this problem. The PnP team have solved this issue now by adding -RowLimit parameter. If you set the RowLimit high enough you can return all items, as internally, it seems to implement a paging mechanism.

I now use the below script to export the result to a CSV file.

<#
.SYNOPSIS
Loops through the recycle bin and output a csv string.
Uses PNP Powershell.
.EXAMPLE
-URL:'https://<tenant&gt;.sharepoint.com/sites/<siteCollection>' -Stage:First -Path:.\FirstRecycleBin.csv
-URL:'https://<tenant&gt;.sharepoint.com/sites/<siteCollection>' -Stage:First -Path:.\FirstRecycleBin.csv -RowLimit:200000
#>
[CmdletBinding(SupportsShouldProcess)]
param(
# The url to the site containing the Site Requests list
[Parameter(Mandatory)][string]$URL,
[Parameter(Mandatory)][ValidateSet("First", "Second")][string]$Stage,
[Parameter(Mandatory)][string]$Path,
[int]$RowLimit=150000
)
Connect-PnPOnline -Url:$URL -UseWebLogin
Write-Host "Getting recycle bin items..."
$RecycleStage;
if ($Stage -eq "First") {
$RecycleStage = Get-PnPRecycleBinItem -FirstStage -RowLimit 150000
}
else {
$RecycleStage = Get-PnPRecycleBinItem -SecondStage -RowLimit 150000
}
$Output = @()
$RecycleStage | ForEach-Object {
$Item = $PSItem
$Obj = "" | Select-Object Title, AuthorEmail, AuthorName, DeletedBy, DeletedByEmail, DeletedDate, Directory, ID, ItemState, ItemType, LeafName, Size
$Obj.Title = $Item.Title
$Obj.AuthorEmail = $Item.AuthorEmail
$Obj.AuthorName = $Item.AuthorName
$Obj.DeletedBy = $Item.DeletedByName
$Obj.DeletedByEmail = $Item.DeletedByEmail
$Obj.DeletedDate = $Item.DeletedDate
$Obj.Directory = $Item.DirName
$Obj.ID = $Item.ID
$Obj.ItemState = $Item.ItemState
$Obj.ItemType = $Item.ItemType
$Obj.LeafName = $Item.LeafName
$Obj.Size = $Item.Size
$output += $Obj
}
$Output | Export-csv $Path -NoTypeInformation
Write-Host "Done"

Once I have the CSV file, I’m able to filter further in excel and save back to CSV to use to either Restore / Delete the items out of the recycle bin.

Restoring Deleted Items using a csv file.

It seemed that now that I can use RowLimit with Get-PnPRecycleBinItem I should be able to call Restore-PnpRecycleBinItem to restore the item. However, this isn’t the case. Even just passing the Identity of one item within the Recycle Bin, you get the same error message.

The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator“.

There is no RowLimit option on the Restore-PnpRecycleBinItem. The code must internally make a call to get all RecycleBin Items first without using RowLimit. Interestingly though, a user could go to a recycle bin, see items, and restore them if they wanted to. By looking through the network traffic, I was able to see that the GUI uses the following API to Restore Items.

POST /_api/site/RecycleBin/RestoreByIds

Passing in the following JSON body.

{
"ids": [
"b1a30d73-917a-4fdc-82f0-ba6e9881710b",
"2dbce811-cbaa-4818-a458-6ef3de70530b",
"17a0d047-efbb-4524-b7f4-32289b01cc3c",
"8c90ef0d-8b2d-468d-8b8e-f9af4c10f58b"
]
}

There can be one or many Ids.

The trouble with using REST API you need an accessToken. Using the Invoke-PnPSPRestMethod it automatically provides the AccessToken in the call.

This is what I do in the below code. Loop through every item in a CSV file to restore, and call “/_api/sites/RecycleBin/RestoreByIds” using Invoke-PnPSPRestMethod.

[CmdletBinding(SupportsShouldProcess)]
param(
# The URL of the Sitecollection where the recycle bin is.
[Parameter(Mandatory)]
[string]
$SiteUrl,
# Full Path of CSV file of Get-AllRecycleBin.ps1
[Parameter(Mandatory)]
[string]
$Path
)
function Restore-RecycleBinItem {
param(
[Parameter(Mandatory)]
[String]
$Id
)
$siteUrl = (Get-PnPSite).Url
$apiCall = $siteUrl + "/_api/site/RecycleBin/RestoreByIds"
$body = "{""ids"":[""$Id""]}"
Write-Verbose "Performing API Call to Restore item from RecycleBin..."
try {
Invoke-PnPSPRestMethod -Method Post -Url $apiCall -Content $body | Out-Null
}
catch {
Write-Error "Unable to Restore ID {$Id}"
}
}
$ErrorActionPreference = 'Continue'
$InformationPreference = 'Continue'
Connect-PnPOnline -Url:$SiteUrl -UseWebLogin
@($(Import-Csv -Path:"$Path")).ForEach({
$csv = $PSItem
Write-Information -MessageData:"Restore item $($csv.Title)"
Restore-RecycleBinItem -Id $($csv.ID)
})

Deleting Deleted Items using a csv file.

I discovered that I also get the error message when using Clear-PnpRecycleBinItem.

Again, I was able to do this in the GUI, and looking at the network traffic there is an API to delete the items.

POST /_api/site/RecycleBin/DeleteByIds

The JSON body is same format as the RestoreByIds, where it passes in one or many Ids.

The code below is almost identical to the Restore-RecycleBinItems.ps1. Passing in a CSV file with the IDs of files to delete permanently.

[CmdletBinding(SupportsShouldProcess)]
param(
# The URL of the Sitecollection where the recycle bin is.
[Parameter(Mandatory)]
[string]
$SiteUrl,
# Full Path of CSV file of Get-AllRecycleBin.ps1
[Parameter(Mandatory)]
[string]
$Path
)
function Clear-RecycleBinItem {
param(
[Parameter(Mandatory)]
[String]
$Id
)
$siteUrl = (Get-PnPSite).Url
$apiCall = $siteUrl + "/_api/site/RecycleBin/DeleteByIds"
$body = "{""ids"":[""$Id""]}"
Write-Verbose "Performing API Call to delete item from RecycleBin..."
try {
Invoke-PnPSPRestMethod -Method Post -Url $apiCall -Content $body | Out-Null
}
catch {
Write-Error "Unable to Delete ID {$Id}"
}
}
$ErrorActionPreference = 'Continue'
$InformationPreference = 'Continue'
Connect-PnPOnline -Url:$SiteUrl -UseWebLogin
@($(Import-Csv -Path:"$Path")).ForEach({
$csv = $PSItem
Write-Information -MessageData:"Delete item $($csv.Title)"
Clear-RecycleBinItem -Id $($csv.ID)
})

Dive into the code for O365 Audit logs webhooks


This is part two of a 2-part blog post.

  1. Walkthrough Setting up WebHook for O365 Audit Logs
  2. Dive into the code for O365 Audit Log webhooks to see how it works – (This Post)

The previous blog post showed how to get you up and running with O365 Audit logs and webhooks. In this blog post I’m going to show and explain parts of the code that ties everything together.

The full code can be found at my Github repo https://github.com/pmatthews05/O365AuditWebHook

PowerShell to initialize the Webhook to the Audit logs

.\Set-AuditLogs.ps1 -ClientId:<ClientID>
-ClientSecret:<AppSecret>
-TenantDomain:<Tenant>.onmicrosoft.com
-TenantGUID:<Directory ID>
-WebHookUrl:https://<Environment>-auditwebhook.azurewebsites.net/API/AuditWebHook
-ContentType:Audit.SharePoint

Run on one line.

From inside the PowerShell folder (.\O365AuditWebhook\PowerShell) there is a PowerShell file called Set-AuditLogs.ps1 This PowerShell file Starts a subscription to the given Audit Content Type. This is done by calling:

https://manage.office.com/api/v1.0/{tenant_id}/activity/feed/start?contentType={ContentType}

The above call is a POST call and uses the ClientID and Secret to authenticate against the tenant. The body is a Json object

{
"webhook": {
"authId": "365notificationaad_Audit.Sharepoint",
"expiration": "",
"address": "https://environment-auditwebhook.azurewebsites.net/API/AuditWebHook"
}
}
  • authId – Optional string that will be included as the WebHook-AuthID header in notifiations sent to the webhook as a means of identifying and authorizing the source of the request to the webhook
  • expiration – Optional datetime that indicates the datatime after which notifications should no longer be sent to the webhook. By leaving it empty, indicates the subscription will be active for the next 180 days.
  • address – Required HTTPS endpoint that can receive notifications. A test message will be sent to the webhook to validate the webhook before creating the subscription.

When the /start operation is called, the webhook URL specified in the address will be sent a validation notification to validate that an active listener can accept and process notifications.

The Azure Function AuditWebhook found in the O365AuditWebhook.cs file has two parts to it.

string stringvalue = await req.Content.ReadAsStringAsync();
log.LogInformation($"Req.Content {stringvalue}");
try
{
log.LogInformation("Getting validation code");
dynamic data = await req.Content.ReadAsAsync<object>();
string validationToken = data.validationCode.ToString();
log.LogInformation($"Validation Token: {validationToken} received");
HttpResponseMessage response = req.CreateResponse(HttpStatusCode.OK);
response.Content = new StringContent(validationToken);
return response;
}
catch (Exception)
{
log.LogInformation("No ValidationCode, therefore process WebHook as content");
}

The first part, as shown above, handles the validation. It looks for a validation code within the content, and if found it response back with a 200 status (OK) and includes the validation code.

If an OK is not received back, then the webhook will not be added and the subscription will remain unchanged.

The second part of the AuditWebhook Azure function is explained in the next section.

Webhook handling O365 notifications

After the initial validation, notifications will be sent to the webhook as the content logs become available.

From the first part of the AuditWebHook Azure Function, notifications do not have the validationCode, this allows us to determine that notifications have been sent, instead of a new subscription.

The content of these notifications contains an array of one or more JSON objects that represent the available content blobs.

log.LogInformation($"Audit Logs triggered the webhook");
string content = await req.Content.ReadAsStringAsync();
log.LogInformation($"Received following payload: {content}");
List<AuditContentEntity> auditContents = JsonConvert.DeserializeObject<List<AuditContentEntity>>(content);
foreach (var auditcontent in auditContents)
{
if (AuditContentUriQueue == null)
{
string cloudStorageAccountConnectionString = System.Environment.GetEnvironmentVariable("AzureWebJobsStorage", EnvironmentVariableTarget.Process);
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(cloudStorageAccountConnectionString);
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
AuditContentUriQueue = queueClient.GetQueueReference("auditcontenturi");
await AuditContentUriQueue.CreateIfNotExistsAsync();
}
log.LogInformation($"Content Queue Message: {auditcontent.ContentUri}");
AuditContentQueue acq = new AuditContentQueue
{
ContentType = auditcontent.ContentType,
ContentUri = auditcontent.ContentUri,
TenantID = auditcontent.TenantId
};
string message = JsonConvert.SerializeObject(acq);
log.LogInformation($"Adding a message to the queue. Message content: {message}");
await AuditContentUriQueue.AddMessageAsync(new CloudQueueMessage(message));
log.LogInformation($"Message added")
}
return new HttpResponseMessage(HttpStatusCode.OK);

On line 5 of the above code, show where I handle the content of deserialize json object (notifications) to a list of AuditContentEntity.

The notification/AuditContentEntity contains the following:

  • tenantId
    The GUID of the tenant to which the content belongs
  • clientId – The GUID of your application that created the subscription
  • contentType – Indicates the content type
  • contentId – An opaque string that uniquely identifies the content
  • contentUri – The URL to use when retrieving the content
  • contentCreated – The datetime when the content was made available
  • contentExpiration – The datetime after which the content will no longer be available for retrieval.

At this point you do not have any log information, you just have a collection of contentUri which when called will provide you with the logs. To ensure that the webhook response quickly so that it can continue to handle incoming requests, we place the contentUri, contentType, and TenantId onto an Azure Storage Queue. This allows a different Azure function to handle getting the actual logs.

Lines 9-16 will set up the storage queue if it doesn’t exist.

Lines 19-26 prepares my queue object and serialize it to a json string.

Line 28 adds the message to the Azure Storage Queue.

Once all notifications/AuditContentEntity have been processed, a 200 status (OK) is passed back. The subscription that calls our webhook is waiting for an OK response. If it encounters failure, it has a built in retry mechanism that will exponentially increase the time between retries. If the subscription continues to receive failure response, the subscription can disable the webhook and stop sending notifications. The subscription will need to be started again to re-enable the disabled webook.

Processing the Storage Queue AuditContentUri

As items are put on the Storage Queue the Azure Function AuditContentUri found in the O365AuditWebhook.cs file fires.

string token = await AcquireTokenForApplication();
var uri = auditContentQueue.ContentUri;
do
{
uri = uri.Contains("?") ? $"{uri}&PublisherIdentifier={auditContentQueue.TenantID}" : $"{uri}?PublisherIdentifier={auditContentQueue.TenantID}";
log.LogInformation($"URL:{uri}");
var results = await RestAPI.GetRestDataAsync(uri, token);
var array = JArray.Parse(results.RestResponse);
foreach(var logEntry in array)
{
log.LogInformation(logEntry.ToString());
}
uri = results.WebHeaderCollections.Get("NextPageUri");
} while (uri != null);
view raw AuditContent.cs hosted with ❤ by GitHub

First you need an authorization token to read the audit logs, we do this with AcquireTokenForApplication method. This uses the Tenant Name, ClientId and Secret that is stored within your Azure configurations. See ‘How to acquire token for application?’ below.

It grabs the ContentUri and then goes into a do loop. This is because the logs that come back, if it is a very busy tenant, not all the logs will be returned, and there will be a NextPageUri value in the header of the response to allow you to obtain the next page of logs.

Line 7 – This adds your tenantID to the end of the URI as a PublisherIdentifier. This parameter is used for throttling the request rate. Make sure this parameter is specified in all issued requests to get a dedicated quota. All requests received without this parameter will share the same quota. The IF statement ensures it is added to the end of the URI correctly.

Line 9 – This calls the ContentUri and gets a results and request headers. You can see the file .\O365AuditWebHook\AuditWebHook\Utilities\RestAPI.cs
The Method GetRestDataAsync is very similar to the GetRestData call you find within PNP Core code. Creates a HttpWebRequest, passing in Authorization Token, and calling the ContentUri. Only difference in my code is that I’m grabbing the response.Headers to find out if there are additional logs, and passes them back with the results.

Line 10 – This parse the results into a JArray. (Json Array object). Here you can manipulate what comes back. For example, instead of grabbing all results and then displaying them out, you can query the results for a particular log type.

In the example code below, this would be using the Audit.General logs, and it will grab any logs that are of RecordType 25 (Indicates Microsoft Teams event) where the operation is creating a new channel, and the Channel type is Private. I then convert the JArray to an object list of AuditGeneralEntity.

For further details about properties of the audit logs can be found here: https://docs.microsoft.com/en-us/microsoft-365/compliance/detailed-properties-in-the-office-365-audit-log

Line 14 – Logs out an individual log entry, this is in a json format. Different schema’s can be found here: https://docs.microsoft.com/en-gb/office/office-365-management-api/office-365-management-activity-api-schema

Line 17 – If there are any additional pages, then this will return a value, and the loop will loop until no more pages are found.

How to acquire token for application?

In the previous section, I called a method AcquireTokenForApplication. This is a helper class and method that I use quite often, when I need to obtain an AccessToken. You can find this in the repo at .\O365AuditWebHook\AuditWebHook\Utilities\AuthenticationHelper.cs. This solution has a cut down version of the helper class I use. It is cut down as it just gets an access token for Audit Logs using AppId and Secret.

internal static async Task<string> AcquireTokenForApplication()
{
var tenant = System.Environment.GetEnvironmentVariable("Tenant", EnvironmentVariableTarget.Process);
var clientId = System.Environment.GetEnvironmentVariable("ClientId", EnvironmentVariableTarget.Process);
var secret = System.Environment.GetEnvironmentVariable("Secret", EnvironmentVariableTarget.Process);
var authorityUri = $"https://login.microsoftonline.com/{tenant}.onmicrosoft.com";
var resourceUri = "https://manage.office.com";
var microsoftToken = await GetTokenRetry(resourceUri, authorityUri, clientId, secret);
return microsoftToken;
}
private static async Task<string> GetTokenRetry(string resourceUri, string authorityUri, string clientId, string secret, int retryCount = 5, int delay = 500)
{
...
AuthenticationContext authContext = new AuthenticationContext(authorityUri, false);
ClientCredential clientCred = new ClientCredential(clientId, secret);
var authenticationResult = await authContext.AcquireTokenAsync(resourceUri, clientCred);
token = authenticationResult.AccessToken;
return token;
...
}

Above is a snippet, as you can see it is wrapped in a retry method in case there is throttling.

PowerShell to stop the Audit logs

Within the PowerShell folder I have also included a file called Remove-AuditLogs.ps1

.\Remove-AuditLogs.ps1 -ClientId:<ClientID>
-ClientSecret:<AppSecret>
-TenantDomain:<Tenant>.onmicrosoft.com
-TenantGUID:<Directory ID>
-WebHookUrl:https://<Environment>-auditwebhook.azurewebsites.net/API/AuditWebHook
-ContentType:Audit.SharePoint

Run on one line.

This works exactly like the Set-AuditLogs.ps1 file except it calls the /stop endpoint:

https://manage.office.com/api/v1.0/{tenant_id}/activity/feed/stop?contentType={ContentType}

Once the subscription is stopped, no notifications will be sent to your webhook, and you will not be able to retrieve available content. Please note, if you decide to start the subscription again later using the Set-AuditLogs.ps1 you will not receive any content that was available between the stop and start time of the subscription.

 

This is quite a heavy post; I hope it has helped you in some way. It is just a starter, as you will probably want to do something with the logs instead of just writing them out to the Azure Logs. Maybe capturing a given process to then implement some logic to react. You might also want to put different Audit content types ContentUri onto different Azure Storage queue, so that different Azure Functions can process the ContentUri.