Category Archives: SharePoint

App-Only Auth Connect to SharePoint Online with MSAL and Azure KeyVault

Posted on by

Now that SharePoint Online CSOM is now works with .NET Framework, I thought I would put together a demo using Visual Studio code that connects to SharePoint.

A previous colleague and still good friend of mine Vardhaman Deshpande in June wrote a blog post showing how to connect to SharePoint Online using MSAL. It is so well written that really writing another blog about it seems a little pointless, so I have taken his blog a little further by connecting to a KeyVault in Azure and grabbing the certificate directly from there.

.NET Standard CSOM of SharePoint Online now uses OAuth for authentication. This means an Access Token needs to be grabbed and passed to every call that is made to SharePoint Online. We will do this by grabbing the AppID and Certificate from the KeyVault and then get the Access Token through ConfidentialClientApplicationBuilder. The Access Token will then be passed into the ClientContext so all calls are made with the Access Token to SharePoint Online.

Walk-through Demo

The demo I have put together can be found at my GitHub repository. By using a Azure AD App Registration and a client certificate, I will walk-through the steps here to set up the following:

  • Create a Resource Group in Azure
  • Create a KeyVault
  • Create a Certificate and Store it in the KeyVault
  • Create an Azure AD App registration
  • Store the ClientID in the KeyVault Secrets
  • Grant Azure AD Application permission for SharePoint – Sites.FullControl.All
  • Console code that connects to SharePoint Online.

Setup

To perform all the steps above as a manual walk-through would take a lot of time to go through. Also, where I can automate things I do. Therefore in the GitHub project under the PowerShell Folder there is a PowerShell file called Install-AzureEnvironment.ps1.

This uses AZ Cli and running the below script will create the above for you in your Azure environment. Replace “Contso” with the name of your tenant.

az login
$tenantName = "contso"
# Defaults to UK South
.\Install-AzureEnvironment.ps1 -Environment $tenantName -Name "SharePointMSAL"
# If wish to change location
#.\Install-AzureEnvironment.ps1 -Environment $tenantName -Name "SharePointMSAL" -Location:'<Location>'
view raw SetUp-AppCertSharPoint.ps1 hosted with ❤ by GitHub

The above will create the following Azure Resources (using the example of Consto as tenant name)

  • Resource Group: Contso-SharePointMSAL
  • App Registration: Contso-SharePointMSAL (Granted with SharePoint > Sites.FullControl.All)
  • Key Vault: Contso-SharePointMSAL (Will be truncated to 24 characters if longer)
  • CertificateName stored in KeyVault Certificates: Contso-SharePointMSAL
  • ClientId stored in KeyVault Secret: ConstoSharePointMSAL

Console Application

Using Visual Studio Code, I’ve create a .NET Core 3.1 console application and added the following nuget packages. Please see my previous blog post “Basic dotnet commands to create C# project in Visual Studio Code” on how to create a Console application and add nuget packages.

  • Microsoft.SharePointOnline.CSOM
    • Used for the SharePoint CSOM calls
  • Microsoft.Identity.Client
    • Used for OAuth authentication
  • Azure.Identity
    • Used for KeyVault authentication
  • Azure.Security.KeyVault.Secrets
    • Used for getting the Secret and Certificate from the vault
  • Microsoft.Extensions.Configuration
    • Used for collecting app.config values
  • Microsoft.Externsions.Configuration.FileExtensions
    • Used for collecting app.config values
  • Microsoft.Extensions.Configuration.Json
    • Used for collecting app.config values

Next you will need to create (or update if cloned the github project) the appsettings.json file. Replace the environment and site values for your environment.

{
"environment": "<tenantName>",
"name": "SharePointMSAL",
"site": "<relative URL e.g, /sites/teamsite>"
}
view raw appsettings.json hosted with ❤ by GitHub

Then update the program.cs file with the following code. The code has been written assuming your Azure resources using the .\Install-AzureEnvironment.ps1.

using System;
using Microsoft.Identity.Client;
using Microsoft.SharePoint.Client;
using System.Security.Cryptography.X509Certificates;
using System.Threading.Tasks;
using Microsoft.Extensions.Configuration;
using Azure.Identity;
using Azure.Security.KeyVault.Secrets;
namespace SharePointMSAL
{
class Program
{
static async Task Main(string[] args)
{
IConfiguration config = new ConfigurationBuilder()
.AddJsonFile("appsettings.json", true, true)
.Build();
string siteUrl = $"https://{config["environment"]}.sharepoint.com{config["site"]}";
string identity = $"{config["environment"]}-{config["name"]}";
string keyVaultName = GetKeyVaultName(identity);
string certificateName = identity;
string tenantId = $"{config["environment"]}.onmicrosoft.com";
string clientIDSecret = identity.Replace("_","").Replace("-","");
string clientId = GetSecretFromKeyVault(keyVaultName, clientIDSecret);
//For SharePoint app only auth, the scope will be the Sharepoint tenant name followed by /.default
var scopes = new string[] { $"https://{config["environment"]}.sharepoint.com/.default" };
var accessToken = await GetApplicationAuthenticatedClient(clientId, keyVaultName, certificateName, scopes, tenantId);
var ctx = GetClientContextWithAccessToken(siteUrl, accessToken);
Web web = ctx.Web;
ctx.Load(web);
await ctx.ExecuteQueryAsync();
Console.WriteLine(web.Title);
}
private static string GetKeyVaultName(string identity)
{
var keyVaultName = identity;
if (keyVaultName.Length > 24)
{
keyVaultName = keyVaultName.Substring(0, 24);
}
return keyVaultName;
}
private static async Task<string> GetApplicationAuthenticatedClient(string clientId, string keyVaultName, string certificateName, string[] scopes, string tenantId)
{
var certificate = GetAppOnlyCertificate(keyVaultName, certificateName);
IConfidentialClientApplication clientApp = ConfidentialClientApplicationBuilder
.Create(clientId)
.WithCertificate(certificate)
.WithTenantId(tenantId)
.Build();
AuthenticationResult authResult = await clientApp.AcquireTokenForClient(scopes).ExecuteAsync();
string accessToken = authResult.AccessToken;
return accessToken;
}
public static ClientContext GetClientContextWithAccessToken(string targetUrl, string accessToken)
{
ClientContext clientContext = new ClientContext(targetUrl);
clientContext.ExecutingWebRequest += delegate (object oSender, WebRequestEventArgs webRequestEventArgs)
{
webRequestEventArgs.WebRequestExecutor.RequestHeaders["Authorization"] = "Bearer " + accessToken;
};
return clientContext;
}
public static X509Certificate2 GetAppOnlyCertificate(string keyVaultName, string certificateName)
{
var keyVaultUrl = $"https://{keyVaultName}.vault.azure.net";
var client = new SecretClient(new Uri(keyVaultUrl), new DefaultAzureCredential());
KeyVaultSecret keyVaultSecret = client.GetSecret(certificateName);
X509Certificate2 certificate = new X509Certificate2(Convert.FromBase64String(keyVaultSecret.Value), string.Empty,
X509KeyStorageFlags.MachineKeySet |
X509KeyStorageFlags.PersistKeySet |
X509KeyStorageFlags.Exportable);
return certificate;
}
public static string GetSecretFromKeyVault(string keyVaultName, string secretName){
var keyVaultUrl = $"https://{keyVaultName}.vault.azure.net";
var client = new SecretClient(new Uri(keyVaultUrl), new DefaultAzureCredential());
KeyVaultSecret keyVaultSecret = client.GetSecret(secretName);
return keyVaultSecret.Value;
}
}
}
view raw CSOM.Standard.MSAL.Auth.AzureKeyVault.cs hosted with ❤ by GitHub

After the code runs it will display the site Title. In my case ‘TestAPISite’.

The important piece of code to get the certificate from Azure Key Vault is GetAppOnlyCertificate function on line 78. This is using the new Azure.Identity and Azure.Security.KeyVault.Secrets libraries.

The Azure.Identity information can be found here: https://docs.microsoft.com/en-us/dotnet/api/overview/azure/identity-readme?view=azure-dotnet

The key to authentication to the KeyVault is on line 82 using DefaultAzureCredential, as this attempts to connect using different authentication methods. Once connected, it retrieves the certificate value and creates a X509Certificate2 certificate in memory. The only confusing part of the code is using Azure.Security.KeyVault.Secrets to get the value not Azure.Security.KeyVault.Certificates.

The image below taken from the Microsoft documentation, shows how the DefaultAzureCredential will attempt to authenticate via the following mechanisms in order.

https://docs.microsoft.com/en-us/dotnet/api/overview/azure/identity-readme?view=azure-dotnet#defaultazurecredential
  • Environment – The DefaultAzureCredential will read account information specified via environment variables and use it to authenticate.
  • Managed Identity – If the application is deployed to an Azure host with Managed Identity enabled, the DefaultAzureCredential will authenticate with that account.
  • Visual Studio – If the developer has authenticated via Visual Studio, the DefaultAzureCredential will authenticate with that account.
  • Visual Studio Code – If the developer has authenticated via the Visual Studio Code Azure Account plugin, the DefaultAzureCredential will authenticate with that account.
  • Azure CLI – If the developer has authenticated an account via the Azure CLI az login command, the DefaultAzureCredential will authenticate with that account.
  • Interactive – If enabled the DefaultAzureCredential will interactively authenticate the developer via the current system’s default browser.

Note: When I first ran my code on Visual Studio Code, I kept getting an authentication issue, it was because I had Visual Studio Enterprise installed on my machine and it was picking up the authentication method selected there which was pointing to a different tenant. You can see this in Visual Studio Enterprise by going into Tools > Options

The great thing about DefaultAzureCredential, is that if this code was within a Azure Function, I could run it first on my computer, then deploy it to Azure Functions with a Managed Identity, and it would still work without any changes to the code.

I hope you find this blog post useful.

Removing External Users fully from a SharePoint Tenancy using PowerShell

Posted on by

This blog post has all come about as the client I was working for was having problems sharing documents in SharePoint with some external users. It turned out that the user was already in Azure AD as a Contact which is part of Exchange. This meant when an internal person attempted to share/Invite into SharePoint/MSTeams it all appeared to work correctly for the external user, but sometimes it didn’t. When looking at external users through the Admin portal, this external user was showing, but their email address was blank. After speaking with Microsoft, it turns out, because the email address was already found within the tenancy, it creates a unique violation when adding the external user to the Active Directory.

I have been working with Microsoft support regarding this, and the resolution was that this is as design!!??! Only by feeding back on the Office 365 uservoice this issue “might” looked at and fixed. See resolution notes below:

Symptom:
When you invite external users who exist as contacts in your environment, their email does not get populated in their guest user ID which results in them not being able to login to your environment and access the shared data.
Cause:
The issue is coming from a conflict caused by the email address which is already populated for the mail contact.
Resolution:
This is behavior by design as all objects in Azure AD have to be unique.
You cannot have 2 objects with the same email address.

When you invite one of your contacts to your content in O365, it actually creates a completely new guest user object in your environment and since the email address which is supposed to be populated in the email attribute is already in use by the contact, the email address does not get populated.

The only way to resolve this issue at the moment is to eliminate any conflicts that are in place, by removing the conflicting email contact and re-invite the user to your content.
More information:
The best thing I can offer to you is the following:

Please go to our UserVoice portal where other people are facing the same behavior and up-vote it, comment and have the whole IT department do the same as well.

Allow a “Guest User” to be converted to a different account type
https://office365.uservoice.com/forums/273493-office-365-admin/suggestions/19966537-allow-a-guest-user-to-be-converted-to-a-differen

This led me to working on a process and script that would remove the users from everywhere.

Locations to remove the External User from:

  • Contacts
  • Azure AD Guest Users
  • Azure AD Deleted Users
  • All SharePoint Sites
  • All SharePoint Hidden User lists
  • SharePoint User Profile

Contacts

To remove the External User from the contacts you will need to use the MSOL PowerShell module.

$UserEmail = "<ExternalUserEmailAddress>"
Connect-MsolService
Get-MsolContact | ? EmailAddress -eq $UserEmail | Remove-MsolContact -Force
view raw Remove-ContactFromTenant.ps1 hosted with ❤ by GitHub

Or you can manually do this by going to admin.microsoft.com and under Users -> Contacts select the user and click Delete contacts.

Azure AD

To remove the External User from Azure AD you will still require using the MSOL PowerShell module. In fact, this script and the above script could be merged.

$Environment = "<TenantName>"
$UserEmail = "<ExternalUserEmailAddress>"
Connect-MsolService
$externalConversionEmail = ($UserEmail -replace '@', '_') + "#EXT#@" + $Environment + ".onmicrosoft.com"
$FoundUser = Get-MsolUser | ? UserPrincipalName -eq $externalConversionEmail
if($FoundUser){
Remove-MsolUser -UserPrincipalName $($FoundUser.UserPrincipalName) -Force
#To see All Deleted User Get-MsolUser -ReturnDeletedUsers
Remove-MsolUser -UserPrincipalName $($FoundUser.UserPrincipalName) -RemoveFromRecycelBin -Force
#To Remove All Deleted Users Get-MsolUsers -ReturnDeletedUsers | Remove-MsolUser -RemoveFromRecycleBin -Force
}
view raw Remove-ExternalAzureUserFromAD.ps1 hosted with ❤ by GitHub

To do this manually, in admin.microsoft.com under Users -> Guest Users, select the user and click delete.

Then go into Users -> Deleted users and remove them from there.

Remove from SharePoint

To remove from SharePoint, if you have a large tenancy and you don’t know all the places where the external user could have been shared with, then you will have to use the following script. This script will remove the external user from the SharePoint Site, ensure that they are removed from the User Information list, and then lastly it will clear the person from the SharePoint User Profile.

I discovered that if I didn’t remove them from the User Profile, when attempted to reshare a document with that user, the people picker would grab the internal userprincipalname (<ExternalUserEmail>#EXT#@<Tenant>.onmicrosoft.com) as the email address and then prevent me clicking the Sharing button. This is because the people picker uses Graph API /Me/People and grabs the value from there. Once removed from everywhere, including the User Profile this no longer happens.

The following script uses SPO PowerShell Module and you will need to connect first using Connect-SPOService. The account that you use, needs to be a SharePoint Global Administrator.

The script checks if it can find the ExternalUser, and if it can remove the user using Remove-SPOExternalUser.

Then it loops through every site collection and looks for the user using Get-SPOUser with the internal userprincipalname. If found it removes the user using Remove-SPOUser. Once it has looped through all SharePoint sites, it then checks the SharePoint User Profile and removes the user from UserProfile Remove-SPOUserProfile. This command will remove a user from the UserProfile if they in the “Active Profiles” or the “Profiles Missing from Import”

<#
.SYNOPSIS
Loops through the SharePoint sites of the tenant, looking for the external user and removing them.
You need to have already connected to the Tenant as a SharePoint Global Adminstrator using Connect-SPOService -url:https://<tenant>-admin.sharepoint.com
.EXAMPLE
.\Remove-ExternalUserFromTenant.ps1 -Environment:<tenant> -UserEmail:<externalEmailAddres>
#For Tenant called Dev34223 and external email address fred.bloggs@outlookdomain.com
.\Remove-ExternalUserFromTenant.ps1 -Environment:Dev34223 -UserEmail:fred.bloggs@outlookdomain.com
#>
[CmdletBinding(SupportsShouldProcess)]
param(
[Parameter(Mandatory)]
[string]
$Environment,
[Parameter(Mandatory)]
[string]
$UserEmail
)
Clear-Host
$sites = Get-SPOSite -Limit ALL
$externalConversionEmail = ($UserEmail -replace '@', '_') + "#EXT#@" + $Environment + ".onmicrosoft.com"
$ErrorActionPreference = 'Stop'
$InformationPreference = 'Continue'
Write-Information -MessageData "Get $UserEmail External User within SharePoint"
$ExtUser = Get-SPOExternalUser -Filter $UserEmail
if ($null -ne $ExtUser) {
Write-Information -MessageData "Remove $UserEmail within SharePoint"
Remove-SPOExternalUser -UniqueIDs @($ExtUser.UniqueId) -Confirm:$false
}
$found = $false
$Sites | ForEach-Object {
$site = $PSItem
$i = $i + 1
try {
Get-SPOUser -site:$($site.Url) -LoginName:$externalConversionEmail
write-Information "Found user $UserEmail in site $($site.Title) Url:$($site.Url)"
Remove-SPOUser -site:$($site.Url) -LoginName:$externalConversionEmail
$found = $true;
}
catch {
#User not found.
}
Write-Progress -Activity "Removing User - $UserEmail" -Status "Progress:$($site.Url)" -PercentComplete ($i / $Sites.count * 100)
}
if ($found) {
Write-Information "User $UserEmail removed from SharePoint Sites"
}
else {
Write-Information "User $UserEmail wasn't found within SharePoint Sites"
}
Write-Information -MessageData "Remove $externalConversionEmail from SharePoint User profile"
try {
Remove-SPOUserProfile -LoginName $externalConversionEmail
}
catch {
Write-Information "Unable to find $externalConversionEmail in the user profiles."
}
view raw Remove-ExternalUserFromTenant.ps1 hosted with ❤ by GitHub

If the plan is to add the external person back into your tenant, once the script has run, you will need to wait at least a few hours (maybe leave it for a day to be sure) to ensure all back end processes of Microsoft have completed.

When you share a document/folder with the external user they will get the invited link and enter a code experience, this way they do not turn up inside you Azure AD. However, if you share a site with them, or add them to a MS Teams, they will appear in your Azure AD correctly.

How does the privacy in O365 Groups/Teams affect the SharePoint Site

Posted on by

When you create a O365 Group or Teams you get given the option for Privacy. Either Public or Private. When an O365 group is created a Modern SharePoint Team site is created, but how is the site affected depending on the privacy?

Private Privacy

In a “Private” group, in the SharePoint site, the members of the O365 Group get added to the SharePoint Members Group.

And the owners of the O365 Group get added to the Site Collection Administrators.

Public Privacy

In a public site, the Members and Owners of the group get added to the SharePoint Members group and Site Collection Administrators respectively. The extra difference I have noticed, is that the “Everyone except external users” group is also added to the SharePoint members group.

What does this mean?

A private site is what it says. It is private. If it’s joined to a Teams anyone who is not a member of the group, cannot see MS Teams, search for site or documents inside it, or have access to the SharePoint site.

A public site is different. If it’s joined to Teams, a person who isn’t a member to the group will not see the MS Teams, meaning they cannot see channel chats, but can search to join it.

The documents however are available to them. They can search for these documents, access the SharePoint Site contribute on the documents too if they like.

Testing this

I have created two MS Teams. The first one is called Public OTB Permissions and set the privacy to Public. I am the only member of this site. The second site is called Private OTB and set the privacy to Private. Again, I am the only member of this site. In both “General” channels I have uploaded a Word document.

Public Privacy

Private Privacy


I have a second user called Alan Brewer, and in SharePoint I’m going to search for the word “Place” as this word is in both test documents.

Alan can only see the document from the public site, and when clicked through can access the document, contribute towards it. He can even view the whole SharePoint site. This is because he is being let into the site via the “Everyone except external users” claim that has been added to the SharePoint Members group for the site.

Just to complete this, I have added another user called Adam Barr to the Private site.

If I now sign in as Adam and perform the search “Place” like before, Adam can see both documents.

What happens when I switch the privacy for the O365 group.

When you decided you want to make a “Private site” public or a “Public site” Private, there is a process that happens in the back end which will either add/remove the “Everyone Except External users” to the SharePoint members group. I’ve noticed this can take up to 15 minutes or so before it is reflected in the SharePoint site.

Should I remove/change the “Everyone Except External users” from a public SharePoint site?

You can, but I don’t recommend it. If like me, personally it might be better to have the Everyone Except External Users in the Visitors group. Allowing the site to still be public, but you must be a member to contribute or change anything. The trouble with this, the site would be set up they way you want it, but then if an Owner decides to change the O365 Groups Privacy to Private, the backend process does not remove the “Everyone Except External Users” from the SharePoint Visitors Group, leaving the documents open to all. Then if they decided to switch it back or change an O365 Group from Private to Public Privacy, the backend process would add the “Everyone Except External Users” to the SharePoint members group. Making the site now open and everyone can contribute again. You could train all owners to ensure they remove/add this group if they switch the privacy, but who will remember to do that!

You could, if you are a developer, have a webhook using MS Graph that is attached to monitor O365 Group changes. https://docs.microsoft.com/en-us/graph/webhooks. Although possible, you would need to ensure that there is a delay from capturing the O365 Group webhook privacy change, and updating SharePoint, as you need to run after the backend process has been and flipped the “Everyone Except External Users”. If switching from Public to Private policy, any process you develop, if runs too late would leave the site open for a given amount of time, or if it fails could leave the site open where the users believe it is all locked down.

Dive into the code for O365 Audit logs webhooks

Posted on by

This is part two of a 2-part blog post.

  1. Walkthrough Setting up WebHook for O365 Audit Logs
  2. Dive into the code for O365 Audit Log webhooks to see how it works – (This Post)

The previous blog post showed how to get you up and running with O365 Audit logs and webhooks. In this blog post I’m going to show and explain parts of the code that ties everything together.

The full code can be found at my Github repo https://github.com/pmatthews05/O365AuditWebHook

PowerShell to initialize the Webhook to the Audit logs

.\Set-AuditLogs.ps1 -ClientId:<ClientID>
-ClientSecret:<AppSecret>
-TenantDomain:<Tenant>.onmicrosoft.com
-TenantGUID:<Directory ID>
-WebHookUrl:https://<Environment>-auditwebhook.azurewebsites.net/API/AuditWebHook
-ContentType:Audit.SharePoint
view raw Call-SetAuditLogs.ps1 hosted with ❤ by GitHub

Run on one line.

From inside the PowerShell folder (.\O365AuditWebhook\PowerShell) there is a PowerShell file called Set-AuditLogs.ps1 This PowerShell file Starts a subscription to the given Audit Content Type. This is done by calling:

https://manage.office.com/api/v1.0/{tenant_id}/activity/feed/start?contentType={ContentType}

The above call is a POST call and uses the ClientID and Secret to authenticate against the tenant. The body is a Json object

{
"webhook": {
"authId": "365notificationaad_Audit.Sharepoint",
"expiration": "",
"address": "https://environment-auditwebhook.azurewebsites.net/API/AuditWebHook&quot;
}
}
view raw StartSubscription-JsonBody.json hosted with ❤ by GitHub
  • authId – Optional string that will be included as the WebHook-AuthID header in notifiations sent to the webhook as a means of identifying and authorizing the source of the request to the webhook
  • expiration – Optional datetime that indicates the datatime after which notifications should no longer be sent to the webhook. By leaving it empty, indicates the subscription will be active for the next 180 days.
  • address – Required HTTPS endpoint that can receive notifications. A test message will be sent to the webhook to validate the webhook before creating the subscription.

When the /start operation is called, the webhook URL specified in the address will be sent a validation notification to validate that an active listener can accept and process notifications.

The Azure Function AuditWebhook found in the O365AuditWebhook.cs file has two parts to it.

string stringvalue = await req.Content.ReadAsStringAsync();
log.LogInformation($"Req.Content {stringvalue}");
try
{
log.LogInformation("Getting validation code");
dynamic data = await req.Content.ReadAsAsync<object>();
string validationToken = data.validationCode.ToString();
log.LogInformation($"Validation Token: {validationToken} received");
HttpResponseMessage response = req.CreateResponse(HttpStatusCode.OK);
response.Content = new StringContent(validationToken);
return response;
}
catch (Exception)
{
log.LogInformation("No ValidationCode, therefore process WebHook as content");
}
view raw AuditWebhookTrigger-Part1.cs hosted with ❤ by GitHub

The first part, as shown above, handles the validation. It looks for a validation code within the content, and if found it response back with a 200 status (OK) and includes the validation code.

If an OK is not received back, then the webhook will not be added and the subscription will remain unchanged.

The second part of the AuditWebhook Azure function is explained in the next section.

Webhook handling O365 notifications

After the initial validation, notifications will be sent to the webhook as the content logs become available.

From the first part of the AuditWebHook Azure Function, notifications do not have the validationCode, this allows us to determine that notifications have been sent, instead of a new subscription.

The content of these notifications contains an array of one or more JSON objects that represent the available content blobs.

log.LogInformation($"Audit Logs triggered the webhook");
string content = await req.Content.ReadAsStringAsync();
log.LogInformation($"Received following payload: {content}");
List<AuditContentEntity> auditContents = JsonConvert.DeserializeObject<List<AuditContentEntity>>(content);
foreach (var auditcontent in auditContents)
{
if (AuditContentUriQueue == null)
{
string cloudStorageAccountConnectionString = System.Environment.GetEnvironmentVariable("AzureWebJobsStorage", EnvironmentVariableTarget.Process);
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(cloudStorageAccountConnectionString);
CloudQueueClient queueClient = storageAccount.CreateCloudQueueClient();
AuditContentUriQueue = queueClient.GetQueueReference("auditcontenturi");
await AuditContentUriQueue.CreateIfNotExistsAsync();
}
log.LogInformation($"Content Queue Message: {auditcontent.ContentUri}");
AuditContentQueue acq = new AuditContentQueue
{
ContentType = auditcontent.ContentType,
ContentUri = auditcontent.ContentUri,
TenantID = auditcontent.TenantId
};
string message = JsonConvert.SerializeObject(acq);
log.LogInformation($"Adding a message to the queue. Message content: {message}");
await AuditContentUriQueue.AddMessageAsync(new CloudQueueMessage(message));
log.LogInformation($"Message added")
}
return new HttpResponseMessage(HttpStatusCode.OK);
view raw AuditWebhookTrigger-Part2.cs hosted with ❤ by GitHub

On line 5 of the above code, show where I handle the content of deserialize json object (notifications) to a list of AuditContentEntity.

The notification/AuditContentEntity contains the following:

  • tenantId –
    The GUID of the tenant to which the content belongs
  • clientId – The GUID of your application that created the subscription
  • contentType – Indicates the content type
  • contentId – An opaque string that uniquely identifies the content
  • contentUri – The URL to use when retrieving the content
  • contentCreated – The datetime when the content was made available
  • contentExpiration – The datetime after which the content will no longer be available for retrieval.

At this point you do not have any log information, you just have a collection of contentUri which when called will provide you with the logs. To ensure that the webhook response quickly so that it can continue to handle incoming requests, we place the contentUri, contentType, and TenantId onto an Azure Storage Queue. This allows a different Azure function to handle getting the actual logs.

Lines 9-16 will set up the storage queue if it doesn’t exist.

Lines 19-26 prepares my queue object and serialize it to a json string.

Line 28 adds the message to the Azure Storage Queue.

Once all notifications/AuditContentEntity have been processed, a 200 status (OK) is passed back. The subscription that calls our webhook is waiting for an OK response. If it encounters failure, it has a built in retry mechanism that will exponentially increase the time between retries. If the subscription continues to receive failure response, the subscription can disable the webhook and stop sending notifications. The subscription will need to be started again to re-enable the disabled webook.

Processing the Storage Queue AuditContentUri

As items are put on the Storage Queue the Azure Function AuditContentUri found in the O365AuditWebhook.cs file fires.

string token = await AcquireTokenForApplication();
var uri = auditContentQueue.ContentUri;
do
{
uri = uri.Contains("?") ? $"{uri}&PublisherIdentifier={auditContentQueue.TenantID}" : $"{uri}?PublisherIdentifier={auditContentQueue.TenantID}";
log.LogInformation($"URL:{uri}");
var results = await RestAPI.GetRestDataAsync(uri, token);
var array = JArray.Parse(results.RestResponse);
foreach(var logEntry in array)
{
log.LogInformation(logEntry.ToString());
}
uri = results.WebHeaderCollections.Get("NextPageUri");
} while (uri != null);
view raw AuditContent.cs hosted with ❤ by GitHub

First you need an authorization token to read the audit logs, we do this with AcquireTokenForApplication method. This uses the Tenant Name, ClientId and Secret that is stored within your Azure configurations. See ‘How to acquire token for application?’ below.

It grabs the ContentUri and then goes into a do loop. This is because the logs that come back, if it is a very busy tenant, not all the logs will be returned, and there will be a NextPageUri value in the header of the response to allow you to obtain the next page of logs.

Line 7 – This adds your tenantID to the end of the URI as a PublisherIdentifier. This parameter is used for throttling the request rate. Make sure this parameter is specified in all issued requests to get a dedicated quota. All requests received without this parameter will share the same quota. The IF statement ensures it is added to the end of the URI correctly.

Line 9 – This calls the ContentUri and gets a results and request headers. You can see the file .\O365AuditWebHook\AuditWebHook\Utilities\RestAPI.cs
The Method GetRestDataAsync is very similar to the GetRestData call you find within PNP Core code. Creates a HttpWebRequest, passing in Authorization Token, and calling the ContentUri. Only difference in my code is that I’m grabbing the response.Headers to find out if there are additional logs, and passes them back with the results.

Line 10 – This parse the results into a JArray. (Json Array object). Here you can manipulate what comes back. For example, instead of grabbing all results and then displaying them out, you can query the results for a particular log type.

In the example code below, this would be using the Audit.General logs, and it will grab any logs that are of RecordType 25 (Indicates Microsoft Teams event) where the operation is creating a new channel, and the Channel type is Private. I then convert the JArray to an object list of AuditGeneralEntity.

For further details about properties of the audit logs can be found here: https://docs.microsoft.com/en-us/microsoft-365/compliance/detailed-properties-in-the-office-365-audit-log

Line 14 – Logs out an individual log entry, this is in a json format. Different schema’s can be found here: https://docs.microsoft.com/en-gb/office/office-365-management-api/office-365-management-activity-api-schema

Line 17 – If there are any additional pages, then this will return a value, and the loop will loop until no more pages are found.

How to acquire token for application?

In the previous section, I called a method AcquireTokenForApplication. This is a helper class and method that I use quite often, when I need to obtain an AccessToken. You can find this in the repo at .\O365AuditWebHook\AuditWebHook\Utilities\AuthenticationHelper.cs. This solution has a cut down version of the helper class I use. It is cut down as it just gets an access token for Audit Logs using AppId and Secret.

internal static async Task<string> AcquireTokenForApplication()
{
var tenant = System.Environment.GetEnvironmentVariable("Tenant", EnvironmentVariableTarget.Process);
var clientId = System.Environment.GetEnvironmentVariable("ClientId", EnvironmentVariableTarget.Process);
var secret = System.Environment.GetEnvironmentVariable("Secret", EnvironmentVariableTarget.Process);
var authorityUri = $"https://login.microsoftonline.com/{tenant}.onmicrosoft.com";
var resourceUri = "https://manage.office.com&quot;;
var microsoftToken = await GetTokenRetry(resourceUri, authorityUri, clientId, secret);
return microsoftToken;
}
private static async Task<string> GetTokenRetry(string resourceUri, string authorityUri, string clientId, string secret, int retryCount = 5, int delay = 500)
{
...
AuthenticationContext authContext = new AuthenticationContext(authorityUri, false);
ClientCredential clientCred = new ClientCredential(clientId, secret);
var authenticationResult = await authContext.AcquireTokenAsync(resourceUri, clientCred);
token = authenticationResult.AccessToken;
return token;
...
}
view raw AcquireTokenForApplication.cs hosted with ❤ by GitHub

Above is a snippet, as you can see it is wrapped in a retry method in case there is throttling.

PowerShell to stop the Audit logs

Within the PowerShell folder I have also included a file called Remove-AuditLogs.ps1

.\Remove-AuditLogs.ps1 -ClientId:<ClientID>
-ClientSecret:<AppSecret>
-TenantDomain:<Tenant>.onmicrosoft.com
-TenantGUID:<Directory ID>
-WebHookUrl:https://<Environment>-auditwebhook.azurewebsites.net/API/AuditWebHook
-ContentType:Audit.SharePoint
view raw Call-RemoveAuditLog.ps1 hosted with ❤ by GitHub

Run on one line.

This works exactly like the Set-AuditLogs.ps1 file except it calls the /stop endpoint:

https://manage.office.com/api/v1.0/{tenant_id}/activity/feed/stop?contentType={ContentType}

Once the subscription is stopped, no notifications will be sent to your webhook, and you will not be able to retrieve available content. Please note, if you decide to start the subscription again later using the Set-AuditLogs.ps1 you will not receive any content that was available between the stop and start time of the subscription.

 

This is quite a heavy post; I hope it has helped you in some way. It is just a starter, as you will probably want to do something with the logs instead of just writing them out to the Azure Logs. Maybe capturing a given process to then implement some logic to react. You might also want to put different Audit content types ContentUri onto different Azure Storage queue, so that different Azure Functions can process the ContentUri.

Setting up Webhook for O365 Audit logs

Posted on by

This is part one of a 2-part blog post.

  1. Walkthrough Setting up WebHook for O365 Audit Logs – (This Post)
  2. Dive into the code for O365 Audit Log webhooks to see how it works

In this blog post I’m going to show you how to get the O365 Audit logs using WebHooks. The full code can be found at my Github repo https://github.com/pmatthews05/O365AuditWebHook. My post will show you how to set up with screenshots and the expected results. In my next blog post I will dive into the important parts of the code to get this Audit WebHook connected and working.

Set up – Walkthrough

Creating an App Only Token

Once you have downloaded a copy from my Repo you will need to set up your environment. First thing we are going to do is create an App Only Token that will be able to read the Audit Logs.

  • For your Office 365 Tenant go to https://portal.azure.com
  • Select Active Directory
  • Select App Registrations
  • Click Create New Registration
    • Name: Audit Logs Retrieval
    • Supported Account types:
      Accounts in this organizational directory only
    • Click Register
  • Take a copy of the Application (client) ID
  • Take a copy of the Directory (tenant) ID
  • Click View API Permissions
  • Click Add a Permission
  • Select Office 365 Management APIs -> Application Permissions -> ActivityFeed.Read
  • Click Add permissions

  • Click Grant Admin Consent for [tenant] and accept the permissions.
  • Click on Certificates & Secrets
  • Click New Client Secret
    • Description: Audit Web Hook
    • Expires: Never
  • Take a copy of the Secret value

Setting up Azure

You will need to set up your Azure Environment, this will consist of the following:

  • Resource Group
  • Azure Function V1
  • Applications Insights
  • Storage Account

I like to automate where I can, also it saves me creating loads of screenshots which are probably all out of date after 2 months. I have written an Az CLI PowerShell script that will create the above for you in your Azure Environment. In the next blog post I will explain the code.

  • Download the latest version of Az Cli.
  • Using a PowerShell window – Sign into your Azure Environment using ‘az login’
  • If you have multiple subscriptions, ensure you are pointing to the correct subscription ‘az account set –subscription [SubscriptionName]‘
  • Change the directory to .\O365AuditWebhook\powershell
  • Run the following: ‘.\Install-AzureEnvironment.ps1 -Environment “[Environment]” -Name:”AuditWebHook”‘ replacing the [Environment] with your tenant name. For example, I’ve used cfcodedev.
  • Once the script has run, you will have the basic template Azure resources you need within the Resource group named [Environment]-AuditWebHook

Deploying Azure Function from Visual Studio 2019

Firstly, you don’t have to deploy this way. If you prefer to use Visual Studio code, create an AZ install script or manually deploy using Kudu, that is your choice, and all are valid. My choice of doing this is simplicity for screen shots and steps.

  • Open the solution using Visual Studio Code 2019
  • Right click on the project AuditWebHook and select Publish
  • From the Pick a publish target dialog (click Start if you are not seeing a dialog), and under Azure Functions Consumption Plan click Select Existing, and select Create Profile.
  • Sign into your account if you need to, then pick your subscription, resource group, and then you can either search, or just pick the Azure Function. Click OK.
  • This takes you back to the Summary page. Under Actions click Edit Azure App Service settings
  • The Application Settings dialog will show you the values Local and what is found within Azure Function in the cloud. You will need to update the Remote value for the following:
    • FUNCTIONS_EXTENSION_VERSION: ~1
  • You will need to add the following Settings, by clicking on Add Setting creating the setting name, and put the value in afterwards. Repeat for each setting below.
    • Tenant: [Name of your Tenant, do not include .onmicrosoft.com]
    • ClientId: [Client ID created in step ‘Creating an App Only Token’ earlier]
    • AppSecret: [Secret Value created in step ‘Creating an App Only Token’ earlier]
  • Click OK
  • Back on the Publish screen, click the Publish button. This will push the code to your environment, with the correct Application Settings.
  • By going to your Azure Function at portal.azure.com, you will see 2 Azure Functions
  • Then clicking on Configuration, it will take you to the Application settings page, click Show Values and you will see your values.

At this point you just have the Azure function as a Webhook in place. Next steps are to tie the O365 Audit log to the WebHook.

Connecting O365 Audit Logs to your webhook

The last step is tying the Audit logs to your webhook. The webhook can be used for the different Audit logs. There are 5 different types of logs.

  1. Audit.AzureActiveDirectory
  2. Audit.Exchange
  3. Audit.SharePoint
  4. Audit.General
  5. DLP.All -Note: DLP sensitive data is only available in the activity feed API to users that have been granted “Read DLP Sensitive Data” permission.

I have written a PowerShell script for you that will register the webhook for you. You will find this in the repo.

  • Open PowerShell
  • Change the directory to .\O365AuditWebhook\powershell
  • Run the following PowerShell script (Run on one line), change the parameters to match your environment. I’ve picked Audit.SharePoint, but you can use any listed above, and run the PowerShell script multiple times to connect all logs to the webhook.

.\Set-AuditLogs.ps1 -ClientId:<ClientID>
-ClientSecret:<AppSecret>
-TenantDomain:<Tenant>.onmicrosoft.com
-TenantGUID:<Directory ID>
-WebHookUrl:https://<Environment>-auditwebhook.azurewebsites.net/API/AuditWebHook
-ContentType:Audit.SharePoint
view raw Call-SetAuditLogs.ps1 hosted with ❤ by GitHub

The above codes login with the ClientID and Secret and Starts a subscription to the given ContentType audit, using the WebHookUrl for the webhook.

If successful, you will receive a 200 Status Code message like below.

Your Azure Function (AuditWebHook) would have fired, and you would see something like the following within your logs.

Viewing the results

Directly from the Microsoft Page on Office 365 management api it states in this note:

“When a subscription is created, it can take up to 12 hours for the first content blobs to become available for that subscription. The content blobs are created by collecting and aggregating actions and events across multiple servers and datacenters. As a result of this distributed process, the actions and events contained in the content blobs will not necessarily appear in the order in which they occurred. One content blob can contain actions and events that occurred prior to the actions and events contained in an earlier content blob. We are working to decrease the latency between the occurrence of actions and events and their availability within a content blob, but we can’t guarantee that they appear sequentially.“

If you are using a Development environment – like myself – and setup the Audit.SharePoint content type then I suggest you go into SharePoint, and start using SharePoint. Just so the logs start to fill.

Please note, it can take up to 30 minutes or up to 24 hours after an event occurs for the corresponding audit log entry to be displayed in the search results, depending on the service of Office 365. See the table at the bottom of this section Search the audit log in security and compliance – Before you begin

Viewing the AuditWebHook azure function, you will see that it has fired more times since your initial setup.

If you look at your latest call, (note: logs can display out of order in azure functions) you will see that it attempts to find the validation code, which is what it needs to set up the webhook. When it is unable to find the validation code, the code assumes that content contains log information. It grabs the URI of the log that has been created and then it adds it to our Azure Storage Queue for our other azure function to process. Depending on how busy your environment is, this request could hold multiple URL’s to logs. A webhook has to respond quickly back to the calling code with a 200 status code. Therefore we are adding the URI’s of the logs directly to a Storage Queue to allow a different process to interrogate the logs.

The second Azure Function (AuditContentUri) will fire every time an item lands on the Storage Queue. This will grab the information from within the log file by calling the URI.

If we select one of the calls and view the logs of that Azure Function call, every entry within that Audit log file URI will be displayed in a JSON format. Clicking on a row in the logs, will display the full details of the line. At this point in the code, would be where you process the line and do whatever you need to do with the Audit log. I’m just printing it out to the Azure Function Logs.

Remove O365 Audit Logs from your webhook

To remove the webhook from the Audit log just run the following PowerShell script. You will find this in the repo.

  • Open PowerShell
  • Change the directory to .\O365AuditWebhook\powershell
  • Run the following PowerShell script (Run on one line), change the parameters to match your environment. I’ve picked Audit.SharePoint, but you can use any listed above, and run the PowerShell script multiple times to remove all logs to the webhook.

The below codes login with the ClientID and Secret and stops the subscription of the given ContentType audit.

.\Remove-AuditLogs.ps1 -ClientId:<ClientID>
-ClientSecret:<AppSecret>
-TenantDomain:<Tenant>.onmicrosoft.com
-TenantGUID:<Directory ID>
-WebHookUrl:https://<Environment>-auditwebhook.azurewebsites.net/API/AuditWebHook
-ContentType:Audit.SharePoint
view raw Call-RemoveAuditLog.ps1 hosted with ❤ by GitHub

Hopefully, if you have followed this correctly, (and I have written decent enough instructions for you), you should have a basic Audit Log Webhook working in your environment. This isn’t anywhere near production ready code, but it gives you an idea where to start. In my next blog post I will be going though parts of the code, to explain how it all fits together.

Configuring the Email settings on O365 Groups

Posted on by

All modern SharePoint Team sites have an O365 group associated with them. The O365 groups have an email address assigned to them. This is usually in the format of Alias@[tenantName].onmicrosoft.com.

As an owner of a group, you can configure the settings to allow external people to email the group, and you can ensure all emails that are received by the group are sent to all members own email inbox.

The examples below are using the browser version of outlook, this can also be done in the outlook client. (Please note buttons and dialog are in different places and look different too)

Configuring Group email settings as an Owner

  • As an Owner of a Group open browser version of Outlook
  • Under Groups find your Group name
  • Click on the ellipse and click Settings.
  • In Group Settings click Edit group

  • Within the Edit group, there are two check boxes.
    • Let people outside the organisation email the group – Until this is ticked, only people within the organisation domain will be able to send an email to the group.
    • Send all group conversations and events to members’ inbox. They can stop following this group later if they want to. – When this is ticked, all emails sent to the group and events, will be automatically sent to all members own inbox. Note: Members can override this setting for themselves, see below “Configuring Group Email settings as a Member”

    The Email address for the group can be found on the above dialog, under Group email address. Typically the email address is <groupAlias>@<tenant>.onmicrosoft.com. See below on “Global Administrators steps to change the Group Email address to the current domain email“.

Configuring Group Email settings as a Member

As a member of a group you cannot change the settings for everyone in the group, nor can you turn on Let people outside the organisation email the group. You can, however, change what emails come to you.

  • As a member of the group open browser version of Outlook.
  • Under Groups find your Group name
  • Click on the ellipse and click Settings.
  • In Group Settings click Edit group

  • You have the option of 4 settings, which are quite self-explanatory:
    • Follow in inbox
      • Receive all emails and events
    • Stop following in inbox
      • Receive only replies to you and the group events (default)
      • Receive only replies to you
      • Don’t receive any group messages

Global Administrators steps to change the Group Email address to the current domain email

To make this change you need the following pre-requisites:

  • Global Adminstrator / Exchange Administrator
  • Microsoft Exchange Online PowerShell Module downloaded and installed
  • The email Domain you are changing to is an accepted domain for your organisation. 

In the below example, it shows how to change the Group (alias: EmailEnabling) email address from EmailEnabling@abcd.onmicrosoft.com to EmailEnabling@abcd.co.uk address.

Note: The below script is untested by myself, as I don’t have access to a environment that has a accepted domain.


#Connect
Connect-EXOPSSession -UserPrincipalName [your UPN];

#Add the Email Address
Set-UnifiedGroup -Identity "EmailEnabling" -EmailAddresses @{Add="EmailEnabling@abcd.co.uk"}

#Promote alias as a primary SMTP address
Set-UnifiedGroup -Identity "EmailEnabling" -PrimarySmtpAddress "EmailEnabling@abcd.co.uk"

#Optional, remove original email address
Set-UnifiedGroup -Identity "EmailEnabling" -EmailAddresses @{Remove="EmailEnabling@abcd.onmicrosoft.com"}

Fixing a Document Content Type that I could not change in SharePoint

Posted on by

I have come across a problem today, that initially had me stumped. A word document had a custom content type assigned to it, but it was the wrong one. The user was unable to change the content type. First, I thought it might be permissions, but I also couldn’t change the content type. The version number continued to go up, which indicates something was being saved, but the content type just wasn’t changing.

Steps to attempt to change the Content Type

  • On the library select the document you wish to change the content type for.
  • Go to the Information Panel and scroll down to Properties.

  • I first tried to change the Content type directly in the Information Panel, but it just flicked back. Next, I tried to click Edit all.

  • I clicked on the Content Type, and from the drop down I change the Content Type to Document.

  • The screen shot above, shows the document after 2 attempts of changing the Content Type. Notice how the version number has changed, but the Content Type still stuck.

How to fix

  • Open the document in the desktop version.

  • In the client application click File.
  • Note: You won’t be able to do the following if Protect Document is enabled, and you don’t have the password.
  • In the Info section, where is says Inspect Document click the Check for Issues button, then Inspect Document

  • On the Document Inspector dialog, Click Inspect

  • Once Inspected, click Remove All button under the Document Properties and Personal Information section.

  • Close the Document Inspector dialog.
  • Save the file (If Autosave isn’t on)
  • Close the Client Application
  • The SharePoint list will show that the file is now of Content Type “Document” (Or whatever is the first/default Content Type in your library) and the version number has gone up once more.

If you need to, you should be able to change the content type without any issues.

I’m not 100% sure, but I believe this is happening because the content type that has been saved within the document is corrupted/different from the same name content type with the one in the library. I believe this has happened in the user’s environment, where the document was originally in a library with an older version of the content type in a different site collection. Then moved to a newer library. The content type exists in the newer library (as we provision all our sites with PNP), but it has changed slightly, for example a column has the same name but different ID.

When you clear the content type from the document, when it is saved back to the SharePoint library, it grabs the information from the library and puts the new content type information back into the document. Going forward, there will be no more corruption or conflict. Although it might be possible to have the issue again if you move the document back to the other library in the other site collection with the older version of a content type.

Top 10 blog post for 2019 from Cann0nf0dder.wordpress.com

Posted on by

In the last year, I let myself down a little by not blogging as much as I wanted to. I’m hoping this year I will blog a bit more often. The issue I sometimes find is there are so many decent blogs out there, writing something that is already out there feels like I’m just copying someone else’s idea. I also don’t write a blog about really simple things, that took me a minute to work out. If it took me a moment, is it really worth blogging about? Then there is the time to write the blog, I like to provide code, screenshots and test my walkthroughs to ensure they work and make sense. (I still have one blog that I haven’t finished yet, I started in August!)

After saying in the last paragraph I don’t want to copy someone else’s idea, the idea of posting my Top 10 blog posts for 2019, I have nicked from Nate Chamberlain. https://natechamberlain.com/2020/01/01/top-10-posts-of-2019-on-natechamberlain-com. I don’t know Nate personally, but I hope he doesn’t mind I’m copying his blog post idea.

Anyway, thank you for reading my blog and here are a list of the most popular blog posts in 2019. (I’m surprised from the list that people still go to them, as they are quite old now, dating back to 2013! Only one of them was written in 2019.)

Top 10 blog post based on views for 2019

TitleNumber of Views
TypeScript error in Visual Studio – Cannot find module, problem with the tsconfig.json file6,125
SharePoint Designer 2013 Workflows and GET REST API5,002
Building SharePoint 2016 Development environment – Part 8 – Installing SQL 2016 ready for SharePoint 20164,505
Getting sound to work within your Windows Server Hyper V client4,081
Upgrading Windows 10 – Resolution stuck at 1024 x 7683,839
Unable to download the Exchange Online PowerShell Module – ‘Deployment and application do not have matching security zones3,294
Using REST to upload a File to Document library (Including different site collection)3,167
Building SharePoint 2016 development environment – Part 10 – Configuring Central Administration for SSL2,959
SharePoint Designer 2013 Workflows and POST REST API2,937
Accessing Taxonomy Term Store with JSOM2,883
Cann0f0dder’s top 10 blog posts of 2019

As the above list one has one blog post I wrote in 2019, below is a list of the top ten blog post that were written in 2019.

The Top 10 blog post based on views that were written in 2019

TitleNumber of Views
Unable to download the Exchange Online PowerShell Module – “Deployment and application do not have matching security zones”3,294
Programmatically change the New Menu in SharePoint Online using PowerShell884
SharePoint Online Custom Format View issue with @now and UK Date format808
Setting up an O365 Dev Tenant – Part 1 – Getting the Tenant795
Access denied when attempting to move SharePoint documents395
Visual Studio – NuGet – No connection could be made because the target machine actively refused it 127.0.0.1:8888263
Setting up an O365 Dev Tenant – Part 6 – Set up SharePoint Tenant188
Finding the related Site from Teams Private Channel Site181
Setting up an O365 Dev Tenant – Part 2 – Create users from CSV file168
Setting up an O365 Dev Tenant – Part 5 – Turning on O365 Auditing104
Cann0f0dder’s top 10 blog posts of 2019 written in 2019

Finding the related Site from Teams Private Channel Site

Posted on by

Private Channels gives the ability to restrict the membership further within a Team site. A person can create a private channel, like creating a public channel, except they can add owners/members to the channel from a subset of members from the Team site.

When a private channel is created, what is happening under the covers is a creation of another SharePoint site. A cut down version of a SharePoint site, using the Template TEAMCHANNEL#0. (ID: 69 for those that want to know)

As this is my first blog post about Private Channels, let me demonstrate quickly how to create a Private Channel.

How to add a private channel

  • From MS Teams click on the ellipse next to your Team name, and select Add Channel.
  • Give the channel a name, optional description, and select “Private – Only accessible to a specific group of people within the team”
  • Click Next. On the next page you can add people from the Team to have access to the Private Channel. They can be an Owner of the channel even if they are only a member within the Team.
  • The private channel will show up as a channel underneath your team, with a pad lock next to it, indicating that it is a private site. You will only see this channel if you are a owner/member of the channel.
  • The SharePoint site – which you can get to by clicking on files Open in SharePoint – has the URL made up of https://<tenant>.sharepoint.com/sites/<TeamName>-<ChannelName> and the home page of the site is the root of the Shared Document library.

Finding the related site

There are a couple of places I have found out where to get the related site.

Property Bag and Graph API

When I did a PNP Get-ProvisioningTemplate pointing at a private channel site, I discovered in the property bag there is a value called RelatedGroupId and it is Indexed.

  <pnp:PropertyBagEntry Key=“RelatedGroupId” Value=“d99aa865-cd55-46cc-b256-177975ad3e13” Overwrite=“false” Indexed=“true” />

With this value you can then get the SharePoint site of the MS Team using Graph API

https://graph.microsoft.com/v1.0/groups/<group-id>/sites/root?$select=webUrl

or

https://graph.microsoft.com/v1.0/groups/<group-id>/sites/root/weburl

Note: On the parent Team site, there is a property bag value called GroupId. It also has RelatedGroupId, which has the same value.

CSOM using the Site object

The related GroupID can also be obtained in CSOM via the Site object.

Site site = context.Site;
context.Load(site, s => s.RelatedGroupId);
context.ExecuteQueryRetry();

Showing all the private channels from the main SharePoint site

When I used PNP to obtain the Private Channel template and discovered the RelatedGroupId was in the property bag and that it was indexed, means that it is searchable. If you check the Manage Search Schema, you will find the managed property.

This means doing a simple search like below, will return all the private channel sites.

RelatedGroupId:<GroupID> contentclass:STS_Site

Using the Microsoft Search PnP Modern Search SPFX (https://github.com/microsoft-search/pnp-modern-search/releases/tag/3.8.0), very quickly I was able to display links.

For someone who only has access to one of the Private Channels, they will only see one in search.

Updating an expired Client Secret of a SharePoint Add-in using Azure-AD/Az Cli

Posted on by

Back in May 2016, I wrote a post to show you how to update the Client Secret of a SharePoint add-in. This used the PowerShell module MSOL. https://cann0nf0dder.wordpress.com/2016/05/18/updating-an-expired-client-secret-of-sharepoint-add-in/

The MSOL module is now (or going to be) deprecated. Therefore, I needed to find a different way of doing this, and ideally something that could be done with Azure Dev-ops pipelines. I originally started with AZ CLI. Although the Client Secret is tied to a Service Principal in Azure AD, I was unable to change the Key Credentials or Password Credentials for it.

Due to the issues I was getting with AZ CLI, I used Azure-AD module instead. See at end of blog post, how I got Az CLI to work with a workaround.

Updating Client Secret

Connect to Azure-AD

First you will need to connect to Azure-AD

#Install AzureAD
Write-Information -MessageData:"Getting if the AzureAD powershell module is available..."
if(-not (Get-Module AzureAD)) {
Write-Information -MessageData:"Installing the NuGet Package provider..."
Install-PackageProvider -Name:NuGet -Force -Scope:CurrentUser
Write-Information -MessageData:"Installing the AzureAD Powershell Module..."
Install-Module AzureAD -Scope:CurrentUser -Force
}
$Credential = Get-Credential
Connect-AzureAD -Credential $Credential
view raw Connect-AzureAD.ps1 hosted with ❤ by GitHub

The above code ensures that you have Azure AD installed on your machine, and logs you in.

Getting the Add-in as a Service Principal

Once you have logged in, you will be able to call back your SharePoint Add-in. The SharePoint Add-in is actually a Service Principal within Azure AD, and we will grab this using Get-AzureADServicePrincipal. You can do this by using the AppId, or the AppName. The AppId is fine to use, but when you want to use the same script across multiple environments, you will need to ensure you are passing in the different AppId for each environment. THis is why I have used the Name of the Add-In. (Assuming you have given your App the same name in each environment)

$serviceprincipal = Get-AzureADServicePrincipal -All:$true -Filter "DisplayName eq 'Demo App'"
#OR If using APP ID.
$serviceprincipalByID = Get-AzureADServicePrincipal -All:$true -Filter "AppId eq 'ab739749-827d-4437-90e5-bf181c5407e0'"
view raw GettingServicePrincipal.ps1 hosted with ❤ by GitHub

Create a new Secret

Next you need to be able to create a new secret. This is done by creating random bytes and converting to a Base64String. I ensure the password is valid for an additional 2 years.

$bytes = New-Object Byte[] 32
$rand = [System.Security.Cryptography.RandomNumberGenerator]::Create()
$rand.GetBytes($bytes)
$rand.Dispose()
$newClientSecret = [System.Convert]::ToBase64String($bytes)
$dtStart = [System.DateTime]::Now
$dtEnd = $dtStart.AddYears(2)
write-output $newClientSecret
view raw CreatingPassword.ps1 hosted with ❤ by GitHub

Create Key Credentials and Password Credential for the App

The SharePoint Add-In requires 2 Key Credentials (One Sign and one Verify) and 1 Password Credentials. The following script creates new ones, this allows both the old password and the new password to continue working at the same time, until you are able to update the code that uses the ClientID and ClientSecret.

Write-Information "Updating KeyCredential Usage Sign..."
New-AzureADServicePrincipalKeyCredential -ObjectId $serviceprincipal.ObjectId -Type:Symmetric -Usage:Sign -Value $newClientSecret -StartDate $dtStart -EndDate $dtEnd | Out-Null
Write-Information "Updating KeyCredential Usage Verify..."
New-AzureADServicePrincipalKeyCredential -ObjectId $serviceprincipal.ObjectId -Type:Symmetric -Usage:Verify -Value $newClientSecret -StartDate $dtStart -EndDate $dtEnd | Out-Null
Write-Information "Updating PasswordCredential..."
New-AzureADServicePrincipalPasswordCredential -ObjectId $serviceprincipal.ObjectId -Value $newClientSecret -StartDate $dtStart -EndDate $dtEnd | Out-Null
view raw NewCredentials.ps1 hosted with ❤ by GitHub

Removing the original Key and Password Credential for the App

The following code shows how to loop round the Key and Password credentials and remove the original ones. It does this by looking for any Credentials that were created before the start date of the new ones. I would only run this part of my code, once I know I have updated my application to use the new password. Not in my example here, but where I’m using it in the real world, my Client Secret is stored within a Keyvault. I would update the keyvault value (ensuring to disable the previous version).

Write-Information "Remove all KeyCredential started before $(Get-Date $dtStart -Format 'O' )..."
$serviceprincipal = Get-AzureADServicePrincipal -All:$true -Filter "DisplayName eq '$SharePointAddInName'"
$serviceprincipal.KeyCredentials | ForEach-Object{
$credential = $PSItem
if($($credential.StartDate) -lt $dtStart)
{
Write-Information -MessageData:"Removing KeyCredential $($credential.KeyId)"
Remove-AzureADServicePrincipalKeyCredential -ObjectId:$serviceprincipal.ObjectId -KeyId:$credential.KeyId
}
}
Write-Information "Remove all PasswordCredential started before $(Get-Date $dtStart -Format 'O' )..."
$serviceprincipal.PasswordCredentials | ForEach-Object{
$credential = $PSItem
if($($credential.StartDate) -lt $dtStart)
{
Write-Information -MessageData:"Removing PasswordCredential $($credential.KeyId)"
Remove-AzureADServicePrincipalPasswordCredential -ObjectId:$serviceprincipal.ObjectId -KeyId:$credential.KeyId
}
}
view raw RemoveOutOfDatePasswords.ps1 hosted with ❤ by GitHub

Connecting with AZ Cli Workaround

Using Azure Dev-Ops Pipeline, I really wanted to use AZ cli to be able to update the Client Secret of a SharePoint Add-in. Due to the error messages when I attempted to update the Service Principal Key Credential and Password Credentials, I was forced to use Azure-AD instead. So how can I uses AZ Cli.

My Dev-Ops Pipeline uses a service account, and I have ensured this service account has permissions to update the directory.

Then I can connect from Az Cli to Azure-AD doing the following:

#Once signed into Azure CLI
$Token = az account get-access-token --resource-type "aad-graph" | ConvertFrom-Json
$AzAccount = az account show | ConvertFrom-Json
Connect-AzureAD -AadAccessToken $($Token.accessToken) -AccountId:$($AzAccount.User.Name) -TenantId:$($AZAccount.tenantId)
view raw AzCliToAzureAD.ps1 hosted with ❤ by GitHub

I add the above code in the full script just after the parameter, as the pipeline will already be signed in as the Pipeline Service Principal. It will then grab the Access token to sign in with Azure AD, and then able to run the rest of the script.

Full Script

<#
.SYNOPSIS
Updates the SharePoint Add-in Secret everytime.
It expects that you are already connected to Azure AD
.EXAMPLE
.\Update-SharePointAddIn.ps1 -SharePointAddInName "Demo App"
#>
param(
[Parameter(Manadatory)]
[string]
$SharePointAddInName
)
$ErrorActionPreference = 'Stop'
$InformationPreference = 'Continue'
#Call AzCliToAzureAD.ps1 here for Pipeline.
#Create Pasword
$bytes = New-Object Byte[] 32
$rand = [System.Security.Cryptography.RandomNumberGenerator]::Create()
$rand.GetBytes($bytes)
$rand.Dispose()
$newClientSecret = [System.Convert]::ToBase64String($bytes)
$dtStart = [System.DateTime]::Now
$dtEnd = $dtStart.AddYears(2)
Write-Information "Getting service principal named: $SharePointAddInName..."
$serviceprincipal = Get-AzureADServicePrincipal -All:$true -Filter "DisplayName eq '$SharePointAddInName'"
if($null -eq $serviceprincipal)
{
Write-Error "Unable to find service principal named: $SharePointAddInName"
}
Write-Information "Updating KeyCredential Usage Sign..."
New-AzureADServicePrincipalKeyCredential -ObjectId $serviceprincipal.ObjectId -Type:Symmetric -Usage:Sign -Value $newClientSecret -StartDate $dtStart -EndDate $dtEnd | Out-Null
Write-Information "Updating KeyCredential Usage Verify..."
New-AzureADServicePrincipalKeyCredential -ObjectId $serviceprincipal.ObjectId -Type:Symmetric -Usage:Verify -Value $newClientSecret -StartDate $dtStart -EndDate $dtEnd | Out-Null
Write-Information "Updating PasswordCredential..."
New-AzureADServicePrincipalPasswordCredential -ObjectId $serviceprincipal.ObjectId -Value $newClientSecret -StartDate $dtStart -EndDate $dtEnd | Out-Null
#Update the application here.
#For example add the secret to a key vault that the application is getting the secret from.
Write-Information "Remove all KeyCredential started before $(Get-Date $dtStart -Format 'O' )..."
$serviceprincipal = Get-AzureADServicePrincipal -All:$true -Filter "DisplayName eq '$SharePointAddInName'"
$serviceprincipal.KeyCredentials | ForEach-Object{
$credential = $PSItem
if($($credential.StartDate) -lt $dtStart)
{
Write-Information -MessageData:"Removing KeyCredential $($credential.KeyId)"
Remove-AzureADServicePrincipalKeyCredential -ObjectId:$serviceprincipal.ObjectId -KeyId:$credential.KeyId
}
}
Write-Information "Remove all PasswordCredential started before $(Get-Date $dtStart -Format 'O' )..."
$serviceprincipal.PasswordCredentials | ForEach-Object{
$credential = $PSItem
if($($credential.StartDate) -lt $dtStart)
{
Write-Information -MessageData:"Removing PasswordCredential $($credential.KeyId)"
Remove-AzureADServicePrincipalPasswordCredential -ObjectId:$serviceprincipal.ObjectId -KeyId:$credential.KeyId
}
}
view raw Update-SharePointAddIn.ps1 hosted with ❤ by GitHub