Setting MS Teams Policies for Users using PowerShell


In my past two blogs I have shown you how to obtain all the users policies and output to a csv file, and how to create a new policy. In this blog post, I’m going to show you a couple of ways of setting users to new policies.

  • Change policy for an individual user
  • Change policy for a group of users
  • Change policies for group of users using a csv file

Change policy for an individual user

In this example I’m going to stick to just the messaging policy.

.\Set-IndivdualUserMessagingPolicy.ps1 -UserNameToSetPolicy:”Jeff.Hay@mytenant.onmicrosoft.com” -PolicyName:”NoGiphyOrStickersMessagePolicy”

It can take a while before the change is reflected in the Teams Administration.

Change policy for a group of users

There are a couple of ways you can do this. If you have your people data filled in correctly, such as Department, Office, City etc, you could assign all the people from one of these areas to a given policy. For example the following script grabs everyone from the sales department and assign them the sales policy for messaging:

Alternatively you could do it based on a AD Group, you first need to connect to grab the members of the given group, I would connect with AZ cli.

Change policies for a group of users using a csv file

In a previous blog post, I showed you how to obtain all the valid users from the tenant with their policies. We are going to use the csv it produces to change users policies.

The screen shot below shows my users with licenses, originally when I ran my script I only had DisplayName, UserPrincipalName and SipAddress showing, everything else was blank because my users were all in the global policies.

I have now filled in the CSV file with either SalesPolicy, HRPolicy, or NoGiphyOrStickersMessagePolicy. Left my account as global. These policy have already been created in my environment, help with doing that can be found in this blog post.

The following script requires your MS Teams Administrator username, and the path to the csv file. It loops through each item and then sets the polices for each user.

.\set-UserTeamPolicies.ps1 -UserName:admin@mytenant.onmicrosoft.com -Path:.\teamsuserpolicies.csv

If you have a lot of users to update, the session might timeout. On line 52 – 57 there is a check to see if the session has timed out and then gets you to reauthenticate. If anyone else knows a better way to do this, please add a comment below, or get in touch.

Creating a new MS Teams Policy using PowerShell


In my previous blog I showed you how to obtain all the policies for users. This blog post, I’m going to show an example of how to create a new Policy for Meeting.

The script I’m providing below is a simple script that will:

  • Name the policy – Line 1
  • Import the SkypeOnlineConnector module – Line 4-7
  • Create the policy if it doesn’t exist – Lines 9 – 12
  • Set the properties of the Policy – Lines 15 – 31

This policy that is being created will prevent users of this policy creating Giphy, Memes or Stickers in their chats. (Basically, taking the fun out of teams. [insert wicked laugh])

As you can see from the above screen shot, I now have a policy called NoGiphyOrStickersMessagePolicy, and below is a screen shot of the settings within Teams Administration.

My previous blog showed you how to get 13 different policies for each user. Below are the links to those PowerShell Set policy commands with the different parameters.

Getting all MS Teams User Policies using PowerShell


With everyone working at home at the moment, you might need to grab a report of the User Policies for MS Teams. These few steps will show you how to grab all the users and display in a csv file.

To work with policies you first need to obtain the PowerShell module – SkypeOnlineConnector. Which you can download from this link https://www.microsoft.com/en-us/download/details.aspx?id=39366

Installing the SkypeOnlineConnector and creating a session

Once you have installed the PowerShell module you will need to import the module and create a PowerShell session.

You will be prompted for MS Teams administrator username and password, you can pass your credentials in with the -credential parameter at the end of New-CsOnlineSession, however this doesn’t work with MFA.

Once connected you will be able to grab all users using the Get-CsOnlineUser cmdlet, or grab one user by providing the users Identity.

Policies

If you call the above for a single user you will see that there are loads of policies that can be set. Not all are MS Teams related. The ones I will be focusing on are the same 12 policies you see when you view Assigned policies for a user in Teams Administration.

These policies are named slightly different in the results compared to the display name shown above.

Display Name Policy Name
Meeting policy TeamsMeetingPolicy
Messaging policy TeamsMessagingPoliy
Live events policy TeamsMeetingBroadcastPolicy
App permission policy TeamsAppPermissionPolicy
App setup policy TeamsAppSetupPolicy
Call park policy TeamsCallParkPolicy
Calling policy TeamsCallingPolicy
Caller ID policy CallerIdPolicy
Teams policy TeamsChannelsPolicy
Emergency calling policy TeamsEmergencyCallingPolicy
Emergency call routing policy TeamsEmergencyCallRoutingPolicy
Dial plan TenantDialPlan
Teams Upgrade TeamsUpgradePolicy

The following script will grab all users and their current policy for the above polices, with the provided path it will output to csv file.

Please note: Anything that is set to Global policy will be blank.

.\Get-UserTeamPolicies.ps1 -Path:.\AllTeamUsersPolicies.csv

In a later blog, I will be using the csv file to update user policies.

Bonus: If you want to get all users for just a single policy you can perform a filter on the Get-CsOnlineUser

Removing External Users fully from a SharePoint Tenancy using PowerShell


This blog post has all come about as the client I was working for was having problems sharing documents in SharePoint with some external users. It turned out that the user was already in Azure AD as a Contact which is part of Exchange. This meant when an internal person attempted to share/Invite into SharePoint/MSTeams it all appeared to work correctly for the external user, but sometimes it didn’t. When looking at external users through the Admin portal, this external user was showing, but their email address was blank. After speaking with Microsoft, it turns out, because the email address was already found within the tenancy, it creates a unique violation when adding the external user to the Active Directory.

I have been working with Microsoft support regarding this, and the resolution was that this is as design!!??! Only by feeding back on the Office 365 uservoice this issue “might” looked at and fixed. See resolution notes below:

Symptom:
When you invite external users who exist as contacts in your environment, their email does not get populated in their guest user ID which results in them not being able to login to your environment and access the shared data.
Cause:
The issue is coming from a conflict caused by the email address which is already populated for the mail contact.
Resolution:
This is behavior by design as all objects in Azure AD have to be unique.
You cannot have 2 objects with the same email address.

When you invite one of your contacts to your content in O365, it actually creates a completely new guest user object in your environment and since the email address which is supposed to be populated in the email attribute is already in use by the contact, the email address does not get populated.

The only way to resolve this issue at the moment is to eliminate any conflicts that are in place, by removing the conflicting email contact and re-invite the user to your content.
More information:
The best thing I can offer to you is the following:

Please go to our UserVoice portal where other people are facing the same behavior and up-vote it, comment and have the whole IT department do the same as well.

Allow a “Guest User” to be converted to a different account type
https://office365.uservoice.com/forums/273493-office-365-admin/suggestions/19966537-allow-a-guest-user-to-be-converted-to-a-differen

This led me to working on a process and script that would remove the users from everywhere.

Locations to remove the External User from:

  • Contacts
  • Azure AD Guest Users
  • Azure AD Deleted Users
  • All SharePoint Sites
  • All SharePoint Hidden User lists
  • SharePoint User Profile

Contacts

To remove the External User from the contacts you will need to use the MSOL PowerShell module.

Or you can manually do this by going to admin.microsoft.com and under Users -> Contacts select the user and click Delete contacts.

Azure AD

To remove the External User from Azure AD you will still require using the MSOL PowerShell module. In fact, this script and the above script could be merged.

To do this manually, in admin.microsoft.com under Users -> Guest Users, select the user and click delete.

Then go into Users -> Deleted users and remove them from there.

Remove from SharePoint

To remove from SharePoint, if you have a large tenancy and you don’t know all the places where the external user could have been shared with, then you will have to use the following script. This script will remove the external user from the SharePoint Site, ensure that they are removed from the User Information list, and then lastly it will clear the person from the SharePoint User Profile.

I discovered that if I didn’t remove them from the User Profile, when attempted to reshare a document with that user, the people picker would grab the internal userprincipalname (<ExternalUserEmail>#EXT#@<Tenant>.onmicrosoft.com) as the email address and then prevent me clicking the Sharing button. This is because the people picker uses Graph API /Me/People and grabs the value from there. Once removed from everywhere, including the User Profile this no longer happens.

The following script uses SPO PowerShell Module and you will need to connect first using Connect-SPOService. The account that you use, needs to be a SharePoint Global Administrator.

The script checks if it can find the ExternalUser, and if it can remove the user using Remove-SPOExternalUser.

Then it loops through every site collection and looks for the user using Get-SPOUser with the internal userprincipalname. If found it removes the user using Remove-SPOUser. Once it has looped through all SharePoint sites, it then checks the SharePoint User Profile and removes the user from UserProfile Remove-SPOUserProfile. This command will remove a user from the UserProfile if they in the “Active Profiles” or the “Profiles Missing from Import”

If the plan is to add the external person back into your tenant, once the script has run, you will need to wait at least a few hours (maybe leave it for a day to be sure) to ensure all back end processes of Microsoft have completed.

When you share a document/folder with the external user they will get the invited link and enter a code experience, this way they do not turn up inside you Azure AD. However, if you share a site with them, or add them to a MS Teams, they will appear in your Azure AD correctly.

How does the privacy in O365 Groups/Teams affect the SharePoint Site


When you create a O365 Group or Teams you get given the option for Privacy. Either Public or Private. When an O365 group is created a Modern SharePoint Team site is created, but how is the site affected depending on the privacy?

Private Privacy

In a “Private” group, in the SharePoint site, the members of the O365 Group get added to the SharePoint Members Group.

And the owners of the O365 Group get added to the Site Collection Administrators.

Public Privacy

In a public site, the Members and Owners of the group get added to the SharePoint Members group and Site Collection Administrators respectively. The extra difference I have noticed, is that the “Everyone except external users” group is also added to the SharePoint members group.

What does this mean?

A private site is what it says. It is private. If it’s joined to a Teams anyone who is not a member of the group, cannot see MS Teams, search for site or documents inside it, or have access to the SharePoint site.

A public site is different. If it’s joined to Teams, a person who isn’t a member to the group will not see the MS Teams, meaning they cannot see channel chats, but can search to join it.

The documents however are available to them. They can search for these documents, access the SharePoint Site contribute on the documents too if they like.

Testing this

I have created two MS Teams. The first one is called Public OTB Permissions and set the privacy to Public. I am the only member of this site. The second site is called Private OTB and set the privacy to Private. Again, I am the only member of this site. In both “General” channels I have uploaded a Word document.

Public Privacy

Private Privacy


I have a second user called Alan Brewer, and in SharePoint I’m going to search for the word “Place” as this word is in both test documents.

Alan can only see the document from the public site, and when clicked through can access the document, contribute towards it. He can even view the whole SharePoint site. This is because he is being let into the site via the “Everyone except external users” claim that has been added to the SharePoint Members group for the site.

Just to complete this, I have added another user called Adam Barr to the Private site.

If I now sign in as Adam and perform the search “Place” like before, Adam can see both documents.

What happens when I switch the privacy for the O365 group.

When you decided you want to make a “Private site” public or a “Public site” Private, there is a process that happens in the back end which will either add/remove the “Everyone Except External users” to the SharePoint members group. I’ve noticed this can take up to 15 minutes or so before it is reflected in the SharePoint site.

Should I remove/change the “Everyone Except External users” from a public SharePoint site?

You can, but I don’t recommend it. If like me, personally it might be better to have the Everyone Except External Users in the Visitors group. Allowing the site to still be public, but you must be a member to contribute or change anything. The trouble with this, the site would be set up they way you want it, but then if an Owner decides to change the O365 Groups Privacy to Private, the backend process does not remove the “Everyone Except External Users” from the SharePoint Visitors Group, leaving the documents open to all. Then if they decided to switch it back or change an O365 Group from Private to Public Privacy, the backend process would add the “Everyone Except External Users” to the SharePoint members group. Making the site now open and everyone can contribute again. You could train all owners to ensure they remove/add this group if they switch the privacy, but who will remember to do that!

You could, if you are a developer, have a webhook using MS Graph that is attached to monitor O365 Group changes. https://docs.microsoft.com/en-us/graph/webhooks. Although possible, you would need to ensure that there is a delay from capturing the O365 Group webhook privacy change, and updating SharePoint, as you need to run after the backend process has been and flipped the “Everyone Except External Users”. If switching from Public to Private policy, any process you develop, if runs too late would leave the site open for a given amount of time, or if it fails could leave the site open where the users believe it is all locked down.

Viewing, Restoring and Removing Items from the SharePoint Recycle Bin – The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator.


I’ve had a script for a while that allows you to view all the items in the Recycle Bin for a Site Collection and prints out to a CSV file. Recently the environment I’ve been running this in has been throwing an error saying;

The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator“.

Getting all items out of the recycle bin.

Originally, I used the PNP Powershell command Get-PnPRecycleBinItem and it was only when I did a Google search for this issue, I found that other people were also having this problem. The PnP team have solved this issue now by adding -RowLimit parameter. If you set the RowLimit high enough you can return all items, as internally, it seems to implement a paging mechanism.

I now use the below script to export the result to a CSV file.

Once I have the CSV file, I’m able to filter further in excel and save back to CSV to use to either Restore / Delete the items out of the recycle bin.

Restoring Deleted Items using a csv file.

It seemed that now that I can use RowLimit with Get-PnPRecycleBinItem I should be able to call Restore-PnpRecycleBinItem to restore the item. However, this isn’t the case. Even just passing the Identity of one item within the Recycle Bin, you get the same error message.

The attempted operation is prohibited because it exceeds the list view threshold enforced by the administrator“.

There is no RowLimit option on the Restore-PnpRecycleBinItem. The code must internally make a call to get all RecycleBin Items first without using RowLimit. Interestingly though, a user could go to a recycle bin, see items, and restore them if they wanted to. By looking through the network traffic, I was able to see that the GUI uses the following API to Restore Items.

POST /_api/site/RecycleBin/RestoreByIds

Passing in the following JSON body.

There can be one or many Ids.

The trouble with using REST API you need an accessToken. Using Connect-PnPOnline using just your Username and Password, you are unable to call back the AccessToken value. The easiest way to do this is using the PNPO365ManagementShell. When called a browser window will open, paste in the code that is showing in your PowerShell window (It is copied to the clipboard already) and then sign in with your account. This will allow you to grab an accesstoken using Get-PNPAccessToken.

This is what I do in the below code. I grab an access token, then loop through every item in a CSV file to restore.

Deleting Deleted Items using a csv file.

I discovered that I also get the error message when using Clear-PnpRecycleBinItem.

Again, I was able to do this in the GUI, and looking at the network traffic there is an API to delete the items.

POST /_api/site/RecycleBin/DeleteByIds

The JSON body is same format as the RestoreByIds, where it passes in one or many Ids.

The code below is almost identical to the Restore-RecycleBinItems.ps1. Passing in a CSV file with the IDs of files to delete permanently.

Dive into the code for O365 Audit logs webhooks


This is part two of a 2-part blog post.

  1. Walkthrough Setting up WebHook for O365 Audit Logs
  2. Dive into the code for O365 Audit Log webhooks to see how it works – (This Post)

The previous blog post showed how to get you up and running with O365 Audit logs and webhooks. In this blog post I’m going to show and explain parts of the code that ties everything together.

The full code can be found at my Github repo https://github.com/pmatthews05/O365AuditWebHook

PowerShell to initialize the Webhook to the Audit logs

Run on one line.

From inside the PowerShell folder (.\O365AuditWebhook\PowerShell) there is a PowerShell file called Set-AuditLogs.ps1 This PowerShell file Starts a subscription to the given Audit Content Type. This is done by calling:

https://manage.office.com/api/v1.0/{tenant_id}/activity/feed/start?contentType={ContentType}

The above call is a POST call and uses the ClientID and Secret to authenticate against the tenant. The body is a Json object

  • authId – Optional string that will be included as the WebHook-AuthID header in notifiations sent to the webhook as a means of identifying and authorizing the source of the request to the webhook
  • expiration – Optional datetime that indicates the datatime after which notifications should no longer be sent to the webhook. By leaving it empty, indicates the subscription will be active for the next 180 days.
  • address – Required HTTPS endpoint that can receive notifications. A test message will be sent to the webhook to validate the webhook before creating the subscription.

When the /start operation is called, the webhook URL specified in the address will be sent a validation notification to validate that an active listener can accept and process notifications.

The Azure Function AuditWebhook found in the O365AuditWebhook.cs file has two parts to it.

The first part, as shown above, handles the validation. It looks for a validation code within the content, and if found it response back with a 200 status (OK) and includes the validation code.

If an OK is not received back, then the webhook will not be added and the subscription will remain unchanged.

The second part of the AuditWebhook Azure function is explained in the next section.

Webhook handling O365 notifications

After the initial validation, notifications will be sent to the webhook as the content logs become available.

From the first part of the AuditWebHook Azure Function, notifications do not have the validationCode, this allows us to determine that notifications have been sent, instead of a new subscription.

The content of these notifications contains an array of one or more JSON objects that represent the available content blobs.

On line 5 of the above code, show where I handle the content of deserialize json object (notifications) to a list of AuditContentEntity.

The notification/AuditContentEntity contains the following:

  • tenantId
    The GUID of the tenant to which the content belongs
  • clientId – The GUID of your application that created the subscription
  • contentType – Indicates the content type
  • contentId – An opaque string that uniquely identifies the content
  • contentUri – The URL to use when retrieving the content
  • contentCreated – The datetime when the content was made available
  • contentExpiration – The datetime after which the content will no longer be available for retrieval.

At this point you do not have any log information, you just have a collection of contentUri which when called will provide you with the logs. To ensure that the webhook response quickly so that it can continue to handle incoming requests, we place the contentUri, contentType, and TenantId onto an Azure Storage Queue. This allows a different Azure function to handle getting the actual logs.

Lines 9-16 will set up the storage queue if it doesn’t exist.

Lines 19-26 prepares my queue object and serialize it to a json string.

Line 28 adds the message to the Azure Storage Queue.

Once all notifications/AuditContentEntity have been processed, a 200 status (OK) is passed back. The subscription that calls our webhook is waiting for an OK response. If it encounters failure, it has a built in retry mechanism that will exponentially increase the time between retries. If the subscription continues to receive failure response, the subscription can disable the webhook and stop sending notifications. The subscription will need to be started again to re-enable the disabled webook.

Processing the Storage Queue AuditContentUri

As items are put on the Storage Queue the Azure Function AuditContentUri found in the O365AuditWebhook.cs file fires.

First you need an authorization token to read the audit logs, we do this with AcquireTokenForApplication method. This uses the Tenant Name, ClientId and Secret that is stored within your Azure configurations. See ‘How to acquire token for application?’ below.

It grabs the ContentUri and then goes into a do loop. This is because the logs that come back, if it is a very busy tenant, not all the logs will be returned, and there will be a NextPageUri value in the header of the response to allow you to obtain the next page of logs.

Line 7 – This adds your tenantID to the end of the URI as a PublisherIdentifier. This parameter is used for throttling the request rate. Make sure this parameter is specified in all issued requests to get a dedicated quota. All requests received without this parameter will share the same quota. The IF statement ensures it is added to the end of the URI correctly.

Line 9 – This calls the ContentUri and gets a results and request headers. You can see the file .\O365AuditWebHook\AuditWebHook\Utilities\RestAPI.cs
The Method GetRestDataAsync is very similar to the GetRestData call you find within PNP Core code. Creates a HttpWebRequest, passing in Authorization Token, and calling the ContentUri. Only difference in my code is that I’m grabbing the response.Headers to find out if there are additional logs, and passes them back with the results.

Line 10 – This parse the results into a JArray. (Json Array object). Here you can manipulate what comes back. For example, instead of grabbing all results and then displaying them out, you can query the results for a particular log type.

In the example code below, this would be using the Audit.General logs, and it will grab any logs that are of RecordType 25 (Indicates Microsoft Teams event) where the operation is creating a new channel, and the Channel type is Private. I then convert the JArray to an object list of AuditGeneralEntity.

For further details about properties of the audit logs can be found here: https://docs.microsoft.com/en-us/microsoft-365/compliance/detailed-properties-in-the-office-365-audit-log

Line 14 – Logs out an individual log entry, this is in a json format. Different schema’s can be found here: https://docs.microsoft.com/en-gb/office/office-365-management-api/office-365-management-activity-api-schema

Line 17 – If there are any additional pages, then this will return a value, and the loop will loop until no more pages are found.

How to acquire token for application?

In the previous section, I called a method AcquireTokenForApplication. This is a helper class and method that I use quite often, when I need to obtain an AccessToken. You can find this in the repo at .\O365AuditWebHook\AuditWebHook\Utilities\AuthenticationHelper.cs. This solution has a cut down version of the helper class I use. It is cut down as it just gets an access token for Audit Logs using AppId and Secret.

Above is a snippet, as you can see it is wrapped in a retry method in case there is throttling.

PowerShell to stop the Audit logs

Within the PowerShell folder I have also included a file called Remove-AuditLogs.ps1

Run on one line.

This works exactly like the Set-AuditLogs.ps1 file except it calls the /stop endpoint:

https://manage.office.com/api/v1.0/{tenant_id}/activity/feed/stop?contentType={ContentType}

Once the subscription is stopped, no notifications will be sent to your webhook, and you will not be able to retrieve available content. Please note, if you decide to start the subscription again later using the Set-AuditLogs.ps1 you will not receive any content that was available between the stop and start time of the subscription.

 

This is quite a heavy post; I hope it has helped you in some way. It is just a starter, as you will probably want to do something with the logs instead of just writing them out to the Azure Logs. Maybe capturing a given process to then implement some logic to react. You might also want to put different Audit content types ContentUri onto different Azure Storage queue, so that different Azure Functions can process the ContentUri.