View Trace Logs with PnP Provisioning Templates using PowerShell


When running the Get-PnPProvisioningTemplate and the Apply-PnPProvisioningTemplate it is sometime necessary to see what is going on while the call is processing.

Just run the following command first before calling your Get/Apply -PNPProvisioningTemplate command.

Set-PnPTraceLog -On -Level:Debug

If you need to turn it off again

Set-PnPTraceLog -Off

PnP Provisioning Templates Adding – Everyone except external users


On a recent project of mine, I’ve been working with the PnP Provisioning engine using PowerShell. It’s really the first time I’ve really worked with it, and I must say I’m impressed at how easy it is to use.

If you have never used it before I recommend you checking out the following articles on MSDN.

https://msdn.microsoft.com/en-us/pnp_powershell/pnp-powershell-overview

https://msdn.microsoft.com/en-us/pnp_articles/pnp-provisioning-framework

https://msdn.microsoft.com/en-us/pnp_articles/pnp-remote-provisioning

Basic installation setup

If you have a Windows 10 device or have installed PowerShellGet, to check if you have the latest version installed in your PowerShell environment, run the following below

Get-Module SharePointPnPPowerShell* -ListAvailable | Select-Object Name, Version | Sort-Object Version -Descending

I already have the latest version at the time of writing this 2.17.1708.1.

You can install or Upgrade the SharePointPNPPowerShell with the following commands.

To Install

#SharePoint Online
Install-Module SharePointPnpPowerShellOnline

#SharePoint 2016
Install-Module SharePointPnPPowerShell2016

#SharePoint 2013
Install-Module SharePointPnPPowerShell2013

To Upgrade

Update-Module SharePointPnPPowerShell*

Everyone and Everyone Except External Users

The quickest way to create a PnP Provisioning template ready to use again, is to create your site within SharePoint using point and click.

I’ve created my site with Members set up to “Everyone except external users” and Visitors set up to “Everyone”.

After you have created your site, you use the following commands to connect and export the template.

Connect-PnpOnline -Url:https://mytenant.sharepoint.com/sites/pnpexamples -Credentials: (Get-Credential)
Get-PnPProvisioningTemplate -Out 'pnpExample.xml'

If you open the XML file, and look in the <pnp:Security> section, you will see that Additional Members has c:0-.f|rolemanager|spo-grid-all-users/{GUID} and Additional Visitors has c:0(.s|true. These represent Everyone except external users and Visitors respectively.

If you are only using this template to create more sites in the same tenant that you exported it from, then you are good. The GUID after ‘spo-grid-all-users’ will always be that GUID in your tenant. However, when you want to use this template in other tenants – for example you have a staging and production environment – then this GUID will not work and the importing of the template will not add Everyone except external users to your members group.

What is the GUID after spo-grid-all-users?

It turns out that the GUID relates to the Authentication Realm ID of your tenant, and luckily PnP have a PowerShell command

Get-PnPAuthenticationRealm

How does that help us?

You can pass parameters into the Apply-PnPTemplate. First we need to change the XML inside <pnp:AdditionalMembers>

From:

<pnp:User Name="c:0-.f|rolemanager|spo-grid-all-users/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx" />

To:

<pnp:User Name="c:0-.f|rolemanager|spo-grid-all-users/{parameter:AuthenticationRealm}" />

Save the XML template and now you can run the following commands to apply the template to a new site in a different tenant. (Please note my team site I’m applying this to has already been created)

Connect-PnpOnline -Url:https://myOtherTenant.sharepoint.com/sites/pnpexamples -Credentials: (Get-Credential)
$AuthenticationRealm = Get-PnPAuthenticationRealm
Apply-PnPProvisioningTemplate -Path:'pnpExample.xml' -Parameters:@{"AuthenticationRealm"=$AuthenticationRealm}

After applying the template to another tenant, you will see your “Everyone except external users” inserted correctly.

Using the Windows Credentials Manager with PnP PowerShell


Do you get fed up keep typing in your username and password when you are connection to SharePoint via PowerShell? Or you have to keep changing between multiple tenants and get fed up keep typing in the username and password? Did you know you could use the built-in Windows Credential Manager to help ease your pain?

  • Open your Credential Manager

  • Under the Generic Credentials, click ‘Add a generic credential

  • For each tenant/user account you need, create a Generic Credential.
    • Put a label name to indicate what the permissions are for (e.g DevAdmin, TestAdmin, TestUser etc)
    • Put the username of the account
    • Put the password of the account

  • After you have created your Generic Credential(s), when you try to Connect to SharePoint using PNP, you can pass your Label to the credentials.
    Connect-PnPOnline -Url "https://mydevtenant.sharepoint.com/sites/pnpexamples" -Credentials:devAdmin
    

    In the screenshot below, I’m connecting to my tenant using a label I created called CFAdmin1

Using the above technique of Credential Manager labels, you can make your PowerShell scripts easier by creating a string variable called label and pass it in. This will make running the same script for multiple environments easier.

[CmdletBinding(SupportsShouldProcess)]
param(
    # The environment label to use for connection
    [Parameter(Mandatory)]
    [string]
    $Label,

    # The URL of the tenancy to create the site collections in, do not include the -admin
    [Parameter(Mandatory)]
    [string]
    $URL,
)
if ($VerbosePreference) {
    Set-PnPTraceLog -On -Level:Debug
}
else {
    Set-PnPTraceLog -Off
}
Connect-PnPOnline -Url:$URL -Credentials:$Label
…
… #Additional code to update Web
…

If the above file was called UpdateWebSite.ps1 I would type in the following:


.\UpdateWebSite.ps1 -Url:'https://mydevtenant.sharepoint.com/sites/pnpexamples' -Label:devAdmin

Power BI using Parameters to use the same PBIX file for different tenants


In the past I, have created a Power BI file connecting it to the developer tenant data, design the reports based on the dummy data in my development tenant, before publishing it to the production tenant using the production data. For those that have done this before, you will know it’s not easy because all your data queries are pointing to the URL you first used, every query you have created will have this URL in, also GUIDs for lists filters etc could all be different in another tenant.

I will be continuing from my last example of a movie database from my previous post. I will be updating the queries so that in future, there is one location to change my parameters. My previous post has 4 queries which was my main Movies list from SharePoint, the TaxonomyHiddenList from SharePoint, and two references of the TaxonomyHiddenList renamed and filtered for Genre and Country.

First I need to think what might be different in the other tenant compared to my original one.

  • Tenant Name
  • Site/Web URL
  • Genre TermSet Id
  • Country TermSet Id

I need to create 4 paramters and name them to correspond with possible changing values. To do this, I need to ensure I’m in the Edit Queries. From your report, on the ribbon click Edit Queries. This will bring up the Edit Queries editor.

Underneath your current queries, right click and from the context menu, select New Group. Name the group Parameters. Your original queries will be places in a group folder called Other Queries.

Right click the Parameters folder, and from the context menu, select New Parameter…

Add 4 new parameters:

  • Tenant:
    • Name: Tenant
    • Required: True
    • Type: Text
    • Suggested Values: Any
    • Current Value: mytenant.sharepoint.com
  • WebSiteUrl:
    • Name: WebSiteUrl
    • Required: True
    • Type: Text
    • Suggested Values: Any
    • Current Value: /sites/taxonomy
  • GenreTermSetId:
    • Name: GenreTermSetId
    • Required:True
    • Type: Text
    • Suggested Values: Any
    • Current Value: f46ebfeb-cc18-479f-bac8-48fdca36dd6c
  • CountryTermSetId:
    • Name: CountryTermSetId
    • Required: True
    • Type: Text
    • Suggested Values: Any
    • Current Value: 6e0c3d2e-5514-4744-a08d-7318fd437a45

Click OK.

You should now have 4 parameters showing in your Query window. Now we need to use these.

Starting with the CountryTermSetId, previously my Country query was filtered on a text value that was the GUID of the TermSetId. Now I have a parameter I can use instead. I will want to update my query to use a parameter instead of free text. In the Applied Steps for Country, I can click the cog next to Filtered Rows which is step 2.

This brings up a Filter Rows dialog, where you can see it current is using a Text value to query against.

By selecting the dropdown where it currently shows ABC, you can select parameter instead. Then in the last dropdown box you are given all your parameters, here I would select CountryTermSetId.

I then repeat those steps for Genre query, using the GenreTermSetId instead.

For the movies query, I need to click the Advanced editor from the ribbon bar to amend the query to use parameters.

The text in this window is the query that has been currently set.

let
    Source = SharePoint.Tables("https://mytenant.sharepoint.com/sites/taxonomy", [ApiVersion = 15]),
    #"acd616ae-ee59-4597-86f3-3d5b65e64547" = Source{[Id="acd616ae-ee59-4597-86f3-3d5b65e64547"]}[Items],
    #"Renamed Columns" = Table.RenameColumns(#"acd616ae-ee59-4597-86f3-3d5b65e64547",{{"ID", "ID.1"}}),
    #"Removed Other Columns" = Table.SelectColumns(#"Renamed Columns",{"Title", "Total Worldwide Box_", "Genre", "Country"}),
    #"Expanded Genre" = Table.ExpandRecordColumn(#"Removed Other Columns", "Genre", {"TermGuid"}, {"Genre.TermGuid"}),
    #"Expanded Country" = Table.ExpandRecordColumn(#"Expanded Genre", "Country", {"TermGuid"}, {"Country.TermGuid"})
in
    #"Expanded Country"

We need to change the Source row (Line 2), and the #”acd616ae-ee59-4597-86f3-3d5b65e64547″ (Line3).

The Source row (Line 2), would need the URL part changed from

Source = SharePoint.Tables("https://mytenant.sharepoint.com/sites/taxonomy", [ApiVersion = 15]),

to

Source = SharePoint.Tables("https://"&tenant&WebSiteUrl, [ApiVersion = 15]),

You can see we are inserting the parameters names and concatenating them using the ampersand.

The Line 3 row we are using the Title of the list instead of the ID as this would be different in another tenant.

#"acd616ae-ee59-4597-86f3-3d5b65e64547" = Source{[Id="acd616ae-ee59-4597-86f3-3d5b65e64547"]}[Items],

to

#"acd616ae-ee59-4597-86f3-3d5b65e64547" = Source{[Title="Movies"]}[Items],

We would also do something similar to the TaxnomyHiddenList query too. Changing the first 3 lines from:

let
    Source = SharePoint.Tables("https://mytenant.sharepoint.com/sites/taxonomy", [ApiVersion = 15]),
    #"63f7f485-a5ca-4be5-815f-d0e3235e96d1" = Source{[Id="63f7f485-a5ca-4be5-815f-d0e3235e96d1"]}[Items],

to

let
    Source = SharePoint.Tables("https://"&Tenant&WebSiteUrl", [ApiVersion = 15]),
    #"63f7f485-a5ca-4be5-815f-d0e3235e96d1" = Source{[Title="TaxonomyHiddenList"]}[Items],
 

If you have followed this correctly, you would find that your data still works, and there are no issues. Click Close & Apply from the ribbon. The apply query changes will happen but your data will be the same. In the future when you want to publish to a different tenant you just need to change your parameters, instead of going to each query and updating the URL ID’s/GUIDs.

SharePoint Designer 2013 Workflows and POST REST API


Following on from my last post of using GET REST API in workflows, this post will show how you can create an item in a list using the POST REST API.

Creating a list Item using REST API.

Using the same list as before, on my SharePoint site, I have a list. The list is nothing special, it’s just an example of some data.

Title: String
Person: User field
Colour: Choice field

You wouldn’t do this in real world scenario, but my demo will demonstrate when you run the workflow on any item, it will just create a new item in the list.

A better real world scenario might be when an item is added to this list, some of the information needs to be copied and created in another list, or when an item is changed a workflow fires to copy the information to another list similar to a history list.

Creating the workflow

In SharePoint Designer, I have gone to my list, and created a new workflow.

  • First, we need to add a Dictionary Item. This is so we can create a header for our HTTP Call. Type in “Dictionary
    and then in the options to choose from select Build Dictionary.

    Click on “this”.

    A dialog will appear, and it is here we need to add two headers.
Name Type Value
accept String application/json;odata=verbose
content-type String application/json;odata=verbose

Click on Variable:dictionary, and create a new variable called Headers.

  • We need to create another Dictionary now, this is to store the “type”. When you create an item with REST API, you need to provide the __metadata with the ListItemEntityType. Typically this item is SP.Data.[ListName]Item. However, to be absolutely sure, you can use this REST API call in the browser to find out.

     

    https://<tenant>/sites/<Site>/_api/web/lists/getbytitle('	<listTitle>')/ListItemEntityTypeFullName
    
Name Type Value
type String SP.Data.DemoListItem

  • One more dictionary needs to be created, this is the RequestContent for the POST Call.
Name Type Value
__metadata Dictionary Variable:EntityType
Title String New Item
Colour String Blue
PersonId String Current Item:Created By (return field as: User Id Number)

When you create this, ensure that __metadata has 2 underscores in front of it.
All fields are the internal names.

User fields all have an Id field with it, in my case the Person field has a PersonId field, it is this that I set with another User Id.

Choice fields just accept a valid string.

Taxonomy fields are a little different, see section below.

  • Now we can call a HTTP web service. Underneath the dictionary, start typing “http” and press enter to insert Call HTTP Web Service.
    Click on “this” to open up the dialog box, and then click on the … for the string builder dialog box.

    I like to use as much dynamic values as possible so my URL in here looks like the following:

    [%Workflow Context:Current Site URL%]/_api/web/lists/getbytitle('[%Workflow Context:List Name%]')/items
    

    Set the HTTP method to HTTP POST.

    After entering the URL and setting the Method, click the dropdown at the far right on the line in SPD and select properties. Set the RequestHeaders to the Dictionary Variable:Headers you made in the first step. Set the RequestContent to the dictionary Variable:NewItemMetadata. Set the ResponseContent to a new variable, and the ResponseStatusCode to a new variable. Click OK.

    The ResponseContent will be populated with the results, and the ResponseStatusCode will be populated with Created if successful, or a different value if something went wrong.

  • After the Web service call, I add an If Statement, to test that the variable responseCode equals to Created.
  • I log success to the history list.
  • In the else statement of if “responseCode equals Created” I grab the error message from the ResponseContent dictionary and output the value to an error message. Then I display the error message in the history list. To grab the error the dictionary path is error/message/value

  • Don’t forget to Transition to Stage “End of Workflow” at the end.

Publish your workflow.

When I run the workflow against any list item, after a moment a new entry is added to the list.

Updating Taxonomy/managed metadata column in POST REST API calls.

To be able to update a Taxonomy column in a list you would need to create the following 2 dictionary’s first, then assign 2nd dictionary to the column.

TaxonomyMetadata Dictionary

Name Type Value
type String SP.Taxonomy.TaxonomyFieldValue

TaxonomyValue Dictionary

Name Type Value
__metadata Dictionary TaxonomyMetadata Dictionary
Label String <Label value>
TermGuid String <TermGuid>
WssId String -1

<Label Value> – the actual value of the taxonomy term
<TermGuid> – The guid of the taxonomy term found in the termstore.
The WssId can always be -1 as SharePoint can work that out itself, but it requires a value.

PostRequestContent

Name Type Value
__metadata Dictionary Variable:EntityType
Title String New Item
<TaxonomyColumnInternalName> Dictionary Variable:TaxonomyValue Dictionary

It is the final dictionary above that you would set in the web service call as the RequestContent.

SharePoint Designer 2013 Workflows and GET REST API


SharePoint Workflows have been around for a long time, and with Microsoft flow now firmly taking hold the need for SharePoint workflows might be less. However, I recently had to work with SharePoint Workflows and learnt a few things around using REST API and I wanted to share my knowledge.

Reading a list/Item using REST API.

On my SharePoint site, I have a list. The list is nothing special, it’s just an example of some data.

Title: String
Person: User field
Colour: Choice field

All I’m going to do in the workflow, is when it is run, it will grab the items in the list, and then display the results in the history list. The point of this demo is to show how to make REST API calls and grab/loop through data.

Creating the workflow

In SharePoint Designer, I have gone to my list, and created a new workflow.

  • First, we need to add a Dictionary Item. This is so we can create a header for our HTTP Call. Type in “Dictionary
    and then in the options to choose from select Build Dictionary.

    Click on “this”.

    A dialog will appear, and it is here we need to add two headers.
Name Type Value
accept String application/json;odata=verbose
content-type String application/json;odata=verbose

Click on Variable:dictionary, and create a new variable called Headers.

Because SPD is a pain, it has already created a variable called dictionary, if you click on Local Variables from the ribbon, you can delete this variable. This will ensure you keep only the variables you are using within your workflow and save confusion later. Always give your variables sensible names, just as if you were writing code.

  • Now we can call a HTTP web service. Underneath the dictionary, start typing “http” and press enter to insert Call HTTP Web Service.
    Click on “this” to open up the dialog box, and then click on the … for the string builder dialog box.

    I like to use as much dynamic values as possible so my URL in here looks like the following:

    [%Workflow Context:Current Site URL%]/_api/web/lists/getbytitle('[%Workflow Context:List Name%]')/items?$select=Title,Colour,Person/Title,Person/Name,Person/EMail&$expand=Person
    

    In the above API call I’m bringing back additional information about the Person from the person column. See at the end of this post about bringing back other information on a person column.

    After entering the URL, click the dropdown at the far right on the line in SPD and select properties. Set the RequestHeaders to the Dictionary you made in the first step. Set the ResponseContent to a new variable, and the ResponseStatusCode to a new variable. Click OK.

    The ResponseContent will be populated with the results, and the ResponseStatusCode will be populated with OK if successful, or a different value if something went wrong.

  • After the Web service call, I add an If Statement, to test that the variable responseCode equals to OK.
  • Now we want to grab the results. So now we will grab a dictionary item. Type “Get” and press enter to insert Get an Item from a Dictionary.
    Click “item by name or path” and type “d/results”
    Click “dictionary” and select “ResponseContent”
    Click “item” and create a new variable. This variable is a dictionary too that will contain the individual items. I’ve called mine DemoList.
  • We need to count the number of items in the new dictionary variable. I do this to ensure that we have returned items.
    Type “Count” and press enter to insert count items in a dictionary.

    Click “dictionary” and select variable “DemoList”.
    Can leave output Variable to count.

  • Put another If statement, and check that Count is greater than 0.
  • We now need to create a variable that will be our indexloop.
  • Add a loop with a condition. (Alternatively, we could have skipped the last step and added a “loop n times” instead).
    Set the condition to read. “The contents of this loop will run repeatedly while: Variable: indexloop is less than Variable: count

  • Now I’m going to get the values of Title, Colour, Person/Title and Person/Name from the DemoList dictionary.
    Type “Get” and press enter to insert Get an Item from a Dictionary
    Click “item by name or path” and type “([%Variable: indexloop%)/Title”

    Click “dictionary” and select “DemoList”
    Click “item” and create a new variable.

    Repeat this for all the parameters.

  • Still inside the loop underneath gathering the parameters we need to raise the index variable by one.
    Type “Calc” then press enter to insert Do Calculations
    Click first “value” and select the Variable: “indexloop”

    Click second “value” and set to 1.

    Set the output to a number variable. I called mine NewIndex

  • Now you need to set the variable NewIndex to indexloop.
  • Lastly, for demo purpose, I’m logging the results I’ve got to the history list. This logging to history lists is still within the loop.
    Type “Log” then press Enter to insert log to History List.
    Then I’m just logging out the variables I’ve captured.
  • Don’t forget to Transition to Stage “End of Workflow” at the end.

Publish your workflow.

When I run the workflow against any list item I see the following results in my History List.

Bringing back additional Person Data in REST API calls.

To bring back a person column data (or a lookup column), you need to expand it in a REST API call, you also need to include the values you want to bring back from the expanded column. A list of all possible values you can bring back for a user column can be found if you type the following in a browser (after already signing into SharePoint that is).

https://<tenant>/sites/<site>/_api/web/lists/getbytitle('<ListTitle>')/items(1)/<UserFieldInternalNameColumn>

*I’m using XV – XML Viewer chrome extension for the layout.