AZ CLI putting message on a storage queue – not a valid Base-64 string

Using Az CLI a lot recently, it has made interactions with Azure so much easier using PowerShell.

I had to write a simple PowerShell script that added items to a Storage queue. Once the items were added to a queue an Azure function picked up the items and processed them.

According to the documentation of Azure CLI you need to use az storage message put.

az login
#Get the connection string
$connectionString = az storage account show-connection-string --name "mystorageaccountname" –resource-group "MyResourceGroup" --query connectionString | ForEach-Object { $PSItem -join '' } | ConvertFrom-Json
#Create message
$message = "{""Name"":""Paul"",""LastName"":""Matthews"",""ID"":""d4b2ffc9-4380-46e4-a0bf-8a9ca58734d2""}"
#Add item to the queue
az storage message put --content $message --queue-name "myQueueName" --connection-string $connectionString

This fires successfully and I can see the item on the queue. However, when my Azure Function starts to run, I get the error message:

The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.

To solve this problem you just need to convert the string.

az login
#Get the connection string
$connectionString = az storage account show-connection-string --name "mystorageaccountname" –resource-group "MyResourceGroup" --query connectionString | ForEach-Object { $PSItem -join '' } | ConvertFrom-Json
#Create message
$message = "{""Name"":""Paul"",""LastName"":""Matthews"",""ID"":""d4b2ffc9-4380-46e4-a0bf-8a9ca58734d2""}"
#Encode to bytes
$b = [System.Text.Encoding]::UTF8.GetBytes($message)
#Convert to Base64String
$message64Base = [System.Convert]::ToBase64String($b)
#Add item to the queue
az storage message put --content $message64Base --queue-name "myQueueName" --connection-string $connectionString

Amazingly the value you push upto the queue is something like below:


But it appears on the queue correctly.

Unable to download the Exchange Online PowerShell Module – “Deployment and application do not have matching security zones”

If you want to use multi-factor authentication (MFA) to connect to Exchange Online PowerShell, Microsoft states you are required to install the Exchange Online Remote PowerShell Module, and use the Connect-EXOPSSession cmdlet to connect.

To download this, you need to go to the Exchange Admin Center (EAC) for your Exchange Online organization.

  • Head to Microsoft 365 Admin centre for your tenant and sign in.
  • In the left hand navigation, expand out Admin Centers and select Exchange
  • A new tab opens and loads up the Exchange admin center.
  • In the navigation select Hybrid, and then click the second configuration button.

  • This downloads and runs Microsoft.Online.CSE.PSModule.Client.application.

It was at this point I got an error message stated, “Application cannot be started Contact the application vendor.”

From clicking on the Details under summary email it stated: “Deployment and application do not have matching security zones.”

The problem here, is that, it is trying to run a click-once application. This only works in IE/Edge. I was accessing the EAC through chrome. Therefore, log into your tenant again using Edge or IE.

  • Once loaded, you will find Microsoft Exchange Online PowerShell available in your Windows 10 menu. Also, a PowerShell window will open ready for you to connect to Exchange online using PowerShell.

Connect-EXOPSSession -UserPrincipalName <your UPN>

Summary: When downloading the Microsoft Exchange Online PowerShell from the Microsoft Exchange Admin centre, ensure you have logged in using IE or Edge.

Programmatically change the New Menu in SharePoint Online using PowerShell

Back in October 2018 Microsoft changed the way the New menu on a document library worked. You can now edit what is shown in the New menu directly from the library.

I needed to know how to do this in code, and it looked like PNP automatically does this for you. After doing a get-pnpprovisioningtemplate, when you look at the list you will see a XML element called NewDocumentTemplates, which has a JSON formatted string. The property NewDocumentTemplates exists on the view of the list. This made me think that you could have different menu items for different views, but this doesn’t seem to be the case.

{"contentTypeId":"0x0101009348E1CE5767914598D327EAE7EB97D500245281855DF00844BF9BD64ADD983B7D","isContentType":true,"templateId":"0x0101009348E1CE5767914598D327EAE7EB97D500245281855DF00844BF9BD64ADD983B7D","title":"CF New Request","visible":true},
{"templateId":"NewDOC","title":"Word document","visible":true},
{"templateId":"NewXSL","title":"Excel workbook","visible":true},
{"templateId":"NewPPT","title":"PowerPoint presentation","visible":true},
{"templateId":"NewONE","title":"OneNote notebook","visible":true},
{"templateId":"NewXSLForm","title":"Forms for Excel","visible":true},

Now this seems to work, when applying it back to the same list. However, I found that if I set my own content types with visible to false, and then tried on a different list that did have that content type, it didn’t hide it.

So why not?

The reason is because the contentTypeId it is expecting, is the ListContentTypeId, not the actual ContentTypeId. For each list, the ListContentTypeId will be different.

When you add a site content type to a list:

“SharePoint list makes a local copy of the site content type and adds the copy to the content type collection on the list. The new list content type is a child of the site content type. The value of the Id property for the list content type is different from the value of the Id property for its parent site content type, but otherwise the two content types are initially the same.”

I’m sure PNP will pick up on this and update the code at some point.

PowerShell script

I decided to write my own PowerShell script to solve this issue. First, I needed to understand the JSON format. For the built in Microsoft menu items the JSON looks as follow:

  "title": "Word Document",
  "templateId": "NewDOC",
  "visible": true

There are quite a few built in ones, and from using PNP to export out, I was able to find what they were called. (If you find out any more, that I don’t have listed here please let me know.)

  • TemplateId – Name
  • NewFolder – Folder
  • NewDOC – Word Document
  • NewXSL – Excel workbook
  • NewPPT – PowerPoint presentation
  • NewONE – OneNote notebook
  • NewVSDX – Visio drawing
  • NewXSLForm – Forms for Excel
  • Link – Link

Note: If you don’t have a license for Visio, or do not have Forms enabled for your E3 license, then the menu item will not show up anyway.

The JSON format for content types are the following:

  "title": "<Content Type Name>",
  "templateId": "<ListContentTypeID>",
  "visible": <true/false>,
  "contentTypeId": "<ListContentTypeID>",
  "isContentType": <true/false>

Below is my function script. Set-ListNewMenuItems.ps1 With this script you can hide any content types/default menu items by using the titles.

-Url:’; -ListTitle:’Documents’ -ContentTypesToHide:’OneNote notebook’,’PowerPoint’,’Custom CT Name’

You can also hide all default menu items, and let your content types show. It will still show links and folder.

-Url:’; -ListTitle:’Documents’ -HideDefault:$true

If you want to hide link and folder too, you would need to include their names in the ContentTypesToHide.

-Url:’; -ListTitle:’Documents’ -ContentTypesToHide:’Folder’,’Link’ -HideDefault:$true

After running my script to remove Document, OneNote notebook, Word Document and Forms For Excel

-Url:’; -ListTitle:’Documents’ -ContentTypesToHide:’Document’,’OneNote notebook’,’Word Document’,’Forms For Excel’ -HideDefault:$false

As you can see from above, it states that it is Showing Visio drawing. However, on my tenant I do not have a license for it. Below shows you how my New menu has been updated.

The code isn’t perfect, and could do with improvements such as ordering. Please feel free to use it.

View Trace Logs with PnP Provisioning Templates using PowerShell

When running the Get-PnPProvisioningTemplate and the Apply-PnPProvisioningTemplate it is sometime necessary to see what is going on while the call is processing.

Just run the following command first before calling your Get/Apply -PNPProvisioningTemplate command.

Set-PnPTraceLog -On -Level:Debug

If you need to turn it off again

Set-PnPTraceLog -Off

PnP Provisioning Templates Adding – Everyone except external users

On a recent project of mine, I’ve been working with the PnP Provisioning engine using PowerShell. It’s really the first time I’ve really worked with it, and I must say I’m impressed at how easy it is to use.

If you have never used it before I recommend you checking out the following articles on MSDN.

Basic installation setup

If you have a Windows 10 device or have installed PowerShellGet, to check if you have the latest version installed in your PowerShell environment, run the following below

Get-Module SharePointPnPPowerShell* -ListAvailable | Select-Object Name, Version | Sort-Object Version -Descending

I already have the latest version at the time of writing this 2.17.1708.1.

You can install or Upgrade the SharePointPNPPowerShell with the following commands.

To Install

#SharePoint Online
Install-Module SharePointPnpPowerShellOnline

#SharePoint 2016
Install-Module SharePointPnPPowerShell2016

#SharePoint 2013
Install-Module SharePointPnPPowerShell2013

To Upgrade

Update-Module SharePointPnPPowerShell*

Everyone and Everyone Except External Users

The quickest way to create a PnP Provisioning template ready to use again, is to create your site within SharePoint using point and click.

I’ve created my site with Members set up to “Everyone except external users” and Visitors set up to “Everyone”.

After you have created your site, you use the following commands to connect and export the template.

Connect-PnpOnline -Url: -Credentials: (Get-Credential)
Get-PnPProvisioningTemplate -Out 'pnpExample.xml'

If you open the XML file, and look in the <pnp:Security> section, you will see that Additional Members has c:0-.f|rolemanager|spo-grid-all-users/{GUID} and Additional Visitors has c:0(.s|true. These represent Everyone except external users and Visitors respectively.

If you are only using this template to create more sites in the same tenant that you exported it from, then you are good. The GUID after ‘spo-grid-all-users’ will always be that GUID in your tenant. However, when you want to use this template in other tenants – for example you have a staging and production environment – then this GUID will not work and the importing of the template will not add Everyone except external users to your members group.

What is the GUID after spo-grid-all-users?

It turns out that the GUID relates to the Authentication Realm ID of your tenant, and luckily PnP have a PowerShell command


How does that help us?

You can pass parameters into the Apply-PnPTemplate. First we need to change the XML inside <pnp:AdditionalMembers>


<pnp:User Name="c:0-.f|rolemanager|spo-grid-all-users/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxx" />


<pnp:User Name="c:0-.f|rolemanager|spo-grid-all-users/{parameter:AuthenticationRealm}" />

Save the XML template and now you can run the following commands to apply the template to a new site in a different tenant. (Please note my team site I’m applying this to has already been created)

Connect-PnpOnline -Url: -Credentials: (Get-Credential)
$AuthenticationRealm = Get-PnPAuthenticationRealm
Apply-PnPProvisioningTemplate -Path:'pnpExample.xml' -Parameters:@{"AuthenticationRealm"=$AuthenticationRealm}

After applying the template to another tenant, you will see your “Everyone except external users” inserted correctly.

Using the Windows Credentials Manager with PnP PowerShell

Do you get fed up keep typing in your username and password when you are connection to SharePoint via PowerShell? Or you have to keep changing between multiple tenants and get fed up keep typing in the username and password? Did you know you could use the built-in Windows Credential Manager to help ease your pain?

  • Open your Credential Manager

  • Under the Generic Credentials, click ‘Add a generic credential

  • For each tenant/user account you need, create a Generic Credential.
    • Put a label name to indicate what the permissions are for (e.g DevAdmin, TestAdmin, TestUser etc)
    • Put the username of the account
    • Put the password of the account

  • After you have created your Generic Credential(s), when you try to Connect to SharePoint using PNP, you can pass your Label to the credentials.
    Connect-PnPOnline -Url "" -Credentials:devAdmin

    In the screenshot below, I’m connecting to my tenant using a label I created called CFAdmin1

Using the above technique of Credential Manager labels, you can make your PowerShell scripts easier by creating a string variable called label and pass it in. This will make running the same script for multiple environments easier.

    # The environment label to use for connection

    # The URL of the tenancy to create the site collections in, do not include the -admin
if ($VerbosePreference) {
    Set-PnPTraceLog -On -Level:Debug
else {
    Set-PnPTraceLog -Off
Connect-PnPOnline -Url:$URL -Credentials:$Label
… #Additional code to update Web

If the above file was called UpdateWebSite.ps1 I would type in the following:

.\UpdateWebSite.ps1 -Url:'' -Label:devAdmin

Building SharePoint 2016 development environment – Part 15 – Configuring Workflow

A few years ago I wrote “Build your SharePoint 2013 development machine on Windows Server 2012” series, I mainly work in the cloud now, but as the blogs was so popular, I thought I would create a new series for the newer version of SharePoint.

You can access other parts of this post below.

The configuration of the Workflow Manager for SharePoint 2016 is the same as it was for SharePoint 2013. Not only do you need to install the separate Workflow Manager components, once installed SharePoint designer will show SharePoint 2013 Workflow in a dropdown when deciding which platform to build the workflow on.

We will be installing Workflow Manager 1.0 CU3. Although my instructions add all this to the SharePoint Machine, the reason why Microsoft have made the Workflow manager separate is for scaling. There is no need for this to be installed on the SharePoint box. You could create another Windows Server 2012 R2 and add that to the domain and run the Workflow manager on that. There are probably a few more steps required in configuring. Here is a full walkthrough provided by Microsoft Technet if you wish to delve in deeper.

Installing SharePoint Designer 2013

Wait! SharePoint Designer 2013? Yes.

There is no SharePoint Designer 2016, there is no plan to release one either. Microsoft have stated that they will support SPD 2013 with SharePoint 2016. We are installing SharePoint Designer here because I can use it to prove if you have correctly configured Workflow Manager with SharePoint 2016 correctly.

SharePoint Designer 2013 is a free tool from Microsoft.

  1. Download SharePoint Designer 2013 32bit from the Microsoft Site.
  2. Once downloaded run the file sharepointdesigner_32bit.exe
  3. Accept the License terms and click Continue.
  4. Click Install Now, (Unless you wish to customise and change file location, user information etc)
  5. Once installed, I’d recommend performing a Windows Update. From the Start Menu, type Windows Update, open the application and run any updates required. Reboot if necessary.

Check to see Workflow settings in SharePoint Designer

  1. From the start menu, type SPD and open SharePoint Designer 2013.
  2. Once it has loaded up, click Open Site
  3. Type the URL click Open
  4. If prompted, enter your credentials
    User: cfcode2016\SP_SetupPassword: Pa55w0rd
  5. From the Navigation menu, select Workflows

  6. On the ribbon menu, select List Workflows > Documents

  7. In the Create List Workflow dialog, at the bottom you will see a dropdown box for Choose the platform to build your workflow on. Only SharePoint 2010 will be listed.

  8. When we come back to this later, we will see SharePoint 2013 Workflow. Close SharePoint Designer for now.

Configuring Workflow Manger accounts

The Workflow Manager will run under new accounts that we haven’t created yet.

  1. On the Domain Controller machine, in the start menu, type Active Directory Users and Computers and open it.
  2. Expand the tree in the left hand pane to see the Managed Service Accounts OU. Select the Managed Service Accounts OU.
  3. Right click in the right hand pane, and select New > User.
  4. Create a user called SP_Workflow. Set the full name and log on name to SP_Workflow. Click Next.
  5. In the password dialog screen, enter the following and click Next
    1. Password and Confirm Password as: Pa55w0rd
    2. Untick User must change password at next logon.
    3. Leave User cannot change password as unticked
    4. Tick Password never expires
    5. Leave Account is disabled as unticked
    6. Click Next. Then click Finished.

Setting up SQL with the correct Security Accounts

  1. On the SharePoint Machine, from the start menu, type SQL Server Management Studio and open up the application
  2. In SQL Server click Connect. (This should be to SQL2016 database instance).
  3. In the left hand menu expand Security. Right click Logins. And select New Login…
  4. In the Login – New dialog box, click the Search button.
  5. Click the Locations button and select Entire Directory.
  6. Type SP_Workflow in the Enter the object name to select, and click Check Names. This will resolve the name. Click OK.
  7. In the left hand panel select Server Roles.
  8. Tick both securityadmin and dbcreator then click OK.
  9. Close down SQL Server Management Studio

Giving SP_Workflow administrative rights on the SharePoint machine.

  1. From the start menu, type Edit local users and groups and open up the application.
  2. In the left hand panel, select Groups
  3. In the right hand pane, double click Administrators
  4. On the Administrators Properties dialog box, click Add
  5. Type SP_Workflow in the Enter the object name to select, and click Check Names. This will resolve the name. Click OK.
  6. Close Edit local users and groups.

Install the Microsoft Web Platform Installer 5.0

  1. Go to the URL and download the latest Microsoft Web Platform Installer
  2. Once downloaded run the file wpilauncher.exe
  3. If like my machine it is already on there, it will just open the Web Platform Installer 5.0 else it will install it for you. Accept the License Agreement and click Install. Then click Finish when complete.

Install Workflow manager

  1. From the Start menu, type Web Platform Installer and open the application

  2. In the search box in the top right of the screen, type Workflow Manager and press Enter.
  3. Click Add on the Workflow manager 1.0 Refresh (CU2) and click Install at the bottom.

  4. Click I Accept

  5. When complete, click Continue.

  6. Click Finish.

  7. Close the Workflow Manger Configuration Wizard that has popped up.

Apply Cumulative Update 3.0 for Workflow Manager 1.0

  1. Close and re-open the Web Platform Installer 5.0 we are going to install the CU 3. (You need to close and re-open otherwise the installer thinks Workflow Manger 1.0 hasn’t been installed)
  2. Type Workflow Manger and press Enter in the top right search box.
  3. Click Add for Workflow Manager 1.0 Cumulative Update 3, then click Install at the bottom.
  4. Click I Accept. Once installed click Finish. Click Exit on the Web Platform Installer.

Configure the Workflow manager

  1. From the start menu, type Workflow Manager Configuration
  2. Click on Configure Workflow Manger with Custom Settings
  3. In the Configure Farm Management Database,
    1. Enter your SQL Server Instance:
    2. Tick Use the above SQL Server Instance and Settings for all Databases
    3. Enter the Database Name: WF_ManagementDB
    4. Click Test Connection button to ensure all working OK.
  4. In the Configure Instance Management Database
    1. Enter the Database Name: WF_InstanceManagementDB
    2. Click Test Connection button to ensure all working OK.
  5. In the Configure Resource Management Database
    1. Enter the Database Name: WF_ResourceManagementDB
    2. Click Test Connection button to ensure all working OK.
  6. In the Configure Service Account
    1. Enter the User ID: CFCODE2016\SP_Workflow
    2. Enter the Password: Pa55w0rd
  7. In Configure Certificates
    1. Leave Auto-generate ticked
    2. Certificate Generation Key: Pa55w0rd
    3. Confirm Certificate Generation Key: Pa55w0rd
  8. In Configure Ports leave default port numbers
    1. https: 12290
    2. http: 12291
    3. Leave Allow Workflow management over HTTP on this Computer unticked
    4. Leave Enable firewall rules on this compute unticked (As we have disabled our firewall)
  9. In Configure Admin Group
    1. Leave BUILTIN\Administrators
  10. Click Next button
  11. On the Service Bus Configuration page, please provide the following
  12. In Configure Farm Management Database
    1. Enter the Database Name: Sb_ManagementDB
    2. Click Test Connection button to ensure all working OK
  13. In Configure Gateway Database
    1. Enter the Database Name: Sb_GatewayDB
    2. Click Test Connection button to ensure all working OK
  14. In Configure Message Container Database
    1. Enter the Database Name: Sb_MessageContainerDB
    2. Click Test Connection button to ensure all working OK
  15. In Configure Service Account
    1. Tick Use the same service account credentials as provided for Workflow Manager
  16. In Configure Certificate
    1. Tick Auto-generate
    2. Tick Use the same certificate generation key as provided for Workflow Manager
  17. In Configure Ports
    1. https: 9355
    2. tcp: 9354
    3. Message Broker Port: 9356
    4. Internal communication Port Range: 9000
    5. Untick Enable firewall rules on this computer (as we have disabled out firewall)
  18. In Configure Admin Group
    1. Leave BUILTIN\Administrators
  19. Click Next button
  20. On the Summary page, click the Tick button at the bottom right of the screen to start installation.
  21. The configuration process can take up to 10 minutes to complete. Once complete, you will see a success pag

Add Workflow Manager Certificate into SharePoint

  1. In Start Menu, type IIS and open Internet Information Services (IIS) Manager
  2. Expand your server name, and Sites. You will now see a site called Workflow Management Site

  3. Click on Workflow Management Site, then on the right hand pane, click Bindings
  4. Select https and click edit.

  5. On the Edit Site Binding, under SSL certificate you will see a Certificate that matches your Server Name. Click the View button.

  6. On the Certificate dialog, click on the Details tab.
  7. Then click Copy to File button.
  8. On the Certificate Export Wizard click Next.
  9. On the Export Private Key page, select No, do not export the private key, click Next

  10. On Export File Format page, select DER encoded binary X.509 (.CER) Click Next
  11. On File to Export page, select a path and filename on your machine. Click Next.

  12. Click Finish. You will receive a successful export message.

Import Certificate into SharePoint Trust

  1. Open SharePoint 2016 central administration
  2. Under Security > General Security click Manage Trust
  3. Click the New button in the ribbon.
  4. On the Establish Trust Relationship page, enter following information:
    1. Name: Workflow Manager
    2. Root Authority Certificate: <Select your file from previous steps>
  5. Click OK.
  6. You will see your certificate in the store.

Register Workflow Service Proxy

  1. In Start Menu, type SharePoint 2016 management Shell (run as administrator) and open the application
  2. In the console type:

Verify the Configuration of Workflow Manager.

  1. Open SharePoint 2016 central administration
  2. Click Application Management
    > Manage services applications
  3. At the bottom of the Manage Services Applications page, there will be Workflow Service Application Proxy

  4. If you click on Workflow Service Application Proxy it will take you a status page that will show you that workflow is now connected.

Check to see Workflow Settings are working in SharePoint Designer

  1. From the start menu, type SharePoint Designer and open the application
  2. Once SharePoint designer has opened, click Open Site.
  3. Type the URL click Open.
  4. If prompted, enter your credentials
    User: CFCode2016\SP_Setup
    Password: Pa55w0rd
  5. From the Navigation menu, select Workflows
  6. On the ribbon menu, select List Workflow > Documents
  7. In the Create List Workflow dialog, at the bottom you will see a dropdown box for Choose the platform to build your workflow on. Both SharePoint 2010 and SharePoint 2013 should be listed if the Workflow is set up correctly.

We are almost at the end. You SharePoint farm is configured to give you a good start as a development machine. Only thing left now is actual development tools. That will be covered in my final post of the series. Shut down your machines, take a checkpoint. (We will remove checkpoints in the last post)