Updating an expired Client Secret of a SharePoint Add-in using Azure-AD/Az Cli

Back in May 2016, I wrote a post to show you how to update the Client Secret of a SharePoint add-in. This used the PowerShell module MSOL. https://cann0nf0dder.wordpress.com/2016/05/18/updating-an-expired-client-secret-of-sharepoint-add-in/

The MSOL module is now (or going to be) deprecated. Therefore, I needed to find a different way of doing this, and ideally something that could be done with Azure Dev-ops pipelines. I originally started with AZ CLI. Although the Client Secret is tied to a Service Principal in Azure AD, I was unable to change the Key Credentials or Password Credentials for it.

Due to the issues I was getting with AZ CLI, I used Azure-AD module instead. See at end of blog post, how I got Az CLI to work with a workaround.

Updating Client Secret

Connect to Azure-AD

First you will need to connect to Azure-AD

The above code ensures that you have Azure AD installed on your machine, and logs you in.

Getting the Add-in as a Service Principal

Once you have logged in, you will be able to call back your SharePoint Add-in. The SharePoint Add-in is actually a Service Principal within Azure AD, and we will grab this using Get-AzureADServicePrincipal. You can do this by using the AppId, or the AppName. The AppId is fine to use, but when you want to use the same script across multiple environments, you will need to ensure you are passing in the different AppId for each environment. THis is why I have used the Name of the Add-In. (Assuming you have given your App the same name in each environment)

Create a new Secret

Next you need to be able to create a new secret. This is done by creating random bytes and converting to a Base64String. I ensure the password is valid for an additional 2 years.

Create Key Credentials and Password Credential for the App

The SharePoint Add-In requires 2 Key Credentials (One Sign and one Verify) and 1 Password Credentials. The following script creates new ones, this allows both the old password and the new password to continue working at the same time, until you are able to update the code that uses the ClientID and ClientSecret.

Removing the original Key and Password Credential for the App

The following code shows how to loop round the Key and Password credentials and remove the original ones. It does this by looking for any Credentials that were created before the start date of the new ones. I would only run this part of my code, once I know I have updated my application to use the new password. Not in my example here, but where I’m using it in the real world, my Client Secret is stored within a Keyvault. I would update the keyvault value (ensuring to disable the previous version).

Connecting with AZ Cli Workaround

Using Azure Dev-Ops Pipeline, I really wanted to use AZ cli to be able to update the Client Secret of a SharePoint Add-in. Due to the error messages when I attempted to update the Service Principal Key Credential and Password Credentials, I was forced to use Azure-AD instead. So how can I uses AZ Cli.

My Dev-Ops Pipeline uses a service account, and I have ensured this service account has permissions to update the directory.

Then I can connect from Az Cli to Azure-AD doing the following:

I add the above code in the full script just after the parameter, as the pipeline will already be signed in as the Pipeline Service Principal. It will then grab the Access token to sign in with Azure AD, and then able to run the rest of the script.

Full Script

Building SharePoint 2016 development environment – Part 12 – Configuring Hosting Apps and HNSC

A few years ago I wrote “Build your SharePoint 2013 development machine on Windows Server 2012” series, I mainly work in the cloud now, but as the blogs was so popular, I thought I would create a new series for the newer version of SharePoint.

You can access other parts of this post below.

Before I create the App Management Service, I’m going to create a separate Domain for the Apps. By creating a separate domain, it helps you write apps that won’t allow cross-site scripting between apps and SharePoint site.

Configuring Hosting Apps

First we need to configure DNS

  1. Go to you Domain Controller and from the Start Menu type DNS, and open the application.
  2. In the Left Hand panel, right click Forward Lookup Zones and select New Zone… Click Next
  3. Keep the Primary zone selected and Store the zone in Active Directory ticked.
    Click Next
  4. Leave the option To all DNS servers running on domain controllers in this domain: cfcode2016.com. Click Next
  5. Here you enter the domain name, type cfapps.com. Click Next
  6. Leave the top option selected and click Next
  7. Click Finish. You will see your new domain showing in the Forward Lookup Zones in DNS.
  8. Now right click on cfapps.com and select New Alias (CNAME) …
  9. Type * for Name
  10. Set the FQDN of the server that hosts the SharePoint sites, CFSP2016.cfcode2016.com in my case. Click OK.

    If you are using more than one server, you should be pointing to the DNS record of the web server in here. This is either the DNS A record for the web server, or the DNS record of the primary cluster address for NLB environments.

    Now if you open a command window and type in nslookup something.cfapps.com it will resolve to your SharePoint server.

Configuring SharePoint 2016 for Hosting Apps

I would recommend to copy the following powershell script and running it as a ps1 file (CreateAppService.ps1 from my one drive). Change the Change any of the variables to match your environments.

  1. On the SharePoint box, logged in as SP_Setup, from the Start Menu, type SharePoint 2016 Management Shell.
  2. Run the Script
  3. To verify the script configured SharePoint 2016 correctly open Central Administration
    1. Under Application Management click Manage Service Application.
    2. You should now have two new service application created
      1. App Management Service Application
      2. Subscription Settings Service Application
    3. Now Navigate to System Settings by clicking the link on the left menu
    4. Under Servers
      click the link Manage Services on Server.
    5. Check that the following services have started
      1. App Management Service
      2. Microsoft SharePoint Foundation Subscription Setting Service
    6. On the left hand menu, click on Apps
    7. Under App Management, click the link Configure App URLs
    8. Verify that:
      1. App Domain: cfapps.com
      2. App Prefix: app

Configuring SharePoint Server 2013 for Host-Named Site Collection and create Initial Site Collections.

Here we are going to create Host Named Site Collection (HNSC) for testing and hosting our apps. Microsoft recommends this because the Office 365 environment uses host-named site collections, new features are optimized for these site collections and they are expected to be more reliable. More can be found out directly from the technet article: http://technet.microsoft.com/en-us/library/cc424952.aspx . The only sites within your environment you should use Path Based Site Collections (PBSC) are Search Center and MySites. HNSC aren’t really needed for Search Center. The only way you can create HNSC is via powershell. So this is what we are going to do.

Register SP_Content

  1. Open SharePoint Central Administration
  2. Select Security > Configure managed Accounts.
  3. Click Register Managed Account
  4. Type Username as cfcode2016\SP_Content and the password as Pa55w0rd. Then click OK.

Create a new Web Application

Open up a PowerShell window and put the following: (change the port number if you wish) (CreateHNSC.ps1)

Configuring the Alternative Access Mapping

  • From the Start Menu
    open SharePoint 2016 Central Administration, this ensures it runs as Administrator.
  • Click Application Management, then under Web applications,
    click Configure alternative mappings.
  • On the right hand side of the screen, Change the Alternate Access Mapping Collection to point to SharePoint HNSC Web Application.
  • Click the internal URL for https://cfsp2016:11111 so that you can edit it. Change the URL protocol, host to https://hnsc.cfcode2016.com
  • Click OK.
  • Back on the Alternate Access Mapping Screen, click Add Internal URLs and add a new Internal URL for each of the following listed below. Screenshot below

Add certificates to IIS

  • In Start type IIS and open IIS Manager
  • Navigate to SharePoint HNSC Web Application and then on the right hand panel, click Bindings…
  • On the Bindings dialog, click Add…
  • In the Add Site Binding page, select https from the Type dropdown, leave the IP address as All Unassigned, the Port should say 443. Enter the Host name as hnsc.cfcode2016.com,
    and tick Require Server Name Indication then select your certificate you created earlier. Click OK
  • Add the binding for host names dev.cfcode2016.com and intranet.cfcode2016.com, ensure the Type is https, you have ticked Require Server Name Indication and you have selected your certificate.

Creating the Top level Site

Because the top-level site is an HNSC is not going to be used by anyone in the site. Therefore, this PowerShell script will create a blank site. (CreateHNSC.ps1)

  1. In PowerShell run the following script:

Site Collections

Here we are going to create a TeamSite called Intranet.cfcode2016.com and a developer site called dev.cfcode2016.com. Please note you can only create, debug and test apps using a developer site. You could type the PowerShell into notepad, save the file as PS1 and run it from SharePoint 2016 Management Shell, instead of typing each row directly. We are first going to create 2 databases, one for each Site collection. This is good practice for backups and restore purposes.

  1. From the Start Menu, type SharePoint 2016 Management Shell, and open the application. (CreateHNSC.ps1)
  2. Type

To verify that the host-name site collections are created:

  1. Open up SharePoint 2016 Central Administration
  2. Under Application Management click View all Site Collections
  3. Ensure the Web Application is pointing to the HNSC web and you should see the two site collections plus the root site.
  4. By clicking on the different site collections, you will also see that the Database Name is assigned correctly to the correct database as set up in our PowerShell script.
  5. You can also navigate in a browser to https://dev.cfcode2016.com or https://intranet.cfcode2016.com. Notice that the SSL certificate is valid.

Configuring SSL for Apps

As our App domain is on a different domain to our SharePoint domain, we should create a different SSL certificate for it.

  • Ensure you are on the SharePoint box with a Domain Admin Account. (cfcode2016\Administrator)
  • We have already configured the Certificate Authority earlier on the Domain Controller. Here we are going to request the certificate using Internet Information Services on the SharePoint Server. From the Start Menu, type IIS and open Internet Information Services (IIS) Manager
  • Once IIS opens, click on the Server Name. (CFSP2016) You will be prompted with a dialog asking to get started with Microsoft Web Platform, click do not show this message and then click No.
  • From the IIS section, double click Server Certificates

  • From the right hand side of the screen, click Create Domain Certificate

  • Complete the form for the Domain Certificate as follows (Change to match your environment if not following exactly along)
    • Common Name : *.cfapps.com
    • Organisation: CF Code
    • Organizational Unit: Computers
    • City/Locality: London
    • State/Province: London
    • Country/Region: GB

  • Click Next
  • On the Online Certification Authority enter the common name you gave your Authority Name\Server Name. (For example mine is MY-CA\CFAD.cfcode2016.com), You can also use the select button if you have configured everything correctly. You can put anything in the friendly name box, ensure it is different from your other certificate friendly name, and easy identifiable as the Apps certificate. Click Finish.

  • You should now see the certificate in the Server certificates window.

  • If there were other servers in your farm, you would need to export the .pfx file so that it can be imported into the other servers.

Configure SharePoint for Apps

We need to configure our SharePoint and IIS to use a different certificate for Apps, and also our Web Application needs to know to use our App Domain.

  1. Sign back into the SharePoint machine as SP_Setup.
  2. Run as administrator, SharePoint 2016 Management Shell
  3. Run the following PowerShell Script

  4. Next we need run the following command:
  5. In Start type IIS and open IIS Manager
  • Navigate to SharePoint HNSC Web Application and then on the right hand panel, click Bindings…
  • On the Bindings dialog, select the one without the Host Name on Port 11111 and click Edit
  • Change the Port to 443, and select the SSL Certificate as your App Certificate.
  • Click OK.

Add Apps to your Intranet Zone.

To prevent getting prompted for your login, configure the intranet zone in IE.

  • Open up Internet explorer
  • Click on the cog symbol, and select Intranet Options
  • Select the Security tab, and then click on Local Intranet. Then click on the Sites button.
  • On the Local intranet dialog, click the Advanced button.
  • Type *.cfapps.com and click Add. (You might need to untick Require server verification (https) for all sites in this zone )
  • Then click Close, OK, and OK

Checking that Apps are new set up for your farm

  1. Open up your intranet site https://intranet.cfcode2016.com
  2. At the top right of the screen click the cog icon.
  3. From the drop down, click Add an app
  4. On the App page, in the quick launch menu area, click on SharePoint Store
  5. If you have connected up correctly you will now see Apps that you can download from the SharePoint store.
  6. Pick a free one to install. I’m selecting Bright Banner. (Have no idea if it’s any good, so not endorsing, just using for testing purposes)
  7. Click Add it.
  8. Confirm that you wish to add the app. Click Continue
  9. A page will state that you have just go this app for everyone in your organization. Click Return to Site
  10. A prompt will appear, asking if you trust the application. Click Trust It.
  11. After a moment you will be returned to your Site Contents. You will also note that your app that you downloaded is currently being added to your site. Once complete the adding text will disappear.

  12. Click on the App. It will load. Take note of the URL. It is being called from the domain you created earlier cfapps.com. Congratulations you have got Apps working!

So glad I finally got Apps certificates to work. Took me a couple of attempts. Thank you to Anupam Shrivastava and his blog post http://akforsharepoint.blogspot.co.uk/2015/05/sharepoint-hosted-apps-in-aam-or-host.html for helping finally cracking it.

I recommend shutting down and taking checkpoints again.

Updating an expired Client Secret of SharePoint Add-in

Updated with Azure-AD / AZ Cli here https://cann0nf0dder.wordpress.com/2019/11/25/updating-an-expired-client-secret-of-a-sharepoint-add-in-using-azure-ad-az-cli/ 

Been working with SharePoint Add-in tokens for a while now, but this week has been the first time I’m still working with an add-in longer than a year in one environment. My application start throwing the error message:

Error:Token request failed., InnerMessage:System.Net.WebException: The remote server returned an error: (401) Unauthorized.”

I knew that they did expire after a year, but never really thought about (until now) how to go about renewing them. Luckily Microsoft documentation nowadays is a lot better than it used to be. I found this walk through https://msdn.microsoft.com/en-us/library/office/dn726681.aspx. In case Microsoft takes the link down, or changes the URL, I will explain the steps below. I have also changed the code slightly for the Report on Client ID expiry dates, as the Microsoft one didn’t return the results I expected.

Report on Client ID expiry dates.

  • Open Windows Powershell and run the following cmdlet:
  • A login prompt will appear, here enter the tenant-administrator credentials for the Office 365 tenancy where the add-in was registered with AppRegNew.aspx
  • You can generate a report that list each add-in in the tenant with the date that the secret expires with the following PowerShelll code.

    $applist = Get-MsolServicePrincipal -all  |Where-Object -FilterScript { ($_.DisplayName -notlike "*Microsoft*") -and ($_.DisplayName -notlike "autohost*") -and  ($_.ServicePrincipalNames -notlike "*localhost*") }
    $output = "";
    foreach ($appentry in $applist)
        $principalId = $appentry.AppPrincipalId
        $principalName = $appentry.DisplayName
        $results =  Get-MsolServicePrincipalCredential -AppPrincipalId $principalId -ReturnKeyValues $false | Where-Object { ($_.Type -ne "Other") -and ($_.Type -ne "Asymmetric") }
        if($results.count -gt 0)
             $output += "PrincipalId:`t$principalId`n"
             $output += "PrincipalName:`t$principalName`n"
             $output += "Keys`n" foreach($result in $results) { 
             $output += "Type:`t" + $result.Type + "`n" 
             $output += "Value:`t" + $result.Value + "`n" 
             $output += "KeyId:`t" + $result.KeyId + "`n" 
             $output += "StartDate:`t" + $result.StartDate + "`n" 
             $output += "EndDate:`t" + $result.EndDate + "`n" 
             $output += "Usage:`t" + $result.Usage+ "`n" $output += "`n"        
          $output += "-----------------------------------------------`n" } 
          $output | Out-File "c:\temp\appsec.txt" 
    • The above code first filters out Microsoft’s own applications, add-ins still under development (and a now-deprecated type of add-in that was called autohosted).
    • Filters out non-SharePoint add-ins and add-in like workflow.
  • Open the file at c:\temp\appsec.text to see the report.

An example of the report below:

Note: The PrincipalID is your Client ID

Generate a new secret for another year.

To generate a new secret, you just need to run the following PowerShell script:

$clientId = <#Replace with your ClientID of the add-in>
$bytes = New-Object Byte[] 32
$rand = [System.Security.Cryptography.RandomNumberGenerator]::Create()
$newClientSecret = [System.Convert]::ToBase64String($bytes)
New-MsolServicePrincipalCredential -AppPrincipalId $clientId -Type Symmetric -Usage Sign -Value $newClientSecret
New-MsolServicePrincipalCredential -AppPrincipalId $clientId -Type Symmetric -Usage Verify -Value $newClientSecret
New-MsolServicePrincipalCredential -AppPrincipalId $clientId -Type Password -Usage Verify -Value $newClientSecret

The output of the above PowerShell file will give you a new Client Secret, take note of this:

Generate a new secret for 3 years.

It is possible to create a secret that will last 3 years, the PowerShell script is very similar to the above script, but now it has a start and end date.

$clientId = <#Replace with your ClientID of the add-in>
$bytes = New-Object Byte[] 32
$rand = [System.Security.Cryptography.RandomNumberGenerator]::Create()
$newClientSecret = [System.Convert]::ToBase64String($bytes)
$dtStart = [System.DateTime]::Now
$dtEnd = $dtStart.AddYears(3)
New-MsolServicePrincipalCredential -AppPrincipalId $clientId -Type Symmetric -Usage Sign -Value $newClientSecret -StartDate $dtStart -EndDate $dtEnd
New-MsolServicePrincipalCredential -AppPrincipalId $clientId -Type Symmetric -Usage Verify -Value $newClientSecret -StartDate $dtStart -EndDate $dtEnd
New-MsolServicePrincipalCredential -AppPrincipalId $clientId -Type Password -Usage Verify -Value $newClientSecret -StartDate $dtStart -EndDate $dtEnd

Updating the remote web application using the new secret

  • If you are redeploying from Visual Studio you will need to update the client Secret in your Web.Config
  • If you are just updating directly in Azure, you can just go to the configurations and update the new secret there.

If you are using the default TokenHelper.cs file in your project, and it’s not the prerelease version then you can add a second app setting called SecondaryClientSecret. Here you would put in your old secret, and in the ClientSecret put in the new one. This is so if your token is going to expire soon, the application will still work as it will try one first then the other.

Time to propagate Client Secret to SharePoint

According the Microsoft link you should wait at least 24 hours to propagate the ClientSecret. However, I found as soon as I changed the secret I could use it straight away. After changing your Client Secret if you run the ‘Report on Client ID expiry dates’ powershell again, those dates didn’t update for me until the following day.

I ran the report the following day, and as you can see below, Demo App 1 which was shown in the screen shot above now has 3 new keys with new dates.

Deleting expired client secrets

As expired client secrets do not seem to get removed, it is recommended to delete them. The code below will grab any keys that are expired for a ClientID and delete them.

$clientId = <#Replace with your ClientID of the add-in>;
$keys = Get-MsolServicePrincipalCredential -AppPrincipalId $clientId -ReturnKeyValues $false
$dtNow = [System.DateTime]::Now
foreach($key in $keys)
 if($key.EndDate -lt  $dtNow)
   write-host $key.KeyId "Expired"
   Remove-MsolServicePrincipalCredential -KeyIds @($key.KeyId) -AppPrincipalId $clientId

Creating lookup and dependency columns in CSOM

What is a dependency column?

When you create a list in SharePoint this list can be used as a lookup. So for example (a very bad example, but gives you the idea), you have a customer list and an order list. On the order list you want to lookup to your Customer List. You would pick the column you want to point to for the lookup, in this case it would probably be the Customer column. However there might be other columns from the Customer list that you want to pull through to your orders list. (Customer ID, Date Joined etc). These extra columns are the dependency columns.

The customer you pick to associate with the order, these extra columns are pulled through. On the lookup list, if any of the data changes for the customer, this data will automatically be updated on the order list.

Not all columns can be used as a lookup/dependency column, only columns that can are:

  • Single Line of text
  • Number
  • Date
  • Calculated

The Demo

I have put together a demo, using a SharePoint hosted Add-In (SharePoint App). This SharePoint Add-in acts as a provisioning page to create my columns and lists, (similar to how OfficeDevPnp samples work) I will not be explaining in this post how to create a SharePoint Add-In.

The main point of the code that adds the dependant lookups uses the method AddDependentLookup which is part of the Microsoft.SharePoint.Client namespace.

public Field AddDependentLookup(string displayName, Field primaryLookupField, string lookupField ) 

I will be explaining the relevant sections of the code. All the methods have been written so that if anything already exists, it will not recreate it. My code also using OfficeDevpnp dlls, which I have obtained through NuGet. It is not until near the end after creating the Order List will I be adding the dependency columns.

Creating the Customer List

This is standard SharePoint CSOM code, I’m checking if the list already exists, if it doesn’t then I create it.

List customerList = null; 
  customerList = ctx.Web.CreateList(ListTemplateType.GenericList, "Customers", false, false); 
else { 
  customerList = ctx.Web.Lists.GetByTitle("Customers"); 

I then change the Title display value from Title to Customer Name.

var title = customerList.Fields.GetByInternalNameOrTitle("Title"); 
title.Title = "Customer Name"; 

Lastly I create 3 fields if they don’t exist, Account ID, Address, Date Joined. Once the Date Joined column is created, I’m then ensuring that the Date column is just using Date Only, instead of Date Time values.

   var accountId = new FieldCreationInformation(FieldType.Text){ 
                           AddToDefaultView = true, 
                           DisplayName = "Account ID", 
                           Id = new Guid("E99BF256-BC01-4A37-B35A-B39BCC5FB82E"), 
                           InternalName = "AccountID" 
    customerList.CreateField(accountId, true); 

   var address = new FieldCreationInformation(FieldType.Note){
                           AddToDefaultView = true, 
                           DisplayName = "Address", 
                           Id = new Guid("2A8ABC2E-B7F0-4187-ADCA-7831648AFAD3"), 
                           Group = "Lookups", 
                           Required =true, 
                           InternalName = "CustomerAddress" 
   customerList.CreateField(address, true); 

   var dateJoined = new FieldCreationInformation(FieldType.DateTime){
                             AddToDefaultView = true, 
                             DisplayName = "Date Joined", 
                             Id= new Guid("C73123E9-85C8-4156-B280-E7783EEB119C"), 
                             Group = "Lookups", 
                             Required = true, 
                             InternalName = "DateJoined" 
   customerList.CreateField(dateJoined, true); 

   var dateJoinedField = ctx.CastTo<FieldDateTime>(customerList.Fields.GetById(new Guid("C73123E9-85C8-4156-B280-E7783EEB119C"))); 

 if(dateJoinedField.DisplayFormat == DateTimeFieldFormatType.DateTime) 
  dateJoinedField.DisplayFormat = DateTimeFieldFormatType.DateOnly; 


As this is just a demo, I want to ensure that my Customer list already has some data in it ready to use. Therefore I’ve added a method that just adds some data to the Customer list.

private void CreateCustomerData(ClientContext ctx) 
  List customerList = ctx.Web.Lists.GetByTitle("Customers"); 
  if(customerList.ItemCount == 0) { 
      //Add Data. 
      Microsoft.SharePoint.Client.ListItem cust1 = customerList.AddItem(new ListItemCreationInformation()); 
      cust1["Title"] = "Customer A"; 
      cust1["AccountID"] = "A12345"; 
      cust1["CustomerAddress"] = "85 Abbott Close, \r\nLondon"; 
      cust1["DateJoined"] = new DateTime(2015, 6, 4).ToString("o"); 

      Microsoft.SharePoint.Client.ListItem cust2 = customerList.AddItem(new ListItemCreationInformation()); 
      cust2["Title"] = "Customer B"; 
      cust2["AccountID"] = "B26554"; 
      cust2["CustomerAddress"] = "745 Rose Drive, \r\nLondon"; 
      cust2["DateJoined"] = new DateTime(2014, 8, 14).ToString("o"); 
      Microsoft.SharePoint.Client.ListItem cust3 = customerList.AddItem(new ListItemCreationInformation()); 
      cust3["Title"] = "Customer C"; 
      cust3["AccountID"] = "C44575"; 
      cust3["CustomerAddress"] = "547 Cooper Way, \r\nLondon"; 
      cust3["DateJoined"] = new DateTime(2011, 1, 24).ToString("o"); 



Now we can move onto creating the Orders list. I have written this code very similar to how I started to write the Customer list, where I’m first checking if it exists first before creating it. I’m then changing the Title display value from Title to Orders. So that I have access to the Customers list columns I’m loading the list and columns.

 List orderList = null; 
 if (!ctx.Web.ListExists("Orders")) 
   orderList = ctx.Web.CreateList(ListTemplateType.GenericList, "Orders", false, false); 
   orderList = ctx.Web.Lists.GetByTitle("Orders"); 

 //Change Title 
 var title = orderList.Fields.GetByInternalNameOrTitle("Title"); 
 title.Title = "Order Item"; 

 //Get the customer list 
 List customerList = ctx.Web.Lists.GetByTitle("Customers");
 //Load Lists, fields and views ready to add more fields and lookup fields. 

I now need to add the columns to the Order list. I’m going to start with the Cost column, as this is a standard currency column. I’m checking if the column exists first before adding it to the list.

 Field cost = null; 
 if (!orderList.FieldExistsById(new Guid("70420D11-3D40-4B53-AAF4-21B57D51C033"))) 
  FieldCreationInformation orderCost = new FieldCreationInformation(FieldType.Currency) 
    DisplayName = "Cost", 
    Id = new Guid("70420D11-3D40-4B53-AAF4-21B57D51C033"), 
    AddToDefaultView = true, 
    Group = "Lookups", 
    Required = true, 
    InternalName = "Cost" 

 cost = orderList.CreateField(orderCost, true); 

This is the section where the AddDependentLookup method is being used. First I need to create the lookup column from the Customers list to the Orders list. If it already exists I need to load this column as I require the Field when I call the AddDependentLookup method. Once this is created, I will check to see if the dependency column has already been added. Unfortunately when it gets added, you don’t have control over what the GUID of the column will be. Therefore you will need to check by internal name. This name will be the title of the column that you give it encoded the way SharePoint encodes spaces, punctuation etc. If the column doesn’t exist it is then added to the Order list using the AddDependentLookup method passing in the column display name, the lookupfield, and the internal name of the dependant column within the customer list.

FieldLookup customerLookupField = null; 
if (!orderList.FieldExistsById("FE9ED460-02E7-4124-A4F0-BFE5A3DDA4D0")) 
  FieldCreationInformation customerLookup = new FieldCreationInformation(FieldType.Lookup) { 
       DisplayName = "Customer", 
       Id = new Guid("FE9ED460-02E7-4124-A4F0-BFE5A3DDA4D0"), 
       Group = "Lookups",
       Required = true, 
       AddToDefaultView = true, 
       InternalName = "CustomerLookup" 
  customerLookupField = ctx.CastTo<FieldLookup>(orderList.CreateField(customerLookup, false)); 
  customerLookupField.LookupList = customerList.Id.ToString(); 
  customerLookupField.LookupField = "Title"; 
  customerLookupField = ctx.CastTo<FieldLookup>(orderList.Fields.GetById(new Guid("FE9ED460-02E7-4124-A4F0-BFE5A3DDA4D0"))); 
 //Add Dependency fields. AccountID, DateJoined 
 Field accountDependency = null; 
 if (!orderList.FieldExistsByName("Cust_x002e__x0020_Account")) 
   accountDependency = orderList.Fields.AddDependentLookup("Cust. Account", customerLookupField, "AccountID"); 
  Field dateJoinedDependency = null; 
  if (!orderList.FieldExistsByName("Cust_x002e__x0020_Joined")) 
    dateJoinedDependency = orderList.Fields.AddDependentLookup("Cust. Joined", customerLookupField, "DateJoined"); 

After deploying my app, and loading it up, I am able to create the lookup lists.

Customer List

Creating a new Order

Orders List



Link to Visual Studio Project


AMS – App Model Samples

A good set of examples of Apps have been released by Microsoft. They can be found on CodePlex. Most will work on 365 and on Prem.

Quick list of included scenarios.

  • Cloud base site collection provisioning
  • Creating site collections remotely using SP Apps in on-premises
  • Provision and integrate Yammer group into site provisioning
  • Manage and update theme settings in host web
  • Changing host web rendering with custom CSS
  • Site policy management with CSOM
  • Wiki page manipulation using CSOM
  • Site collection enumeration
  • Setting up Apps to Windows Azure with specific roles
  • People picker implementation for provider hosted app
  • Taxonomy picker implementation for provider hosted app
  • Utilization of JavaScript injection pattern to modify the UI for end users
  • Uploading of large files to SharePoint
  • Deploy branding to personal OneDrive Pro sites automatically
  • Connect to and manipulate Office365 sites using console application
  • Provide custom template pattern for sub site creation
  • Manipulation of user profile properties remotely
  • Build sync of user profile properties in cloud
  • Taxonomy driven navigation control using JavaScript
  • Mass deployment of branding elements from console app
  • Hybrid self-service site collection provisioning – one UI for cloud and on-prem
  • Synchronization of user profile pictures from on-prem to SharePoint Online
  • Dynamic permission handling
  • Remote event receivers from host web


Cross Domain and SharePoint Hosted Apps using CSOM

Continuing on from last week blog post on Cross Domain and SharePoint Hosted Apps using REST, today I’m going to show you how to do the same Cross Domain calls but by using CSOM.

If you remember the 2 different calls I made in my project were a call back to the host domain to retrieve the title, and to retrieve all the lists in the host web. Here I’m going to extend that project and include two additional buttons that are going to do exactly the same thing, but using CSOM.

Below is a screen shot where we last left off.

If you were following along, open the solution from last week, and let us continue.

  • Open the Default.aspx page. Here we will add the two buttons to the relevant sections. Under the last button with the id=”btnStandardRestGetTitle” but before the </div> lets add the button to get Title via CSOM.
<span><input type="button" id="btnCSOMGetTitle" value="Get Title via CSOM" onclick="getTitleCSOM()" /></span>
  • While we are in the Default.aspx page, let us add the button to get the lists via CSOM as well. Under the last button with the id=”btnStandardRestGetLists” but before the </div> lets add the button to get Lists via CSOM.
<span><input type="button" id="btnCSOMGetLists" value="Get Lists via CSOM" onclick="getListsCSOM()" /></span>
  • Open up your App.js file. Let us add the two click event handlers for the buttons we just placed on default.aspx
/*Button Click Get CSOM*/
function getTitleCSOM() {
/*button Click Get List CSOM*/
function getListsCSOM() { execCSOMListRequest();}
  • Lastly we are going to add the two functions that returns the title and Lists, with their corresponding Success and Fail handlers.
//CSOM Cross Domain call to obtain HostWeb Title
function execCSOMTitleRequest() {
    var context;
    var factory;
    var appContextSite;
    var collList;
    //Get the client context of the AppWebUrl
    context = new SP.ClientContext(appwebUrl);
    //Get the ProxyWebRequestExecutorFactory
    factory = new SP.ProxyWebRequestExecutorFactory(appwebUrl);
    //Assign the factory to the client context.
    //Get the app context of the Host Web using the client context of the Application.
    appContextSite = new SP.AppContextSite(context, hostwebUrl);
    //Get the Web
    this.web = appContextSite.get_web();
    //Load Web.
        Function.createDelegate(this, successTitleHandlerCSOM),
        Function.createDelegate(this, errorTitleHandlerCSOM)
    //success Title
    function successTitleHandlerCSOM(data) {
        $('#lblResultTitle').html("<b>Via CSOM the title is:</b> " + this.web.get_title());
    //Error Title
    function errorTitleHandlerCSOM(data, errorCode, errorMessage) {
        $('#lblResultLists').html("Could not complete CSOM call: " + errorMessage);
//CSOM Cross domain call to obtain HostWeb Lists
function execCSOMListRequest(){
    var context;
    var factory;
    var appContextSite;
    var collList;
    //Get the client context of the AppWebUrl
    context = new SP.ClientContext(appwebUrl);
    //Get the ProxyWebRequestExecutorFactory
    factory = new SP.ProxyWebRequestExecutorFactory(appwebUrl);
    //Assign the factory to the client context.
    //Get the app context of the Host Web using the client context of the Application.
    appContextSite = new SP.AppContextSite(context, hostwebUrl);
    //Get the Web
    this.web = appContextSite.get_web();
    // Get the Web lists.
    collList = this.web.get_lists();
    //Load Lists.
        Function.createDelegate(this, successListHandlerCSOM),
        Function.createDelegate(this, errorListHandlerCSOM)
    //Success Lists
    function successListHandlerCSOM() {
            var listEnumerator = collList.getEnumerator();
            $('#lblResultLists').html("<b>Via CSOM the lists are:</b><br/>");
        while (listEnumerator.moveNext()) {
            var oList = listEnumerator.get_current();
            $('#lblResultLists').append(oList.get_title() + " (" + oList.get_itemCount() + ")<br/>");
    //Error Lists
    function errorListHandlerCSOM(data, errorCode, errorMessage) {
        $('#lblResultLists').html("Could not complete CSOM Call: " + errorMessage);
  • If we now run out solution, you should see all three ways of obtaining the title and lists from the host.

  • By clicking Get Title via CSOM this will return the title “Cannonfodder Development” in my case, exactly the same result as if we tried via REST.

  • By clicking Get Lists via CSOM this will return all the lists from my host Cannonfodder development, exactly the same list as if we tried via REST.

I hope these two blogs gave you a basic understanding of the different ways you can interact with the host in SharePoint 2013.

Cross Domain and SharePoint Hosted Apps using REST

When building SharePoint Apps, because they sit in a different Domain to your SharePoint website there are “blocking mechanisms” that prevents it from retrieving data from the SharePoint website. To be able to communicate, you will need to use the Cross Domain library. The cross domain library is a JavaScript file known as SP.RequesterExecutor.js, which is referenced by the SharePoint app, and hosted in the SharePoint website. You can find the SP.RequesterExecutor.js file under the “_layouts/15/” directory.

In SharePoint 2013, REST (Representational State Transfer) is a great way of communicating with SharePoint websites. You can test out REST directly in the browser. For example, in your browser type:


In the above REST call, I have requested the Web and got the Title only. If I didn’t add $select=Title I would have retrieved all information about dev.cannonfodder.local web. For more information on to how to use REST it can be found at the following Microsoft website http://msdn.microsoft.com/en-gb/library/fp142385.aspx

Cannot see feeds?

If when you did the above REST query you didn’t see the XML feed, but received the following message, just follow these steps.

  • Access Internet Options. Then on the Content tab, click Settings.

  • In the Feed and Web Slice Settings untick Turn on feed reading view. Click OK, OK and then refresh your browser with the REST query in it. You should now be seeing the data similar to my initial screenshot.

Cross Domain project

I’m going to walk you through building a SharePoint Hosted App, that will make 2 different Cross Domain calls. One will obtain the Title of the Host website, and the second will obtain all the lists in the host site.

  • Open Visual Studio and create a new App for SharePoint 2013 project.
  • Point it to your development site, and set the hosting to SharePoint Hosted.
  • First open the AppManifest.xml file, and on the Permission tab, give the scope of Web the permission of Read.
  • On your default.aspx page, within the PlaceHolderMain ContentPlaceHolder put the following.
    <h2>Please select a way to bring back the title from the host web:</h2>
    <div><input id="btnCrossDomainGetTitle" onclick="getTitleXd()" type="button" value="Get Title via REST Cross Domain" /></div>
    <h2>Please select a way to bring back all lists in the host web:</h2>
    <div><input id="btnCrossDomainGetLists" onclick="getListsXd()" type="button" value="Get Lists via REST Cross Domain" /></div>
  • Now switch to the App.js file.
  • First thing we need to do is set up the page to load the SP.RequestExecutor.js script. We also need to obtain the SPHostUrl.
    //global variables.
    var hostwebUrl
    var appwebUrl;
    var web;
    // This code runs when the DOM is ready and creates a context object which is needed to use the SharePoint object model
    /*Get the page ready*/
    $(document).ready(function () {
        hostwebUrl = decodeURIComponent(getQueryStringParameter("SPHostUrl"));
        appwebUrl = decodeURIComponent(getQueryStringParameter("SPAppWebUrl"));
        var scriptbase = hostwebUrl + "/_layouts/15/";
        $.getScript(scriptbase + "SP.RequestExecutor.js");
    function getQueryStringParameter(paramToRetrieve) {
        var params = document.URL.split("?")[1].split("&");
        var strParams = "";
        for (var i = 0; i &amp;lt; params.length; i = i + 1) {
            var singleParam = params[i].split("=");
            if (singleParam[0] == paramToRetrieve)
                return singleParam[1];
  • Now we need to add the button click handler, and the code to call and display the Title.
    /*Button Click Get Title Cross Domain*/
    function getTitleXd() {
    //Cross Domain Call to obtain HostWeb Title.
    function execCrossDomainTitleRequest() {
        var executor;
        executor = new SP.RequestExecutor(appwebUrl);
        var url = appwebUrl + "/_api/SP.AppContextSite(@target)/web/Title?@target='" + hostwebUrl + "'";
            url: url,
            method: "GET",
            headers: { "Accept": "application/json; odata=verbose" },
            success: successTitleHandlerXD,
            error: errorTitleHandlerXD
    //Success Title
    function successTitleHandlerXD(data) {
        var jsonObject = JSON.parse(data.body);
        $('#lblResultTitle').html"<b>Via Cross Domain the title is:</b> " + jsonObject.d.Title);
    //Error with Title.
    function errorTitleHandlerXD(data, errorCode, errorMessage) {
        $('#lblResultTitle').html("Could not complete cross-domain call: " + errorMessage);
  • Now let’s add the button click handler and the code to display the lists from the host web.
/*Button Click Get Lists Cross Domain*/
function getListsXd() { execCrossDomainListRequest(); }
//Cross Domain Call to obtain Host Web Lists
function execCrossDomainListRequest() {
    var executor;
    executor = new SP.RequestExecutor(appwebUrl);
    var url = appwebUrl + "/_api/SP.AppContextSite(@target)/web/lists?@target='" + hostwebUrl + "'";
        url: url,
        method: "GET",
        headers: {"Accept" : "application/json; odata=verbose"},
        success: successListHandlerXD,
        error: errorListHandlerXD
//Success Lists
function successListHandlerXD(data){
    var jsonObject = JSON.parse(data.body);
    //Get LIsts
    var lists = jsonObject.d.results;
    $('#lblResultLists').html("<"b>Via Cross Domain the lists are:</b>");
    //Loop through each item adding to the label.
    var listsHtml = $.each(lists, function(index, list){
        $('#lblResultLists').append(list.Title + " (" + list.ItemCount + ")
//Error Lists
function errorListHandlerXD(data, errorCode, errorMessage){
    $('#lblResultLists').html("Could not complete cross-domain call: " + errorMessage);
  • Now let’s run the app and see it working.

Cross domain call without using SP.RequestExecutor.js

Now during my learning journey of Cross Domain, I discovered that I can still call into my host site, using a REST URL based on the App web URL. I don’t believe this is a supported way, as I haven’t found anything on Microsoft sites that confirm this, however I have got this working On-Prem and Autohosted in 365.

  • Back on our app, open up the Default.aspx file again.
  • Add the two following buttons, put the btnStandardRestGetTitle after the btnCrossDomainGetTitle button, but before the </div>. And put the btnStandardRestGetLists after the btnCrossDomainGetLists, but before the </div>

<input id="btnStandardRestGetTitle" onclick="getTitle()" type="button" value="Get Title standard REST" />

<input id="btnStandardRestGetLists" onclick="getLists()" type="button" value="Get Lists standard REST" />

  • Back in the App.js file. First we need to add a function that gets the actual path of the application Url.
//Obtains the path upto the actual application. E.g. http://app123.app.code/SubSite/CrossDomainApp
//gets http://app123.app.com/SubSite
function getUrlPath() {
    var webRel = _spPageContextInfo.webAbsoluteUrl;
    var lastIndex = webRel.lastIndexOf('/');
    var urlpath = webRel.substring(0, lastIndex);
    return urlpath;
  • Now we need to add the button click handler, and the code to call and display the Title.
/*Button Click Get Title Rest*/
function getTitle() {
//REST Call to obtain HostWeb Title
function execRESTTitleRequest() {
    var url = getUrlPath() + "/_api/web/?$select=Title";
        url: url,
        method: "GET",
        headers: { "Accept": "application/json; odata=verbose" },
        success: successTitleHandler,
        error: errorTitleHandler
//Success Title
function successTitleHandler(data) {
    $('#lblResultTitle').html("<b>Via REST the title is:</b>" + data.d.Title);
//Error Title
function errorTitleHandler(data, errorCode, errorMessage) {
    $('#lblResultTitle').html("Could not complete REST call: " + errorMessage);
  • Now let’s add the button click handler and the code to display the lists from the host web.
/*Button Click Get List Rest*/
function getLists() { execRESTListRequest(); }
//REST Call to obtain HostWeb Lists
function execRESTListRequest() {
    var url = getUrlPath() + "/_api/web/Lists";
        url: url,
        method: "GET",
        headers: { "Accept": "application/json; odata=verbose" },
        success: successListHandler,
        error: errorListHandler
//Success List
function successListHandler(data) {
    var lists = data.d.results;
    $('#lblResultLists').html("<b>Via REST the lists are:</b><br/>");
    var listsHtml = $.each(lists, function (index, list) {
        $('#lblResultLists').append(list.Title + " (" + list.ItemCount + ")<br/>");
//Error Lists
function errorListHandler(data, errorCode, errorMessage) {
    $('#lblResultLists').html("Could not complete REST call: " + errorMessage);
  • Now let’s run the App

You have learnt two ways of doing a Cross Domain call using REST. One way is the correct way recommend by Microsoft. The other way is a way I found, but I’m unsure if this is a valid way. It works and sometimes that’s all that matters.