Azure Blob Storage and managing concurrency


There are different ways you can manage concurrency within a blob storage.

  1. Blob leases work where a process will obtain a lease/lock on a file, that process will be able to update that file. Any other process will not be able to access that file to update, until the lease has been released and the process can acquire its only lease. This is known as Pessimistic concurrency.
  2. If no lease is used, and a process updates a file, it checks to see if the data has changed since it last obtained the file, if the file has been changed since then it prevents the process from updating the file. This is known as Optimistic concurrency.
  3. Ignoring concurrency completely, a process could just update the file, basically last writer wins. This is best used when there is no chance that multiple users will access the same data at the same time.

Optimistic concurrency

Every blob that is added to the storage is assigned an ETag. Every time the blob changes, the ETag will also change. It is this ETag that is used to calculate if the blob has changed since the process has last read it. The steps are:

  • Read the Blob file, and grab the ETag header value.
  • Update the Blob file, passing in the ETag value from reading the blob file.
  • The ETag passed in, matches the ETag of the blob currently in the storage. The file is updated, and a new ETag assigned.
  • If the ETag passed in, doesn’t match the ETag of the blob currently in the storage, due to another process changes it, then an HttpStatusCode of 412 (PreconditionFailed) is returned and the file isn’t updated.

The below code (taken partly from Managing Concurrency using Azure Storage – Sample Application) shows an example of optimistic concurrency. It also shows how to ignore concurrency completely to simulate a different process updating the blob. You will need to following NuGet Packages installed for the code to work:

  • Microsoft.WindowsAzure.ConfigurationManager
  • WindowsAzure.Storage
public static void DemonstrateOptimisticConcurrencyUsingBlob()
{
    Console.WriteLine("Demo - Demonstrate optimistic concurrency using a blob");

    CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
    String blobStorageContainerRef = System.Configuration.ConfigurationManager.AppSettings["BlobStorageContainerReference"];
    String filename = "Optimistic.txt";

    CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
    var container = blobClient.GetContainerReference(blobStorageContainerRef);
    container.CreateIfNotExists();

    // Create test blob - default strategy is last writer wins - so UploadText will overwrite existing blob if present
    CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
    blockBlob.UploadText("Hello this is my text");

    // Retrieve the ETag from the newly created blob
    // Etag is already populated as UploadText should cause a PUT Blob call to storage blob service which returns the etag in response.
   string orignalETag = blockBlob.Properties.ETag;
   Console.WriteLine("Blob added. Orignal ETag = {0}", orignalETag);

   // This code simulates an update by different process.
   string helloText = "Blob updated simulates a different process.";

   // No etag, provided so orignal blob is overwritten (generating a new etag)
   blockBlob.UploadText(helloText);

   Console.WriteLine("Blob updated. Updated ETag = {0}", blockBlob.Properties.ETag);

   // update the blob using the orignal ETag provided when the blob was created
   try
   {
      Console.WriteLine("Trying to update blob using orignal etag to generate if-match access condition");
      blockBlob.UploadText(helloText, accessCondition: AccessCondition.GenerateIfMatchCondition(orignalETag));
    }
    catch (StorageException ex)
    {
       if (ex.RequestInformation.HttpStatusCode == (int)System.Net.HttpStatusCode.PreconditionFailed)
       {
          Console.WriteLine("Precondition failure as expected. Blob's orignal etag no longer matches");
       }
       else
       {
           throw;
       }
    }

   Console.WriteLine("Press enter to exit");
   Console.ReadLine();
}

My app.config file for my console application has the following in it for the above to work.

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
   <startup> 
       <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />
   </startup>
   <appSettings>
     <add key="BlobStorageContainerReference" value="leaseobjects"/>
     <add key="StorageConnectionString" value="[Insert your own DefaultEndpointsProtocol to your Azure Storage] "/>
   </appSettings>
</configuration>

The results from running the DemonstrateOptimisticConcurrencyUsingBlob() method.

Pessimistic concurrency

If you assign a lease on a blob for exclusive use, you can say how long you need the lease for. This lease is from 15 seconds to 60 seconds, or infinite which will lock the blob until the code releases the lease. A lease can be renewed and broken (generally used by administrators to reset the lease state). The steps to acquire a lock is:

  • Get the blob
  • Acquire a lease
  • Update the blob
  • Release the blob.
  • If a different process try to acquire a lease when the blob is already leased, then an HttpStatusCode of 409 (Conflict) is returned.
  • If you try to update the blob that has a lease on it, without the lease id, then an HttpStatusCode of 412 (PreconditionFailed) is returned.

The below code (taken partly from Managing Concurrency using Azure Storage – Sample Application) shows an example of optimistic concurrency.

private static void DemonstratePessimisticConcurrencyUsingBlob()
{
   Console.WriteLine("Demo - Demonstrate pessimistic concurrency using a blob");
   CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("StorageConnectionString"));
   String blobStorageContainerRef = System.Configuration.ConfigurationManager.AppSettings["BlobStorageContainerReference"];
   String filename = "Optimistic.txt";

   CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
   var container = blobClient.GetContainerReference(blobStorageContainerRef);
   container.CreateIfNotExists();

   //Create test blob - default strategy is last writer wins - so UploadText will overwrite existing blob if present
   CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
   blockBlob.UploadText("Hello this is my text");

   // Acquire lease for 15 seconds
   string lease = blockBlob.AcquireLease(TimeSpan.FromSeconds(15), null);
   Console.WriteLine("Blob lease acquired. Lease = {0}", lease);

   // Update blob using lease. This operation will succeed
   string helloText = "Blob updated";
   var accessCondition = AccessCondition.GenerateLeaseCondition(lease);
   blockBlob.UploadText(helloText, accessCondition: accessCondition);

   Console.WriteLine("Blob updated using an exclusive lease");
   //Simulate third party update to blob without lease

   try
   {
       // Below operation will fail as no valid lease provided
       Console.WriteLine("Trying to update blob without valid lease");
       blockBlob.UploadText("Update without lease, will fail");
    }
    catch (StorageException ex)
    {
        if (ex.RequestInformation.HttpStatusCode == (int)System.Net.HttpStatusCode.PreconditionFailed)
       {
          Console.WriteLine("Precondition failure as expected. Blob's lease does not match");
       }
       else
       {
          throw;
       }
    }

    //Simulate another process trying to obtain a new lease
    try
    {
        Console.WriteLine("Trying to obtain a new lease when previous lease still exists.");
        string newLease = blockBlob.AcquireLease(TimeSpan.FromSeconds(15), null);
    }
    catch(StorageException ex)
    {
        if (ex.RequestInformation.HttpStatusCode == (int)System.Net.HttpStatusCode.Conflict)
        {
            Console.WriteLine("Conflict. Another process already has this blob lease");
        }
        else
        {
            throw;
        }
     }

     // Release the blob. Lease expires anyways after 15 seconds
     blockBlob.ReleaseLease(accessCondition);
     Console.WriteLine("Press enter to exit");
     Console.ReadLine();
}

The results from running DemonstratePessimisticConcurrencyUsingBlob()

If you want to see the blob file you could use visual studio. In the Server explorer, if you sign into your Azure account, under storage you should see the blob you created. Mine is called leaseobjects, which I defined in my app.config “BlobStorageContainerReference”.

Within our blob container, we have the two files we created in our application.

References: https://azure.microsoft.com/en-gb/blog/managing-concurrency-in-microsoft-azure-storage-2/

http://justazure.com/azure-blob-storage-part-8-blob-leases/

You can download my code from here.

Code: http://1drv.ms/1LNvvET

Advertisements