Flexibility is great until it isn’t… Careful with these Azure Tables!!

Azure, Azure Storage Explorer, Azure Tables, Technical stuff, Work
5 Comments

As part of my ongoing discovery of Azure features and services, I am working on a few applications and samples that use various features of Azure such as Azure Functions, Azure Tables and more. Recently I had a bug in one of my applications, a link shortener that takes in a link in the form http://gslb.ch/5t and returns another longer link, in this case https://expertday.forxamarin.com.

The application uses an HTTP Module that detects the short domain gslb.ch and performs the table lookup. This is straightforward enough. Where things become interesting is that by deploying this solution on Azure and switching Application Insights on, I get some feedback on who is clicking what. For instance, the short link above has a few possible variations. http://gslb.ch/5t is used for Twitter, as indicated by the trailing T. I could also use http://gslb.ch/5f for Facebook, http://gslb.ch/5l for LinkedIn, you get the idea. It allows me to see where the majority of my community is active, which is interesting information.

Using analytics

As part of the service, I also have the possibility to add some analytics information for the destination system. For some Microsoft links, we use analytics like the following:

“?WT_mc_id=redshirtdevtour-twitter-lbugnion”

This type of analytics is used for the Red Shirt Tour taking place this Fall, where Scott Guthrie is visiting 5 locations in the USA and talking about Azure with a LOT of demos. If you haven’t done so yet, you should really check it out, some great content for free, and a chance to meet a lot of the Cloud Developer Advocates, including myself in Boston and NYC. Sometimes I want to use analytics, and sometimes I don’t, so I have foreseen a boolean column for this in the database, as shown below.

Using Azure Tables for storage

I found one small issue that cost me a few head scratches. You see, I use Azure Tables for the storage of the link information. This is quick to use and super easy to maintain. As time goes, I might port it to CosmosDB but for now, this works well.

From the Azure Tables page: “A NoSQL key-value store for rapid development using massive semi-structured datasets”. This is a very flexible, schema-less system. It means that you can easily add new entries in a table, and create new properties on the fly, for example with the Microsoft Azure Storage Explorer which I documented earlier. In my case, here is a screenshot of the table:

2017-10-01_11-05-22
(Click for full size)

As you can see, the last column is titled SkipAnalytics. If true, the analytics portion described above is omitted from the long link. Obviously I made this value a boolean as shown in the class code:

public class ShortenedLinkEntity : TableEntity
{
    public string LongLink { get; set; }
    public string Alias { get; set; }
    public string Channel { get; set; }
    public string EventName { get; set; }
    public bool SkipAnalytics { get; set; }
    public string LinkAlias { get; set; }
}

The entities are retrieved with the following code. In this example, I hardcoded the index “5”, which corresponds to the RowKey column in the table shown above.

var account = CloudStorageAccount.Parse(Constants.ConnectionString);
var tableClient = account.CreateCloudTableClient();
var linksTable = tableClient.GetTableReference(LinksTableName);
await linksTable.CreateIfNotExistsAsync();

var retrieveOperation = TableOperation.Retrieve<ShortenedLinkEntity>(
    "partition",
    "5");
var operation = await linksTable.ExecuteAsync(retrieveOperation);
var link = operation.Result as ShortenedLinkEntity;
return link;

The code above starts by creating a CloudStorageAccount corresponding to the connection string I obtained from the Azure Storage Explorer as shown here (Primary Connection String):

2017-10-01_11-20-54

Then I create a CloudTableClient which I use to retrieve the CloudTable for the short links. To retrieve the entity itself, I use here the TableOperation.Retrieve method, which is very fast and convenient if you happen to know the PartitionKey and the RowKey for a given entity. Since in my code the RowKey is the index passed in the URL (here “5”), it is easy.

That sounds great, so where’s the issue?

Here is the bug: I wanted to add a new row. Eventually I will have a client for this (planning to develop it with Xamarin, of course, so I can use it on Windows, iOS, Android) but right now I add rows directly in the Azure Storage Explorer. Let’s see if you can spot the issue.

First I add a new entity in the table. Notice that I don’t enter a value for the SkipAnalytics column, this is an error that I will fix later.

2017-10-01_11-25-29

2017-10-01_11-26-21

Then I see ooops I forgot the SkipAnalytics column, let’s correct that now by selecting the row and clicking the Edit button.

2017-10-01_11-32-01

Note that the empty columns are missing from this dialog. This is because as I explained, the entities stored in Azure Tables can take any shape, there is no strict schema. Freedom is great but in that case it will cause a small issue. Let’s add the missing SkipAnalytics column by pressing the Add Property button.

2017-10-01_11-33-47

Looking good right? Now if I press Update, the entity is updated in the Table. I can then run the code to retrieve the entity in my web application and… SkipAnalytics is false. Why???

Did you spot the issue?

Let’s go back to the place where I edited the entity.

2017-10-01_11-33-48

Wait… how can the column SkipAnalytics be a string? All the other rows use Boolean for this value in the table!! Well here you go, this is the issue. Because of the added freedom, I can have an object with a SkipAnalytics value of type bool, and another object in the same table where the SkipAnalytics value is of type string. When I retrieve the value in a strongly typed language like C#, I don’t get an error, but the value “true” maps to the Boolean false. Ugh…

Unfortunately, (1) the Azure Explorer doesn’t perform a check when you add an entity, (2) the Azure Table doesn’t have a problem storing what we would consider incompatible values and (3) there is no exception when the entity is retrieved and the values are mapped to the corresponding entity. This creates a bug in the application that can be quite difficult to understand.

What now?

For now, I didn’t ask the team yet why this is possible. I’d love to know if this is by design or if they would consider it a flaw of the system. What do you guys think? Personally I would be happier if the Azure Storage Explorer would prevent me from doing that. Maybe an update in the future? If and when I get a reply from the team, I will update this post with what I learn.

Happy coding
Laurent

GalaSoft Laurent Bugnion
Laurent Bugnion (GalaSoft)

Share on Facebook

5 Responses to “Flexibility is great until it isn’t… Careful with these Azure Tables!!”

  1. David Says:

    I don’t think that this should be considered as a bug, because, as you said, there is not constraint on the schema. There is no harm to have the same column with different type depending the row, because it could be totally different data stored in this column and even a different TableEntity implementation for each row.

  2. lbugnion Says:

    Yes i am also reluctant to call it a bug. But i feel that the user experience in the tool should be better. It’s in this spirit that i approached the team

  3. David Says:

    Agree.
    But your call for carefulness is good, because somebody not used to schema-less tables could easily do this this kind of mistake. Thank you for that :)

  4. Nadav Says:

    I don’t think the behaviour of the Azure explorer is a bug, my problem is with the c# code that casts string value to bool property without throwing an exception….

  5. lbugnion Says:

    IMHO the fact that the explorer lets you close the window without a value for a Bool property is bad.If they were requiring a value (and they know the type at that moment, notice the dropdown) or a least setting that to false, we wouldn’t have this issue

Leave a Reply