Azure Service Bus 1.5

DCC 2011.2 Lap Around Azure Service Bus: The Goods

Rick G. Garibay — Sat, 05 Nov 2011 21:33:25 GMT

Thanks to all that came out to my “Lap Around Azure Service Bus Brokered Messaging” talk at Desert Code Camp today.

We covered a ton of content including things a few folks didn’t know about relay messaging capabilities in Azure Service Bus and demonstrated how simple it is to expose a REST or SOAP endpoint from behind the firewall. I also demonstrated the brand new load balancing capabilities that were just released last week.

From there, we dove deep into the .NET API to walk through how to provision queues and topics from code and start messaging within minutes by simply grabbing the Azure Service Bus NuGet package and writing a few lines of code.

Next, we explored the REST API, and how simple it is for any HTTP client, regardless of platform to take advantage of the robust messaging capabilities that Azure Service Bus queues and topics have to offer.

Last but not least, we wrapped up with a quick walkthrough of the NeMessagingBinding and how simple it is to send and receive messages over a queue using the familiar WCF programming model.

I hope that each of you will unlock new possibilities with the power that these hybrid messaging capabilities have to offer.

I’d also like to thank Pluralsight for sponsoring my session. The quiz is now up for the first 5 smartest attendees.

Search for hashtags #dcc11 #Azure #ServiceBus #Q1 to #Q5.

Good Luck!

Code Demos

Happy Messaging!

Azure Service Bus Relay Load Balancing Demo

Rick G. Garibay — Mon, 31 Oct 2011 05:30:00 GMT

This week, the highly anticipated support for load balancing relay endpoints was released to the Azure Service Bus messaging fabric as first hinted by Velery Mizonov and later officially announced by Avkash Chauhan.

Richard Seroter and Sam Vanhoutte posted some very helpful walkthroughs of the functionality, so I thought I would follow up with a quick webcast to show Azure Service Bus Relay load balancing in action.

In the demo, I show starting 3 relay endpoints using the NetTcpRelayBinding with no address already in use exceptions and firing a series of messages over the bus which arrive at one of the 3 endpoints.

This is a feature that many customers and the community has been clamoring for for some time and it is great to see this live!

New Article in CODE Magazine on Azure Service Bus Queues & Topics

Rick G. Garibay — Wed, 26 Oct 2011 17:20:03 GMT

I am pleased to share that my new article on Azure Service Bus Queues and Topics has just been published by CODE Magazine.

CODE Magazine is a leading Microsoft technical journal with a worldwide in-print circulation in excess of 20,000 along with on-line, Xiine and Amazon Kindle media distribution. CODE is distributed to a combination of paid subscriptions, qualified requests, and newsstand sales. In addition, CODE Magazine has bonus distribution at targeted Microsoft-oriented conferences and targeted industry events throughout the year such as Microsoft Professional Developer Conference (PDC), Tech Ed, DevTeach, MVP Global Summit, DevConnections, Devscovery, QCon, Code Camps, and more!

Here is the link to the article: http://www.code-magazine.com/article.aspx?quickid=1112041

The article covers critical capabilities provided by Azure Service Bus for composing distributed messaging solutions for the hybrid enterprise and how the latest release delivers on the Software + Services vision that was first laid out over five years ago.

The new release includes the addition of Queues and Topics which build on top of an already robust set of capabilities introducing new levels of reliability for building loosely coupled distributed solutions across a variety of clients and services, be they on-premise, in the cloud, or a combination of the two.

There are many exciting changes happening within Microsoft around integration and middleware, and the release of Service Bus Brokered Messaging/Queues and Topics is a strong reflection of the commitment to the platform that I believe is going to make this new wave of innovation more exciting than ever before.

It has been a tremendous privilege to have the opportunity to work with the Azure Service Bus team and experiment with the early bits ahead of release. I’d like to thank Todd Holmquist-Sutherland, Clemens Vasters, Abhishek Lal and David Ingham for the unprecedented access to their team, resources and information as well as kindly and patiently answering my many questions over the last several weeks.

Long live messaging!

Azure AppFabric Service Bus Brokered Messaging GA & Rude CTP Diffs

Rick G. Garibay — Wed, 14 Sep 2011 16:46:52 GMT

Today at the Build conference in Anaheim California, Satya Nadella, President Server and Tools business announced general availability of the production release of AppFabric Queues and Topics, otherwise known as Brokered Messaging.

Brokered Messaging introduces durable queue capabilities and rich, durable pub-sub with topics and subscriptions that compliment the existing Relayed Messaging capabilities.

I covered Brokered Messaging following the May CTP release of Queues and followed up shortly with an overview and exploration of Topics (please see some other great resources at the end of this post).

Since then, there was a June CTP release which included the new AppFabric Application and no visible changes to Brokered Messaging, however since its release, the AppFabric Messaging team has been hard at work refining the API and behaviors based on feedback from Advisors, MVPs and the community at large.

Since I’ve already covered Queues and Topics in the aforementioned posts, I’ll dive right in to some terse examples which demonstrate the API changes. Though not an exhaustive review of all of the changes, I’ve covered the types that your most likely to come across and will cover Queues, Topics and Subscriptions extensively in my upcoming article in CODE Magazine which will also include more in-depth walk-throughs of the .NET Client API, REST API and WCF scenarios.

Those of you who have worked with the CTPs will find some subtle and not so subtle changes, but all in all I think all of the refinements are for the best and I think you’ll appreciate them as I have. For those new to Azure AppFabric Service Bus Brokered Messaging, you’ll benefit most from reading my first two posts based on the May CTP (or any of the resources at the end of this post) to get an idea of the why behind queues and topics and then come back here to explore the what and how.

A Quick Note on Versioning

In the CTPs that preceded the release of the new Azure AppFabric Service Bus features, a temporary assembly called “Microsoft.ServiceBus.Messaging.dll” was added to serve a container for new features and deltas that were introduced during the development cycle. The final release includes a single assembly called “Microsoft.ServiceBus.dll” which contains all of the existing relay capabilities that you’re already familiar with as well as the addition of support for queues and topics. If you are upgrading from the CTPs, you’ll want to get ahold of the new Microsoft.ServiceBus.dll version 1.5 which includes everything plus the new queue and topic features.

The new 1.5 version of the Microsoft.ServiceBus.dll assembly targets the .NET 4.0 framework. Customers using .NET 3.5 can continue using the existing Microsoft.ServiceBus.dll assembly (version 1.0.1123.2) for leveraging the relay capabilities, but must upgrade to .NET 4.0 to take advantage of the latest features presented here.

.NET Client API

Queues

May/June CTP	General Availability	Comments
ServiceBusNamespaceClientSettings	NamespaceManagerSettings	New class for encapsulating endpoint and security settings.
N/A	TokenProvider	New class for acquiring a WRAP token from ACS.
ServiceBusNamespaceClient	NamespaceManager	Root management object for creating Queues, Topics, Subscriptions.
Queue	QueueDescription	In May/June CTP, Topic / Queue / Subscription were management objects. All create/delete operations were moved to NamespaceManager and the state operations are now on TopicDescription/QueueDescription etc.
MessagingFactorySettings	MessagingFactorySettings	New class for encapsulating security settings.
MessagingFactory	MessagingFactory
BrokeredMessage	BrokeredMessage	No longer a factory. Simply instantiate a BrokeredMessage.
MessageSender	MessageSender	Optional, for use when you want to abstract away queue or topic.
MessageReceiver	MessageReceiver	Optional, for use when you want to abstract away queue or topic.

Below is a representative sample for creating, configuring, sending and receiving a message on a queue:

Administrative Operations

  1: 
  2:             // Configure and create NamespaceManager for performing administrative operations
  3:             NamespaceManagerSettings settings = new NamespaceManagerSettings();
  4:             TokenProvider tokenProvider = settings.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuer,key);
  5:             
  6:             NamespaceManager manager = new NamespaceManager(ServiceBusEnvironment.CreateServiceUri("sb", serviceNamespace, string.Empty), settings);
  7: 
  8:             // Check for existence of queues on the fabric
  9:             var qs = manager.GetQueues();
 10: 
 11:             var result = from q in qs
 12:                          where q.Path.Equals(queueName, StringComparison.OrdinalIgnoreCase)
 13:                          select q;
 14: 
 15:             if (result.Count() == 0)
 16:             {
 17:                 Console.WriteLine("Queue does not exist");
 18: 
 19:                 // Create Queue
 20:                 Console.WriteLine("Creating Queue...");
 21: 
 22:                 manager.CreateQueue(new QueueDescription(queueName) { LockDuration = TimeSpan.FromSeconds(5.0d) });
 23:                 
 24:             }

Runtime Operations

  1:             // Create and Configure Messaging Factory to provision QueueClient
  2:             MessagingFactorySettings messagingFactorySettings = new MessagingFactorySettings();
  3:             messagingFactorySettings.TokenProvider = settings.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuer, key);
  4:             MessagingFactory messagingFactory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb", serviceNamespace, string.Empty), messagingFactorySettings);
  5:             
  6:             QueueClient queueClient = messagingFactory.CreateQueueClient(queueName, ReceiveMode.PeekLock);
  7: 
  8:             Order order = new Order();
  9:             order.OrderId = 42;
 10:             order.Products.Add("Kinect", 70.50M);
 11:             order.Products.Add("XBOX 360", 199.99M);
 12:             order.Total = order.Products["Kinect"] + order.Products["XBOX 360"];
 13: 
 14:             // Create a Brokered Message from the Order object
 15:             BrokeredMessage msg = new BrokeredMessage(order);
 16: 
 17:             /***********************
 18:             *** Send Operations  ***
 19:             ************************/
 20: 
 21:             queueClient.Send(msg);
 22: 
 23:             /**************************
 24:              *** Receive Operations ***
 25:             ***************************/
 26:             
 27:             BrokeredMessage recdMsg;
 28:             Order recdOrder;
 29: 
 30:             // Receive and lock message
 31:             recdMsg = queueClient.Receive();
 32: 
 33:             if(recdMsg != null)
 34:             {
 35:                 // Convert from BrokeredMessage to native Order
 36:                 recdOrder = recdMsg.GetBody<Order>();
 37: 
 38:                 Console.ForegroundColor = ConsoleColor.Green;
 39:                 Console.WriteLine("Received Order {0} \n\t with Message Id {1} \n\t and Lock Token:{2} \n\t from {3} \n\t with total of ${4}", recdOrder.OrderId, recdMsg.MessageId, recdMsg.LockToken, "Receiver 1", recdOrder.Total);
 40:                 recdMsg.Complete();
 41:             }
 42:             queueClient.Close();

Note that MessageSender and MessageReceiver are now optional. Here’s an example that shows PeekLocking a message, simulating an exception and trying again:

  1: 
  2:             // Alternate receive approach using agnostic MessageReceiver
  3:             MessageReceiver receiver = messagingFactory.CreateMessageReceiver(queueName);            // Recieve, complete, and delete message from the fabric
  4: 
  5:             try
  6:             {
  7:                 // Receive and lock message
  8:                 recdMsg = receiver.Receive();
  9: 
 10:                 // Convert from BrokeredMessage to native Order
 11:                 recdOrder = recdMsg.GetBody<Order>();
 12: 
 13:                 // Complete read, release and delete message from the fabric
 14:                 receiver.Complete(recdMsg.LockToken);
 15: 
 16:                 Console.ForegroundColor = ConsoleColor.Green;
 17:                 Console.WriteLine("Received Order {0} \n\t with Message Id {1} \n\t and Lock Token:{2} \n\t from {3} \n\t with total of ${4} \n\t at {5}", recdOrder.OrderId, recdMsg.MessageId, recdMsg.LockToken, "Receiver 2", recdOrder.Total, DateTime.Now.Hour + ":" + DateTime.Now.Minute + ":" + DateTime.Now.Second);
 18:             }
 19:             catch
 20:             {
 21:                 // Should processing fail, release the lock from the fabric and make message available for later processing.
 22:                 if (recdMsg != null)
 23:                 {
 24:                     receiver.Abandon(recdMsg.LockToken);
 25:                     
 26:                     Console.ForegroundColor = ConsoleColor.Red;
 27:                     Console.WriteLine("Message could not be processed.");
 28: 
 29:                 }
 30:             }
 31:             finally
 32:             {
 33:                 receiver.Close();
 34:             }

As shown below, this sample results in order 42 being received by the QueueClient:

Topics

May/June CTP	General Availability	Comments
Topic	TopicDescription	In May/June CTP, Topic / Queue / Subscription were management objects. All create/delete operations were moved to NamespaceManager and the state operations are now on TopicDescription/QueueDescription etc.
TopicClient	TopicClient	As noted in the Queues section, you can use TopicClient or MessageSender in the event you want to abstract details of using Topics.
SubscriptionClient	SubscriptionClient	As noted in the Queues section, you can use SubscriptionClient or MessageReceiver in the event you want to abstract details of using a Topic/Subscription.
Subscription	SubscriptionDescription	Changes to constructors and use of properties (see code samples below), but intent is the same.
RuleDescription	RuleDescription	Changes to constructors and use of properties (see code samples below), but intent is the same.
FilterExpression	Filter	Base for Filter types such as SqlFilter
SqlFilterExpression	SqlFilter
FilterAction	RuleAction

Below is a representative sample for creating, configuring, sending and receiving a message on a topic:

Administrative Operations

  1:             // Configure and create NamespaceManager for performing administrative operations
  2:             NamespaceManagerSettings settings = new NamespaceManagerSettings();
  3:             settings.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuer, key);
  4:             NamespaceManager manager = new NamespaceManager(ServiceBusEnvironment.CreateServiceUri("sb", serviceNamespace, string.Empty), settings);
  5: 
  6:             // Check for existence of topics on the fabric
  7:             var topics = manager.GetTopics();
  8: 
  9:             var result = from t in topics
 10:                          where t.Path.Equals(topicName, StringComparison.OrdinalIgnoreCase)
 11:                          select t;
 12: 
 13:             if (result.Count() == 0)
 14:             {
 15:                 Console.WriteLine("Topic does not exist");
 16: 
 17:                 // Create Queue
 18:                 Console.WriteLine("Creating Topic...");
 19: 
 20:                 TopicDescription topic = manager.CreateTopic(topicName);
 21:             }
 22: 
 23:             // Create Subscriptions for InventoryServiceSubscription and CreditServiceSubscription and associate to OrdersTopic:
 24:             SubscriptionDescription inventoryServiceSubscription = new SubscriptionDescription(topicName, "InventoryServiceSubscription");
 25:             SubscriptionDescription creditServiceSubscription = new SubscriptionDescription(topicName, "CreditServiceSubscription");
 26: 
 27: 
 28:             // Set up Filters for NorthAmericaFulfillmentServiceSubscription
 29:             RuleDescription northAmericafulfillmentRuleDescription = new RuleDescription();
 30:             northAmericafulfillmentRuleDescription.Filter = new SqlFilter("CountryOfOrigin = 'USA' OR CountryOfOrigin ='Canada' OR CountryOfOrgin ='Mexico'");
 31:             northAmericafulfillmentRuleDescription.Action = new SqlRuleAction("set FulfillmentRegion='North America'");
 32: 
 33: 
 34:             // Create Subscriptions
 35:             SubscriptionDescription northAmericaFulfillmentServiceSubscription = new SubscriptionDescription(topicName, "NorthAmericaFulfillmentServiceSubscription");
 36: 
 37:             // Delete existing subscriptions
 38:             try { manager.DeleteSubscription(topicName, inventoryServiceSubscription.Name); } catch { };
 39:             try { manager.DeleteSubscription(topicName, creditServiceSubscription.Name); } catch { };
 40:             try { manager.DeleteSubscription(topicName, northAmericaFulfillmentServiceSubscription.Name); } catch { };
 41: 
 42:             
 43:             // Add Subscriptions and Rules to Topic
 44:             manager.CreateSubscription(inventoryServiceSubscription);
 45:             manager.CreateSubscription(creditServiceSubscription);
 46:             manager.CreateSubscription(northAmericaFulfillmentServiceSubscription, northAmericafulfillmentRuleDescription);
 47:

Runtime Operations

  1:             // Create and Configure Messaging Factory to provision TopicClient
  2:             MessagingFactorySettings runtimeSettings = new MessagingFactorySettings();
  3:             runtimeSettings.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuer, key);
  4:             MessagingFactory messagingFactory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb",serviceNamespace,String.Empty),runtimeSettings);
  5:             
  6:             // Create Topic Client for sending messages to the Topic:
  7:             TopicClient client = messagingFactory.CreateTopicClient(topicName);
  8:          
  9:             /***********************
 10:              *** Send Operations ***
 11:              ***********************/
 12: 
 13:             // Prepare BrokeredMessage and corresponding properties
 14:             Order order = new Order();
 15:             order.OrderId = 42;
 16:             order.Products.Add("Kinect", 70.50M);
 17:             order.Products.Add("XBOX 360", 199.99M);
 18:             order.Total = order.Products["Kinect"] + order.Products["XBOX 360"];
 19: 
 20:             // Set the body to the Order data contract
 21:             BrokeredMessage msg = new BrokeredMessage(order);
 22:             
 23:             // Set properties for use in RuleDescription
 24:             msg.Properties.Add("CountryOfOrigin", "USA");
 25:             msg.Properties.Add("FulfillmentRegion", "");
 26:             
 27:             // Send the message to the OrdersTopic
 28:             client.Send(msg);
 29:             client.Close();
 30:             
 31:            /**************************
 32:              *** Receive Operations ***
 33:            ****************************/
 34:         
 35:             BrokeredMessage recdMsg;
 36:             Order recdOrder;
 37: 
 38:             // Inventory Service Subscriber
 39:             SubscriptionClient inventoryServiceSubscriber = messagingFactory.CreateSubscriptionClient(topicName, "InventoryServiceSubscription",ReceiveMode.PeekLock);
 40:             
 41:             // Read the message from the OrdersTopic
 42:             while ((recdMsg = inventoryServiceSubscriber.Receive(TimeSpan.FromSeconds(5))) != null)
 43:             {   
 44:                 // Convert from BrokeredMessage to native Order
 45:                 recdOrder = recdMsg.GetBody<Order>();
 46: 
 47:                 // Complete read, release and delete message from the fabric
 48:                 inventoryServiceSubscriber.Complete(recdMsg.LockToken);
 49: 
 50:                 Console.ForegroundColor = ConsoleColor.Green;
 51:                 Console.WriteLine("Received Order {0} \n\t on {1} \n\t with Message Id {2} \n\t and Lock Token {3}.", recdOrder.OrderId, "Inventory Service Subscriber", recdMsg.MessageId, recdMsg.LockToken);
 52:             }
 53:             inventoryServiceSubscriber.Close();
 54: 
 55:             // Credit Service Subscriber
 56:             SubscriptionClient creditServiceSubscriber = messagingFactory.CreateSubscriptionClient(topicName, "CreditServiceSubscription");
 57: 
 58:             // Read the message from the OrdersTopic
 59:             recdMsg = creditServiceSubscriber.Receive();
 60: 
 61:             // Convert from BrokeredMessage to native Order
 62:             recdOrder = recdMsg.GetBody<Order>();
 63: 
 64:             // Complete read, release and delete message from the fabric
 65:             creditServiceSubscriber.Complete(recdMsg.LockToken);
 66: 
 67:             Console.ForegroundColor = ConsoleColor.Green;
 68:             Console.WriteLine("Received Order {0} \n\t on {1} \n\t with Message Id {2} \n\t and Lock Token {3}.", recdOrder.OrderId, "Credit Service Subscriber", recdMsg.MessageId, recdMsg.LockToken);
 69: 
 70:             creditServiceSubscriber.Close();
 71: 
 72:             // Fulfillment Service Subscriber for the North America Fulfillment Service Subscription
 73:             SubscriptionClient northAmericaFulfillmentServiceSubscriber = messagingFactory.CreateSubscriptionClient(topicName, "northAmericaFulfillmentServiceSubscription");
 74:             
 75:             // Read the message from the OrdersTopic for the North America Fulfillment Service Subscription
 76:             recdMsg = northAmericaFulfillmentServiceSubscriber.Receive(TimeSpan.FromSeconds(5));
 77: 
 78:             
 79:            if(recdMsg != null)
 80:             {
 81:                 // Convert from BrokeredMessage to native Order
 82:                 recdOrder = recdMsg.GetBody<Order>();
 83: 
 84:                 // Complete read, release and delete message from the fabric
 85:                 northAmericaFulfillmentServiceSubscriber.Complete(recdMsg.LockToken);
 86: 
 87:                 Console.ForegroundColor = ConsoleColor.Green;
 88:                 Console.WriteLine("Received Order {0} \n\t on {1} \n\t with Message Id {2} \n\t and Lock Token {3}.", recdOrder.OrderId, "North America Fulfillment Service Subscriber", recdMsg.MessageId, recdMsg.LockToken);
 89:             }
 90:             else
 91:             {
 92:                 Console.ForegroundColor = ConsoleColor.Yellow;
 93:                 Console.WriteLine("No messages for North America found.");
 94:             }
 95:             
 96:             northAmericaFulfillmentServiceSubscriber.Close();

When running this sample, you’ll see that I have received Order 42 on my Inventory, Credit and North America Fulfillment Service subscriptions:

WCF

One of the great things about the WCF programming model is that it abstracts much of the underlying communication details and as such, other than dropping in a new assembly and and refactoring the binding and configuration, it is not greatly affected by the API changes from the May/June CTP to GA.

As I mentioned, one thing that has changed is that the ServiceBusMessagingBinding has been renamed to NetMessagingBinding. I’ll be covering and end to end example of using the NetMessagingBinding in my upcoming article in CODE Magazine.

REST API

The REST API is key to delivering these new capabilities across a variety of client platforms and remains largely unchanged, however one key change is how message properties are handled. Instead of individual headers for each, there is now one header with all the properties JSON encoded. Please refer to the updated REST API Reference doc for details. I’ll also be covering and end-to-end example of using the REST API to write an read to/from a queue in my upcoming article in CODE Magazine.

More Coming Soon

As I mentioned, in my upcoming article in CODE Magazine, I’ll cover the Why, What, and How behind Azure AppFabric Service Bus Brokered Messaging including end to end walkthroughs with the .NET Client API, REST API and WCF Binding. The November/December issue should be on newsstands (including Barnes and Noble) or your mailbox towards the end of October. You can also find the article online at http://code-magazine.com

Resources

You can learn more about this exciting release as well as download the GA SDK version 1.5 by visiting the following resources:

Azure AppFabric SDK 1.5:http://www.microsoft.com/download/en/details.aspx?id=27421
Clemens Vasters on the May CTP: http://vasters.com/clemensv/2011/05/16/Introducing+The+Windows+Azure+AppFabric+Service+Bus+May+2011+CTP.aspx
Great video by Clemens Vasters on Brokered Messaging: http://vasters.com/clemensv/2011/06/11/Understanding+Windows+Azure+AppFabric+Queues+And+Topics.aspx
David Ingham on Queues: http://blogs.msdn.com//b/appfabric/archive/2011/05/17/an-introduction-to-service-bus-queues.aspx
David Ingham on Topics: http://blogs.msdn.com//b/appfabric/archive/2011/05/25/an-introduction-to-service-bus-topics.aspx
My Introduction to Queues: http://rickgaribay.net/archive/2011/05/17/appfabric-service-bus-v2-ctp.aspx
My Introduction to Topics:http://rickgaribay.net/archive/2011/05/31/exploring-appfabric-service-bus-v2-may-ctp-topics.aspx

String Comparison Gotcha

Rick G. Garibay — Wed, 10 Aug 2011 23:08:34 GMT

I was burning the midnight oil last night writing some samples for a project I’m working on with Azure AppFabric Service Bus Queues and Topics. In short, I was modifying some existing code I had written to defensively check for the existence of a queue so that I could avoiding entity conflicts when attempting to create the same queue name twice.

Given a queueName of “Orders”, the first thing I did was to check to see if it exists by using the Client API along with a very simple LINQ query:

  1: var qs = namespaceClient.GetQueues();
  2: 
  3: var result = from q in qs
  4: 
  5: where q.Path == queueName
  6: 
  7: select q;

If the result came back as empty, I’d go ahead and create the queue:

  1: if (result.Count() == 0)
  2: {
  3: 
  4:      Console.WriteLine("Queue does not exist");
  5: 
  6:     // Create Queue
  7: 
  8:     Console.WriteLine("Creating Queue...");
  9: 
 10:     Queue queue = namespaceClient.CreateQueue(queueName);
 11: 
 12: }

This code worked fine the first time, since the queue did not exist, but the next time I ran the code, and each time following, Line 10 above would execute each time because the result.Count() was always zero.

Looking at the queue properties, it was clear that the queue name was stored in lower case, regardless of the case used in my queueName variable:

On the surface, this created an interesting dilemma: Even though case didn’t matter when creating the queue, when I was searching for the queue name in Line 5 of my LINQ query, it was always resulting in a false match. Could this be a bug?

My first thought was to go back and make queueName lowercase, changing to “orders”. This fixed the problem on the surface, but it felt dirty and fragile.

After reviewing the code with some colleagues (thanks @clemensv and @briannoyes) it turns out that I had made an esoteric, but key error when checking for equality of the queue path:

where q.Path == queueName

The full explanation is available on the C# Programming Guide on MSDN, but in a nutshell, I had made the error of using the “==” operator to check for equality. From MSDN: “Use basic ordinal comparisons when you have to compare or sort the values of two strings without regard to linguistic conventions. A basic ordinal comparison (System.StringComparison.Ordinal) is case-sensitive, which means that the two strings must match character for character: "and" does not equal "And" or "AND".”

The solution was to modify the where clause to use the Equals method which takes the object you are comparing to along with a StringComparison rule of StringComparison.OrdinalIgnoreCase:

  1: var result = from q in qs
  2:      where q.Path.Equals(queueName,StringComparison.OrdinalIgnoreCase)
  3: select q;

Explanation from MSDN: “A frequently-used variation is System.StringComparison.OrdinalIgnoreCase, which will match "and", "And", and "AND". StringComparison.OrdinalIgnoreCase is often used to compare file names, path names, network paths, and any other string whose value does not change based on the locale of the user's computer.”

The AppFabric team is doing exactly the right thing since the queue path corresponds to a network path.

Here is the full code, which allows me to check for the existence of the queue path ensuring that I am ignoring case and the rest is as it was:

  1: var qs = namespaceClient.GetQueues();
  2: 
  3: var result = from q in qs
  4:              where q.Path.Equals(queueName,StringComparison.OrdinalIgnoreCase)
  5:              select q;
  6:             
  7: if (result.Count() == 0)
  8: {
  9:    Console.WriteLine("Queue does not exist");
 10: 
 11:    // Create Queue
 12:    Console.WriteLine("Creating Queue...");
 13:    Queue queue = namespaceClient.CreateQueue(queueName);
 14: }

When you’ve been programming for a while, it’s sometimes easy to forget the little things, and it is often the least obvious things that create the nastiest bugs.

Hope this saves someone else from having a Doh! moment like I did

Exploring AppFabric Service Bus V2 May CTP: Topics

Rick G. Garibay — Tue, 31 May 2011 10:23:00 GMT

In my previous post, I discussed Azure AppFabric Service Bus Queues, a key new capability in the first CTP of the Azure AppFabric Service Bus V2 release that was announced on May 17th.

Queues are an important addition to Azure AppFabric Service Bus capabilities because they provide a solid foundation on which to build loosely coupled distributed messaging solutions. The natural decoupling of queues introduces a number of natural side effects that can further benefit non-functional quality attributes of your solution such as performance, scalability and availability.

The graphic on the right is taken from my recent whitepaper “Developing and Extending Apps for Windows Azure with Visual Studio” and shows the perpetual mismatch of supply and demand of IT capacities. If we think of this mismatch as load on the Y axis being introduced over time, the result is either failure to deliver a service or spending too much on hardware. The goal, then is to align the demand with capacity.

Queues allow us to get closer to the drawing on the left because capacity can be tuned to scale as needed and at its own pace. This is effective because with queues, a consumer/worker can be throttled to only consume what it can handle. If the consumer/worker is offline, items in the queue will queue up, providing classic “store and forward” capabilities. If the consumer/worker is very busy, it will only consume the messages it is able to reliably pull from the queue. If we add more consumers/workers, each consumer/worker will consume messages at its optimal rate (determined by processing capacity, tuning, etc.), resulting in a natural distribution of work. Of course, stronger, more capable consumers/workers may consume more messages, but as long as there are messages in the queue, there is work to be done and the capacity can be allocated accordingly.

As you can see, queues are a great pattern for building loosely coupled distributed solutions, and the ability to add consumers/workers to a process or message exchange in a manner that is transparent from the client/producer perspective makes queues even more useful.

This is the idea behind topics. Before I dive into topics though, let’s talk about the problem that topics are trying to solve.

In distributed systems, it is useful to design message exchanges in terms of publishers and subscribers. Publishers are clients that are either sending a request-response message and will wait for a response; or are sending a single, one way message and expect no response. Subscribers care about these messages for one reason or another and thus subscribe to these messages. This is the essence of the Publish-Subscribe, or more succinct “Pub-Sub” messaging pattern. A one-way pub-sub message exchange pattern is modeled to my left, and again to my right to build on a concrete example. Purchases, be they on-line or at brick-and-mortar retail outlets typically involve a point-of-sale (POS) system.

One of the first things a smart, modern POS software does when a unit is sold is to update inventory on that product so that the company can make proactive, intelligent decisions about managing inventory levels. In most cases, this is an administrative function, that is (or should be) transparent to the customer. When an order/sale is placed, an event occurs which is of interest to an Inventory Service that is responsible for decrementing the inventory count on a shared store. Of course, this is just one of several things that likely need to happen when an order is placed. Credit card authorization as well as fulfillment (whatever that means in the context of the purchase) needs to take place as shown below on your left.

All of a sudden things are more complex than they were before. I want to decouple the POS Client from the downstream business of authorizing a credit card and shipping the product to the customer’s doorstep. Depending on the context, the credit authorization process may be request-response or one-way. For most high-volume online retailers, the financial outcome of the transaction is transparent to the purchasing experience. Ever gotten an email from Amazon.com after you made your purchase letting you know that your order is in a pending state because you need to update your expiration date on file so it can authorize your credit card? This asynchronous approach is common to facilitate scale and performance, and it is also good business.

It is very important to note that when the credit card authorization is designed as one-way, there are a number of guarantees that must be made. First, the Credit Service must receive the message, no matter how busy the front-end or back-end services are. Second, but of equal importance, is that the Credit Service must receive the message once and only once. Failure to deliver on the first or second guarantee will lead to lost revenue, either due to lost transactions or very disgruntled customers.

Using queues is a first step towards achieving these desired outcomes. However, now we need to reason about who should have the responsibility of sending the message to each subscriber? It can’t be the POS Client, because we want to decouple it from this kind of intimate knowledge. Adding this responsibility to each subscriber is also just as bad, or arguably worse.

What we need is an intermediary that forms both a logical and physical relationship between publishers and subscribers such that the minimum degree of coupling is accomplished between the two and no more. This is exactly what a topic provides.

Topics are nothing new. They have been the mainstay of JMS-based systems for years, and thus have proven their usefulness in the field of distributed computing as a great way to logically associate actions to events, thus achieving pub-sub in a minimally coupled manner.

In our scenario, when a sale occurs, the POS Client publishes a message to the “Orders” topic to signal a new order event. At this point, corresponding subscribers are notified by the topic. This logical relationship is modeled to your right. Each one of the subscribers might receive a copy of the same message, in which case we would define the message exchange pattern as multi-cast. It is also possible-and likely- that each service exposes a different contract and thus, transformation between a canonical message and the expected message must take place, but this is a subject for a later post.

How the subscribers receive the message, be it one-way or request-response, is a physical implementation decision. In the classic sense, the logical abstraction of a topic affords us some latitude in choosing a physical transport that will fuse the publisher and subscriber(s) together.

Note: Theory aside, everything I am sharing here is based on my experience over the last few days jumping into these early bits. I am not an expert on the implementation of these capabilities or the features of the CTP and am merely sharing my learnings and thoughts as I explore these exciting new features. If you have questions, know of a different or better way to do something as it applies to the CTP, or have any any corrections, please use the comments below and I’ll gladly consider them and/or share them with the product team and post updates back here.

Exploring Azure AppFabric Service Bus V2 Topics

In Azure AppFabric Service Bus V2, the physical transport used in Topics is Azure AppFabric Service Bus Queues, which allows us to harness the physical advantages of queues and the logical abstraction that topics provide to design our distributed solutions at internet scale.

If you’ve played with the Azure AppFabric Service Bus Queues, or read my introduction to this series you might be wondering what makes Azure AppFabric Service Bus Topic so special. So far, you might be thinking that we could accomplish much of what I’ve discussed with Queues and you wouldn’t be alone. I struggled with this initially as well.

The way that Queues and Topics are implemented in the current technology preview, Topics don’t really seem all that useful until you want to refine when or under what conditions a Subscriber should receive a message beyond simply being subscribed to a Topic, and this can be pretty powerful.

In fact, we can code the scenario shown in the article on Queues to be functionally equivalent with Topics without immediately gaining much.

Creating Topics and Subscriptions

Start by creating a ServiceBusNamespaceClient and MessagingFactory just as before:

  1: ServiceBusNamespaceClient namespaceClient = new ServiceBusNamespaceClient( 
  2:         ServiceBusEnvironment.CreateServiceUri( 
  3:         "sb", serviceNamespace,  
  4:         string.Empty),  
  5:         sharedSecretCreds); 
  6:  
  7: MessagingFactory messagingFactory = MessagingFactory.Create( 
  8:     ServiceBusEnvironment.CreateServiceUri( 
  9:     "sb",  
 10:     serviceNamespace,  
 11:     string.Empty),

Next, create a Topic called “Orders”:

  1: Topic ordersTopic = namespaceClient.CreateTopic("Orders");

The underlying queue infrastructure is created for you.

Now, create a subscription for the Inventory Service on the “Orders” topic:

  1: Subscription inventoryServiceSubscription = ordersTopic.AddSubscription("InventoryServiceSubscription");

At this point, the creation of the Topic and Subscription above would take place in a management context without regard to, or any knowledge of the actual publisher or subscriber(s).

I would expect tooling either from Microsoft or the community or both to start to crop up soon to provide a user experience for these types of management chores, including the ability to enumerate Queues, Topics, etc. For example, before I create a Topic, I need to ensure that the Topic doesn’t already exist or I will get an MessagingEntityAlreadyExistsException. I used the GetQueues method on the namespace client, and if the Topic or Queue entity exists, use the DeleteQueue or DeleteTopic method.

Publishing on a Topic

Now, the client/publisher creates a TopicClient and MessageSender and sends the message on the Orders Topic:

  1: TopicClient pOSClientPublisher = messagingFactory.CreateTopicClient("Orders"); 
  2:  
  3: MessageSender msgSender = pOSClientPublisher.CreateSender(); 
  4:  
  5: Order order = new Order(); 
  6: order.OrderId = 42; 
  7: order.Products.Add("Kinect", 70.50M); 
  8: order.Products.Add("SamsungFocus", 199.99M); 
  9: order.Total = order.Products["Kinect"] + order.Products["SamsungFocus"]; 
 10:  
 11: var msg = BrokeredMessage.CreateMessage(order); 
 12:             
 13: msgSender.Send(msg); 
 14: msgSender.Close();

Note that the client/publisher knows nothing about the subscriber. It is only bound to a logical Topic called “Orders” on line 1 above. It is running in some other process somewhere in the world (literally) that has an internet connection and can make an outbound connection on TCP port 9354***.

Subscribing to a Topic

On the receiving end, a SubscriptionClient is created along with a MessageReciever. The SubscriptionClient instance is created from the MessagingFactory instance which accepts the name of the topic and the subscription itself.

Note the RecieveMode is the same as before which will have the effect of ensuring that the Inventory Service Subscriber receives the message at most once (FWIW, I think PublisherClient and SubscriberClient make more sense that SubscriptionClient and TopicClient respectively, but the intent of the classes is pretty clear and again, these are early bits so expect changes as the team gets feedback and continues to bake the API):

  1: SubscriptionClient inventoryServiceSubscriber = messagingFactory.CreateSubscriptionClient("Orders", "InventoryServiceSubscription"); 
  2:              
  3: MessageReceiver msgReceiver = inventoryServiceSubscriber.CreateReceiver(ReceiveMode.PeekLock); 
  4:  
  5: var recdMsg = msgReceiver.Receive(); 
  6: msgReceiver.Close(); 
  7:  
  8: var recdOrder = recdMsg.GetBody<Order>(); 
  9:  
 10: Console.WriteLine("Received Order {0} on {1}.", recdOrder.OrderId, "Inventory Service Subscriber");

The code above would be wrapped into a polling algorithm that allows you to have fine control over the polling interval, which as Clemens Vasters pointed out in a side conversation recently is a key capability that allows you to throttle your subscribers. The samples in the SDK show a polling technique which works, but it would be nice to see an option for setting some config and letting the API do this for you.

Regardless of the approach you take to checking for messages, the only thing the subscriber knows about is the Topic name and the name of the subscription on line 1 above (having to provide the subscription name in addition to the topic name, seems a bit redundant to me and more coupling than is needed).

Multiple Subscribers

At this point, we’ve emulated the functionality of the Queuing example shown in my first post in the series.

While we’ve increased our level of abstraction, and are no longer thinking about (or care) about the fact that there’s a queue in between the publisher and subscriber, so far, topics haven’t bought us much just yet…

Think back to the scenario above. There are two other services that care about orders. We can create subscriptions for them just as we did for the Inventory Service at management/configuration time:

  1: Subscription fulfillmentServiceSubscription = ordersTopic.AddSubscription("FulfillmentServiceSubscription"); 
  2: Subscription creditServiceSubscription = ordersTopic.AddSubscription("CreditServiceSubscription");

Of course, new subscriptions can be added long after the Topic has been created, and this is one of the many powerful aspects of this logical abstraction from publishers and subscribers. This approach introduces agility into your solutions because you can add subscribers with minimal friction, and in a fully location transparent manner.

As with the publisher (TopicClient), subscribers (SubscriberClients) live in their own process anywhere in the world with an internet connection and can be fired up at will. If one is offline, or unavailable, the message will be queued (provided that previous subscribers have peeked the message (ReceiveMode.PeekLock) as opposed to popping it off the queue (ReceiveMode.ReceiveAndDelete). Below is the simple code for adding listeners/Subscribers for the Credit Service and Fulfillment Service:

  1: // Credit Service Subscriber 
  2: SubscriptionClient creditServiceSubscriber = messagingFactory.CreateSubscriptionClient("Orders", "CreditServiceSubscription"); 
  3:  
  4: msgReceiver = creditServiceSubscriber.CreateReceiver(ReceiveMode.PeekLock); 
  5: recdMsg = msgReceiver.Receive(); 
  6: msgReceiver.Close(); 
  7:  
  8: recdOrder = recdMsg.GetBody<Order>(); 
  9:  
 10: Console.WriteLine("Received Order {0} on {1}.", recdOrder.OrderId, "Credit Service Subscriber"); 
 11:  
 12: // Fulfillment Service Subscriber 
 13: SubscriptionClient fulfillmentServiceSubscriber = messagingFactory.CreateSubscriptionClient("Orders", "FulfillmentServiceSubscription"); 
 14:  
 15: msgReceiver = fulfillmentServiceSubscriber.CreateReceiver(ReceiveMode.PeekLock); 
 16: recdMsg = msgReceiver.Receive(); 
 17: msgReceiver.Close(); 
 18:  
 19: recdOrder = recdMsg.GetBody<Order>(); 
 20:  
 21: Console.WriteLine("Received Order {0} on {1}.", recdOrder.OrderId, “Fulfillment Service Subscriber");

Creating two additional SubscriptionClients for the Credit Service and Fulfillment Service results in all three subscribers getting the message as shown on the right. Again, in my examples, I am running each subscriber in the same process, but in the real world, these subscribers could be deployed anywhere in the world provided they can establish a connection to TCP 9354.

Rules/Actions, Sessions/Groups

Now, what if we wanted to partition the subscribers such that in addition to subscribing to a Topic, additional logic could be evaluated to determine if the subscribers are really interested in the message? Our online retailer probably (err, hopefully) has a centralized inventory management system and credit card processor, but may have different fulfillment centers across the world.

Based on the customer’s origin, the order should go to the closest fulfillment center to minimize cost and ship times (i.e. North America, South America, Africa, East, Europe, Asia, Australia).

Azure AppFabric Service Bus V2 supports this approach with Sessions, Rules and Actions. I group these into the idea of a message pipeline. In addition to the subscriptions, the Topic evaluates additional context or content of the published message configured at management time to introduce some additional filtering and very lightweight orchestration. The topic subscription is the right place for this to happen because again, it is a management-time task. Publishers and subscribers merely send/receive messages. It is the benefit of a logically centralized, yet physically distributed messaging model that affords us the ability to manage these details in a centralized way.

You can create a RuleDescription to evaluate some property or field in the message that indicates country of origin, and as an action, set a property on the message to identify the fulfillment center.

To illustrate this, first, I’ve added two properties to the BrokerMessage that I am publishing on the “Orders” Topic. I’ll use these properties when I configure my rule and action next:

  1: msg.Properties.Add("CountryOfOrigin", "USA"); 
  2: msg.Properties.Add("FulfillmentRegion", "");

Notice that in line 2 above, I’ve intentionally created the “FulfillmentRegion” property with an empty string, since we are going to apply some logic to determine the fulfillment region.

Now, I use a RuleDescription and SqlFilterExpression to determine if the CountryOfOrigin is the United States. If the SqlFilterExpression evaluates to true, then the SqlFilterAction fires and sets the FulfillmentRegion to “North America”:

  1: RuleDescription fulfillmentRuleDescription = new RuleDescription(); 
  2: fulfillmentRuleDescription.FilterExpression = new SqlFilterExpression("CountryOfOrigin = 'USA'"); 
  3: fulfillmentRuleDescription.FilterAction = new SqlFilterAction("set fulfillmentRegion='North America'");

Of course, in the real world, there would be a more sophisticated process for identifying the country of origin, but simple, contrived examples make it so that articles get published.

The evaluation and any corresponding actions must fire when the message is in-flight as any actions taken could influence the routing of the message, as with the example above which will meet a subscription rule we’ll configure on the Fulfillment Service Description next.

OK, so now we have some properties we can play with and we’ve defined a RuleDescription. The last thing we need to do is modify the FulfillmentServiceSubscription to include the RuleDescription I just created. This makes the FulfillmentSubscription conditional, based on the conditions we've defined in the instance of the RuleDescription called fulfillmentRuleDescription:

  1: Subscription fulfillmentServiceSubscription = ordersTopic.AddSubscription("FulfillmentServiceSubscription", fulfillmentRuleDescription);

Now, when I run my code, all three subscribers receive the order message just as before, however this time, we know that the only reason that the Fulfillment Service is getting the message is because it is acting as the North America fulfillment center. If I modify the CountryOfOrigin property in line 1 in the 3rd code sample up from here to anything but “North America” the Fulfillment Service will not receive the message at all.

As I continue to model out my subscribers, I could create a subscription for each fulfillment center that is capable of receiving real-time orders and then create RuleDescriptions accordingly. This would allow me to distribute fulfillment geographically (good for scale, reliability and happy customers) as well as ensuring that I am always only pulling back messages that I need. If, during peak times around holidays, the volume of orders increases, I can simply add additional fulfillment subscribers for that region to ensure that packages ship as quickly as possible and that no orders are lost.

Closing Thoughts

So far, I’m pretty impressed with the powerful messaging capabilities that Azure AppFabric Service Bus V2 Topics introduced in the May CTP, and I’m excited to see where things are going.

As Azure AppFabric Service Bus matures further, I would love to see additional transport channels supported by Azure AppFabric Service Bus Topics. Just as with the creation of the Topic and Subscriptions as a management function, the transport would also be defined and created/configured at management time. This is really where the power and elegance of Topics shines through in my opinion because the publisher and subscriber don’t know or care about the transport- they’re just connecting to a topic and sending and/or receiving messages.

By way of some nit picks, I think that PublisherClient makes more sense than TopicClient, and along with considering a modification to SubscriptionClient, having a PublisherClient and SubscriberClient that publish and subscribe to a Topic seems a lot cleaner and more intuitive to me.

I’m also trying to get used the fact that we need Clients and Senders/Receivers. Again, to me it would seem more intuitive to simply have a PublisherClient and SubscriberClient that own the sending and receiving. Perhaps we’re jus seeing visible seams in the API due to the early nature, or there’s a good reason for this that I haven’t thought of yet.

At PDC 10, the AppFabric team announced that they are investing in an “Integration Service” that will provide additional messaging capabilities by way of transformation and advanced mapping similar to how we leverage these capabilities in BizTalk today. I can see Topics getting much more robust when, in addition to modifying properties on the BrokerMessage, we can mediate and transform a message in-flight just before reaching a subscriber, and I can also think of some nice message enrichment patterns that would be very nice.

*** One important thing to note is that in the current May CTP, Queues and Topics do not provide the same NAT/Firewall traversal capabilities of their relay siblings. For the .NET Messaging API (which I’ve been using to share my learnings thus far) as well as the WCF ServiceBusMessagingBinding, outbound TCP 9354 is required. Also note that the channel type is Net.Tcp (think WCF). This means that in the current CTP, the only way to ensure full interoperability across publishers/subscribers and guarantee an outbound connection (assuming HTTP ports aren’t locked down) is to use the REST API, but I suspect we’ll see more parity of this important feature across client types for Queues and Topics.

What’s Next?

There’s still much to explore around sessions, filtering, the WCF ServiceBusMessagingBinding, the REST API and how we might bridge Azure AppFabric Service Bus with on-premise messaging capabilities. Exciting stuff- stay tuned!