SOA

Azure Service Bus Connect EAI and EDI “Integration Services” CTP

Rick G. Garibay — Fri, 16 Dec 2011 18:46:51 GMT

I am thrilled to share in the announcement that the first public CTP of Azure Service Bus Integration Services is now LIVE at http://portal.appfabriclabs.com.

The focus of this release is to enable you to build hybrid composite solutions that span on-premise investments such as Microsoft SQL Server, Oracle Database, SAP, Siebel eBusiness Applications, Oracle E-Business Suite, allowing you to compose these mission critical systems with applications, assets and workloads that you have deployed to Windows Azure, enabling first-class hybrid integration across traditional network and trust boundaries.

In a web to web world, many of the frictions addressed in these capabilities still exist, albeit to a smaller degree. The reality is that as the web and cloud computing continue to gain momentum, investments on-premise are, and will continue to be critical to realizing the full spectrum of benefits that cloud computing provides both in the short and long term.

So, what’s in this CTP?

Azure Service Bus Connect provides a new server explorer experience for LOB integration exposing a management head that can be accessed on-prem via Server Explorer or PowerShell to create, update, delete or retrieve information from LOB targets. This provides a robust extension of the Azure Service Bus relay endpoint concept, which acts a LOB conduit (LobTarget, LobRelay) for bridging these assets by extending the WCF LOB Adapters that ship with BizTalk Server 2010. The beauty of this approach is that you can leverage the LOB Adapters using BizTalk as a host, or, for a lighter weight way approach, use IIS/Windows Server AppFabric to compose business operations on-premise and beyond.

In addition, support for messaging between trading partners across traditional trust boundaries in business-to-business (B2B) scenarios using is EDI is also provided in this preview, including AS2 protocol support with X12 chaining for send and receive pipelines, FTP as transport for X12, agreement templates, partners view with profiles per partner, resources view, and an intuitive, metro style EDI Portal.

Just as with on-premise integration, friction always exists when integrating different assets which may exist on different platforms, implement different standards and at a minimum have different representations of common entities that are part of your composite solution’s domain. What is needed is a mediation broker that can be leveraged at internet-scale, and apply message and protocol transformations across disparate parties and this is exactly what the Transforms capability provides. Taking an approach that will be immediately familiar to the BizTalk developer, a familiar mapper-like experience is provided within Visual Studio for interactively mapping message elements and applying additional processing logic via operations (functoids).

In addition, XML Bridges which include the XML One-Way Bridge and XML Request-Reply Bridge are an extension to the Azure Service Bus which supports critical patterns such as protocol bridging, routing actions, external data lookup for message enrichment and support for both WS-I and REST endpoints and any combination thereof.

As shown below in the MSDN documentation, “bridges are composed of stages and activities where each stage is a message processing unit in itself. Each stage of a bridge is atomic, which means either a message completes a stage or not. A stage can be turned on or off, indicating whether to process a message or simply let it pass through”.

Taking a familiar VETR approach to validate, extract, transform and route messages from one party to another, along with the ability to enrich messages by composing other endpoint in-flight (supported protocols include HTTP, WS-HTTP and Basic HTTP, HTTP Relay Endpoint, Service Bus Queues/Topics and any other XML bridge) the Bridge is a very important capability and brings very robust capabilities for extending Azure Service Bus as a key messaging broker across integration disciplines.

In reality, these patterns have no more to do with EAI than with traditional, contemporary service composition and become necessary once you move from a point-to-point approach and need to elegantly manage integration and composition across assets. As such, this capability acts as a bridge to Azure Service Bus that is very powerful in and of itself, even in non-EAI/EDI scenarios where endpoints can be virtualized increasing decoupling between parties (clients/services). In addition, this capability further enriches what is possible when using the BrokeredMessage property construct as a potential poor-man’s routing mechanism.

In closing, the need to address the impedance mismatch that exists between disparate applications that must communicate with each other is a friction that will continue to exist for many years to come, and while traditionally, many of these problems have been solved by expensive, big iron middleware servers, this is changing.

As with most technologies, often new possibilities are unlocked that are residual side-effects of something bigger, and this is certainly the case with how both innovative and strategic Azure Service Bus is to Microsoft’s PaaS strategy. Azure Service Bus continues to serve as a great example of a welcomed shift to a lightweight capability-based, platform-oriented approach to solving tough distributed messaging/integration problems while honoring the existing investments that organizations have made and benefiting from a common platform approach which is extremely unique in the market. And while this shift will take some time, in the long-run enterprises of all shapes and sizes only stand to benefit.

To get started, download the SDK & samples from http://go.microsoft.com/fwlink/?LinkID=184288 and the tutorial & documentation from http://go.microsoft.com/fwlink/?LinkID=235197 and watch this and the Windows Azure blog for more details coming soon.

Happy Messaging!

Azure AppFabric Service Bus Brokered Messaging GA & Rude CTP Diffs

Rick G. Garibay — Wed, 14 Sep 2011 16:46:52 GMT

Today at the Build conference in Anaheim California, Satya Nadella, President Server and Tools business announced general availability of the production release of AppFabric Queues and Topics, otherwise known as Brokered Messaging.

Brokered Messaging introduces durable queue capabilities and rich, durable pub-sub with topics and subscriptions that compliment the existing Relayed Messaging capabilities.

I covered Brokered Messaging following the May CTP release of Queues and followed up shortly with an overview and exploration of Topics (please see some other great resources at the end of this post).

Since then, there was a June CTP release which included the new AppFabric Application and no visible changes to Brokered Messaging, however since its release, the AppFabric Messaging team has been hard at work refining the API and behaviors based on feedback from Advisors, MVPs and the community at large.

Since I’ve already covered Queues and Topics in the aforementioned posts, I’ll dive right in to some terse examples which demonstrate the API changes. Though not an exhaustive review of all of the changes, I’ve covered the types that your most likely to come across and will cover Queues, Topics and Subscriptions extensively in my upcoming article in CODE Magazine which will also include more in-depth walk-throughs of the .NET Client API, REST API and WCF scenarios.

Those of you who have worked with the CTPs will find some subtle and not so subtle changes, but all in all I think all of the refinements are for the best and I think you’ll appreciate them as I have. For those new to Azure AppFabric Service Bus Brokered Messaging, you’ll benefit most from reading my first two posts based on the May CTP (or any of the resources at the end of this post) to get an idea of the why behind queues and topics and then come back here to explore the what and how.

A Quick Note on Versioning

In the CTPs that preceded the release of the new Azure AppFabric Service Bus features, a temporary assembly called “Microsoft.ServiceBus.Messaging.dll” was added to serve a container for new features and deltas that were introduced during the development cycle. The final release includes a single assembly called “Microsoft.ServiceBus.dll” which contains all of the existing relay capabilities that you’re already familiar with as well as the addition of support for queues and topics. If you are upgrading from the CTPs, you’ll want to get ahold of the new Microsoft.ServiceBus.dll version 1.5 which includes everything plus the new queue and topic features.

The new 1.5 version of the Microsoft.ServiceBus.dll assembly targets the .NET 4.0 framework. Customers using .NET 3.5 can continue using the existing Microsoft.ServiceBus.dll assembly (version 1.0.1123.2) for leveraging the relay capabilities, but must upgrade to .NET 4.0 to take advantage of the latest features presented here.

.NET Client API

Queues

May/June CTP	General Availability	Comments
ServiceBusNamespaceClientSettings	NamespaceManagerSettings	New class for encapsulating endpoint and security settings.
N/A	TokenProvider	New class for acquiring a WRAP token from ACS.
ServiceBusNamespaceClient	NamespaceManager	Root management object for creating Queues, Topics, Subscriptions.
Queue	QueueDescription	In May/June CTP, Topic / Queue / Subscription were management objects. All create/delete operations were moved to NamespaceManager and the state operations are now on TopicDescription/QueueDescription etc.
MessagingFactorySettings	MessagingFactorySettings	New class for encapsulating security settings.
MessagingFactory	MessagingFactory
BrokeredMessage	BrokeredMessage	No longer a factory. Simply instantiate a BrokeredMessage.
MessageSender	MessageSender	Optional, for use when you want to abstract away queue or topic.
MessageReceiver	MessageReceiver	Optional, for use when you want to abstract away queue or topic.

Below is a representative sample for creating, configuring, sending and receiving a message on a queue:

Administrative Operations

  1: 
  2:             // Configure and create NamespaceManager for performing administrative operations
  3:             NamespaceManagerSettings settings = new NamespaceManagerSettings();
  4:             TokenProvider tokenProvider = settings.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuer,key);
  5:             
  6:             NamespaceManager manager = new NamespaceManager(ServiceBusEnvironment.CreateServiceUri("sb", serviceNamespace, string.Empty), settings);
  7: 
  8:             // Check for existence of queues on the fabric
  9:             var qs = manager.GetQueues();
 10: 
 11:             var result = from q in qs
 12:                          where q.Path.Equals(queueName, StringComparison.OrdinalIgnoreCase)
 13:                          select q;
 14: 
 15:             if (result.Count() == 0)
 16:             {
 17:                 Console.WriteLine("Queue does not exist");
 18: 
 19:                 // Create Queue
 20:                 Console.WriteLine("Creating Queue...");
 21: 
 22:                 manager.CreateQueue(new QueueDescription(queueName) { LockDuration = TimeSpan.FromSeconds(5.0d) });
 23:                 
 24:             }

Runtime Operations

  1:             // Create and Configure Messaging Factory to provision QueueClient
  2:             MessagingFactorySettings messagingFactorySettings = new MessagingFactorySettings();
  3:             messagingFactorySettings.TokenProvider = settings.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuer, key);
  4:             MessagingFactory messagingFactory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb", serviceNamespace, string.Empty), messagingFactorySettings);
  5:             
  6:             QueueClient queueClient = messagingFactory.CreateQueueClient(queueName, ReceiveMode.PeekLock);
  7: 
  8:             Order order = new Order();
  9:             order.OrderId = 42;
 10:             order.Products.Add("Kinect", 70.50M);
 11:             order.Products.Add("XBOX 360", 199.99M);
 12:             order.Total = order.Products["Kinect"] + order.Products["XBOX 360"];
 13: 
 14:             // Create a Brokered Message from the Order object
 15:             BrokeredMessage msg = new BrokeredMessage(order);
 16: 
 17:             /***********************
 18:             *** Send Operations  ***
 19:             ************************/
 20: 
 21:             queueClient.Send(msg);
 22: 
 23:             /**************************
 24:              *** Receive Operations ***
 25:             ***************************/
 26:             
 27:             BrokeredMessage recdMsg;
 28:             Order recdOrder;
 29: 
 30:             // Receive and lock message
 31:             recdMsg = queueClient.Receive();
 32: 
 33:             if(recdMsg != null)
 34:             {
 35:                 // Convert from BrokeredMessage to native Order
 36:                 recdOrder = recdMsg.GetBody<Order>();
 37: 
 38:                 Console.ForegroundColor = ConsoleColor.Green;
 39:                 Console.WriteLine("Received Order {0} \n\t with Message Id {1} \n\t and Lock Token:{2} \n\t from {3} \n\t with total of ${4}", recdOrder.OrderId, recdMsg.MessageId, recdMsg.LockToken, "Receiver 1", recdOrder.Total);
 40:                 recdMsg.Complete();
 41:             }
 42:             queueClient.Close();

Note that MessageSender and MessageReceiver are now optional. Here’s an example that shows PeekLocking a message, simulating an exception and trying again:

  1: 
  2:             // Alternate receive approach using agnostic MessageReceiver
  3:             MessageReceiver receiver = messagingFactory.CreateMessageReceiver(queueName);            // Recieve, complete, and delete message from the fabric
  4: 
  5:             try
  6:             {
  7:                 // Receive and lock message
  8:                 recdMsg = receiver.Receive();
  9: 
 10:                 // Convert from BrokeredMessage to native Order
 11:                 recdOrder = recdMsg.GetBody<Order>();
 12: 
 13:                 // Complete read, release and delete message from the fabric
 14:                 receiver.Complete(recdMsg.LockToken);
 15: 
 16:                 Console.ForegroundColor = ConsoleColor.Green;
 17:                 Console.WriteLine("Received Order {0} \n\t with Message Id {1} \n\t and Lock Token:{2} \n\t from {3} \n\t with total of ${4} \n\t at {5}", recdOrder.OrderId, recdMsg.MessageId, recdMsg.LockToken, "Receiver 2", recdOrder.Total, DateTime.Now.Hour + ":" + DateTime.Now.Minute + ":" + DateTime.Now.Second);
 18:             }
 19:             catch
 20:             {
 21:                 // Should processing fail, release the lock from the fabric and make message available for later processing.
 22:                 if (recdMsg != null)
 23:                 {
 24:                     receiver.Abandon(recdMsg.LockToken);
 25:                     
 26:                     Console.ForegroundColor = ConsoleColor.Red;
 27:                     Console.WriteLine("Message could not be processed.");
 28: 
 29:                 }
 30:             }
 31:             finally
 32:             {
 33:                 receiver.Close();
 34:             }

As shown below, this sample results in order 42 being received by the QueueClient:

Topics

May/June CTP	General Availability	Comments
Topic	TopicDescription	In May/June CTP, Topic / Queue / Subscription were management objects. All create/delete operations were moved to NamespaceManager and the state operations are now on TopicDescription/QueueDescription etc.
TopicClient	TopicClient	As noted in the Queues section, you can use TopicClient or MessageSender in the event you want to abstract details of using Topics.
SubscriptionClient	SubscriptionClient	As noted in the Queues section, you can use SubscriptionClient or MessageReceiver in the event you want to abstract details of using a Topic/Subscription.
Subscription	SubscriptionDescription	Changes to constructors and use of properties (see code samples below), but intent is the same.
RuleDescription	RuleDescription	Changes to constructors and use of properties (see code samples below), but intent is the same.
FilterExpression	Filter	Base for Filter types such as SqlFilter
SqlFilterExpression	SqlFilter
FilterAction	RuleAction

Below is a representative sample for creating, configuring, sending and receiving a message on a topic:

Administrative Operations

  1:             // Configure and create NamespaceManager for performing administrative operations
  2:             NamespaceManagerSettings settings = new NamespaceManagerSettings();
  3:             settings.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuer, key);
  4:             NamespaceManager manager = new NamespaceManager(ServiceBusEnvironment.CreateServiceUri("sb", serviceNamespace, string.Empty), settings);
  5: 
  6:             // Check for existence of topics on the fabric
  7:             var topics = manager.GetTopics();
  8: 
  9:             var result = from t in topics
 10:                          where t.Path.Equals(topicName, StringComparison.OrdinalIgnoreCase)
 11:                          select t;
 12: 
 13:             if (result.Count() == 0)
 14:             {
 15:                 Console.WriteLine("Topic does not exist");
 16: 
 17:                 // Create Queue
 18:                 Console.WriteLine("Creating Topic...");
 19: 
 20:                 TopicDescription topic = manager.CreateTopic(topicName);
 21:             }
 22: 
 23:             // Create Subscriptions for InventoryServiceSubscription and CreditServiceSubscription and associate to OrdersTopic:
 24:             SubscriptionDescription inventoryServiceSubscription = new SubscriptionDescription(topicName, "InventoryServiceSubscription");
 25:             SubscriptionDescription creditServiceSubscription = new SubscriptionDescription(topicName, "CreditServiceSubscription");
 26: 
 27: 
 28:             // Set up Filters for NorthAmericaFulfillmentServiceSubscription
 29:             RuleDescription northAmericafulfillmentRuleDescription = new RuleDescription();
 30:             northAmericafulfillmentRuleDescription.Filter = new SqlFilter("CountryOfOrigin = 'USA' OR CountryOfOrigin ='Canada' OR CountryOfOrgin ='Mexico'");
 31:             northAmericafulfillmentRuleDescription.Action = new SqlRuleAction("set FulfillmentRegion='North America'");
 32: 
 33: 
 34:             // Create Subscriptions
 35:             SubscriptionDescription northAmericaFulfillmentServiceSubscription = new SubscriptionDescription(topicName, "NorthAmericaFulfillmentServiceSubscription");
 36: 
 37:             // Delete existing subscriptions
 38:             try { manager.DeleteSubscription(topicName, inventoryServiceSubscription.Name); } catch { };
 39:             try { manager.DeleteSubscription(topicName, creditServiceSubscription.Name); } catch { };
 40:             try { manager.DeleteSubscription(topicName, northAmericaFulfillmentServiceSubscription.Name); } catch { };
 41: 
 42:             
 43:             // Add Subscriptions and Rules to Topic
 44:             manager.CreateSubscription(inventoryServiceSubscription);
 45:             manager.CreateSubscription(creditServiceSubscription);
 46:             manager.CreateSubscription(northAmericaFulfillmentServiceSubscription, northAmericafulfillmentRuleDescription);
 47:

Runtime Operations

  1:             // Create and Configure Messaging Factory to provision TopicClient
  2:             MessagingFactorySettings runtimeSettings = new MessagingFactorySettings();
  3:             runtimeSettings.TokenProvider = TokenProvider.CreateSharedSecretTokenProvider(issuer, key);
  4:             MessagingFactory messagingFactory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb",serviceNamespace,String.Empty),runtimeSettings);
  5:             
  6:             // Create Topic Client for sending messages to the Topic:
  7:             TopicClient client = messagingFactory.CreateTopicClient(topicName);
  8:          
  9:             /***********************
 10:              *** Send Operations ***
 11:              ***********************/
 12: 
 13:             // Prepare BrokeredMessage and corresponding properties
 14:             Order order = new Order();
 15:             order.OrderId = 42;
 16:             order.Products.Add("Kinect", 70.50M);
 17:             order.Products.Add("XBOX 360", 199.99M);
 18:             order.Total = order.Products["Kinect"] + order.Products["XBOX 360"];
 19: 
 20:             // Set the body to the Order data contract
 21:             BrokeredMessage msg = new BrokeredMessage(order);
 22:             
 23:             // Set properties for use in RuleDescription
 24:             msg.Properties.Add("CountryOfOrigin", "USA");
 25:             msg.Properties.Add("FulfillmentRegion", "");
 26:             
 27:             // Send the message to the OrdersTopic
 28:             client.Send(msg);
 29:             client.Close();
 30:             
 31:            /**************************
 32:              *** Receive Operations ***
 33:            ****************************/
 34:         
 35:             BrokeredMessage recdMsg;
 36:             Order recdOrder;
 37: 
 38:             // Inventory Service Subscriber
 39:             SubscriptionClient inventoryServiceSubscriber = messagingFactory.CreateSubscriptionClient(topicName, "InventoryServiceSubscription",ReceiveMode.PeekLock);
 40:             
 41:             // Read the message from the OrdersTopic
 42:             while ((recdMsg = inventoryServiceSubscriber.Receive(TimeSpan.FromSeconds(5))) != null)
 43:             {   
 44:                 // Convert from BrokeredMessage to native Order
 45:                 recdOrder = recdMsg.GetBody<Order>();
 46: 
 47:                 // Complete read, release and delete message from the fabric
 48:                 inventoryServiceSubscriber.Complete(recdMsg.LockToken);
 49: 
 50:                 Console.ForegroundColor = ConsoleColor.Green;
 51:                 Console.WriteLine("Received Order {0} \n\t on {1} \n\t with Message Id {2} \n\t and Lock Token {3}.", recdOrder.OrderId, "Inventory Service Subscriber", recdMsg.MessageId, recdMsg.LockToken);
 52:             }
 53:             inventoryServiceSubscriber.Close();
 54: 
 55:             // Credit Service Subscriber
 56:             SubscriptionClient creditServiceSubscriber = messagingFactory.CreateSubscriptionClient(topicName, "CreditServiceSubscription");
 57: 
 58:             // Read the message from the OrdersTopic
 59:             recdMsg = creditServiceSubscriber.Receive();
 60: 
 61:             // Convert from BrokeredMessage to native Order
 62:             recdOrder = recdMsg.GetBody<Order>();
 63: 
 64:             // Complete read, release and delete message from the fabric
 65:             creditServiceSubscriber.Complete(recdMsg.LockToken);
 66: 
 67:             Console.ForegroundColor = ConsoleColor.Green;
 68:             Console.WriteLine("Received Order {0} \n\t on {1} \n\t with Message Id {2} \n\t and Lock Token {3}.", recdOrder.OrderId, "Credit Service Subscriber", recdMsg.MessageId, recdMsg.LockToken);
 69: 
 70:             creditServiceSubscriber.Close();
 71: 
 72:             // Fulfillment Service Subscriber for the North America Fulfillment Service Subscription
 73:             SubscriptionClient northAmericaFulfillmentServiceSubscriber = messagingFactory.CreateSubscriptionClient(topicName, "northAmericaFulfillmentServiceSubscription");
 74:             
 75:             // Read the message from the OrdersTopic for the North America Fulfillment Service Subscription
 76:             recdMsg = northAmericaFulfillmentServiceSubscriber.Receive(TimeSpan.FromSeconds(5));
 77: 
 78:             
 79:            if(recdMsg != null)
 80:             {
 81:                 // Convert from BrokeredMessage to native Order
 82:                 recdOrder = recdMsg.GetBody<Order>();
 83: 
 84:                 // Complete read, release and delete message from the fabric
 85:                 northAmericaFulfillmentServiceSubscriber.Complete(recdMsg.LockToken);
 86: 
 87:                 Console.ForegroundColor = ConsoleColor.Green;
 88:                 Console.WriteLine("Received Order {0} \n\t on {1} \n\t with Message Id {2} \n\t and Lock Token {3}.", recdOrder.OrderId, "North America Fulfillment Service Subscriber", recdMsg.MessageId, recdMsg.LockToken);
 89:             }
 90:             else
 91:             {
 92:                 Console.ForegroundColor = ConsoleColor.Yellow;
 93:                 Console.WriteLine("No messages for North America found.");
 94:             }
 95:             
 96:             northAmericaFulfillmentServiceSubscriber.Close();

When running this sample, you’ll see that I have received Order 42 on my Inventory, Credit and North America Fulfillment Service subscriptions:

WCF

One of the great things about the WCF programming model is that it abstracts much of the underlying communication details and as such, other than dropping in a new assembly and and refactoring the binding and configuration, it is not greatly affected by the API changes from the May/June CTP to GA.

As I mentioned, one thing that has changed is that the ServiceBusMessagingBinding has been renamed to NetMessagingBinding. I’ll be covering and end to end example of using the NetMessagingBinding in my upcoming article in CODE Magazine.

REST API

The REST API is key to delivering these new capabilities across a variety of client platforms and remains largely unchanged, however one key change is how message properties are handled. Instead of individual headers for each, there is now one header with all the properties JSON encoded. Please refer to the updated REST API Reference doc for details. I’ll also be covering and end-to-end example of using the REST API to write an read to/from a queue in my upcoming article in CODE Magazine.

More Coming Soon

As I mentioned, in my upcoming article in CODE Magazine, I’ll cover the Why, What, and How behind Azure AppFabric Service Bus Brokered Messaging including end to end walkthroughs with the .NET Client API, REST API and WCF Binding. The November/December issue should be on newsstands (including Barnes and Noble) or your mailbox towards the end of October. You can also find the article online at http://code-magazine.com

Resources

You can learn more about this exciting release as well as download the GA SDK version 1.5 by visiting the following resources:

Azure AppFabric SDK 1.5:http://www.microsoft.com/download/en/details.aspx?id=27421
Clemens Vasters on the May CTP: http://vasters.com/clemensv/2011/05/16/Introducing+The+Windows+Azure+AppFabric+Service+Bus+May+2011+CTP.aspx
Great video by Clemens Vasters on Brokered Messaging: http://vasters.com/clemensv/2011/06/11/Understanding+Windows+Azure+AppFabric+Queues+And+Topics.aspx
David Ingham on Queues: http://blogs.msdn.com//b/appfabric/archive/2011/05/17/an-introduction-to-service-bus-queues.aspx
David Ingham on Topics: http://blogs.msdn.com//b/appfabric/archive/2011/05/25/an-introduction-to-service-bus-topics.aspx
My Introduction to Queues: http://rickgaribay.net/archive/2011/05/17/appfabric-service-bus-v2-ctp.aspx
My Introduction to Topics:http://rickgaribay.net/archive/2011/05/31/exploring-appfabric-service-bus-v2-may-ctp-topics.aspx

A Middle-Tier Guy’s Take on HTML 5

Rick G. Garibay — Mon, 22 Aug 2011 22:50:58 GMT

(My) Early Beginnings

I started my career in software development as a very junior web developer in 1999. I taught myself HTML, VBScript and JavaScript. The browser wars between Microsoft and Netscape were raging. I still remember how exciting my first Classic ASP page was. The fact that I was able to connect to an Access or SQL Server 7 database with just a few lines of code inside my markup was incredible at the time. I consumed new recipes from “4 Guys from Rolla” with relish and kept Scott Mitchell’s titles like “Teach Yourself Classic ASP 3.0 in 21 Days” on my desk at all times (it’s still on my bookshelf which itself has become a sort of Smithsonian for software development over the last decade). Even more exciting was that fact that it seemed like the beginning of a new era in which I could relegate JavaScript as a necessary evil for handling tricks like focus and validation on the UI.

At the time, I was doing some really fun and interesting work as a young Padawan for the credit card division inside the #1 Visa issuer in the country (we didn’t have a fancy name for it other than “reports” back then but we were doing very early work on what would become “BI”). By rendering data-driven operational reports dynamically on the browser, we had revolutionized how metrics like Occupancy, Average Handle Time, and Multiple-Call-Rate were disseminated within the bank ushering in a new era of productivity, transparency and accountability for everyone from agent to VP. Through this experience, we built a center of excellence which served as a benchmark for other call centers to follow (in fact, we had the likes of Amex come in and review how we did it). Sure, we had displaced an army of Excel Macro and Access developers, but such was the price of progress. As this little rogue IT shop basking in our success, I remember a number of “true programmer” personas in the “Real IT” group that tended to undermine what was happening. These were programmers who came from C++ or Java (and whose managers felt threatened by what we were able to do with such few resources) and mostly thumbed their nose at things like lack of strong typing, OO, etc. They looked at JavaScript as just a tool for dirty hippie web masters (remember that term) and VBScript as as something OK for administrators to use to walk AD hives, but being inferior for lacking OO and being handicapped by things like variants. Despite our incredibly visible success (at the CEO level and above) by applying these scripting technologies, I was hungry to see the forest for the trees and experimented a bit with COM and COM+, learning how to encapsulate business logic in components and delighting in being able to wire up my COM libraries with Classic ASP even though with the exception of my manager and mentor at the time, no one else even had the tools to debug my components.

The Server Era

Then, this thing called Windows DNA came along which promised to marry Windows, ActiveX and Internet into one big cluster… well you know. Fortunately, it’s fate was short-lived, but I remember attending a couple MSDN events where it seemed like concurrency and single threaded apartments would become mainstream topics on the web. Maybe us script kiddies would earn some respect after all? Then, just like that, this new, new thing called .NET happened. All of a sudden, just like that, I had a ton to learn. All my Classic ASP and JavaScript skills were superseded by Web Forms. I still remember stepping through every line of code in the I Buy Spy reference app and being completely blow away. ASP.NET offered something that was OO, strongly typed, and would even render JavaScript for you. Cumbersome JavaScript validation was replaced by server-side templates and controls. Form fields magically remembered their values across post backs. And, as I learned, WebForms offered a better separation of concerns with a nice, clean code-behind model that I would later leverage to introduce patterns like MVP, MVC Page Controller and Front Controller. I built a nice CMS portal for a multi-national bank (which according to Forbes Magazine is the largest public trading company in the world today) with these new skills and I hear it is still running today. Life was good. This was the age of the web server, and the only major argument within the Microsoft web community at that time was “C# or VB.NET”?

At this point in my career, I’d grown a bit bored with web development. I felt like I’d accomplished everything I wanted to with ASP.NET, and the roles I found myself in started dealing with problems at a more holistic, program level which would involve a handful of web apps and coordination between them and new and existing enterprise resources. I discovered Web Services, and the problems I was now trying to solve led to the gradual gravitation to enterprise architecture and middleware and before I knew it, I was hooked. Admittedly, it was a great time to make the shift. SOA was king. COM+ had grown up with support for Enterprise Services in .NET and in parallel, this amazing new messaging framework codenamed “Indigo” was in development that would provide a black belt for hungry ninjas like me who wanted to take over the world with SOA. When it came to Indigo, there were two types of members in the community: Those on this inside, and the rest of the world. I was very much part of the rest of the world, but I consumed every bit of content I could get my hands on from folks like Don Box, Juval Lowy, Wenlong Dong, Michele Bustamante and Dr. Nick.

Around the same time, a major re-engineering of a product called BizTalk Server was nearing release which took full advantage of the .NET Framework. My employer then, a mid-sized auto retail and finance company was one of the first BizTalk Server 2004 customers in Phoenix. For a fledging enterprise integration architect, this was an awesome opportunity. I learned a ton from my friend Todd Sussman, Brian Loesgen and Adam Smith, the latter of the two I wouldn’t meet in person for a few years, but I had read “BizTalk Server 2004 Unleashed” and “The Blogger’s Guide to BizTalk” from cover to cover more than once. Even better was that I was leading the development of our first SOA and 802.11 enterprise mobility project. We had decided to build the mobile apps- which were a superset of a desktop control center- with ASP.NET. Users would hit the same URL whether they were on the desktop or in the field with their device, and the right screen would render. All of our business logic was wrapped in an ASMX façade which then communicated with our BizTalk orchestrations. With my first real enterprise program under my belt, and WCF nearing GA, I decided that this was what I wanted to do when I grew up, or at least for the next 5 years.

Along with WCF, WPF was nearing release. WPF offered a completely different paradigm on which to build traditional Windows apps. Support for rich media like video and sound, flat controls, new gradients all with an incredibly “webby” DHTML-looking design aesthetic. At the time, I remember introspecting that if presentation technologies like this were successful at winning over users, then one day, users won’t care if they are using a browser or an OS to interact with software. What I didn’t realize was to what extent everyone’s cheese was about to be moved. Here we had two tremendously powerful additions to the .NET Framework, poised to revolutionize how we write software from a presentation and back-end perspective, and yet, something subtle was happening that was bigger than Microsoft, bigger than the marvel that is .NET.

The Web Reborn

Content building SOA solutions in the walled gardens of my employers and clients, almost overnight, I remember when web developers started insisting on me starting to expose JSON endpoints on my services. Apparently, while I was in messaging la la land, users had grown tired of refreshing their browsers and posting back to the server every time they submitted a form. Turns out, I too was one of them! Users were demanding not just a dynamic web experience, but one that was interactive, and felt more like a rich client (a great example at this time was Outlook Web Access). But if you wanted a rich experience, isn’t that why you stuck to the desktop and used WPF? Following the promulgation of XML as the second coming, wasn’t JSON nothing more than an esoteric relic of JavaScript? AJAX had arrived. What followed was a complete disruption of the seeming balance between intended purpose that would shift the pendulum once again to the web.

This was about the same time that Microsoft had shared its vision for Software + Services, first unveiled at Mix 07. On the outside, the Mix conference was all about the web, but the reality was that this was also part of a pretty massive campaign to court and win over designers from Adobe/Macromedia to a new technology: WPF/Everywhere, aka Silverlight. While many had been signaling WPF as the return of the smart client, users were accepting an alternate, even degraded user experience in exchange for interop. Silverlight offered a superset of WPF capabilities, delivering a somewhat equally productive design time and development experience as .NET and thus began penetrating Apple and Linux via the ubiquity of the browser. This was the first time I saw Microsoft really understanding that it wasn’t only about .NET and Windows anymore. This was evidenced by prominent PMs on stage demonstrating Silverlight apps on Macs, and running Linux distros on VMs to show that you could write the app once, and run it (almost) everywhere. A number of incredible apps were released on Silverlight including examples like Netflix and Hard Rock Café Memorabilia which were each both a sign of the times and a hint at what was to come.

To be sure, this had indeed become a software + services world, and for a while Silverlight looked promising despite the tremendous market footprint that Flash had and continues to have. I remember writing about how remarkable Silverlight was and what a game changer it could be. But then, an interesting thing happened. JavaScript and REST started appearing more and more in web apps, particularly in the consumer space. At first, despite the popularity of Fielding’s paper, REST seemed like a fringe thing, and in many ways a step backward. Here we had a tremendously powerful consortium of standards built around SOAP representing the intersection of the very few things big players like Microsoft, IBM, Sun and Oracle agreed on. What’s more, it seemed that Microsoft had timed this SOAP bubble (no pun intended) perfectly, with its shiny, new, equally efficacious messaging stack called WCF which was, and still today remains unrivaled by other platform vendors. In addition to HTTP, WCF brought TCP, MSMQ and IPC to the enterprise, offering (proprietary) binary encoding and MTOM for optimizing message exchanges. The programming model had (and continues to have) a learning curve over ASMX, but once you got over the hump, you were on a high summit and could see the world for miles around from this new vantage point. So, why in the world would anyone want to go back to using HTTP POST and POX? How could it be that the world was settling for REST and JavaScript?

Simple. The world (and the internet) was changing. A gradual, yet viral shift was taking place fueled by the success of Ruby on Rails and PHP which built the foundation for what is today known as Web 2.0. All of a sudden, anyone with a laptop and an internet connection could download a few packages and get an app up and running in no time. The barrier to entry was financially negligible and because these languages fully embraced HTTP, a tremendous community was born that was as smart as they were entrepreneurial. Interop and reuse were mere side-effects that led to tremendous adoption by everyone with a browser. Fully embracing JavaScript with their productivity boosting libraries that took the sting out of writing JavaScript, similar approaches to packaging robust functionality into libraries such as JQuery followed. In addition, while at first, Ruby on Rails embraced SOAP, it was later replaced with REST. Ask your mother or grandmother what Twitter or Groupon is and you’ll have the answer as to why REST and JavaScript have persevered.

In response to all of this, WCF added support for JSON first, followed by REST, and by the release of .NET 4, both were in the box. In addition, WCF RIA Services was introduced, adding an easy button for integrating with Silverlight clients with the stack, and WCF Data Services provided a REST-friendly approach to managing CRUD operations using ATOM as the message contract. The success of the ASP.NET MVC framework, which has all but subsumed its older, less cool ASP.NET Web Forms sibling, is further evidence of the developer and user community embracing the browser as a conduit for interoperability on the client.

The Mobility Wars

Even amidst the mobile revolution, which has been largely built on the ubiquity of broadband connections and increasingly capable handheld devices, proprietary platforms have emerged which in many ways are more restrictive, and costly than any other platform. Want to build apps for iOs? Learn Objective C. Android? Got Java? Windows Phone? .NET. Even though they all sit on similar devices and depend on the same infrastructure for messaging (the internet), apps are hardly interoperable with one another. I am sure you know at least two people that carry multiple headsets with them for this very reason, and tablets, sure to be the next wave of mobile innovation suffer from the same dilemma (if you ask me why HP abandoned WebOS, I think it has more to do with the writing on the wall regarding HTML5 than anything else, but more on that shortly). At first, this dilemma seems somewhat benign, perhaps only affecting developers. The truth is it affects everyone. Talk to any iOS developer (that’s not a complete Apple zealot) and they’ll tell you that Objective C isn’t the most productive language to write apps with. Aside from Java not being too sexy these days, I don’t hear many Android (or WP7) users raving about the selection of apps in Android Market. Same goes for WP7 and Windows Marketplace- I remember how long I had to wait to get Angry Birds on my Windows Phone 7! But the most salient example I can think of is the fact that after a decade of browser wars, tremendous innovation on the client and the server, I still can’t play Flash videos on my iPad or WP7. My iPad refuses to run Silverlight apps, even though its browser on the desktop is fully capable of doing so. This is a situation that is just plain broken, and it isn’t just me that feels this way…

The World Wants Native Interoperability on the Client, and Today, the Answer is HTML 5.

Like it or not, HTTP has become the ubiquitous interface for both the client and its conduit to the backend. This incredibly simple protocol has had more influence on software over the last decade than any other technology, completely reshaping the strategies of the biggest players. I don’t have to tell you that without HTTP, you don’t have cloud computing. For example, who would have thought that Amazon, after pioneering e-commerce would get into the PaaS business by being the first to truly innovate in commercial cloud computing at scale? Who would think that Microsoft would completely reinvent itself on Windows Azure and invest as deeply in REST as it has, not only with standards and technologies like OData and WCF Data Services, but also in exposing their incredibly rich and powerful Azure APIs as REST heads? Again, the answer is simple. HTTP has become the lingua franca of the interconnected world and the disruption started with the first packet in the early 60s.

Just as SOAP was developed to aid in interop between vendor platforms, banks and partners, REST has increased the native interoperability of applications on the web. Hold your rotten tomatoes, but I am afraid that so too is the fate of iOS, Android, WP7, WPF, ASP.NET and Silverlight. Are they going away tomorrow, next year or 5 years from now? Nope. SOAP still has a very important place in back-end systems and I don’t mean just for legacy applications. When you you want to work with contracts and interfaces (very important when designing critical message exchanges for business processes), need support for heavy lifting such as distributed transactions, reliable messaging, multiple transports and the like, SOAP is your tool. Case in point as I mentioned briefly above is Windows Azure. While the investment in REST has been significant, these REST endpoints are merely designed for optimizing interop allowing any client or platform to take advantage of the services offered by Microsoft’s cloud. Want to start or stop a compute instance? Need to write a file to blob storage, retrieve an entity from table storage or publish a message for hundreds of subscribers to consume over the AppFabric Service Bus? There’s a REST API for that. While adoption is key to the success of any product or platform, the API, and thus REST is not the end itself but merely a means to the end, and as such is only the tip of the iceberg. Below the water’s surface, there is, and will continue to be a ton of SOAP and .NET.

The same is happening on the client with HTML5. While Silverlight and ASP.NET MVC were a step in the right direction and aren’t going to just vanish tomorrow, HTML5 offers true interop at the native (browser) level, and since native interop is what the world wants, it will win, at least for now. I say at least for now because as tempting as it is to chock this up to just another trend, unlike the crusty programmer personas I mentioned when I started this stream of consciousness that has become a rather long post (thanks for staying with me this far, btw), I’ve been doing this long enough to have seen software reinvent itself a few times now. I’ve learned that rather than cry over spilled milk, it is important to embrace change and this means you have to expect and be prepared for anything. HTML 5 could fail, and companies that have already invested significantly in ASP.NET, Silverlight, Flash and one (or several) mobile platforms aren’t going to just jump in right away, but they are going to watch very, very carefully. If I am building a web app or a rich client app today from scratch though, I’m going to think very, very hard before I decide to do so in anything but HTML 5.

Who Moved my Cheese?

Who would have thought that Microsoft, with an incredibly lucrative productivity, OS, server and tools business would bet the farm on Windows Azure? As innovative as I think Microsoft’s (PaaS) cloud story is, in many ways, it is the software giant’s response to its cheese being moved by the web. And make no mistake, it is a massive bet. Initial buzz around Windows 8 has so far been met with both positive and quite negative feedback after the revelation that Windows 8 will make HTML 5 a first class citizen on the desktop and the tablet. Viewed simplistically, the seams between client and server/backend are exposed with Windows 8 and Microsoft Azure respectively. At first, this seems quite alarming (Joel Spolsky saw this coming over 8 years ago), but if you think about it, it makes perfect sense. If the client is moving to the browser, the value proposition of a beefy desktop or a rack of servers in an opaque data center is diminished significantly. However, all that data, records, images, videos, files still have to be stored and served up some where, and that somewhere needs to be natively interoperable with the the client at the iceberg and get the heavy lifting done below the water’s surface. The need for middleware- integration between the client, that somewhere and its data, applications and systems has never been greater.

Even though I’ve joked to friends that stayed on the front end that I didn’t miss anything by skipping WPF and Silverlight because we’re back to where I first began with HTML and JavaScript, the reality is that the last decade has been incredibly important in reinforcing that innovation is bigger than any platform vendor or standards body because unlike them, it is you and me that determine the fate of technology, and for that we should all be proud.

So, What Now?

I provide no value without designing distributed solutions that can be consumed by the client applications and automate the business processes they serve. So just as before, its time to buckle down once again and learn the client technologies that one of my primary customers- the UI developer- will soon be using. First in line is Mango. Next is HTML 5. And who knows, after specializing in integration for the last 5 years, I just might start generalizing a bit and get back into web development again.

See you at the other end of the wire.

Exploring AppFabric Service Bus V2 May CTP: Topics

Rick G. Garibay — Tue, 31 May 2011 10:23:00 GMT

In my previous post, I discussed Azure AppFabric Service Bus Queues, a key new capability in the first CTP of the Azure AppFabric Service Bus V2 release that was announced on May 17th.

Queues are an important addition to Azure AppFabric Service Bus capabilities because they provide a solid foundation on which to build loosely coupled distributed messaging solutions. The natural decoupling of queues introduces a number of natural side effects that can further benefit non-functional quality attributes of your solution such as performance, scalability and availability.

The graphic on the right is taken from my recent whitepaper “Developing and Extending Apps for Windows Azure with Visual Studio” and shows the perpetual mismatch of supply and demand of IT capacities. If we think of this mismatch as load on the Y axis being introduced over time, the result is either failure to deliver a service or spending too much on hardware. The goal, then is to align the demand with capacity.

Queues allow us to get closer to the drawing on the left because capacity can be tuned to scale as needed and at its own pace. This is effective because with queues, a consumer/worker can be throttled to only consume what it can handle. If the consumer/worker is offline, items in the queue will queue up, providing classic “store and forward” capabilities. If the consumer/worker is very busy, it will only consume the messages it is able to reliably pull from the queue. If we add more consumers/workers, each consumer/worker will consume messages at its optimal rate (determined by processing capacity, tuning, etc.), resulting in a natural distribution of work. Of course, stronger, more capable consumers/workers may consume more messages, but as long as there are messages in the queue, there is work to be done and the capacity can be allocated accordingly.

As you can see, queues are a great pattern for building loosely coupled distributed solutions, and the ability to add consumers/workers to a process or message exchange in a manner that is transparent from the client/producer perspective makes queues even more useful.

This is the idea behind topics. Before I dive into topics though, let’s talk about the problem that topics are trying to solve.

In distributed systems, it is useful to design message exchanges in terms of publishers and subscribers. Publishers are clients that are either sending a request-response message and will wait for a response; or are sending a single, one way message and expect no response. Subscribers care about these messages for one reason or another and thus subscribe to these messages. This is the essence of the Publish-Subscribe, or more succinct “Pub-Sub” messaging pattern. A one-way pub-sub message exchange pattern is modeled to my left, and again to my right to build on a concrete example. Purchases, be they on-line or at brick-and-mortar retail outlets typically involve a point-of-sale (POS) system.

One of the first things a smart, modern POS software does when a unit is sold is to update inventory on that product so that the company can make proactive, intelligent decisions about managing inventory levels. In most cases, this is an administrative function, that is (or should be) transparent to the customer. When an order/sale is placed, an event occurs which is of interest to an Inventory Service that is responsible for decrementing the inventory count on a shared store. Of course, this is just one of several things that likely need to happen when an order is placed. Credit card authorization as well as fulfillment (whatever that means in the context of the purchase) needs to take place as shown below on your left.

All of a sudden things are more complex than they were before. I want to decouple the POS Client from the downstream business of authorizing a credit card and shipping the product to the customer’s doorstep. Depending on the context, the credit authorization process may be request-response or one-way. For most high-volume online retailers, the financial outcome of the transaction is transparent to the purchasing experience. Ever gotten an email from Amazon.com after you made your purchase letting you know that your order is in a pending state because you need to update your expiration date on file so it can authorize your credit card? This asynchronous approach is common to facilitate scale and performance, and it is also good business.

It is very important to note that when the credit card authorization is designed as one-way, there are a number of guarantees that must be made. First, the Credit Service must receive the message, no matter how busy the front-end or back-end services are. Second, but of equal importance, is that the Credit Service must receive the message once and only once. Failure to deliver on the first or second guarantee will lead to lost revenue, either due to lost transactions or very disgruntled customers.

Using queues is a first step towards achieving these desired outcomes. However, now we need to reason about who should have the responsibility of sending the message to each subscriber? It can’t be the POS Client, because we want to decouple it from this kind of intimate knowledge. Adding this responsibility to each subscriber is also just as bad, or arguably worse.

What we need is an intermediary that forms both a logical and physical relationship between publishers and subscribers such that the minimum degree of coupling is accomplished between the two and no more. This is exactly what a topic provides.

Topics are nothing new. They have been the mainstay of JMS-based systems for years, and thus have proven their usefulness in the field of distributed computing as a great way to logically associate actions to events, thus achieving pub-sub in a minimally coupled manner.

In our scenario, when a sale occurs, the POS Client publishes a message to the “Orders” topic to signal a new order event. At this point, corresponding subscribers are notified by the topic. This logical relationship is modeled to your right. Each one of the subscribers might receive a copy of the same message, in which case we would define the message exchange pattern as multi-cast. It is also possible-and likely- that each service exposes a different contract and thus, transformation between a canonical message and the expected message must take place, but this is a subject for a later post.

How the subscribers receive the message, be it one-way or request-response, is a physical implementation decision. In the classic sense, the logical abstraction of a topic affords us some latitude in choosing a physical transport that will fuse the publisher and subscriber(s) together.

Note: Theory aside, everything I am sharing here is based on my experience over the last few days jumping into these early bits. I am not an expert on the implementation of these capabilities or the features of the CTP and am merely sharing my learnings and thoughts as I explore these exciting new features. If you have questions, know of a different or better way to do something as it applies to the CTP, or have any any corrections, please use the comments below and I’ll gladly consider them and/or share them with the product team and post updates back here.

Exploring Azure AppFabric Service Bus V2 Topics

In Azure AppFabric Service Bus V2, the physical transport used in Topics is Azure AppFabric Service Bus Queues, which allows us to harness the physical advantages of queues and the logical abstraction that topics provide to design our distributed solutions at internet scale.

If you’ve played with the Azure AppFabric Service Bus Queues, or read my introduction to this series you might be wondering what makes Azure AppFabric Service Bus Topic so special. So far, you might be thinking that we could accomplish much of what I’ve discussed with Queues and you wouldn’t be alone. I struggled with this initially as well.

The way that Queues and Topics are implemented in the current technology preview, Topics don’t really seem all that useful until you want to refine when or under what conditions a Subscriber should receive a message beyond simply being subscribed to a Topic, and this can be pretty powerful.

In fact, we can code the scenario shown in the article on Queues to be functionally equivalent with Topics without immediately gaining much.

Creating Topics and Subscriptions

Start by creating a ServiceBusNamespaceClient and MessagingFactory just as before:

  1: ServiceBusNamespaceClient namespaceClient = new ServiceBusNamespaceClient( 
  2:         ServiceBusEnvironment.CreateServiceUri( 
  3:         "sb", serviceNamespace,  
  4:         string.Empty),  
  5:         sharedSecretCreds); 
  6:  
  7: MessagingFactory messagingFactory = MessagingFactory.Create( 
  8:     ServiceBusEnvironment.CreateServiceUri( 
  9:     "sb",  
 10:     serviceNamespace,  
 11:     string.Empty),

Next, create a Topic called “Orders”:

  1: Topic ordersTopic = namespaceClient.CreateTopic("Orders");

The underlying queue infrastructure is created for you.

Now, create a subscription for the Inventory Service on the “Orders” topic:

  1: Subscription inventoryServiceSubscription = ordersTopic.AddSubscription("InventoryServiceSubscription");

At this point, the creation of the Topic and Subscription above would take place in a management context without regard to, or any knowledge of the actual publisher or subscriber(s).

I would expect tooling either from Microsoft or the community or both to start to crop up soon to provide a user experience for these types of management chores, including the ability to enumerate Queues, Topics, etc. For example, before I create a Topic, I need to ensure that the Topic doesn’t already exist or I will get an MessagingEntityAlreadyExistsException. I used the GetQueues method on the namespace client, and if the Topic or Queue entity exists, use the DeleteQueue or DeleteTopic method.

Publishing on a Topic

Now, the client/publisher creates a TopicClient and MessageSender and sends the message on the Orders Topic:

  1: TopicClient pOSClientPublisher = messagingFactory.CreateTopicClient("Orders"); 
  2:  
  3: MessageSender msgSender = pOSClientPublisher.CreateSender(); 
  4:  
  5: Order order = new Order(); 
  6: order.OrderId = 42; 
  7: order.Products.Add("Kinect", 70.50M); 
  8: order.Products.Add("SamsungFocus", 199.99M); 
  9: order.Total = order.Products["Kinect"] + order.Products["SamsungFocus"]; 
 10:  
 11: var msg = BrokeredMessage.CreateMessage(order); 
 12:             
 13: msgSender.Send(msg); 
 14: msgSender.Close();

Note that the client/publisher knows nothing about the subscriber. It is only bound to a logical Topic called “Orders” on line 1 above. It is running in some other process somewhere in the world (literally) that has an internet connection and can make an outbound connection on TCP port 9354***.

Subscribing to a Topic

On the receiving end, a SubscriptionClient is created along with a MessageReciever. The SubscriptionClient instance is created from the MessagingFactory instance which accepts the name of the topic and the subscription itself.

Note the RecieveMode is the same as before which will have the effect of ensuring that the Inventory Service Subscriber receives the message at most once (FWIW, I think PublisherClient and SubscriberClient make more sense that SubscriptionClient and TopicClient respectively, but the intent of the classes is pretty clear and again, these are early bits so expect changes as the team gets feedback and continues to bake the API):

  1: SubscriptionClient inventoryServiceSubscriber = messagingFactory.CreateSubscriptionClient("Orders", "InventoryServiceSubscription"); 
  2:              
  3: MessageReceiver msgReceiver = inventoryServiceSubscriber.CreateReceiver(ReceiveMode.PeekLock); 
  4:  
  5: var recdMsg = msgReceiver.Receive(); 
  6: msgReceiver.Close(); 
  7:  
  8: var recdOrder = recdMsg.GetBody<Order>(); 
  9:  
 10: Console.WriteLine("Received Order {0} on {1}.", recdOrder.OrderId, "Inventory Service Subscriber");

The code above would be wrapped into a polling algorithm that allows you to have fine control over the polling interval, which as Clemens Vasters pointed out in a side conversation recently is a key capability that allows you to throttle your subscribers. The samples in the SDK show a polling technique which works, but it would be nice to see an option for setting some config and letting the API do this for you.

Regardless of the approach you take to checking for messages, the only thing the subscriber knows about is the Topic name and the name of the subscription on line 1 above (having to provide the subscription name in addition to the topic name, seems a bit redundant to me and more coupling than is needed).

Multiple Subscribers

At this point, we’ve emulated the functionality of the Queuing example shown in my first post in the series.

While we’ve increased our level of abstraction, and are no longer thinking about (or care) about the fact that there’s a queue in between the publisher and subscriber, so far, topics haven’t bought us much just yet…

Think back to the scenario above. There are two other services that care about orders. We can create subscriptions for them just as we did for the Inventory Service at management/configuration time:

  1: Subscription fulfillmentServiceSubscription = ordersTopic.AddSubscription("FulfillmentServiceSubscription"); 
  2: Subscription creditServiceSubscription = ordersTopic.AddSubscription("CreditServiceSubscription");

Of course, new subscriptions can be added long after the Topic has been created, and this is one of the many powerful aspects of this logical abstraction from publishers and subscribers. This approach introduces agility into your solutions because you can add subscribers with minimal friction, and in a fully location transparent manner.

As with the publisher (TopicClient), subscribers (SubscriberClients) live in their own process anywhere in the world with an internet connection and can be fired up at will. If one is offline, or unavailable, the message will be queued (provided that previous subscribers have peeked the message (ReceiveMode.PeekLock) as opposed to popping it off the queue (ReceiveMode.ReceiveAndDelete). Below is the simple code for adding listeners/Subscribers for the Credit Service and Fulfillment Service:

  1: // Credit Service Subscriber 
  2: SubscriptionClient creditServiceSubscriber = messagingFactory.CreateSubscriptionClient("Orders", "CreditServiceSubscription"); 
  3:  
  4: msgReceiver = creditServiceSubscriber.CreateReceiver(ReceiveMode.PeekLock); 
  5: recdMsg = msgReceiver.Receive(); 
  6: msgReceiver.Close(); 
  7:  
  8: recdOrder = recdMsg.GetBody<Order>(); 
  9:  
 10: Console.WriteLine("Received Order {0} on {1}.", recdOrder.OrderId, "Credit Service Subscriber"); 
 11:  
 12: // Fulfillment Service Subscriber 
 13: SubscriptionClient fulfillmentServiceSubscriber = messagingFactory.CreateSubscriptionClient("Orders", "FulfillmentServiceSubscription"); 
 14:  
 15: msgReceiver = fulfillmentServiceSubscriber.CreateReceiver(ReceiveMode.PeekLock); 
 16: recdMsg = msgReceiver.Receive(); 
 17: msgReceiver.Close(); 
 18:  
 19: recdOrder = recdMsg.GetBody<Order>(); 
 20:  
 21: Console.WriteLine("Received Order {0} on {1}.", recdOrder.OrderId, “Fulfillment Service Subscriber");

Creating two additional SubscriptionClients for the Credit Service and Fulfillment Service results in all three subscribers getting the message as shown on the right. Again, in my examples, I am running each subscriber in the same process, but in the real world, these subscribers could be deployed anywhere in the world provided they can establish a connection to TCP 9354.

Rules/Actions, Sessions/Groups

Now, what if we wanted to partition the subscribers such that in addition to subscribing to a Topic, additional logic could be evaluated to determine if the subscribers are really interested in the message? Our online retailer probably (err, hopefully) has a centralized inventory management system and credit card processor, but may have different fulfillment centers across the world.

Based on the customer’s origin, the order should go to the closest fulfillment center to minimize cost and ship times (i.e. North America, South America, Africa, East, Europe, Asia, Australia).

Azure AppFabric Service Bus V2 supports this approach with Sessions, Rules and Actions. I group these into the idea of a message pipeline. In addition to the subscriptions, the Topic evaluates additional context or content of the published message configured at management time to introduce some additional filtering and very lightweight orchestration. The topic subscription is the right place for this to happen because again, it is a management-time task. Publishers and subscribers merely send/receive messages. It is the benefit of a logically centralized, yet physically distributed messaging model that affords us the ability to manage these details in a centralized way.

You can create a RuleDescription to evaluate some property or field in the message that indicates country of origin, and as an action, set a property on the message to identify the fulfillment center.

To illustrate this, first, I’ve added two properties to the BrokerMessage that I am publishing on the “Orders” Topic. I’ll use these properties when I configure my rule and action next:

  1: msg.Properties.Add("CountryOfOrigin", "USA"); 
  2: msg.Properties.Add("FulfillmentRegion", "");

Notice that in line 2 above, I’ve intentionally created the “FulfillmentRegion” property with an empty string, since we are going to apply some logic to determine the fulfillment region.

Now, I use a RuleDescription and SqlFilterExpression to determine if the CountryOfOrigin is the United States. If the SqlFilterExpression evaluates to true, then the SqlFilterAction fires and sets the FulfillmentRegion to “North America”:

  1: RuleDescription fulfillmentRuleDescription = new RuleDescription(); 
  2: fulfillmentRuleDescription.FilterExpression = new SqlFilterExpression("CountryOfOrigin = 'USA'"); 
  3: fulfillmentRuleDescription.FilterAction = new SqlFilterAction("set fulfillmentRegion='North America'");

Of course, in the real world, there would be a more sophisticated process for identifying the country of origin, but simple, contrived examples make it so that articles get published.

The evaluation and any corresponding actions must fire when the message is in-flight as any actions taken could influence the routing of the message, as with the example above which will meet a subscription rule we’ll configure on the Fulfillment Service Description next.

OK, so now we have some properties we can play with and we’ve defined a RuleDescription. The last thing we need to do is modify the FulfillmentServiceSubscription to include the RuleDescription I just created. This makes the FulfillmentSubscription conditional, based on the conditions we've defined in the instance of the RuleDescription called fulfillmentRuleDescription:

  1: Subscription fulfillmentServiceSubscription = ordersTopic.AddSubscription("FulfillmentServiceSubscription", fulfillmentRuleDescription);

Now, when I run my code, all three subscribers receive the order message just as before, however this time, we know that the only reason that the Fulfillment Service is getting the message is because it is acting as the North America fulfillment center. If I modify the CountryOfOrigin property in line 1 in the 3rd code sample up from here to anything but “North America” the Fulfillment Service will not receive the message at all.

As I continue to model out my subscribers, I could create a subscription for each fulfillment center that is capable of receiving real-time orders and then create RuleDescriptions accordingly. This would allow me to distribute fulfillment geographically (good for scale, reliability and happy customers) as well as ensuring that I am always only pulling back messages that I need. If, during peak times around holidays, the volume of orders increases, I can simply add additional fulfillment subscribers for that region to ensure that packages ship as quickly as possible and that no orders are lost.

Closing Thoughts

So far, I’m pretty impressed with the powerful messaging capabilities that Azure AppFabric Service Bus V2 Topics introduced in the May CTP, and I’m excited to see where things are going.

As Azure AppFabric Service Bus matures further, I would love to see additional transport channels supported by Azure AppFabric Service Bus Topics. Just as with the creation of the Topic and Subscriptions as a management function, the transport would also be defined and created/configured at management time. This is really where the power and elegance of Topics shines through in my opinion because the publisher and subscriber don’t know or care about the transport- they’re just connecting to a topic and sending and/or receiving messages.

By way of some nit picks, I think that PublisherClient makes more sense than TopicClient, and along with considering a modification to SubscriptionClient, having a PublisherClient and SubscriberClient that publish and subscribe to a Topic seems a lot cleaner and more intuitive to me.

I’m also trying to get used the fact that we need Clients and Senders/Receivers. Again, to me it would seem more intuitive to simply have a PublisherClient and SubscriberClient that own the sending and receiving. Perhaps we’re jus seeing visible seams in the API due to the early nature, or there’s a good reason for this that I haven’t thought of yet.

At PDC 10, the AppFabric team announced that they are investing in an “Integration Service” that will provide additional messaging capabilities by way of transformation and advanced mapping similar to how we leverage these capabilities in BizTalk today. I can see Topics getting much more robust when, in addition to modifying properties on the BrokerMessage, we can mediate and transform a message in-flight just before reaching a subscriber, and I can also think of some nice message enrichment patterns that would be very nice.

*** One important thing to note is that in the current May CTP, Queues and Topics do not provide the same NAT/Firewall traversal capabilities of their relay siblings. For the .NET Messaging API (which I’ve been using to share my learnings thus far) as well as the WCF ServiceBusMessagingBinding, outbound TCP 9354 is required. Also note that the channel type is Net.Tcp (think WCF). This means that in the current CTP, the only way to ensure full interoperability across publishers/subscribers and guarantee an outbound connection (assuming HTTP ports aren’t locked down) is to use the REST API, but I suspect we’ll see more parity of this important feature across client types for Queues and Topics.

What’s Next?

There’s still much to explore around sessions, filtering, the WCF ServiceBusMessagingBinding, the REST API and how we might bridge Azure AppFabric Service Bus with on-premise messaging capabilities. Exciting stuff- stay tuned!

Exploring Azure AppFabric Service Bus V2 May CTP: Queues

Rick G. Garibay — Tue, 17 May 2011 07:17:18 GMT

Today, the AppFabric team announced the first Community Technology Preview of the Azure AppFabric Service Bus V2.

The second release of Azure AppFabric Service bus is a significant milestone that the team has been hard at work on for several months. While I’ve had the privilege of attending a number of SDRs and watching this release move from ideation to actual bits, this is the first time I’ve been able to actually get my hands on the code, so I’ve spent the better part of this evening diving in.

First, if you are new to the Azure AppFabric Service Bus, there are many resources available to get you up to speed. I highly recommend this whitepaper by Aaron Skonnard: A Developer’s Guide to the Service Bus. If you are a visual learner, please consider checking out my webcast on AppFabric Service Bus here: http://rickgaribay.net/archive/2011/01/12/neudesic-appfabric-webcast-series.aspx.

In a nutshell, like most ESBs, Azure AppFabric Service Bus is the manifestation of a number of core messaging patterns that provide the ability to design messaging solutions that are loosely coupled. What makes the Azure AppFabric Service Bus unique is that it provides these capabilities at “internet scale”, meaning that it is designed to decouple clients and services regardless of whether they are running on premise or in the cloud. As a result, the AppFabric Service Bus is a key technology for enabling hybrid scenarios at the platform level (i.e. PaaS) and serves as a key differentiator in the market today for enabling organizations to adopt cloud computing in a pragmatic way.

Messaging patterns in general provide a common frame on which to think about and build composite applications. Cloud and hybrid computing necessitate many of the same messaging patterns found within the on-premise enterprise and introduce new complexities that are somewhat unique. Azure AppFabric Service Bus V2 introduces tried and true messaging capabilities such as Queues, Topics, Pipes and Filters (Rules/Actions) as well as sequencing semantics and of course durability.

It is important to note that the Azure AppFabric Service Bus is not a replacement for on-premise publish-subscribe messaging. It enables new scenarios that allow you to integrate your current on-premise messaging and provide the ability to compose clouds, be they your own, your partners or those of private cloud providers such as Microsoft. The drawing on the right is from the Microsoft whitepaper I mentioned in the introduction. Notice that the Azure AppFabric Service Bus is providing the ability to integrate clients and services regardless of where they reside, and for non-trivial SOA, on-premise pub-sub for decoupling composed services and clients is essential.

Queues

Queues are typically used to provide temporal decoupling, which provides support for occasionally connected clients and/or services. With a queue, a client writes to a known endpoint (private/local or public/somewhere on the network) without regard to the state of the downstream service. If the service/consumer is running, it will read work items or messages from the queue. Otherwise, the queue will retain the message(s) until the service is available to retrieve the message or the message expires.

As promised, Azure AppFabric Service Bus V2 delivers on durable messaging capabilities beyond the current Message Buffer feature with the introduction of Queues.

Queues are supported in the .NET API (shown below), REST API and with a new WCF Binding called “ServiceBusMessagingBinding”. While my preferred approach will certainly be WCF, the .NET API helps to understand the new API. In addition, the process of creating a queue is required even with WCF since queue creation is outside of WCF’s immediate area of concern.

The first thing you need to do when working with Queues, Topics and Subscribers is create a Service Bus Namespace Client which is an anchor class for managing Service Bus entities:

ServiceBusNamespaceClient namespaceClient = new 
ServiceBusNamespaceClient(ServiceBusEnvironment.CreateServiceUri("sb", 
serviceNamespace, string.Empty), 
sharedSecretCreds);

Once you have an instance of the ServiceBusNamespaceClient, you can create a queue by simply instantiating the Microsoft.ServiceBus.Messaging.Queue class (I’ve set the queueName field to “Orders”):

  Queue queue = namespaceClient.CreateQueue(queueName);

Next, create a Queue client. The Queue Client manages both send and receive operations:

   1:    MessagingFactory messagingFactory = 
MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb", serviceNamespace, string.Empty), sharedSecretCreds);

   2:  var queueClient = messagingFactory.CreateQueueClient(queue);

The code above uses the Microsoft.ServiceBus.Messaging.MessagingFactory which accepts your namespace and security credentials and returns an instance of the MessagingFactory.

Next, you create an instance of Microsoft.ServiceBus.Messaging.MessageSender using the CreateSender method:

   1:  var messageSender = queueClient.CreateSender();

   2:  messageSender.Send(msg);

   3:  messageSender.Close();

Line 7 below creates a BrokeredMessage, which is a new message class which represents a unit of communication between Service Bus clients. Note that this class has nothing to do with the System.ServiceModel.Channels Message class, however when using the ServiceBusMessagingBinding the classic WCF Message class is used. The BrokeredMessage class consists of a number of methods and some interesting properties including ContentType, CorrelationId, MessageId, Label and a property bag called Properties which we’ll explore as we progress through the new features.

In this example, I’m using an overload of CreateMessage that accepts a serializable object. The method uses the DataContractSerializer with a a binary XmlDictionaryWriter to create a BrokeredMessage.

   1:  Order order = new Order();

   2:  order.OrderId = 42;

   3:  order.Products.Add("Kinect",70.50M);

   4:  order.Products.Add("SamsungFocus", 199.99M);

   5:  order.Total = order.Products["Kinect"] + order.Products["SamsungFocus"];

6:

   7:  var msg = BrokeredMessage.CreateMessage(order);

Finally, we can send the message:

   1:  var messageSender = queueClient.CreateSender();

   2:  messageSender.Send(msg);

   3:  messageSender.Close();

With much of the infrastructure code out of the way, we can use the same queueClient instance to create a MessageReciever and request the BrokeredMessage message from the Orders queue:

   1:  var messageReceiver = queueClient.CreateReceiver(ReceiveMode.ReceiveAndDelete);

   2:  var recdMsg = messageReceiver.Receive();

   3:  messageReceiver.Close();

Note the ReceiveMode in line 1 above. This has the effect of enforcing an ~~“Exactly-Once”~~ “At Most Once” (thanks David Ingham) receive semantic since the first consumer to read the message will pop it off the queue. The opposite option is RecieveMode.PeekLock which provides “At Least Once” delivery semantics. As David Ingham, Program Manager on the AppFabric team kindly adds in his comments below: “If the consumer is able to record the MessageIds of messages that it has processed then you can achieve “ExactlyOnce” processing semantics with PeekLock mode too.” –Thanks David!

Once we have the BrokeredMessage, we can retrieve the body:

   1:  var recdOrder = recdMsg.GetBody<Order>();

   2:  Console.WriteLine("Received Order {0} with total of ${1}",recdOrder.OrderId,recdOrder.Total);

The cool thing about the CTP is that Microsoft is offering you a free labs environment in which to explore and play. In the CTP, you can create a maximum of 100 queues, with a maximum size of 100MB each and messages can not exceed a payload size of 256KB- pretty workable constraints for a first CTP.

To get started, and dive in for yourself, be sure to download the Azure AppFabric V2 SDK at: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=D89640FC-C552-446E-AEAD-B1E0D940F31B

In my next post, we’ll explore Topics and Subscriptions which allow for rich pub-sub including the ability to multicast to thousands of potential subscribers.

Hats off to the AppFabric Messaging team for all of their hard work with this release!

Microsoft Business Integration Roadshow–The Goods

Rick G. Garibay — Mon, 02 May 2011 18:03:56 GMT

We just wrapped up a great roadshow with Microsoft in 5 US cities including Philadelphia, Columbus, Houston, Phoenix and Mountain View highlighting the exciting new developments in the AppFabric platform.

The event included a keynote briefing by a Microsoft Product Manager, a deep dive technical session led by our Connected Systems consultants and either a customer session or “Ask the Experts” panel to wrap things up.

The event was well received and there was lots of good discussion and questions.

As promised, I am posting the deck as well as a link to a recorded version of the presentation for those who attended and would like to reference the material or those who were unable to attend.

Abstract: Building Composite Application Services with AppFabric and BizTalk Server 2010

BizTalk Server 2010 and AppFabric provide a comprehensive middleware platform for developing, deploying, and managing composite enterprise capabilities both on-premise and in the cloud. Come learn how AppFabric and BizTalk Server can benefit your approach to building and supporting application services at enterprise scale while transcending traditional trust boundaries and enabling the hybrid enterprise.

Special thanks to the Neudesic team, including Justin Yanta, Monish Nagisetty, Brendon Birdoes and Stuart Celearier for doing such a great job on the deep dive sessions!

Note: Session starts at 2:30 mark.

Richard Seroter's Architecture Musings

Rick G. Garibay — Tue, 01 Feb 2011 19:42:16 GMT

I am honored to have been invited by Richard Seroter to participate in his 27th interview in his series on Connected Systems technology for February 2011.

Richard is an absolute thought leader in the CSD space and epitomizes much of what we discussed in his day-to-day role as a Solution Architect for his very lucky employer. His blog posts, articles, speaking engagements and books cover the full breadth of the Microsoft middleware stack and his command of the technology and breadth is truly an achievement to aspire to.

As we discuss, there really has never been a better time for the Microsoft platform, and I believe that AppFabric really is a unification of on-premise workloads while providing first-class tooling for cloud and hybrid.

You can read the full interview here: http://seroter.wordpress.com/2011/01/31/interview-series-four-questions-with-rick-garibay/

I’d love your thoughts and comments and would also encourage you to read the other interviews in the series: http://seroter.wordpress.com/category/four-questions/

Pat Filoteo and Fellow MVPs on AppFabric & More

Rick G. Garibay — Sat, 11 Dec 2010 22:17:54 GMT

I was fortunate to participate in a one hour discussion with Microsoft architect Pat Filoteo and a few fellow MVPs including my Neudesic colleague David Pallmann on Windows Azure and AppFabric a few weeks ago when I was on campus for PDC 10.

The discussion was filmed and posted by the MVP team, uncut in its entirety.

In the segment below, we talk about the potential for AppFabric to transform how we think about composite applications and how important hybrid composition will be to the enterprise as it identifies the chemistry and seeks the right psychology for leveraging the cloud in a manner that leads to increased effectiveness while preserving and more importantly extending the reach of on-premise assets to the cloud and beyond.

If you are interested in watching all segments, please check out the MVP Award Blog: http://blogs.msdn.com/b/mvpawardprogram/archive/2010/12/08/windows-azure-q-amp-a-discussion-with-microsoft-azure-architect.aspx

Composite Applications Roadshow – Dallas & Houston

Rick G. Garibay — Wed, 08 Dec 2010 17:44:26 GMT

Microsoft is hosting a two events in Dallas and Houston on 12/8 and 12/9 covering composite application scenarios, governance, composite application roadmap and upgrading to BizTalk Server 2010.

I just got done presenting the keynote, “Building Composite Application Services with AppFabric” at the Microsoft Las Colinas Campus in Dallas and will be presenting once again at the Houston Microsoft Campus tomorrow (12/9), so if you are in the area but missed today’s event, please feel free to register and attend: https://msevents.microsoft.com/cui/EventDetail.aspx?culture=en-US&EventID=1032469800&IO=ycqB%2bGJQr78fJBMJTye1oA%3d%3d

In this session, I cover how to enable hybrid composition scenarios leveraging AppFabric, Azure and BizTalk Server 2010 by looking at a hybrid travel & hospitality scenario that manages reservation requests on-premise by composing services hosted in an Azure Web Role and a BizTalk Server 2010 Orchestration hosted out in the edge (such as a restaurant location itself) which receives new reservation manifests and reserves a table. The on-premise application is implemented with WF 4 as a Workflow Service and is hosted in Server AppFabric and consumes a WCF 4 service hosted in an Azure Web Role which in turn consumes the BizTalk Orchestration using AppFabric Connect for Web Services.

Below is the agenda for both events and I am also attaching the deck from my talk for any attendees or others who would like to reference it.

09:00 – 10:00 Composite Application (Windows AppFabric, Azure AppFabric, BizTalk 2010)

10:00 – 11:00 Accelerate Adoption of SOA – Tools, Best Practices, Governance

11:00 – 12:00 BizTalk 2010 and Beyond Roadmap

12:00 – 01:00 (Lunch) Upgrading BizTalk Server 2006 R2 / BizTalk 2009 to BizTalk 2010

SOA 2.0

Rick G. Garibay — Thu, 28 Oct 2010 23:09:04 GMT

In my previous post, I talked about how AppFabric is really coming together as a cohesive platform for building, deploying and managing composite service-oriented applications both on-premise and in the cloud.

Key to recent advancements announced today is the AppFabric Composition Model.

The Azure AppFabric Composition Model is a set of .NET Framework extensions for composing applications & services consistently across Windows Azure & Windows Server platforms, delivering on the promise of a true hybrid platform as a service that transends on-premise and cloud.

It includes Developer Tooling via a Visual Studio based designer to compose, deploy, and manage a cloud application as a single logical entity.

In addition, managed runtime services consume the Composition Model to provide deployment & management of the end to end application in a simple template allowing bringing click-once deployment to the cloud for multi-role applications.

Key to the value proposition of the AppFabric Composition Model is a new "role" called the AppFabric Container which provides a scaled-out, multi-tenant, isolated, high-performance runtime optimized for cloud-scale services & mid-tier components. With the AppFabric Container, we no longer think about Web Roles or Worker Roles, or how many VMs to allocate. It is essentially the 'easy button' for deploying your WCF, WF or ASP.NET apps as first class citizens int he Composition Model. You can almost think of an AppFabric Container as the premier logical hosting environment. A good analogy would be thinking about how self-hosting a WCF or WF service felt like compared to pushing to Server AppFabric hosting. With the AppFabric Container, we don't think about Web or Worker roles. We just deploy.

The beauty of the Composition Model is that you compose your application by dragging and dropping these services at design time, choosing services such as service bus, caching, containers, etc.

Of course, composing Azure roles in itself is only half of the story. As you would probably expect, AppFabric Service Bus provides the integration backbone for composing applications across the cloud and on-premise providing location transparency and a logically centralized, yet physically distributed management model via the AppFabric Management Portal. Announced publically today is the addition of durable messaging to the Service Bus.

And please keep in mind, this is not a Microsoft only party. Any WS compliant service or REST endpoint can be consumed from the cloud, regardless of vendor. This is a WCF client thing. In terms of exposing non-Microsoft services in Azure in particular, Microsoft has committed to investing in other platforms such as Java and Ruby to bring the benefits of cloud to anyone who wants to take advantage of the benefits that the cloud provides.

To me, this is SOA 2.0: the further democratization of building service-oriented applications by commoditizing the composition of services across traditional boundaries, both in the commercial and enterprise space at virtually unlimited scale.

In terms of timeline, caching in Web and Worker Roles are now available in CTP form as is durable message buffer support in AppFabric Service Bus, with full RTW versions following in H1 or 2011 along with the first public CTP of the Composite Model.