Architecture & Design

Desert Code Camp 2010.1 – The Goods

Rick G. Garibay — Thu, 20 May 2010 02:16:02 GMT

This was a great event as usual, and I really have to hand it to @jguadango, @coneybeer, Devry University and all the volunteers for putting this together. The logistics, facilities, scheduling and planning for 55+ sessions is not easy.

I’ve presented at every Desert Code Camp since the very first one, so I think this one makes 6 or 7. Code Camp is always a great day to interact with Phoenix’s community of technologists through conversation, teaching and learning and this one was no exception.

Below are the goods for each session, and as promised I’ve included videos of my WCF and AppFabric Service Bus demos.

Session

Content

Code

Demo Vids

Let the Good Guys in with Azure AppFabric Service Bus

Demo1: Unicast One-Way Messaging with NetOneWayRelay Binding

Demo 2: Multicast On-Way Messaging with NetRelayBinding

Demo 3: World Peace & Fireworks with NetTcpRelayBinding (HD)

Beyond the Whiteboard: Enforcing Conceptual Integrity with Visual Studio 2010

N/A

New Features in WCF 4 that will Instantly Make you More Productive

Demo 1: Convention over Configuration (HD)

Demo 2: Dynamic Endpoint Discovery (HD)

Demo 3: Monitoring Service Availability with WS-Discovery Announcements (HD)

If you have any comments or questions, drop me a line.

Visual Studio 2010 Ultimate Event Content Download

Rick G. Garibay — Sun, 14 Mar 2010 01:45:01 GMT

Thanks to all who came out to the event in Phoenix, Salt Lake City & Las Vegas. We had a great turnout at all events with great questions and discussions.

Remember that the conversation continues on twitter.

If you attended the event and have a question or comment for any of the presenters, use the following hashtags:

Phoenix: #vs2010phx
Salt Lake City: #vs2010slc
Las Vegas: #vs2010vegas

Below are the presentations from all four sessions in each city:

Phoenix	A Lap Around Visual Studio 2010	Rick G. Garibay
Phoenix	Agile Management with TFS	Brendon Birdoes	*
Phoenix	No More "No Repro"	Brendon Birdoes	*
Phoenix	Architecture for Everyone	Rick G. Garibay
Salt Lake City	A Lap Around Visual Studio 2010	Rick G. Garibay
Salt Lake City	Agile Management with TFS	Jeff Leite	*
Salt Lake City	No More "No Repro"	Jeff Leite	*
Salt Lake City	Architecture for Everyone	Rick G. Garibay
Las Vegas	A Lap Around Visual Studio 2010	Rick G. Garibay
Las Vegas	Agile Management with TFS	Mickey Williams	*
Las Vegas	No More "No Repro"	Mickey Williams	*
Las Vegas	Architecture for Everyone	Rick G. Garibay

* Note that content for all venues is very similar but may vary from presenter to presenter. ~~All content for both venues will be available soon so please check back shortly.~~

.NET 4.0 and Visual Studio 2010 Beta 2 Announcements

Rick G. Garibay — Mon, 19 Oct 2009 23:30:00 GMT

.NET 4.0 and Visual Studio 2010 Beta 2 is now available to MSDN subscribers as of 9:30 AM PST. This release includes the new version of Connected Framework products like WCF and WF 4.0, Entity Framework 4.0 and a host of additional new features across work streams including MEF, Parallel Extensions, and significant enhancements to Visual Studio Team System and Team Build.

Having spent some time with both WCF and WF limited CTP, Beta and pre-Beta 2 bits, this release includes significant improvements, re-design and new features specifically around WCF and WF 4.0. If you are like me, you will likely welcome many of the designer enhancements in WF 4.0 and may lament a number of designer design changes within messaging activities. You’ll just need to rip off the shrink wrap on Beta 2 to find out yourself!

I plan to start doing a significant amount of writing around WF 4.0 because I think that having been re-written from the ground up, WF 4.0 addresses many, many of the pain points in the 3.0 and 3.5 version, and I believe that WF + WCF 4.0 will lower the barrier of entry for writing workflow and service/model-oriented applications. In addition, the improvements within Workflow Services and the ease with which those new to WCF can get a service up and running are both significant and necessary to increasing the reach of these fantastic technologies.

Another major area of VS 2010 I am excited about is in the new architecture and design tools in Microsoft Visual Studio Team System 2010 Architecture Edition which introduces new UML designers, use cases, activity diagrams, sequence diagrams and the ability to explore and visualize existing code, design and apply layering to enforce dependency rules, and physical designers to visualize, analyze, and refactor your software. In May of the year, I demonstrated how VS 2010 extends UML logical views into physical views of your code along with how to create relationships from these views to work items and project metrics, and how to programmatically transform models into patterns for other domains and disciplines.

Needless to say, I’m excited about VS 2010 Beta 2, and am even more excited to announce that the official launch of Visual Studio 2010 will take place on March 22nd, 2010.

Until then, you can download Beta 2 on October 21st here: http://msdn.microsoft.com/en-us/vstudio/dd582936.aspx

Some other notes of interest:

Microsoft is announcing a new, streamlined packaging lineup and licensing options for Visual Studio 2010 and MSDN. This includes an "Ultimate Offer" to help customers upgrade to the latest developer tools. For more information on the new packaging, licensing, and pricing of Visual Studio 2010, visit: http://go.microsoft.com/fwlink/?LinkID=166249
As you may have noticed, MSDN has undergone a significant redesign which includes new benefits for all MSDN subscribers: http://msdn.microsoft.com/en-us/default.aspx

March 22nd will be here before you know it, so jump on the Beta 2 bits now and get ready!

Oh, and don’t forget, PDC 2009 is right around the corner. Register now for key insights you just won’t find at any other conference: http://microsoftpdc.com/

Unit Testing Microsoft Sync Framework 2.0 & WCF

Rick G. Garibay — Tue, 18 Aug 2009 04:46:33 GMT

I just wrapped up some work for a client that entails exposing the Sync Framework 2.0 via WCF.

We are using Sync Framework 2.0 CTP 2 to support occasionally connected client scenarios between applications deployed out in the edge and core services hosted in the client’s cloud. I am using WCF to wrap the Sync Fx APIs to support interoperable synchronization support via SOAP 1.1. This way, any client can on-ramp with the core services and essentially become self-provisioned by initiating and initial synchronization to download required data necessary to come on line. In addition, when events happen out on the edge, the client can synchronize each event back to the cloud using an upload semantic.

There are many benefits to using the Sync Fx as opposed to rolling your own, and one benefit is the extremely simple provider model which makes basic synchronization scenarios a breeze, particularly between relational replicas such as SQL Server. Note however, that Sync Fx is fully extensible and capable of synchronizing any two memory streams.

So, the basic idea is that given a SyncProvider for the local and remote replica, any two data sources/applications can be synchronized:

In my scenario, the local client out in the edge utilizes the out of the box SqlSyncProvider. The goal was to expose the cloud over HTTP using SOAP, which required some understanding of the underlying APIs. From there, it was simply a matter of wrapping the Sync Fx APIs in a service layer, so that I would arrive at something like this:

By providing a proxy to the orchestrator, I was able to implement a proxy which is polymorphic with RelationalSyncProvider.

Immediately it should be obvious that this is not your typical integration scenario. Essentially, I am extending a framework by exposing it over WCF so that any orchestrator can leverage the WebSyncProvider.

I started the implementation by writing some tests. It was simple enough to extract an interface for the RelationalSyncProvider type because it is abstract. I used RhinoMocks to stub it, which worked great but where I ran into problems was in recording expectations on the return types because there are not a lot of test seams in the framework. With the absence of setters on main properties that I needed to assert, I needed a way to reach into the framework types and manipulate them so that I could develop the CUT (in this case the ServiceContract implementation) independent of the framework.

Publicize.exe works pretty nicely for what I was looking for.

For example, let’s say I want to stub a closed/framework type so that I can actually write my test without having to resort to integrating with the framework (which wouldn’t be a unit test at all), in this case, a couple of types in the Microsoft.Syncronization.dll assembly.

I run Publicize.exe Microsoft.Syncronization.dll and it generates a shadow assembly called Microsoft.Syncronization_Accessor.dll, which takes all types, adds _Accessor and exposes all non-public members as public as shown in Reflector on the right.

This pretty powerful.

I can now stub the return type of any closed/framework class and test as long as I am asserting *properties*, which is pretty common:

1:

   2:          [TestMethod]

   3:          public void GetChangesShouldReturnNumberOfChangesApplied()

   4:          {

   5:              MockRepository mockRepository = new MockRepository();

6:

   7:              RelationalSyncProvider provider = mockRepository.Stub<RelationalSyncProvider>();

8:

9:

  10:              using (mockRepository.Record())

  11:              {

12:

  13:                  // Set up the out parameter on RelationalSyncProvider Stub

  14:                  DbSyncContext changeDataRetriever = new DbSyncContext();

15:

  16:                  // Set up ScopeProgress property on DbySyncContext instance

  17:                  /*

  18:                   * Note that in order to test for number of changes applied, we need to reach into the

  19:                   * DbSyncScopeProgress and access the _totalChanges private member

  20:                   * so that public properties can be asserted. This is because the TotalChanges property only has a

  21:                   * getter, so the only way to influence the value is through it's backing field.

  22:                   */

23:

  24:                  DbSyncScopeProgress dbSyncScopeProgress = new DbSyncScopeProgress();

25:

  26:                  // Reference the actual object which will be accessed by the generated accessor

  27:                  PrivateObject obj = new PrivateObject(dbSyncScopeProgress);

28:

  29:                  // Generated accessor which wraps the type to be exposed using reflection.

  30:                  // Instead of writing the reflection code manually, Publicize.exe was run against the real assembly,

  31:                  // which in turn generated reflection wrappers for *all* types.

32:

  33:                  DbSyncScopeProgress_Accessor dbSyncScopeProgressAccessor = new DbSyncScopeProgress_Accessor(obj);

34:

  35:                  // Look ma, I am accessing a private field

  36:                  dbSyncScopeProgressAccessor._tablesProgress = new List<DbSyncTableProgress>();

37:

38:

  39:                  // Now we need to repeat the same process again for this type, which is at the heart of what we need to

  40:                  // gain access to.

  41:                  DbSyncTableProgress dbSyncTableProgress = new DbSyncTableProgress();

42:

  43:                  // Provides the accessor with a reference to the actual object

  44:                  obj = new PrivateObject(dbSyncTableProgress);

45:

  46:                  // Generated accessor which wraps the type to be exposed using reflection.

  47:                  // Instead of writing the reflection code manually, Publicize.exe was run against the real assembly,

  48:                  // which in turn generated reflection wrappers for *all* types.

  49:                  DbSyncTableProgress_Accessor dbSyncTableProgressAccessor = new DbSyncTableProgress_Accessor(obj);

50:

  51:                  // Set the _rowsApplied field so that the stubbed return instance of DbSyncScopeProgress can be asserted.

  52:                  dbSyncTableProgressAccessor._rowsApplied = 42;

53:

  54:                  // Add the dbSyncTableProgress to the private collection

  55:                  dbSyncScopeProgressAccessor._tablesProgress.Add(dbSyncTableProgress);

56:

57:

  58:                  // Now, set the property to our used and abused instance of DbSyncScopeProgress

  59:                  changeDataRetriever.ScopeProgress = dbSyncScopeProgress;

60:

  61:                  // Provider is implemented in COM so out params are prevelant (yuk)

  62:                  object changeDataRetrieverObj = changeDataRetriever;

63:

  64:                  // Set the expectation

  65:                  provider.GetChangeBatch(0, null, out changeDataRetrieverObj);

66:

  67:                  // Initialize params

  68:                  SyncIdFormatGroup idFormats = new SyncIdFormatGroup();

  69:                  SyncKnowledge destinationKnowledge = new SyncKnowledge();

  70:                  ForgottenKnowledge sourceForgottenKnowledge = new ForgottenKnowledge();

71:

  72:                  // Stub the return, including the object reference to DbSyncContext

  73:                  LastCall.Return(new ChangeBatch(idFormats, destinationKnowledge, sourceForgottenKnowledge))

  74:                      .OutRef(changeDataRetrieverObj).IgnoreArguments();

  75:              }

76:

  77:              IRelationalSyncManager manager = new RelationalSyncManager();

78:

  79:              // Inject the stub

  80:              manager.Provider = provider;

81:

  82:              SyncKnowledge localKnowledge = new SyncKnowledge();

  83:              DbSyncContext returnedContext = new DbSyncContext();

  84:              returnedContext.ScopeProgress = new DbSyncScopeProgress();

85:

86:

  87:              var changeBatch = manager.GetChanges(0, localKnowledge, out returnedContext);

88:

  89:              DbSyncContext context = returnedContext;

90:

  91:              Assert.IsTrue(context.ScopeProgress.TotalChangesApplied ==42);

92:

  93:          }

There is a bit of work here, but with a bit of elbow grease thanks to Visual Studio Test Tools, RhinoMocks, reflection and a little code generation the framework is testable.

Phoenix Visual Studio Team System Big Event was BIG!

Rick G. Garibay — Tue, 12 May 2009 00:29:16 GMT

Thanks to all who came out for the Visual Studio Team System Big Event on Thursday, May 7th! We had a great time with the Microsoft Developer Evangelist team, Microsoft Technical Specialists, MVPs and influencers who at the end of the full day of action packed training left no doubt that Visual Studio Team System is BIG!

Dan Weinman, Senior Consultant at Neudesic kicked off with the keynote on "Development Best Practices & How Microsoft Helps" and set the stage for what the day full of training had in store. Dan talked about the various roles that VSTS supports and how each role comes together to enact process under a single tool that maximizes transparency and communication on Agile teams.

Next, Brendon Birdoes, Principal Consultant at Neudesic delivered a great talk on Test-Driven-Development with Visual Studio. Brendon covered unit testing, including test first development and showed off the refactoring capabilities right within the IDE along with code coverage. Brendon talked about how important code coverage and quality assertions are to ensuring that your unit tests are as useful as possible in maintaining high degrees of quality within your software projects.

Following the excellent session on TDD, Visual Studio Team System MVP Dave McKinstry delivered a session entitled ""It Works on My Machine!" Closing the Loop Between Development & Testing" which discussed the close integration between testing tools in Visual Studio Team System and the code itself which minimizes the cost and complexity in task switching.

We were then treated by another excellent talk by Rob Bagby, Developer Evangelist on "Treating Databases as First-Class Citizens in Development". In this session, Rob made clear that database work can no longer hide from unit testing and how the Database Project in Visual Studio Team System makes synchronizing DDL and DML objects seamless and eminently testable.

I was lucky enough to show off the new Visual Studio Team System 2010 Team Architect bits which include fully compliant UML 2.1 diagram support as well as assembly, namespace, class and method visualization using the VSTS 2010 Architecture Explorer. I talked about VSTS 2010 Team Architect being yet another clear signal of Microsoft's commitment to modeling which is, and will continue to manifest itself in technologies like Windows Workflow, BizTalk Server, Entity Framework, and Quadrant to name just a few. As a Connected Systems guy, VS2010 Team Architect is really going to go a long way in helping me and my teams visualize the problem domain and partition our big balls of mud into the right architecture.

Speaking of which, I think my favorite new modeling feature in Team Architect is the Layer Diagram which lets you define your layers and then validate them at build time! What this means is that I can use a layer diagram to define separation of concerns and single responsibility principle and then make sure that team members are honoring the layer rules. It is just awesome to be able to map namespaces to a layer diagram, wire it up to the automated build and keep violations from every being delivered.

The day winded up with a great talk by Steve Lange, Technical Specialist on how to get the most out of TFS. The session was aptly called "Bang for Your Buck" Getting the Most out of Team Foundation Server, and Steve gave away some great tips on things you can do with your investment in TFS that you may not have even known about.

I want to thank Steve and Rob for inviting us to speak at this event. The room was full of great folks with lots of passion and great questions which is exactly what we like to see at these events. If you have any follow up questions that I wasn't able to get answered at the event, please don't hesitate to contact me.

SOA Patterns Book

Rick G. Garibay - The more I learn, the less I kno — Sat, 01 Mar 2008 14:28:11 GMT

Excellent SOA Patterns book underway with a public preview and collaboration site here: http://www.soapatterns.org/

Several of us at Neudesic have either contributed, are reviewing or are contributing candidate patterns based on proven practices.

I would encourage anyone to check out this resource and add it to your architecture and design toolbox.

Managed Windows NT Services, Application Domains & Principal Policy

Rick G. Garibay - The more I learn, the less I kno — Thu, 10 May 2007 17:34:58 GMT

Update to orginal post: After doing some more research, the actual problem turned out to be that the legacy Windows NT service never specified the WindowsPrincipal and had nothing to do with the Gateway assembly. In fact, it turns out that when you load a class using Activator.CreateInstanceFrom(), the class loads into the same parent application domain. I've updated the solution below appropriately.

Thanks to Rick Strahl for clarifying the Activator.CreateInstanceFrom(), app domain behavior. You can also find an excellent article by Rick in CoDe magazine that covers this extensively: http://www.code-magazine.com/article.aspx?quickid=0211081&page=1

- - -

I recently ran into a security problem in a somewhat typical scenario for which the specific details I imagine are fairly esoteric because I was unable to find anything relevant on the web or on MSDN forums. It is amazing how easy it is to forget (or at least suffer from delayed recall) fundamental aspects of the framework when you are in the thick of it and trying to ship a product!

So, I figured I’d post an overview of the scenario and the solution to the problem in hopes of helping anyone else who runs into this.

Scenario

A common approach for setting up any kind of automated process is to use a Windows NT service which is continually running, monitoring a queue or database for records/messages that represent events for wich some work should be performed. For example, I can have a Windows NT Service monitor a private queue every 5 seconds, and when a message is available, process it (whatever process means for the given service).

One option for the processing bit would be to raise and event and delegate the work out to a method that would maintain all of the event handling logic. This handler could reside in a single class within the same assembly as the service, but for obvious reasons this could quickly get unruly and introduce a maintanance nightmare. If every time you need to change the event handling business logic you had to recompile and redeploy the service executable, although you might be building job security, you'd find yourself getting frustrated at best and at worst, you would be tightly coupled to a single assembly making it difficult to leverage this pattern in a more abstract, loosely coupled manner.

One possible solution might be to apply a typical publish and subscribe pattern where the Windows NT service polls a backing store (or ideally was notified automagically) and when a record (or message) meets a specific criteria fire an event. The event would then be mapped to a delegate which implements the handling of the event. However, to keep the implementation of the handler from being static and hard coded, we could further delegate the implementation of the work to an external component, making the original event handling delegate only worry about the plumbing for dispatching the work to another component. Ideally, this would allow a plug and play approach where assemblies could be "dropped" in and just work. Of course, this would require that the message in a queue has some kind of metadata about the component to execute (such as assembly name, type and URI) and when the event is fired, the handler gets access to this URI and dispatches the call to it.

This is precisely what the System.Activator class allow you to. If you know the name of the assembly that contains the component you want to call, simply call the CreateInstanceFrom method on the Activator class. The CLR will then probe the local private directory for the assembly name, load it and instantiate an instance of the type. Figure 1 below summarizes what this might look like.

Figure 1: Mock Sequence Diagram

When the IGateway object is “unwrapped” and made available to the MyNTService Windows NT Service, the service can call a method (or methods) on the instance to do some work. To achieve polymorphism, since the Gateway component that is unwrapped implements the IGateway interface, the service is guaranteed to be able to call a particular method (Execute, in this case).

Figure 2 (below, right) provides a possible object model that supports this design.

The purpose of the Gateway class is to serve as wrapper around a component/service call while maintaining location transparency. This can thought of as a "super proxy" and is really just the classic Service Gateway/Service Agent pattern. The MyNTService Windows NT Service has no idea if the component is local to the machine or process or somewhere out in the cloud. With this flexibility comes significant power in being able to make decisions on distribution in a post-deployment manner.

As the saying goes “with great power comes great responsibility”. Looking again at the sequence diagram in Figure 1, two important things need to be considered.

First, this scenario demonstrates WS-I Profle interop between a WCF service and a managed Windows NT Service running on .NET 1.1 CLR (pretty cool huh?).

Second, between every call in the sequence is an authentication and authorization boundary. Though impractical, the Gateway component could ensure that only the identity of the Windows account running the MyNTService Windows NT Service can call it’s Execute method. More importantly, however, the Gateway should flow the identity of the Windows account running the MyNTService Windows NT Service process to the WCF service that is out in the cloud so that the WCF service can authenticate and authorize the call. Pretty simple,right?

Problem

Well, one of the things that may not be overly apparent is that the assembly in which the Gateway type that implements the IGateway interface resides will actually load in the same application domain that exists within the host NT service process. The MyNTService Windows NT Service knows NOTHING about the Gateway component (other than that it implements a standard interface). When an instance of the Gateway component is activated, we want to run in the security context of the parent process (of the MyNTService Windows NT Service ), however if the host application domain has not specifically set the the Principal Policy, the component will execute with no security context, or to be a bit more technical, will load using the default UnauthenticatedPrincipal enumeration flag. This basically means that the principal under which the MyNTService Windows NT Service is running will not be attached to the thread on which the Gateway component runs. In fact, as the MSDN documentation for the System.Security.Principal.PrincipalPolicy enumeration states “Specifies how principal and identity objects should be created for an application domain. The default is UnauthenticatedPrincipal.”.

Solution

Those keen to the default behavior regarding application domains and Windows Principals already know where this is going, but for those who aren’t sure, stay with me.

Although the MyNTService Windows NT Service was running as a fixed domain account, the WCF service requires transport level authentication and uses role-based authorization to ensure the caller (MyNTService) is authorized. However, calls from the MyNTService Windows NT Service via the Gateway were failing with "The request failed with HTTP status 401: Authorization Required."

I triple checked IIS security settings. My unit tests worked just fine. My test harness also worked and even browsing to the service’s metadata page was working. For some reason, the Gateway was calling the WCF service in a seamingly anonymous manner. This threw me for a loop for a couple of days.

Then, a few days ago, while in the shower (I get many of my “light-bulb on” moments in the shower or in my dreams for some odd reason) the solution hit me like a thunderbolt! The Gateway assembly was being loaded without an authenticated principal, and hence the solution was simple, sweet and elegant:

AppDomain.CurrentDomain.SetPrincipalPolicy(System.Security.Principal.PrincipalPolicy.WindowsPrincipal);

Now the Gateway took on the identity of the Windows AD account that was running the Windows NT Service process and was able to authenticate to and be authorized by the WCF service.

You can find more info on principal policy and application domain behavior here: http://msdn2.microsoft.com/en-us/library/90395801.aspx

Taking the Agile Plunge? Hire a Competent Architect

Rick G. Garibay — Mon, 12 Mar 2007 19:15:49 GMT

Before the agile community swarms my home with pitchforks and torches, let me get one thing straight right now: I have nothing against agile development practices (ADP) or theory and do not dispute their efficacy when applied reasonably and pragmatically. I have learned much from, and have a deep respect for many agile evangelists such as Martin Fowler and Kent Beck, and more recently Mike Cohn of Mountain Goat Software. In fact, I believe that many of the engineering practices purported as “agile” are simply a handful of best practices that have emerged from modern software engineering. I stand behind, and use many of these practices in my own work and continue to learn from many of the aforementioned visionaries (although I so wish we could banish the term "agile" as it has become such a polarizing term in software process community).

But while I believe that the tenets of the Agile Manifesto are noble and well intended, I can’t help but wonder if these tenets are being somewhat misinterpreted by the community at large and should either be clarified or ratified.

One questionable trend that I believe we are witnessing in the agile gold rush is the wholesaling of agile methodologies into canned process frameworks that are peddled like snake oil to desperate and unsuspecting souls. These frameworks are often deployed very prescriptively, as if to “cleanse the masses” and often result in a carte blanche implementation where previous learnings are encouraged to be forgotten or disregarded.

In fact, I have seen teams adopt agile methodologies with such zealotry that they cling to every letter of the tenets as if the words were inscribed on a marble tome. Worse, they then go beyond the written guidelines to form their own interpretations which are then applied conveniently to meet their needs. This is a dangerous practice that is common to all religions, and the fact that I am even describing a software methodology in this context suggests that things are getting a little ridiculous. The industry doesn't need to be born again, it just needs a unified representation of how to accomplish the job by doing more of the right things and a reminder that no one is smart enough to figure the whole puzzle out before getting their hands dirty. This doesn't mean you scrap everything for this new snake oil- it means you identify some key areas of opportunity and lend some good old fashioned process improvement.

There is an exception to this, where wholesaling ADP does make sense, and that is in organizations that have very little understanding or appreciation for the engineering aspects that drive product development. These are the companies that are 100% date and sale driven and blame IT for all of the failed projects because they lack the transparency to identify the real problems keeping them from meeting their objectives. But few of these organizations will actually stick it out because the cultural transformation is often insurmountable (then again, one would have to ask themselves why they would stick around in this environment to begin with)?

But the biggest problem I see developing within the agile movement -that has gone largely unchecked- is the de-emphasis on so called up-front design and architecture, a key component that I believe is fundamental to (successful) software engineering. Although it is not stated anywhere in the Agile Manifesto to abandon architecture and design, the dismissal of up-front architecture and design as draconian casualties of “traditional” or “waterfall” methods is a common thread within the agile community. As an architect, when I challenge this point (and trust me, it happens all the time), too often tenet # 3 (“Working Software over comprehensive documentation”) is used to support this belief that up front design is the modern leprosy of agile development. Martin Fowler has done some work to help dispel this myth, but there is still much work to be done around educating the community on this topic, and I am confident that many reading this actually firmly believe that all so called up front design (BDUF) is evil.

The reality is that this tenet is being used as a fig leaf to cover up the real reasons for not doing the right things, and this has nothing to do with being agile or not agile, but instead points to what I think boils down to a lack of experience and maturity.

Some up-front, pragmatic architecture and design work which sets the foundation for the project's conceptual integrity and then is consistently refined is a sound engineering practice which is the responsibility of a competent architect to perform. Such critical design aspects include layer and corresponding artifiact definitions, assembly allocation, messaging patterns, process allocation, identity management, transaction boundaries and deployment plans- these are fundamental to software projects that you don't need a crystal ball to figure out, just some good old fashioned experience.

Of course, all architecture and design is to some extent evolutionary, but modern software projects of any significance apply this technique as a critical step for gaining consensus, forming conceptual integrity and orienting teams on the work that lies ahead. In my opinion, this step is just as critical to the success of any non-trivial endeavor as writing unit tests and automating the build process. Skipping it will almost always yield lost time, productivity and result in a fragile product, regardless of how agile or motivated the team may be.

There is nothing written in stone that says that all BDUF is bad, and I believe that there are a fair amount of Agile projects that do put this critical step into an architecture/core-foundation iteration, but I just wish that the Agile community at large would distinguish this a bit better so that the lay person interpreting the impact of ADP has a more educated perspective. This is certainly an optimistic perspective, and for as many moderates on this subject, I am sure there are radicals, but the I think that the kind of thinking that rationalizes the omission of this practice threatens the software industry’s ability to evolve as an engineering body of knowledge and will further demarcate the gap between enterprise software and toys.

Worse, I believe it will continue to prevent the agile movement from ever graduating to a mainstream, disciplined SDLC process, instead being forever relegated as a fringe software development process. This is unfortunate too, because I think ADP has a real potential for forming the bedrock of what modern software engineering practices look like and mobilizing the industry to do more of the "right things", lest we skip the critical design aspects.