thinking-out-loud

FABRIQ Released

Clemens Vasters announced the release to the wild of the Fabriq sample that newtelligence has been working on with Microsoft EMEA, et al.

I downloaded the bits and have started reading through the code. Clemens explains the architecture and configuration in several recent posts, so you probably want to start there if you didn't get to see this stuff at the various Tech Eds.

One interesting observation is that, though you will find several references to some of the wse namespaces such as Addressing, Security.*, and the base namespace, you will not find references to the WSE messaging layer. Fabriq roles its own transport(s). I have recently come to that decision myself. I have been trying to implement a msmq-based, reliable, transacted transport for wse 2.0. I basically gave up the ghost Friday, after a week of load testing revealed nit after nit that made the code far more complicated than it really needed to be. I basically reimplemented the whole transport in a much more dumbed-down, straight forward fashion, and it took a lot less code. I am going to try to take some time to write up the experience notes, and I will publish them here when I get a chance.

One Nice-to-have would be for the code to be structured so that the architecture was as clear in the projects as it is in the documentation, e.g. adding sub-namespaces to the Core project to package up the concepts, maybe MessageHandler, Pipeline, Node, Action, Router, Transaction, Transport, IO, Utility. It would improve understandability, I think. FWIW -- I tend to use the [most likely fabricated] 7+- 2 rule in packaging things. If a namespace has 10+ classes, I strongly consider reorganizing it. Most of the time, this ends up making the code much better, if only because it is easier to understand. This is what I have always called Extract Package, stealing the pattern naming convention from Martin Fowler. Apparently, I am not the only one. Patterns are like that.

More .NET TDD and Reflector

In my recent post on TDD (and other .NET) resources, I mentioned that I would like for Reflector to be a VS.NET add-in. If I had a dollar for every time I posted something like that and then realized that it was already available later the same day, I would have *at* *least* ten dollars! Check out the .NET Reflector Add-ins GDN Workspace. Silly me.

Might want to check back soon, because there appear to be some recently checked in files by JdH related to the type treemap. Treemaps are a very cool visualization technique. I think I have posted this before, but here it is again:

One of my favorite implementations of the treemap visualization technology is sequoiaview. SequoiaView uses "squarified treemaps," which try to keep the rectangles as square as possible. Another interesting one: SmartMoney.com's MarketMap.

Treemaps seem like one of those hammers that turns everything into a nail. One idea I had along those lines was to create a treemap view of a small world network where the tree is organized by the number of relations.

Congratulations

They made it! Very exciting stuff. Having dreamt, like many children, of one day being an astronaut, I feel honored to be part of the time when this is possible.

.NET TDD Resources

I was doing a little browsing and ran across several items of interest.

First, let me join the voice of many who love Lutz Roeder's .NET Reflector. This tool absolutely rocks. I wish it were a visual studio add-in replacement for the "Go To Definition" and "Go To Reference" options, because it is sooo much nicer.

There is an add-in (yes, Reflector has an add-in api!) by Jonathan de Halleux called Reflector.Graph. You can see some snapshots of what it can do on his blog. His blog is excellent reading material. Apparently, Microsoft took notice, because they recently hired Jonathan.

Advanced Unit Testing is a collection of tools, practices, and patterns for unit testing, especially focused on .NET. Core to AUT is the MBUnit testing framework by (again) Jonathan de Halleux. Jonathan also has a production grammar framework, some graphing libraries (implemented on top of AT&T Research's GraphViz), unit test generation tools, and more. Very, very good stuff.

Marc Clifton created AUT, and he has a number of articles on Unit Testing over at Code Project. Good stuff as well.

Finally, Roy Osherove has an interesting article that extends some ideas ideas presented in James Newkirk, et al.'s Testing in Microsoft.NET to leverage Enterprise Services in database testing to rollback any changes that are the result of the test.

In my experience, even in organizations that are aware or actively trying to adopt test-driven development or *at* *least* implement decent regression test suites for their applications, there is a constant struggle to develop the tests. In my experience, the amount of test code necessary to test a component well is 2-3 times the amount of code necessary to implement the functionality. it is good to see so much activity in this area in terms of helping people understand best practices and providing tools to automate some of the more redundant bits can help narrow the gap. Of course, tools alone won't necessarily make the job easier. You must properly apply them. But all of the infrastructure certainly lowers the barrier to entry to increasing the quality of code.

I was talking to a one-time cohort about TDD a while back, and he influenced how I think about it. He basically said (paraphrased) that TDD is not so much a testing method or a coding method as it is a *design* method.

I hope you had a nice weekend.

Generic WSE Soap Router Pt. 4

OK -- here is the latest cut. Straight inline. Sorry if the format sucks. Once again, I didn't compile it, but it is a moral equivalent of what I am using successfully in a project, so there ya go. I am still interested in monkeying around with the peer-to-peer, federated referral stuff, so I will keep you posted. I am thinking ip multicast wse transport with really tiny messages.

mmm. multicast.

 
 public class AnyRouter:SoapReceiver
  {
    public AnyRouter()
    {
      Pipeline.IsIntermediary = true;
    }

    protected override void Receive(SoapEnvelope envelope)

    {

      Uri newAddress = ReferralCache.ResolvePath(envelope.Context.Addressing.To.Value);

      if( newAddress == null )

      {

        throw new AddressingFault(

          AddressingFault.EndpointUnavailableMessage, 		

          AddressingFault.EndpointUnavailableCode );

      }

      SoapSender sender = new SoapSender(new EndpointReference(

	envelope.Context.Addressing.To.Value, 

	newAddress));

      sender.Pipeline.IsIntermediary=true;

      sender.Send(envelope);

    }

  }

Trail of Dead

I have promised but not delivered on posting about rock-n-roll. You may know this already - they have been around for quite a while - but And you will know us by the trail of dead rock. They ROCK. I am revisiting their excellent album Source Tags & Codes. Pitchfork gave it a 10.0, we-dis-god-for-indie-street-cred mongering sonsabitches, so that should mean something.

If you haven't heard of them before, no thanks necessary. You are welcome. Very welcome indeed.

We now return to our regularly scheduled program.

Generic Soap Router Pt. 3

I fished, Steve bit, and then fought back with a "don't you go using channels directly," um, smack down (sorry for mixing fishing and outdated urban street cred metaphors, but I *am* from Texas). And he is right, excusing the bit about it being a bug. I was using the channel directly because of The Problem. The Problem was that using SoapSender resulted in a seriously bizarre set of soap headers, with problems including duplicate was:Action elements. Now we're talking bug! Of course, the *Real Problem* was that I didn't set IsIntermediary = true on the *Sender's Pipeline*. Silly me. This resulted in a very similar "slap my forehead, I already thought of that, why didn't I do it for both" sort of reaction. Reminds me of the old joke:

patient: "Doctor, doctor, it hurts when I hold my arm behind my back like this..."
patient demonstrates.
doctor: "Well, stop doing that."

I added the IsIntermediary bit to the sender's pipeline, and voila: Success. Consider my version revised. Thanks, Steve.

Now.

He poses a couple of questions that are interesting aspects of intermediary design.

Is trust point-to-point, or end-to-end?

Of course, being a f'in dork and a wanker consultant to boot, I realize that the only appropriate response [that could contribute to additional billable hours] is "It depends." There definitely isn't a one-or-the-other answer to this one. The question depends on the requirements, e.g. how the trust relationship is defined--is it end-to-end or point-to-point? Sorry to answer the question with the same question, but as far as I am concerned, that *is* the answer. I also think it begs additional questions about general intermediary design -- routers that do nothing more than route, in my opinion, should not modify the envelope. As I understand it, that one of the primary motivating factors for moving away from WS-Routing to the next-hop routing and WS-Addressing. Other intermediaries that route as a byproduct rather than as their core competency may or may not need to add their mark to the message, depending on role. I think the answer to the question depends on how far up the stack the role of the intermediary goes. More on this later. Back to work. I would be really interested in any comments anyone has on their experience with intermediary design with respect to how the messages are handled. (ahhhh, Fishing).

Generic WSE Soap Router Pt.2

I did try out the code I mentioned -- it worked with a caveat.

I needed to change:


envelope.Context.Addressing.Via = newAddress;
ISoapOutputChannel channel
= SoapTransport.StaticGetOutputChannel(envelope.Context.Addressing.Destination);


to something like:


envelope.Context.Addressing.Via = newAddress;
ISoapOutputChannel channel
= SoapTransport.StaticGetOutputChannel(new EndpointReference(envelope.Context.Addressing.To.Value, newAddress));


This may not be the best approach, but it does work, and I don't have to clone the soap envelope, which is nice for response time and memory utilization reasons, as I mentioned before. I don't have the code in front of me, so please forgive if something isn't quite right. You should get the general idea. I want to add async sending and correlation of responses, but I need to think through the ramifications a bit. I am also thinking more about the multi-hop routing. I have a need for multi-hop forward-only routing. I don't have to worry about chaining back through the intermediaries. I would like to provide the facility to reply back to the originating systems, but that would be short-circuited, rather than traveling through the intermediaries. Unfortunately, the politics of the design aren't allowing for it. I think it will come back to haunt us a bit, but the world won't end if it does. They'll just have to do the billing by hand when it screws up :)

Generic WSE Soap Router

Caveat Lector: The code that I mention, though it compiles, hasn't even been executed, and certainly hasn't been tested.

Steve Maine has a number of good posts on WSE 2.0, including a couple of detailed entries on his attempts to write a transport-independent, next-hop routing intermediary. His first article highlighted several interesting problems:

Routing Roadblock #1: What endpoint is my router listening on, exactly?

The answer involves WSAddressing.AnyRole

Routing Roadblock #2: “This envelope’s already been processed!”

He basically does a clone of the envelope. I am thinking about a different solution from his, because I would like to avoid the memory and service time hit. I skirt the issue on the inbound side by setting the Pipeline.IsIntermediary to true.

Routing Roadblock #3: What to do about MessageId?

WSE is pretty picky about the messageIds. His cloning approach works, as I am hopeful mine does as well.

The second article digs into the replyTo processing issues. I also have an interest in this, though it is not addressed below. I am thinking that you could dyamically modify the ReferralCache and use ReferralCache.Query(...), but that runs into scalability problems that might require wrapping the referral cache in a service interface. Then you would have to deal with clustering/federating the referral services, which would be a really fun problem to solve, but certainly non-trivial. Graham Glass could probably hack it up in Glue in no time, though. And it would be really quite compact!

I removed the code, because it just looked like crapola. I will include a link.

-------------------------------------------------------------
My [hopeful] code follows. The formatting got a bit screwy, so please forgive the k&r if it isn't your preference. It is not mine either. My approach is to avoid the clone, munge the original soap envelope and mess with the transports and channels directly, bypassing the soap sender side of things.

Marketing is hard, didn't you know?

Kevin Schofield over at Microsoft rants on marketing, how hard it is, and how most techies dis marketing types [case in point].

Marketing is a difficult discipline, that requires both theoretical and practical knowledge. Just as marketing people should respect technical people, technical people should respect marketing people. I think in the long run one of the most important business lessons that people will take away from studying Microsoft is that when you put sharp technical people and sharp marketing people together and things click, you can create a very, very profitable business.

Luke Hohmann has a book title Beyond Software Architecture in the AW Martin Fowler Signature series of books that addresses some of the topics around Product and Marketing Management.

Luke's book provides an excellent introduction to product marketing and management, especially for software architects. He drills down into the method of what may sometimes seem to be marketing madness. Luke is a fairly engaging writer, and I enjoyed the work. I have a couple of nits I would pick: the chapter on logging seems like it is not congruent with the rest of the book, and I was yearning for more insight and depth in the effect of brand on software design than making brand names configurable, but of course that leaves room for book 2 (or 3, really). Well worth the read, even if he ought to lose the beard pic on his site.

Via the scobleizer.