thinking-out-loud

TexMex Issue List, WS-MetadataExchange

As I am releasing more [and more correct] functionality in TexMex, I wanted to present a list of issues/features I know are outstanding or I am focusing on fleshing out:

WS-MetadataExchange Semantics

1. GetMetadata w/ Dialect where no satisfactory dialect exists returns an Empty Metadata element.
2. GetMetadata w/ Identifier but no Dialect should fail
3. Support Multiple MetadataSections with the same Dialect/Identifier pair
4. GetRequest with valid To
5. GetRequest with invalid To

Additionally, I am concerned with the correctness of the semantics of my ws-addressing specific code, as well as ensuring I have all of the namespaces, mime types, encodings and such correct. 

Other things that I want to accomplish

1. WSE support (difficult to be correct given the addressing spec version, but I'll gracefully degrade in the presence of the earlier spec Xmlns)
2. Performance (specifically, caching)
3. Interop with IBM's ETTK -- haven't gotten there yet.
4. TexMex client support -- there is a lot of redundancy in the tests that could be a reasonable basis for a client [set of] classes.
5. DTOs should be useful in either a serialization or a streaming context.
6. (Related to 7) -- a useful EPR class that makes it [relatively] simple to work with either ws-addressing's 2004-03 or 2004-08 schema and ws-metadataexchange's MetadataReference. The 2004-03 requirement is not addressed yet.

TexMex 1.2: WS-MetadataExchange

Another point release of TexMex is available. I have added more tests, refactored the code a bit, improved my XmlReader/XmlWriter code, and fleshed out the EndpointReference support a bit more. One test fails currently as a placeholder for what to do next. It is still definitely alpha quality code, if that, but it is getting warmer to something I could take home to Momma :)

TexMex 1.1

I create a point release of TexMex. I have added support for Location and partial support for EndpointReference. I say partial because I haven't really tested out the roundtrip for EndpointReference. I do think it *should* work, but I will reserve confidence due to lack of empirical evidence.

I also did a quick little code review and cleaned up the code considerably. I was chaining streams in the SoapExtension, and I didn't need to be, and then I was misusing the streams a little to boot. That has been rectified.

Because of how I am choosing to write my soap envelopes to stream (e.g. WriteRaw(string) for the static chunks of the response envelope), the namespaces were not working as I had intended on the non-raw writes. I [am hopeful] that I fixed that as well. I am going to continue to refactor it as I get time.

WS-When

There were a number of good predictions for 2005. Some of my favorites (in no particular order):

• Graham Glass's
• Don Box's
• Rory Blyth's
• Mark Baker's

Mine is slightly more pithy:

• Ubiquitous WS-Splat is yet another year away, though use cases requiring message level encryption and signing may gain.
• Use of Xml over HTTP will continue to flourish despite the fact that the implementors will neither get nor conform to either REST or SO principles.
• Semantic Web is SO+3 years away.
• I will publicly state at least three times that a) I wish I had more time to play with Lisp and that b) despite the fact that I know so little about Lisp, that we should ditch all of this DSL/MDA crap and use it instead.
• Python and Ruby will gain market.
• Static typing will once again be recognized for what it is: a terrible, nasty, but sometimes necessary evil.

WS-MetadataExchange thoughts

At the XmlDevCon, Don Box talked about the 5 specs he would want on a desert island. Implementing TexMex (zip) and re-reading Steve Maine's WS-Kernel post got me to thinking about what Don was saying and what it means. (FWIW, Steve also had a nice post on WS-Addressing and hack.net, the latter of which caused me to lose a good number of hours of my life :).

To recap, Don's Top 5 were:

1. XML
2. SOAP
3. WS-Addressing
4. WS-MetadataExchange
5. The conjoined, evil twins, XSD/WSDL.

Of these, XSD, WSDL, and WS-MetadataExchange reify themselves as metadata. So basically, if you were on a desert island with Don Box, you could ask his services for their schema, their wsdl, and their meta-metadata. Now, how would you ask for that information, and what would you do with it? Well, you might want to generate a client library from the wsdl and schema to access the service. When would you do this dynamically? You might use it to dynamically generate not just client libraries, but also full-blown end-user clients, ala InfoPath or Thinktecture/Christian Weyer's DynWSLib. Business process orchestration/choreography also comes to mind.

It seems like there are two uses for the metadata exchange, one that happens at design/build time and one that happens at runtime. The latter makes me think of how the SSL/TLS handshake occurs: versions and algorithms are negotiated, and the protocol gets on its way. That negotiation allows for extensibility and graceful degradation of service. The analogy can be applied back to WS-MetadataExchange. Today, we use XSD/WSDL. Lots of people openly complain about it, but it continues to be the approach that is widely supported. WS-MetadataExchange opens the door to replace WSDL and XSD with some newer, better approach. If the client supports the new dialect and the server serves it up, everyone is happy.

TexMex: A spicy WS-MetadataExchange Implementation

Tex-Mex.... mmmmm mmmm.

We at Improving are pleased to introduce TexMex, a spicy ws-metadataexchange implementation. As I have mentioned before, it is implemented as an ASP.NET soap extension. The solution contains three projects:

TexMex - this is the core soap extension.

TexMex.Service - a test asp.net web service project.

TexMex.Test - an NUnit (2.2) test project.

There isn't a lot of code, and that is indicative of two things:

1. I wanted to have a really simple solution.
2. I haven't really put much time into it yet :)

In the implementation, TexMex avoids DOM-style parsing in favor of XPathNavigator+XmlWriter. It does support either Soap 1.1 or Soap 1.2, which is to say if you send 1.1 you will receive 1.1, and likewise for 1.2. It assumes that the metadata is stored in files relative to the service itself, that they do not contain an xml declaration, and that they are well-formed and valid for the given dialect. One way to put it, I suppose, is that TexMex treats dialects as completely opaque arrays of [UTF-8] chars.

TexMex does *not* respect all of the WS-Addressing rules yet, in particular, it does not properly deal with ReferenceProperties and ReferenceParameters. I don't think I am handling the replyto exactly correctly, either, but that should be fairly trivial to fix. I hope to get time to remedy the Reference* soon. If someone else is interested in taking it on and submitting changes, let me know. TexMex currently only supports inlined metadata in responses. I think I will amend the solution shortly to support the other two options, url and epr, via additional attributes, e.g. MetadataLocation (...)  and MetadataReference(...). Finally, I would like to try out some very low level (byte-array-style) caching just for grins, because most of the messaging is static content.

To use TexMex, you must declare the soap extension in your web.config:

<configuration>
  <system.web>
    <webServices>
      <soapExtensionTypes>
        <add type="TexMex.MetadataExchangeExtension, TexMex" priority="1" group="0" />
      </soapExtensionTypes>
     </webServices>
...

And then add MetadataAttribute declarations to your service class. The example from the TexMex.Service. This service actually just returns the wsdl and schema for ws-metadataexchange itself, as well as a sample policy from the WS-Policy specification.

using System;
using System.Web;
using System.Web.Services;

using TexMex;

namespace TexMex.Service
{
  [
  Metadata(Dialects.WSPolicy, "TestServicePolicy.xml"),
  Metadata(Dialects.Wsdl11, "metadataexchange.wsdl", Dialects.WSMetadataExchange), 
  Metadata(Dialects.XmlSchema10, "metadataexchange.xsd", Dialects.WSMetadataExchange)
  ]
  public class TestService : System.Web.Services.WebService
  {
    public TestService()
    {
    }

    [WebMethod]
    public string HelloWorld()
    {
      return "Hello World";
    }
  }
}

WS-MetadataExchange update

OK. I have changed my mind. I think I am going to ignore sweating the ws-addressing details (e.g. processing ReferenceProperties & ReferenceParameters) and cut an "alpha" of the WS-MetadataExchange ASP.NET Soap Extension. I wrap up what I have this weekend and post it.  I am also thinking about setting up an endpoint for interop testing just for grins. I don't intend for the interop service to do much, but it will be fun to play around with. Anyway, stay tuned. I promise it is coming soon :)

WS-MetadataExchange Implementation Update

Just a brief update: I am narrowing in on a version of the software that I feel is reasonably presentable. I haven't had much time over the holidays, but I got a little time last night before bed, and so I hacked in the code to allow the extension to be indifferent to soap versions. This may be a silly thing to do given that even WSE doesn't support 1.2 to my knowledge. I have a bit of testing to round out on the versioning to feel comfortable, and then I will move on to ensuring that I am handling EPR reference properties in a conformant manner. I will post the solution shortly thereafter.

Schneider elaborates, still not sure

If you haven't, you should read this first.

Jeff, your heart really seems in the right place. Really. I tend to agree with you, though it may not have been obvious from my posts, that we need to elaborate on underlying principles of service orientation. I don't think that the four tenets are in any way perfect. In fact, in hat tip to your redundancy statement, I would probably claim that 'services share schema and contract, not class" is practically a corollary to the "boundaries are explicit," especially if you use this dictionary definition of explicit:

1. Fully and clearly expressed; leaving nothing implied.

I think my basic points are that the four tenets add value at a certain level of abstraction, and that we need to refine from there. I will have to be brief [last minute shopping beckons],

But:

Bible reference: Is that your "New Schneider Version?" :) Dude, you should look into a good seminary course -- your exegesis could use some help. The first statement stands on its own merit in the original hebrew. Some of the fault lies in our overloaded use of the word 'love' in English. Your point is, none-the-less, taken.

No, you didn't clarify the difference between philosophy and constraints, though you gave some reasonable examples of constraints, and I get it.

You are wrong on the constraint you derived from "Boundaries are Explicit." That isn't what it is saying at all. Nope. No sir. What you are talking about are implementation details at this [I hestitate to say] level of abstraction.

The second part of boundaries are explicit is really, in my mind, more about the "not classes" part of the "schemas and contracts" tenet later. The redundancy shows.

I don't know that I buy your versioning constraint under autonomy. That is a should, not a must. We should be aware of the consequences, but services are going to get versioned out of existence.

Schema and contract doesn't necessarily (in my mind, at least) mean XML Schema, though we are basically stuck with it for now. This could instead be a guiding principle for where schemas need to go to be able to adequately describe services. I don't have much more on that either, though.

I haven't made it all of the way through the MS video yet, but they don't seem to butcher it too much. The point I am trying to make is that you shouldn't be berate the butcher when your over there carving the pig up. :)

Now, that being said, I will try to provide more constructive feedback, following your lead. Just after the shopping and such. In the meantime, I will point you all back to Pat Helland's excellent section on messages from his article on data inside and out, which I think is a wonderful start in the right direction. Scroll down to Table 1 for the brief version.

Autonomy, AI, and Memories of [Travises]

OK. So I was reading Jeff Schneider's reply to my post questioning his post about MS's Ron Jacob's... oh *!@&@^ it! Anyway, while thinking about services and autonomy, I happened across Jeff Sutherland's scrum log, and I was floored by these posts. They seem particularly relevant, but I don't know that I have the picture complete enough to explain anything at all.

Also, while reading Mr. Schneider's post, I couldn't help but be overwhelmed by two childhood (or practically childhood) memories. The tamer of the two was of the first time I visited my high school friends John, Chip, and Von at UT Austin.

Chip and I were in the same class -- Chip was the Valedictorian of our class, a super-smart *and* amazingly nice guy, and (though he may not know it) also responsible for my interest and pursuit of a degree in physics. John and Von were a year younger than me. John was my best friend at the time, and is one of the funniest guys I have ever known. I owe both him and Ronnie a call (in case you're watching). Von is easily one of the most unique people I have ever known, and interestingly, though he, Chip, and I all started out pursing physics PhDs, he is the only one who finished. He studied lasers, and every time I think of that fact, it reminds me of a David Letterman show where there was a discussion of lasers in the context, I think, of cds. Dave said "Don't look at them, them's LAY-ZERS," which still makes me chuckle to this day. Von, wherever you are, you are a better man than me.

Back to the point. When I first visited them at the campus, they were in the dorms at Jester. Jester is a monster dorm on the campus -- I was told that it was so big, in fact, that it has its own zip code. We were heading out somewhere -- dinner, something -- and we walked by one of their neighbor's dorms. I think the guy's name was Travis. He must of been between 200-300 lbs, and he was standing in the open doorway of his dorm, in front of his mirror, performing a slow motion upper cut-to-body blow, while quoting Mike Tyson's Punch Out a little too slowly for comfort "Uuuuppppper ccuuut. Booodddy blooow." We lost it. I wasn't aware of anyone typing the sequence of characters "ROTFL" at that point in my life, but had it been fashionable at the time, and had a neighbor of their neighbor Travis been watching and usenetting at the same time, that is likely what would have described John and me at the time.

Now. As I mentioned, reading Jeff's post evoked two memories. The second is really a collection of memories, all related to [my fear of/involvement in] fighting. When I was a kid, I was smaller than the average person in my grade. When I graduated high school, for instance, I weighed a buck and a quarter, soaking wet. And *that* was after a recent "growth spurt." This was partly due to the fact that I started school early. Call me "eager." My sister was going, and I wanted to go, too. So my parents looked at their budgets, and figured out a way to afford getting me into the pre-school in the school where my sister was going to kindergarten at my ripe age of 3. I seemed to do ok, but I was always small. Because I was small, a wimp, and generally a pacifist, I tended to try to avoid fights. But this isn't about most of the time.

There were four [count them] times in particular where I didn't feel like I had a choice. One time, I got into an argument with a guy named Travis. I don't have a clue what it was about, but something relevant to a group of 10-somethings. Push came to shove, and I seem to recall having hit him, I think in the nose. It pretty much ended at that, which is good, because I think he really could have taken me -- he was an angry kid, and I was a scrawny wimp. Then again, it might not have ended in a punch, maybe just some shoves, but my memory of the time is really fuzzy. One thing I do remember well, though, is what happened just a little while later. We had all met to play soccer near Travis's house before the brouhaha ensued. Post-brouhaha, we kissed and made up, modulo the kissing, and proceeded to play soccer. Well, needless to say, Travis was on one team, and I, the other. At one point, he had the ball, and I was defending, and he reared back and kicked the ball *as* *hard* *as* *he* *could* right at me. And, though I wouldn't admit it at the time, it had hurt. Worse than that, I am pretty sure that I had closed my eyes, turned my head, held up my arms, and winced like a little girl at the oncoming cannon ball he had hammered at me. But the pain and disgrace soon faded as the ball deflected off my shin and into his neighbor's window, shattering it into a million pieces. It was pretty obvious that he had kicked it like he did out of pure spite, and he ended up being responsible for the repairs. His dad was actually seemed kind of a jerk about it, and I always felt sorry that he had such a mean seeming dad. Anyway, that day, I somehow came out on top.

The second fight of any consequence involved a kid shooting a b-b gun at me and my friend Josh. Long story short, the kid was younger and smaller than me, and so it wasn't difficult for me to, um, take him out. I ended up with him pinned on the ground, me straddling him, asking him to apologize, and then hitting him in the face when he wouldn't. The sad part is that I really couldn't bring myself to hit him that hard. I didn't really want to hurt him, but I didn't like the fact that he shot at Josh and me.

While I was "pummeling" this poor kid (who was a royal ass, and I still believe he deserved what he got), one of his friends ran off to get another of their friends. The friend he went to get turned out to be none other than the Travis kid from above. Several years and a number of karate classes had honed his skill and focused his anger, which he then proceeded to direct at me. I tried to persuade him that it wasn't his fight, and that the kid deserved it, but to be honest, I think he was still bitter after all those years. He threw a number of kicks at me, and he actually hit me with a roundhouse kick -- and it jammed my thumb. It hurt pretty bad, and I backed down (my usual response as a child to any sort of conflict -- I could fill this blog with *those* entries).

I later learned that Travis was afflicted with a terrible disease late in his high school years, and had to spend a lot of time in the hospital. I don't remember exactly what it was, but I remember that it involved a major organ and was life-threatening. I hear that he actually expressed considerable regret for being such a bully in his younger years and stuff. It made me feel awful for despising him so much as a kid.

There was another time that is worthy of it's own postThe short version: my friend Josh had moved into an apartment complex, befriended a speed and lsd dealer from Boston named Pat who claimed to have robbed banks before. While I visiting Josh, Pat somehow got the mistaken impression that I had made inappropriate suggestions to his girlfriend, and he decided that the best way to clear it up was to deck me. Hard. Like "knock me to the ground, big swollen black eye" hard. Again, though, that is worthy of its own post.

Oh, actually, maybe the appropriate number is five, no, six! I didn't actually get into a fight for one of them -- it did involve my friend John, Deep Ellum, A Cure concert, two *big* black guys, and me screaming somewhat like a girl. That also deserves its own post, so you will have to wait. The other involves the aforementioned Ronnie, a *late* night, and a long story. That one might not make it into a post, but it *is* at the same time funny and regretful.

The final "fight" I got into was when I was in college. As you can tell, I have lost the energy to type it all out. To sum, I kept getting threatened by this guy who was another person who wore his anger on his sleeve, we were playing sand volleyball, and he didn't like what I was saying, so he threatened to kick my ass. I kept backing down, and then kicking sand on him when he walked away or shooting my mouth off (I was considerably more arrogant/Stupid at whatever age that was -- 18-20?), and finally he came at me in such a way that I thought he actually meant it, and so I hit him. As hard as I could. Square in the nose. And it *bled.*

He stood there. I stood there.

Nothing else really happened.

He said something like "I don't want to get kicked out of the park for fighting. But mark my words..."

Ok. Maybe he didn't say "Mark my words." But it was something like that. To tell you the truth, I was scared poopless. But I didn't back down, and calmly went to the side of the court, dusted off the sand, got dressed, and left. In a strange coincidence, this guy ended up dating and I think eventually marrying a girl I knew fairly well from my first job out of college, which was about 75 miles away. Small world.

Speaking of small world, Jeff (may I call you Jeff?), I got an IM from Greg Vaughn mentioning you because of your post. Small world.

I am soooo glad that I am no longer < 25. It was all a lot of fun, even the scary stuff, but I am sooo glad...

I do think it is worth mentioning that not all Travises are bad. In fact, I have a wonderful nephew named Travis that is a really cool kid. He taught me how to play pokemon on his Gameboy oh so many years ago. Good times. Good times.

Now, when it comes to services, I will duke it out! Buhhhhh-ring it! Four Tenets are good if they are spot on. If we have to get to 100, then I think that something is wrong. Leave precision for the specs. Principles and prescriptions don't have to be precise to be effective guidance. For instance, "No business logic in servlets" is a useful prescription, even though the term "business logic" is far from precise. So, I still like the four tenets. Do we need to elaborate? Sure. But they are still, imnsho, spot on. So, in the new year, keep your boundaries explicit.