Mapping Code Pairs in the WABS Mapper

By Nick Hauenstein

This post is the twenty-third in a weekly series intended to briefly spotlight those things that you need to know about new features in BizTalk Server 2013.

Assuming that one of the major benefits of using Windows Azure BizTalk Services is being able to offload some of our EDI processing to the cloud, then a major use case for the new WABS mapper becomes EDI mapping. One of the fun tasks that comes out of that is dealing with code pairs (if you’re interested in the on-premise side of this problem, the best coverage to date is found here). In this post, I’m going to examine how one might approach this problem using the new tooling available in the WABS SDK.

Give Me an Example of Code Pairs

Maybe you’re reading this and thinking to yourself, “I don’t really care about EDI, but WABS looks interesting,” and you still want to be able to gain something from this discussion without getting buried in the esoterica that is EDI. If that’s the case for anyone out there, here’s an example of the problem I’m going to try to solve using the WABS mapper.

We have an input that is a repeating record of items where there is a qualifier that describes that type of item we’re looking at, and a value that provides the value of the item. Consider this schema:

image

Here we have (at some point in the schema) a list of phone numbers, identified by a Type field element (our qualifier) and the Number field element (our value). The Type field is defined as follows:

image

We have an output that is a flat listing of items where we have a set of fields representing a few of these different types of numbers:

image

So what we are trying to do is iterate through the list of numbers in the input, map over the value for the item with the qualifier Mobile for the CellNumber output, and grab the value of the Home qualified number and toss it into the HomeNumber output field. It’s something that looks trivial, and yet is annoying to do en masse using the BizTalk Mapper:

image

In a Land Without FLATTENING Value MAPPERS

So what does this look like when brought over to the land of WABS using .trfm style maps? Well, looking into the toolbox, we do have an item called Conditional Assignemnt, which is essentially a Value Mapper:

image

If we run with that (combined with some Logical Expression map operations, we end up with something like this:

image

Unfortunately, this isn’t a Value Mapper (Flattening) type map operation. As a result, this actually will only work if we are interested only in a single qualifier, and that qualifier happens to show up in the first code pair that appears in the document.

So what do we do now? Do we reach to XSLT and code it up by hand (after-all, it is super easy to pull off)? Let’s take a look at how the migration tool that comes with the SDK handles this scenario:

image

It adds two MapEach operations to the map, that are configured something like this:

image

As you can see, the MapEach was made to be conditional upon the Type field element being compared to a constant string input. However, this conditional check is repeated inside the MapEach operation within the chained Logical Expression and Conditional Assignment map operations. While the map technically works as generated by the tool, it looks horrible, and it really feels like there ought to be a better way.

Another Approach

If you don’t want to use the MapEach approach (which really wouldn’t be so bad if our two nodes weren’t hanging out at the root as siblings), then we can take advantage of the ability of the new mapper to generate lists of items in memory while working through the mapping (really this is kind of similar to the Table Looping and Extracting functoid).

In order to make that happen, we use the Create List map operation, and then at some point inside include the Add Item operation to add items to the list. The Create List operation is configured to include the names of the fields of each item in the list. It then includes the ForEach operation with a nested Add Items. The ForEach is pointed to by the parent repeating node of our code pairs. The Add Items operation is pointed to by each value we want to include for each source item:

image

Here’s what the configuration of the Create List map operation looks like:

image

Once we have the list created, we can select values out of the list using the Select Value operation (in fact we’ve used two of them):

image

Does it Work?

Technically, both approaches worked, and took this input, and generated the output shown below it:

image

image

I’m going to be honest here, it still feels comparatively clunky, and I’m probably going to be reaching for that XSLT override pretty frequently. That is, unless there’s yet another way that’s cleaner and more simple than those presented thus far (this is one of those posts that I hope to revisit in the future with a correction showing an even better way to approach the problem).

Next Week

Next week, I will be wrapping up this series and shifting focus back to day-to-day BizTalk Development and all the fun things that we’re actively working on there.

Until then, happy mapping with shiny new tools!

Building Blocks of Windows Azure BizTalk Services

By Nick Hauenstein

This post is the twenty-second in a weekly series intended to briefly spotlight those things that you need to know about new features in BizTalk Server 2013.

Since the last BizTalk Summit, there has been a lot of coverage within the greater BizTalk Community of Windows Azure BizTalk Services – the Azure-backed offering that brings pieces of the core BizTalk Server functionality to the cloud. This is something that QuickLearn had early opportunity to get hands-on with, and also something for which we were able to create labs (even before public availability). At the same time, it’s something that we haven’t really written about until now.

This week, rather than going through specific how-to guidance for pieces of the offering, I instead want to take a step back and look at the offering as a whole, so that we all understand the pieces that are there (and can see the forest through the trees). If you want to follow along, go ahead and download the Windows Azure BizTalk Services Preview SDK.

Once installed, you’ll find that Visual Studio has been juiced up with two additional project templates found under the BizTalk Services heading:

image

The first project type BizTalk Service allows you to define Itineraries (not in the ESB sense, though they do display a complete message flow, much like ESB), and the configuration for Bridges (much like a BizTalk Pipeline).

The second project type BizTalk Service Artifacts allows you to define schemas and maps (.trfm files, though you can convert your existing .btm files with some limitations using a tool provided at the SDK download link). The mapper is something I’m not going to be discussing any further, and as a result I will instead recommend this excellent blog post on the subject by Glenn Colpaert, and then also the official documentation on the conversion process/limitations.

So we can create these Itineraries, configure these Bridges, and deal with Schemas and Maps, but how does this all fit together within the context of Windows Azure BizTalk Services? To understand the answer to that, we have to first address the three things that Windows Azure BizTalk Services is trying to do:

  • Rich Messaging Endpoints (Itineraries + Bridges + Maps)
  • BizTalk Adapter Service (Service Bus Relay + Local Helper Service + BIzTalk Adapter Pack)
  • Business to Business Messaging (EDI Schemas + All of the Above)

Rich Messaging Endpoints

image

I’m going to start with the weirdest of the bunch – Rich Messaging Endpoints. The developer experience of this is similar (to a point) to defining an ESB Toolkit itinerary, but only inasmuch as you’re seeing the entire message flow within a single diagram. Here you can define certain sources (FTP/SFTP) for messages, and have them routed through Bridges (similar to Pipelines), and then routed out to certain destinations (for which CBR can be used to route the messages to one or multiple destinations).

In order to specify the settings necessary to connect to those destinations, you often find yourself within a configuration file defining WCF related binding settings directly:

image

Double-clicking on a bridge within the itinerary will bring up a semi-clone of the Pipeline Designer with a fixed set of components in place (as well as a location to specify maps to execute directly as part of the pipeline):

image

In terms of integration patterns, the bridges here are giving us a subset of the VETRO pattern (namely the VETR part of it): Validate, Enrich, Transform and Route. Where is the Route part of the equation you might ask? Well, you’ll actually find it in the Properties window for the bridge, by clicking the button in the Route Ordering Table property:

image

Here, we can do context-based routing that helps determine the destination for each message coming from the destination (though here, I have only connected a single destination endpoint).

BizTalk Adapter Service

image

What happens if the list of destinations doesn’t suit me? What if I want to take information from an SFTP drop, transform it into something that I could use to generate a record within a table in SQL, and then directly insert that record into my on-premise system? In that case, I’ll find myself reaching to the BizTalk Adapter Service feature.

This one has a nice list of dependencies that need to be in place before you can even use it (e.g., Windows Server AppFabric, BizTalk Server Adapter Pack), but once you have those, it’s fairly straight-forward to setup.

What it’s really providing is a single WCF endpoint that is exposed over a Service Bus Relay endpoint (and thus accessible from anywhere in the world – even if hosted behind a fairly strict firewall). This single endpoint can be passed messages destined for any number of internal systems that you can setup through the Server Explorer interface within Visual Studio. The BizTalk Team Blog actually had a pretty decent article on the topic back in June that sadly generated 0 comments.

Essentially, this is allowing you to bring your LOB systems into the mix to play along with everything else already mentioned (assuming those LOB systems have WCF adapters included in the BizTalk Adapter Pack).

Business to Business Messaging

The final capability that WABS is bringing to the table is B2B messaging (i.e., EDI). We have the ability through a special portal (still in preview) within the Windows Azure Management interface to create Parties and Agreements a la BizTalk Server on-prem. In fact, the same exact schemas are used to define the X12 messages, so if you’ve already had to do some schema customizations on a per-partner basis for in-house integrations, those same changes can now be brought to the cloud.

Pulling it All Together

Is this replacing BizTalk Server on-premise as we know it? Not quite. We have a lot of the same pieces: Transport (Limited)/Translation (Limited)/Transformation and Context-based Routing (Content-based Routing when using the Enrich stage of the Bridge). We are missing more complete process orchestration with exception handling / transactions / compensation (Orchestrations). We are missing rule-driven injectable logic (Business Rule Engine), among other things (though I’m stopping the list here to avoid debate about similar (but not quite) functionality in Azure in general).

So what do we do with this? Use it to solve integration problems that use a hybrid between cloud-based and on-premise resources (e.g., http://msdn.microsoft.com/en-us/library/windowsazure/hh859742.aspx). Use it when an on-premise BizTalk installation would be overkill for the task, but it can be done happily using the tools available in WABS. Use it to take over the most taxing parts of your B2B processing, leaving BizTalk Server on-premise to do those things it’s great at.

Ultimately, it’s up to you. BizTalk Server 2013 and Windows Azure BizTalk Services together give you the power to decide how to break up your integration work, and how you approach the integration challenges that are thrown your way.

If you haven’t yet given the SDK a download yet, go do it. Though the service is still in preview, the SDK has been available for quite some time now, and great things are right around the corner – so keep your skills sharp.

That’s all for now!