Author Archives: Rob Callaway

BizTalk, BizTalk Server 2016, Blog , , , 1 Comment

BizTalk Server 2016 Feature Pack 1

By Rob Callaway

Ooh, shiny!

On April 26th 2017, Microsoft released Feature Pack 1 (FP1) for BizTalk Server 2016 and it’s been a while since I was this excited for a BizTalk Server release. Yeah, I just said that. I’m more excited for this feature pack than I was for BizTalk Server 2016 or even 2013 and 2013 R2, and here’s why… this is the first ever Feature Pack for any release of BizTalk Server, and is setting a precedent that we have never seen before in the 16+ years of the product.

So, what is a feature pack?

A feature pack is a release of new non-breaking features for the product. These are not bug fixes or anything like that (those are distributed quarterly through Cumulative Updates). These are brand-new features that extend the product in new ways and help customers get the most out of their BizTalk Server investment.

The product team has confirmed that other feature packs are in the works, but they have not publicly confirmed when we can expect them. In discussions I’ve had with Tord Glad Nordahl (a program manager at Microsoft and longtime lover of BizTalk Server), he said:

“If it takes 6 months to build new features there will be another feature pack in 6 months, and if it takes 2 months there will be a new one in 2 months.”

My takeaway is that the team’s goal is to offer real answers to problems that customers face in the timeliest manner possible.

I’m excited that the BizTalk Engineering team at Microsoft will be releasing new features on a more regular cadence. It’s awesome that they are building new features to address long-overlooked issues and not making us wait another 2 years to get our hands on them.

FP1 is available to customers with Software Assurance who are using the Developer or Enterprise editions of BizTalk Server 2016.

FP1 introduces some innovative new features to BizTalk Server and addresses some longstanding concerns that many customers have had. The new features break down into three categories.

Deployment

Anyone who’s worked with BizTalk Server knows that the deployment/ALM story has left something to be desired. For years, the “official” deployment story has been to deploy applications using BizTalk MSI Packages. Although the BizTalk MSIs are pretty easy to use and they work well for simple applications, they tend to be inflexible and break with the complexities of a real application in the real world.

For example, to create a BizTalk MSI Package I have to deploy all my assets to the BizTalk Server Management database and then I can generate an MSI with those assets. It sounds easy and it is. The issue is that if I introduce a new assembly, or port, or anything at all, I have to add that new resource to the BizTalkMgmtDb and then generate a new MSI. In the modern world of DevOps and continuous integration/deployment, the standard MSI-based deployment is pretty cumbersome and most teams wanting to adopt those types of strategies need a different answer.

In the past, those teams have used community-designed tools such as the BizTalk Deployment Framework to automate building an MSI from source code repositories (like most modern ALM solutions) and therefore eliminating the need to deploy to a BizTalk Server system to create the deployment package. Feature Pack 1 for BizTalk Server 2016 introduces two new features that will serve as a foundation for more sophisticated deployment strategies in the future (which my inside sources at Microsoft have confirmed are coming in later feature packs).

  • Deploy with VSTS – Enable Continuous Integration to automatically deploy and update applications using Visual Studio Team Services (VSTS).
    For anyone who has used the build/release features of TFS or VSTS, this will be immediately familiar. This is a deployment task that you add to your release pipelines to deploy new or redeploy/update existing BizTalk Server applications.
    If you haven’t used the build/release features of TFS or VSTS check out this post where I explain how to use those features to enable continuous release for a Logic App.
  • New management APIs – Manage your environment remotely using the new REST APIs with full Swagger support.
    Imagine having RESTful web APIs for updating, adding, or querying the status of your BizTalk Server applications and their resources… now stop imagining it because it’s a real thing!

Analytics

The tracking capabilities in BizTalk Server are extensive, but the configuration is often unintuitive, and no one likes digging through the BizTalkDTADb for the instance data they need.

FP1 enables you to send your tracking data to Azure Application Insights and feed operational data (subscriptions, batching status, message instance counts, etc.) to Power BI.

  • Application Insights – Tap into the power of Azure for tracking valuable application performance, usage, diagnostics, and availability.
    Enabling this is super easy: after creating an Application Insights instance, in the BizTalk Settings Dashboard there’s a new section for enabling analytics.

    Enabling Application Insights for the BizTalk Server group as introduced in BizTalk Server 2016 Feature Pack 1

    Once you’ve enabled analytics and provided your App Insights instrumentation key, in your ports and orchestrations you will have a new setting to output the tracking data to App Insights.

    Enabling Application Insights for a BizTalk Server orchestration as introduced in BizTalk Server 2016 Feature Pack 1

  • Leverage operational data – View operational data from anywhere and with any device using Power BI.
    This operational data is the same kind of information that you’d typically view using the BizTalk Group Hub (suspended instances, subscriptions, tracked events, etc.). If you can build a query for it using the Group Hub, you can output that to Power BI… but why would you?
    Because Power BI gives you the ability to view that data from anywhere (without having direct access to the BizTalk Group), and the tools in Power BI make querying that data surprisingly easy. With Power BI, you can ask questions in plain English and have MDX-style queries created for you in the background. FP1 comes with a pre-built Power BI template, but you of course have the ability to build your own. I’m interested to see what the BizTalk community can create using these tools.

Runtime

If I’m being completely honest, the two features in this runtime category weren’t really on my radar at all until Tord Glad Nordahl stopped by one of my classes last month and discussed them with the students. But now that I’ve seen them, I’m excited for the potential and happy that customers with these requirements are getting some much needed love.

  • Support for Always Encrypted – Use the WCF-SQL adapter to connect to SQL Server secure Always Encrypted columns.
    Basically, SQL Server 2016 introduced a feature that enables client applications to read/write encrypted data within a SQL table without actually providing the encryption keys to SQL Server. This gives a new level of data security since the owners of the secure data (i.e., the client applications) can see it, but the manager of the data (i.e., SQL Server) cannot.
    This ensures that on-premises or cloud database administrators or other high-privileged (but unauthorized) users cannot access the sensitive data.
    With Feature Pack 1 of BizTalk Server 2016, the WCF-SQL adapter now offers an Always Encrypted property where you can simply enable or disable the feature as your needs dictate.

    WCF-SQL Adapter Always Encrypted property as introduced in BizTalk Server 2016 Feature Pack 1

  • Advanced Scheduling – Set up advanced schedules for BizTalk receive locations.
    The Schedule page of receive locations has additional options for shifting time zones and setting up recurrence schedules.

    BizTalk Server receive location advanced scheduling options as introduced in BizTalk Server 2016 Feature Pack 1

As always, the QuickLearn Training team is already looking for the best ways to incorporate these new features into our courses, but until we do you should grab the Feature Pack for yourself and give these new features a spin for yourself. While you’re at it go to the BizTalk Server User Voice page and vote for the features that you’d like to see in the next feature pack, or if you have an original idea for a feature add it there and see how much love it gets.

BizTalk, BizTalk Server 2016, Blog, Student Tips Leave a comment

BizTalk Server 2016: Using the XslCompiledTransform Class

By Rob Callaway

A few weeks ago, QuickLearn Training hosted a webinar with an overview of a few of the new features in BizTalk Server 2016. This post serves as a proper write-up of the feature that I shared. In this write-up, I’d like to drill a little deeper into the things that were discussed and even explore some other aspects that we simply didn’t have time for. In my portion of the webinar, I spoke about using the XslCompiledTransform class in maps. If you missed the full webinar, check it out over on the QuickLearn Training YouTube channel. If you prefer you can just watch the section on the XslCompiledTransform class.

Comparing XslCompiledTransform and XslTransform Performance

There are so many resources comparing these .NET classes against one another that I’m really not sure that there’s anything I can add to the discussion. The best of these resources is the one I cited in the webinar by Anton Lapounov. Simply put, if you put these two classes in a straight-up foot race, the XslCompiledTransform is going to take longer to load, but will blow away the XslTransform class when it comes to the actual time to transform.

Comparing XslCompiledTransform and XslTransform Behaviors

Now, if you’ve closely followed the features of other releases you may think this isn’t actually a new feature, and you would be mostly correct. You see, in BizTalk Server 2013 the mapping engine was changed to utilize the XslCompiledTransform class instead of the XslTransform class that the mapping engine had been using since BizTalk Server 2004. While this change was made to reap the performance benefits of the XslCompiledTransform class over the XslTransform class, it was a change that Microsoft made unilaterally to all BizTalk maps compiled for BizTalk Server 2013. While the intention was pure, this change wasn’t universally welcomed by BizTalk developers. There are several great write-ups exploring issues that arose in existing maps when updated (for example this great blog post from Dan Rosanova, or just the Known Issues for BizTalk Server 2013). In case you’re pressed for time and can’t read those, the issues arise from the differences in behavior between the XslCompiledTransform class and the XslTransform class. The specific differences are:

  • If an input XML field is empty or contains a false value, the Scripting functoid will treat the input as a true
  • The XslCompiledTransform class only supports calling public methods
  • The XslCompiledTransform class does not support returning null
  • In the XslCompiledTransform class, function overloads are differentiated by number of parameters rather than types
  • The XslCompiledTransform class utilizes the XPathArrayIterator type rather than the XPathSelectionIterator type for looping through repeating records within the Scripting Your script must call the MoveNext() method to advance properly

While these five changes may seem relatively trivial, for some people they presented issues that completely broke their maps. There is a registry setting which allows you to tell the mapping engine to use the XslTransform class, but the change is applied globally to all maps and negates the potential performance gains offered by the XslCompiledTransform.

Setting the Transform Class for a Map

BizTalk Server 2016 now surfaces the option for the transformation class as a UI element. Furthermore, the transformation class isn’t specified globally, but instead we can set it for each map separately. So, any maps I’m upgrading that are negatively affected by the XslCompiledTransform class can target the XslTransform class. This ability to opt-in or opt-out gives us true backward compatibility that was sorely missing in the previous releases.

By default, any new or existing maps that you are upgrading will target the XslTransform class (for backward compatibility), but setting the transform class for the map couldn’t be easier. In your map, if you go to the Properties for the map grid, you will see a new property named Use XSL Transform (the default is True and indicates that the XslTransform class will be used). If you change it to False, the XslCompiledTransform is targeted instead. There is a third option for the property, Undefined. If you choose this option, that map will check the same registry settings used in earlier releases to control the transformation class.

Configuring the Use XSL Transform property

Answering an Unanswered Question

I don’t really want to rehash my test process or results, as you can watch the webinar or download the slides to get all of that. Instead, I wanted to answer a question that was posed in the webinar that I was unprepared to answer. Niyati asked if the increase in performance would have been the same if I were calling the map in an orchestration. This configuration was not part of my initial round of tests, but I cracked open my solution, and created an orchestration so I could definitively show whether it was true or not.

My orchestration has the simplest design possible. It receives the message, runs the map, and sends the transformed message out.

Orchestration used to test the XslCompiledTransform map

I made one of these babies that runs the XslCompiledTransform map, and another that runs the XslTransform map.

I went crazy in my testing and processed 10,000 instances of each orchestration using my batch file submission method. I’m not going to write a full-blown analysis of the results because I think the numbers speak for themselves.

The XslCompiledTransform class once again executed much faster

But That’s Not All

I had a lot of fun exploring transformation classes in preparation for this webinar and I really look forward to the next one. Please be on the lookout for details on that webinar in the coming weeks.

If you need to learn more about measuring and analyzing BizTalk Server performance, tuning performance, or controlling the throttling behaviors in BizTalk Server you really should check out our BizTalk Server Administrator Deep Dive course.