Friday, November 29, 2013

BizTalk User Group NL 28-11-2013

On 28-11-2013 the BizTalk User Group (LinkedIn group BTUG NL) meeting took place in Amsterdam, which was organized by Estreme. The purpose of the BizTalk User Group is to have regular meetings with members in the community on the topic of integration. Since Azure provides more and more integration capabilities, by means of the Azure Service Bus, and the Go-Live of Windows Azure BizTalk Services (WABS), the meetings are diverse and very interesting.

As Azure is very broad, the BTUG focuses on the following elements of the Microsoft Integration stack:

  • On Premise (WCF/SSIS/BizTalk/Windows Server Service Bus etc)
  • Cloud - Windows Azure (Windows Azure BizTalk Services / Service Bus etc).

Announcements

  • An upcoming event in January is the BizTalk Saturday, focused on Windows Azure BizTalk Services
  • Next year, a BTUG Beach event is organized, an informal community event
  • The next upcoming meeting will be held in March

Feedback BizTalk Summit - Steef - Jan Wiggers
Steef - Jan Wiggers provided a summary after attending the BizTalk Summit. This showed that BizTalk is here to stay with an improved release cadence:

  • Annual cumulative updates
  • 2 - yearly platform updates
  • Next year there will be a BizTalk 2013 R2
  • In 2015, a new version will be released
  • Improvements included in the upcoming releases are in the area of JSON support, HealthCare / SWIFT adapter-additions and an updated SB Adapter

Windows Azure BizTalk Services is now live and can be used in production and is improved in the areas of monitoring, archiving, EDI support and management by using PowerShell Command Let.

KAS Integrator - Johan Vrielink

At KAS Bank BizTalk has been implemented to handle transactions for stock exchanges. The KAS Integrator is a framework built on top of BizTalk which allows fully automated configuration of the environment. There are several services defined, on top of BizTalk, and a management portal which provides business rules, publish / subscription configurations, which has some similarities with the EDI partnering, which was pretty interesting. A customer with a clear vision and story was very great to have presenting a session. It showed some typical demands in the market; the automated configuration of middleware, ability to minimize development efforts for interfaces and gave great insight in how to think about challenges in future projects, e.g. by using PowerShell.

Integration Challenge : Custom Service Bus - Rob Kits
During the integration challenge, non-BizTalk products / solutions are shown and compared to BizTalk, which allows you to think about integration in a broader sense, where not every problem can be solved with a single tool. In this case it was a custom solution that used in locations where gas is distributed. In this environment, it is necessary that operators can configure/adjust/monitor the environment, and middleware such as BizTalk is too complex. It was based on PLC technology presented a solution that was brilliant in its simplicity. It again showed that an integration problem must be based on the needs and requirements, and not always with the potential features provided. I found that to be a nice analogy with cloud technology, where one of the advantages is that you pay for what you need, not necessarily what the technology can do.

Synchronous Service Bus - Martin Rienstra
BizTalk is not a golden hammer and certainly not suitable for all issues. At a client, about 80 interfaces were implemented in an intranet environment using a request-response pattern (synchronously). As BizTalk is designed with the principle of guaranteed delivery using the asynchronous pub/sub architecture (polling), BizTalk is not designed for low latency solutions. This does not mean BizTalk is not capable of handling these, this is possible, by using separate hosts, scaling out, separating the databases, however, there is due to the architecture, unpredictable latency.

The BizTalk product team has recognized this and stated that this is due to the architecture in BizTalk and will not be resolved, this kind of issues can be addressed by using different technologies.

Martin had previously looked at the Service virtualization platform MSE (Microsoft Service Engine), but this product is no longer developed (in this space there is only Sentinet). The requirements; configurable, manageable, and re-using the BizTalk maps. The solution consisted of an interesting mix of WCF custom behaviors allowing dynamic service to be generated using a configuration, which uses BizTalk artifacts (mappings / assemblies etc), with the great advantage that the existing BizTalk used solution could be reused. The disadvantage is that the services should run on BizTalk the machine because of the usage of BizTalk artefacts.

Summary

A great event and very interesting content, in future meetings we can expect a lot of great Integration Challenges, and I’m trying to arrange a session where one of my colleagues from Caesar will explain differences and comparisons between Sonic vs BizTalk vs Azure as I’ve seen a lot of interesting things after comparing BizTalk and Sonic;

  • Sonic has the choice between durable subscriptions and non-durable (using queues), where BizTalk always uses durable subscriptions, Azure provides in this context durable (Topic) and non-durable subscriptions (Queues)
  • Routing can be done schema based, where Sonic does this without enforcing a schema, where BizTalk requires a schema
  • Similarities between logical and physical separation of concerns (where Sonic works with an ESB Container and Broker concept) and BizTalk uses a Logical and Physical port)
  • And more….

 

Great to see everyone and I hope a lot of events like this will follow.

 

Regards,

Sander

Friday, November 22, 2013

‘ETW2.0’ - High performance tracing using EntLib SLAB

Are you writing an application that has high performance requirements, are you wondering how Azure Diagnostics works, do you want to write your own logging framework….this might help you out.

Not so long ago, the Application Server Group ISV Partner Advisory Team posted an excellent article on how to instrument specifically BizTalk applications, by leveraging the ETW infrastructure.

This allowed for significant high performance tracing and was measured against other frameworks as you can see in the diagram below;

image

In the latest EntLib releases, this has been included in the Semantic Logging Application Block (SLAB).

What’s really interesting is that there are 2 patterns which you can implement:

1. In Process, where the Host which performs the log data is written to the ETW Infrastructure and the Listener is subscribed to the ETW data

Follow link to expand image

2. Out of Process, where the Listener can be a Service outside of your application (most suitable for OnPremise usage)

Follow link to expand image

 

  • EventSource

With the Semantic logging application block, the idea is that the logging infrastructure is predefined (e.g. Start / Stop events which are logged) and that the application only provides the data/parameter used to log. You need to create an EventSource which contains all the LogEvents you would like to log. This means that, as well as with unit testing….you need to think, before you build. An example is shown below;

image

  • Sink

The great thing, and the reason I like this framework is that you are able to create the sinks, and due to the Out-Of-Process model, can leverage sinks which in itself are not high performance. Out of the box there are a number of sinks: SQL database, Windows Azure table, flat files and some others.

1. Example SQL Server Sink

image

2. Example SQL Server Result

image

  • Result and extension points

Writing using the EventSource will write to the ETW Infrastructure, which has almost no performance impact. The Out-Of-Process listener will pick up the messages in a windows service (can also be downloaded from the EntLib download link), the Sink writes the data to the destination of your choice.

In a post of Tomasso Groenendijk the option of using MongoDb is explained, with the idea of having a high performance tracing mechanism. With SLAB the same functionality is available.

Additionally, writing large amounts of data can be something you don’t want to do on you database used for your primary process, so creating a MongoDb Sink is still a viable option, however, for different purposes.

  • Getting started with SLAB

You can use the SLAB quite easily by using NuGet and search for Semantic, which will display the Application Block and the available Sinks for Windows Azure Tables and SQL Server as wel.

image

The hands-on labs and documentation should get you going quickly. As the EntLib settings can be configured outside of your code (recommended), diving in the EntLib config might not be as much fun as you would expect. For this, there is an EntLib configuration tool available.

  • Getting started with EntLib Config

The following link, contains a EntLib 6 configuration add-in which helps you create the Configuration settings for some of the Application blocks and the Windows Service for out-of-process logging.

1. Select the configuration console

image

image

2. Click on the Config file and open the editor

image

3. Select the block and visually configure the block

image

4. Example TransientFaulthandling Config

image

5. Usage TransientFaultHandling Config

image

 

 

 

Cheers,

Sander