Thursday, September 26, 2013

Setting up your Azure Subscriptions – learned the hard way

For a project i’ve been developing a Pilot using Azure VM’s. The VM was all nicely configured, IIS/network/TFS integration/all running smoothly. 2 weeks ago, i could see i would hit spending limit, within a couple of days.

I created a support ticket, with questions about the spending limit, where i found out that removing it could only be done by the account owner (on holiday), resetting it was not possible, and that hitting your limit (what i did), would remove the Virtual machine, disable access to the subscription, so that you are effectively…toast…

The good news…this won’t happen to you! Scott Guthrie announced a number of Windows Azure Improvements today, which would have solved my scenario, and prevents this from happening again, so really nice updates, however in my case….with a disabled subscription….this does not help…

….for me it meant i had to find a workaround…my plan was to create a new dedicated project subscription, and migrate my VM to that subscription….there was only 1 slight problem, for testing purposes i had created a Windows Active Directory.

“...An Azure Active Directory is created and shared among all your subscriptions where you are the Service Administrator of...”

This imposes a challenge:

#1 When migrating data between subscriptions, the first requirement the support team issues is having the same service administrator for the source and target subscription

#2 When you create a brand new subscription, and use the same service administrator account the Active Directory is also in the new subscription

#3 When migrating data between subscriptions, the target subscription must be empty

#1+#2 != #3

Lessons learned:

  • Use a dedicated project subscription
  • Use a dedicated dev / poc subscription
  • Think about the service administrator which is linked to your account, use different service administrators for your DEV / Project subscriptions
  • Be aware that when you create an Active Directory, it cannot be removed without creating a support ticket
  • A support ticket is not free, and can take some time to process
  • Make sure that you have planned your Azure configuration in advance

 

HTH,

Sander

Monday, September 16, 2013

BizTalk Services Questions and answers

After attending the UKCSUG and BTUG I have some answered questions to some challenges I faced during my implementation with BizTalk Services. Hereby my findings;

Challenges

I'm wondering how to cope with some of the following challenges in this scenario, and would like to know resource information available which addresses some of these challenges;

Validating / Signing of messages

A requirement is to validate the incoming message by inspecting an signature in the message body, and signing the outgoing message. I now that a

custom inspector would help, however, i have no idea where to store these certificates as these would be used inside the BizTalk Service.

Q: Is this functionality possible within BizTalk Services?

A: In the custom inspector this functionality can be added, this however means that the certificate must be stored somewhere. This cannot be stored within the BizTalk Service, however could be stored in a BLOB storage container.

 

Handling errors when connecting to an Endpoint using BizTalk Services.

Detect an error and if one occurs, execute a mapping to return a custom Error message to the client.

Q: Is this possible by using an OnEnterInspector in the Enrich stage? (Overruling the response by setting custom headers or something which triggers an different mapping than the happy flow)?

A: This could be done by ensuring that the correct MessageType is set on the response

 

 

In case of a protocol error, i would like to be able to perform a retry

Send a POST request Using the HTTP protocol, handle technical errors gracefully by doing a retry. This means that the response for an endpoint needs to be evaluated,

Q: should this be implemented using custom bindings/behaviors for the endpoint, or will this be added as a feature in the future?

A: This could be done by implementing a custom WCF Service, which could then promote a MessageType based on the response. This would require some extensive testing / debugging as this is not readily available. It is at this moment not in preview, the AS/2 – EDI uses a mechanism ‘error queue’ which runs outside the BizTalk Service

When communicating with the backend, what approach would be wise for managing certificates

My requirement is to place a call to an endpoint using TLS, this would mean i need to manage the certificate for the endpoint after the bridge. At this point, i'm only aware configuring a certificate to communicate with the bridge.

Q: Would this be possible by custom WCF behaviors?

A: This could be done by implementing a custom WCF Service, which could retrieves the certificate from a BLOB store

Custom components

As a reference i have found this blog on using inspects, which helps with some of my questions (http://blog.codit.eu/post/2013/06/03/wabs-part3.aspx)

Note: At the moment I'm using BizTalk, however, i am working on a platform and would like to migrate to BizTalk Services as soon as possible. Any insight in the roadmap would help a lot.

A: Not known publicly

Migration path

For now, we are evaluating BizTalk and would like to migrate to BizTalk Services at a later stage. For now, this would mean that we need to purchase a license and again purchase a subscription when we migrate.

Q: Are there any intermediate licensing options where we can use BizTalk for a certain period with the migration to BizTalk services in mind (e.g. 1 year license model + subscription).

A: It is possible to purchase the premium subscription, this comes with a license of 1 year. Other than that, the licenses cannot be mixed.

 

Pricing

At this moment the costs of BizTalk Services are discounted with 50%, that quite a lot and could make/brake a use case cost-wise, depending on when BizTalk Services is out of preview.

Q: Is there any way to provide use BizTalk Services with a transaction based subscription, where we pay for the amount of transaction done through BizTalk Services?

A: Not known publicly, however the prices are being evaluated.

Additional information

·         Top feature

o    AS2/EDI

·         AS2/EDI Bridge

o    Generated Bridge

o    Errors to dedicated queue

·         Bridge == Custom Code

·         1 Live ID is linked to 1 Trading management portal

·         Everything is executed in memory

·         MiniSAP (http://www.sdn.sap.com/irj/scn/nw-downloads)

·         Yossih Dahan (WABS reflections)

·         Challenges

o    Separation of projects (maps / schemas)

o    Complex Schemas (imports)

o    Retrieve information / state using PowerShell

o    Mapping (no XSLT)

o    Routing (first match) like WCF-Routing

·         Roadmap

o    Hopefully this year live

o    From ServiceBus

o    Settings in Code รจ Configurable (connection strings etc)

o    Mapping (breaking change)

§  Mapper does not use XSLT (feature support JSON/Object2Object)

§  Migration tool available

 

Monday, September 09, 2013

UK Connected Systems User Group – BizTalk Services questions

This Wednesday I’m attending the CSUG – The Hybrid Organisation (sold out!). During my evaluation of several Integration products (BizTalk / BizTalk Services / Neuron ESB / MuleESB) I had some concerns/questions on BizTalk Services.

My goal is to get answers on the following scenario

  • a) I receive a HTTP Post with an Xml Body payload
  • b) This contains a signature which i need to validate
  • c) Transform the Xml body payload to another Xml message
  • d) Perform a HTTP Post with the new Xml message to an external HTTP url and sign the message
  • e) Route the response back, similar to the aforementioned steps, where an error needs to be mapped to a dedicated error response

Challenges

I'm wondering how to cope with some of the following challenges in this scenario, and would like to know resource information available which addresses some of these challenges;

Validating / Signing of messages

A requirement is to validate the incoming message by inspecting an signature in the message body, and signing the outgoing message. I now that a

custom inspector would help, however, i have no idea where to store these private certificates as these would be used inside the BizTalk Service (in the BizTalk Services portal I can only store public certificates).

Q: Is this functionality possible within BizTalk Services?

Handling errors when connecting to an Endpoint using BizTalk Services.

Detect an error and if one occurs, execute a mapping to return a custom Error message to the client.

Q: Is this possible by using a OnEnterInspector in the Enrich stage? (overruling the response by setting custom headers or something which triggers an different mapping than the happy flow)?

In case of a protocol error, i would like to be able to perform a retry

Send a POST request Using the HTTP protocol, handle technical errors gracefully by doing a retry. This means that the response for an endpoint needs to be evaluated,

Q: should this be implemented using custom bindings/behaviors for the endpoint, or will this be added as a feature in the future?

When communicating with the backend, what approach would be wise for managing certificates

My requirement is to place a call to an endpoint using TLS, this would mean i need to manage the certificate for the endpoint after the bridge. At this point, i'm only aware configuring a certificate to communicate with the bridge.

Q: Would this be possible by custom WCF behaviors?

Custom components

As a reference i have found this blog on using inspects, which helps with some of my questions (http://blog.codit.eu/post/2013/06/03/wabs-part3.aspx)

Note: At the moment I'm using BizTalk, however, i am working on a platform and would like to migrate to BizTalk Services as soon as possible. Any insight in the roadmap would help a lot.

Migration path

For now, we are evaluating BizTalk and would like to migrate to BizTalk Services at a later stage. For now, this would mean that we need to purchase a license and again purchase a subscription when we migrate.

Q: Are there any intermediate licensing options where we can use BizTalk for a certain period with the migration to BizTalk services in mind (e.g. 1 year license model + subscription).

Pricing

At this moment the costs of BizTalk Services are discounted with 50%, that quite a lot and could make/brake a use case cost-wise, depending on when BizTalk Services is out of preview.

Q: Is there any way to provide use BizTalk Services with a transaction based subscription, where we pay for the amount of transaction done through BizTalk Services?

 

 

Hopefully, more answers at the end of this week!

 

Regards,

Sander

Sunday, September 08, 2013

BizTalk deployment framework – things to think about

Update (12-09-2013): slides can be found here http://www.slideshare.net/SanderNefs/biz-talk-deployment-framework-stripped

As a consultant, my role is to give advice, work on projects and help out customers. As I don’t want to be the Single Point of Truth, my approach is to  transfer my knowledge in workshops/sessions with the internal developers or people who I think should know about certain technologies.
In the past i’ve used the BizTalk Deployment Framework (BTDF) quite often. I have experienced that getting to know the concepts and working with it, is a developers choice, however this choice impacts the way administrators need to work.
This post summarizes my experiences and contains some of the slides i used when having workshops on the BTDF, with some caveats, and thinks to think about.
First of all, why? Why would i need a new framework, what is wrong with the way i do my deployments right now? Nothing, however, in certain cases, you might run into some limitations, in the slide below some pros and cons of applying the classic deployment vs using the BTDF.
image
As we can see, the main advantages are having a consistent approach, automated deployments and a way to perform multi server deployments. However, keep in mind, to achieve this, we need to learn a complex framework, where you can make mistakes, which will not give you all the pros! So learning it, before applying it is crucial, and as always, it is not a golden hammer!
Below an overview of important features
image
Note: The reason for deviating from a standard product and using the out-of-the-box features by choosing a tool, framework should be that it add features that are not possible in the standard product.
My #1 requirement for a tool, framework, utility is the support provided. My experiences with the BTDF are just amazing. I’ve posted a feature suggestion to include some of the VS.Net menu items as startmenu items upon deployment, and a couple of days later it was in the new 5.1 release!  (authors: Thomas F. Abraham / Scott Colestock)
Considerations
Before applying it, after having in depth knowledge of the problems you can overcome by the framework. Ensure that the people involved in the project/integration platform are familiar with the BTDF, know about the advantages, agree on the usage, are willing to use it. The BTDF works different, so unless you are the dev/admin you need to agree that this is the framework for you!
image
Usage
In stead of deploying your solution and creating an MSI by exporting this application from the BizTalk Administration console. The BTDF will generate an MSI based on a .target file which contains steps such as packaging the bindings, settings, dll’s etc. The BTDF MSI in different as it packages the BTDF along with it. So when deploying on a machine, you are actually deploying using the BTDF in a different mode.
So design time we are working from visual studio, we generate the MSI (or it is generated by the BuildServer) and deployment it ont he target machine.
image
Usage
Assuming we have a BizTalk project with artefacts, we can add a new BTDF project after following the steps mentioned below;
image
Right….nothing is shown? Correct, the BTDF project is not a known type in VS.Net. We need to manually add the artefacts used. 
image
There are 3 concepts extremely important, on which we need to focus, but which is something you should learn before making any decisions in your project.
  • SettingsFile generator
This is the ‘master’ configuration file, this contains all the settings used, possibly for your different environment. So when following an DTAP strategy, this possible contains all the settings for each environment.
Caveat: this means your environments must be static, known in advance, and should not change often, as this file needs to be administrated and is per solution/application!
  • Masterbindings
You can implement the BTDF in various ways, one is to have a Masterbinding (binding with macros), which is used to generate environment spefiic bindings;
Masterbinding (macros ABC) + settingsfile with environment values ABC =   binding with replaced values for ABC.
In this post, i am not going to use Masterbindings, but alternative bindings, similar to ‘classic bindings with BizTalk’.
  • MSBuild
MSBuild is a xml configuration based framework used by the BTDF. The project file created, is basically a configuration file for MSBuild tasks.
image
The Deployment.btdfproj works with Propertygroups
image
You can override, configure custom properties, in the example below i am overwriting the output path so that all my build files are generated on a specified locations.
The ItemGroup contains the Artefact specific configuration
image
Dedicated bindings
So as stated, i’m not using Masterbindings, as i don’t like to use macros etc within my team. I would like them to export a binding file, and add it to the project. We have this flexibility too! Follow the steps mentioned below;
image
image
image
2 Step redeployment
The last thing to mention, is one of the features that can explain why an automated framework is good. Let’s consider an application, which contains a ‘Common’ application.
image
Our deployment options are limited with the ‘classic’ approach
image
How does the BTDF solution helps us? We have 2 steps
1) Undeploy
Removes the application from BizTalk
2) Uninstall
Removes the artefacts from the machine + configuration.
In our case…..we only need to really do step #2 for the Common. Replace it and deploy, we do not need to Uninstall everything (which already worked). So now we can, we can Undeploy all the applications from BizTalk, and leave the dll’s components, configuration on the system.
For the common we need to
Step #1: Undeploy all applications
Step #2: Uninstall the application Common
Step #3: Deploy the application Common
Step #4: Redeploy all other applications
The advantage of the BTDF is that we can script this, so we can actually perform a much more complex deployment and do it in a consistent way, as described in the slide below
image
Start learning the BTDF
So, how to start learning the BTDF? Follow the lab ‘Helloworld’ with the steps mentioned below;
image

Note: I cannot provide the original slides, hopefully these prints suffice..

HTH,
Sander