Microsoft Azure Development Best Practices for Cloud Computing

Cloud computing is the set of technologies and infrastructure capabilities offered in a utility-based consumption model. Microsoft Cloud Computing offers Microsoft Azure, previously known as Windows Azure, to help its customers realize the benefits of cloud computing.

How Successful Teams Leverage the Cloud

Download Our Guide 

Applying existing development ideologies to Microsoft Azure is not straightforward, as direct one-to-one conversion is rarely possible. How should you alter your methodology and what Microsoft Azure development best practices should you use?

Key Features of Cloud Computing

• On-demand self-service. Less capital investment is required of businesses.
• Measured service. The pay-per-use model aligns IT investment to operational expenses.
• Broad network access. Cloud computing improves business agility and the ability to innovative with reduced barriers to entry.
• Resource pooling. Securely separating resources on a logical level represents a key architectural principle used to minimize interaction points and achieve high cohesion/low coupling between the layers of the software solution.
• Rapid elasticity. Cloud computing allows businesses to be flexible when they need to scale. Businesses can meet their needs more dynamically and provision resources on-demand or with an automated process using parameters and triggers.

Cloud computing that replaces the traditional on-premise (locally built and deployed) software is broadly divided into three categories:

Azure Development Lifecycle

The development lifecycle of software that uses the Azure platform mainly follows two processes:

Application Development

During the application development stage the code for Azure applications is most commonly built locally on a developer’s machine. Microsoft has recently added additional services to Azure Apps named Azure Functions. They are a representation of ‘serverless’ computing and allow developers to build application code directly through the Azure portal using references to a number of different Azure services.

The application development process includes two phases: 1) Construct + Test and 2) Deploy + Monitor.

Construct & Test

In the development and testing phase, a Windows Azure application is built in the Visual Studio IDE (2010 or above). Developers working on non-Microsoft applications who want to start using Azure services can certainly do so by using their existing development platform. Community-built libraries such as Eclipse Plugins, SDKs for Java, PHP or Ruby are available and make this possible.

Visual Studio Code is a tool that was created as a part of Microsoft efforts to better serve developers and recognize their needs for lighter and yet powerful/highly-configurable tools. This source code editor is available for Windows, Mac and Linux. It comes with built-in support for JavaScript, TypeScript and Node.js. It also has a rich ecosystem of extensions and runtimes for other languages such as C++, C#, Python, PHP and Go.

That said, Visual Studio provides developers with the best development platform to build Windows Azure applications or consume Azure services.

Visual Studio and the Azure SDK provide the ability to create and deploy project infrastructure and code to Azure directly from the IDE. A developer can define the web host, website and database for an app and deploy them along with the code without ever leaving Visual Studio.

Microsoft also proposed a specialized Azure Resource Group deployment project template in Visual Studio that provides all the needed resources to make a deployment in a single, repeatable operation. Azure Resource Group projects work with preconfigured and customized JSON templates, which contain all the information needed for the resources to be deployed on Azure. In most scenarios, where multiple developers or development teams work simultaneously on the same Azure solution, configuration management is an essential part of the development lifecycle.

Cloud Development 101

Download Our Guide 

Tools to Build and Deploy an Application with Microsoft Azure

Microsoft Azure provides a number of different ways to build and deploy an application. They can be grouped in several categories by Azure cloud service type:

Azure App Service

Azure App Service can be used for easy publishing of web-based projects in one of the following categories:

• Web Apps: Websites or web applications written in .NET, Java, PHP, Node.js and Python
• Mobile Apps: Projects that provide support for mobile devices and enable authentication with social providers and with Azure Active Directory (Azure AD); Projects that provide backend storage and integrate with Azure Notification Hubs for push notifications
• API Apps: Securely-exposed APIs in the cloud with Swagger metadata so clients can easily find out how to consume them
• Logic Apps: Powerful integration solutions that can be built quickly using the numerous connectors — The connectors can be custom-built or out-of-the-box. If out-of-the-box, they are already prepared to work with data and devices anywhere — on premise or in the cloud. Some of the currently available SaaS and cloud-based connectors include Salesforce, Office 365, Twitter, Dropbox, Google Services and more.

Azure Web Apps are built with continuous integration and delivery in mind. They support various tools like GitHub Webhooks, Jenkins, Visual Studio Team Services, TeamCity, etc.

Azure Virtual Machines

Azure Virtual Machines are a well-known standard in VMs with a large collection of pre-configured Windows or Linux Servers. VMs allow full control over machine configuration along with responsibility for support, updates, software installations and administration. Basically, they offer the Azure IaaS layer (infrastructure-as-a-service).

Azure Functions

Azure Functions are the ‘serverless’ style response to business’ needs and developers’ demands to be able to write code directly through the web portal. With Azure Functions, a team can consistently and securely connect each function to different Azure services.

Azure Service Fabric

Azure Service Fabric is used to package, deploy and manage scalable and reliable microservices.

Azure Cloud Services

Azure Cloud Services is the PaaS layer of Azure, lying between App Service and Virtual Machines. Cloud Services employ the App Service approach to deliver various capabilities that enable developers to build n-tier cloud apps with more control over the OS.

Azure Container Service (Docker)

This Azure feature functions as a kind of OS virtualization allowing teams to deploy applications in a more efficient and predictable way. The containerized application approach works the same way in development, in test and in production systems. Azure Container Service supports standard Docker tools to manage containers.

Microsoft has dramatically extended their source control solutions in the cloud by adding Visual Studio Team Services — a wide range of tools and services to support DevOps work for continuous integration and delivery. With support for a range of tools including Jenkins, GitHub, Puppet, Chef, TeamCity, Ansible and VSTS for Azure services, a developer can work with existing tools enabling productivity and maximizing his/her experience.

After the application development is complete, development teams can move on to the quality assurance stage. Unit testing of the code is normally completed locally by the developer using different types of testing tools, such as automated unit tests and web tests. Azure Developer Tools contain emulators for some of the Azure environments, including cloud services and storage. These tools are provided for developers to unit test their application code before deploying it to Azure.

To ensure a smooth transition of the application to the cloud, I recommend using the following method:

Microsoft has implemented remote debugging for Visual Studio. The remote debugging process can be attached to any App Service (web or running in background) through the Visual Studio UI using Server or Cloud Explorer abilities. I recommend a hybrid approach to ensure our code will run on Azure — test locally using emulators and then move the application to cloud.

Microsoft Azure Development Best Practices to Deploy and Monitor

There are a few steps to take in order to do an Azure deploy. First the code is tested and then synchronized with the source code manager. Then the engine and versions are added to all assemblies. And finally after the code is compiled, a package is created so it can be uploaded to the Azure platform.

Preparing the Application for Publishing

All applications are prepared for a release and published at some point in their lifecycle. Uploading to the environment though is different in our case. We need to validate our Azure hosting environment first. Then, we need to focus on the host, management and execution of our cloud application. After that, we purchase a subscription, and finally, we can manually upload our deployment package to our staging or production environments.

Azure Tools to Optimize, Automate and Monitor

Microsoft provides several tools that can help optimize and automate the deployment process. Some of them are well-known: MSBuild scripts and the Azure Service Management API. The API can be used to interface with the management portal and create a package (one config file and one package file) directly from a VS project. That package can then be uploaded to the specific cloud environment (Azure Service). An alternative method to do that is using some of the DevOps tools I referred to above in order to deploy to Azure.

With your application up and running in Azure, you need to be able to monitor performance, watch for issues and see how customers are using your app.

Azure provides several monitoring options:

• Visual Studio Application Insights — An Azure-hosted extensible analytics service that integrates with Visual Studio to monitor your live web applications
• Azure Monitor — A new service created to help development teams visualize, query, route, archive and act on the metrics and logs that are generated by your Azure infrastructure and resources

Using the Azure Management Portal

Finally, a developer can use the Azure Management portal to manage an already deployed application. Each Azure service provides access to a few managed environments — the so-called deployment slots. They can be used for Staging, Production or any other environment we want to deploy to and use in the Azure cloud. There is no rule dictating the environments we must use. We can deploy our application directly to the production environment or deploy it to staging.

Microsoft provides a SWAP procedure that will do a transition (switch) between one deployment slot and another one. It is important for the production environment IP address to stay the same. That is the Virtual IP address on Azure. Any change to the Public IP Address will lead to changes in all DNS records and name servers. Microsoft tries to set as standard practice using CNAME records when we configure our DNS servers. Azure provides a DNS service (DNS Zone) as well.

As a part of some teams’ Microsoft Azure development best practices, the SWAP process can be automated to run each time new code is pushed as part of the continuous delivery procedure. It can even be used in preview mode so we can be absolutely sure that it will work after the SWAP process has finished.

From a development perspective, I recommend deploying the package first to the staging environment, so the application can be tested in a separate QA-like environment. Then, promote it to production for release. This approach will ensure the tested release of the application is ready for public use.

According to Microsoft Azure development best practices, the Deployment & Release stage involves the following two key phases.

System Testing. In this phase, different application tests are carried out. The tests can start with a basic “Smoke Test” to ensure that all basic functionality of the deployed services is running as it should on the cloud. This can be followed by “Integration Testing” to ensure that all external touch points to the services are functioning as expected. Subsequently followed by a “User Acceptance Test”, the services would be tested by a sample set of the application users. These are a representative set of tests that can be carried out as a part of the development lifecycle on the Azure platform, which an enterprise can run as a part of their standard project delivery practices.

All of these tests are carried out on the Azure platform without any investments required for procuring or setting up separate test environments on-premise. These tests can be carried out in the default staging environment, as discussed previously, or separate environments, in the case of larger projects, to carry out each test.

Production Release. After all of the test cycles have been completed, the tested Azure Service’s code can be released into production by promoting the services from the Windows Azure portal. The promoted services will execute from the production regions of the Azure data center fabric. As a part of the promotion process, the hosted services will have a public facing URL like “http:// [project].cloudapp.net”, where [project] is the name of the project defined at the time of creating a new hosted instance. In the production stage, the services are configured, managed and monitored from the Windows Azure portal.

Some of the activities an administrator can control and manage are:

• Configuring the number of running instances of the hosted service
• Starting/stopping service instances
• Upgrading or deleting deployed service packages
• Monitoring the usage of the hosted services

What You Need to Know About a Cloud Migration

Download Our Guide 

Control in Cloud Management

Unlike traditional on-premise systems, Azure does not provide full control of the computing environment and resources to administrators. Azure’s datacenter is fully managed and controlled by Microsoft. Users are only provided with private virtualized instances, packaged as services, as the unit of computation and storage for hosting their apps.

This will require a shift in the IT support operations. The Azure of today does not permit administrators to deploy some of the custom or third-party tools, utilities, agents, etc., that might be in use extensively with the existing operational processes to support and administer. Administrators use these tools to investigate production-related issues such as poor performance and crashes.

Sensitivity in the Cloud-Based Applications

A cloud-based application can run into different problems related to environments and architecture. Microsoft works consistently to avoid most of the problems by creating and proposing specific patterns to be used during the application development process. They are grouped in following areas.

Availability

The time that the system is functional and working. It will be affected by system errors, infrastructure problems, malicious attacks and system load. It is usually measured as a percentage of uptime.

Data Management

A key feature of most cloud applications is the ability to manage data under different conditions. Data is typically hosted in different locations and across multiple servers for reasons such as performance, scalability or availability, and this can present a range of challenges.

Design and Implementation

Microsoft Azure development best practices for design considerations should include consistency and coherence in component design and deployment, maintainability to simplify administration and development and the reusability of components and subsystems. Decisions made during the design and implementation phase have a huge impact on the quality and the total cost of ownership of cloud-hosted applications and services.

Messaging

Cloud-based applications need a reliable communication channel that connects components and services ideally in a loosely coupled manner in order to maximize the scalability of distributed applications. Asynchronous messaging is widely used and provides many benefits, but also brings challenges such as the ordering of messages, poison message management and idempotency etc.

Considerations in Managing Cloud Solutions

Most cloud applications use the PaaS layer of the cloud. It brings great flexibility, scalability and expense reduction in development and administration efforts but can make management and monitoring more difficult than an on-premises deployment. Applications must expose runtime information that administrators and operators can use to manage and monitor the system, as well as support changing business requirements and customization without requiring the application to be stopped or redeployed.

Performance and Scalability

Performance is an indication of the responsiveness of a system to execute any action within a given time interval, while scalability is the ability of a system either to handle increases in load without an impact on performance or increase available resources. Cloud applications typically encounter variable workloads and peaks in activity. Predicting these, especially in a multi-tenant scenario, is almost impossible.

Resiliency

Resiliency is the ability of a system to gracefully handle and recover from failures. The nature of cloud hosting, where applications are often multi-tenant, 1) use shared platform services, 2) compete for resources and bandwidth, 3) communicate over the Internet and 4) run on commodity hardware. This means there is an increased likelihood that both transient and more permanent faults will arise. Detecting failures, and recovering quickly/efficiently, is necessary to maintain resiliency.

Security

Security is the capability of a system to prevent malicious or accidental actions outside of the designed usage and to prevent disclosure or loss of information. Cloud applications are exposed on the Internet outside the trusted on-premise boundaries. They are often open to the public, and may serve untrusted users. Applications must be designed and deployed in a way that protects them from malicious attacks, restricts access to only approved users and protects sensitive data.

Tips to Choose a Cloud Architecture

Download Our Guide 

Useful Cloud Design Patterns

Rather than sharing final comments, I’ve added the most-used design patterns for the cloud.

Circuit Breaker Pattern

This pattern can improve the stability and resiliency of an application — handling faults that may take a variable amount of time to rectify when connecting to a remote service or resource.

Competing Consumers Pattern

Enable multiple concurrent consumers to process messages received on the same messaging channel. This pattern enables a system to process multiple messages concurrently to optimize throughput, to improve scalability and availability and to balance the workload.

Federated Identity Pattern

Delegate authentication to an external identity provider. This pattern can simplify development, minimize the requirement for user administration and improve the user experience of the application.

Index Table Pattern

Indexes are a key feature of database management. Currently, they are presented in a new light for the cloud-based solutions.

Materialized View Pattern

The pre-populated view has returned to support the performance of the cloud data services. When the data is spread over couple of data tables it is better to combine it in a special view for the actual application needs.

Queue-based Load Leveling Pattern

Communication between the application in the cloud or an app in a hybrid environment is of high importance for the stability, the performance and the quality of the solution. To achieve better scalability and loose coupling, one can modify the project architecture by organizing the communication between the different layers of the solution through a message queue-like communication channel. Implementing messaging communication can minimize the impact on availability and responsiveness for both the task and the service in peak traffic situations.

Retry Pattern

To create a stable cloud application, the pattern must be applied on almost every layer of the application that communicates with external services. For example database or storage access, web services access, etc.

Static Content Hosting Pattern

This pattern can reduce the requirement for potentially expensive computing instances. With the Static Host Couple Pattern, static content is deployed to a cloud-based storage service that can deliver the resources directly to the client.

Enterprise Integration Patterns with Windows Azure Service Bus (1) – Dynamic Routers

The book Enterprise Integration Patterns by Gregor Hohpe et al. has been sitting on my desk for several years now, and I’ve always been planning writing a library that brings those patterns into easy-to-use building blocks for .Net developers. However , the motion of writing a MOM system by myself doesn’t sound quite plausible to myself, even though I’ve been taking on some insane projects from time to time. On the other hand, the idea is so fascinating that I keep looking for opportunities to actually start the project. This is why I was really excited when I first learned about Windows Azure Service Bus. Many of the patterns, including (but not limited to) Message Filter, Content-Based Router, and Competing Consumers, are either supported out-of-box or can be implemented with easy customizations. However, I want more. I want ALL the patterns in the book supported and can be applied to applications easily. So, after a couple years of waiting, I finally started the project to implement Enterprise Integration Patterns.

About the project

The project is open source, which can be accessed here. I’m planning to add more patterns to the project on a monthly basis (as I DO, maybe surprisingly, have other things to attend to). I’d encourage you, my dear readers, to participate in the project if you found the project interesting. And you can always send me comments and feedbacks to hbai@microsoft.com. At the time when this post is written, the code is at its infant stage and needs lots of work. However, it does demonstrates the basic ideas such as independent processing units and configuration supports.

Extended Windows Azure Service Bus constructs

As this is the first post of the series, I’ll spend sometime to explain the basic entities used in this project, and then give you an overview of the overall architecture.

• Processing Units

Windows Azure Service Bus provides two types of “pipes” – queues and topics, which are fundamental building blocks  of a messaging system. Customization points, such as filters and rules, are provided on these pipes so you can control how these pipes are linked into processing pipelines. This works for many of the simple patterns perfectly. However, for more complex patterns, these pipes are linked in more convoluted ways. And very often the connectors among the pipes assume enough significance to be separate entities. In this project I’m introducing “Processing Unit” as separate entities to provide greater flexibilities and capabilities to building processing pipelines.  A Processing Unit can be linked to a number of inbound channels, outbound channels as well as control channels. It’s self-contained so it can be hosted in any processes  –  it can be hosted on-premises or on cloud, in cloud services or console applications, or wherever else. And in most cases you can easily scale out Processing Units by simply hosting more instances. In addition, Processing Units support events for event-driven scenarios.

• Channel

Channels are the “pipes” in a messaging system. These are where your messages flow from one place to another. Channels connect Processing Units, forming a Processing Pipeline.

• Processing Pipeline

Process Pipelines comprise Processing Units and Channels. It provides management boundaries so that you can construct and manage messaging routes more easily. Here I need to point out that Processing Pipeline is more a management entity than runtime entity. Parts of  a Processing Pipeline can be (and usually are) scattered across different networks, machines and processes.

• Processing Unit Host

Processing Unit Host, as its name suggests, hosts Processing Units. It manages lifecycle of hosted Processing Units as well as governing overall system resource consumption.

Pattern 1: Dynamic Router

• Problem

Message routing can be achieved by a topic/subscription channel with filters that control which recipients get what messages. In other words, messages are broadcasted to all potential recipients before they are filtered by receiver-side filters. If we only want to route messages to specific recipients, Message Routers are more efficient because there’s only one message sent to designated recipient. Further more, routing criteria may be dynamic, requiring the routing Processing Unit to be able to take in feedbacks from recipients so that it can make informed decisions when routing new messages.

• Pattern

Dynamic Router pattern is well-defined in Gregor’s book, so I won’t bother to repeat the book here, other than stealing a diagram from it (see below). What makes this pattern interesting is that the recipients are provided with a feedback channel so that they can report their updated states back to the router to affect future routing decisions. As you can see in the following diagram, the rule processor (as well as its associated storage, which is not shown in the following diagram) doesn’t belong to any specific channels, so it should exist as a separate entity.

In my implementation the Dynamic Router is implemented as a Processing Unit, which is hosted independently from message queues. Specifically, DynamicRouter class, which has the root class ProcessingUnit, defines a virtual PickOutputChannel method that returns a index to the channel that is to be selected for next message. The default implementation of this method doesn’t take feedbacks into consideration. Instead, it simply selects output channels in a round-robin fashion:

protected virtual int PickOutputChannel(ChannelMessageEventArgs e)
{
  mChannelIndex = OutputChannels.Count > 0 ? (mChannelIndex + 1) % OutputChannels.Count : 0;
  return mChannelIndex;
}

Obviously the method can be overridden in sub classes to implement more sophisticated routing logics. In addition, the Processing Unit can subscribe to MessageReceviedOnChannel on its ControlChannels property to collect feedbacks from recipients. The architecture of the project doesn’t mandate any feedback formats or storage choices, you are pretty much free to implement whatever fits the situation.

• Sample Scenario – dynamic load-balancing

In this sample scenario, we’ll implement a Dynamic Message Router that routes messages to least pressured recipients using a simple greedy load-balancing algorithm. The pressure of a recipient is defined as the current workload on that recipient. When the router tries to route a new message, it will always routes the message to the recipient with least workload. This load-balancing scheme often yields better results than round-robin load-balancing. The following screenshot shows the running result of such as scenario. In this scenario, we have 309 seconds worth of work to be distributed to 2 nodes. If we followed round-robin pattern, this particular work set will take 161.5 seconds to finish. While using the greedy load-balancing, it took the two nodes 156.4 seconds to finish all the tasks, which is pretty close to the theoretical minimum processing time – 154.8 seconds. Note that it’s virtually impossible to achieve the optimum result as the calculation assumes any single task can be split freely without any penalties.

This scenario uses GreedyDynamicRouter class, which is a sub-class of DynamicRouter. The class subscribes to MessageReceivedOnchannel event of its ControlChannels collection to collect recipient feedbacks, which are encoded in message headers sent back from the recipients. As I mentioned earlier, the architecture doesn’t mandate any feedback formats, the recipients and the router need to negotiate a common format offline, or use the “well known” headers, as used by the GreedyDynamicRouter implementation. This implementation saves all its knowledge about the recipients in memory. But obviously it can be extended to use external storages or distributed caching for the purpose.

Pattern 2: Content-Based Router

• Problem

In many integration scenarios, we need capability to route messages to different recipients based on message contents. The messages should be sent to designated recipients instead broadcasted to all recipients to filter on.  In addition, it’s desirable to have routing logic defined in a centralized place instead of scattered across different recipients. Centralized routing logic is easier to maintain and it can avoid the risk of inconsistent filtering that may cause lose of message or duplicated processing.

• Pattern

The library doesn’t have a specific Content-Based Router - DynamicRouter class suffices in this case. You can easily override its virtual PickOutputChannel method to implement your custom routing logics. The method is triggered whenever a message is about to be routed, and you have access to the triggering channel as well as the message via method parameters.

• Sample Scenario – dynamic load-balancing

In this sample scenario, we’ll generate a number of messages with a header value randomly set to “Cars” or “Trucks”. Then, we implement a Content-Based Router and send all “Cars” requests to one channel and the rest to another. You can check out ContentBasedRouter method in test console application for detailed implementation. It’s really easy to implement such a router, as show in the following code snippet:

public class HeaderBasedRouter: DynamicRouter
{
  protected override int PickOutputChannel(Infrastructure.ChannelMessageEventArgs e)
  {
    if (e.Message.Headers.ContainsKey("ProductType"))
      return (string)e.Message.Headers["ProductType"] == "Cars" ? 0 : 1;
    else
      return base.PickOutputChannel(e);
  }
}

Note: because the routing logic is externalized, none of the recipients is aware of any routing logics or conditions. In the following screenshot you can see how tasks are routed to different processors based on their headers:

Summary

In this post we had a brief overview of the overall architecture of Enterprise Integration Patterns project, and we also went through the first couple of patterns – Dynamic Router and Content-Based Router. We are starting with harder patterns in this project so that we can validate and improve the overall infrastructure during earlier phases of the project. At this point I haven’t decided which pattern to go after next, but message transformation patterns and other routing patterns are high in the list. See you next time!

Microsoft Azure Integration Platform as a Service (iPaaS) – Logic Apps, Azure Functions, API Management and Service Bus working together

Introduction

If you work for an established organisation going through Digital Transformation or in a modern company born in the digital era, the chances are that IT is required to implement integration solutions more than ever before. Whether an organisation is an incumbent or a new entrant, they need to leverage the power of technology to lead the change or embrace it. Monolithic systems will not be capable of meeting the needs of digital organisations, creating the need to interconnect existing systems with best-of-breed cloud and distributed apps and to expose all these systems to other systems, business partners or consumer apps through easy to consume APIs.

Microsoft Azure provides different technologies which enable us to build robust application, data and process integration solutions. One of the core offerings for Azure integration is Logic Apps – however there are other Azure PaaS offerings which when used with Logic Apps form the robust building blocks of first-class integration solutions. These technologies together provide a very rich and fully-managed Integration Platform as a Service (iPaaS). In this post, I will try to describe these Azure technologies that we can leverage to make our life easier, and perhaps even more fun, when implementing enterprise integration solutions.

Azure Services to build integration solutions

Out of the very large and growing list of Azure services, there are some that we can utilise to build our integration solutions, including Logic Apps, Azure Functions,  API Management, and Service Bus Messaging. I will briefly describe each of them below.

Logic Apps

Logic Apps is a very robust and powerful platform to orchestrate and automate integration workflows. The visual designer together with all the available connectors, and deployment and management tools make it a great tool for this kind of scenarios. Logic Apps also provides the concept of serverless computing, in which you just focus on what you want to achieve, without worrying at all about servers, patching, scaling, etc.

Workflow and Orchestration Capabilities

Logic Apps workflows, which can be easily designed and implemented graphically (via the Azure Portal or Visual Studio), are based on the Workflow Definition Language. These workflows provide rich ways to process and manipulate data than can be obtained or pushed via different connectors. Below there is a figure which shows a simple Logic App workflow with different actions, a condition and a loop.

In Logic Apps workflows we can implement different actions, like calling an HTTP endpoint, calling Azure API Apps, calling WebHooks, waiting, calling a Logic App nested workflow, implementing conditions and loops, query arrays, terminating a workflow, or returning an HTTP Response.

Triggers, Input and Outputs

Logic Apps provides a growing list of connectors that allow us to trigger workflows and get data from and push data to many different protocols, SaaS applications, other Azure and Power Apps Services, and on-premises systems. Below there is a snapshot of the 100+ connectors available at the time of writing (few of them are still in preview).

Protocol Connectors

Azure Services and Power Apps Connectors

SaaS Connectors

B2B, XML, EDI and AS2 Connectors

Hybrid Connectors

As we can see, with all these connectors, Logic Apps allows us to very easily and quickly connect to many different protocols, apps, and also to other Azure Services. Additionally, the Enterprise Integration Pack brings some of the BizTalk Server functionality to Logic Apps and makes really easy to implement B2B integrations based on AS2 and EDI.

Azure Functions

Another Azure service that is very useful when building integration solutions is Azure Functions. Azure Functionsare the evolution of Azure WebJobs, which allow us to create microservices or small pieces of code that can run synchronously or asynchronously as part of composite and distributed cloud solutions. Azure Functions are built on top of the WebJobs SDK but with some additional benefits and capabilities, including the option of being deployed on a serverless-style Dynamic Service Plan, in which you pay per consumption only, and also the capability of being triggered via HTTP. Additionally, with Functions you can deploy from very simple to quite complex pieces of code developed on different languages, such as bash (.sh), batch (.bat / .cmd), C#, F#, Node.Js,PHP, PowerShell, and Python.

Triggers, Inputs and Outputs

Azure Functions support different triggers, input and output bindings, which I summarise in the table below. These bindings make Functions very easy to be called and allow them to get data from and push data to other microservices, Logic Apps or other systems or services.

Type / Service	Trigger	Input	Output
Schedule	*
HTTP Call	*	*	*
Azure Blob Storage	*	*	*
Azure Event Hubs	*	*	*
Azure Storage Queues	*	*	*
Azure Service Bus  Messaging	*	*	*
Azure Storage Tables		*	*
Azure Mobile Apps  Tables		*	*
Azure DocumentDB		*	*
Azure Notification  Hubs			*
Twilio SMS Message			*
SendGrid Emails (Not  fully  documented yet)			*
Files in Cloud File  Storage  SaaS, such as  Box,  DropBox,  OneDrive, FTP  and  SFTP (Not fully    documented yet)	*	*	*

Processing

Integration Processes usually require custom data and message validation, enrichment, transformation or routing. These can simply be done via custom code on Azure Functions.

Service Bus Messaging

Azure Service Bus Messaging provides reliable message queuing and publish-subscribe messaging capabilities. Azure Service Bus Queues and Topics allow us to decouple in time upstream and downstream systems or different interrelated services. Service Bus Queues provide ordered message delivery to competing consumers, while Service Bus Topics enable publish-subscribe messaging scenarios where multiple consumers can process the same message.

The following Service Bus features make it a key component of most integration solutions on Azure:

• temporal decoupling;
• the ability to load balance message processing using competing consumers;
• the capability of implementing Publish/Subscribe architecture via Service Bus Topics;
• and the fact that Functions and Logic Apps can very easily read and write messages from and to Service Bus.

API Management

API Management is an Azure Service which functions as a API Gateway or Front-End for backend APIs in the cloud or on-prem. In addition, it provides a Developer portal which helps to speed up the adoption and use of the implemented APIs. Some of the benefits of API Management are:

• Ability to secure APIs by implementing different authentication and authorisation methods, like Azure AD or use of client certificates
• Managing consumer accounts
• VNET connectivity to connect to other secured resources
• Caching to improve API response times
• Ability to throttle requests based on the caller or the product
• API call routing or forwarding
• Transforming requests, by changing the method or implementing richer transformation policies
• Tracing calls, or logging requests via Event Hubs
• Monitoring and analysing usage and health of APIs
• Having a developer’s portal which allows to publicise APIs and speed up adoption and the on-boarding process
• Ability to monetise APIs

Thanks to all these features, API Management can be leveraged on many integration solutions which require to expose RESTful APIs and require any kind of mediation.

 

When to use each of these technologies and how they complement each other

Now that I have described the Azure iPaaS offerings, it’s worth analysing when we could use each of these technologies and how they complement each other. I summarise this in the table below.

Technology	When to use it	Use together with
Logic Apps	To implement and orchestrate visually designed integration workflows. To orchestrate distributed microservices. To leverage the 100+ connectors to interact with different protocols, SaaS systems and services, other Azure services, and on-premises systems. To implement cloud-based B2B integrations with AS2 and EDI.	Functions - so custom logic can be implemented in microservices to be orchestrated by LogicApps. Service Bus - to decouple in time different microservices and steps in the integration process. API Management - for those HTTP triggered apps when some of the capabilities of API Management are required.
Azure  Functions	To implement code-based microservices or processing.	Logic Apps - so different microservices can be orchestrated. Azure Service Bus Messaging - to decouple in time different microservices. API Management - for those HTTP triggered functions when some of the capabilities of API Management are required.
Service Bus	To decouple in time upstream systems from downstream systems or different microservices. To implement multiple consumers on a Publish/Subscribe pattern. To allow load distribution with multi-instance competing consumers.	Functions and Logic Apps - to decouple in time different microservices and steps in the integration process.
API  Management	When any of the API Management features is required, for example: securing backend APIs, API response caching, request throttling, request routing, request transformation, API calls tracing or logging, usage and health analytics, or providing a rich portal for developers.	Functions and Logics Apps - that are triggered by an HTTP call and require some kind of mediation.

 

Summary

Azure provides a very robust Integration Platform as a Service (iPaaS), which is based on Logic Apps and can be complemented with Azure Functions, Service Bus Messaging and API Management

The breadth and capabilities of many different Azure technologies and how they complement each other is what differentiates Azure against other iPaaS vendors. We can leverage many different services to build first-class integration solutions. Logic Apps is the core engine to implement these. Logic Apps can connect to many different protocols, SaaS apps and Services, to other Azure and Power App Services, to on-premises systems and to B2B trading partners via a growing list of connectors. Logic Apps integration workflows can easily be extended with custom code implemented as microservices on Azure Functions. In order to decouple in time these integration processes, we can leverage Service Bus Messaging services. And in case we need to expose our integration services as RESTful APIs, we might want to make use of all the features and capabilities of API Management. Additionally, these integration solutions can be enhanced by other Azure services, such as Cognitive Services to, for example get the sentiment from social media feeds or Azure Active Directory for authenticating and authorising calls to our APIs. All of this with all the PaaS magic and the powerful DevOps capabilities of Azure.

Thanks for reading, and happy clouding!