Microservices with minimum overhead using ASP.NET Web API and Azure – part 2 – Deployment

If you haven’t read the first part, I strongly recommend to do so.
Just to remind, my goal is to create environment that lets us build fast, but build a system that can be easily scaled in the future. Last week, I’ve proposed architecture of .NET service oriented system, which can be both hosted on a single machine and easily spread on multiple microservices. In short, it consists of multiple .NET class libraries containing ASP.NET MVC (Web API) controllers. They can be hosted just by referencing them from one or multiple Web Application projects.

Below, a draft presenting both scenarios:

This week I’m going to focus on how to use Azure PaaS level services to leverage goal of building system fast.

SHOW ME WHAT YOU GOT – DEPLOYMENT

We all know it is important to deliver pieces of software to product owners and stakeholders as soon as possible so they can see if we are building what they really need. For this reason we want to setup environment for deployments quickly. Time spent on configuring and managing servers is a waste here. Instead, we can deploy system to one of Platform as a Service solutions giving us a lot of setup out of the box. With them, you can really quickly configure platform that runs application without bothering you with machine performance, security, operating system, server management etc.

Let’s quickly deploy source code of initial version of MarketFinder, created in the previous post
https://github.com/FutureProcessing/Microservices—ASP.NET-Web-API—Azure/tree/analytics_service_impl

1. Create Azure SQL database
   1. Go to portal.azure.com,
   2. Select New -> Data + Storage -> SQL Database,
   3. Enter DB name, select or create new server – this is only for authorization purpose,
   4. Select Pricing tier – basic is sufficient for this moment,
   5. Create resource group (you should create one new group for all services associated with particular system),
   6. Use CreateDatabaseTables.sql script from repository to create its structure – initially this can be done manually, however we will soon automate this process to make sure, DB is always up to date with code.
2. Create new Web App
   1. Select New -> Web + Mobile -> Web App,
   2. Enter name, resource group (one created with database),
   3. Select pricing plan. For sake of early testing and publishing application to stakeholders, free or shared plan is perfect. You can scale it up in seconds, at any time.
3. Make them work together. We are going to configure Web App to use created SQL Database by overriding web.config settings. This Web Apps feature lets us avoid storing environment dependent keys and passwords in source code repository
   1. Go to created SQL Database blade,
   2. Click “Show database connection string”,
   3. Copy ADO.NET connection string,
   4. Go to created Web App blade,
   5. Select Settings, then Application Settings, scroll down to Connection Strings section and paste database connection string. Do not forget to fill it with password to your server. Azure can’t get it for you because it is hashed.

4. Configure continuous deployment
   1. Go to Web App blade,
   2. In Deployment section click “Set up continuous deployment”,
   3. Follow wizard to setup deployment of your choice. In my case, I login to Github, select project linked above and select master branch. You can try this by forking my Github project.
   4. Every time we deploy something to master branch, it is automatically deployed to Web App,
   5. If the above solution do not fit your process, Web App can be configured to wait until we explicitly push source code into it.

Voilà, our application is up and running.

GOING LIVE – PRODUCTION ENVIRONMENT

Created environment is good for development and testing purposes, however before we ship it to public we should care about more aspects. Let’s list some of them, and see what we need to achieve with and without Azure PaaS services.

[table “14” not found /]

TIME FOR SCALING

Everybody wishes to be successful with their application. However, when this happens, one can realize he cannot handle such popularity. Let’s see how we are prepared to scale.

Scale using Azure capabilities

The simplest thing you can do is to scale up, to do so just change pricing plan for service to the higher one. For most PaaS services it takes just few seconds and does not cause service downtime. However, this approach is limited to what machine behind the highest pricing plan can handle.

Another option is scaling out. That means to create multiple instances of our service which are automatically load balanced. To do this we can simply move slider in Azure Portal, or better, configure rules for autoscaling. Typical ruleset will increase number of service instances when average CPU utilization is higher than, for example 50% for 10min, and decrease it when it is lower than 20% for 20min.

There is one important thing to remember. To scale out your application must be stateless, all states need to be moved to external store, like SQL Database of Redis cache. It is good to keep this in the back of your head since the beginning of project development.

Scale by breaking code on services

At some point, the above solution might be insufficient. For example, your SQL Database does not handle traffic, even at the highest pricing plan. Or, simply one component is using a lot of CPU at some point in time and we do not want the whole application to suffer because of this.

Architecture proposed in the previous post lets us simply extract number of components of the system to independent hosts. Then, we can deploy them to independent Azure Web App and SQL databases.

OPTIMIZING COSTS

When our system is getting more traffic, the entire environment can cost quite a sum of money every month. It is hard to anticipate whether it is cheaper to host system on a small number of big services, or on a lot of small ones. With proposed architecture, it is relatively easy to create both configuration, perform tests and look for cost optimum setup for particular system.

Another important aspect of cloud, is that you can easily scale or even shut down services when they are not used. This can be easily automated, and in some cases can cut our costs by more than half. So even though Azure or AWS services might be more expensive than local datacenters, having mentioned flexibility you can pay only for traffic you actually get, not just for being ready to handle a lot of it.

SUMMARY

In this article I’ve tried to present how we can use PaaS level services available in Azure, to both build MVP fast, and meet requirements of live systems when we get to this stage. When buying such services from Microsoft, we are saving time for managing servers and network. Additionally, we are able to achieve system high availability, scalability, security and monitoring without hiring army of Ops.

I’ve briefly described just a couple of services available in Azure, however there are way more of them; just to mention some: Storage, Document DB, Azure Search, Mobile Apps, Logic Apps, Machine Learning. You can really solve most of your problems without leaving comfort of sold as a service platforms.

Microservices with minimum overhead using ASP.NET Web API and Azure – part 1 – Architecture

In the era of building systems that aim to provide services at global scale, requirements for scalability and high availability are becoming our bread and butter. What is more, it is absolutely normal that stakeholders want first shippable version of software as soon as possible.

Due to recent hype, microservices architecture comes to our mid as the first answer to mentioned challenges. However, as it is commonly known: “(…) microservices introduce complexity on their own account. This adds a premium to a project’s cost and risk – one that often gets projects into serious trouble” (Martin Fowler). We can see this requirement does not help to deliver system fast. So where is the golden mean? How to build fast, but be prepared to scale and provide high availability without dramatic, expensive changes to system? Let me propose architecture that answers those questions. At least some of them.

OVERVIEW

The main idea of this article is to organize system into multiple lightweight, logical, decoupled components, rather than multiple independently hosted services. With this approach we can start with hosting all components in single service and dividing them into multiple ones overtime.

It looks nice as an idea, however what actually hosts and those “mysterious” components here are.

Hosts are, in short, ASP.NET Web API (or MVC) applications that can be hosted as Azure Web Apps or started on developers’ machine. Their only responsibility is to host and initially configure application, but there should be no application logic, or simply: no controllers.

What is this component then? Let’s zoom in!

In the suggested architecture, component is a bunch of Web API (MVC) controllers with all related business logic. All those classes are placed outside of ASP.NET Web API host application, usually in simple class library. Single component should consist of controllers and logic related to single business functionality. Usually, each component has its own place to store data, it can be common or dedicated database.

To put all this together, host project just need reference class libraries containing components code. Since now, controllers are available to process HTTP requests sent to host application. Moving component to separate host is as simple as changing reference between projects.

With this approach we benefit from most of microservice architecture advantages, and initially limit pain that comes with it by keeping development environment easy to run on one machine, limiting DevOps work, and cutting off a lot of distributed system problems. Additionally, we achieve flexibility of production environment costs. Initially limited number of machines do not dry out our accounts, however, when necessary, we are able to distribute components to multiple independent hosts, achieving better scalability.

BUILDING ACTUAL SYSTEM

Let’s assume we are creating social system, where users can find and rate nearby shops and markets. For scope of this article we want to:

• Create, read, update and delete data about markets and user’s ratings.
• Generate suggestions what another markets user may like too.

And of course our stakeholders want the first version of project quickly, however, at some point after release, they expect system to handle millions of users from entire word.

Let’s build it

Source code of a sample project described in this article can be found here:
https://github.com/FutureProcessing/Microservices—ASP.NET-Web-API—Azure

1. Project structure
   First of all, we need basic project structure.
   1. Create empty ASP.NET Web Application with Web API.
   2. Create empty Class Library and install Microsoft.AspNet.WebApi NuGet Package in it.
   3. Add reference from Web API project to Class Library.
After those steps your solution should look like this:

2. Market management service implementation
As the next step, we want to implement CRUD operations within MakretManagementService. For simplicity of this example I’ve just scaffolded MarkedController and RatingsController using Visual Studio tools.
Source code of this state of system can be found in repository at following revision
https://github.com/FutureProcessing/Microservices—ASP.NET-Web-API—Azure/tree/market_management_service_impl
Now, we can test if this solution actually works. To do so we need appropriate tool e.g. Fiddler or PostMan to send HTTP requests to our services. Works on my machine!
3. Create Analytics service
   The last feature we need before release is suggestions generation. Such feature is expected to be time consuming operation that we want to run periodically, and store results in database so they can be accessed by client applications later on. What is more, in future we may not want this logic to run at the same machine that MakretManagementService, because this heavy operation might slow system down. For this reason we are going to create it as a separate component.To achieve that we just need to:
   1. Create empty Class Library and install Microsoft.AspNet.WebApi NuGet Package in it.
   2. Add reference from Web API project to Class Library.
   Code itself is not important here, so just skip discussion on this.
   Source code of this state of system can be found in repository at following revision
   https://github.com/FutureProcessing/Microservices—ASP.NET-Web-API—Azure/tree/analytics_service_impl
   Now we can send POST request to http://<<base_address>/RecommendationsAnalysis and have our suggestions created.
   At this stage system looks as follows:
   
   It is important to notice at this point that there is absolutely no coupling between two services on the code level, however as you might noticed it doesn’t come for free. There is some duplication in data access code and data model classes. Although, at this moment, it can be generalized and placed in common class library, it may not be the best solution when we think about long term perspective and further independent development of those services. There are no good and bad solutions, there are only pros and cons in particular context.
   Sounds like we are ready to ship our Minimum Viable Product to customers!

Time for scaling

Until now we have created application optimized for low development and hosting costs. It runs as a single application and store data in single database. Let’s now react to scenario where we already have a lot of data about markets, single database is getting large, calculating user recommendation takes long enough to significantly slow website down for long periods.

Some improvements that should help in a given problem are:

1. Move recommendations data to different database.
2. Run Analysis service in separate host application, so we can run it in independent container.

The first change is nearly trivial. We need to create another database with Recommendations table. Next, because our system is already running, we need to migrate data from first database. After that there is a single connection string to change in web.config.

Such change will be that easy only if we took care not to introduce coupling between data behind different components. It is not simple and, again, usually requires some data duplication and synchronization, however that is the cost for possibility of keeping parts of system independent. If you are familiar with Domain Driven Design concept of bounded contexts, you have probably noticed that what I’m suggesting here is similar to concept of aggregates and persisting them as an internally cohesive, decoupled from each other set of entities.

Changes to database creation scripts and project configuration can be analyzed here:
https://github.com/FutureProcessing/Microservices—ASP.NET-Web-API—Azure/commit/scale_db

The second change isn’t complicated too, all you need to do is to create new host projects (same way as previously) and add references to appropriate class libraries.

Changes to C# code can be analyzed here:
https://github.com/FutureProcessing/Microservices—ASP.NET-Web-API—Azure/commit/scale_csharp 

Now you can enjoy system running its components in two microservices. From broader perspective it works this way:

You might have noticed that I’ve left in solution MarketFinder.CommonHost project that is hosting both applications. This project is useful during development, as it is usually faster to start single web site than multiple ones, especially when we have a lot of them.

SUMMARY

Most of community advice against starting system development with microservice architecture, because it comes with huge costs and complexity overhead. In return, it is recommended to build monolithic system and extract microservices overtime when required. Advised approach sounds great, but to divide system into services in future, we need to make sure we do not create highly coupled ball of mud. Architectural approach presented in this article may help us by creating some logical boundaries of system components. Having them prevent us, and our less experiences teammates, from introducing hidden coupling.

However, nothing in software development is the silver bullet. Suggested approach introduces less complexity and cost overhead than fully blown microservices architecture, however it still does. The biggest problem when cutting system on independent components, is that it can never be done without actually cutting of some dependencies that exist. If we find boundaries where those dependencies are minimal, we will benefit from low coupling in future, however when we set those boundaries wrongly, we will heavily suffer from development complexity and probably poor system performance.

Application Design: Going Stateless on Azure

★★★★

★★★

★★

★

Application Design: Going Stateless on Azure

The components of a cloud application are distributed and deployed among multiple cloud resources (virtual machines) to benefit from the elastic demand driven environment. One of the most important factor in this elastic cloud is the ability to add or remove application components and resources as and when required to fulfill scalability needs.

However, while removing the components, this internal state or information may be lost.

That’s when the application needs to segregate their internal state from an in-memory store to a persistent data store so that the scalability and reliability are assured even in case of reduction of components as well as in the case of failures.  In this article, we will understand ‘being stateless’ and will explore strategies like Database-driven State Management and Cache driven State Management.

Being stateless

Statelessness refers to the fact that no data is preserved in the application memory itself between multiple runs of the strategy (i.e. action). When same strategy is executed multiple times, no data from a run of strategy is carried over to another. Statelessness allows our system to execute the first run of the strategy on a resource (say X) in cloud, the second one on another available resource (say Y, or even on X) in cloud and so on.

This doesn’t mean that applications should not have any state. It merely means that the actions should be designed to be stateless and should be provided with the necessary context to build up the state.

If our application has a series of such actions (say A1, A2, A3…) to be performed, each action (say A1) receives context information (say C1), executes the action and builds up the context (say C2) for next action (say A2). However, Action A2 should not necessarily depend on Action A1 and should be able to be executed independently using context C2 available to it.

How can we make our application stateless?

The conventional approach to having stateless applications is to push the state from web/services out of the application tier to somewhere else – either in configuration or persistent store. As shown in diagram below, the user request is routed through App Tier that can refer to the configuration to decide the persistent store (like, database) to store the state. Finally, an application utility service (preferably, isolated from application tier) can perform state management

The App Utility Service (in the above diagram) takes the onus of state management. It requires the execution context from App Tier so that it can trigger either a data-driven state machine or an event-drive state machine. An example of state machine for bug management system would have 4 states as shown below

To achieve this statelessness in application, there are several strategies to push the application state out of the application tier. Let’s consider few of them.

Database-drive State Management

Taking the same bug management system as an example, we can derive the state using simple data structures stored in database tables.

Current State	Event	Action	Next State
START	NewBug	OpenNew	Bug Opened
Bug Opened	Assigned	AssignForFix	Fix Needed
Not A Bug	MarkClosed	Bug Closed
Fix Needed	Resolved	MarkResolved	Bug Fixed
ReOpened	AssignForFix	Fix Needed
Bug Fixed	Tested	MarkClosed	Bug Closed
ReOpened	MarkOpen	Fix Needed
Bug Closed	–	–	END

The above structure only defines the finite states that a bug resolution can visit. Each action needs to be context-aware (i.e. minimal bug information and sometimes the state from which the action was invoked) so that it can independently process the bug and identify the next state (especially when multiple end-states are possible).

When we look at database-drive state management on Azure, we can leverage one of these out-of-the-box solutions

• Azure SQL Database: The Best choice when we want to work with relational & structured data using relations, indexes, constraints, etc. It is a complete suite of MS-SQL database hosted on Azure.
• Azure Storage Tables: Works great when we want to work with structured data without relationships, possibly with larger volumes. A lot of times better performance at lower cost is observed with Storage Tables especially when used for data without relationships. Further read on this topic – SQL Azure and Microsoft Azure Table Storage by Joseph Fultz
• DocumentDB: DocumentDB, a NoSQL database, pitches itself as a solution to store unstructured data (schema-free) and can have rich query capabilities at blazing speeds. Unlike other document based NoSQL databases, it allows creation of stored procedures and querying with SQL statements.

Depending on our tech stack, size of the state and the expected number of state retrievals, we can choose one of the above solutions.

While moving the state management to database works for most of the scenarios, there are times when these read-writes to database may slow down the performance of our application. Considering state is transient data and most of it is not required to be persisted across two sessions of the user, there is a need of a cache system that provides us state objects with low-latency speeds.

Cache driven state management

To persist state data using a cache store is also an excellent option available to developers.  Web developers have been storing state data (like, user preferences, shopping carts, etc.) in cache stores ever since ASP.NET was introduced.  By default, ASP.NET allows state storage in memory of the hosting application pool.  In-memory state storage is required following reasons:

• The frequency at which ASP.NET worker process recycles is beyond the scope of application and it can cause the in-memory cache to be wiped off
• With a load balancer in the cloud, there isn’t any guarantee that the host that processed first request will also receive the second one. So there are chances that the in-memory information on multiple servers may not be in sync

The typical in-memory state management is referred as ‘In-role’ cache when this application is hosted on Azure platform.

Other alternatives to in-memory state management are out-of-proc management where state is managed either by a separate service or in SQL server – something that we discussed in the last section.  This mechanism assures resiliency at the cost of performance.  For every request to be processed, there will be additional network calls to retrieve state information before the request is processed, and another network call to store the new state.

The need of the hour is to have a high-performance, in-memory or distributed caching service that can leverage Azure infrastructure to act as a low-latency state store – like, Azure Redis Cache.

Based on the tenancy of the application, we can have a single node or multiple node (primary/secondary) node of Redis Cache to store data types such as lists, hashed sets, sorted sets and bitmaps.

Azure Redis cache supports master-slave replication with very fast non-blocking first synchronization and auto-reconnection on net split. So, when we choose multiple-nodes for Redis cache management, we are ensuring that our application state is not managed on single server. Our application state get replicated on multiple nodes (i.e. slaves) at real-time. It also promises to bring up the slave node automatically when the master node is offline.

Fault tolerance with State Management Strategies

With both database-driven state management and cache-driven state management, we also need to handle temporary service interruptions – possibly due to network connections, layers of load-balancers in the cloud or some backbone services that these solutions use. To give a seamless experience to our end-users, our application design should cater to handle these transient failures.

Handling database transient errors

Using Transient Fault Handling Application Block, with plain vanilla ADO.NET, we can define policy to retry execution of database command and wait period between tries to provide a reliable connection to database. Or, if our application is using any version of Entity Framework, we can include SqlAzureExecutionStrategy, an execution strategy that configures the policy to retry 3 times with an exponential wait between tries.

Every retry consumes computation power and slows down the application performance. So, we should define a policy, a circuit breaker that prevents throttling of service by processing the failed requests. There is no-one-size-fits all solution to breaking the retries.

There are 2 ways to implement a circuit breaker for state management –

• Fallback or Fail silent – If there is a fallback mechanism to complete the requested functionality without the state management, the application should attempt executing it. For example, when the database is not available, the application can fallback on cache object. If no fallback is available, our application can fail silent (i.e. return a void state for a request).
• Fail fast – Error out the user to avoid flooding the retry service and provide a friendly response to try later.

Handling cache transient errors

Azure Redis cache internally uses ConnectionMultiplexer that automatically reconnects to Redis cache should there be disconnection or Internet glitch. However, the StackExchange.Redis does not retry for the get and set commands. To overcome this limitation, we can use library such as Polly that provide policies like Retry, Retry Forever, Wait and Retry and Circuit Breaker in a fluent manner.

The take-away!

The key take-away is to design applications considering that the infrastructure in cloud is elastic and that our applications should be designed to leverage its benefits without compromising the stability and user experience. It is, hence, utmost important to think about application information storage, its access mechanisms, exception handling and dynamic demand.