Cloud Scale, means Out, not Up

Traditional application architectures have relied on scaling to handle increases in load by adding more power to the server. This has meant increasing the amount of RAM, or moving to a faster, more powerful CPU. This is called scaling up. The shift to the cloud has changed the way application architectures are designed, and it has meant an increase in distributed systems. Distributed systems are built to scale out rather than up; this means adding additional servers to distribute the load across a multitude of nodes in the system. There are many benefits to scaling out, and this article walks through a few of them to explain the benefits to scaling out, rather than up.

ScalingDefinition

Pitfalls of Scaling Up

There are many times when scaling UP is a valid option, and it still does have a valid place in cloud based systems. However, there are some drawbacks to relying solely on scaling up and the only option of scaling server resources. These drawbacks are the same in the cloud as they are on-premises, and have been on-premises forever.

Here are some drawbacks to solely relying on scaling up:

  • Servers can only be made so big. Eventually you wont be able to add more resources necessary to scale properly.
  • Scaling up may be able to add the necessary resources, but a single server is still a single point of failure that can bring and entire system down.
  • Relying on moving to larger and larger servers can get very expensive, since it can mean (when running on-premises) abandoning old hardware and completely replacing it.

ScalingUp

Benefits of Scaling Out

While scaling up has a place in the cloud, it’s generally better to architect solutions to support scaling out instead. Scaling out addes more server instances to a system and can achieve higher levels of redundancy and scalability.

Here are some benefits to scaling out:

  • There’s no limit to the number of server you can add.
  • Scaling out adds redundancy, and load balancing offers seamless failover for reaching high availability.
  • Stateless systems are designed to scale out, and can benefit greatly from spreading the load across servers.
  • Stateful systems may need extra services for state caching / sharing, and may require more work to scale out. But, once scaled out the first time, they generally are as easy to scale out as a Stateless system is.

ScalingOut

Up and Out

Most enterprise systems will rely on 2 server to start. This give the system basic redundancy and failover ability to achieve a basic level of high availability. Generally when these systems go live into production for the first time they are using smaller server instance sizes. Then as the app grows in usage there is a need to add more server resources.

To add more server resources, there are 2 options: Up and Out. There are perfectly valid times when scaling either Up or Out will work for a given solution. Depending on how the enterprise system is architected, it may be valid to scale up or out with the same result in many cases. This is especially true if the system already has redundancy of 2 or more servers.

However, there are times when scaling Up or Out is a decision that needs to be considered carefully. Here are some scenarios to think about when deciding to scale:

  • The hosting / operation cost of the system needs to be considered when scaling up or out. Sometimes it may be cheaper to one over the other giving your organization some cost savings on the monthly bill.
  • Choosing to scale out can offer expanded redundancy and a higher level of availability with lower system downtime or degraded performance in case of a server or data center outage, or even a spike in load.

There are many other points to keep in mind when deciding how to scale. These are just a couple examples.

Happy scaling!

Microsoft Azure – Security Center Partner Integration

Being very ‘partner-centric’ has always been core to Microsoft’s success. This has never been more true than in the cloud era. Microsoft’s biggest bet and largest platform is, of course, Azure, and it will certainly take partners of all kinds to help Microsoft win in this market.

A great example of this close relationship with partners is the way Microsoft releases their own services in Azure that directly integrate with partner solutions. Azure Security Center, is a case in point. Security Center solves a huge problem for customers taking their first leap into Azure. Once a customer deploys a workload in the public cloud, how can they be sure they followed security best practices? This can be hard to answer if you’re new to the platform. Security Center helps to bridge this gap through prevention and detection. Security Center helps prevent issues by evaluating a customer’s deployments and offering recommendations on ways to improve the security configuration. Beyond these recommendations, Security Center also leverages machine learning and an expansive set of telemetry from Microsoft’s global security datasets, to detect probable and certain security incidents as they happen.

This is great, but the aspect of Security Center I want to highlight is the integration with partner offerings. To show you how this works, I’ll quickly deploy the solution.

Deploying a WAF Solution from Security Center

Here is a view of Security Center’s Prevention tiles.

Security Center - Prevention Tiles

Security Center – Prevention Tiles

Notice the Application section on the Resource security health tile (outlined in purple). This is showing me that there is a high-severity application health finding. If I click on that, I can see what the finding is.

High Severity Finding

High Severity Finding

If I then click on the Web Application at the bottom, I see a recommendation.

Security Center Recommendation

Security Center Recommendation

Click on the recommendation and you can add a web application firewall. Notice this is still within Security Center, in the context of the specific finding. Clicking on Create New shows me the partner web application firewalls that can be deployed from within Security Center. For illustration sake, I’m choosing the F5 solution.

Choosing the Partner Solution

Choosing the Partner Solution

On the Azure Marketplace page for the F5 web application firewall, notice that this solution will be deployed and configured automatically, so that your web app will be protected against the most common threats and vulnerabilities.

F5 WAF Marketplace Page

F5 WAF Marketplace Page

When I click Create I fill out a few bits of information and my web application firewall starts deploying. It will take about 20 minutes (or so) to complete.

Once the solution is deployed, there is one final step to configure the web application firewall. Back on the Resource Security Health tile, click on Applications.

Security Center - Applications Findings

Security Center – Applications Findings

Within Applications notice that the Application Recommendations lists Pending WAF finalization. Click this recommendation.

Pending WAF Finalization

Pending WAF Finalization

On the Pending WAF finalization menu click your web application. On the subsequent dialog click Finalize web application firewall setup.

Security Center Recommendation

Security Center Recommendation

The next step involves the manual step of pointing your web site’s DNS name to the public IP address of the web application firewall. Once you have done this, check the box beside I updated my DNS record and click Restrict traffic.

Final Step to Restrict Traffic

Final Step to Restrict Traffic

At this point, rules are written in a Network Security Group that effectively force all incoming traffic through the firewall. Now your web application is protected!

Network Security Group is Altered

Network Security Group is Altered

Note the ‘green’ status of the application now

Application Finding Remediated

Application Finding Remediated

The integration goes deeper than just deploying the partner solution from Security Center. After the deployment, security threats that are detected by the F5 solution are fed back into Security Center, so I can see and investigate the findings within the Security Alerts tile.

Security Center Alerts

Security Center Alerts

When you click on an alert you can see that many were detected by the F5 web application firewall, and fed into Security Center.

F5 Events Logged in Security Center

F5 Events Logged in Security Center

Clicking on an alert gives more information, and gives the correct path so you can view the finding within the F5 interface.

More Detail in Security Center

More Detail in Security Center


F5 Management Interface - Security Event

F5 Management Interface – Security Event

Conclusion

I hope this quick overview of the F5 Web Application Firewall, as deployed from Azure Security Center, illustrates the deep level with which Microsoft is working with their partners. This level of integration, both from the ease of deployment, and the sharing of security incident information, is a powerful example of strong partnerships bringing amazing value to customers. For more information check out the links below:

F5 BIG-IP Virtual Editions: https://www.f5.com/pdf/products/big-ip-virtual-editions-datasheet.pdf

Request F5 Trial Licenses: https://interact.f5.com/azuref5trial.html

Azure Security Center: https://azure.microsoft.com/en-us/services/security-center

Azure IoT Architecture: 2-way Messaging

Inter-Application messaging is important. For this reason there are a couple different Messaging options within Microsoft Azure; Storage Queues, Azure Service Bus Queues/Topics. However, these are message queues meant to facilitate the sending and receiving of inter-application messages between disparate application components. When it comes to the Internet of Things, systems need even more robust messaging that these Message Queue offerings provide. For this reason, Microsoft created the Azure Event Hubs to provide a highly scalable method of receiving a very large stream of message ingress. Event Hubs handles the scale with being able to scale up to log millions of events per second. Although, Event Hubs are just for ingress. What about egress? What about sending messages back to application components or Internet of Things (IoT) devices? This article goes through an explanation on how the Azure IoT Hub service builds on top of Event Hubs to provide a secure 2-way messaging platform for IoT integration.

Device-to-Cloud Messaging

Device-to-Cloud Messaging is sort of the traditional method of messaging using Queues. A device or application sends messages to the Queue that are picked up by a receiver of some kind on the other end. These messages can be information for new orders coming into the system, or from an IoT device they could be sensor reading telemetry, or status updates to report the health of the device. There are many different kinds of messages that can be sent from IoT devices; the main key of Device-to-Cloud is that this is the ingress stream of messages received by the Message Queue.

For this ingress there are a couple different options of services to use in Azure. For lower scale you could use Storage Queues or Service Bus Queues/Topics. However, for a much larger ingress stream of messages, you’ll want to use Event Hubs or IoT Hubs. Event Hubs will handle the scale, but IoT Hubs extends scalability with additional Device Management features that are extremely useful with handling messages from dozens, hundreds, even millions of devices.

Cloud-to-Device Messaging

Cloud-to-Device messaging is the method that used to send messages to individual devices or applications from the system running in the Azure cloud. The types of messages send to a specific application instance or IoT Device can be in multiple forms. These messages are used to send command and control, or other messages to individual or groups of devices. Here’s a list of some of the examples of Cloud-to-Device messages that could be sent:

  • Configuration setting updates
  • Signals to tell the device to start or stop reporting sensor telemetry
  • Signal to tell the device to update it’s firmware
  • Open a gate or turn on an LED, Fan, or other controlled device

Building applications to be message receivers and listen to a Storage Queue or Service Bus Queue/Topic can feasibly work to implement this, but it will have great difficulty in scaling; especially scaling out to hundreds or millions of devices. Also, Azure Event Hubs is meant for handling an Ingress stream of messages into the Cloud, so it doesn’t lend itself well to Cloud-to-Device messaging.

2-way Messaging Architecture

To provide a little more clarity on what 2-way Messaging for IoT solutions provide, consider the following simple diagram how 2-way messaging could be used to receive telemetry from a Smart Thermostat that is controlled from a Smartphone App:

IoT2WayMessaging

  1. Device-to-Cloud messaging is used to send telemetry readings, such as current temperature and/or humidity, to the Cloud.
  2. The Smartphone App can receives notifications or other updated data to provide the User feedback on what the current conditions are. Then the User can send commands to alter the environmental controls from the Smartphone App.
  3. Cloud-to-Device messaging is used to subsequently send the User’s command from the Cloud down to the Device allowing the environmental conditions to be controlled remotely, from anywhere.

The above sample is simple, but it demonstrates how 2-way Messaging fits together within an IoT solution. The diagram shows only a single device, but a solution that’s scaled out to thousands or millions of devices can truly benefit greatly from implementing 2-way Messaging properly. If the IoT Device were communicating directly with the Smartphone App then this system wouldn’t be able to scale out very much at all.

2-way Messaging between the cloud and devices or application can be extremely useful, and can be difficult to implement if built from scratch. Microsoft has thought this through, and they built out the Azure IoT Hub service that extends what the Azure Event Hub service already offered with added ability to provide 2-way Messaging and Device Management capabilities. This allows systems to more easily control access and message delegation to devices through platform features that allow developers to focus on the applications and devices functionality rather than building out this crucial underlying infrastructure.

What is Azure IoT Hub?

Azure IoT Hub is a fully managed service within Microsoft Azure that enables secure and reliable bidirectional communication (2-way messaging) between millions of IoT devices and the solution back-end. It also provides device management and monitoring, along with support for many of the most popular programming languages and platforms.

Here are some of the features that Azure IoT Hub provides:

  • 1-way (Device-to-Cloud) AND 2-way messaging (Device-to-Cloud + Cloud-to-Device)
  • Device Management and per-device Security with security keys or X.509 certificates
  • Monitoring of device connectivity and device identity management events
  • Declarative message routing to other Azure services
  • Queryable store for device metadata and synchronized state
  • Device libraries for many popular languages and platforms; some of which include: .NET, Java, Node.js, Windows IoT Core, Linux, and more

More Resources

If you’re interested in more information about IoT Messaging Architecture and building IoT Solutions within Microsoft Azure and Azure IoT Hub, then you’ll want to check out the following Opsgility courses:

What IoT devices and solutions are you building, or looking to build? Feel free to post in the comments what cool Internet of Things projects you’re working on. We’d love to hear all about it!

Provision NVidia GPUs and N-Series Virtual Machines in Azure

It’s been over a year since Microsoft initially announced they were bringing NVidia GPUs into Azure and making them available through the new N-series Virtual Machine instance sizes. The Azure N-Series VMs have now hit General Availability, as of December 1, and are no longer in Preview! This new VM tier and sizes offer NVidia Tesla K80 and NVidia Tesla M60 GPUs in the cloud. The N-Series VMs support both Windows and Linux operating systems.

AzureNSeriesGeneralAvailabilityDec-1-2016

The Azure NC virtual machines are powered by the NVidia Tesla K80 GPUs and provide the compute power required to accelerate the most demanding HPC or High-Performing Compute and AI workloads. You can not use these GPUs to accelerate deep learning models and algorithms that run in the cloud. The NC series VMs are optimized for Compute workloads.

The Azure NV virtual machines are powered by NVidia Tesla M60 GPUs and provide NVidia GRID capabilities on demand. You can use these NVidia powered VMs to virtualize workstation-class workloads in the cloud that require cutting edge GPU performance. The NV series VMs are optimized for GPU visualization workloads.

Provision GPUs in the Cloud!

To provision the new N-Series GPUs Virtual Machines, you can simply go to the Azure Portal and provision a new VM choosing an N-Series instance size. If you view the list of Instance Sizes, but don’t see the N-series listed, then you may have chosen an Azure Region that currently doesn’t have them available. The screenshot below shows what was available in the South Central US region at the time of writing this post.

AzureNSeriesVMSizesAzurePortal

The easiest way to see if the N-Series VMs are available for a specific region is to check the Azure Virtual Machine Pricing page. Just choose you desired Region, then expand the +GPU heading, and it’ll show you the pricing information for that region.

Driver Installation

Win2016DeviceManagerNVidiaTeslaM60GPUOnce you provision an N-Series Virtual Machine, you need to install the NVidia drivers for the GPUs. The Operating System wont automatically detect them and install drivers. The “Set up GPU drivers for N-series VMs” page within the Azure documentation contains the necessary links you’ll need to follow in order to download the NVidia drivers onto the VM. Once installed, you’ll be able to take full advantage of the GPUs.