Cloud Scale, means Out, not Up
January 2, 2017
Traditional application architectures have relied on scaling to handle increases in load by adding more power to the server. This has meant increasing the amount of RAM, or moving to a faster, more powerful CPU. This is called scaling up. The shift to the cloud has changed the way application architectures are designed, and it has meant an increase in distributed systems. Distributed systems are built to scale out rather than up; this means adding additional servers to distribute the load across a multitude of nodes in the system. There are many benefits to scaling out, and this article walks through a few of them to explain the benefits to scaling out, rather than up.
Pitfalls of Scaling Up
There are many times when scaling UP is a valid option, and it still does have a valid place in cloud based systems. However, there are some drawbacks to relying solely on scaling up and the only option of scaling server resources. These drawbacks are the same in the cloud as they are on-premises, and have been on-premises forever.
Here are some drawbacks to solely relying on scaling up:
- Servers can only be made so big. Eventually you wont be able to add more resources necessary to scale properly.
- Scaling up may be able to add the necessary resources, but a single server is still a single point of failure that can bring and entire system down.
- Relying on moving to larger and larger servers can get very expensive, since it can mean (when running on-premises) abandoning old hardware and completely replacing it.
Benefits of Scaling Out
While scaling up has a place in the cloud, it’s generally better to architect solutions to support scaling out instead. Scaling out addes more server instances to a system and can achieve higher levels of redundancy and scalability.
Here are some benefits to scaling out:
- There’s no limit to the number of server you can add.
- Scaling out adds redundancy, and load balancing offers seamless failover for reaching high availability.
- Stateless systems are designed to scale out, and can benefit greatly from spreading the load across servers.
- Stateful systems may need extra services for state caching / sharing, and may require more work to scale out. But, once scaled out the first time, they generally are as easy to scale out as a Stateless system is.
Up and Out
Most enterprise systems will rely on 2 server to start. This give the system basic redundancy and failover ability to achieve a basic level of high availability. Generally when these systems go live into production for the first time they are using smaller server instance sizes. Then as the app grows in usage there is a need to add more server resources.
To add more server resources, there are 2 options: Up and Out. There are perfectly valid times when scaling either Up or Out will work for a given solution. Depending on how the enterprise system is architected, it may be valid to scale up or out with the same result in many cases. This is especially true if the system already has redundancy of 2 or more servers.
However, there are times when scaling Up or Out is a decision that needs to be considered carefully. Here are some scenarios to think about when deciding to scale:
- The hosting / operation cost of the system needs to be considered when scaling up or out. Sometimes it may be cheaper to one over the other giving your organization some cost savings on the monthly bill.
- Choosing to scale out can offer expanded redundancy and a higher level of availability with lower system downtime or degraded performance in case of a server or data center outage, or even a spike in load.
There are many other points to keep in mind when deciding how to scale. These are just a couple examples.
Microsoft Azure – Security Center Partner Integration
December 16, 2016
Being very ‘partner-centric’ has always been core to Microsoft’s success. This has never been more true than in the cloud era. Microsoft’s biggest bet and largest platform is, of course, Azure, and it will certainly take partners of all kinds to help Microsoft win in this market.
A great example of this close relationship with partners is the way Microsoft releases their own services in Azure that directly integrate with partner solutions. Azure Security Center, is a case in point. Security Center solves a huge problem for customers taking their first leap into Azure. Once a customer deploys a workload in the public cloud, how can they be sure they followed security best practices? This can be hard to answer if you’re new to the platform. Security Center helps to bridge this gap through prevention and detection. Security Center helps prevent issues by evaluating a customer’s deployments and offering recommendations on ways to improve the security configuration. Beyond these recommendations, Security Center also leverages machine learning and an expansive set of telemetry from Microsoft’s global security datasets, to detect probable and certain security incidents as they happen.
This is great, but the aspect of Security Center I want to highlight is the integration with partner offerings. To show you how this works, I’ll quickly deploy the solution.
Deploying a WAF Solution from Security Center
Here is a view of Security Center’s Prevention tiles.
Notice the Application section on the Resource security health tile (outlined in purple). This is showing me that there is a high-severity application health finding. If I click on that, I can see what the finding is.
If I then click on the Web Application at the bottom, I see a recommendation.
Click on the recommendation and you can add a web application firewall. Notice this is still within Security Center, in the context of the specific finding. Clicking on Create New shows me the partner web application firewalls that can be deployed from within Security Center. For illustration sake, I’m choosing the F5 solution.
On the Azure Marketplace page for the F5 web application firewall, notice that this solution will be deployed and configured automatically, so that your web app will be protected against the most common threats and vulnerabilities.
When I click Create I fill out a few bits of information and my web application firewall starts deploying. It will take about 20 minutes (or so) to complete.
Once the solution is deployed, there is one final step to configure the web application firewall. Back on the Resource Security Health tile, click on Applications.
Within Applications notice that the Application Recommendations lists Pending WAF finalization. Click this recommendation.
On the Pending WAF finalization menu click your web application. On the subsequent dialog click Finalize web application firewall setup.
The next step involves the manual step of pointing your web site’s DNS name to the public IP address of the web application firewall. Once you have done this, check the box beside I updated my DNS record and click Restrict traffic.
At this point, rules are written in a Network Security Group that effectively force all incoming traffic through the firewall. Now your web application is protected!
Note the ‘green’ status of the application now
The integration goes deeper than just deploying the partner solution from Security Center. After the deployment, security threats that are detected by the F5 solution are fed back into Security Center, so I can see and investigate the findings within the Security Alerts tile.
When you click on an alert you can see that many were detected by the F5 web application firewall, and fed into Security Center.
Clicking on an alert gives more information, and gives the correct path so you can view the finding within the F5 interface.
I hope this quick overview of the F5 Web Application Firewall, as deployed from Azure Security Center, illustrates the deep level with which Microsoft is working with their partners. This level of integration, both from the ease of deployment, and the sharing of security incident information, is a powerful example of strong partnerships bringing amazing value to customers. For more information check out the links below:
F5 BIG-IP Virtual Editions: https://www.f5.com/pdf/products/big-ip-virtual-editions-datasheet.pdf
Request F5 Trial Licenses: https://interact.f5.com/azuref5trial.html
Azure Security Center: https://azure.microsoft.com/en-us/services/security-center
Azure IoT Architecture: 2-way Messaging
December 14, 2016
Inter-Application messaging is important. For this reason there are a couple different Messaging options within Microsoft Azure; Storage Queues, Azure Service Bus Queues/Topics. However, these are message queues meant to facilitate the sending and receiving of inter-application messages between disparate application components. When it comes to the Internet of Things, systems need even more robust messaging that these Message Queue offerings provide. For this reason, Microsoft created the Azure Event Hubs to provide a highly scalable method of receiving a very large stream of message ingress. Event Hubs handles the scale with being able to scale up to log millions of events per second. Although, Event Hubs are just for ingress. What about egress? What about sending messages back to application components or Internet of Things (IoT) devices? This article goes through an explanation on how the Azure IoT Hub service builds on top of Event Hubs to provide a secure 2-way messaging platform for IoT integration.
Device-to-Cloud Messaging is sort of the traditional method of messaging using Queues. A device or application sends messages to the Queue that are picked up by a receiver of some kind on the other end. These messages can be information for new orders coming into the system, or from an IoT device they could be sensor reading telemetry, or status updates to report the health of the device. There are many different kinds of messages that can be sent from IoT devices; the main key of Device-to-Cloud is that this is the ingress stream of messages received by the Message Queue.
For this ingress there are a couple different options of services to use in Azure. For lower scale you could use Storage Queues or Service Bus Queues/Topics. However, for a much larger ingress stream of messages, you’ll want to use Event Hubs or IoT Hubs. Event Hubs will handle the scale, but IoT Hubs extends scalability with additional Device Management features that are extremely useful with handling messages from dozens, hundreds, even millions of devices.
Cloud-to-Device messaging is the method that used to send messages to individual devices or applications from the system running in the Azure cloud. The types of messages send to a specific application instance or IoT Device can be in multiple forms. These messages are used to send command and control, or other messages to individual or groups of devices. Here’s a list of some of the examples of Cloud-to-Device messages that could be sent:
- Configuration setting updates
- Signals to tell the device to start or stop reporting sensor telemetry
- Signal to tell the device to update it’s firmware
- Open a gate or turn on an LED, Fan, or other controlled device
Building applications to be message receivers and listen to a Storage Queue or Service Bus Queue/Topic can feasibly work to implement this, but it will have great difficulty in scaling; especially scaling out to hundreds or millions of devices. Also, Azure Event Hubs is meant for handling an Ingress stream of messages into the Cloud, so it doesn’t lend itself well to Cloud-to-Device messaging.
2-way Messaging Architecture
To provide a little more clarity on what 2-way Messaging for IoT solutions provide, consider the following simple diagram how 2-way messaging could be used to receive telemetry from a Smart Thermostat that is controlled from a Smartphone App:
- Device-to-Cloud messaging is used to send telemetry readings, such as current temperature and/or humidity, to the Cloud.
- The Smartphone App can receives notifications or other updated data to provide the User feedback on what the current conditions are. Then the User can send commands to alter the environmental controls from the Smartphone App.
- Cloud-to-Device messaging is used to subsequently send the User’s command from the Cloud down to the Device allowing the environmental conditions to be controlled remotely, from anywhere.
The above sample is simple, but it demonstrates how 2-way Messaging fits together within an IoT solution. The diagram shows only a single device, but a solution that’s scaled out to thousands or millions of devices can truly benefit greatly from implementing 2-way Messaging properly. If the IoT Device were communicating directly with the Smartphone App then this system wouldn’t be able to scale out very much at all.
2-way Messaging between the cloud and devices or application can be extremely useful, and can be difficult to implement if built from scratch. Microsoft has thought this through, and they built out the Azure IoT Hub service that extends what the Azure Event Hub service already offered with added ability to provide 2-way Messaging and Device Management capabilities. This allows systems to more easily control access and message delegation to devices through platform features that allow developers to focus on the applications and devices functionality rather than building out this crucial underlying infrastructure.
What is Azure IoT Hub?
Azure IoT Hub is a fully managed service within Microsoft Azure that enables secure and reliable bidirectional communication (2-way messaging) between millions of IoT devices and the solution back-end. It also provides device management and monitoring, along with support for many of the most popular programming languages and platforms.
Here are some of the features that Azure IoT Hub provides:
- 1-way (Device-to-Cloud) AND 2-way messaging (Device-to-Cloud + Cloud-to-Device)
- Device Management and per-device Security with security keys or X.509 certificates
- Monitoring of device connectivity and device identity management events
- Declarative message routing to other Azure services
- Queryable store for device metadata and synchronized state
- Device libraries for many popular languages and platforms; some of which include: .NET, Java, Node.js, Windows IoT Core, Linux, and more
If you’re interested in more information about IoT Messaging Architecture and building IoT Solutions within Microsoft Azure and Azure IoT Hub, then you’ll want to check out the following Opsgility courses:
What IoT devices and solutions are you building, or looking to build? Feel free to post in the comments what cool Internet of Things projects you’re working on. We’d love to hear all about it!
Provision NVidia GPUs and N-Series Virtual Machines in Azure
December 8, 2016
It’s been over a year since Microsoft initially announced they were bringing NVidia GPUs into Azure and making them available through the new N-series Virtual Machine instance sizes. The Azure N-Series VMs have now hit General Availability, as of December 1, and are no longer in Preview! This new VM tier and sizes offer NVidia Tesla K80 and NVidia Tesla M60 GPUs in the cloud. The N-Series VMs support both Windows and Linux operating systems.
The Azure NC virtual machines are powered by the NVidia Tesla K80 GPUs and provide the compute power required to accelerate the most demanding HPC or High-Performing Compute and AI workloads. You can not use these GPUs to accelerate deep learning models and algorithms that run in the cloud. The NC series VMs are optimized for Compute workloads.
The Azure NV virtual machines are powered by NVidia Tesla M60 GPUs and provide NVidia GRID capabilities on demand. You can use these NVidia powered VMs to virtualize workstation-class workloads in the cloud that require cutting edge GPU performance. The NV series VMs are optimized for GPU visualization workloads.
Provision GPUs in the Cloud!
To provision the new N-Series GPUs Virtual Machines, you can simply go to the Azure Portal and provision a new VM choosing an N-Series instance size. If you view the list of Instance Sizes, but don’t see the N-series listed, then you may have chosen an Azure Region that currently doesn’t have them available. The screenshot below shows what was available in the South Central US region at the time of writing this post.
The easiest way to see if the N-Series VMs are available for a specific region is to check the Azure Virtual Machine Pricing page. Just choose you desired Region, then expand the +GPU heading, and it’ll show you the pricing information for that region.
Once you provision an N-Series Virtual Machine, you need to install the NVidia drivers for the GPUs. The Operating System wont automatically detect them and install drivers. The “Set up GPU drivers for N-series VMs” page within the Azure documentation contains the necessary links you’ll need to follow in order to download the NVidia drivers onto the VM. Once installed, you’ll be able to take full advantage of the GPUs.
Azure Subscription Spending Limit Reached! Now What?
December 8, 2016
When using an Azure Free Trial or an MSDN Subscription you get a certain amount of Free credit for Azure Services. This credit is really helpful for Dev/Test purposes, or just to use for learning and studying for the Azure certification exams. However, there is a spending limit of the Free credits ($200 for Free Trial / $150/mo for MSDN Subscriptions) and once you reach that limit your Azure Account will be suspended. There are a few reasons you may use up all your credits, on purpose or by accident, and it would be helpful to be able to keep using Azure Services. This article answers the question of, “What do I do when my Azure Subscription spending limit is reached?” as it pertains to a Free Trial or MSDN Subscription.
Spending Limit Reached!
Your Azure Subscription will be disabled when you’ve used up all your credits and reached the maximum spending limit. When this happens, all the resources in your subscription remain configured, but they are all stopped from running. Essentially they’re still there, but they don’t work.
When this happens there are 2 notices you will see to let you know you’ve exhausted your spending limit:
- A notification email will be sent to you.
- A notification prompt will display when you log into the Azure Portal.
The image to the right is what the notification prompt in the Azure Portal looks like, and below is an example of the email notification you will receive.
How did this happen?
Whether you purposely provisioned extra resources that used up all your credit, or you accidentally left something running, you really need to know exactly what caused the spending limit to be reached. There are many reasons this could happen. Here’s a short list of some of the possibilities that could cause you to use up all your spending limit earlier than the end of the month:
- You purposely provisioned a few resources that you needed, and you were aware this may happen.
- You forgot to Deallocate or Delete some resources when you were done with them and left them running overnight, over the weekend, or longer.
- You accidentally chose a higher, more pricey Instance Size or Pricing Tier for a Virtual Machine or other resource
- and, so on…
What ever the reason, it’s useful to drill into your usage and find out what service or services caused you to spend so much unexpectedly. Fortunately, the Billing area in the Azure Portal let’s you drill into a “Breakdown of current changes” on your Azure Subscription for the current billing period. It also displays a “Burn rate” graph and “Costs by service” breakdown to give you full visibility at any time; not just when you’ve reached your limit.
The above screenshot shows that the spending limit was reached within an Azure Subscription associated with an MSDN Subcription; which gets $150/mo Azure credit. In this case the spending limit was reached and the subscription was disabled. There was some small charges per day for a little while, and then one day the spending just spiked.
Looking at the bottom portion of the screenshot you can see a breakdown of spending by Azure service. This shows some smaller spending by a few different services, like Compute, App service, etc. However, one item really stands out at the top! The “Premium API Management – API Management” service spiked the spending within the Azure Subscription, and as a result is the culprit for the spending limit being reached.
In this example, I actually forget to reconfigure an Azure API Management service back to use the Developer pricing tier and left it running for an entire day on the Premium Tier with multiple Units configured. While this is frustrating, it’s also very nice, since Azure is helping us from “breaking the bank” with overrun spending that we may not even know about.
Reactivate and Lift the Spending Limit
In order to get your services running again and be able to use the Azure Subscription after it’s been suspended / disabled, you need to go into the Azure Account Portal and remove the spending limit.
When logged into the Azure Account Portal, it will display your Azure Subscriptions with notification messages by them in the list that will tell you that, “The subscription reached a spending limit and has been disabled to prevent charges.” It will also display the status of the Subscription as “Disabled.”
Clicking on the Azure Subscription in the list will bring you to a page that will list out your current usage within the billing period as well as display how much spending limit you have left. In this screenshot it shows $0.00 in red since the spending limit has been reached.
To get things running again, just click on the Remove spending limit button. This will open a prompt that will let you choose to remote the spending limit, and choose the payment method to purchase additional services.
While it’s important to know how much you’re spending in Azure, it can sounds scary to remove the spending limit. But, Microsoft does ease that fear a little by offering 2 options for how you want to remove the spending limit:
- Remove spending limit for the current billing period
- Remove spending limit indefinitely
When choosing the “Remove spending limit for the current billing period” option, it will lift the spending limit for the remainder of the current billing period only. At the end of the current billing period, it will re-enable the spending limit so it’s in place again for the next billing period.
Remember that when you remove the spending limit and reactivate your Azure Subscription, you’ll certainly want to Deallocate, Delete, or Reconfigure your Azure Services to decrease spending. If you forget to do this once the spending limit is reached, you will no longer get a notification and your spending could run away from you. Don’t forget to do this, or you could get a really big surprise on your credit card statement at the end of the month, and your boss may not be too happy with you if this happens.