Category Archives: Microsoft Azure

Raspberry Pi GPIO Pin Reference

Microsoft Azure IoT Suite can be used to build extremely scalable Internet of Things (IoT) solutions. With any IoT solution the cloud platform is only half of what needs to be built. The other half resides on physical IoT hardware devices such as a Raspberry Pi that are connected to some combination of sensors and / or actuators to provide the real-world integration side of the IoT solution. Both the Raspberry Pi 2 & 3 offer a 40 pin GPIO header to allow for many different components to be connected, in addition for the capability of providing both 3.3 volt and 5 volt output to those components. Each component needs to be connected to the correct pins, so a proper reference diagram is always necessary to ensure correct pin locations. Continue reading

Can an Azure Resource Manager policy be overridden?

Azure Resource Manager (ARM) policies are a powerful governance capability that allows administrators to accomplish many things, such as:

  • Define a ‘service catalog’ (which services may be created)
  • Control the Azure regions in which resources may be deployed
  • Enforce the use of ARM tags on resources
  • etc.

The use cases for ARM policies is limited only (well, almost only) by your imagination. However I had a question about the possibility of overriding policies. You see, an ARM policy defined at the subscription scope will be ‘inherited’ by all resource groups and resources within the subscription. Could someone with the ‘Owner’ role on a resource group remove the policy association from their resource group? Or could they put in place a policy on their resource group that overrode something defined by the policy at the subscription scope? Let’s try it and find out.

On my subscription I define and associate a policy that limits the resource providers which can be created to compute, network, storage, and resources (for resource groups). My ServiceCatalog.json file looks like this:

ServiceCatalog.json

My ServiceCatalog.json

Now we switch the context to the resource group owner. This user has the ‘owner’ role on a resource group called ‘MyRG’. Here is how the application of the subscription-scoped policy looks from this user’s perspective:

Scope of ServiceCatalog policy

Scope of ServiceCatalog policy

You can see that the policy applies to the scope of the user’s resource group. SO can they un-associate it? It certainly appears so:

Removing the ServiceCatalog assignment

Removing the ServiceCatalog assignment

However, the association is not truly removed…subsequent runs of ‘Get-AzureRmPolicyAssignment with a scope of MyRG shows it is still in place. This answers the question of whether a resource group owner can remove a policy ‘inherited’ from the subscription. What about overwriting a setting put in place by the subscription-scoped policy? Let’s try that too.

While signed on as the resource group owner within PowerShell, I attempt to define and associate another policy called ServiceCatalogUpdate. You can see below that this JSON file adds another resource provider:

ServiceCatalogUpdate.json

ServiceCatalogUpdate.json

Notice what happens when this user tries to define the policy (a prerequisite to applying it):

Results of overwrite attempt

Results of overwrite attempt

The error calls out that the user is trying to ‘write over scope’ of the subscription policy. So even though the user has the ‘Owner’ role on the resource group, they cannot overwrite a policy defined at the subscription level.

One more question seems appropriate. What if the resource group administrator is attempting to define and assign a policy which does not override any settings from a subscription-scoped policy? Here is another policy definition JSON file that requires tags on resources. This policy should not contradict anything in the subscription-scoped policy:

JSON with no contradictory settings

JSON with no contradictory settings

Here are the results when the resource group owner tries to define and apply this policy:

Policy definition results

Policy definition results

The results are the same. So it appears that a resource group owner cannot apply an ARM policy on their resource group if a policy is defined at the subscription level. The takeaway seems to be, when designing solutions using ARM policies, apply your policies at the lowest level possible to achieve your goals.

For more information on Azure Resource Manager policies, visit the Azure documentation page for this powerful feature: https://azure.microsoft.com/en-us/documentation/articles/resource-manager-policy/

Azure Advanced Data Center Bootcamp class now available

In the past 6 months we have taught over 7,500 students worldwide our Microsoft Azure Infrastructure as a Service Training. The widespread adoption of Azure is clearly accelerating into more advanced workloads.

To manage these workloads and to start implementing Azure at scale a new set of skills are needed.

These involve planning for Azure Governance – to ensure that you still have control over your data center, even if it does happen to reside in the cloud. Another key skill is advanced networking. Efficient routing and configuration of virtual firewalls to protect your workloads is critical as more important as sensitive workloads make their way into Azure. Other important skills are hybrid management and monitoring as well as implementing a dependable and understandable plan for business continuity.

With these goals in mind, the experts here at Opsgility have developed the next wave of Azure training to take your infrastructure teams to the next level. We call it the: “Azure Advanced Datacenter Boot camp”.

This class is a full 5 day instructor-led class that focuses on the following topics:

  • Azure Governance in the Enterprise
  • Advanced Azure Networking
  • Implementing Monitoring and Automation with Operations Management Suite
  • Business Continuity with Site Recovery and Backup

The target audience for this class are IT Professionals that have significant experience with Microsoft Azure Infrastructure as a Service. So if you can deploy and configure virtual machines, virtual networks, automate workloads with the command line or templates and need to move to the next level this is the class for you.

The full agenda for the class is online now: https://www.opsgility.com/courses/details/advanced_azure_data_center_bootcamp

This class is now available to schedule for private deliveries. An open enrollment schedule will be announced soon.

If you are interested in learning more about this class you can contact us at info@opsgility.com or email me directly at michael@opsgility.com

 

Rethinking Paradigms in Networking: Firewalls in the Public Cloud

If you have ever implemented a firewall in a traditional network, almost certainly it had at least two network interfaces. One was on an untrusted side, perhaps directly on the Internet, and the other was on a trusted side. The goal, of course, is to keep unwanted traffic from reaching the trusted network. There are more complex implementations, but this serves for illustration sake.

Traditional firewall configuration

Traditional firewall configuration

I bring up these foundational topics to point out one way in which the public cloud makes us rethink our paradigms…in this case, that of the firewall. Continue reading

Set up continuous deployment with unit tests to your Azure App Service

If you want to set up continuous deployment for an App Service Web app in Microsoft Azure, you can either set up a build server (or service) on your own and have it copy the final build artifacts into the folder structure of the Web app -or- you can enable the built in continuous deployment features of the App Service to pull, build, and deploy code from a nice list of available source code locations.

App service publishing settings

App service publishing settings

All you have to do to enable this is to click on the Settings blade of the App Service and click on the Deployment source link. From there, you will see a list of common source code locations. Once you select one of these, provide the correct credentials, and then choose the right repository and branch, Azure will enable a service called Kudu that will immediately copy the source code into the App service, start a build, and then deploy the code straight into the live site (or deployment slot).

So far so good, and also amazingly easy! When I was doing this, I decided to do an experiment and see what would happen when I pushed a commit that included a failing unit test. To my shock and frustration, Kudu took my updated code with the broken test and happily built the app and published it to the live sight without any hesitation. That’s right, it pushed bad code straight through without checking the unit tests. Not good.

I did some research online and came up with practically nothing. I only found one blog post on the matter and it was someone complaining about the same problem but there wasn’t a good alternative solution provided. I did some more research of my own and finally found a workable solution. Yes, you can make Kudu build and run your tests! Read on to see how.

Web app blade Tools icon

Web app blade Tools icon

The first thing you need to do is navigate to the blade for your Web app in the Azure portal and click on the Tools icon in the blade header.

Kudu blade

Kudu blade

 

From there, click the Kudu link and then click the link called Go.

 

 

 

 

This will take you to the Kudu portal page where you can access the command line of the App service server and see details about deployments, app settings, and a lot of other interesting information. What you need from this screen, specifically, is to download the deployment script that Kudu is using when it builds and deploys your application’s source code. To get this, click the Tools link and then click the Download deployment script link that shows up below it. When you click this, it will download a zip file to your computer that contains two files: deploy.cmd and .deployment. Open the deploy.cmd file in a text editor.

Download the deployment script

Download the deployment script

In the deploy.cmd file, find the “Deployment” section that looks like this screen shot.

Deployment section in the script

Deployment section in the script

Solution structure of my app

Solution structure of my app

In between steps 2 and 3, you need to add the instructions to build your test project and to run the tests in it. In the screenshot and the code sample below, my solution was called WebWithTests and I had two projects, a web application project called WebWithTests and a unit tests project called WebWithTests.Tests. All you need to do is configure the build script appropriately for your solution. It should involve building the test project or projects (this is the Step 4 that I added in the code sample below) and then running the test framework by passing it the location of the final .dll files from the build step (Step 5 that I added in the code sample below). Since you can check the result of the test framework executable tool, you can stop the build if any of the tests failed. For your test framework tool, you should be able to use three different test frameworks natively: vstest, nUnit, and xUnit. Make sure the “KuduSync” step still appears at the end of the steps you added just like it was before you modified the file.

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
:: Deployment
:: ----------

echo Handling .NET Web Application deployment.

:: 1. Restore NuGet packages
IF /I "WebWithTests.sln" NEQ "" (
  call :ExecuteCmd nuget restore "%DEPLOYMENT_SOURCE%\WebWithTests.sln"
  IF !ERRORLEVEL! NEQ 0 goto error
)

:: 2. Build to the temporary path
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
  call :ExecuteCmd "%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\WebWithTests\WebWithTests.csproj" /nologo /verbosity:m /t:Build /t:pipelinePreDeployCopyAllFilesToOneFolder /p:_PackageTempDir="%DEPLOYMENT_TEMP%";AutoParameterizationWebConfigConnectionStrings=false;Configuration=Release;UseSharedCompilation=false /p:SolutionDir="%DEPLOYMENT_SOURCE%\.\\" %SCM_BUILD_ARGS%
) ELSE (
  call :ExecuteCmd "%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\WebWithTests\WebWithTests.csproj" /nologo /verbosity:m /t:Build /p:AutoParameterizationWebConfigConnectionStrings=false;Configuration=Release;UseSharedCompilation=false /p:SolutionDir="%DEPLOYMENT_SOURCE%\.\\" %SCM_BUILD_ARGS%
)

IF !ERRORLEVEL! NEQ 0 goto error

:: ADDED THIS BELLOW --------------------------
:: 3. Building test projects 
rem echo Building test projects 
"%MSBUILD_PATH%" "%DEPLOYMENT_SOURCE%\WebWithTests.sln" /p:Configuration=Release;VisualStudioVersion=14.0 /verbosity:m /p:Platform="Any CPU" 

IF !ERRORLEVEL! NEQ 0 (
 	echo Build failed with ErrorLevel !0!
 	goto error
)

:: 4. Running tests 
echo Running tests 
vstest.console.exe "%DEPLOYMENT_SOURCE%\WebWithTests.Tests\bin\Release\WebWithTests.Tests.dll" 
IF !ERRORLEVEL! NEQ 0 goto error 
:: ADDED THIS ABOVE --------------------------

:: 5. KuduSync
IF /I "%IN_PLACE_DEPLOYMENT%" NEQ "1" (
  call :ExecuteCmd "%KUDU_SYNC_CMD%" -v 50 -f "%DEPLOYMENT_TEMP%" -t "%DEPLOYMENT_TARGET%" -n "%NEXT_MANIFEST_PATH%" -p "%PREVIOUS_MANIFEST_PATH%" -i ".git;.hg;.deployment;deploy.cmd"
  IF !ERRORLEVEL! NEQ 0 goto error
)

::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::

Once you have updated this file on your machine with your own solution and project names, you need to add these files (deploy.cmd and .deployment) to your source code so that the Kudu service will use them when it builds and deploys your app in the cloud. The trick here is that you need to add them at the root of the solution, in the same place as the solution file and the .gitignore file.

Deployment scripts in the folder

Deployment scripts in the folder

Then you just need to commit these files to your repository, and do a push. Kudu will see your new commit and start a build and deploy process. You can see in the console output of the Deployment source blade that it is running your tests. Push up a failing test and try it out, the build should fail and it won’t deploy the bad code into the slot.

Azure Marketplace Free Trials and Subscription Spending Limits

If you have tried to deploy an ‘Free Trial’ Azure Marketplace image within an MSDN subscription you probably have seen this error:

2016-06-20_19-11-34

They key text of the deployment error is: “We could not find a credit card on file for your azure subscription. Please make sure your azure subscription has a credit card.

The Azure Marketplace FAQ gives the reason why you get this failure:

Do I need to have a payment instrument (e.g. credit card) on file to deploy Free Tier offerings?
No. A payment instrument is not required to deploy Free Tier offerings. However, for Free Trial offerings a payment instrument is required.

The issue here is 2-fold. Continue reading

Move Resources via ARM Portal

Have you ever deployed a resource or resources in Azure only to realize you’ve deployed the resources into an incorrect resource group or perhaps your misspelled the name of the resource group and want to correct it? I have. Many times. Moving those resources was cumbersome at best, frustrating at worst. It could be done via a number of PowerShell cmdlets and the full resource ID. This was helpful, but not extremely user friendly.

When I was doing some onsite training a few weeks ago, one of the students stumbled upon a way to move those resources via the Azure Portal. What?! I didn’t know this had been a feature. Wow! I asked him to show me how he did this and after he showed me, we shared it with the rest of the students. I then investigated a little further and found out it can be done multiple ways.

To prepare for example resource group moves, I utilized the GitHub Quick Start Templates located here: https://github.com/Azure/azure-quickstart-templates to deploy a Simple Windows and a Simple Linux VM environment into two separate resource groups.

The first resource group is entitled TemplateRG1 and contains an Ubuntu Linux VM and the associated resources for this virtual machine. NOTE: I did have to change the name of the NIC, PublicIP, and VNET due to potential naming conflicts for my moving exercise.
image

The second resource group is entitled TempRG and contains a Windows VM and the associated resources for this virtual machine.

image

Let’s say I wanted to move the resources from TempRG to a new resource group called ProdRG. The following are the ways I’ve found how to move resources (and even associated resources) from one resource group to another, and even creating the new resource group in the process if needed.

Option one:

  1. In the Azure Portal, click on the resource you would like to move. In my case, SimpleWindowsVM is the name of the virtual machine I’d like to relocate.
  2. When the information blade appears, notice the “pencil” icon next to the Computer name.
    image
  3. Click on this “pencil” icon.
  4. You will now be prompted with a Move resources blade.
  5. The VM is listed as the resource that will be moved and you have the optional choice to move the related resources along with the VM.
  6. In my case, I check the checkbox beside all resources as I’m going to move them all to a new resource group.
    image
  7. Then I click on the Create a new group option to create my ProdRG resource group and type in the name of the new resource group. Otherwise, I could choose an existing resource group from the drop down menu if desired.
  8. After checking the checkbox that this action will result in new resource IDs so scripts and tools may need to be modified, I click the OK button to continue the process.
    image
  9. After the blade validates the move, the portal takes care of the rest. Watch the notifications that indicate the resources are being moved, and then finally, they have been moved successfully.
    image
    image
  10. Voilà! You have successfully moved a virtual machine and all the associated resources to a brand new resource group! NOTE: You might want to clean up your empty resource group now just for housecleaning purposes.

Now let’s say that I’d like to move all the resources from the TemplateRG1 to my newly created ProdRG group because they should be located in this resource group along with my Windows VM and its resources.

Option two:

  1. Again, in the Azure Portal, click on the resource you would like to move. In my case, MyUbuntuVM is the name of the virtual machine I’d like to relocate this time.
  2. In the Settings blade under the GENERAL heading, select Properties this time.
    image
  3. In the resulting properties information, find the RESOURCE ID property. Notice right below it there is an option for Change resource group.
    image
  4. Once you click this, you will be presented with the Move resources blade as in Option one above.
  5. The VM is listed as the resource that will be moved and you have the optional choice to move the related resources along with the VM.
  6. In my case, I check the checkbox beside all resources as I’m going to move them all to an existing resource group.
    image
  7. Choose ProdRG from the drop down list of existing resource groups, check the checkbox for the resource id information and click the OK button to begin the process.
    image
  8. After the blade validates the move, the portal takes care of the rest. Watch the notifications that indicate the resources are being moved, and then finally, the notification that they have been moved successfully.
    image
    image
  9. Presto! You have successfully moved a virtual machine and all the associated resources to an existing resource group! NOTE: You might want to clean up your empty resource group now just for housecleaning purposes.

As you can see below, I have consolidated two resource groups into one with the two options presented.
image

There you have it. Multiple ways to relocate resources to a new or existing resource group. Be sure and be aware of resources that may not fully support moving (start here) and be aware of some potential impact during the move process. I have tested it on running VMs and they have not been required to restart or lose connectivity, but test fully before performing this on production workloads for safety!

@EdwardFBear

Trying out Backup and Site Recovery (OMS): Lessons Learned

Summary of the items I learned:

  • Reminder: Azure changes often
  • Register Resource Providers
  • Chocolatey is a cool tool
  • ARMClient is a cool tool, too

For details keep reading below!

I wanted to take a little time and share some of the things I’ve learned as I have started to utilize the new items in Azure Resource Manager. Specifically, surrounding the Backup and Site Recovery (OMS) feature that is now GA. The first thing I have learned is that I need to be flexible when looking for an item in ARM because of Rule #1: Things change rapidly in Azure.

Originally, I had been evaluating this item in ARM as Recovery Services but it recently was renamed to Backup and Site Recovery (OMS). Okay, that’s maybe not a huge revelation to many of you, but a gentle reminder to all of us: Don’t get too used to things in Azure being static. It is a dynamic environment to be sure!

The first thing I ran into, was that in my MSDN Subscription, the Location was actually blank. What? Why can’t I deploy a Recovery Services Vault in ARM with my MSDN Subscription? I had another Subscription that I used for some of my training courses and using that one, I saw Location populated with all Regions that I could create a Recovery Services Vault in.

After some investigation, attempting other subscriptions and reaching out to peers (Thank you, @mscloud_stever!), it was discovered that the Resource Provider for Recovery Services was not registered. Even though I had Site Recovery registered, it was not enough to display the Location regions for my Recovery Services Vault. So, learn from my discovery: Make sure to register your Recovery Services Provider Namespace.

To check your Registered Providers, run the following command in PowerShell:

Get-AzureRmResourceProvider -ListAvailable | Where-Object -FilterScript { $PSItem.RegistrationState -eq 'Registered'; }

Here is the sample output of this command:

image_thumb.png

If you look closely, Microsoft.RecoveryServices is not in the list. If this is the case for you, run the following command to register this Provider for your subscription:

Register-AzureRmResourceProvider -ProviderNamespace Microsoft.RecoveryServices -Verbose

Once this completes, run the command to list out the registered providers again, and you should now see it registered like the screen shot below:

2016-05-12_10-56-38

Now that you see this, you may need to refresh your portal or log-out and log-in again for this to now show you the region options in the Location. NOTE: This took about an hour for mine to show correctly, so you might need to wait a little time for it to recognize the registration, but most of the time it is an immediate process.

After doing this, now I could see the regions and deploy a Recovery Services Vault without issue from within the ARM Portal. Now on to some other stuff I learned.

After registering a Windows 10 client for Files and Folders Backup, registering a Azure VM to backup, registering a System Center VMM server for protection and a VMware vCenter Server for protection, I performed backups and test failovers, etc. All of this is great stuff in the ARM portal and being able to see the backups, replicate VMs, test failovers individually, and more, I started the process of removal and deletion to clean up my environment.

All was great until in the process of deleting the Resource Group, the Site Recovery Vault would not delete. Nothing I attempted to do allowed me to remove it. It kept telling me that there were registered servers in the vault that I needed to remove before attempting the deletion of the vault.

In the Azure portal, I could find no servers listed in any area of the vault. I tried many things, but nothing I did would show that a server was still registered in the vault to delete it. I opened a support incident in Azure and was contacted my Microsoft. They gave me the following option to run and ARMClient command to clean the vault. You might be asking yourself, like I did as well, “How is this done?” So, read on!

First, if you have not done so, install Chocolatey on your Windows machine. If you’re not familiar with Chocolatey, here is the location to get instructions on installation: https://chocolatey.org/ – From the website: “Chocolatey NuGet is a Machine Package Manager, somewhat like apt-get, but built with Windows in mind.

Apt-get for Windows? Cool! So, I browsed to to the homepage and found the PowerShell command to download and install via the easy install of the main page. Bingo! Once you install Chocolatey, check out the packages you can install! https://chocolatey.org/packages – You mean I can install sysinternals tools with just a command line!? C:\> choco install sysinternals – Okay, I’m distracted easily. Back to the topic at hand.

Okay, now I’ve installed Chocolatey, which I hope you’ll agree is a cool tool, now I need to install ARMClient. ARMClient is a simple command-line tool to invoke the Azure Resource Manager API. At the Chocolatey package page, type ARMClient in the Search Packages prompt and click the magnifying glass to perform the search. See screenshot below:

image.png

Once you find the results, click on the ARMClient 1.1.1 to read about the package and the commands to both install and upgrade the package. NOTE: The version number may change should there be any updates after this article was published.

Once I ran choco install armclient and ARMClient was installed, I typed ARMClient at the command prompt showing me the initial help screen and some sample commands. See the screenshot below:

image_thumb.png

After I ran ARMClient login, I entered my credentials for my Azure subscription and successfully authenticated. ARMClient enumerated my tenants and my subscriptions associated with those tenants in the resulting output. Now comes some of the fun part. I had been provided a command to try to remove registered servers in the vault. Some of the information I had to gather and enter specific to my situation, but here is the command that was useful for me:

ARMClient.exe delete subscriptions/<subscriptionId>/resourceGroups/<resourceGroupName>/providers/Microsoft.RecoveryServices/Vaults/<vaultName>/backupContainers/<serverFQDNname>/UnRegisterContainer?api-version=2015-03-15 -verbose

The highlighted areas are the edits I needed to make putting the information in that corresponded to my specific subscription, resource group, vault, and server FQDN name. After modifying the command with my info, I was able to run this command and after doing so was able to delete the vault without issue. See example output (with some information redacted for privacy) in the screenshot below:

image.png

So, all-in-all, this was a great learning exercise for me. Not only did a learn a lot about Recovery Services, Backup, ARM and Resource Providers, I learned a few new cool tools as well: Chocolatey and ARMClient. I hope you find this information useful and learn along with me!

@EdwardFBear

Automate finding a unique Azure storage account name (ARM)

When building ARM-based virtual machines in Azure with PowerShell, one important step involves creating a storage account. And because the storage account name must be unique within Azure, you first need to find an unused name. To accomplish this we can use the Get-AzureRmStorageAccountNameAvailability cmdlet.

This is great, but I may have to manually try several names before I stumble upon one that is unique. The last time I was working through this process I thought it would be helpful to let PowerShell find a unique name for me, retrying as needed until a usable name was determined. I looked around and to my surprise I could not find something that was already written to accomplish this. Continue reading