The 7 Steps to your own Azure Stack using the Azure Stack Development Kit (ASDK)
September 15, 2017
By David Summers, Microsoft Azure Lead at Data#3 Limited
Connect
You have heard about Azure Stack either through Bala’s blog post or from other research. For this post, I won’t be going into what Azure Stack is or how you can benefit from your own Stack infrastructure, instead I will take you through the end-to-end deployment of the ASDK (Azure Stack Development Kit) and include important information to maximise your deployment success (based on my own experiences and failures).
I will track the progress of the deployment using the Progressionator – my very own implementation progression indicator!
We will be performing the following:
- Hardware Deployment
- Stack Pre-Requisites
- Deploy Base Operating System
- Software Pre-Requisites
- Install Stack
- Configure Stack
- Finalise Stack
Stage 1: Hardware Deployment
Our friends at Dell kindly supplied us with a PowerEdge R630 Server for us to deploy and test Azure Stack on.
We racked and powered our device within our demonstration area at our Head Office. The device was connected to our dedicated demo network with direct internet connectivity. We also connected the iDRAC remote console to our internal network for deployment.
Note: it is totally possible to deploy the ASDK within a virtualised environment and even within Azure. If you are interested in this as an option then feel free to contact me for details.
Stage 1: Hardware Deployment | Stage 2: Stack Pre-Requisites | Stage 3: Deploy Base Operating System | Stage 4: Software Pre-Requisites | Stage 5: Install Stack | Stage 6: Configure Stack | Stage 7: Finalise Stack
Stage 2: Stack Pre-Requisites
Before we enter the exciting world of on-premises Azure Cloud, we need to fulfil the below requirements.
Hardware
| Component | Minimum | Recommended |
|---|---|---|
| Disk drives: Operating System | 1 OS disk with minimum of 200 GB available for system partition (SSD or HDD) | 1 OS disk with minimum of 200 GB available for system partition (SSD or HDD) |
| Disk drives: General development kit data | 4 disks. Each disk provides a minimum of 140 GB of capacity (SSD or HDD). | 4 disks. Each disk provides a minimum of 250 GB of capacity (SSD or HDD). |
| Compute: CPU | Dual-Socket: 12 Physical Cores (total) | Dual-Socket: 16 Physical Cores (total) |
| Compute: Memory | 96 GB RAM | 128 GB RAM (This is the minimum to support PaaS resource providers.) |
| Compute: BIOS | Hyper-V Enabled (with SLAT support) | Hyper-V Enabled (with SLAT support) |
| Network: NIC | Windows Server 2012 R2 Certification required for NIC; no specialised features required | Windows Server 2012 R2 Certification required for NIC; no specialised features required |
HBA configuration options
- (Preferred) Simple HBA
- RAID HBA – Adapter must be configured in “pass through” mode
- RAID HBA – Disks should be configured as Single-Disk, RAID-0
Supported bus and media type combinations
- SATA HDD
- SAS HDD
- RAID HDD
- RAID SSD (If the media type is unspecified/unknown*)
- SATA SSD + SATA HDD
- SAS SSD + SAS HDD
* RAID controllers without pass-through capability can’t recognise the media type. Such controllers will mark both HDD and SSD as Unspecified. In that case, the SSD will be used as persistent storage instead of caching devices. Therefore, you can deploy the development kit on those SSDs.
Azure
You will need an Azure subscription for Azure Stack to register against. You will also need an Azure AD account that is either a Subscription owner or a Co-Administrator.
Note: it is possible to deploy Azure Stack in a disconnected state that utilises ADFS for authentication to the Portal, however this guide is focused on the online connected scenario.
Networking
Stack requires direct access to Azure for the installation and operation. Proxies are supported for Stack as long as a transparent proxy is used and the Azure endpoints are whitelisted for the Stack Appliance.
Do not connect the development kit machine to the following subnets:
- 192.168.200.0/24
- 192.168.100.0/27
- 192.168.101.0/26
- 192.168.102.0/24
- 192.168.103.0/25
- 192.168.104.0/25
These subnets are reserved for the internal networks within the development kit environment. Also, the DNS server that the Stack Appliance will use cannot be within these subnet ranges.
At this time, IPv6 networks are not supported for Stack. DHCP is supported for the Stack host networking, however it is recommended to select 3 IP Addresses.
- Address 1 – Base OS Installation
- Address 2 – Azure Stack Host
- Address 3 – Azure Stack BGB Nat Gateway
Internet access
Azure Stack requires access to the Internet, either directly or through a transparent proxy. Azure Stack does not support the configuration of a web proxy to enable Internet access. Both the host IP and the new IP assigned to the MAS-BGPNAT01 (by DHCP or static IP) must be able to access Internet. HTTP and HTTPS access must be enabled for the following Domains:
- *.graph.windows.net
- *.login.microsoft.com
- vortex-win.data.microsoft.com.ftonline.com
Stage 1: Hardware Deployment | Stage 2: Stack Pre-Requisites | Stage 3: Deploy Base Operating System | Stage 4: Software Pre-Requisites | Stage 5: Install Stack | Stage 6: Configure Stack | Stage 7: Finalise Stack
Stage 3: Deploy base Operating System (OS)
First, we need to deploy a base OS to the hardware that we will be using for Stack. We can use either Windows Server 2012R2 or Windows Server 2016.
This base OS is disposable, so it does not matter which edition you install. The only critical decision you need to make during this phase is the OS disk location. For my Stack hardware, I had the following Disks:
| Slot | Type | Capacity |
|---|---|---|
| 0 | SSD | 400 GB |
| 1 | ||
| 2 | SSD | 200 GB |
| 3 | SSD | 200 GB |
| 4 | HDD | 1 TB |
| 5 | HDD | 1 TB |
| 6 | HDD | 1 TB |
| 7 | HDD | 1 TB |
| 8 | HDD | 1 TB |
| 9 | HDD | 1 TB |
The Stack OS requires at least 190 GB, the Storage Spaces Direct configuration requires at least 2 SSD’s and an even number of disks in total (Since S2D will be configured in a three way mirror) – So I selected the Disk in slot 0.
Windows Server Setup Instructions
Stage 1: Hardware Deployment | Stage 2: Stack Pre-Requisites | Stage 3: Deploy Base Operating System | Stage 4: Software Pre-Requisites | Stage 5: Install Stack | Stage 6: Configure Stack | Stage 7: Finalise Stack
Stage 4: Software Pre-Requisites
After the OS has been installed, log in and download the Azure Stack installer.
The first thing we need to check is the Disk configuration. From Server manager, review the installed Disks – you will want to see something like below where we have an even number of SSD and HDD drives with no partitioning information. If this is a re-install or re-purposing of hardware, then we should reset each disk.
The next thing to check is the media type. The Azure Stack installer will fail if the media cannot be determined. Possible causes for this are unsupported HBAs Raid configuration. Back at the start, I mentioned that we need pass through support for the disks so that the OS can determine the media Type. If the media type is marked as unspecified or unknown then we will need to resolve that first.
Note: there is an unsupported method to set the media type via Powershell. Below example will set any disk that I have that is greater than 400GB to be a HDD. Once again, this is not supported.
Get-PhysicalDisk | select friendlyname, size, mediatype | where {$_.Size -gt '400088457216'} | Set-PhysicalDisk -MediaType HDD
Now we can run through the pre-requisite check script.
Pre-requisite Check Instructions
Download the script and copy to the Stack Server.
Rename the downloaded script and take off the .txt extension. Launch a new PS or ISE console as an Administrator (this is crucial, as the script will be performing hardware checks).
Below is my sample output, which has an all clear result. This script is checking the hardware (CPU\RAM\BIOS Virtualisation features) storage (Disk capacity\Type) and Internet connectivity.
Key takeaway: if there are any issues reported here then they should be resolved prior to installing Stack. Most deployments will fail due to Internet connectivity issues and in certain circumstances; you will need to start from scratch depending on the phase of the deployment failure. Ensuring that these checks are passed will result in a high success rate for a deployment and save a lot of time in the event of a redeployment.
Once we have a green light, launch the Azure Stack Downloader. Optionally select to download the Server 2016 ISO if you do not have one available (this will be used later to deploy a Server Image to the Azure Stack Gallery).
Select Browse and pick a location that will have the available space for the download and the extracted download (I needed 35GB free space in total).
Hit the download button and grab a coffee or 10.
While the download is in progress, you might be tempted to work on the OS to bring it in line with your other server infrastructure. Don’t.
This OS will be replaced with a bootable VHDX, which has been preconfigured with 2016 server with the applicable Hyper-V roles and services. The OS that you are looking at now will likely never be used again (unless you need to re-deploy Azure Stack).
So, what can we do while we wait for the download?
Well, getting Stack up and running is just one step in a long process, so there are some things that we can do in the meantime. There is a deployment script that we will be calling that is a PowerShell script with a very nice HTA wrapper. Execute the following to download the Script:
# Variables
$Uri = 'https://raw.githubusercontent.com/Azure/AzureStack-Tools/master/Deployment/asdk-installer.ps1'
$LocalPath = 'C:\Installers\AzureStack'
# Download file
Invoke-WebRequest $uri -OutFile ($LocalPath + ‘\’ + ‘asdk-installer.ps1’)
We will need Git later on while setting up some of the gallery images, so might as well download the 64Bit Git client.
Once the download has completed, hit the Run button.
Note: if you accidentally closed the downloader window, then execute C:\Installers\AzureStack\AzureStackDevelopmentKit.exe).
Follow the ASDK setup instructions below.
ASDK Setup Instructions
Always read the EULA.
Pro tip: The Azure Stack installation is 100% based on the CloudBuilder.vhdx bootable OS volume. To preserve the source OS from the download, it is recommended to copy the VHDX file to the root of the C:\ Drive and use the copy to configure Stack. This gives you a rollback option to redeploy Stack in the event of a deployment failure (and save you from having to download the Stack installation again).
Another tip: To free up a bit of space you can remove the following files since they are the compressed versions of the CloudBuilder.VHDX file.
For the Azure connected scenario, only one Network Adapter can be used for the Stack host, to avoid potential issues, I disable any other network adapters with the below command (caution when running this though).
Get-NetAdapter -Physical | where {$_.name -notlike "*NDIS*"} | where {$_.Status -ne "Up"} | Disable-NetAdapter
Depending on your hardware, you may need to install additional drivers for the Stack OS. Pull down any Certified for Windows 2016 Drivers for your hardware. Since I was working with a Dell PowerEdge R630, I downloaded the Windows Driver Pack from the Systems Management section and extracted the ISO to C:\Installs\AzureStack\Dell-Drivers.
Stage 1: Hardware Deployment | Stage 2: Stack Pre-Requisites | Stage 3: Deploy Base Operating System | Stage 4: Software Pre-Requisites | Stage 5: Install Stack | Stage 6: Configure Stack | Stage 7: Finalise Stack
Stage 5: Install Stack
We are now ready to get deployed!
Azure Stack Installer Instructions
From an elevated Powershell prompt, launch C:\Installers\AzureStack\asdk-installer.ps1 (Deployment script that we downloaded earlier).
Select Prepare and then browse for the path of the CloudBuilder.VHDX that we copied to the C:\ drive as well as the optional folder for Drivers (If required).
Enter in your options, DHCP is supported for the network interface, but I like to select a static IP. Since we also have a Stack in our Adelaide office, I chose to denote that the host belongs in our Brisbane office. Although this is purely cosmetic, as the Hostname is not reflected in the Azure Stack portal.
For the networking, select the adapter that has a default gateway.
We are done for now. Select Reboot and for fun, jump onto the server’s local or remote console. As we can see from below, the Boot loader has been configured to boot from the CloudBuilder.VHDX with the original Host as an option.
The server will restart at least once for HAL driver detection and installation. And we are done (for this stage anyway).
Stage 1: Hardware Deployment | Stage 2: Stack Pre-Requisites | Stage 3: Deploy Base Operating System | Stage 4: Software Pre-Requisites | Stage 5: Install Stack | Stage 6: Configure Stack | Stage 7: Finalise Stack
Stage 6: Configure the Stack Deployment
Log back onto the host, but this time use the new LocalAdmin Stack credentials:
UserName = .\Administrator
Password = The one you specified during part one of the setup
The previous OS volume will be mounted as D:\ so open an elevated Powershell console and change directory to D:\Installers\AzureStack and execute asdk-installer.ps1
Now we have a new option for the script, begin installation.
ASDK Installation Instructions
Select Install.
Credential Time. This will be the Azure account that has either Subscription Owner or Co-Administrator within your Azure Tenancy.
At some stage, the Stack OS will restart. When it does, it will be joined to the Azure Stack domain. If you accidentally log back into the host using the local Administration account, you will not see the deployment progress – so log back in using the new Azure Stack domain admin account.
After a period of time (1 hour and 22 minutes) you will see another VM.
Intermission
While we are waiting for the script to execute, let’s take a moment to look at the Stack host itself.
The Stack deployment has taken all of our available disks and created a Storage Spaces Direct Pool, tiering has been enabled for our SSD and HDD disks.
Taking a look at the Failover Cluster Manager, we can see that a Scale Out File server has also been created and the Volume within the Storage Space has been added to the Cluster as a Cluster Shared Volume. Here we can also see the SMB shares that our VM’s and Gallery items will be provisioned into.
Notice the Cluster VM names are all GUID-based? Ever wondered why you need to supply a deployment ID for an Azure VM to Microsoft support? Well this is how support locate your VM among the millions of VMs running in Azure.
Taking a look at the Networking, we can see that several Hyper-V switches have been deployed. Note here that if we named out Primary Network adapter to Deployment, Storage1 or Management then the Stack deployment would have failed!
Now a look into the Hyper-V virtual switch configuration, we have 2 new switches that allow the Stack fabric to communicate using Software-Defined Networking.
Next up, taking a peek into Active directory for the Azure Stack domain. Multiple Managed Service accounts have been created that are used for the service fabric communication between the Stack VMs.
Also, multiple AD groups have been created along with standard user accounts for elevated access.
Note: The script generates a random name for these accounts during provisioning.
Let’s check back in with our deployment, we are now up to step 60!
The magical line of completion. If you get to this stage first time, then you had a well-prepared environment that met all of the pre-requisites.
You can now check out the local Azure Stack portal.
Note: This URL can only be accessed locally on the Azure Stack host at this stage. In the next blog, we will go into setting up a VPN into Stack to allow for outside connectivity.
Stage 1: Hardware Deployment | Stage 2: Stack Pre-Requisites | Stage 3: Deploy Base Operating System | Stage 4: Software Pre-Requisites | Stage 5: Install Stack | Stage 6: Configure Stack | Stage 7: Finalise Stack
Stage 7: Finalise the Stack Deployment
Our final step in this deployment is to enable Marketplace syndication by registering Stack with our Azure subscription.
For this we will need the Git client, so let’s install that first. Normally, I would not include instructions on a simple install like this, but there is one important configuration item to take note of.
Git Client Installation Instructions
From an elevated PS prompt, run the following:
# Set the Azure Powershell repository location
Set-PSRepository -Name "PSGallery" -InstallationPolicy Trusted
# Install the AzureRM.Bootstrapper module. Select Yes when prompted to install NuGet
Install-Module -Name AzureRm.BootStrapper
# Install and import the API Version Profile required by Azure Stack into the current PowerShell session.
Use-AzureRmProfile -Profile 2017-03-09-profile -Force
Install-Module -Name AzureStack -RequiredVersion 1.2.10
Once that has completed, install the Azure Stack utilities using the previously installed Git client.
# Change directory to the root directory
cd \
# clone the repository
git clone https://github.com/Azure/AzureStack-Tools.git –recursive
# Change to the tools directory
cd AzureStack-Tools
# Change directory to the root directory
cd \
# Download the tools archive
invoke-webrequest https://github.com/Azure/AzureStack-Tools/archive/master.zip -OutFile master.zip
# Expand the downloaded files
expand-archive master.zip -DestinationPath . -Force
# Change to the tools directory
cd AzureStack-Tools-master
$AzureCredential = Get-Credential D3-AZ-MAS-01@outlook.com
Login-AzureRmAccount -Environment AzureCloud -Credential $AzureCredential
C:\AzureStack-Tools\Registration\RegisterWithAzure.ps1 -azureSubscriptionId 73515f85-5a1f-4d98-a1ae-64f54c962d88 -azureDirectoryTenantName d3azmas01outlook.onmicrosoft.com -azureAccountId D3-AZ-MAS-01@outlook.com
Once completed, you can now navigate to the Stack Admin portal and then add Market place gallery images to your own Stack.
Stage 1: Hardware Deployment | Stage 2: Stack Pre-Requisites | Stage 3: Deploy Base Operating System | Stage 4: Software Pre-Requisites | Stage 5: Install Stack | Stage 6: Configure Stack | Stage 7: Finalise Stack
The End
That’s all for now. In my next blog, I will walk you through deploying your own iAAS VM template along with the Application Service Environment and SQL resource providers.
If you are interested in running a POC, you can learn more about our offering here.
I will also cover connectivity scenarios to open up Stack to your environment.
