Windows Server 2012 and PowerShell 3.0 offer thousands of new cmdlets to make any scripter happy. One set of cmdlets had me intrigued when I first heard about them. The cmdlets in the storage module interested me because frequently I would have to deploy a bunch of new Exchange servers and depending on the design sometimes I would need to prepare a lot of disks and create mountpoints. Depending on the numbers of servers and disks involved I would either do everything manually or use a diskpart script. I always for automating things like this as much as possible so when I heard about the new storage cmdlets in 2012 I was pleasantly surprised.
For anyone who has used diskpart scripts you’ll know they aren’t always easy to implement and require some testing and tweaking to get them fully automated. Good luck trying to put a lot of logic in a diskpart script too. The only switch I found useful was the NOERR one, which basically told the script to continue and “ignore the man behind the curtain” when an error was encountered. Here’s a snippet of one such script. In it I’m creating two disks, one for a database and one for a log, then linking them to a directory as a mountpoint.
select Disk 1
online disk NOERR
select Disk 1
convert GPT NOERR
attributes disk clear readonly NOERR
create partition primary NOERR
format FS=NTFS LABEL=DB1 UNIT=64K QUICK NOERR
assign mount=E:\Mountpoints\Databases\DB1
select Disk 2
online disk NOERR
select Disk 2
convert GPT NOERR
attributes disk clear readonly NOERR
create partition primary NOERR
format FS=NTFS LABEL=Logs1 UNIT=64K QUICK NOERR
assign mount=E:\Mountpoints\Logs\Logs1
I know. It looks a little messy, but it actually works and saved me a ton of time. What you don’t see is how I actually created this script. I used a combination of PowerShell scripting to get the disk information by using output from diskpart, then with some CSV file manipulation in Excel I mapped my disks along with their respective database and log names and paths. Then I used PowerShell once again to import my edited CSV file and spit out this diskpart script shown above. The final part was obviously running diskpart in script mode (i.e. “diskpart /s drives.txt”). I even took my PowerShell script a step further and as I had my script create my diskpart commands to layout the drives as I wanted them, I also had it create an “undo” script which reverted all the disks back to their initial states. This proved to be very helpful in my lab for testing as well as giving me an out when dealing with 30+ LUNs x 16 servers in production.
The downside is that these cmdlets are only available with Windows Server 2012 and Windows 8 so if you’re trying to automate disk management for Windows Server 2008 or other OS, you’ll need to stick with diskpart for now.
So how do we translate diskpart scripts to Windows Server 2012 and PowerShell 3.0?
First, let’s take a look at the new storage commands available in 2012. There’s more storage cmdlets than these but we’ll be focusing on this group of commands for this exercise.
Get-Command -Module storage
CommandType Name ModuleName
----------- ---- ----------
Alias Initialize-Volume storage
Function Add-InitiatorIdToMaskingSet storage
Function Add-PartitionAccessPath storage
Function Add-PhysicalDisk storage
Function Add-TargetPortToMaskingSet storage
Function Add-VirtualDiskToMaskingSet storage
Function Clear-Disk storage
Function Connect-VirtualDisk storage
Function Disable-PhysicalDiskIndication storage
Function Disconnect-VirtualDisk storage
Function Dismount-DiskImage storage
Function Enable-PhysicalDiskIndication storage
Function Format-Volume storage
Function Get-Disk storage
Function Get-DiskImage storage
Function Get-FileIntegrity storage
Function Get-InitiatorId storage
Function Get-InitiatorPort storage
Function Get-MaskingSet storage
Function Get-OffloadDataTransferSetting storage
Function Get-Partition storage
Function Get-PartitionSupportedSize storage
Function Get-PhysicalDisk storage
Function Get-ResiliencySetting storage
Function Get-StorageJob storage
Function Get-StoragePool storage
Function Get-StorageProvider storage
Function Get-StorageReliabilityCounter storage
Function Get-StorageSetting storage
Function Get-StorageSubSystem storage
Function Get-SupportedClusterSizes storage
Function Get-SupportedFileSystems storage
Function Get-TargetPort storage
Function Get-TargetPortal storage
Function Get-VirtualDisk storage
Function Get-VirtualDiskSupportedSize storage
Function Get-Volume storage
Function Get-VolumeCorruptionCount storage
Function Get-VolumeScrubPolicy storage
Function Hide-VirtualDisk storage
Function Initialize-Disk storage
Function Mount-DiskImage storage
Function New-MaskingSet storage
Function New-Partition storage
Function New-StoragePool storage
Function New-StorageSubsystemVirtualDisk storage
Function New-VirtualDisk storage
Function New-VirtualDiskClone storage
Function New-VirtualDiskSnapshot storage
Function Optimize-Volume storage
Function Remove-InitiatorId storage
Function Remove-InitiatorIdFromMaskingSet storage
Function Remove-MaskingSet storage
Function Remove-Partition storage
Function Remove-PartitionAccessPath storage
Function Remove-PhysicalDisk storage
Function Remove-StoragePool storage
Function Remove-TargetPortFromMaskingSet storage
Function Remove-VirtualDisk storage
Function Remove-VirtualDiskFromMaskingSet storage
Function Rename-MaskingSet storage
Function Repair-FileIntegrity storage
Function Repair-VirtualDisk storage
Function Repair-Volume storage
Function Reset-PhysicalDisk storage
Function Reset-StorageReliabilityCounter storage
Function Resize-Partition storage
Function Resize-VirtualDisk storage
Function Set-Disk storage
Function Set-FileIntegrity storage
Function Set-InitiatorPort storage
Function Set-Partition storage
Function Set-PhysicalDisk storage
Function Set-ResiliencySetting storage
Function Set-StoragePool storage
Function Set-StorageSetting storage
Function Set-StorageSubSystem storage
Function Set-VirtualDisk storage
Function Set-Volume storage
Function Set-VolumeScrubPolicy storage
Function Show-VirtualDisk storage
Function Update-Disk storage
Function Update-HostStorageCache storage
Function Update-StorageProviderCache storage
(Get-Command -Module storage).count
84
Sweeeet! (a la Eric Cartman)
So the scenario I’m going to is one where I have the following set of Exchange servers with these disk requirements:
-
C: – 40GB boot volume
-
D: – CD-ROM
-
E: – 30GB pagefile drive
-
F: – 60GB application drive (Exchange, mountpoint root, other apps)
-
Three databases with associated log volumes:
-
3 x 20GB disks – our log LUNs
-
3 x 100GB disks – our database LUNs
-
1 x 150GB disk – our restore LUN
-
-
F:\ExchangeData\ – root directory for our mountpoints. Each database and associated log mountpoints will be linked to a subdirectory based on the database name (i.e. F:\ExchangeData\DB1\Database and F:\ExchangeData\DB1\Logs)
Let’s take a look at the disks on my lab machine.
Get-Disk | ft -AutoSize
Number Friendly Name OperationalStatus Total Size Partition Style
------ ------------- ----------------- ---------- ---------------
0 VMware, VMware Virtual S SCSI Disk Device Online 40 GB MBR
7 VMware, VMware Virtual S SCSI Disk Device Online 60 GB MBR
9 VMware, VMware Virtual S SCSI Disk Device Online 30 GB MBR
2 VMware, VMware Virtual S SCSI Disk Device Offline 100 GB RAW
6 VMware, VMware Virtual S SCSI Disk Device Offline 20 GB RAW
4 VMware, VMware Virtual S SCSI Disk Device Offline 20 GB RAW
3 VMware, VMware Virtual S SCSI Disk Device Offline 100 GB RAW
8 VMware, VMware Virtual S SCSI Disk Device Offline 150 GB RAW
1 VMware, VMware Virtual S SCSI Disk Device Offline 100 GB RAW
5 VMware, VMware Virtual S SCSI Disk Device Offline 20 GB RAW
get-disk|ft -AutoSize Number,@{label="Size(MB)";expression={"{0:N0}" -f (($_.size)/1MB)}},@{label="Allocated Size(MB)";expression={"{0:N0}" -f (($_.allocatedsize)/1MB)}},NumberOfPartitions,PartitionStyle,isReadOnly,isSystem,isBoot
Number Size(MB) Allocated Size(MB) NumberOfPartitions PartitionStyle isReadOnly isSystem isBoot
------ -------- ------------------ ------------------ -------------- ---------- -------- ------
0 40,960 40,960 2 MBR False True True
7 61,440 61,439 1 MBR False False False
9 30,720 30,719 1 MBR False False False
2 102,400 0 0 RAW True False False
6 20,480 0 0 RAW True False False
4 20,480 0 0 RAW True False False
3 102,400 0 0 RAW True False False
8 153,600 0 0 RAW True False False
1 102,400 0 0 RAW True False False
5 20,480 0 0 RAW True False False
Now, what generally happens is that you would give the server and storage guys these requirements and they would most likely give you a server with the OS already installed along with some basic applications (anti-Virus, desktop management tools, etc) and all of the non-Exchange data disks already configured. For example, they would give you the C:, D:, E: and F: drives and a bunch of RAW disks for you to deal with. So we don’t want to touch the already provisioned disks except for creating some directories for the mountpoints we’ll be using.
So how do we exclude these drives from our script? Well we could filter them out a few ways, by size, disk number, name, etc. If we truly have a standard build for these machines, and we should, then we can probably just use the size as another filter. You need to determine which filter best works in your situation. If you have similar sizes across different sets of disks, then maybe you need to add a couple of filters to narrow it down to just the Exchange data disks.
For example, excluding the system disk would go something like this:
$nonsysdisks=get-disk | where {$_.issystem -eq $false -and $_.isboot -eq $false}
Now let’s extend that further to excluding the E: (Pagefile) and F: (Applications) drives so that we’re left with only the database, log and restore LUNs. For my environment I’m going to filter by the allocated size. Since it should be ‘0’ for all unprovisioned disks the query looks like this.
$exchdatadisks= Get-Disk| where {$_.allocatedsize -eq 0}
Number Friendly Name OperationalStatus Total Size Partition Style
------ ------------- ----------------- ---------- ---------------
2 VMware, VMware Virtual S SCSI Disk Device Offline 100 GB RAW
6 VMware, VMware Virtual S SCSI Disk Device Offline 20 GB RAW
4 VMware, VMware Virtual S SCSI Disk Device Offline 20 GB RAW
3 VMware, VMware Virtual S SCSI Disk Device Offline 100 GB RAW
8 VMware, VMware Virtual S SCSI Disk Device Offline 150 GB RAW
1 VMware, VMware Virtual S SCSI Disk Device Offline 100 GB RAW
5 VMware, VMware Virtual S SCSI Disk Device Offline 20 GB RAW
Now group them by size to confirm how many of each you have. Hopefully the numbers you end up with match with your Exchange design.
$exchdatadisks|group size|ft count,@{label="Size(GB)";expression={$_.name/1GB}} –AutoSize
Count Size(GB)
----- --------
3 100
3 20
1 150
Assuming everything checks out we can now move onto provisioning our Exchange data disks. I suggest working with the disks according to their sizes and intended function. This will make it easier to loop through the groups of disks and perform operations on them.
If you recall our directory structure is going to look like this:
Now let’s create the folder structure we need.
# Create root level directory for ExchangeData and sub folder for Restores
New-Item -ItemType directory -Path F:\ExchangeData # Creates the root-level folder
New-Item -ItemType directory -Path F:\ExchangeData\Restore # Creates the sub-level for the Restore volume
# Create Database and Log directories
1..3|%{
New-Item -ItemType directory -Path F:\ExchangeData\DB$_ # Creates the first sub-level DBx directories (i.e. DB1, DB2, DB3)
New-Item -ItemType directory -Path F:\ExchangeData\DB$_\Database # Creates the second sub-level dirs for the databases (mountpoint dirs)
New-Item -ItemType directory -Path F:\ExchangeData\DB$_\Logs # Creates the second sub-level dirs for the Logs (mountpoint dirs)
}
Verify the structure in Windows Explorer.
Here’s the completed script that ties everything together, creating the directory structure plus preparing the disks and mountpoints. You would just run this on each server. You’ll notice that I put a delay in the script. After running this multiple times I noticed occasionally I would get some strange “read-only” errors when creating the partition and formatting the volume so I gave it a few seconds delay before formatting the volume and that seemed to do the trick.
# Create root level directory for ExchangeData and sub folder for Restores
New-Item -ItemType directory -Path F:\ExchangeData # Creates the root-level folder
New-Item -ItemType directory -Path F:\ExchangeData\Restore # Creates the sub-level for the Restore volume
# Create Database and Log directories
1..3|%{
New-Item -ItemType directory -Path F:\ExchangeData\DB$_ # Creates the first sub-level DBx directories (i.e. DB1, DB2, DB3)
New-Item -ItemType directory -Path F:\ExchangeData\DB$_\Database # Creates the second sub-level dirs for the databases (mountpoint dirs)
New-Item -ItemType directory -Path F:\ExchangeData\DB$_\Logs # Creates the second sub-level dirs for the Logs (mountpoint dirs)
}
# Gather DB, Log and Restore disks
$exchdbdisks= Get-Disk| where {$_.allocatedsize -eq 0 -and $_.size -eq 107374182400}
$exchlogdisks= Get-Disk| where {$_.allocatedsize -eq 0 -and $_.size -eq 21474836480}
$restoredisk=Get-Disk| where {$_.size -eq 161061273600}
# Initialize and format restore volume
Initialize-Disk $restoredisk.number -PartitionStyle GPT
$newpartition=New-Partition -DiskNumber $restoredisk.number -UseMaximumSize
Start-Sleep -Seconds 3
$newpartition|Format-Volume -FileSystem NTFS -NewFileSystemLabel Restore -AllocationUnitSize 65536 -Confirm:$false
# "Mount up"
Add-PartitionAccessPath -DiskNumber $restoredisk.number -PartitionNumber 2 -AccessPath "F:\ExchangeData\Restore\"
1..3|ForEach {
# Initialize and format database volumes
Initialize-Disk -Number $exchdbdisks[($_-1)].Number -PartitionStyle GPT
$newpartition=New-Partition -DiskNumber $exchdbdisks[($_-1)].Number -UseMaximumSize
Start-Sleep -Seconds 3
$newpartition|Format-Volume -FileSystem NTFS -NewFileSystemLabel DB$_ -AllocationUnitSize 65536 -Confirm:$false
# "Mount up"
Add-PartitionAccessPath -DiskNumber $exchdbdisks[($_-1)].Number -PartitionNumber 2 -AccessPath "F:\ExchangeData\DB$_\Database\"
# Initialize and format log volumes
Initialize-Disk -Number $exchlogdisks[($_-1)].Number -PartitionStyle GPT
$newpartition=New-Partition -DiskNumber $exchlogdisks[($_-1)].Number -UseMaximumSize
Start-Sleep -Seconds 3
$newpartition|Format-Volume -FileSystem NTFS -NewFileSystemLabel Logs$_ -AllocationUnitSize 65536 -Confirm:$false
# "Mount up"
Add-PartitionAccessPath -DiskNumber $exchlogdisks[($_-1)].Number -PartitionNumber 2 -AccessPath "F:\ExchangeData\DB$_\Logs\"
}
Here’s a quick summary showing the mountpoints we created:
SystemName Name Size(GB) FreeSpace(GB) % Free Label
---------- ---- -------- ------------- ------ -----
WIN2K12 C:\ 39.7 27.4 69 %
WIN2K12 F:\ 59.9 59.8 100 % Applications
WIN2K12 F:\ExchangeData\DB1\Database\ 99.9 99.8 100 % DB1
WIN2K12 F:\ExchangeData\DB2\Database\ 99.9 99.8 100 % DB2
WIN2K12 F:\ExchangeData\DB3\Database\ 99.9 99.8 100 % DB3
WIN2K12 F:\ExchangeData\DB1\Logs\ 19.9 19.8 100 % Logs1
WIN2K12 F:\ExchangeData\DB2\Logs\ 19.9 19.8 100 % Logs2
WIN2K12 F:\ExchangeData\DB3\Logs\ 19.9 19.8 100 % Logs3
WIN2K12 E:\ 29.9 29.8 100 % Pagefile
WIN2K12 F:\ExchangeData\Restore\ 149.9 149.8 100 % Restore
WIN2K12 \\?\Volume{1e1d049b-00ea-11e2-93e8-806e6f6e6963}\ 0.3 0.1 31 % System Reserved
The mini script used to produce this report:
####################################################################
# Begin function for checking mountpoints and free space
function getDiskFreeSpace
{
Get-WmiObject Win32_Volume -Namespace root/cimv2 -computername $args -Filter "DriveType='3'" | `
Select-Object SystemName, Name, @{Name="Size(GB)";Expression={"{0:N1}" -f($_.Capacity/1gb)}},@{Name="FreeSpace(GB)";Expression={"{0:N1}" -f($_.freespace/1gb)}},@{Name="% Free";Expression={"{0:P0}" -f($_.freespace/$_.capacity)}},Label | `
Sort-Object -property "Label" | `
Format-Table -AutoSize
}
# End function for checking mountpoints and free space
####################################################################
getDiskFreeSpace localhost|out-default
When scripting against disks you might want to use filters like this to avoid destroying disks you didn’t intend to touch. And that would be bad (a la Mr. Mackey).
I hope this post inspires you develop your own automated storage scripts.
Until next time.
