CosmosDb is a non-relational, distributed database you can to build highly scalable, global applications in Azure. It is:
– Always on
– Offers throughput scalability
– Low latency, guaranteed
– No schema or indexes management
– Globally distributed
– Enterprise ready
– Use popular no-Sql APIs
https://docs.microsoft.com/en-us/azure/cosmos-db/introduction
It’s all great from a development perspective, but when it comes to management things are a bit different. CosmosDb can be managed to a certain extent through Azure CLI but as of this writing there is no Powershell module available:
PS /> Find-Module *cosmos* | select Name, Version, Author Name Version Author ---- ------- ------ CosmosDB 3.2.3.359 Daniel Scott-Raynsford AxCosmosDB 1.0.0 Axel Bøg Andersen CosmosDBCmdlets 19.0.7045.0 CData Software Inc.
I admit I have only superficially explored the existing modules and while it is great to see the Powershell Community sharing modules and scripts, it would probably be nice to have an official module by Microsoft as many other Resource Provider offer.
This is a good opportunity though to continue exploring the topic I have introduced in my previous post about calling Resource Provider actions and since these script will likely be reused (I can actually use them at work), why not build a reusable module?
Module basics
Writing a Windows Powershell Module is a good point to start to get an overview of this topic; I’ll write a script module and I’ll skip some of the introduction and go to the first important decision: how do we call this new module? 🤓
Long ago my team was known as LiveSite Engineering and most of the modules I wrote to share with my colleagues at work use the name LSE* (LSEAzure, LSEFunctions, LSECompliance and so on) so I guess I’ll just call this one LSECosmos (I know, I won’t win the Nobel prize for originality 😏). Why is this important? Because in Powershell the files building a module must be contained in a folder whose name matches the module name, so my first step is to create a new folder called LSECosmos:
➜ Modules git:(dev) mkdir LSECosmos ➜ Modules git:(dev) ls -l total 0 drwxr-xr-x 19 carlo staff 608 May 1 22:10 Archive drwxr-xr-x 46 carlo staff 1472 May 1 22:10 LSEAzure drwxr-xr-x 10 carlo staff 320 Mar 30 2018 LSECompliance drwxr-xr-x 2 carlo staff 64 May 1 22:12 LSECosmos drwxr-xr-x 37 carlo staff 1184 May 1 22:10 LSEFunctions drwxr-xr-x 83 carlo staff 2656 May 1 22:10 LSEOaaS drwxr-xr-x 6 carlo staff 192 Feb 5 19:37 OaaSCdoAdminModule drwxr-xr-x 13 carlo staff 416 Feb 18 14:02 Orchestrator.AccountKeyMigration https://github.com/carlocardella/LSECosmos
Next we need to create the module manifest with New-ModuleManifest:
PS /Users/carlo/Git/LSECosmos> New-ModuleManifest -Author 'Carlo Cardella' -ModuleVersion '0.0.1' -Path .\LSECosmos.psd1 -CompatiblePSEditions Desktop, Core -Guid (New-Guid).Guid -PowerShellVersion '5.1' -DotNetFrameworkVersion '4.7.2' -PassThru
#
# Module manifest for module 'LSECosmos'
#
# Generated by: Carlo Cardella
#
# Generated on: 5/1/19
#
@{
# Script module or binary module file associated with this manifest.
RootModule = 'LSECosmos.psm1'
# Version number of this module.
ModuleVersion = '0.0.1'
# Supported PSEditions
CompatiblePSEditions = 'Desktop', 'Core'
# ID used to uniquely identify this module
GUID = '4e4bd46a-81bf-48c2-b499-c367c4f043b7'
# Author of this module
Author = 'Carlo Cardella'
# Company or vendor of this module
CompanyName = ''
# Copyright statement for this module
Copyright = '(c) Carlo Cardella. All rights reserved.'
# Description of the functionality provided by this module
# Description = ''
# Minimum version of the PowerShell engine required by this module
PowerShellVersion = '5.1'
# Name of the PowerShell host required by this module
# PowerShellHostName = ''
# Minimum version of the PowerShell host required by this module
# PowerShellHostVersion = ''
# Minimum version of Microsoft .NET Framework required by this module. This prerequisite is valid for the PowerShell Desktop edition only.
DotNetFrameworkVersion = '4.7.2'
# Minimum version of the common language runtime (CLR) required by this module. This prerequisite is valid for the PowerShell Desktop edition only.
# CLRVersion = ''
# Processor architecture (None, X86, Amd64) required by this module
# ProcessorArchitecture = ''
# Modules that must be imported into the global environment prior to importing this module
# RequiredModules = @()
# Assemblies that must be loaded prior to importing this module
# RequiredAssemblies = @()
# Script files (.ps1) that are run in the caller's environment prior to importing this module.
# ScriptsToProcess = @()
# Type files (.ps1xml) to be loaded when importing this module
# TypesToProcess = @()
# Format files (.ps1xml) to be loaded when importing this module
# FormatsToProcess = @()
# Modules to import as nested modules of the module specified in RootModule/ModuleToProcess
# NestedModules = @()
# Functions to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no functions to export.
# FunctionsToExport = '*'
# Cmdlets to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no cmdlets to export.
# CmdletsToExport = '*'
# Variables to export from this module
# VariablesToExport = '*'
# Aliases to export from this module, for best performance, do not use wildcards and do not delete the entry, use an empty array if there are no aliases to export.
# AliasesToExport = '*'
# DSC resources to export from this module
# DscResourcesToExport = @()
# List of all modules packaged with this module
# ModuleList = @()
# List of all files packaged with this module
# FileList = @()
# Private data to pass to the module specified in RootModule/ModuleToProcess. This may also contain a PSData hashtable with additional module metadata used by PowerShell.
PrivateData = @{
PSData = @{
# Tags applied to this module. These help with module discovery in online galleries.
# Tags = @()
# A URL to the license for this module.
# LicenseUri = ''
# A URL to the main website for this project.
# ProjectUri = ''
# A URL to an icon representing this module.
# IconUri = ''
# ReleaseNotes of this module
# ReleaseNotes = ''
} # End of PSData hashtable
} # End of PrivateData hashtable
# HelpInfo URI of this module
# HelpInfoURI = ''
# Default prefix for commands exported from this module. Override the default prefix using Import-Module -Prefix.
# DefaultCommandPrefix = ''
}
PS /Users/carlo/Git/LSECosmos> Test-ModuleManifest ./LSECosmos.psd1
ModuleType Version Name ExportedCommands
---------- ------- ---- ----------------
Manifest 0.0.1 LSECosmos
The module manifest only contains basic information for now but I want to call out a couple important point:
CompatiblePSEdition: I am setting it to bothDesktopandCorebecause this module can be loaded and used on both Powershell versionsGUID: I am passing a random GUID, its value does not really matter but it must be unique, not shared with other modulesPowerShellVersion: this represent the minimum version required to run the module. I do not really need to be strict on version 5.1 for this module but I prefer to use the latest version everywhere on my machines (and my colleagues will excuse me if I force them to keep their systems updated 😉)DotNetFrameworkVersion: same reasoning as PowerShellVersion above
Now for the fun part: the actual module’s code.
Typically you’ll find a script module’s code in a .psm1 file and this is where all the functions (and other code) exposed by the module is supposed to live; anyway I find this file can quickly become very large and hard to maintain, so I prefer to keep every function in its own separate .ps1 file and use the .psm1 to dot source them. Let’s add a LSECosmos.psm1 file with this simple line:
Get-ChildItem "$PSScriptRoot\*.ps1" | foreach { . $_.FullName }
When we’ll load the module, Powershell will execute the content of the .psm1 file which, in turn, will simply list all the .ps1 files in the module folder and dot source them, thus making the functions available to the user. This allows to easily add new functions (new .ps1 files) into the folder and they will automatically become part of the module; it also makes it easier to identify files that need to be updated. Even if not strictly required, I make sure the .ps1 file name matches the function name it contains, so if I find a bug in a certain function I know which file I need to update.
List CosmosDb Accounts
Let’s start with something easy: first off, we need a function to list the existing CosmosDb accounts in the current Azure Subscription:
function Get-AzCosmosDbAccount {
<#
.SYNOPSIS
Returns information about CosmosDB accounts in the current subscription.
You can pass a single Account name or ResourceGroup name
.PARAMETER Name
CosmosDB Account name to return information for
Accepts wildcards
.PARAMETER ResourceGroupName
Resource Group containing the CosmosDB account to look for
Accepts wildcards
.EXAMPLE
Get-AzCosmosDbAccount -Name ncusoaasprodcosmos
Name : mycosmosaccount
ResourceGroupName : mycosmosaccountRG
ResourceType : Microsoft.DocumentDb/databaseAccounts
Location : northcentralus
ResourceId : /subscriptions/f897c2fa-a735-4e03-b019-890cd2f7109e/resourceGroups/mycosmosaccountRG/providers/Microsoft.DocumentDb/databaseAccounts/mycosmosaccount
.NOTES
This function uses Get-AzResource -ApiVersion '2018-11-01'
#>
[CmdletBinding()]
param (
[parameter(ValueFromPipeline, ValueFromPipelineByPropertyName, Position = 1)]
[string]$Name = '*',
[parameter(ValueFromPipelineByPropertyName, Position = 2)]
[string]$ResourceGroupName = '*'
)
process {
Get-AzResource -ResourceType 'Microsoft.DocumentDb/databaseAccounts' -ApiVersion '2018-11-01' | Where-Object 'ResourceGroupName' -Like $ResourceGroupName | Where-Object 'Name' -Like $Name
}
}
This is a very simple function: it uses Get-AzResource to return all resource with type Microsoft.DocumentDb/databaseAccounts unless we pass a specific Account Name and/or ResourceGroupName, in which case the function will use them as query filter.
Get-AzCosmosDbAccount -Name mycosmos* Name : mycosmosaccount ResourceGroupName : mycosmosaccountRG ResourceType : Microsoft.DocumentDb/databaseAccounts Location : northcentralus ResourceId : /subscriptions/f897c2fa-a735-4e03-b019-890cd2f7109e/resourceGroups/mycosmosaccountRG/providers/Microsoft.DocumentDb/databaseAccounts/mycosmosaccount
Create a CosmosDb account
Listing existing CosmosDb Accounts is easy, let’s try to create a new CosmosDb account. The user need to provide basic information such as the Cosmos Account name and Resource Group, the location where to create the account in and I’m going to request the disaster recovery location (DR is good practice and a requirement for any serious Cloud solution). To make things simple with this first iteration of the function I’ll hard code a few policy parameters, we can improve this part in a future version if needed.
function New-AzCosmosDbAccount {
<#
.SYNOPSIS
Create a new CosmosDb account
.PARAMETER AccountName
CosmosDb Account Name
.PARAMETER ResourceGroupName
Resource Group to contain the new CosmosDb Account
.PARAMETER Location
Location where to create the new CosmosDb Account
.PARAMETER DRLocation
Disaster Recover (failover) location for the new CosmosDb Account
.EXAMPLE
New-AzCosmosDbAccount -AccountName carloctestcosmosaccount -ResourceGroupName carloctestrg -Location centralus -DisasterRecoveryLocation southcentralus
#>
[CmdletBinding()]
param (
[parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$AccountName,
[parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$ResourceGroupName,
[parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$Location,
[parameter(Mandatory)]
[ValidateNotNullOrEmpty()]
[string]$DisasterRecoveryLocation
)
$locations = @(
@{"locationName" = $Location;
"failoverPriority" = 0
},
@{"locationName" = $DisasterRecoveryLocation;
"failoverPriority" = 1
}
)
$consistencyPolicy = @{
"defaultConsistencyLevel" = "Session";
"maxIntervalInSeconds" = "5";
"maxStalenessPrefix" = "100"
}
$DBProperties = @{
"databaseAccountOfferType" = "Standard";
"locations" = $locations;
"consistencyPolicy" = $consistencyPolicy;
}
New-AzResource -ResourceType "Microsoft.DocumentDb/databaseAccounts" `
-ApiVersion "2015-04-08" `
-ResourceGroupName $ResourceGroupName `
-Location $Location `
-Name $AccountName `
-PropertyObject $DBProperties `
-Force
}
List CosmosDb account keys
Going forward we’ll see a lot of the actions we’ll want to take on CosmosDb accounts or its Collections require authentication (as expected), which means we need to be able to retrieve either the connection string or the MasterKey from the Cosmos Account and pass it to the next call. There is no direct Powershell cmdlet to retrieve those essential pieces for authentication, using using my previous post as reference we can invoke the listKeysaction from Microsoft.DocumentDb/databaseAccounts. Also, I’m adding pipeline support since I can easily see myself using Get-AzCosmosDbAccount to search for a specific account name and pipe it to GetAzCosmosDbAccountKey rather than explicitly passing Name and ResourceGroupName. I am returning the output as generic PSCustomObject for now
function Get-AzCosmosDbAccountKey {
<#
.SYNOPSIS
Returns the Primary and Secondary keys for a Cosmos DB account
.EXAMPLE
Get-AzCosmosDbAccountKey -Name mycosmosdbaccount -ResourceGroupName mycosmosdbaccountRG
Name PrimaryMasterKey SecondaryMasterKey
---- ---------------- ------------------
carloctestcosmosaccount xxxxx xxxxx
.EXAMPLE
Get-AzCosmosDbAccount -Name mycosmos* | Get-AzCosmosDbAccountKey | Format-List
Name : mycosmosdbaccount
PrimaryMasterKey : xxxxx
SecondaryMasterKey : xxxxx
.NOTES
This function uses Get-AzResource -ApiVersion '2016-03-31'
#>
[CmdletBinding()]
param (
[parameter(Mandatory, ValueFromPipelineByPropertyName)]
[Alias('AccountName')]
[string]$Name,
[parameter(Mandatory, ValueFromPipelineByPropertyName)]
[string]$ResourceGroupName
)
process {
$keys = $null
$keys = Invoke-AzResourceAction -Action 'listKeys' -ResourceType "Microsoft.DocumentDb/databaseAccounts" -ApiVersion "2016-03-31" -ResourceGroupName $ResourceGroupName -Name $Name -Force
if ($keys) {
[PSCustomObject]@{
'Name' = $Name;
'PrimaryMasterKey' = $keys.primaryMasterKey;
'SecondaryMasterKey' = $keys.secondaryMasterKey
}
}
$keys = $null
}
}
Get-AzCosmosDbAccount -Name mycosmos* | Get-AzCosmosDbAccountKey | Format-List Name : mycosmosdbaccount PrimaryMasterKey : xxxxx SecondaryMasterKey : xxxxx
Follow along on Github, download the module, use it (I realize it is very limited now but it will improve over time 🤓), leave comments, send pull requests… you know the drill 😉
| You can’t shake hands with a clenched fist. – Mahatma Gandhi |