Using Bicep to create an Azure Machine Learning Workspace resources and get started with Azure…

10 mins read

💪Using Bicep to create an Azure Machine Learning Workspace resources and get started with Azure Machine Learning

In this article, we will leverage Bicep, the Azure domain-specific language (DSL) to deploy a workspace for Azure Machine Learning.

Machine Learning Workspace in Bicep Language

Azure Machine Learning is a cloud service for accelerating and managing the machine learning project lifecycle. In Azure, you can create a model using the Machine Learning Service or use a model from an Open Source platform like TensorFlow or Pytorch.

If you’re implementing Machine Learning(ML) Operations within your organization to bring ML models to production, then you can benefit from Azure Machine Learning as you will be able to:

Leverage Azure compute resourcesShare data, notebooks, and environmentsKeep track of the versions of your model

Azure Machine Learning can be integrated with other services to support your project, like Azure Synapse Analytics to process and stream data with Spark. You can also integrate Azure SQL Database, Storage Blobs, and App Services to deploy and manage ML-powered apps.

Your entire ML project can be managed with Azure Machine Learning Service. It provides a workspace that allows for collaboration for many users. You can share the results of the runs from the experiments and use versioned assets for jobs like environments and storage references.

Through Azure Machine Learning, you will be able to manage your complete project lifecycle, including tasks definitions, data preparation, training, validation of your models, deployment of models, monitoring and publishing models.

Azure Machine Learning

Before working with Azure Machine Learning, we have to create a workspace and then add compute resources to the workspace.

The workspace is the top resource for all your machine learning activities and a centralized place to view and manage artifacts.

The image below shows a high-level architecture of Azure Machine Learning and the components included in the workspace:

Azure Machine Learning

That said, we will use Bicep to deploy the Azure Machine Learning Workspace.

Taxonomy of the workspace

The Bicep file will include the following components:

Application InsightsContainer RegistryKey VaultMachine Learning workspaceStorage accountAzure Machine Learning with Bicep

Bicep file — Parameters

We will define a few parameters for the deployment’s name, environment, and location. We will also define a parameter bool that specifies whether to reduce telemetry collection and enable additional encryption.

The code below shows the parameter definition:

@description(‘Specifies the name of the deployment.’)
param name string@description(‘Specifies the name of the environment.’)
param environment string@description(‘Specifies the location of the Azure Machine Learning workspace and dependent resources.’)
param location string = resourceGroup().location@description(‘Specifies whether to reduce telemetry collection and enable additional encryption.’)
param hbi_workspace bool = false

Now we will define the variables section.

Bicep file — Variables

The code below shows the definition of the variables to be utilized. We will use variables to specify the name of the resources to be created:

var tenantId = subscription().tenantId
var storageAccountName_var = ‘st${name}${environment}’
var keyVaultName_var = ‘kv-${name}-${environment}’
var applicationInsightsName_var = ‘appi-${name}-${environment}’
var containerRegistryName_var = ‘cr${name}${environment}’
var workspaceName_var = ‘mlw${name}${environment}’
var storageAccount = storageAccountName.id
var keyVault = keyVaultName.id
var applicationInsights = applicationInsightsName.id
var containerRegistry = containerRegistryName.id

Now let’s define the resources.

Bicep file — Resources

We will define the following resources:

Storage AccountKey VaultApplication InsightsContainer RegistryMachine Learning Service Workspace

The code below shows the definition of the resources.

resource storageAccountName ‘Microsoft.Storage/storageAccounts@2021-01-01’ = {
name: storageAccountName_var
location: location
sku: {
name: ‘Standard_RAGRS’
}
kind: ‘StorageV2’
properties: {
encryption: {
services: {
blob: {
enabled: true
}
file: {
enabled: true
}
}
keySource: ‘Microsoft.Storage’
}
supportsHttpsTrafficOnly: true
}
}resource keyVaultName ‘Microsoft.KeyVault/vaults@2021-04-01-preview’ = {
name: keyVaultName_var
location: location
properties: {
tenantId: tenantId
sku: {
name: ‘standard’
family: ‘A’
}
accessPolicies: []
enableSoftDelete: true
}
}resource applicationInsightsName ‘Microsoft.Insights/components@2020-02-02’ = {
name: applicationInsightsName_var
location: (((location == ‘eastus2’) || (location == ‘westcentralus’)) ? ‘southcentralus’ : location)
kind: ‘web’
properties: {
Application_Type: ‘web’
}
}resource containerRegistryName ‘Microsoft.ContainerRegistry/registries@2019-12-01-preview’ = {
sku: {
name: ‘Standard’
}
name: containerRegistryName_var
location: location
properties: {
adminUserEnabled: true
}
}resource workspaceName ‘Microsoft.MachineLearningServices/workspaces@2021-07-01’ = {
identity: {
type: ‘SystemAssigned’
}
name: workspaceName_var
location: location
properties: {
friendlyName: workspaceName_var
storageAccount: storageAccount
keyVault: keyVault
applicationInsights: applicationInsights
containerRegistry: containerRegistry
hbiWorkspace: hbi_workspace
}
dependsOn: [
storageAccountName
keyVaultName
applicationInsightsName
containerRegistryName
]
}

Note that we define a few dependencies in the workspace to ensure all the rest of the resources needed to deploy the workspace exists before deploying the workspace resource type.

Now we will define the parameters file. This file will contain only a few parameters: the name of the deployment, the name of the environment, and the location.

The code below shows the definition of the parameters file.

{
“$schema”: “https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#“,
“contentVersion”: “1.0.0.0”,
“parameters”: {
“name”: {
“value”: “azinsiderb”
},
“environment”: {
“value”: “azinsiderb”
},
“location”: {
“value”: “eastus”
}
}
}

Now we will proceed to deploy this resource. We previously created a resource group called ‘AzInsiderBicep’. We will use the following code to deploy this Bicep file:

$date = Get-Date -Format “MM-dd-yyyy”
$deploymentName = “AzInsiderDeployment”+”$date”New-AzResourceGroupDeployment -Name $deploymentName -ResourceGroupName AzInsiderBicep -TemplateFile .main.bicep -TemplateParameterFile .azuredeploy.parameters.json -c

Note that we use the flag -c to preview the deployment before executing it.

The figure below shows the preview of the deployment.

Azure Machine Learning — Bicep deployment preview

Once the Bicep file is valid, we will proceed to execute the deployment. The image below shows the output from this deployment.

Azure Machine Learning — Bicep deployment output

You can go to the Azure Portal and see the resources deployed as shown below:

Azure Machine Learning Workspace deployment— Azure Portal

From this point, you can start working on your own workspace and add compute instances and compute clusters as needed for your model.

Here’s the complete Bicep code to deploy an Azure Machine Learning Workspace:

@description(‘Specifies the name of the deployment.’)
param name string@description(‘Specifies the name of the environment.’)
param environment string@description(‘Specifies the location of the Azure Machine Learning workspace and dependent resources.’)
param location string = resourceGroup().location@description(‘Specifies whether to reduce telemetry collection and enable additional encryption.’)
param hbi_workspace bool = falsevar tenantId = subscription().tenantId
var storageAccountName_var = ‘st${name}${environment}’
var keyVaultName_var = ‘kv-${name}-${environment}’
var applicationInsightsName_var = ‘appi-${name}-${environment}’
var containerRegistryName_var = ‘cr${name}${environment}’
var workspaceName_var = ‘mlw${name}${environment}’
var storageAccount = storageAccountName.id
var keyVault = keyVaultName.id
var applicationInsights = applicationInsightsName.id
var containerRegistry = containerRegistryName.idresource storageAccountName ‘Microsoft.Storage/storageAccounts@2021-01-01’ = {
name: storageAccountName_var
location: location
sku: {
name: ‘Standard_RAGRS’
}
kind: ‘StorageV2’
properties: {
encryption: {
services: {
blob: {
enabled: true
}
file: {
enabled: true
}
}
keySource: ‘Microsoft.Storage’
}
supportsHttpsTrafficOnly: true
}
}resource keyVaultName ‘Microsoft.KeyVault/vaults@2021-04-01-preview’ = {
name: keyVaultName_var
location: location
properties: {
tenantId: tenantId
sku: {
name: ‘standard’
family: ‘A’
}
accessPolicies: []
enableSoftDelete: true
}
}resource applicationInsightsName ‘Microsoft.Insights/components@2020-02-02’ = {
name: applicationInsightsName_var
location: (((location == ‘eastus2’) || (location == ‘westcentralus’)) ? ‘southcentralus’ : location)
kind: ‘web’
properties: {
Application_Type: ‘web’
}
}resource containerRegistryName ‘Microsoft.ContainerRegistry/registries@2019-12-01-preview’ = {
sku: {
name: ‘Standard’
}
name: containerRegistryName_var
location: location
properties: {
adminUserEnabled: true
}
}resource workspaceName ‘Microsoft.MachineLearningServices/workspaces@2021-07-01’ = {
identity: {
type: ‘SystemAssigned’
}
name: workspaceName_var
location: location
properties: {
friendlyName: workspaceName_var
storageAccount: storageAccount
keyVault: keyVault
applicationInsights: applicationInsights
containerRegistry: containerRegistry
hbiWorkspace: hbi_workspace
}
dependsOn: [
storageAccountName
keyVaultName
applicationInsightsName
containerRegistryName
]
}

Next Steps:

I recommend you the following resources as the next steps:

Quickstart: Create workspace resources – Azure Machine LearningWhat is a workspace? – Azure Machine Learning

Join the AzInsider email list here.

-Dave R.

💪Using Bicep to create an Azure Machine Learning Workspace resources and get started with Azure… was originally published in CodeX on Medium, where people are continuing the conversation by highlighting and responding to this story.

Leave a Reply

Your email address will not be published.

Follow Us