az vm image list-offers \  
--publisher Canonical \
--location westeurope \
-o table

Location    Name  
----------  -------------------------------------------
westeurope  0001-com-ubuntu-minimal-focal-daily  
westeurope  0001-com-ubuntu-minimal-groovy-daily  
westeurope  0001-com-ubuntu-minimal-hirsute-daily  
westeurope  0001-com-ubuntu-pro-advanced-sla  
westeurope  0001-com-ubuntu-pro-advanced-sla-att  
westeurope  0001-com-ubuntu-pro-advanced-sla-nestle  
westeurope  0001-com-ubuntu-pro-advanced-sla-servicenow  
westeurope  0001-com-ubuntu-pro-advanced-sla-shell  
westeurope  0001-com-ubuntu-pro-bionic  
westeurope  0001-com-ubuntu-pro-bionic-fips  
westeurope  0001-com-ubuntu-pro-focal  
westeurope  0001-com-ubuntu-pro-hidden-msft-fips  
westeurope  0001-com-ubuntu-pro-trusty  
westeurope  0001-com-ubuntu-pro-xenial  
westeurope  0001-com-ubuntu-pro-xenial-fips  
westeurope  0001-com-ubuntu-server-eoan  
westeurope  0001-com-ubuntu-server-focal  
westeurope  0001-com-ubuntu-server-focal-daily  
westeurope  0001-com-ubuntu-server-groovy  
westeurope  0001-com-ubuntu-server-groovy-daily  
westeurope  0001-com-ubuntu-server-hirsute-daily  
westeurope  0002-com-ubuntu-minimal-bionic-daily  
westeurope  0002-com-ubuntu-minimal-disco-daily  
westeurope  0002-com-ubuntu-minimal-focal-daily  
westeurope  0002-com-ubuntu-minimal-xenial-daily  
westeurope  0003-com-ubuntu-minimal-eoan-daily  
westeurope  0003-com-ubuntu-server-trusted-vm  
westeurope  test-ubuntu-premium-offer-0002  
westeurope  Ubuntu15.04Snappy  
westeurope  Ubuntu15.04SnappyDocker  
westeurope  UbunturollingSnappy  
westeurope  UbuntuServer  
westeurope  Ubuntu_Core  

Thanks for that, new SKUs lists are short:

az vm image list-skus \  
--publisher Canonical \
--offer 0001-com-ubuntu-server-focal-daily \
--location westeurope \
-o table

Location    Name  
----------  --------------------
westeurope  20_04-daily-lts  
westeurope  20_04-daily-lts-gen2  

At the end we have a much shorter list of images:

az vm image list --all \  
--publisher Canonical \
--offer 0001-com-ubuntu-server-focal-daily \
--sku 20_04-daily-lts \
--location westeurope \
-o table

Offer                               Publisher    Sku                   Urn                                                                                Version  
----------------------------------  -----------  --------------------  ---------------------------------------------------------------------------------  ---------------
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts       Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts:20.04.202012100       20.04.202012100  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts       Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts:20.04.202012110       20.04.202012110  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts       Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts:20.04.202101050       20.04.202101050  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts       Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts:20.04.202101060       20.04.202101060  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts       Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts:20.04.202101120       20.04.202101120  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts       Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts:20.04.202101140       20.04.202101140  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts       Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts:20.04.202101180       20.04.202101180  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts-gen2  Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts-gen2:20.04.202011260  20.04.202011260  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts-gen2  Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts-gen2:20.04.202012010  20.04.202012010  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts-gen2  Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts-gen2:20.04.202012100  20.04.202012100  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts-gen2  Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts-gen2:20.04.202012110  20.04.202012110  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts-gen2  Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts-gen2:20.04.202101050  20.04.202101050  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts-gen2  Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts-gen2:20.04.202101060  20.04.202101060  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts-gen2  Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts-gen2:20.04.202101120  20.04.202101120  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts-gen2  Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts-gen2:20.04.202101140  20.04.202101140  
0001-com-ubuntu-server-focal-daily  Canonical    20_04-daily-lts-gen2  Canonical:0001-com-ubuntu-server-focal-daily:20_04-daily-lts-gen2:20.04.202101180  20.04.202101180  

If you are wondering what is the root cause of this change, I have an answer for you:

We need to separate out our different releases into different offers, hence having them all distinct and then the numbering is because you can't delete or fully replace them so if publishing is stuck on 1 vm image in 1 listing, you cannot update any other vm image in that listing.

Install Cloudera CDH

CDH (Cloudera Distribution Hadoop) is an open-source Apache Hadoop distribution provided by Cloudera Inc which is a Palo Alto-based American enterprise software company. CDH (Cloudera's Distribution Including Apache Hadoop) is the most complete, tested, and widely deployed distribution of Apache Hadoop.

Configure HDFS

Edit /etc/hadoop/conf/core-site.xml config file using editor you like, filling-up {{AAD_tenant_ID}}, {{client_id}} and {{client_secret}} variables (Service Principal credentials) in the template below:

Mount HDFS endpoint

Now you can mount your ADLS Gen2 HDFS endpoint in your filesystem filling-up {{storage_account}}, {{container/fs}} and {{mount_point}} variables in command template below:

hadoop-fuse-dfs abfss://{{storage_account}}@{{container/fs}}.dfs.core.windows.net /{{mount_point}}

You (probably) want CentOS to mount your ADLS Gen2 HDFS endpoint on every startup. You can do it using /etc/fstab:

hadoop-fuse-dfs#abfss://{{storage_account}}@{{container/fs}}.dfs.core.windows.net {{mount_point}} fuse allow_other,usetrash,rw 2 0 or similar.

In my case it does not apply because I have few filesystems, I need to mount in each other, so the order of mounting matters. To handle that I added hadoop-fuse-dfs commands in proper order to /etc/rc.local script.

Optimizing Mountable HDFS

As you can find in CDH documentation:

• Cloudera recommends that you use the -obig_writes option on kernels later than 2.6.26. This option allows for better performance of writes.
• By default, the CDH package installation creates the /etc/default/hadoop-fuse file with a maximum heap size of 128 MB. You might need to change the JVM minimum and maximum heap size for better performance. For example: export LIBHDFS_OPTS="-Xms64m -Xmx256m". Be careful not to set the minimum to a higher value than the maximum.

In this configuration there is Apache Web Server (httpd) used to serve your app. If you scale out, you have as many Apache Web Servers as many instances in App Service Plan you had configured. At the front you have a load balancer. App Service Instances are NATed. App Services have inbound and outbound IP address pools and DNS name and domain. Azure Front Door is not integrating into App Service - it is totally separated from it.
But you are running Azure Front Door, because it is cool, and you are pointing to Azure App Service as a backend. And you want to service your website only through Azure Front Door, restricting the traffic.

It's easy. You just need to restrict the traffic to the Web App only to traffic originating from Azure Front Door backend addresses. The list of all public IP classess used by Azure services is public and you can find it here. All you need to do is to find AzureFrontDoor.Backend value on the list of objects, create a JSON definition of ipSecurityRestrictions setting of your Web App. Don't forget to add Azure's basic infrastructure services (through virtualized host IP addresses: 168.63.129.16 and 169.254.169.254) and IPv6 address range, currently limited to 2a01:111:2050::/44.

But what if someone runs it's own Azure Front Door and try to route to your Web App? It will work. You can configured it to work. What you want to do is to restrict the access only to YOUR Azure Front Door instance. It's also easy and documented. All you need is to filter request headers for X-Azure-FDID header value and pass only those requests that match your Front Door ID. For PHP running on built-in App Service on Linux image, you can do it at the Apache Web Server layer. You need to create .htaccess file in your application's repository (in the wwwroot directory) and add Apache2 Web Server directive to restrict access only to requests with X-Azure-FDID header pointing to your Azure Front Door instance:

For other stacks you need to pick proper options - i.e. for Python image you can do it in your app's code or try to do it at Gunicorn layer.

But how to determine Azure Front Door instance ID? It's also easy. What you need to do is to send a GET request to Azure Resource Manager API (version 2020-01-01) for your Azure Front Door instance (by name). In the response you will find a "frontdoorId" property.

The biggest part of the problem is that Azure is not validating custom roles when creating and we are able to create a role which is almost the same as built-in one.

Let's try with built-in Reader role:

{
    "assignableScopes": [
      "/"
    ],
    "description": "Lets you view everything, but not make any changes.",
    "id": "/subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/providers/Microsoft.Authorization/roleDefinitions/acdd72a7-3385-48ef-bd42-f606fba81ae7",
    "name": "acdd72a7-3385-48ef-bd42-f606fba81ae7",
    "permissions": [
      {
        "actions": [
          "*/read"
        ],
        "dataActions": [],
        "notActions": [],
        "notDataActions": []
      }
    ],
    "roleName": "Reader",
    "roleType": "BuiltInRole",
    "type": "Microsoft.Authorization/roleDefinitions"
}

We can see here clearly, that user with this role is able to perform read action on all resources, the role type is BuiltInRole and it's name is Reader.

Now let's check our custom role named Reader. Yes. The same name on the first look because you did not noticed a one, small space.

{
    "assignableScopes": [
  "/subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss"
    ],
    "description": "Lets you view everything, but not make any changes.",
    "id": "/subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/providers/Microsoft.Authorization/roleDefinitions/54f8ccae-dc88-47f1-b425-174ee843f162",
    "name": "54f8ccae-dc88-47f1-b425-174ee843f162",
    "permissions": [
      {
        "actions": [
          "*"
        ],
        "dataActions": [],
        "notActions": [],
        "notDataActions": []
      }
    ],
    "roleName": "Reader ",
    "roleType": "CustomRole",
    "type": "Microsoft.Authorization/roleDefinitions"
}

Notice that actions that user with this role can perform is "*" for whole subscription (which is a scope). The name of the role is "Reader " and the description of the role is the same as in built-in "Reader" role.

It shouldn't be so hard to notice that user is having an elevated permissions because it is not a "Reader" role but "Reader "? False.

Let's check the assignemnts list using Azure CLI:

{
    "canDelegate": null,
    "id": "/subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/providers/Microsoft.Authorization/roleAssignments/f8f14d8b-27c6-4969-8361-0c76a4e64bda",
    "name": "f8f14d8b-27c6-4969-8361-0c76a4e64bda",
    "principalId": "5a59308f-7c05-4512-9001-d3122d7e22e5",
    "principalName": "tester@free-media.eu",
    "roleDefinitionId": "/subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss/providers/Microsoft.Authorization/roleDefinitions/54f8ccae-dc88-47f1-b425-174ee843f162",
    "roleDefinitionName": "Reader ",
    "scope": "/subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss",
    "type": "Microsoft.Authorization/roleAssignments"
}

Of course, when validating, you will check roleDefinitionId? And of course you will validate that the roleDefinitionName is "Reader" not "Reader "? You are a PRO, so probably you will...

But how it is possible? It's simple. You just need to create a custom role with the same name as built-in one and add a hidden character, like space, at the end:

{
  "Name": "Reader ",
  "IsCustom": true,
  "Description": "Lets you view everything, but not make any changes.",
  "Actions": [
    "*"
  ],
  "NotActions": [

  ],
  "dataActions": [

  ],
  "notDataActions": [

  ],
  "AssignableScopes": [
    "/subscriptions/ssssssss-ssss-ssss-ssss-ssssssssssss"
  ]
}

Azure is just not validating it.

a service in Azure that is designed to extend Azure Resource Management by providing efficient and performant resource exploration with the ability to query at scale across all subscriptions and management groups so that you can effectively govern your environment. These queries provide the following capabilities:

• Ability to query resources with complex filtering, grouping, and sorting by resource properties.
• Ability to iteratively explore resources based on governance requirements and convert the resulting expression into a policy definition.
• Ability to assess the impact of applying policies in a vast cloud environment.

What is Azure Resource Graph?

Azure Resource Manager sends data to a cache that exposes some information about resource (Resource name, ID, Type, Resource Group, Subscriptions, and Location). Normally we make calls to each resource provider and request these informations for each resource. It means not only much more calls to make but also a need to create a script which will handle that operation.

With Azure Resource Graph, we can access these informations directly, using complex query language we know, the Kusto query language.

How to use Azure Resource Graph?

To use Azure Resource Graph, you need at least Reader (RBAC) role on the resources you want to query.

To query Azure Resource Graph, you can use Azure CLI, PowerShell, SDK or REST API directly.

Sample queries

How much time we are saving?

In simple words, a lot. Let's consider a simple scenario with just 10 VMs in just one subscription. If we want to summarize Operating Systems we are using Without Azure Resource Graph, we need to call Azure Resource Manager for all the resources with resource type like Microsoft.Compute/virtualMachines and then iterate all VMs for storageProfile.osDisk.osType property. In Azure CLI it will be something like that:

for id in `az resource list --resource-type 'Microsoft.Compute/virtualMachines' --query '[].id' -o tsv`; do az resource show --ids $id --query 'properties.storageProfile.osDisk.osType'; done | uniq -c

The uniq -c command at the end is summarizing every unique OS type. The output of this script for my subscription is:

3 "Windows"
7 "Linux"

and it took a little bit more than 11s to complete.

With Azure Resource Graph I used az graph query command with simple Kusto query:

az graph query -q "where type =~ 'Microsoft.Compute/virtualMachines' | summarize count() by tostring(properties.storageProfile.osDisk.osType)" -s xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx

The output of this command is:

[
  {
    "count_": 7,
    "properties_storageProfile_osDisk_osType": "Linux"
  },
  {
    "count_": 3,
    "properties_storageProfile_osDisk_osType": "Windows"
  }
]

and it took around 1.5s to complete.

As you can see, in graph query I have limited query to just one subscription - it's because Azure Resource Graph is returning results for all resources in all subscriptions we have access to. I have also noticed, that this rule is not true for subscriptions where we are external AAD users with RBAC roles - in that situation we are receiving "Access denied" message.

Schema

The response schema for ARG query is different than resource query. As an example, let's check a schema for query about a VM.

Command:

az vm show -g group -n VM0

Response:

If you are wondering what Aliases are, check here.

But it is not so complicated to do it by yourself. In a request to the ARM API (https://management.azure.com) you need to have Content-Type header and the Authorization header where Bearer token is placed. Use a variable for the token - let say {{access_token}}.

Next, you need to create a Pre-request Script to handle Access Token aquisition from oAuth endpoint in Azure Active Directory - you will find it in "Endpoints" blade inside "Application registration" blade (AAD).

Here you have a code I'm using for Pre-request Script:

As you can see, I'm not hardcoding client_id (Application ID), client_secret (Application Key) and tenant. I have it written in my Postman Environment. This is the same place, where access_token is written, when acquired from oAuth endpoint.

You can find it under this address.

Use cases

• Orchestration systems that manage containerized applications, like DC/OS, Docker Swarm or Kubernetes.
• Azure services based on Docker containers, like Azure Kubernetes Service (AKS), App Service, Batch, Azure Container Instances or Service Fabric.

Azure Container Registry can also be used as part of a standard container development workflow. For example, as a container target for continuous integration and deployment tools like Visual Studio Team Services or Jenkins.

Azure Container Registry is also a suite of features that provides Docker container image builds capability in Azure. Configurable build tasks can help to automate container OS and framework patching pipelines and build images automatically when commits will come to the code repository.

Pricing

Azure Container Registry has three pricing tiers: Basic, Standard and Premium. The following table details the features and limits of the Basic, Standard, and Premium service tiers.

The differences in prices is "radical" but there is no problem with later tier changing when needed.

So, assuming full storage utilization per month we have 10GB for €4.37/month, 100GB for €17.45/month and 500GB for €43.59/month.

We are not limited to GB included in a pricing plan. If we want to extend it, we just need to pay €0.003/GB/day (it's less than €1 per month for additional 10GB of storage in a Basic plan).

If we want to use ACR for cloud-based Docker image builds, we also need to pay for it - it's an additional cost of €0.00005/second (every minute of the build will cost us €0.003, so every three minutes of build is around ¢1).

Competition

Docker Hub is a cloud-based registry service which allows you to link to code repositories, build your images and test them, stores manually pushed images, and links to Docker Cloud so you can deploy images to your hosts. It provides a centralized resource for container image discovery, distribution and change management, user and team collaboration, and workflow automation throughout the development pipeline.

Pricing

In Docker Hub we are not paying for storage and builds. There is also no pricing tiers that depend on performance, limits or features. In contrary to ACR in Docker Hub all repositories are public by default. If you want to have more than one private repository, you need to pay and the price depends on the number of repositories. There is also a difference in a number of possible parallel builds per pricing plan.

Comparison

Of course it's hard to compare Azure Container Registry with Docker Hub. The power of ACR is privacy by default, integration with Azure services and a strong security compliance inherited from Azure itself.
The power of Docker Hub, in the other hand, is being a community hub for image creators and out-of-box integration with GitHub and Bitbucket.

The second part of the comparison is pricing - the choice depends on your repositories characteristics and I will not summarize this comparison - You should do that.

ACR in action

In my case, I have Python script which I need to run periodically. I don't need and I don't want to maintain VM for that. I just want to run this script for time to time. Of course it is not just a script - it has dependencies (Azure SDK for Python), so it's more like a bundle than a script - it's normal for Python and many other languages.
Docker is a perfect solution for me in that case. I can bundle SDK and other dependencies and use it as a base image for my script runtime environment - all without storing any data on image itself.

I have clear prerequisites:

1. I need Azure SDK for Python.
2. I have my script written in Python 3.6.
3. I need to pass some parameters to my script to prevent hardcoding.
4. I wish to run this script on almost any machine -Linux, Mac and Windows.
5. I wish to run this script periodically.

Azure SDK for Python

You can find it on PyPi (https://pypi.org/project/azure/) and you can install it using pip install azure. There is no philosophy here - it's open, it's developed on GitHub and it's available on PyPi.

An additional prerequisite for Azure SDK for Python is keyrings.alt package - due to this issue.

So, I have:

pip install azure keyrings.alt

Python 3.6

I'm working on Python 3.6 environment locally on my computer, where I'm developing, so I wish to have the same environment on the script's runtime. It's probably compatible with 3.5, 2.4 and 3.3 but... I'm working on 3.6.

Let see my script (sample) - it is listing all resource groups in my subscription:

Parameters

As you can see, I'm not hardcoding stuff like tenant, application ID, application key or subscription ID in my script. I'm using os.getenv() to extract it from environment variables. It means, that I need to include some "sensitive" data in my environment.

Interoperability

I don't want to focus on the question if my script will run on Windows Server or Linux... or Mac, that I'm using personally. Python is Python but... environments differs between Operating Systems. And this is the place where Docker comes to the game. it does not matter where you will run your Docker image - it will be totally the same from the code/script perspective.

We have two options in Docker world - first is to use pre-built images, prepared by community or team-mates; second is to use custom images we are building by our own.

If you are looking for ready-to-use images, check on Docker Hub or Docker Store.

But if you are looking for more flexible solution or you just want to have a lot of fun, try to build your own Docker image, using Dockerfile. As I assumed above, we need Python 3.6, azure package and keyrings.alt package. Let's create Dockerfile for that:

As you can see, it's really simple. We are getting Python image with tag pointing for version 3.6 from Docker Hub - a community repository - and it is an official Python image for docker. You can check it here.
The second step is to install packages wee need on top of python image. To do that, we are using pip of course. Image building means, that having Python 3.6 image, we are installing additional packages we want, and then we are generating new image based on the base one and changes we made. After that, we have a static environment image with Python 3.6 and Azure SDk for Python.

Complex Docker image

Having our script and a Docker image based on Python 3.6 official image, we can prepare a complex Docker image with ready-to-use solution. We need to merge the script with the environment image. We will do it using Dockerfile, adding script to it:

Runtime

Assuming, that we have created image above, we have a complex solution: we have a Python 3.6 interpreter, an Azure SDK for Python and keyrings.alt package. But when this image will runs, it will do... nothing. The script is inside, but the command is not declared. We need to declare what the image will do on startup:

And this is a complete solution. On startup, conainer will run run.py script, using Python 3.6 interpreter, where Azure SDk for Python is installed along with keyrings.alt package.

Build

At this point, we need to build our Docker imageand to do so we need to have run.py script and a Dockerfile in the same directory. Using shell where docker is installed - no matter on what OS - go to this directory and perform a command:

docker build -t imagename .

We have built the Docker image based on definition from Dockerfile, tagging it as "imagename" which tag will be used as the image name when running.

Run

No we know three things:

1. We have an "imagename" Docker image.
2. We want to run run.py script which is on that image.
3. We need to "inject" environment variables with sensitive data to the runtime.

The run.py script will run automatically because we built an image in that way. Only thing we need is to pass environment variables to the image on startup. To do it, we will use -e arguments to the *docker run * command. Let's do it:

docker run -e "TENANT_ID=<tenant_id>" -e "CLIENT=<applicatiom_id>" -e "KEY=<key>" -e "SUBSCRIPTION=<subscription_id>" imagename:latest   

Conclusion

The script should run and should start treating Docker as your daily-basis tool. Not because it's fancy and cool. Just because it's easy to use, simple and it works almost everywhere.

If you don't want to wait for image to build with SDK, I have created a ready-to-use image. You can find it here and use as a base:

Who is asking

When I have terminated this interactive prompt I have got a traceback pointing where the prompt starts in a code:

Please enter password for encrypted keyring:
  File "/usr/local/lib/python3.6/dist-packages/msrestazure/azure_active_directory.py", line 448, in __init__
self.set_token()
  File "/usr/local/lib/python3.6/dist-packages/msrestazure/azure_active_directory.py", line 485, in set_token
self._default_token_cache(self.token)
  File "/usr/local/lib/python3.6/dist-packages/msrestazure/azure_active_directory.py", line 207, in _default_token_cache
keyring.set_password(self.cred_store, self.store_key, str(token))
  File "/usr/lib/python3/dist-packages/keyring/core.py", line 47, in set_password
_keyring_backend.set_password(service_name, username, password)
  File "/usr/lib/python3/dist-packages/keyrings/alt/file_base.py", line 135, in set_password
password_encrypted = self.encrypt(password.encode('utf-8'), assoc)
  File "/usr/lib/python3/dist-packages/keyrings/alt/file.py", line 206, in encrypt
cipher = self._create_cipher(self.keyring_key, salt, IV)
  File "/usr/lib/python3/dist-packages/keyring/util/properties.py", line 56, in __get__
return self.fget(obj)
  File "/usr/lib/python3/dist-packages/keyrings/alt/file.py", line 96, in keyring_key
self._unlock()
  File "/usr/lib/python3/dist-packages/keyrings/alt/file.py", line 186, in _unlock
'Please enter password for encrypted keyring: ')
  File "/usr/lib/python3.6/getpass.py", line 77, in unix_getpass
passwd = _raw_input(prompt, stream, input=input)
  File "/usr/lib/python3.6/getpass.py", line 146, in _raw_input
line = input.readline()

I have focused on File "/usr/local/lib/python3.6/dist-packages/msrestazure/azure_active_directory.py", line 207, in _default_token_cache.

What I hound there is:

def _default_token_cache(self, token):
    """Store token for future sessions.

    :param dict token: An authentication token.
    :rtype: None
    """
    self.token = token
    if keyring:
        try:
            keyring.set_password(self.cred_store, self.store_key, str(token))
        except Exception as err:
            _LOGGER.warning("Keyring cache token has failed: %s", str(err))

So, it is nothing more, than using system keyring to store oAuth2 token for future use. msrestazure/azure_active_directory.py - which is used by Azure SDK for Python, is checking if keyring Python module is used in our system and if yes, it is using it to store oAuth token. Clear.

Why it started to happen?

I'm using Ubuntu Server 16.04-LTS for a long time, and there is no issue like that there. Why it is on 18.04? I made some more investigation on that and now I know.

When installing python3-pip package on Ubuntu 18.04 I noticed that python3-keyring and python3-keyrings.alt are also installed. Knowing that it's clear that _default_token_cache() function in msrestazure/azure_active_directory.py will decide to use keyring.

I have checked where it started looking at python3-pip dependencies and recommendations at packages.ubuntu.com. One of the package recommendations, and in fact used modules, is python3-wheel - and this module has:

python3-keyring - store and access your passwords safely - Python 3 version of the package

and

python3-keyrings.alt - alternate backend implementations for python3-keyring

as recommended dependencies. So, when installing python3-pip package in default mode, we will always get python3-keyring also installed.

There is no such recommendation in 16.04-LTS.

Solution

Of course we can try to not install python3-keyring or remove it. But it can be needed or even required by other packages or modules. If we do not need encryption, we can use the PlaintextKeyring backend which does not need any password. We have at least two options to do it and get back to the state where keyring is not used.

Configuration file

Edit ~/.local/share/python_keyring/keyringrc.cfg:

[backend]
default-keyring=keyrings.alt.file.PlaintextKeyring

Python API

Add this to your code:

import keyring.backend
from keyrings.alt.file import PlaintextKeyring

keyring.set_keyring(PlaintextKeyring())

Security

The safety of using PlaintextKeyring() as token storage is another story and it will be probably discussed on GitHub soon. I will inform you about other solutions (using keyring if possible) or security conclusions.

But official, stable image (not DAILY) is already there. To find it and use it, you need to use Azure CLI (or PowerShell).

To find it, use:

az vm image list --all -p Canonical -f UbuntuServer -s 18.04-LTS --query [].urn -o tsv

Find the one with "18.04-LTS" SKU.

Then, to deploy it, use:

az vm create -n MyVm -g MyResourceGroup --image Canonical:UbuntuServer:18.04-LTS:18.04.201804262

1. Implement validation method in your application, which you will be calling using WebHook.
2. Run standalone validator for a moment and then run your application on the same address.

The first option can be expensive or impossible (for example if you are going to use some SaaS tool in the subdomain you own).

The second option is quick and cheap. I created a sample Python web server, which only purpose is to respond to Azure Event Grid WebHook validation request.

Feel free to use, contribute and share: https://github.com/smereczynski/Azure-EventGrid-WebHook-Validator

What you need to do to find more images available is to use Azure CLI. The command we will use for VM images browsing is vm image. But first, we need to clarify some parameters:

1. publisher - the company or entity behind the image.
2. offer - this is the offering form the publisher.
3. sku - Stock Keeping Unit is a id assigned to an image to identify the exact product version.

To list all publishers in West Europe Azure region we will perform:

az vm image list-publishers -l westeurope --query [].name -o tsv

To list all offers in West Europe Azure region from choosen publisher, let say Canonical, we will perform:

az vm image list-offers -l westeurope -p Canonical --query [].name -o tsv

To list all SKUs form UbuntuServer offer in West Europe Azure region we will perform:

az vm image list-skus -l westeurope -p Canonical -f UbuntuServer --query [].name -o tsv

Now we need to check how VM creation is performed from Azure CLI. Typically it looks like this:

az vm create -n MyVm -g MyResourceGroup --image UbuntuLTS

where --image is: The name of the operating system image as a URN alias, URN, custom image name or ID, or VHD blob URI.

We are not using custom image or VHD and our image is not a standard, aliased image (like UbuntuLTS). So, we need to know image URN.
It's easy as:

az vm image list --all -p <publisher> -f <offer> -s <sku> -l <region>

Example for Ubuntu Server 18.04 LTS:

az vm image list --all -p Canonical -f UbuntuServer -s 18.04-DAILY-LTS -l westeurope

What we want to know is URN, so:

az vm image list --all -p Canonical -f UbuntuServer -s 18.04-DAILY-LTS -l westeurope --query [].urn -o tsv

The last part of the URN is an image version - you are probably looking for the last one. For 18.04 LTS daily builds, as for February 27th, we have:

Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802130
Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802140
Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802160
Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802170
Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802180
Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802190
Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802210
Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802220
Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802230
Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802240

If you want to create a VM from the last daily build's image, then:

az vm create -n MyVm -g MyResourceGroup --image Canonical:UbuntuServer:18.04-DAILY-LTS:18.04.201802240

With empty RGs it's quite easy - just list all RGs and for each RG check if there are resources in it - if no, then delete:

for i in `az group list -o tsv --query [].name`; do if [ "$(az resource list -g $i -o tsv)" ]; then echo "$i is not empty"; else az group delete -n $i -y --no-wait; fi; done

Cleaning not-empty RGs is much more complicated because we need to decide which of them are needed and which are not. The quickest way to do it is to add Delete Lock to every RG we want to leave untouched. Then we just need to iterate all RGs (as in previous example) and for each RG check if there is any Lock on it. If no Lock, then delete:

for i in `az group list -o tsv --query [].name`; do if [ "$(az lock list -g $i -o tsv --query [].name)" ]; then echo "$i have a lock"; else az group delete -n $i -y --no-wait; fi; done

Azure Resource Manager API

ARM REST API is well documented here. You will find there information about how to prepare for using REST API (i.e. create a Service Principal) and how to perform API Calls. There is also a (almost?) complete API Reference divided by service or resource type - it is where you will search for methods you can use for target resource type.

To start working with API Calls to ARM API, you need to have 5 things and know where to find 6th.

1. Client ID
2. Client Secret
3. Tenant ID
4. Resource
5. Authority URL
6. API version

First five of six above are related directly to oAuth2 flow, where Azure AD is an Identity Provider. The sixth one is a query-string parameter which points an API version to call.

Client ID is an Application ID you created for RBAC (as described here). This is the AAD Application with a Service Principal object related to it.

Client Secret is an AAD Application's key (password).

Tenant ID is your AD Tenant's ID you can find in AAD Properties or in output of the az ad sp create-for-rbac command.

Resource is https://management.azure.com/ - Azure Resource Manager provider APIs URI.

Authority URL is https://login.microsoftonline.com/ - the Identity Provider address.

API version is a query-string parameter with designated API version you should provide for service you are calling. You can find this parameter in API reference under provider of choice - here an example for Resource Management / Resource Groups / List.

Let say we want to list all Resource Groups in our subscription:

Find all subscription on the tenant

1. We need to find proper page in Azure REST API reference: https://docs.microsoft.com/en-us/rest/api/resources/subscriptions/list
2. We need to determine API Version: 2016-06-01 (from reference page).

3. We need to determine call's method and URI:

   GET https://management.azure.com/subscriptions?api-version=2016-06-01

As an output we will get a list (JSON) of all subscriptions on our Azure tenant.

Find all resource groups in the subscription of choice

1. We need to find proper page in Azure REST API reference: https://docs.microsoft.com/en-us/rest/api/resources/resourcegroups/list
2. We need to determine API Version: 2017-05-10 (from reference page).

3. We need to determine call's method and URI:

   GET https://management.azure.com/subscriptions/{subscriptionId}/resourcegroups?api-version=2017-05-10

As an output we will get a list (JSON) of all resource groups in our Azure subscription of choice.

Python (3)

Handling GET, HEAD, PUT, POST, and PATCH methods in Python can be implemented using many libraries and in many ways of thinking. The same situation is with oAuth flows. I decided to use requests (link) library for HTTP methods and adal (Link) library for Azure AD Authentication.

I choose requests because I know it and I'm using it. It is also probably the most popular library used for handling HTTP requests.
I choose adal library because it is officially a proper way to handle authentication against AAD in Python. It is also used in Azure CLI 2.0 and Azure SDK for Python.

Beside of requests and adal I will also use json library for handling JSON requests bodies and calls responses and os for os environment variables handling (no credentials hardcoding!).

Only requests and adal libraries requires to be installed:

pip install requests adal

So, the import code block of my Python script will be as follows:

import adal
import requests
import os
import json

Next, we will declare necessary variables, storing their values as environment variables:

tenant = os.environ['TENANT']
authority_url = 'https://login.microsoftonline.com/' + tenant
client_id = os.environ['CLIENTID']
client_secret = os.environ['CLIENTSECRET']
resource = 'https://management.azure.com/'

I'm using venv for Python runtime and PyCharm IDE (link) and both venv and env variables are handled by PyCharm.

Next, we are going to handle oAuth flow with adal, receiving an authorization token in an authentication context of https://login.microsoftonline.com/tenant - an identity provider:

context = adal.AuthenticationContext(authority_url)
token = context.acquire_token_with_client_credentials(resource, client_id, client_secret)

We will use token variable to extract a Bearer authorization token from the response which looks like this:

{
    "tokenType": "Bearer",
    "expiresIn": 3600,
    "expiresOn": "2018-02-16 19:55:32.068528",
    "resource": "https://management.azure.com/",
    "accessToken": "eyJ0eXAiOiJKV1QiLC...",
    "isMRRT": true,
    "_clientId": "<client_id>",
    "_authority": "https://login.microsoftonline.com/<tenant>"
}

The accessToken is a JWT - Json Web Token which can be translated on http://jwt.ms/.

Try to decode it by yourself - you will notice the correlation between your Application ID and token's content.

Next we are going to start building a request to the ARM API. First, we need a headers:

headers = {'Authorization': 'Bearer ' + token['accessToken'], 'Content-Type': 'application/json'}

Then, we create query-string parameter with API version (2016-06-01 is proper version according to the documentation):

params = {'api-version': '2016-06-01'}

Next, the url for getting the subscriptions list (according to the documentation):

url = 'https://management.azure.com/' + 'subscriptions'

And at last, performing the request and printing the response:

r = requests.get(url, headers=headers, params=params)
print(json.dumps(r.json(), indent=4, separators=(',', ': ')))

The response looks like this:

{
    "value": [
        {
            "id": "/subscriptions/<subscription_id>",
            "subscriptionId": "<subscription_id>",
            "displayName": "<subscription_name",
            "state": "Enabled",
            "subscriptionPolicies": {
                "locationPlacementId": "Public_2014-09-01",
                "quotaId": "Sponsored_2016-01-01",
                "spendingLimit": "Off"
            },
            "authorizationSource": "RoleBased"
        }
    ]
}

And whole script looks like this:

import adal
import requests
import os
import json


tenant = os.environ['TENANT']
authority_url = 'https://login.microsoftonline.com/' + tenant
client_id = os.environ['CLIENTID']
client_secret = os.environ['CLIENTSECRET']
resource = 'https://management.azure.com/'
context = adal.AuthenticationContext(authority_url)
token = context.acquire_token_with_client_credentials(resource, client_id, client_secret)
headers = {'Authorization': 'Bearer ' + token['accessToken'], 'Content-Type': 'application/json'}
params = {'api-version': '2016-06-01'}
url = 'https://management.azure.com/' + 'subscriptions'

r = requests.get(url, headers=headers, params=params)

print(json.dumps(r.json(), indent=4, separators=(',', ': ')))

Of course GET operations are little bit simpler than PUT or POST. There is no request body to send. Let's create some resource group on our subscription (docs):

import adal
import requests
import os
import json


tenant = os.environ['TENANT']
authority_url = 'https://login.microsoftonline.com/' + tenant
client_id = os.environ['CLIENTID']
client_secret = os.environ['CLIENTSECRET']
resource = 'https://management.azure.com/'
context = adal.AuthenticationContext(authority_url)
token = context.acquire_token_with_client_credentials(resource, client_id, client_secret)
headers = {'Authorization': 'Bearer ' + token['accessToken'], 'Content-Type': 'application/json'}
params = {'api-version': '2017-05-10'}
url = 'https://management.azure.com/subscriptions/<subscription_id>/resourcegroups/mytestrg'

data = {'location': 'northeurope'}

r = requests.put(url, data=json.dumps(data), headers=headers, params=params)

print(json.dumps(r.json(), indent=4, separators=(',', ': ')))

Note change in params and url, adding data and method change to requests.put. The response should look like this:

{
    "id": "/subscriptions/<subscription_id>/resourceGroups/mytestrg",
    "name": "mytestrg",
    "location": "northeurope",
    "properties": {
        "provisioningState": "Succeeded"
    }
}