Deploying a solution in Azure to investigate Network Anomalies

13 mins read

Learn how to deploy an NDR solution using Azure Bicep to reveal threats undetectable by other technologies.

Securing Workloads through Network Detection and Response capabilities might be overwhelming if you’re starting in the Cyber security space.

Here’s a capture of the initial findings in a web application:

Anomaly Detection

How did we get here? First, let’s be clear with terminology. Network detection and response (NDR) refers to a progressive security solution for obtaining complete visibility to known and unknown threats across your network.

This article aims to share with you my personal experience deploying an NDR solution in the Azure Cloud and the configuration needed to detect threats and respond.

I will share how I deployed a solution in Azure to detect threats and anomalies for an application running in an on-premises data center.

Architecture reference

The diagram below shows a high-level overview of the environment.

Pre-requisites:

An active Azure subscriptionA user with owner/contributor rolesAccess to the local appliance and be able to send network telemetry (NetFlow or IPFIX)

Here are the steps to follow:

Deploy the Kemp Flowmon Network Virtual Appliance(NVA) in Azure using BicepLicense and configure the NVAConfigure the local device to send Netflow/IPFIX to Kemp Flowmon in AzureConfigure the Kemp Flowmon NVA for anomaly detection

1. Deploy the Kemp Flowmon Network Virtual Appliance(NVA) in Azure using Bicep

The first step is to prepare the NDR environment. We will use the Kemp Flowmon virtual machine in Azure using Infrastructure-As-Code using Azure Bicepto to simplify the deployment process.

You can refer to this article that shows in detail how you can deploy the Kemp Flowmon appliance in Azure using Bicep. Note you need to create the SSH keys and ensure you use the username ‘flowmon’.

Here’s the complete Bicep template to deploy the Kemp Flowmon NVA in Azure.

// The name of your Virtual Machine.
param vmName string = ‘flowmonVm’// Username for the Virtual Machine.
param adminUsername string// Type of authentication to use on the Virtual Machine. SSH key is recommended.
param authenticationType string = ‘sshPublicKey’// SSH Key or password for the Virtual Machine. SSH key is recommended.
@secure()
param adminPasswordOrKey string// Unique DNS Name for the Public IP used to access the Virtual Machine.
param dnsLabelPrefix string = toLower(‘flowmon-${uniqueString(resourceGroup().id)}’)// Location for all resources.
param location string = resourceGroup().location// The size of the VM.
param vmSize string = ‘Standard_B4ms’// Name of the VNET.
param virtualNetworkName string = ‘vNet’// Name of the subnet in the virtual network.
param subnetName string = ‘Subnet’// Name of the Network Security Group.
param networkSecurityGroupName string = ‘SecGroupNet’var publicIPAddressName = ‘${vmName}PublicIP’
var networkInterfaceName = ‘${vmName}Nic’
var subnetRef = ‘${vnet.id}/subnets/${subnetName}’
var osDiskType = ‘Standard_LRS’
var subnetAddressPrefix = ‘10.5.0.0/24’
var addressPrefix = ‘10.5.0.0/16’
var linuxConfiguration = {
disablePasswordAuthentication: true
ssh: {
publicKeys: [
{
path: ‘/home/${adminUsername}/.ssh/authorized_keys’
keyData: adminPasswordOrKey
}
]
}
}resource nic ‘Microsoft.Network/networkInterfaces@2020-06-01’ = {
name: networkInterfaceName
location: location
properties: {
ipConfigurations: [
{
name: ‘ipconfig1’
properties: {
subnet: {
id: subnetRef
}
privateIPAllocationMethod: ‘Dynamic’
publicIPAddress: {
id: publicIP.id
}
}
}
]
networkSecurityGroup: {
id: nsg.id
}
}
}resource nsg ‘Microsoft.Network/networkSecurityGroups@2020-06-01’ = {
name: networkSecurityGroupName
location: location
properties: {
securityRules: [
{
name: ‘SSH’
properties : {
protocol : ‘Tcp’
sourcePortRange : ‘*’
destinationPortRange : ’22’
sourceAddressPrefix : ‘*’
destinationAddressPrefix: ‘*’
access: ‘Allow’
priority : 1010
direction : ‘Inbound’
sourcePortRanges : []
destinationPortRanges : []
sourceAddressPrefixes : []
destinationAddressPrefixes : []
}
}
{
name : ‘HTTPS’
properties : {
protocol : ‘Tcp’
sourcePortRange : ‘*’
destinationPortRange : ‘443’
sourceAddressPrefix : ‘*’
destinationAddressPrefix : ‘*’
access : ‘Allow’
priority : 1020
direction : ‘Inbound’
sourcePortRanges : []
destinationPortRanges : []
sourceAddressPrefixes : []
destinationAddressPrefixes : []
}
}
{
name : ‘Collector’
properties : {
protocol : ‘Udp’
sourcePortRange : ‘*’
destinationPortRange : ‘3000’
sourceAddressPrefix : ‘*’
destinationAddressPrefix : ‘*’
access : ‘Allow’
priority : 103
direction : ‘Inbound’
sourcePortRanges : []
destinationPortRanges : []
sourceAddressPrefixes : []
destinationAddressPrefixes : []
}
}
{
name : ‘ALL’
properties : {
protocol : ‘*’
sourcePortRange : ‘*’
destinationPortRange : ‘*’
sourceAddressPrefix : ‘*’
destinationAddressPrefix : ‘*’
access : ‘Allow’
priority : 1040
direction : ‘Inbound’
sourcePortRanges : []
destinationPortRanges : []
sourceAddressPrefixes : []
destinationAddressPrefixes : []
}
}
]
}
}resource vnet ‘Microsoft.Network/virtualNetworks@2020-06-01’ = {
name: virtualNetworkName
location: location
properties: {
addressSpace: {
addressPrefixes: [
addressPrefix
]
}
subnets: [
{
name: subnetName
properties: {
addressPrefix: subnetAddressPrefix
privateEndpointNetworkPolicies: ‘Enabled’
privateLinkServiceNetworkPolicies: ‘Enabled’
}
}
]
}
}resource publicIP ‘Microsoft.Network/publicIPAddresses@2020-06-01’ = {
name: publicIPAddressName
location: location
properties: {
publicIPAllocationMethod: ‘Dynamic’
publicIPAddressVersion: ‘IPv4’
dnsSettings: {
domainNameLabel: dnsLabelPrefix
}
idleTimeoutInMinutes: 4
}
sku: {
name: ‘Basic’
}
}resource vm ‘Microsoft.Compute/virtualMachines@2021-03-01’ = {
name: vmName
location: location
properties: {
hardwareProfile: {
vmSize: vmSize
}
storageProfile: {
osDisk: {
createOption: ‘FromImage’
managedDisk: {
storageAccountType: osDiskType
}
}
imageReference: {
publisher: ‘flowmon’
offer: ‘flowmon_collector’
sku: ‘v1101-byol’
version: ‘latest’
}
}
networkProfile: {
networkInterfaces: [
{
id: nic.id
}
]
}
osProfile: {
computerName: vmName
adminUsername: adminUsername
adminPassword: adminPasswordOrKey
linuxConfiguration: any(authenticationType == ‘password’ ? null : linuxConfiguration)
}
diagnosticsProfile: {
bootDiagnostics: {
enabled: true
}
}
}
plan: {
name: ‘v1101-byol’
publisher: ‘flowmon’
product: ‘flowmon_collector’
}
}output administratorUsername string = adminUsername

Once you deploy this virtual machine in Azure, the next step is to apply for a license, and you can get a license from the following URL: https://www.flowmon.com/en/download-free-trial

2. License and configure the Kemp Flowmon NVA

The license is a file that we need to upload in the ‘Configuration Center’ module. You can go to Configuration Center > License > Upload License as shown below:

Kemp Flowmon- upload license

Once you have the license enabled, we will ensure that the NVA can receive traffic from different devices.

We will send network telemetry from the local Kemp LoadMaster to the Kemp Flowmon in Azure. We are talking about the ability to transmit network telemetry from the on-premises device to the Kemp Flowmon in Azure.

The Kemp Flowmon appliance is Flow-based, which means it can receive NetFlow, sFLow, jFlow, NetStram, IPFIX, etc.. from different sources like Firewalls, Core Switches, Load Balancers. Here’s a complete list of the compatible devices:

https://medium.com/media/83d2a0c0648d90be5ee77d7995db8594/href

To ensure that the NVA can receive traffic, we have to verify the Security Rules or NSGs.

The Bicep template contains the NSGs needed to enable communication. The below image shows the inbound rules enabled for the NVA:

Kemp Flowmon — Inbound rules

We have configured the basic setup for the Kemp Flowmon in Azure.

The next step is to configure the local device to send Netflow/IPFIX to Kemp Flowmon in Azure.

3. Configure the local device to send Netflow/IPFIX to Kemp Flowmon in Azure

The Kemp Flowmon appliance is Flow-based, which means it can receive NetFlow v5 /v9, sFLow, jFlow, NetStream, IPFIX, etc. from multiple devices including Alcatel, Brocade, Cisco, HP, Huawei, Juniper, VMware, IXIA, Gigamon, Checkpoint, Palo Alto, SonicWall, Fortinet, and others.

In this specific environment, we have a Kemp Loadmaster in the local datacenter, so we were able to enable network telemetry on that appliance and send it to the Kemp Flowmon Collector in Azure.

To send network telemetry, we need to configure the local device to send IPFIX or any other supported Flow type as shown below:

Enable Network Telemetry

Once this option is enabled, we will see all the network telemetry in the Kemp Flowmon NVA in Azure in a few minutes.

Note: In the Kemp LoadMaster v7.2.55, after enabling network telemetry, I noticed a 15% increase in resource consumption.

The next step is to configure the Kemp Flowmon NVA for anomaly detection.

4. Configure the Kemp Flowmon NVA for anomaly detection

Kemp Flowmon will detect the new source in the ‘Monitoring Center’. From there, we can create a ‘Profile’ and filter what we are interested in monitoring from that specific source.

The image below shows the ‘Monitoring Center’ and some of the profiles created:

Kemp Flowmon — Monitoring Center

Now that Kemp Flowmon detected the source to monitor, we will proceed to configure the anomaly detection. We will work on the configuration for the module called ‘Anomaly Detection System’. This module is included in the trial license.

This ADS module is a security solution that uses artificial intelligence and machine learning to detect anomalies hidden in the network traffic.

In the ADS module, there is a configuration wizard that you can use for the initial setup:

Kemp Flowmon — Anomaly Detection System

In this configuration wizard, we provide the parameters for the configuration of the network. We define the servers, and IP Address ranges of existing services: SMTP Servers, DNS servers, DHCP, local servers, etc.

Once you complete the process of providing all the network parameters, you will be able to start the ADS module and see your data feed and the active methods applied to the data feed as shown below:

Kemp Anomaly Detection System — Methods

Now we can perform our analysis.

In the left menu, there’s an ‘Analysis’ option; this will take you to a page where you can explore all the events related to a data feed.

The image below shows the analysis of the Kemp LoadMaster in a period of 24 hours:

Kemp Flowmon Anomaly Detection System — Analysis

Note we selected to perform the analysis for the last 24 hours and filter by security issues on the Kemp LoadMaster data feed.

Then, ADS will provide a list with the latest events sorted by priority. We can see there’s a high number of events overall, and we can expand on each event to get more evidence about it, as shown below.

Kemp Flowmon Anomaly Detection System — Event detail

From this view, you can also drill down into the details and validate if this is a false positive or a threat:

Kemp Flowmon Anomaly Detection System — Event detail

While most companies rely on perimeter security and endpoint protection, detailed awareness of network behavior and a proactive fight against cyber threats is critical in today’s hybrid environments.

You probably noticed already that this solution is not invasive as we don’t have to perform additional configurations on the server side. This solution uses a serverless-agent approach; therefore, we don’t need any agents installed on the network devices or servers.

Join the AzInsider email list here.

-Dave R.

💪Deploying a solution in Azure to investigate Network Anomalies was originally published in CodeX on Medium, where people are continuing the conversation by highlighting and responding to this story.

Leave a Reply

Your email address will not be published.

Follow Us