Skip to content

Deploy AIDR Container on Azure

These installation steps show you how to install locally into a Minikube cluster. Instructions are also valid for Kubernetes provided by Docker Desktop and a full-fledged Kubernetes cluster. Running locally with Minikube introduces extra latency due to the prompt injection classification model. Latency is not present in a full-fledged Kubernetes cluster.

Before deploying Model Scanner, make sure you understand:

Select OS

Select your operating system and deployment type to view installation instructions.

A Hybrid deployment sends metadata to the HiddenLayer AISec Platform. See Hybrid and Disconnected Deployments for information about each deployment type.

  1. Open a terminal and create an environment variable file with Quay credentials. Use the following examples. Replace %QUAY_USERNAME% and %QUAY_PASSWORD% with actual values.

    cat << EOF > env
HIDDENLAYER_USERNAME=%QUAY_USERNAME%
HIDDENLAYER_PASSWORD=%QUAY_PASSWORD%
EOF

    Special Characters

    Special characters will require wrapping the value in a string.

  2. Load the environment variables into your shell.

    source ./env

  3. Create a file named .env.local.

    • Use the comments in the example to find configurations for your deployment needs.

    • File changes to note:

      • For the EU region, uncomment hl_region=eu.
      • Replace <license> with your AIDR license.
      • Replace <client id> and <client secret> with your HiddenLayer API key and secret.
      • Replace <azure tenant id>, <azure client id>, <azure client secret>, and <azure region> with information for your AWS instance.
    cat << EOF > .env.local
namespace:
  name=aidr-genai

image:
  ## By default, the image below is deployed; this image is for CPU deployments
  repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai
  ## To set up a GPU deployment, uncomment the line below to use the CUDA image
  # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda
  ## By default, the latest version is installed
  ## Uncomment the following to specify a specific version
  # tag=25.5.2

## By default, hybrid connection will connect to the 'US' region
## Uncomment the following option to change to a different supported region  
# hl_region=eu

resources:
  ## For CPU deployments, 8 vCPUs are recommended
  ## For GPU deployments, 4 vCPUs are recommended
  requests:
    cpu=8
  ## For GPU deployments, uncomment this to allocate 1 GPU per instance
  # limits:
    # nvidia.com/gpu=1

## Set number of instances for deployment
replicas:
  min=1
  max=1

## Specify a specific node group if necessary
## For instance, GPU deployments may require specific nodes to access a GPU
# nodeGroup=<nodegroup_name>

config:
  HL_LICENSE=<license>
  HL_LLM_PROXY_AZURE_TENANT_ID=<azure tenant id>
  HL_LLM_PROXY_AZURE_CLIENT_ID=<azure client id>
  HL_LLM_PROXY_AZURE_CLIENT_SECRET=<azure client secret>
  HL_LLM_PROXY_AZURE_REGION=<azure region>

  ## This must match the number of vCPUs assigned above
  OMP_NUM_THREADS=8

## By default, the hybrid connection type is used and requires a client id and secret
  HL_LLM_PROXY_CLIENT_ID=<client_id>
  HL_LLM_PROXY_CLIENT_SECRET=<client_secret>

  ## Uncomment the following to disable the connection to the AISec Platform
  ## in this mode, the client id and secret are not required
  # HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled
EOF

    Policy Configuration Environment Variables

    For other policy configuration environment variables, see AIDR GenAI Configuration.

  4. Make sure Docker Desktop is running.

  5. Run the following command to login to the HiddenLayer Quay repository.

    docker login --username=$HIDDENLAYER_USERNAME --password=$HIDDENLAYER_PASSWORD quay.io

  6. Run the AIDR container.

    docker run --platform linux/amd64 --env-file  .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest

A Disconnected deployment doesn't send data to the HiddenLayer AISec Platform. See Hybrid and Disconnected Deployments for information about each deployment type.

  1. Open a terminal and create an environment variable file with Quay credentials. Use the following examples. Replace %QUAY_USERNAME% and %QUAY_PASSWORD% with actual values.

    cat << EOF > env
HIDDENLAYER_USERNAME=%QUAY_USERNAME%
HIDDENLAYER_PASSWORD=%QUAY_PASSWORD%
EOF

    Special Characters

    Special characters will require wrapping the value in a string.

  2. Load the environment variables into your shell.

    source ./env

  3. Create a file named .env.local.

    • Use the comments in the example to find configurations for your deployment needs.

    • File changes to note:

      • For the EU region, uncomment hl_region=eu.
      • Replace <license> with your AIDR license.
      • Replace <azure tenant id>, <azure client id>, <azure client secret>, and <azure region> with information for your AWS instance.
    cat << EOF > .env.local
namespace:
  name=aidr-genai

image:
  ## By default, the image below is deployed; this image is for CPU deployments
  repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai
  ## To set up a GPU deployment, uncomment the line below to use the CUDA image
  # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda
  ## By default, the latest version is installed
  ## Uncomment the following to specify a specific version
  # tag=25.5.2

## By default, hybrid connection will connect to the 'US' region
## Uncomment the following option to change to a different supported region  
# hl_region=eu

resources:
  ## For CPU deployments, 8 vCPUs are recommended
  ## For GPU deployments, 4 vCPUs are recommended
  requests:
    cpu=8
  ## For GPU deployments, uncomment this to allocate 1 GPU per instance
  # limits:
    # nvidia.com/gpu=1

## Set number of instances for deployment
replicas:
  min=1
  max=1

## Specify a specific node group if necessary
## For instance, GPU deployments may require specific nodes to access a GPU
# nodeGroup=<nodegroup_name>

config:
  HL_LICENSE=<license>
  HL_LLM_PROXY_AZURE_TENANT_ID=<azure tenant id>
  HL_LLM_PROXY_AZURE_CLIENT_ID=<azure client id>
  HL_LLM_PROXY_AZURE_CLIENT_SECRET=<azure client secret>
  HL_LLM_PROXY_AZURE_REGION=<azure region>

  ## This must match the number of vCPUs assigned above
  OMP_NUM_THREADS=8

## By default, the hybrid connection type is used and requires a client id and secret
  # HL_LLM_PROXY_CLIENT_ID=<client_id>
  # HL_LLM_PROXY_CLIENT_SECRET=<client_secret>

  ## Uncomment the following to disable the connection to the AISec Platform
  ## in this mode, the client id and secret are not required
  HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled
EOF

    Policy Configuration Environment Variables

    For other policy configuration environment variables, see AIDR GenAI Configuration.

  4. Make sure Docker Desktop is running.

  5. Run the following command to login to the HiddenLayer Quay repository.

    docker login --username=$HIDDENLAYER_USERNAME --password=$HIDDENLAYER_PASSWORD quay.io

  6. Run the AIDR container.

    docker run --platform linux/amd64 --env-file  .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest

A Hybrid deployment sends metadata to the HiddenLayer AISec Platform. See Hybrid and Disconnected Deployments for information about each deployment type.

  1. Create an environment variable file with Quay credentials. Manually create an env.bat file with the following content. Replace %QUAY_USERNAME% and %QUAY_PASSWORD% with actual values.

    @echo off
set HIDDENLAYER_USERNAME=%QUAY_USERNAME%
set HIDDENLAYER_PASSWORD=%QUAY_PASSWORD%

    Special Characters

    Special characters will require wrapping the value in a string.

  2. Open a command prompt and load the environment variables into your shell.

    env

  3. Create a file named .env.local.

    • Use the comments in the example to find configurations for your deployment needs.

    • File changes to note:

      • For the EU region, uncomment hl_region=eu.
      • Replace <license> with your AIDR license.
      • Replace <client id> and <client secret> with your HiddenLayer API key and secret.
      • Replace <azure tenant id>, <azure client id>, <azure client secret>, and <azure region> with information for your AWS instance.
    cat << EOF > .env.local
namespace:
  name=aidr-genai

image:
  ## By default, the image below is deployed; this image is for CPU deployments
  repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai
  ## To set up a GPU deployment, uncomment the line below to use the CUDA image
  # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda
  ## By default, the latest version is installed
  ## Uncomment the following to specify a specific version
  # tag=25.5.2

## By default, hybrid connection will connect to the 'US' region
## Uncomment the following option to change to a different supported region  
# hl_region=eu

resources:
  ## For CPU deployments, 8 vCPUs are recommended
  ## For GPU deployments, 4 vCPUs are recommended
  requests:
    cpu=8
  ## For GPU deployments, uncomment this to allocate 1 GPU per instance
  # limits:
    # nvidia.com/gpu=1

## Set number of instances for deployment
replicas:
  min=1
  max=1

## Specify a specific node group if necessary
## For instance, GPU deployments may require specific nodes to access a GPU
# nodeGroup=<nodegroup_name>

config:
  HL_LICENSE=<license>
  HL_LLM_PROXY_AZURE_TENANT_ID=<azure tenant id>
  HL_LLM_PROXY_AZURE_CLIENT_ID=<azure client id>
  HL_LLM_PROXY_AZURE_CLIENT_SECRET=<azure client secret>
  HL_LLM_PROXY_AZURE_REGION=<azure region>

  ## This must match the number of vCPUs assigned above
  OMP_NUM_THREADS=8

## By default, the hybrid connection type is used and requires a client id and secret
  HL_LLM_PROXY_CLIENT_ID=<client_id>
  HL_LLM_PROXY_CLIENT_SECRET=<client_secret>

  ## Uncomment the following to disable the connection to the AISec Platform
  ## in this mode, the client id and secret are not required
  # HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled
EOF

    Policy Configuration Environment Variables

    For other policy configuration environment variables, see AIDR GenAI Configuration.

  4. Make sure Docker Desktop and the Kubernetes cluster are running.

  5. Run the following commands to login to the HiddenLayer Quay repository.

    docker login --username=%HIDDENLAYER_USERNAME% --password=%HIDDENLAYER_PASSWORD% quay.io

  6. Run the AIDR container.

    docker run --platform linux/amd64 --env-file  .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest

A Disconnected deployment doesn't send data to the HiddenLayer AISec Platform. See Hybrid and Disconnected Deployments for information about each deployment type.

  1. Create an environment variable file with Quay credentials. Manually create an env.bat file with the following content. Replace %QUAY_USERNAME% and %QUAY_PASSWORD% with actual values.

    @echo off
set HIDDENLAYER_USERNAME=%QUAY_USERNAME%
set HIDDENLAYER_PASSWORD=%QUAY_PASSWORD%

    Special Characters

    Special characters will require wrapping the value in a string.

  2. Open a command prompt and load the environment variables into your shell.

    env

  3. Create a file named .env.local.

    • Use the comments in the example to find configurations for your deployment needs.

    • File changes to note:

      • For the EU region, uncomment hl_region=eu.
      • Replace <license> with your AIDR license.
      • Replace <azure tenant id>, <azure client id>, <azure client secret>, and <azure region> with information for your AWS instance.
    cat << EOF > .env.local
namespace:
  name=aidr-genai

image:
  ## By default, the image below is deployed; this image is for CPU deployments
  repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai
  ## To set up a GPU deployment, uncomment the line below to use the CUDA image
  # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda
  ## By default, the latest version is installed
  ## Uncomment the following to specify a specific version
  # tag=25.5.2

## By default, hybrid connection will connect to the 'US' region
## Uncomment the following option to change to a different supported region  
# hl_region=eu

resources:
  ## For CPU deployments, 8 vCPUs are recommended
  ## For GPU deployments, 4 vCPUs are recommended
  requests:
    cpu=8
  ## For GPU deployments, uncomment this to allocate 1 GPU per instance
  # limits:
    # nvidia.com/gpu=1

## Set number of instances for deployment
replicas:
  min=1
  max=1

## Specify a specific node group if necessary
## For instance, GPU deployments may require specific nodes to access a GPU
# nodeGroup=<nodegroup_name>

config:
  HL_LICENSE=<license>
  HL_LLM_PROXY_AZURE_TENANT_ID=<azure tenant id>
  HL_LLM_PROXY_AZURE_CLIENT_ID=<azure client id>
  HL_LLM_PROXY_AZURE_CLIENT_SECRET=<azure client secret>
  HL_LLM_PROXY_AZURE_REGION=<azure region>

  ## This must match the number of vCPUs assigned above
  OMP_NUM_THREADS=8

## By default, the hybrid connection type is used and requires a client id and secret
  # HL_LLM_PROXY_CLIENT_ID=<client_id>
  # HL_LLM_PROXY_CLIENT_SECRET=<client_secret>

  ## Uncomment the following to disable the connection to the AISec Platform
  ## in this mode, the client id and secret are not required
  HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled
EOF

    Policy Configuration Environment Variables

    For other policy configuration environment variables, see AIDR GenAI Configuration.

  4. Make sure Docker Desktop and the Kubernetes cluster are running.

  5. Run the following commands to login to the HiddenLayer Quay repository.

    docker login --username=%HIDDENLAYER_USERNAME% --password=%HIDDENLAYER_PASSWORD% quay.io

  6. Run the AIDR container.

    docker run --platform linux/amd64 --env-file  .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest

A Hybrid deployment sends metadata to the HiddenLayer AISec Platform. See Hybrid and Disconnected Deployments for information about each deployment type.

  1. Open a terminal and create an environment variable file with Quay credentials. Use the following examples. Replace %QUAY_USERNAME% and %QUAY_PASSWORD% with actual values.

    cat << EOF > env
HIDDENLAYER_USERNAME=%QUAY_USERNAME%
HIDDENLAYER_PASSWORD=%QUAY_PASSWORD%
EOF

    Special Characters

    Special characters will require wrapping the value in a string.

  2. Load the environment variables into your shell.

    source ./env

  3. Create a file named .env.local.

    • Use the comments in the example to find configurations for your deployment needs.

    • File changes to note:

      • For the EU region, uncomment hl_region=eu.
      • Replace <license> with your AIDR license.
      • Replace <client id> and <client secret> with your HiddenLayer API key and secret.
      • Replace <azure tenant id>, <azure client id>, <azure client secret>, and <azure region> with information for your AWS instance.
    cat << EOF > .env.local
namespace:
  name=aidr-genai

image:
  ## By default, the image below is deployed; this image is for CPU deployments
  repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai
  ## To set up a GPU deployment, uncomment the line below to use the CUDA image
  # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda
  ## By default, the latest version is installed
  ## Uncomment the following to specify a specific version
  # tag=25.5.2

## By default, hybrid connection will connect to the 'US' region
## Uncomment the following option to change to a different supported region  
# hl_region=eu

resources:
  ## For CPU deployments, 8 vCPUs are recommended
  ## For GPU deployments, 4 vCPUs are recommended
  requests:
    cpu=8
  ## For GPU deployments, uncomment this to allocate 1 GPU per instance
  # limits:
    # nvidia.com/gpu=1

## Set number of instances for deployment
replicas:
  min=1
  max=1

## Specify a specific node group if necessary
## For instance, GPU deployments may require specific nodes to access a GPU
# nodeGroup=<nodegroup_name>

config:
  HL_LICENSE=<license>
  HL_LLM_PROXY_AZURE_TENANT_ID=<azure tenant id>
  HL_LLM_PROXY_AZURE_CLIENT_ID=<azure client id>
  HL_LLM_PROXY_AZURE_CLIENT_SECRET=<azure client secret>
  HL_LLM_PROXY_AZURE_REGION=<azure region>

  ## This must match the number of vCPUs assigned above
  OMP_NUM_THREADS=8

## By default, the hybrid connection type is used and requires a client id and secret
  HL_LLM_PROXY_CLIENT_ID=<client_id>
  HL_LLM_PROXY_CLIENT_SECRET=<client_secret>

  ## Uncomment the following to disable the connection to the AISec Platform
  ## in this mode, the client id and secret are not required
  # HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled
EOF

    Policy Configuration Environment Variables

    For other policy configuration environment variables, see AIDR GenAI Configuration.

  4. Make sure Docker Desktop is running.

  5. Run the following command to login to the HiddenLayer Quay repository.

    docker login --username=$HIDDENLAYER_USERNAME --password=$HIDDENLAYER_PASSWORD quay.io

  6. Run the AIDR container.

    docker run --platform linux/amd64 --env-file  .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest

A Disconnected deployment doesn't send data to the HiddenLayer AISec Platform. See Hybrid and Disconnected Deployments for information about each deployment type.

  1. Open a terminal and create an environment variable file with Quay credentials. Use the following examples. Replace %QUAY_USERNAME% and %QUAY_PASSWORD% with actual values.

    cat << EOF > env
HIDDENLAYER_USERNAME=%QUAY_USERNAME%
HIDDENLAYER_PASSWORD=%QUAY_PASSWORD%
EOF

    Special Characters

    Special characters will require wrapping the value in a string.

  2. Load the environment variables into your shell.

    source ./env

  3. Create a file named .env.local.

    • Use the comments in the example to find configurations for your deployment needs.

    • File changes to note:

      • For the EU region, uncomment hl_region=eu.
      • Replace <license> with your AIDR license.
      • Replace <azure tenant id>, <azure client id>, <azure client secret>, and <azure region> with information for your AWS instance.
    cat << EOF > .env.local
namespace:
  name=aidr-genai

image:
  ## By default, the image below is deployed; this image is for CPU deployments
  repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai
  ## To set up a GPU deployment, uncomment the line below to use the CUDA image
  # repository=quay.io/hiddenlayer/distro-enterprise-aidr-genai-cuda
  ## By default, the latest version is installed
  ## Uncomment the following to specify a specific version
  # tag=25.5.2

## By default, hybrid connection will connect to the 'US' region
## Uncomment the following option to change to a different supported region  
# hl_region=eu

resources:
  ## For CPU deployments, 8 vCPUs are recommended
  ## For GPU deployments, 4 vCPUs are recommended
  requests:
    cpu=8
  ## For GPU deployments, uncomment this to allocate 1 GPU per instance
  # limits:
    # nvidia.com/gpu=1

## Set number of instances for deployment
replicas:
  min=1
  max=1

## Specify a specific node group if necessary
## For instance, GPU deployments may require specific nodes to access a GPU
# nodeGroup=<nodegroup_name>

config:
  HL_LICENSE=<license>
  HL_LLM_PROXY_AZURE_TENANT_ID=<azure tenant id>
  HL_LLM_PROXY_AZURE_CLIENT_ID=<azure client id>
  HL_LLM_PROXY_AZURE_CLIENT_SECRET=<azure client secret>
  HL_LLM_PROXY_AZURE_REGION=<azure region>

  ## This must match the number of vCPUs assigned above
  OMP_NUM_THREADS=8

## By default, the hybrid connection type is used and requires a client id and secret
  # HL_LLM_PROXY_CLIENT_ID=<client_id>
  # HL_LLM_PROXY_CLIENT_SECRET=<client_secret>

  ## Uncomment the following to disable the connection to the AISec Platform
  ## in this mode, the client id and secret are not required
  HL_LLM_PROXY_MLDR_CONNECTION_TYPE=disabled
EOF

    Policy Configuration Environment Variables

    For other policy configuration environment variables, see AIDR GenAI Configuration.

  4. Make sure Docker Desktop is running.

  5. Run the following command to login to the HiddenLayer Quay repository.

    docker login --username=$HIDDENLAYER_USERNAME --password=$HIDDENLAYER_PASSWORD quay.io

  6. Run the AIDR container.

    docker run --platform linux/amd64 --env-file  .env.local -p 8000:8000 quay.io/hiddenlayer/distro-enterprise-aidr-genai:latest

Health Check

To check that the proxy is up and running, open a new terminal and use the following command to ping the health monitor.

curl http://localhost:8000/health

Generate Manifest

To write the generated manifest to a file (manifest.yml), open a terminal and run the following command.

  • You must run the AI Detection and Response Generative installer before generating a manifest.
  • Change latest to the AI Detection and Response Generative version that you use.
docker run --env-file ./env -v pwd/config:/src/config quay.io/hiddenlayer/distro-enterprise-aidr-genai-installer:latest > manifest.yml