Destroyed 10. Setting the solution up in AKS (Azure Kubernetes Service) (markdown)
Cesar De la Torre
parent
40c1b376cc
commit
4390f2bf37
@ -1,308 +0,0 @@
|
||||
# Introduction
|
||||
eShopOnContainers supports deployment to Kubernetes clusters for "simulated" production scenarios where you need to scale-out your containers.
|
||||
Currently, eShopOnContainers supports [AKS (Azure Kubernetes Service)](https://docs.microsoft.com/en-us/azure/aks/intro-kubernetes) and [ACS-Kuberentes (Azure Container Service for Kuberentes)](https://docs.microsoft.com/en-us/azure/container-service/kubernetes/container-service-intro-kubernetes), but we recommend to use AKS since ACS will be deprecated in the future favoring AKS which offers further PaaS features for the cluster's infrastructure, automatic updates/patch on the VMs, etc.
|
||||
|
||||
Common info for eShopOnContainers on Kuberentes (including both environments, AKS and ACS) is available [here](https://github.com/dotnet-architecture/eShopOnContainers/blob/dev/k8s/README.k8s.md)
|
||||
|
||||
However, this post focuses specifically on AKS (Azure Kubernetes Service) deployment of eShopOnContainers, with further details.
|
||||
|
||||
# Prerequisites
|
||||
|
||||
### A Docker development environment with `docker` and `docker-compose`.
|
||||
|
||||
Visit [docker.com](https://docker.com) to download the tools and set up the environment. Docker's [installation guide](https://docs.docker.com/engine/getstarted/step_one/#step-3-verify-your-installation) covers verifying your Docker installation.
|
||||
|
||||
### Azure cli 2.0 or later
|
||||
Azure cli 2.0 or later must be installed in your dev machine [installation guide](https://docs.microsoft.com/en-us/cli/azure/install-azure-cli).
|
||||
|
||||
### The Kubernetes command line client, `kubectl`.
|
||||
`kubectl` can be installed with the `az` tool as described in the AKS [walkthrough](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough#connect-to-the-cluster). `az` is also helpful for getting the credentials `kubectl` needs to access your cluster. For other installation options, and information about configuring `kubectl` yourself, see the [Kubernetes documentation](https://kubernetes.io/docs/tasks/kubectl/install/).
|
||||
|
||||
## Option A: Manually create the infrastructure in Azure
|
||||
|
||||
### A Kubernetes cluster in AKS.
|
||||
You can follow any of the following approaches in order to create a Kubernetes cluster in AKS:
|
||||
|
||||
* A. Create an AKS cluster with Azure portal: [Walkthrough](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough-portal)
|
||||
|
||||
* B. Create an AKS cluster with Azure CLI: [Walkthrough](https://docs.microsoft.com/en-us/azure/aks/kubernetes-walkthrough)
|
||||
|
||||
### A private Azure Container Registry (ACR)
|
||||
|
||||
[ACR](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-intro) is a managed Docker registry service based on the open-source Docker Registry 2.0. With ACR you create and maintain Azure container registries to store and manage your private Docker container images.
|
||||
|
||||
It is important to have a Docker registry, holding your private images, located in the same Datacenter/area than your orchestrator cluster, so when pulling images it'll go faster than if you use a remote Docker registry like Docker Hub. In addition, ACR offers significant value-added like global availability and global replication, etc.
|
||||
You can follow any of the following approaches:
|
||||
|
||||
* A. Create an ACR registry with Azure portal: [Walkthrough](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-get-started-portal)
|
||||
|
||||
* B. Create an ACR registry with Azure CLI: [Walkthrough](https://docs.microsoft.com/en-us/azure/container-registry/container-registry-get-started-azure-cli)
|
||||
|
||||
## Option B: Automatically create all the infrastructure in Azure with a ready-to-go script
|
||||
|
||||
Create ALL the Azure infrastructure with an eShopOnContainer's ready-to-go script.
|
||||
All previous steps related to Azure infrastructure can be skipped if you run the **gen-k8s-env-aks.ps1** script to automatically create the Azure environment needed for the Kubernetes deployment. That includes the AKS cluster and optionally the ARC container registry.
|
||||
|
||||
For example just by running the following cli script it would create to create the AKS cluster.
|
||||
|
||||
>```
|
||||
>./gen-k8s-env-aks -resourceGroupName YoureShopAksResgroup -location centralus -serviceName YoureShopAksCluster -dnsNamePrefix youreshopaks -registryName YoureShopAcrRegistry -createAcr true -nodeCount 3 -nodeVMSize Standard_D2_v2
|
||||
>```
|
||||
|
||||
**Important**: Note the parameter "-createAcr true". If you are creating the K8s cluster but you want to re-use and existing ACR, say "-createAcr false".
|
||||
|
||||
### Step-by-step procedure
|
||||
|
||||
* Authenticate in Azure using the CLI
|
||||
Before running the eShopOnContainers **gen-k8s-env-aks.ps1** script or manually creating the clusters, you first need to authenticate from a PowerShell window, with:
|
||||
`az login`
|
||||
Follow the procedure.
|
||||
|
||||
* Select the right Azure subscription to use
|
||||
It is very common to have access to multiple Azure subscriptions.
|
||||
You can list your available subscriptions with:
|
||||
|
||||
`az account list -o table`
|
||||
|
||||
And check which subscription is being used by default.
|
||||
If you need to change it, do it with this command:
|
||||
|
||||
`az account set --subscription "YOUR-SUBSCRIPTION-NAME"`
|
||||
|
||||
* Run the script to create the Kubernetes cluster (and optionally the ACR registry) in Azure
|
||||
Open PowerShell and position in the folder where the script is placed:
|
||||
|
||||
`cd <YOUR-PATH>\eShopOnContainers\k8s`
|
||||
|
||||
Run the following script:
|
||||
|
||||
>```
|
||||
>./gen-k8s-env-aks -resourceGroupName YoureShopAksResgroup -location centralus -serviceName YoureShopAksCluster -dnsNamePrefix youreshopaks -registryName YoureShopAcrRegistry -createAcr true -nodeCount 3 -nodeVMSize Standard_D2_v2
|
||||
>```
|
||||
|
||||
You should get a similar execution in PowerShell, as the following:
|
||||
|
||||
|
||||
|
||||
After a successful execution you can check the AKS Kubernetes cluster in Azure portal, like in the following screenshot:
|
||||
|
||||
|
||||
|
||||
|
||||
# Check your Kubernetes Cluster Dashboard
|
||||
|
||||
To open your Kubernetes dashboard, complete the following steps:
|
||||
|
||||
* Open Azure CLI version 2.0.27 or later in PowerShell or CommandLine window. This will not work in cloud shell and must be running on your local machine.
|
||||
|
||||
* If you do not already have `kubectl` installed in your CLI, run the following command:
|
||||
NOTE: Make sure you run with admin privileges so you don't get a "Permission denied: C:\\Program Files (x86)\\kubectl.exe".
|
||||
|
||||
>```
|
||||
>az aks install-cli
|
||||
>```
|
||||
|
||||
|
||||
|
||||
* Get the credentials for your cluster by running the following command:
|
||||
|
||||
>```
|
||||
>az aks get-credentials --resource-group YoureShopAksResgroup --name YoureShopAksCluster
|
||||
>```
|
||||
|
||||
|
||||
|
||||
* Open the Kubernetes dashboard by running the following command:
|
||||
|
||||
>```
|
||||
>az aks browse --resource-group YoureShopAksResgroup --name YoureShopAksCluster
|
||||
>```
|
||||
|
||||
This command runs the Kubernetes proxy (so you can access the remote dashboard through the http://127.0.0.1:8001/ui address ) while it opens the dashboard in a browser. You could also do those steps manually, though.
|
||||
You should see the Kubernetes dashboard, similar to the following screenshot if you click on the nodes menu option:
|
||||
|
||||
|
||||
|
||||
### Select the right Kubernetes context when having multiple clusters
|
||||
|
||||
When you have multiple Kubernetes clusters in AKS or even if you want to switch and deploy to a local Kubernetes cluster in the PC with "Docker for Windows" with Kuberentes support, you need to "point" to the right cluster before running `kubectl` commands.
|
||||
|
||||
In order to see the contexts you have in your local PC, type:
|
||||
|
||||
>```
|
||||
>kubectl config get-contexts
|
||||
>```
|
||||
You should see something like the following:
|
||||
|
||||
|
||||
|
||||
If you need to select a different context because the cluster you want to use is not the current one selected, type:
|
||||
|
||||
>```
|
||||
>kubectl config use-context YoureShopAksCluster
|
||||
>```
|
||||
|
||||
|
||||
|
||||
So when you use `kubectl`, it'll be managing that particular cluster:
|
||||
|
||||
|
||||
|
||||
|
||||
## Important: Store your credentials/secrets for Kubernetes and ACR in a safe place!
|
||||
|
||||
The script and Azure CLI create all the infrastructure very easily. However, if you want to re-use the same Kubernetes credentials or re-use the same Azure ACR registry, it is important that you store your credentials in a safe place.
|
||||
|
||||
### Kubernetes credentials (SSH RSA keys and service principal)
|
||||
|
||||
**Make sure you store the SSH RSA keys and service principal to connect in a safe place**
|
||||
If you use a new client PC/machine, you will need these keys in order to be able to connect to the same cluster.
|
||||
|
||||
**IMPORTANT**: When you created the Kubernetes cluster with the script provided by eShopOnContainers or by using the command `az aks create --resource-group myResourceGroup --name myAKSCluster --node-count 1 --generate-ssh-keys`, it generated the SSH RSA keys and service principal for the Kubernetes cluster.
|
||||
|
||||
Those keys were stored in the client PC used when you created the cluster and you definitely need those in order to connect to the cluster. If you are going to connect to the cluster from the same machine, it'll work directly because the keys are already there. But if you try to connect to the cluster from a different client machine, you'll need to copy the SSH RSA keys and the service principal to that new client machine.
|
||||
|
||||
The SSH RSA private key file and the corresponding public key files generated when you created the cluster are stored in the folder at the client machine you used.
|
||||
|
||||
|
||||
|
||||
The service principal credentials are written to the file ~/.azure/aksServicePrincipal.json on the client machine you used, as shown in the image below.
|
||||
|
||||
|
||||
|
||||
It is critical that you copy and store those files above in a secure place so you can re-used them in the future in other additional client machines.
|
||||
If you are trying to connect from a new client machine, then, copy those files into the same folder paths before trying to connect to the Kubernetes cluster.
|
||||
|
||||
For further info about SSH RSA keys, see these links:
|
||||
|
||||
https://docs.microsoft.com/en-us/azure/virtual-machines/linux/ssh-from-windows
|
||||
|
||||
https://docs.microsoft.com/en-us/azure/virtual-machines/linux/troubleshoot-ssh-connection
|
||||
|
||||
### ACR credentials
|
||||
|
||||
In order to see your ACR credentials type:
|
||||
|
||||
`az acr credential show -n YoureShopAcrRegistry`
|
||||
|
||||
|
||||
|
||||
Keep those credentials in a safe place.
|
||||
|
||||
|
||||
# Deploy the nginx-ingress and eShopOnContainers application into the Kuberentes cluster with deployment scripts
|
||||
|
||||
- Open a PowerShell command line at the `k8s` directory of your local eShopOnContainers repository.
|
||||
- Ensure `docker`, `docker-compose`, and `kubectl` are on the path, and configured for your Docker machine and Kubernetes cluster.
|
||||
|
||||
### Setup the nginx-ingress
|
||||
|
||||
Just the first time you deploy eShopOnContainers into the cluster, you need to deploy the nginx-ingress tier with two scripts.
|
||||
|
||||
None of these PowerShell scripts accept any parameter.
|
||||
The first script creates the generic nginx-ingress resources and the second script creates specific resources for using nginx-ingress under Azure AKS.
|
||||
|
||||
Run first script for ingress:
|
||||
>```
|
||||
>./deploy-ingress.ps1
|
||||
>```
|
||||
|
||||
|
||||
Run second script for ingress:
|
||||
>```
|
||||
>./deploy-ingress-azure.ps1
|
||||
>```
|
||||
|
||||
|
||||
To ensure that everything is installed you can type `kubectl get services -n ingress-nginx`
|
||||
Output should be something like:
|
||||
|
||||
```
|
||||
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
|
||||
default-http-backend ClusterIP 10.0.200.205 <none> 80/TCP 2h
|
||||
ingress-nginx LoadBalancer 10.0.52.205 xx.xx.xx.xx 80:31608/TCP,443:32518/TCP 2h
|
||||
```
|
||||
>**IMPORTANT**: The EXTERNAL-IP of the ingress-nginx takes a few minutes to appear. This will be the external IP of the cluster.
|
||||
You also need to wait for a while until the ingress tiers are created before trying to deploy the application containers. Other than that you will still get the error "Must install ingress first - Run deploy-ingress.ps1 and deploy-ingress-azure.ps1".
|
||||
|
||||
Errors if ingresses are still not available:
|
||||
|
||||
|
||||
**Deploy eShopOnContainers into Kuberentes:**
|
||||
Finally, in order to deploy the application's containers, run `deploy.ps1` with your ACR registry information. The Docker username and password are provided by Azure Container Registry, and can be retrieved from the Azure portal. Optionally, ACR credentials can be obtained by running the following command:
|
||||
|
||||
>```
|
||||
>az acr credential show -n eshopregistry
|
||||
>```
|
||||
|
||||
Once the user and password are retrieved, run the following script for deployment. For example:
|
||||
|
||||
>```
|
||||
>./deploy.ps1 -registry youreshopacrregistry.azurecr.io -dockerUser YoureShopAcrRegistry -dockerPassword YOUR-ACR-PASSWORD -configFile conf_local.yaml
|
||||
>```
|
||||
|
||||
The parameter `configFile` is important (and mandatory) because it contains the configuration used for the Pods in Kubernetes. This allow deploying Pods that use your own resources in Azure or any other cloud provider.
|
||||
|
||||
If the configuration file name you provide is `conf_local.yaml`, in addition to the application containers, it will deploy the infrastructure containers (that is, SQL Server container, RabbitMQ container, Redis container and MongoDB container will also be deployed into the Kubernetes cluster).
|
||||
|
||||
If the configuration file name you provide is `conf_cloud.yaml` it will just deploy the application containers. All the infrastructure should be deployed as Azure PaaS services such as Azure SQL Database, Azure Redis, Azure CosmosDB and Azure Service Bus instead of the dev/test infrastructure containers). This file is, however, provided just as a reference file because it doesn't have valid values for Azure resources. You'd need to provide your Azure configuration if you want to use infrastructure resources which are external to the Kubernetes cluster.
|
||||
|
||||
The execution will start like the following:
|
||||
|
||||
|
||||
|
||||
The script will build the .NET code (microservices and ASP.NET web apps) and build their corresponding Docker images, push the later to your ACR registry, and deploy the containers from ACR into your Kubernetes cluster.
|
||||
|
||||
Notice that the first time you build the Docker images it'll take quite some time.
|
||||
|
||||
Once the Docker images are built (wait a few minutes), you can watch the deployment unfold from the Kubernetes web interface:
|
||||
|
||||
run `kubectl proxy` and open a browser to [http://localhost:8001/#!/deployment](http://localhost:8001/#!/deployment)
|
||||
|
||||
The deployment should finish as in the following screenshot:
|
||||
|
||||
|
||||
Finally, go ahead and run eShopOnContainers in Kuberentes in AKS! 👍
|
||||
|
||||
|
||||
### Possible parameters for the deploy.ps1 script
|
||||
|
||||
The script accepts the following parameters:
|
||||
|
||||
+ `registry`: Fully qualified name of the Docker registry to use, like your ACR registry such as `youreshopacrregistry.azurecr.io`. If not provided, DockerHub registry is assumed.
|
||||
+ `dockerUser`: Login to use for the Docker registry (if needed)
|
||||
+ `dockerPassword`: Password to use for the Docker registry (if needed)
|
||||
+ `execPath`: Location of `kubectl` (if not in the path env-var). If provided, it must finish with the '/' character.
|
||||
+ `kubeconfigPath`: Location of the `kubectl` configuration file. **This parameter is used only in the CI pipeline**, so you don't need to pass it when invoking the script using the CLI in your local PC.
|
||||
+ `configFile`: Location of the Yaml file (such as `conf_local.yaml`) with the `externalcfg` configmap to be deployed. This configmap is used to configure the Pod's environment **This parameter is mandatory**
|
||||
+ `imageTag`: Tag of the images to deploy to Kuberentes. If not provided, the name of the current branch in Git is used.
|
||||
+ `deployCI`: If `true` means that script is running under the context of a VSTS Hosted Build Agent. **You should never use this parameter from CLI**
|
||||
+ `buildImages`: If `true` (default value) Docker images are built and pushed in the Docker registry. If you set this parameter to `false`, Docker images won't be built nor pushed in the Docker registry (but k8s' deployments and services will be redeployed).
|
||||
+ `deployInfrastructure`: If `true` infrastructure containers (rabbitmq, mongo, redis, sql) will be deployed in k8s. If `false` those containers (and its related deployments and services in k8s) won't be deployed.
|
||||
+ `dockerOrg`: Name of the organization in the registry where the images are (or will be pushed). Default value is `eshop` (which has images provided by Microsoft)
|
||||
|
||||
### Typical usages of the script:
|
||||
|
||||
* Build all the images, push the images in a organization called `foo` in DOCKER HUB (instead of ACR). Images will be tagged with my current git branch and containers will use the configuration set in `conf_local.yaml` file when deployed all the containers in Kuberentes, including infrastructure containers:
|
||||
|
||||
```
|
||||
./deploy.ps1 -buildImages $true -pushImages $true -dockerOrg foo -dockerUser MY_USER -dockerPassword MY_PASSWORD -configFile conf_local.yaml
|
||||
```
|
||||
|
||||
* Do not build Docker images, neither push it to the registry. Then, create Kubernetes deployments that will pull images from my private repository such as in ACR registry, in the `foo` organization, using the tag `latest`. Containers will use the configuration set in `conf_cloud.yaml` file.
|
||||
|
||||
```
|
||||
./deploy.ps1 -buildImages $false -pushImages $false -dockerOrg foo -registry MY_REGISTRY_FQDN -dockerUser MY_USER -dockerPassword MY_PASSWORD -configFile conf_cloud.yaml -imageTag latest
|
||||
```
|
||||
|
||||
* Deploy all containers (including infrastructure containers) into Kuberentes using the public eShopOnContainers images that Microsoft provides at Docker Hub, using the tag `dev`:
|
||||
|
||||
```
|
||||
./deploy.ps1 -buildImages $false -pushImages $false -configFile conf_local.yaml -imageTag dev
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user