diff --git a/README.md b/README.md index 4a4e1d3c3..cbae071f8 100644 --- a/README.md +++ b/README.md @@ -17,12 +17,12 @@ The **dockerfiles** in the solution have also been updated and now support [**Do >**PLEASE** Read our [branch guide](./branch-guide.md) to know about our branching policy > ### DISCLAIMER -> **IMPORTANT:** The current state of this sample application is **BETA**, because we are constantly evolving towards new released technologies. Therefore, many areas could be improved and change significantly while refactoring current code and implementing new features. Feedback with improvements and pull requests from the community will be highly appreciated and accepted. +> **IMPORTANT:** The current state of this sample application is **BETA**, because we are constantly evolving towards newly released technologies. Therefore, many areas could be improved and change significantly while refactoring the current code and implementing new features. Feedback with improvements and pull requests from the community will be highly appreciated and accepted. > -> This reference application proposes a simplified microservice oriented architecture implementation to introduce technologies like .NET Core with Docker containers through a comprehensive application. The chosen domain is an eShop/eCommerce but simply because it is a well-know domain by most people/developers. -However, this sample application should not be considered as an "eCommerce reference model", at all. The implemented business domain might not be ideal from an eCommerce business point of view. It is neither trying to solve all the problems in a large, scalable and mission-critical distributed system. It is just a bootstrap for developers to easily get started in the world of Docker containers and microservices with .NET Core. +> This reference application proposes a simplified microservice oriented architecture implementation to introduce technologies like .NET Core with Docker containers through a comprehensive application. The chosen domain is eShop/eCommerce but simply because it is a well-known domain by most people/developers. +However, this sample application should not be considered as an "eCommerce reference model" at all. The implemented business domain might not be ideal from an eCommerce business point of view. It is neither trying to solve all the problems in a large, scalable and mission-critical distributed system. It is just a bootstrap for developers to easily get started in the world of Docker containers and microservices with .NET Core. >

For example, the next step after running the solution in the local dev PC and understanding Docker containers and microservices development with .NET Core, is to select a microservice cluster/orchestrator like Kubernetes in Azure (AKS) or Azure Service Fabric, both environments tested and supported by this solution. -> Additional steps would be to move your databases to HA cloud services (like Azure SQL Database), or switch your EventBus to use Azure Service Bus (instead of bare-bone RabbitMQ) or any other production ready Service Bus in the market. +> Additional steps would be to move your databases to HA cloud services (like Azure SQL Database) or switch your EventBus to use Azure Service Bus (instead of bare-bone RabbitMQ) or any other production-ready Service Bus in the market. ![image](https://user-images.githubusercontent.com/1712635/40397331-059a7ec6-5de7-11e8-8542-a597eca16fef.png) @@ -39,7 +39,7 @@ The architecture proposes a microservice oriented architecture implementation wi > ### Important Note on API Gateways and published APIs > Since April 2018, we have introduced the implementation of the [API Gateway pattern](http://microservices.io/patterns/apigateway.html) and [Backend-For-Front-End (BFF) pattern](https://samnewman.io/patterns/architectural/bff/) in eShopOnContainers architecture, so you can filter and publish simplified APIs and URIs and apply additional security in that tier while hiding/securing the internal microservices to the client apps or outside consumers. These sample API Gateways in eShopOnContainers are based on [Ocelot](https://github.com/ThreeMammals/Ocelot), an OSS lightweight API Gateway solution explained [here](http://threemammals.com/ocelot). The deployed API Gateways are autonomous and can be deployed as your own custom microservices/containers, as it is currently done in eShopOnContainers, so you can test it even in a simple development environment with just Docker engine or deploy it into orchestrators like Kubernetes in AKS or Service Fabric. -> For your production-ready architecture you can either keep using [Ocelot](https://github.com/ThreeMammals/Ocelot) which is simple and easy to use and used in production by significant companies or if you need further functionality and a much richer set of features suittable for commercial APIs, you can also substitute those API Gateways and use [Azure API Management](https://azure.microsoft.com/en-us/services/api-management/) or any other commercial API Gateway, as shown in the following image. +> For your production-ready architecture you can either keep using [Ocelot](https://github.com/ThreeMammals/Ocelot) which is simple and easy to use and used in production by significant companies or if you need further functionality and a much richer set of features suitable for commercial APIs, you can also substitute those API Gateways and use [Azure API Management](https://azure.microsoft.com/en-us/services/api-management/) or any other commercial API Gateway, as shown in the following image.

@@ -56,9 +56,9 @@ The architecture proposes a microservice oriented architecture implementation wi

> ### Important Note on Database Servers/Containers -> In this solution's current configuration for a development environment, the SQL databases are automatically deployed with sample data into a single SQL Server container (a single shared Docker container for SQL databases) so the whole solution can be up and running without any dependency to any cloud or specific server. Each database could also be deployed as a single Docker container, but then you'd need more than 8GB of RAM assigned to Docker in your development machine in order to be able to run 3 SQL Server Docker containers in your Docker Linux host in "Docker for Windows" or "Docker for Mac" development environments. +> In this solution's current configuration for a development environment, the SQL databases are automatically deployed with sample data into a single SQL Server container (a single shared Docker container for SQL databases) so the whole solution can be up and running without any dependency to any cloud or a specific server. Each database could also be deployed as a single Docker container, but then you'd need more than 8GB of RAM assigned to Docker in your development machine in order to be able to run 3 SQL Server Docker containers in your Docker Linux host in "Docker for Windows" or "Docker for Mac" development environments. >

A similar case is defined in regard to Redis cache running as a container for the development environment. Or a No-SQL database (MongoDB) running as a container. ->

However, in a real production environment it is recommended to have your databases (SQL Server, Redis, and the NO-SQL database, in this case) in HA (High Available) services like Azure SQL Database, Redis as a service and Azure CosmosDB instead the MongoDB container (as both systems share the same access protocol). If you want to change to a production configuration, you'll just need to change the connection strings once you have set up the servers in a HA cloud or on-premises. +>

However, in a real production environment it is recommended to have your databases (SQL Server, Redis, and the NO-SQL database, in this case) in HA (High Available) services like Azure SQL Database, Redis as a service and Azure CosmosDB instead the MongoDB container (as both systems share the same access protocol). If you want to change to a production configuration, you'll just need to change the connection strings once you have set up the servers in an HA cloud or on-premises. ## Related documentation and guidance While developing this reference application, we've been creating a reference Guide/eBook focusing on architecting and developing containerized and microservice based .NET Applications (download link available below) which explains in detail how to develop this kind of architectural style (microservices, Docker containers, Domain-Driven Design for certain microservices) plus other simpler architectural styles, like monolithic apps that can also live as Docker containers. @@ -76,12 +76,12 @@ Download in other formats (**eReaders** like **MOBI**, **EPUB**) and other eBook Send feedback to [dotnet-architecture-ebooks-feedback@service.microsoft.com](dotnet-architecture-ebooks-feedback@service.microsoft.com) -However, we encourage to download and review the [Architecting and Developing Microservices eBook](https://aka.ms/microservicesebook) because the architectural styles and architectural patterns and technologies explained in the guidance are using this reference application when explaining many pattern implementations, so you'll understand much better the context, design and decisions taken in the current architecture and internal designs. +However, we encourage you to download and review the [Architecting and Developing Microservices eBook](https://aka.ms/microservicesebook) because the architectural styles and architectural patterns and technologies explained in the guide are using this reference application when explaining many pattern implementations, so you'll understand the context, design and decisions taken in the current architecture and internal designs much better. ## Overview of the application code In this repo you can find a sample reference application that will help you to understand how to implement a microservice architecture based application using .NET Core and Docker. -The example business domain or scenario is based on an eShop or eCommerce which is implemented as a multi-container application. Each container is a microservice deployment (like the basket-microservice, catalog-microservice, ordering-microservice and the identity-microservice) which are developed using ASP.NET Core running on .NET Core so they can run either on Linux Containers and Windows Containers. +The example business domain or scenario is based on an eShop or eCommerce which is implemented as a multi-container application. Each container is a microservice deployment (like the basket-microservice, catalog-microservice, ordering-microservice and the identity-microservice) which is developed using ASP.NET Core running on .NET Core so they can run either on Linux Containers and Windows Containers. The screenshot below shows the VS Solution structure for those microservices/containers and client apps. - (*Recommended when getting started*) Open eShopOnContainers-ServicesAndWebApps.sln for a solution containing just the server-side projects related to the microservices and web applications. diff --git a/docs/Decks/BRK3175_CesarDeIaTorre.pptx b/docs/Decks/BRK3175_CesarDeIaTorre.pptx new file mode 100644 index 000000000..e04d7338e Binary files /dev/null and b/docs/Decks/BRK3175_CesarDeIaTorre.pptx differ diff --git a/docs/Decks/eShopOnContainers-Architecture-v2.1.pptx b/docs/Decks/eShopOnContainers-Architecture-v2.1.pptx new file mode 100644 index 000000000..52398f823 Binary files /dev/null and b/docs/Decks/eShopOnContainers-Architecture-v2.1.pptx differ diff --git a/k8s/deploy-ingress-azure.ps1 b/k8s/deploy-ingress-azure.ps1 index f93cf437b..d0ff702df 100644 --- a/k8s/deploy-ingress-azure.ps1 +++ b/k8s/deploy-ingress-azure.ps1 @@ -1,3 +1 @@ -kubectl patch deployment -n ingress-nginx nginx-ingress-controller --type=json --patch="$(cat nginx-ingress\publish-service-patch.yaml)" -kubectl apply -f nginx-ingress\azure\service.yaml -kubectl apply -f nginx-ingress\patch-service-without-rbac.yaml \ No newline at end of file +kubectl apply -f nginx-ingress\cloud-generic.yaml \ No newline at end of file diff --git a/k8s/deploy-ingress-dockerlocal.ps1 b/k8s/deploy-ingress-dockerlocal.ps1 new file mode 100644 index 000000000..04ffad763 --- /dev/null +++ b/k8s/deploy-ingress-dockerlocal.ps1 @@ -0,0 +1,2 @@ +kubectl apply -f nginx-ingress\cm.yaml +kubectl apply -f nginx-ingress\cloud-generic.yaml \ No newline at end of file diff --git a/k8s/deploy-ingress.ps1 b/k8s/deploy-ingress.ps1 index 694361bfa..37abcbee2 100644 --- a/k8s/deploy-ingress.ps1 +++ b/k8s/deploy-ingress.ps1 @@ -1,12 +1,5 @@ -kubectl apply -f ingress.yaml - # Deploy nginx-ingress core files -kubectl apply -f nginx-ingress\namespace.yaml -kubectl apply -f nginx-ingress\default-backend.yaml -kubectl apply -f nginx-ingress\configmap.yaml -kubectl apply -f nginx-ingress\tcp-services-configmap.yaml -kubectl apply -f nginx-ingress\udp-services-configmap.yaml -kubectl apply -f nginx-ingress\without-rbac.yaml +kubectl apply -f nginx-ingress\mandatory.yaml diff --git a/k8s/deploy.ps1 b/k8s/deploy.ps1 index f0905096a..abeb12aed 100644 --- a/k8s/deploy.ps1 +++ b/k8s/deploy.ps1 @@ -113,6 +113,7 @@ ExecKube -cmd 'delete configmap internalurls' ExecKube -cmd 'delete configmap urls' ExecKube -cmd 'delete configmap externalcfg' ExecKube -cmd 'delete configmap ocelot' +ExecKube -cmd 'delete -f ingress.yaml' # start sql, rabbitmq, frontend deployments if ($deployInfrastructure) { @@ -204,5 +205,8 @@ ExecKube -cmd 'rollout resume deployments/apigwwm' ExecKube -cmd 'rollout resume deployments/apigwws' ExecKube -cmd 'rollout resume deployments/ordering-signalrhub' +Write-Host "Adding/Updating ingress resource..." -ForegroundColor Yellow +ExecKube -cmd 'apply -f ingress.yaml' + Write-Host "WebSPA is exposed at http://$externalDns, WebMVC at http://$externalDns/webmvc, WebStatus at http://$externalDns/webstatus" -ForegroundColor Yellow diff --git a/k8s/helm-rbac.yaml b/k8s/helm-rbac.yaml new file mode 100644 index 000000000..b6180329a --- /dev/null +++ b/k8s/helm-rbac.yaml @@ -0,0 +1,18 @@ +apiVersion: v1 +kind: ServiceAccount +metadata: + name: tiller + namespace: kube-system +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRoleBinding +metadata: + name: tiller +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: cluster-admin +subjects: + - kind: ServiceAccount + name: tiller + namespace: kube-system \ No newline at end of file diff --git a/k8s/helm/app.yaml b/k8s/helm/app.yaml index bd3988c5e..3868d331b 100644 --- a/k8s/helm/app.yaml +++ b/k8s/helm/app.yaml @@ -27,7 +27,7 @@ app: # app global settings catalog: catalog # service name for catalog api ordering: ordering # service name for ordering api orderingbackgroundtasks: orderingbackgroundtasks # service name for orderingbackgroundtasks - orderingsignalrhub: orderingsignalrhub # service name for orderingsignalrhub + orderingsignalrhub: ordering-signalrhub # service name for orderingsignalrhub identity: identity # service name for identity api mvc: webmvc # service name for web mvc spa: webspa # service name for web spa diff --git a/k8s/helm/deploy-all.ps1 b/k8s/helm/deploy-all.ps1 index 1239cc7af..08313cdbb 100644 --- a/k8s/helm/deploy-all.ps1 +++ b/k8s/helm/deploy-all.ps1 @@ -8,11 +8,19 @@ Param( [parameter(Mandatory=$false)][bool]$clean=$true, [parameter(Mandatory=$false)][string]$aksName="", [parameter(Mandatory=$false)][string]$aksRg="", - [parameter(Mandatory=$false)][string]$imageTag="latest" -) + [parameter(Mandatory=$false)][string]$imageTag="latest", + [parameter(Mandatory=$false)][bool]$useLocalk8s=$false + ) $dns = $externalDns +$ingressValuesFile="ingress_values.yaml" + +if ($ingressValuesFile) { + $ingressValuesFile="ingress_values_dockerk8s.yaml" + $dns="localhost" +} + if ($externalDns -eq "aks") { if ([string]::IsNullOrEmpty($aksName) -or [string]::IsNullOrEmpty($aksRg)) { Write-Host "Error: When using -dns aks, MUST set -aksName and -aksRg too." -ForegroundColor Red @@ -58,18 +66,18 @@ $charts = ("eshop-common", "apigwmm", "apigwms", "apigwwm", "apigwws", "basket-a if ($deployInfrastructure) { foreach ($infra in $infras) { Write-Host "Installing infrastructure: $infra" -ForegroundColor Green - helm install --values app.yaml --values inf.yaml --values ingress_values.yaml --set app.name=$appName --set inf.k8s.dns=$dns --name="$appName-$infra" $infra + helm install --values app.yaml --values inf.yaml --values $ingressValuesFile --set app.name=$appName --set inf.k8s.dns=$dns --name="$appName-$infra" $infra } } foreach ($chart in $charts) { Write-Host "Installing: $chart" -ForegroundColor Green if ($useCustomRegistry) { - helm install --set inf.registry.server=$registry --set inf.registry.login=$dockerUser --set inf.registry.pwd=$dockerPassword --set inf.registry.secretName=eshop-docker-scret --values app.yaml --values inf.yaml --values ingress_values.yaml --set app.name=$appName --set inf.k8s.dns=$dns --set image.tag=$imageTag --set image.pullPolicy=Always --name="$appName-$chart" $chart + helm install --set inf.registry.server=$registry --set inf.registry.login=$dockerUser --set inf.registry.pwd=$dockerPassword --set inf.registry.secretName=eshop-docker-scret --values app.yaml --values inf.yaml --values $ingressValuesFile --set app.name=$appName --set inf.k8s.dns=$dns --set image.tag=$imageTag --set image.pullPolicy=Always --name="$appName-$chart" $chart } else { if ($chart -ne "eshop-common") { # eshop-common is ignored when no secret must be deployed - helm install --values app.yaml --values inf.yaml --values ingress_values.yaml --set app.name=$appName --set inf.k8s.dns=$dns --set image.tag=$imageTag --set image.pullPolicy=Always --name="$appName-$chart" $chart + helm install --values app.yaml --values inf.yaml --values $ingressValuesFile --set app.name=$appName --set inf.k8s.dns=$dns --set image.tag=$imageTag --set image.pullPolicy=Always --name="$appName-$chart" $chart } } } diff --git a/k8s/helm/ingress_values_dockerk8s.yaml b/k8s/helm/ingress_values_dockerk8s.yaml new file mode 100644 index 000000000..75597aac9 --- /dev/null +++ b/k8s/helm/ingress_values_dockerk8s.yaml @@ -0,0 +1,5 @@ +ingress: + annotations: + kubernetes.io/ingress.class: "nginx" + ingress.kubernetes.io/ssl-redirect: "false" + nginx.ingress.kubernetes.io/ssl-redirect: "false" diff --git a/k8s/helm/ordering-signalrhub/templates/configmap.yaml b/k8s/helm/ordering-signalrhub/templates/configmap.yaml index cec9565f0..addcf7e9d 100644 --- a/k8s/helm/ordering-signalrhub/templates/configmap.yaml +++ b/k8s/helm/ordering-signalrhub/templates/configmap.yaml @@ -15,4 +15,4 @@ data: all__InstrumentationKey: {{ .Values.inf.appinsights.key }} all__UseAzureServiceBus: "{{ .Values.inf.eventbus.useAzure }}" signalr__StoreConnectionString: {{ .Values.inf.redis.keystore.constr }} - urls__IdentityUrl: {{ $identity }} \ No newline at end of file + urls__IdentityUrl: http://{{ $identity }} \ No newline at end of file diff --git a/k8s/nginx-ingress/azure/service.yaml b/k8s/nginx-ingress/azure/service.yaml deleted file mode 100644 index 8d2f71505..000000000 --- a/k8s/nginx-ingress/azure/service.yaml +++ /dev/null @@ -1,19 +0,0 @@ -kind: Service -apiVersion: v1 -metadata: - name: ingress-nginx - namespace: ingress-nginx - labels: - app: ingress-nginx -spec: - externalTrafficPolicy: Local - type: LoadBalancer - selector: - app: ingress-nginx - ports: - - name: http - port: 80 - targetPort: http - - name: https - port: 443 - targetPort: https diff --git a/k8s/nginx-ingress/cloud-generic.yaml b/k8s/nginx-ingress/cloud-generic.yaml new file mode 100644 index 000000000..945441ab8 --- /dev/null +++ b/k8s/nginx-ingress/cloud-generic.yaml @@ -0,0 +1,21 @@ +kind: Service +apiVersion: v1 +metadata: + name: ingress-nginx + namespace: ingress-nginx + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx +spec: + externalTrafficPolicy: Local + type: LoadBalancer + selector: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx + ports: + - name: http + port: 80 + targetPort: http + - name: https + port: 443 + targetPort: https \ No newline at end of file diff --git a/k8s/nginx-ingress/cm.yaml b/k8s/nginx-ingress/cm.yaml new file mode 100644 index 000000000..7818fd15b Binary files /dev/null and b/k8s/nginx-ingress/cm.yaml differ diff --git a/k8s/nginx-ingress/configmap.yaml b/k8s/nginx-ingress/configmap.yaml deleted file mode 100644 index 6703fc38e..000000000 --- a/k8s/nginx-ingress/configmap.yaml +++ /dev/null @@ -1,11 +0,0 @@ -kind: ConfigMap -apiVersion: v1 -metadata: - name: nginx-configuration - namespace: ingress-nginx - labels: - app: ingress-nginx -data: - ssl-redirect: "false" - proxy-buffer-size: "128k" - proxy-buffers: "4 256k" diff --git a/k8s/nginx-ingress/default-backend.yaml b/k8s/nginx-ingress/default-backend.yaml deleted file mode 100644 index 64f6f58ad..000000000 --- a/k8s/nginx-ingress/default-backend.yaml +++ /dev/null @@ -1,52 +0,0 @@ -apiVersion: extensions/v1beta1 -kind: Deployment -metadata: - name: default-http-backend - labels: - app: default-http-backend - namespace: ingress-nginx -spec: - replicas: 1 - template: - metadata: - labels: - app: default-http-backend - spec: - terminationGracePeriodSeconds: 60 - containers: - - name: default-http-backend - # Any image is permissable as long as: - # 1. It serves a 404 page at / - # 2. It serves 200 on a /healthz endpoint - image: gcr.io/google_containers/defaultbackend:1.4 - livenessProbe: - httpGet: - path: /healthz - port: 8080 - scheme: HTTP - initialDelaySeconds: 30 - timeoutSeconds: 5 - ports: - - containerPort: 8080 - resources: - limits: - cpu: 10m - memory: 20Mi - requests: - cpu: 10m - memory: 20Mi ---- - -apiVersion: v1 -kind: Service -metadata: - name: default-http-backend - namespace: ingress-nginx - labels: - app: default-http-backend -spec: - ports: - - port: 80 - targetPort: 8080 - selector: - app: default-http-backend diff --git a/k8s/nginx-ingress/local-dockerk8s/identityapi-cm-fix.yaml b/k8s/nginx-ingress/local-dockerk8s/identityapi-cm-fix.yaml new file mode 100644 index 000000000..3a3fcf5a5 --- /dev/null +++ b/k8s/nginx-ingress/local-dockerk8s/identityapi-cm-fix.yaml @@ -0,0 +1,3 @@ +data: + mvc_e: http://10.0.75.1/webmvc + \ No newline at end of file diff --git a/k8s/nginx-ingress/local-dockerk8s/mvc-cm-fix.yaml b/k8s/nginx-ingress/local-dockerk8s/mvc-cm-fix.yaml new file mode 100644 index 000000000..1475deec1 --- /dev/null +++ b/k8s/nginx-ingress/local-dockerk8s/mvc-cm-fix.yaml @@ -0,0 +1,3 @@ +data: + urls__IdentityUrl: http://10.0.75.1/identity + urls__mvc: http://10.0.75.1/webmvc diff --git a/k8s/nginx-ingress/local-dockerk8s/mvc-fix.yaml b/k8s/nginx-ingress/local-dockerk8s/mvc-fix.yaml new file mode 100644 index 000000000..b9ecd4cba --- /dev/null +++ b/k8s/nginx-ingress/local-dockerk8s/mvc-fix.yaml @@ -0,0 +1,39 @@ +apiVersion: extensions/v1beta1 +kind: Ingress +metadata: + annotations: + ingress.kubernetes.io/ssl-redirect: "false" + kubernetes.io/ingress.class: nginx + nginx.ingress.kubernetes.io/ssl-redirect: "false" + labels: + app: webmvc + name: eshop-webmvc-loopback + namespace: default +spec: + rules: + - http: + paths: + - backend: + serviceName: webmvc + servicePort: http + path: /webmvc +--- +apiVersion: extensions/v1beta1 +kind: Ingress +metadata: + annotations: + ingress.kubernetes.io/ssl-redirect: "false" + kubernetes.io/ingress.class: nginx + nginx.ingress.kubernetes.io/ssl-redirect: "false" + labels: + app: identity-api + name: eshop-identity-api-loopback + namespace: default +spec: + rules: + - http: + paths: + - backend: + serviceName: identity + servicePort: http + path: /identity \ No newline at end of file diff --git a/k8s/nginx-ingress/mandatory.yaml b/k8s/nginx-ingress/mandatory.yaml new file mode 100644 index 000000000..56b1cc3b5 --- /dev/null +++ b/k8s/nginx-ingress/mandatory.yaml @@ -0,0 +1,238 @@ +apiVersion: v1 +kind: Namespace +metadata: + name: ingress-nginx + +--- + +kind: ConfigMap +apiVersion: v1 +metadata: + name: nginx-configuration + namespace: ingress-nginx + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx + +--- + +apiVersion: v1 +kind: ServiceAccount +metadata: + name: nginx-ingress-serviceaccount + namespace: ingress-nginx + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx + +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRole +metadata: + name: nginx-ingress-clusterrole + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx +rules: + - apiGroups: + - "" + resources: + - configmaps + - endpoints + - nodes + - pods + - secrets + verbs: + - list + - watch + - apiGroups: + - "" + resources: + - nodes + verbs: + - get + - apiGroups: + - "" + resources: + - services + verbs: + - get + - list + - watch + - apiGroups: + - "extensions" + resources: + - ingresses + verbs: + - get + - list + - watch + - apiGroups: + - "" + resources: + - events + verbs: + - create + - patch + - apiGroups: + - "extensions" + resources: + - ingresses/status + verbs: + - update + +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: Role +metadata: + name: nginx-ingress-role + namespace: ingress-nginx + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx +rules: + - apiGroups: + - "" + resources: + - configmaps + - pods + - secrets + - namespaces + verbs: + - get + - apiGroups: + - "" + resources: + - configmaps + resourceNames: + # Defaults to "-" + # Here: "-" + # This has to be adapted if you change either parameter + # when launching the nginx-ingress-controller. + - "ingress-controller-leader-nginx" + verbs: + - get + - update + - apiGroups: + - "" + resources: + - configmaps + verbs: + - create + - apiGroups: + - "" + resources: + - endpoints + verbs: + - get + +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: RoleBinding +metadata: + name: nginx-ingress-role-nisa-binding + namespace: ingress-nginx + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: Role + name: nginx-ingress-role +subjects: + - kind: ServiceAccount + name: nginx-ingress-serviceaccount + namespace: ingress-nginx + +--- +apiVersion: rbac.authorization.k8s.io/v1beta1 +kind: ClusterRoleBinding +metadata: + name: nginx-ingress-clusterrole-nisa-binding + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx +roleRef: + apiGroup: rbac.authorization.k8s.io + kind: ClusterRole + name: nginx-ingress-clusterrole +subjects: + - kind: ServiceAccount + name: nginx-ingress-serviceaccount + namespace: ingress-nginx + +--- + +apiVersion: extensions/v1beta1 +kind: Deployment +metadata: + name: nginx-ingress-controller + namespace: ingress-nginx + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx +spec: + replicas: 1 + selector: + matchLabels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx + template: + metadata: + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx + annotations: + prometheus.io/port: "10254" + prometheus.io/scrape: "true" + spec: + serviceAccountName: nginx-ingress-serviceaccount + containers: + - name: nginx-ingress-controller + image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.20.0 + args: + - /nginx-ingress-controller + - --configmap=$(POD_NAMESPACE)/nginx-configuration + - --publish-service=$(POD_NAMESPACE)/ingress-nginx + - --annotations-prefix=nginx.ingress.kubernetes.io + securityContext: + capabilities: + drop: + - ALL + add: + - NET_BIND_SERVICE + # www-data -> 33 + runAsUser: 33 + env: + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: POD_NAMESPACE + valueFrom: + fieldRef: + fieldPath: metadata.namespace + ports: + - name: http + containerPort: 80 + - name: https + containerPort: 443 + livenessProbe: + failureThreshold: 3 + httpGet: + path: /healthz + port: 10254 + scheme: HTTP + initialDelaySeconds: 10 + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 1 + readinessProbe: + failureThreshold: 3 + httpGet: + path: /healthz + port: 10254 + scheme: HTTP + periodSeconds: 10 + successThreshold: 1 + timeoutSeconds: 1 diff --git a/k8s/nginx-ingress/namespace.yaml b/k8s/nginx-ingress/namespace.yaml deleted file mode 100644 index 6878f0be8..000000000 --- a/k8s/nginx-ingress/namespace.yaml +++ /dev/null @@ -1,4 +0,0 @@ -apiVersion: v1 -kind: Namespace -metadata: - name: ingress-nginx diff --git a/k8s/nginx-ingress/patch-service-without-rbac.yaml b/k8s/nginx-ingress/patch-service-without-rbac.yaml deleted file mode 100644 index 919efc389..000000000 --- a/k8s/nginx-ingress/patch-service-without-rbac.yaml +++ /dev/null @@ -1,40 +0,0 @@ -apiVersion: extensions/v1beta1 -kind: Deployment -metadata: - name: nginx-ingress-controller - namespace: ingress-nginx -spec: - replicas: 1 - selector: - matchLabels: - app: ingress-nginx - template: - metadata: - labels: - app: ingress-nginx - spec: - containers: - - name: nginx-ingress-controller - image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0 - args: - - /nginx-ingress-controller - - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - - --configmap=$(POD_NAMESPACE)/nginx-configuration - - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - - --publish-service=$(POD_NAMESPACE)/ingress-nginx - - --annotations-prefix=nginx.ingress.kubernetes.io - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - ports: - - name: http - containerPort: 80 - - name: https - containerPort: 443 diff --git a/k8s/nginx-ingress/publish-service-patch.yaml b/k8s/nginx-ingress/publish-service-patch.yaml deleted file mode 100644 index f8f52f772..000000000 --- a/k8s/nginx-ingress/publish-service-patch.yaml +++ /dev/null @@ -1,7 +0,0 @@ -[ - { - 'op': 'add', - 'path': '/spec/template/spec/containers/0/args/-', - 'value': '--publish-service=$(POD_NAMESPACE)/ingress-nginx' - } -] diff --git a/k8s/nginx-ingress/service-nodeport.yaml b/k8s/nginx-ingress/service-nodeport.yaml new file mode 100644 index 000000000..dd82ed3ed --- /dev/null +++ b/k8s/nginx-ingress/service-nodeport.yaml @@ -0,0 +1,22 @@ +apiVersion: v1 +kind: Service +metadata: + name: ingress-nginx + namespace: ingress-nginx + labels: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx +spec: + type: NodePort + ports: + - name: http + port: 80 + targetPort: 80 + protocol: TCP + - name: https + port: 443 + targetPort: 443 + protocol: TCP + selector: + app.kubernetes.io/name: ingress-nginx + app.kubernetes.io/part-of: ingress-nginx diff --git a/k8s/nginx-ingress/tcp-services-configmap.yaml b/k8s/nginx-ingress/tcp-services-configmap.yaml deleted file mode 100644 index a963085d3..000000000 --- a/k8s/nginx-ingress/tcp-services-configmap.yaml +++ /dev/null @@ -1,5 +0,0 @@ -kind: ConfigMap -apiVersion: v1 -metadata: - name: tcp-services - namespace: ingress-nginx diff --git a/k8s/nginx-ingress/udp-services-configmap.yaml b/k8s/nginx-ingress/udp-services-configmap.yaml deleted file mode 100644 index 1870931a2..000000000 --- a/k8s/nginx-ingress/udp-services-configmap.yaml +++ /dev/null @@ -1,5 +0,0 @@ -kind: ConfigMap -apiVersion: v1 -metadata: - name: udp-services - namespace: ingress-nginx diff --git a/k8s/nginx-ingress/without-rbac.yaml b/k8s/nginx-ingress/without-rbac.yaml deleted file mode 100644 index 1c46b73eb..000000000 --- a/k8s/nginx-ingress/without-rbac.yaml +++ /dev/null @@ -1,61 +0,0 @@ -apiVersion: extensions/v1beta1 -kind: Deployment -metadata: - name: nginx-ingress-controller - namespace: ingress-nginx -spec: - replicas: 1 - selector: - matchLabels: - app: ingress-nginx - template: - metadata: - labels: - app: ingress-nginx - annotations: - prometheus.io/port: '10254' - prometheus.io/scrape: 'true' - spec: - containers: - - name: nginx-ingress-controller - image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.9.0 - args: - - /nginx-ingress-controller - - --default-backend-service=$(POD_NAMESPACE)/default-http-backend - - --configmap=$(POD_NAMESPACE)/nginx-configuration - - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services - - --udp-services-configmap=$(POD_NAMESPACE)/udp-services - - --annotations-prefix=nginx.ingress.kubernetes.io - env: - - name: POD_NAME - valueFrom: - fieldRef: - fieldPath: metadata.name - - name: POD_NAMESPACE - valueFrom: - fieldRef: - fieldPath: metadata.namespace - ports: - - name: http - containerPort: 80 - - name: https - containerPort: 443 - livenessProbe: - failureThreshold: 3 - httpGet: - path: /healthz - port: 10254 - scheme: HTTP - initialDelaySeconds: 10 - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 1 - readinessProbe: - failureThreshold: 3 - httpGet: - path: /healthz - port: 10254 - scheme: HTTP - periodSeconds: 10 - successThreshold: 1 - timeoutSeconds: 1 diff --git a/src/ApiGateways/ApiGw-Base/OcelotApiGw.csproj b/src/ApiGateways/ApiGw-Base/OcelotApiGw.csproj index d3b1a049b..821755d9d 100644 --- a/src/ApiGateways/ApiGw-Base/OcelotApiGw.csproj +++ b/src/ApiGateways/ApiGw-Base/OcelotApiGw.csproj @@ -10,6 +10,6 @@ - + diff --git a/src/ApiGateways/ApiGw-Base/Startup.cs b/src/ApiGateways/ApiGw-Base/Startup.cs index f6a36b59e..da7cd25f4 100644 --- a/src/ApiGateways/ApiGw-Base/Startup.cs +++ b/src/ApiGateways/ApiGw-Base/Startup.cs @@ -1,11 +1,5 @@ -using System; -using System.Collections.Generic; -using System.Linq; -using System.Threading.Tasks; -using CacheManager.Core; -using Microsoft.AspNetCore.Builder; +using Microsoft.AspNetCore.Builder; using Microsoft.AspNetCore.Hosting; -using Microsoft.AspNetCore.Http; using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; diff --git a/src/BuildingBlocks/EventBus/EventBusRabbitMQ/EventBusRabbitMQ.cs b/src/BuildingBlocks/EventBus/EventBusRabbitMQ/EventBusRabbitMQ.cs index 49a417635..a3b6437ef 100644 --- a/src/BuildingBlocks/EventBus/EventBusRabbitMQ/EventBusRabbitMQ.cs +++ b/src/BuildingBlocks/EventBus/EventBusRabbitMQ/EventBusRabbitMQ.cs @@ -217,14 +217,16 @@ namespace Microsoft.eShopOnContainers.BuildingBlocks.EventBusRabbitMQ if (subscription.IsDynamic) { var handler = scope.ResolveOptional(subscription.HandlerType) as IDynamicIntegrationEventHandler; + if (handler == null) continue; dynamic eventData = JObject.Parse(message); await handler.Handle(eventData); } else { + var handler = scope.ResolveOptional(subscription.HandlerType); + if (handler == null) continue; var eventType = _subsManager.GetEventTypeByName(eventName); var integrationEvent = JsonConvert.DeserializeObject(message, eventType); - var handler = scope.ResolveOptional(subscription.HandlerType); var concreteType = typeof(IIntegrationEventHandler<>).MakeGenericType(eventType); await (Task)concreteType.GetMethod("Handle").Invoke(handler, new object[] { integrationEvent }); } diff --git a/src/BuildingBlocks/EventBus/EventBusServiceBus/EventBusServiceBus.cs b/src/BuildingBlocks/EventBus/EventBusServiceBus/EventBusServiceBus.cs index 2cd86669b..d16eb4625 100644 --- a/src/BuildingBlocks/EventBus/EventBusServiceBus/EventBusServiceBus.cs +++ b/src/BuildingBlocks/EventBus/EventBusServiceBus/EventBusServiceBus.cs @@ -163,14 +163,16 @@ if (subscription.IsDynamic) { var handler = scope.ResolveOptional(subscription.HandlerType) as IDynamicIntegrationEventHandler; + if (handler == null) continue; dynamic eventData = JObject.Parse(message); await handler.Handle(eventData); } else { - var eventType = _subsManager.GetEventTypeByName(eventName); - var integrationEvent = JsonConvert.DeserializeObject(message, eventType); var handler = scope.ResolveOptional(subscription.HandlerType); + if (handler == null) continue; + var eventType = _subsManager.GetEventTypeByName(eventName); + var integrationEvent = JsonConvert.DeserializeObject(message, eventType); var concreteType = typeof(IIntegrationEventHandler<>).MakeGenericType(eventType); await (Task)concreteType.GetMethod("Handle").Invoke(handler, new object[] { integrationEvent }); } diff --git a/src/BuildingBlocks/EventBus/IntegrationEventLogEF/EventStateEnum.cs b/src/BuildingBlocks/EventBus/IntegrationEventLogEF/EventStateEnum.cs index 3efb78e74..079cf7d7e 100644 --- a/src/BuildingBlocks/EventBus/IntegrationEventLogEF/EventStateEnum.cs +++ b/src/BuildingBlocks/EventBus/IntegrationEventLogEF/EventStateEnum.cs @@ -7,7 +7,8 @@ namespace Microsoft.eShopOnContainers.BuildingBlocks.IntegrationEventLogEF public enum EventStateEnum { NotPublished = 0, - Published = 1, - PublishedFailed = 2 + InProgress = 1, + Published = 2, + PublishedFailed = 3 } } diff --git a/src/BuildingBlocks/EventBus/IntegrationEventLogEF/IntegrationEventLogEntry.cs b/src/BuildingBlocks/EventBus/IntegrationEventLogEF/IntegrationEventLogEntry.cs index 3cab9e500..e5c3bc9ad 100644 --- a/src/BuildingBlocks/EventBus/IntegrationEventLogEF/IntegrationEventLogEntry.cs +++ b/src/BuildingBlocks/EventBus/IntegrationEventLogEF/IntegrationEventLogEntry.cs @@ -3,6 +3,9 @@ using System.Collections.Generic; using System.Text; using Newtonsoft.Json; using Microsoft.eShopOnContainers.BuildingBlocks.EventBus.Events; +using System.Linq; +using System.ComponentModel.DataAnnotations.Schema; +using System.Reflection; namespace Microsoft.eShopOnContainers.BuildingBlocks.IntegrationEventLogEF { @@ -11,7 +14,7 @@ namespace Microsoft.eShopOnContainers.BuildingBlocks.IntegrationEventLogEF private IntegrationEventLogEntry() { } public IntegrationEventLogEntry(IntegrationEvent @event) { - EventId = @event.Id; + EventId = @event.Id; CreationTime = @event.CreationDate; EventTypeName = @event.GetType().FullName; Content = JsonConvert.SerializeObject(@event); @@ -20,9 +23,19 @@ namespace Microsoft.eShopOnContainers.BuildingBlocks.IntegrationEventLogEF } public Guid EventId { get; private set; } public string EventTypeName { get; private set; } + [NotMapped] + public string EventTypeShortName => EventTypeName.Split('.')?.Last(); + [NotMapped] + public IntegrationEvent IntegrationEvent { get; private set; } public EventStateEnum State { get; set; } public int TimesSent { get; set; } public DateTime CreationTime { get; private set; } public string Content { get; private set; } + + public IntegrationEventLogEntry DeserializeJsonContent(Type type) + { + IntegrationEvent = JsonConvert.DeserializeObject(Content, type) as IntegrationEvent; + return this; + } } } diff --git a/src/BuildingBlocks/EventBus/IntegrationEventLogEF/Services/IIntegrationEventLogService.cs b/src/BuildingBlocks/EventBus/IntegrationEventLogEF/Services/IIntegrationEventLogService.cs index ed1f74616..6167d8ae8 100644 --- a/src/BuildingBlocks/EventBus/IntegrationEventLogEF/Services/IIntegrationEventLogService.cs +++ b/src/BuildingBlocks/EventBus/IntegrationEventLogEF/Services/IIntegrationEventLogService.cs @@ -9,7 +9,10 @@ namespace Microsoft.eShopOnContainers.BuildingBlocks.IntegrationEventLogEF.Servi { public interface IIntegrationEventLogService { + Task> RetrieveEventLogsPendingToPublishAsync(); Task SaveEventAsync(IntegrationEvent @event, DbTransaction transaction); - Task MarkEventAsPublishedAsync(IntegrationEvent @event); + Task MarkEventAsPublishedAsync(Guid eventId); + Task MarkEventAsInProgressAsync(Guid eventId); + Task MarkEventAsFailedAsync(Guid eventId); } } diff --git a/src/BuildingBlocks/EventBus/IntegrationEventLogEF/Services/IntegrationEventLogService.cs b/src/BuildingBlocks/EventBus/IntegrationEventLogEF/Services/IntegrationEventLogService.cs index a12309482..2712c5e1c 100644 --- a/src/BuildingBlocks/EventBus/IntegrationEventLogEF/Services/IntegrationEventLogService.cs +++ b/src/BuildingBlocks/EventBus/IntegrationEventLogEF/Services/IntegrationEventLogService.cs @@ -1,9 +1,14 @@ using Microsoft.EntityFrameworkCore; using Microsoft.EntityFrameworkCore.Diagnostics; +using Microsoft.eShopOnContainers.BuildingBlocks.EventBus; using Microsoft.eShopOnContainers.BuildingBlocks.EventBus.Events; +using Newtonsoft.Json; using System; +using System.Collections; +using System.Collections.Generic; using System.Data.Common; using System.Linq; +using System.Reflection; using System.Threading.Tasks; namespace Microsoft.eShopOnContainers.BuildingBlocks.IntegrationEventLogEF.Services @@ -12,6 +17,7 @@ namespace Microsoft.eShopOnContainers.BuildingBlocks.IntegrationEventLogEF.Servi { private readonly IntegrationEventLogContext _integrationEventLogContext; private readonly DbConnection _dbConnection; + private readonly List _eventTypes; public IntegrationEventLogService(DbConnection dbConnection) { @@ -21,6 +27,20 @@ namespace Microsoft.eShopOnContainers.BuildingBlocks.IntegrationEventLogEF.Servi .UseSqlServer(_dbConnection) .ConfigureWarnings(warnings => warnings.Throw(RelationalEventId.QueryClientEvaluationWarning)) .Options); + + _eventTypes = Assembly.Load(Assembly.GetEntryAssembly().FullName) + .GetTypes() + .Where(t => t.Name.EndsWith(nameof(IntegrationEvent))) + .ToList(); + } + + public async Task> RetrieveEventLogsPendingToPublishAsync() + { + return await _integrationEventLogContext.IntegrationEventLogs + .Where(e => e.State == EventStateEnum.NotPublished) + .OrderBy(o => o.CreationTime) + .Select(e => e.DeserializeJsonContent(_eventTypes.Find(t=> t.Name == e.EventTypeShortName))) + .ToListAsync(); } public Task SaveEventAsync(IntegrationEvent @event, DbTransaction transaction) @@ -38,11 +58,28 @@ namespace Microsoft.eShopOnContainers.BuildingBlocks.IntegrationEventLogEF.Servi return _integrationEventLogContext.SaveChangesAsync(); } - public Task MarkEventAsPublishedAsync(IntegrationEvent @event) + public Task MarkEventAsPublishedAsync(Guid eventId) + { + return UpdateEventStatus(eventId, EventStateEnum.Published); + } + + public Task MarkEventAsInProgressAsync(Guid eventId) { - var eventLogEntry = _integrationEventLogContext.IntegrationEventLogs.Single(ie => ie.EventId == @event.Id); - eventLogEntry.TimesSent++; - eventLogEntry.State = EventStateEnum.Published; + return UpdateEventStatus(eventId, EventStateEnum.InProgress); + } + + public Task MarkEventAsFailedAsync(Guid eventId) + { + return UpdateEventStatus(eventId, EventStateEnum.PublishedFailed); + } + + private Task UpdateEventStatus(Guid eventId, EventStateEnum status) + { + var eventLogEntry = _integrationEventLogContext.IntegrationEventLogs.Single(ie => ie.EventId == eventId); + eventLogEntry.State = status; + + if(status == EventStateEnum.InProgress) + eventLogEntry.TimesSent++; _integrationEventLogContext.IntegrationEventLogs.Update(eventLogEntry); diff --git a/src/Services/Basket/Basket.API/BasketSettings.cs b/src/Services/Basket/Basket.API/BasketSettings.cs index 9d143545a..064d615ec 100644 --- a/src/Services/Basket/Basket.API/BasketSettings.cs +++ b/src/Services/Basket/Basket.API/BasketSettings.cs @@ -3,7 +3,5 @@ public class BasketSettings { public string ConnectionString { get; set; } - - public string EventBusConnection { get; set; } } } diff --git a/src/Services/Catalog/Catalog.API/IntegrationEvents/CatalogIntegrationEventService.cs b/src/Services/Catalog/Catalog.API/IntegrationEvents/CatalogIntegrationEventService.cs index 1b82251e3..8c550bf27 100644 --- a/src/Services/Catalog/Catalog.API/IntegrationEvents/CatalogIntegrationEventService.cs +++ b/src/Services/Catalog/Catalog.API/IntegrationEvents/CatalogIntegrationEventService.cs @@ -29,9 +29,16 @@ namespace Catalog.API.IntegrationEvents public async Task PublishThroughEventBusAsync(IntegrationEvent evt) { - _eventBus.Publish(evt); - - await _eventLogService.MarkEventAsPublishedAsync(evt); + try + { + await _eventLogService.MarkEventAsInProgressAsync(evt.Id); + _eventBus.Publish(evt); + await _eventLogService.MarkEventAsPublishedAsync(evt.Id); + } + catch (Exception) + { + await _eventLogService.MarkEventAsFailedAsync(evt.Id); + } } public async Task SaveEventAndCatalogContextChangesAsync(IntegrationEvent evt) diff --git a/src/Services/Catalog/Catalog.API/Startup.cs b/src/Services/Catalog/Catalog.API/Startup.cs index 408f870af..5f0080c44 100644 --- a/src/Services/Catalog/Catalog.API/Startup.cs +++ b/src/Services/Catalog/Catalog.API/Startup.cs @@ -232,7 +232,7 @@ namespace Microsoft.eShopOnContainers.Services.Catalog.API public static IServiceCollection AddIntegrationServices(this IServiceCollection services, IConfiguration configuration) { services.AddTransient>( - sp => (DbConnection c) => new IntegrationEventLogService(c)); + sp => (DbConnection c) => new IntegrationEventLogService(c)); services.AddTransient(); diff --git a/src/Services/Identity/Identity.API/Controllers/AccountController.cs b/src/Services/Identity/Identity.API/Controllers/AccountController.cs index 85d17228c..7a1fea312 100644 --- a/src/Services/Identity/Identity.API/Controllers/AccountController.cs +++ b/src/Services/Identity/Identity.API/Controllers/AccountController.cs @@ -216,6 +216,8 @@ namespace Microsoft.eShopOnContainers.Services.Identity.API.Controllers // delete authentication cookie await HttpContext.SignOutAsync(); + await HttpContext.SignOutAsync(IdentityConstants.ApplicationScheme); + // set this so UI rendering sees an anonymous user HttpContext.User = new ClaimsPrincipal(new ClaimsIdentity()); diff --git a/src/Services/Identity/Identity.API/Identity.API.csproj b/src/Services/Identity/Identity.API/Identity.API.csproj index d2ee94316..a7d93aa2a 100644 --- a/src/Services/Identity/Identity.API/Identity.API.csproj +++ b/src/Services/Identity/Identity.API/Identity.API.csproj @@ -55,7 +55,6 @@ - diff --git a/src/Services/Identity/Identity.API/Views/Shared/_Layout.cshtml b/src/Services/Identity/Identity.API/Views/Shared/_Layout.cshtml index 60e420f64..4727229c7 100644 --- a/src/Services/Identity/Identity.API/Views/Shared/_Layout.cshtml +++ b/src/Services/Identity/Identity.API/Views/Shared/_Layout.cshtml @@ -18,7 +18,7 @@ -