Differences

This shows you the differences between two versions of the page.

--- wiki:confluence_aws_deploy [2023/10/16 18:08] – created antisa
+++ wiki:confluence_aws_deploy [2023/10/17 11:14] (current) – apply formatting antisa
@@ Line 6: / Line 6: @@
   helm repo add atlassian-data-center \
- https://atlassian.github.io/data-center-helm-charts
+  https://atlassian.github.io/data-center-helm-charts
   helm repo update
@@ Line 35: / Line 35: @@
     - Again, open the “Instances” tab. Expand your MySQL instance and expand “Instance Action” tab and select "Reboot".
+You might also need in values.yml //?sessionVariables=transaction_isolation='READ-COMMITTED'//
+e.g.
+  url: "jdbc:mysql://your.confluence.ap-southeast-2.rds.amazonaws.com/confluencedb?sessionVariables=transaction_isolation='READ-COMMITTED'"
 Connect to the database (use the URL from the console) and create the DB:
   CREATE DATABASE confluencedb CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;
@@ Line 50: / Line 54: @@
 ===== Configure persistent storage =====
+Configure shared and local volumes see this https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/docs/efs-create-filesystem.md. In short run this, replacing any parameters if not the same:
+<code bash>
 vpc_id=$(aws eks describe-cluster \
 --name atlassian-cluster \
@@ Line 73: / Line 79: @@
 --query 'FileSystemId' \
 --output text)
+</code>
-# configure shared and local volumes -> https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/docs/efs-create-filesystem.md
+Get the subnets in the VPC:
-aws ec2 describe-subnets     --filters "Name=vpc-id,Values=$vpc_id"  --query 'Subnets[*].{SubnetId: SubnetId,AvailabilityZone: AvailabilityZone,CidrBlock: CidrBlock}'  --output table --region ap-southeast-2
+  aws ec2 describe-subnets     --filters "Name=vpc-id,Values=$vpc_id"  --query 'Subnets[*].{SubnetId: SubnetId,AvailabilityZone: AvailabilityZone,CidrBlock: CidrBlock}'  --output table --region ap-southeast-2
-kubectl get nodes
+Check the node ips:
+  kubectl get nodes
-aws efs create-mount-target     --file-system-id $file_system_id     --subnet-id subnet-035ce5337e5f51cbb     --security-groups $security_group_id --region ap-southeast-2
+Now you will need to run the below command for each subnet (from above command) the node (its IP address) is in:
+  aws efs create-mount-target     --file-system-id $file_system_id     --subnet-id subnet-035ce5337e5f51cbb     --security-groups $security_group_id --region ap-southeast-2
+==== Installing the driver for the EBS volumes ====
+This one is used for local storage of pods.
+First we need to create the [[https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html|IAM service account]]:
-kubectl create secret generic counfluence-licence --from-literal=license-key='AAABkw0ODAoPeNp1kVtv2kAQhd/3V6zUl1SVk7VdMEZaqXTtJKS+pJgEWvGyWONg1SxoL1Dz67vYQ Wkq9Wln50hn5nzzoeAa56XGJMCuOx6E40GAWTTHHvF89AvaZ5Cq3gnqDgkJyMj3XcR2QvNSZ3wLl AtdK/5lv73eAip3orq2Sn0AqqUB9GhkueEKIq6Bnh0dlzhkhJK6BKEg/r2vZfuX6DokuNjHKa+b9 /4FyAPIaUS/LonnhD9y15lmo6GT5IM71PSe91xtaMqO7HaS+5NblhTp8WQeFstqPXtezIc3YbYoJ 5vv7eddfFeS+FP283SaLQft7EUbdW8eveOKrmifxa7GGQgNss9TmLUqZb3XZyJdxy5pZcFF+Z80n U9mtmuQefWkLE3quH230FyerSveKLgwsemSaVTEmZO4YeB7gRci+6PvO7l84cKC6RZ5yr5l+SLDr y9iEjrhX+rd0Ncx83YP3QFZnqbxjE0nyYXg5eIeiuAtrb1K1RiwOfHVGQvuuXxcjXF84I3pJqK3s sfzB435xiAwLAIUF8gBu/1jJlBJUuU4y8HE2ZD13CsCFEAiuAmbnfT0zBABubqJOz49P2Z6X02jf'
+Might need to append region like
+  eksctl utils associate-iam-oidc-provider --cluster $cluster_name --approve --region ap-southeast-2
+  eksctl create iamserviceaccount     --name ebs-csi-controller-sa     --namespace kube-system     --cluster atlassian-cluster     --role-name AmazonEKS_EBS_CSI_DriverRole     --role-only     --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy     --approve --region ap-southeast-2
-kubectl apply -f confluence-volume-local.yml
+You should now see new ebs pods in kube-system namespace:
-kubectl apply -f confluence-volume-shared.yml
-helm install confluence-test-install atlassian-data-center/confluence --values values.yaml
+  $ kubectl get pods -n kube-system | grep -i ebs
+  ebs-csi-controller-76f5dcb948-4fkbb   3/3     Running   0          31s
+  ebs-csi-controller-76f5dcb948-xxl7t   3/3     Running   0          31s
+  ebs-csi-node-45rhc                    3/3     Running   0          31s
+  ebs-csi-node-9xxhh                    3/3     Running   0          31s
-Check events for any warnings/errors
+==== Installing the driver for the EFS volumes ====
-kubectl describe pods
+This driver is used for the shared storage of pods.
-# Uninstall
+Run this replacing any values with yours:
-eksctl delete cluster -f eks-cluster-config.yml
+<code>
-Above will probably fail, you may need to go AWS UI and manually delete resources like VPCs, EFS etc.
+export cluster_name=atlassian-cluster
-helm uninstall ingress-nginx ingress-nginx/ingress-nginx --namespace ingress
+export role_name=AmazonEKS_EFS_CSI_DriverRole
-helm uninstall confluence-test-install
+eksctl create iamserviceaccount \
+    --name efs-csi-controller-sa \
+    --namespace kube-system \
+    --cluster $cluster_name \
+    --role-name $role_name \
+    --role-only \
+    --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEFSCSIDriverPolicy \
+    --approve --region ap-southeast-2
+eksctl create iamserviceaccount \
+    --name efs-csi-node-sa \
+    --namespace kube-system \
+    --cluster $cluster_name \
+    --role-name $role_name \
+    --role-only \
+    --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEFSCSIDriverPolicy \
+    --approve --region ap-southeast-2
+TRUST_POLICY=$(aws iam get-role --role-name $role_name --query 'Role.AssumeRolePolicyDocument' --output json | \
+    sed -e 's/efs-csi-controller-sa/efs-csi-*/' -e 's/StringEquals/StringLike/')
+aws iam update-assume-role-policy --role-name $role_name --policy-document "$TRUST_POLICY"
+</code>
+Now create the addon (replace your account id):
+  eksctl create addon --name aws-efs-csi-driver --cluster atlassian-cluster --service-account-role-arn arn:aws:iam::<account id>:role/AmazonEKS_EFS_CSI_DriverRole --force --region ap-southeast-2
-### http://aff805d3a15184e7b8454fca018feb75-639259050.ap-southeast-2.elb.amazonaws.com
+You should now see new efs pods in kube-system namespace:
+  $ kubectl get pods -n kube-system | grep -i efs
+  efs-csi-controller-76f5dcb948-4fkbb   3/3     Running   0          31s
+  efs-csi-controller-76f5dcb948-xxl7t   3/3     Running   0          31s
+  efs-csi-node-45rhc                    3/3     Running   0          31s
+  efs-csi-node-9xxhh                    3/3     Running   0          31s
+===== Create and apply storage configuration =====
-# Errors:
+Create/update the values in yaml files for [[https://atlassian.github.io/data-center-helm-charts/examples/storage/aws/LOCAL_STORAGE/|local]] and for [[https://atlassian.github.io/data-center-helm-charts/examples/storage/aws/SHARED_STORAGE/|shared]] storage:
-kubectl describe pods
+  kubectl apply -f confluence-volume-local.yml
+  kubectl apply -f confluence-volume-shared.yml
-Events:                                                                                                                                                                                                                                Type     Reason            Age                  From               Message                                                                                                                                                           ----     ------            ----                 ----               -------                                                                                                                                                           Warning  FailedScheduling  45m (x6 over 70m)    default-scheduler  0/2 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/2 nodes are available: 2 No preemption victims found for incoming pod..  Warning  FailedScheduling  4m1s (x11 over 44m)  default-scheduler  0/2 nodes are available: 2 Insufficient cpu. preemption: 0/2 nodes are available: 2 No preemption victims found for incoming pod..
-...
-Warning  FailedScheduling  8m57s (x2 over 18m)  default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
+===== Installation =====
+Create the licence secret. Use your license or a [[https://my.atlassian.com/products/index|use a trial one]]:
-# Additional steps
+  kubectl create secret generic counfluence-licence --from-literal=license-key='xxxxxxxxxxxxxxxxxxx'
-https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html
-Might need to append region like
-eksctl utils associate-iam-oidc-provider --cluster $cluster_name --approve --region ap-southeast-2
+<WRAP center round tip 60%>
+Use BX02-9YO1-IN86-LO5G for Server ID if generating the trial licence
+</WRAP>
-eksctl create iamserviceaccount     --name ebs-csi-controller-sa     --namespace kube-system     --cluster atlassian-cluster     --role-name AmazonEKS_EBS_CSI_DriverRole     --role-only     --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy     --approve --region ap-southeast-2
+Update the values.yml with this licence.
-https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html#adding-ebs-csi-eks-add-on
+Finally install the confluence:
-eksctl create addon --name aws-ebs-csi-driver --cluster atlassian-cluster --service-account-role-arn arn:aws:iam::855054226135:role/AmazonEKS_EBS_CSI_DriverRole --force --region ap-southeast-2
+  helm install confluence-test-install atlassian-data-center/confluence --values values.yaml
-$ kubectl get pods -n kube-system | grep ebs
+Check events for any warnings/errors with this:
-ebs-csi-controller-7cf5f8c56d-gpx59   6/6     Running   0          116s
+  kubectl describe pods
-ebs-csi-controller-7cf5f8c56d-nf922   6/6     Running   0          116s
-ebs-csi-node-mwltf                    3/3     Running   0          117s
-ebs-csi-node-ncvnc                    3/3     Running   0          117s
-## Error:
+===== Uninstall =====
-  Warning  Failed                  7m1s (x10 over 8m50s)  kubelet                  Error: secret "confluencedb" not found
-Uninstall and install helm chart again
+  eksctl delete cluster -f eks-cluster-config.yml
+Above will probably fail, you may need to go AWS UI and manually delete resources like VPCs, EFS etc.
+  helm uninstall ingress-nginx ingress-nginx/ingress-nginx --namespace ingress
+  helm uninstall confluence-test-install
+===== Errors =====
+Check with:
+  kubectl describe pods
+<code>
+Events:                                                                                                                                                                                                                                Type     Reason            Age                  From               Message                                                                                                                                                           ----     ------            ----                 ----               -------                                                                                                                                                           Warning  FailedScheduling  45m (x6 over 70m)    default-scheduler  0/2 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/2 nodes are available: 2 No preemption victims found for incoming pod..  Warning  FailedScheduling  4m1s (x11 over 44m)  default-scheduler  0/2 nodes are available: 2 Insufficient cpu. preemption: 0/2 nodes are available: 2 No preemption victims found for incoming pod..
+...
+Warning  FailedScheduling  8m57s (x2 over 18m)  default-scheduler  running PreBind plugin "VolumeBinding": binding volumes: timed out waiting for the condition
+</code>
+Make sure the EFS/EBS drivers above are installed correctly.
-# Can't connect to DB?
+==== Can't connect to DB? ====
+<code>
 helm test confluence-test-install -n default --logs
@@ Line 144: / Line 204: @@
       at DriverManager.getConnection (DriverManager.java:228)
       at (#5:1)
-Establishing connection to jdbc:mysql://confluencedb.chesfhqzuwnk.ap-southeast-2.rds.amazonaws.com/confluencedb
+Establishing connection to jdbc:mysql://confl.something.ap-southeast-2.rds.amazonaws.com/confluencedb
 Error: unable to get pod logs for confluence-test-install-shared-home-permissions-test: pods "confluence-test-install-shared-home-permissions-test" not found
+</code>
-We need to install the mysql driver, but before that we need another pod to mount the jar file see here
+We need to install the mysql driver, but before that we need another pod to mount the jar file
-https://atlassian.github.io/data-center-helm-charts/examples/external_libraries/EXTERNAL_LIBS/
+[[https://atlassian.github.io/data-center-helm-charts/examples/external_libraries/EXTERNAL_LIBS/|see here]]:
-kubectl apply -f shared-home-browser.yaml
+  kubectl apply -f shared-home-browser.yaml
-Now you might get an error like
+If you get an error like
-kubectl describe pod shared-home-browser
+<code>
+$ kubectl describe pod shared-home-browser
 Events:
   Type     Reason       Age                From               Message
@@ Line 159: / Line 221: @@
   Normal   Scheduled    85s                default-scheduler  Successfully assigned default/shared-home-browser to ip-192-168-15-3.ap-southeast-2.compute.internal
   Warning  FailedMount  21s (x8 over 84s)  kubelet            MountVolume.MountDevice failed for volume "confluence-shared-vol-pv" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name efs.csi.aws.com not found in the list of registered CSI drivers
+</code>
-This refers to the EFS driver not the EBS one above. We need to configure this.
+As it states make sure the EFS driver is installed.
-https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html
-```
-export cluster_name=atlassian-cluster
-export role_name=AmazonEKS_EFS_CSI_DriverRole
-eksctl create iamserviceaccount \
-    --name efs-csi-controller-sa \
-    --namespace kube-system \
-    --cluster $cluster_name \
-    --role-name $role_name \
-    --role-only \
-    --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEFSCSIDriverPolicy \
-    --approve --region ap-southeast-2
-eksctl create iamserviceaccount \
-    --name efs-csi-node-sa \
-    --namespace kube-system \
-    --cluster $cluster_name \
-    --role-name $role_name \
-    --role-only \
-    --attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEFSCSIDriverPolicy \
-    --approve --region ap-southeast-2
-TRUST_POLICY=$(aws iam get-role --role-name $role_name --query 'Role.AssumeRolePolicyDocument' --output json | \
-    sed -e 's/efs-csi-controller-sa/efs-csi-*/' -e 's/StringEquals/StringLike/')
-aws iam update-assume-role-policy --role-name $role_name --policy-document "$TRUST_POLICY"
-```
-eksctl create addon --name aws-efs-csi-driver --cluster atlassian-cluster --service-account-role-arn arn:aws:iam::855054226135:role/AmazonEKS_EFS_CSI_DriverRole --force --region ap-southeast-2
-kubectl get pods -n kube-system | grep -i efs
-efs-csi-controller-76f5dcb948-4fkbb   3/3     Running   0          31s
-efs-csi-controller-76f5dcb948-xxl7t   3/3     Running   0          31s
-efs-csi-node-45rhc                    3/3     Running   0          31s
-efs-csi-node-9xxhh                    3/3     Running   0          31s
 Then continue
-kubectl exec -it shared-home-browser -- bash -c "mkdir -p /shared-home/libraries"
+  kubectl exec -it shared-home-browser -- bash -c "mkdir -p /shared-home/libraries"
-wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-j-8.1.0.tar.gz && tar xf mysql-connector-j-8.1.0.tar.gz && \
+  wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-j-8.1.0.tar.gz && tar xf mysql-connector-j-8.1.0.tar.gz && kubectl cp $PWD/mysql-connector-j-8.1.0/mysql-connector-j-8.1.0.jar shared-home-browser:/shared-home/libraries
-kubectl cp $PWD/mysql-connector-j-8.1.0/mysql-connector-j-8.1.0.jar shared-home-browser:/shared-home/libraries
+----
+<code>
-# Error
 No matching mount target in the az ap-southeast-2c. Please create one mount target in ap-southeast-2c, or try the mount target in another AZ by passing the availability zone name option. Available mount target(s) are in az ['ap-southeast-2b', 'ap-southeast-2a']
+</code>
+Go to //Amazon EFS > File systems > select the EFS > Network tab > Manage// and add mount target in the missin az (ap-southeast-2c in this instance)
-Go to Amazon EFS > File systems > select the EFS > Network tab > Manage and add mount target in the missin az (ap-southeast-2c in this instance)
-# Error
+----
+<code>
 Setup Database Failed
@@ Line 219: / Line 247: @@
     Configuring database failed
     StatementCallback; uncategorized SQLException for SQL [CREATE TRIGGER denormalised_space_trigger_on_update AFTER UPDATE ON SPACES FOR EACH ROW sp: BEGIN DECLARE isServiceDisabled BOOL DEFAULT TRUE; CALL space_procedure_for_denormalised_permissions(isServiceDisabled); IF (isServiceDisabled) THEN LEAVE sp; END IF; IF (NEW.LOWERSPACEKEY = OLD.LOWERSPACEKEY) THEN LEAVE sp; END IF; INSERT INTO DENORMALISED_SPACE_CHANGE_LOG(SPACE_ID) VALUES (NEW.SPACEID); END;]; SQL state [HY000]; error code [1419]; You do not have the SUPER privilege and binary logging is enabled (you *might* want to use the less safe log_bin_trust_function_creators variable); nested exception is java.sql.SQLException: You do not have the SUPER privilege and binary logging is enabled (you *might* want to use the less safe log_bin_trust_function_creators variable)
+</code>
+And
+<code>
-Solution: do this > https://stackoverflow.com/a/30874794/6881647
-# Error
 Setup Database Failed
@@ Line 234: / Line 258: @@
     Configuring database failed
     MySQL session isolation level 'REPEATABLE-READ' is no longer supported. Session isolation level must be 'READ-COMMITTED'. See http://confluence.atlassian.com/x/GAtmDg
+</code>
-In values.yml add ?sessionVariables=transaction_isolation='READ-COMMITTED'
-  url: "jdbc:mysql://confluencedb.chesfhqzuwnk.ap-southeast-2.rds.amazonaws.com/confluencedb?sessionVariables=transaction_isolation='READ-COMMITTED'"
-also add in Paramters group transaction_isolation='READ-COMMITTED' in RDS configuration and reboot
+Make sure you added the correct options in parameters group in [[wiki:confluence_aws_deploy#configure_database]] step.
@@ Line 245: / Line 268: @@
 ====== See also ======
+  * [[wiki:confluence_troubleshooting|Confluence troubleshooting]]
+  * [[wiki:installing_confluence_server|Installing Confluence server]]
+  * [[wiki:installing_confluence_server_in_docker_behind_reverse_proxy|Installing Confluence server in Docker behind reverse proxy]]
 ====== References ======
@@ Line 250: / Line 276: @@
   * https://confluence.atlassian.com/confkb/confluence-fails-to-start-and-throws-mysql-session-isolation-level-repeatable-read-is-no-longer-supported-error-241568536.html
   * https://stackoverflow.com/a/30874794/6881647
+  * https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html
+  * https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html
+  * https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html#adding-ebs-csi-eks-add-on
+  * https://atlassian.github.io/data-center-helm-charts/examples/external_libraries/EXTERNAL_LIBS/
+  * https://stackoverflow.com/a/30874794/6881647
+  * https://community.atlassian.com/t5/Confluence-questions/Install-Confluence-on-AWS-via-Kubernetes/qaq-p/2499972