User Tools

Site Tools


wiki:confluence_aws_deploy

This is an old revision of the document!


Confluence AWS deploy

Install prerequisite tooling first.

Install Helm

helm repo add atlassian-data-center \

https://atlassian.github.io/data-center-helm-charts

helm repo update

Install cluster

Use a more powerful VM in the config first otherwise the pods will complain about the lack of CPU, something like c5.xlarge then run:

eksctl create cluster -f eks-cluster-config.yml

Obtain values.yaml

helm show values atlassian-data-center/<product> > values.yaml

Configure database

Create AWS RDS mysql 8.0 database. Now we need to set some parameters for mysql.

  1. Open the RDS web console.
  2. Open the “Parameter Groups” tab.
  3. Create a new Parameter Group. On the dialog, select the MySQL family compatible to your MySQL database version, give it a name and confirm. Select the just created Parameter Group and issue “Edit Parameters”.
  4. Look for the parameter log_bin_trust_function_creators and set its value to 1.
  5. Look for the parameter transaction_isolation and set it to 'READ-COMMITTED'
  6. Save the changes.
  7. Open the “Instances” tab. Expand your MySQL instance and issue the “Instance Action” named “Modify”.
  8. Select the just created Parameter Group and enable “Apply Immediately”.
  9. Click on “Continue” and confirm the changes.
  10. Wait for the “Modifying” operation to be completed.
  11. Again, open the “Instances” tab. Expand your MySQL instance and expand “Instance Action” tab and select “Reboot”.

Connect to the database (use the URL from the console) and create the DB:

CREATE DATABASE confluencedb CHARACTER SET utf8mb4 COLLATE utf8mb4_bin;
CREATE USER 'confluenceusr'@'%' IDENTIFIED BY 'somelongpassword';
GRANT ALL PRIVILEGES ON confluencedb.* TO 'confluenceusr'@'%';

Also create kubernetes secret and update values.yml with it:

kubectl create secret generic counfluencedb --from-literal=username='user' --from-literal=password='pwd'

Configure Ingress

kubectl create namespace ingress && \
helm install ingress-nginx ingress-nginx/ingress-nginx --namespace ingress && \

Configure persistent storage

vpc_id=$(aws eks describe-cluster \ –name atlassian-cluster \ –query “cluster.resourcesVpcConfig.vpcId” \ –output text \ –region ap-southeast-2) && \ cidr_range=$(aws ec2 describe-vpcs \ –vpc-ids $vpc_id \ –query “Vpcs[].CidrBlock” \ –output text \ –region ap-southeast-2) && \ security_group_id=$(aws ec2 create-security-group \ –group-name confluence_efs \ –description “confluence EFS security group” \ –vpc-id $vpc_id \ –output text \ –region ap-southeast-2) && \ aws ec2 authorize-security-group-ingress –group-id $security_group_id –protocol tcp –port 2049 –cidr $cidr_range –region ap-southeast-2 && \ file_system_id=$(aws efs create-file-system \ –region ap-southeast-2 \ –performance-mode generalPurpose \ –query 'FileSystemId' \ –output text)

# configure shared and local volumes → https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/docs/efs-create-filesystem.md aws ec2 describe-subnets –filters “Name=vpc-id,Values=$vpc_id” –query 'Subnets[*].{SubnetId: SubnetId,AvailabilityZone: AvailabilityZone,CidrBlock: CidrBlock}' –output table –region ap-southeast-2

kubectl get nodes

aws efs create-mount-target –file-system-id $file_system_id –subnet-id subnet-035ce5337e5f51cbb –security-groups $security_group_id –region ap-southeast-2

kubectl create secret generic counfluence-licence –from-literal=license-key='AAABkw0ODAoPeNp1kVtv2kAQhd/3V6zUl1SVk7VdMEZaqXTtJKS+pJgEWvGyWONg1SxoL1Dz67vYQ Wkq9Wln50hn5nzzoeAa56XGJMCuOx6E40GAWTTHHvF89AvaZ5Cq3gnqDgkJyMj3XcR2QvNSZ3wLl AtdK/5lv73eAip3orq2Sn0AqqUB9GhkueEKIq6Bnh0dlzhkhJK6BKEg/r2vZfuX6DokuNjHKa+b9 /4FyAPIaUS/LonnhD9y15lmo6GT5IM71PSe91xtaMqO7HaS+5NblhTp8WQeFstqPXtezIc3YbYoJ 5vv7eddfFeS+FP283SaLQft7EUbdW8eveOKrmifxa7GGQgNss9TmLUqZb3XZyJdxy5pZcFF+Z80n U9mtmuQefWkLE3quH230FyerSveKLgwsemSaVTEmZO4YeB7gRci+6PvO7l84cKC6RZ5yr5l+SLDr y9iEjrhX+rd0Ncx83YP3QFZnqbxjE0nyYXg5eIeiuAtrb1K1RiwOfHVGQvuuXxcjXF84I3pJqK3s sfzB435xiAwLAIUF8gBu/1jJlBJUuU4y8HE2ZD13CsCFEAiuAmbnfT0zBABubqJOz49P2Z6X02jf'

kubectl apply -f confluence-volume-local.yml kubectl apply -f confluence-volume-shared.yml

helm install confluence-test-install atlassian-data-center/confluence –values values.yaml

Check events for any warnings/errors kubectl describe pods

# Uninstall eksctl delete cluster -f eks-cluster-config.yml Above will probably fail, you may need to go AWS UI and manually delete resources like VPCs, EFS etc. helm uninstall ingress-nginx ingress-nginx/ingress-nginx –namespace ingress helm uninstall confluence-test-install

### http://aff805d3a15184e7b8454fca018feb75-639259050.ap-southeast-2.elb.amazonaws.com

# Errors: kubectl describe pods

Events: Type Reason Age From Message —- —— —- —- ——- Warning FailedScheduling 45m (x6 over 70m) default-scheduler 0/2 nodes are available: pod has unbound immediate PersistentVolumeClaims. preemption: 0/2 nodes are available: 2 No preemption victims found for incoming pod.. Warning FailedScheduling 4m1s (x11 over 44m) default-scheduler 0/2 nodes are available: 2 Insufficient cpu. preemption: 0/2 nodes are available: 2 No preemption victims found for incoming pod.. … Warning FailedScheduling 8m57s (x2 over 18m) default-scheduler running PreBind plugin “VolumeBinding”: binding volumes: timed out waiting for the condition

# Additional steps https://docs.aws.amazon.com/eks/latest/userguide/enable-iam-roles-for-service-accounts.html Might need to append region like eksctl utils associate-iam-oidc-provider –cluster $cluster_name –approve –region ap-southeast-2

eksctl create iamserviceaccount –name ebs-csi-controller-sa –namespace kube-system –cluster atlassian-cluster –role-name AmazonEKS_EBS_CSI_DriverRole –role-only –attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEBSCSIDriverPolicy –approve –region ap-southeast-2

https://docs.aws.amazon.com/eks/latest/userguide/managing-ebs-csi.html#adding-ebs-csi-eks-add-on eksctl create addon –name aws-ebs-csi-driver –cluster atlassian-cluster –service-account-role-arn arn:aws:iam::855054226135:role/AmazonEKS_EBS_CSI_DriverRole –force –region ap-southeast-2

$ kubectl get pods -n kube-system | grep ebs ebs-csi-controller-7cf5f8c56d-gpx59 6/6 Running 0 116s ebs-csi-controller-7cf5f8c56d-nf922 6/6 Running 0 116s ebs-csi-node-mwltf 3/3 Running 0 117s ebs-csi-node-ncvnc 3/3 Running 0 117s

## Error:

Warning  Failed                  7m1s (x10 over 8m50s)  kubelet                  Error: secret "confluencedb" not found                                                                                                            

Uninstall and install helm chart again

# Can't connect to DB? helm test confluence-test-install -n default –logs

POD LOGS: confluence-test-install-db-connectivity-test Oct 15, 2023 2:20:00 PM java.util.prefs.FileSystemPreferences$1 run INFO: Created user preferences directory. Exception java.sql.SQLException: No suitable driver found for jdbc:mysql:confluencedb.chesfhqzuwnk.ap-southeast-2.rds.amazonaws.com/confluencedb at DriverManager.getConnection (DriverManager.java:702) at DriverManager.getConnection (DriverManager.java:228) at (#5:1) Establishing connection to jdbc:mysql:confluencedb.chesfhqzuwnk.ap-southeast-2.rds.amazonaws.com/confluencedb

Error: unable to get pod logs for confluence-test-install-shared-home-permissions-test: pods “confluence-test-install-shared-home-permissions-test” not found

We need to install the mysql driver, but before that we need another pod to mount the jar file see here https://atlassian.github.io/data-center-helm-charts/examples/external_libraries/EXTERNAL_LIBS/ kubectl apply -f shared-home-browser.yaml

Now you might get an error like kubectl describe pod shared-home-browser Events:

Type     Reason       Age                From               Message
----     ------       ----               ----               -------
Normal   Scheduled    85s                default-scheduler  Successfully assigned default/shared-home-browser to ip-192-168-15-3.ap-southeast-2.compute.internal
Warning  FailedMount  21s (x8 over 84s)  kubelet            MountVolume.MountDevice failed for volume "confluence-shared-vol-pv" : kubernetes.io/csi: attacher.MountDevice failed to create newCsiDriverClient: driver name efs.csi.aws.com not found in the list of registered CSI drivers

This refers to the EFS driver not the EBS one above. We need to configure this. https://docs.aws.amazon.com/eks/latest/userguide/efs-csi.html ``` export cluster_name=atlassian-cluster export role_name=AmazonEKS_EFS_CSI_DriverRole eksctl create iamserviceaccount \

  1. -name efs-csi-controller-sa \
  2. -namespace kube-system \
  3. -cluster $cluster_name \
  4. -role-name $role_name \
  5. -role-only \
  6. -attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEFSCSIDriverPolicy \
  7. -approve –region ap-southeast-2

eksctl create iamserviceaccount \

  1. -name efs-csi-node-sa \
  2. -namespace kube-system \
  3. -cluster $cluster_name \
  4. -role-name $role_name \
  5. -role-only \
  6. -attach-policy-arn arn:aws:iam::aws:policy/service-role/AmazonEFSCSIDriverPolicy \
  7. -approve –region ap-southeast-2

TRUST_POLICY=$(aws iam get-role –role-name $role_name –query 'Role.AssumeRolePolicyDocument' –output json | \

  sed -e 's/efs-csi-controller-sa/efs-csi-*/' -e 's/StringEquals/StringLike/')

aws iam update-assume-role-policy –role-name $role_name –policy-document “$TRUST_POLICY” ```

eksctl create addon –name aws-efs-csi-driver –cluster atlassian-cluster –service-account-role-arn arn:aws:iam::855054226135:role/AmazonEKS_EFS_CSI_DriverRole –force –region ap-southeast-2

kubectl get pods -n kube-system | grep -i efs efs-csi-controller-76f5dcb948-4fkbb 3/3 Running 0 31s efs-csi-controller-76f5dcb948-xxl7t 3/3 Running 0 31s efs-csi-node-45rhc 3/3 Running 0 31s efs-csi-node-9xxhh 3/3 Running 0 31s

Then continue kubectl exec -it shared-home-browser – bash -c “mkdir -p /shared-home/libraries”

wget https://dev.mysql.com/get/Downloads/Connector-J/mysql-connector-j-8.1.0.tar.gz && tar xf mysql-connector-j-8.1.0.tar.gz && \ kubectl cp $PWD/mysql-connector-j-8.1.0/mysql-connector-j-8.1.0.jar shared-home-browser:/shared-home/libraries

# Error No matching mount target in the az ap-southeast-2c. Please create one mount target in ap-southeast-2c, or try the mount target in another AZ by passing the availability zone name option. Available mount target(s) are in az ['ap-southeast-2b', 'ap-southeast-2a']

Go to Amazon EFS > File systems > select the EFS > Network tab > Manage and add mount target in the missin az (ap-southeast-2c in this instance)

# Error

Setup Database Failed

The following error(s) occurred:

  Configuring database failed
  StatementCallback; uncategorized SQLException for SQL [CREATE TRIGGER denormalised_space_trigger_on_update AFTER UPDATE ON SPACES FOR EACH ROW sp: BEGIN DECLARE isServiceDisabled BOOL DEFAULT TRUE; CALL space_procedure_for_denormalised_permissions(isServiceDisabled); IF (isServiceDisabled) THEN LEAVE sp; END IF; IF (NEW.LOWERSPACEKEY = OLD.LOWERSPACEKEY) THEN LEAVE sp; END IF; INSERT INTO DENORMALISED_SPACE_CHANGE_LOG(SPACE_ID) VALUES (NEW.SPACEID); END;]; SQL state [HY000]; error code [1419]; You do not have the SUPER privilege and binary logging is enabled (you *might* want to use the less safe log_bin_trust_function_creators variable); nested exception is java.sql.SQLException: You do not have the SUPER privilege and binary logging is enabled (you *might* want to use the less safe log_bin_trust_function_creators variable)

Solution: do this > https://stackoverflow.com/a/30874794/6881647

# Error

Setup Database Failed

The following error(s) occurred:

  Configuring database failed
  MySQL session isolation level 'REPEATABLE-READ' is no longer supported. Session isolation level must be 'READ-COMMITTED'. See http://confluence.atlassian.com/x/GAtmDg

In values.yml add ?sessionVariables=transaction_isolation='READ-COMMITTED'

url: "jdbc:mysql://confluencedb.chesfhqzuwnk.ap-southeast-2.rds.amazonaws.com/confluencedb?sessionVariables=transaction_isolation='READ-COMMITTED'"

also add in Paramters group transaction_isolation='READ-COMMITTED' in RDS configuration and reboot

Tested on

  • AWS 2023.10.16

See also

References

wiki/confluence_aws_deploy.1697472520.txt.gz · Last modified: 2023/10/16 16:08 by antisa

Except where otherwise noted, content on this wiki is licensed under the following license: CC0 1.0 Universal
CC0 1.0 Universal Donate Powered by PHP Valid HTML5 Valid CSS Driven by DokuWiki