updated paris2024 documentation in Wiki

#!/bin/bash

# Upgrades the entire landscape of servers to a new release ${RELEASE}

# The procedure works in the following steps:

# their INSTALL_FROM_RELEASE points to the new ${RELEASE}

# - Install new releases to sap-p1-1:servers/[master|security_service] and sap-p1-2:servers/[replica|master|security_service]

# - Update all launch configurations and auto-scaling groups in the cloud (update-launch-configuration.sh)

echo "Example: $0 -R build-202106041327 -k Jan"

echo "The procedure works in the following steps:"

echo " their INSTALL_FROM_RELEASE points to the new ${RELEASE}"

echo " - Update all launch configurations and auto-scaling groups in the cloud (update-launch-configuration.sh)"

echo " - Tell all replicas in the cloud to stop replicating (stop-all-cloud-replicas.sh)"

echo " - Tell sap-p1-2 to stop replicating"

echo " * Patching configurations on sap-p1-1 and sap-p1-2 to new release ${RELEASE} and installing"

patch_conf_and_install sap-p1-1 master

patch_conf_and_install sap-p1-1 security_service

patch_conf_and_install sap-p1-2 replica

patch_conf_and_install sap-p1-2 master

patch_conf_and_install sap-p1-2 security_service

So the essential change is that the reverse forward from ``paris-ssh.sapsailing.com:8888`` now targets ``sap-p1-2:8888`` where we now assume the failover master to be running.

When the Internet connection fails, replicating the security service from ``security-service.sapsailing.com`` / ``rabbit.internal.sapsailing.com`` will no longer be possible. Neither will outbound replication to ``rabbit-eu-west-3.sapsailing.com`` be possible, and cloud replicas won't be able to reach the on-site master anymore through the ``paris-ssh.sapsailing.com:8888`` reverse port forward. This also has an effect on the local on-site replica which no longer will be able to reach ``rabbit-eu-west-3.sapsailing.com`` which provides the on-site replica with the operation stream under regular circumstances.

}

```

### Backup

borgbackup is used to backup the ``/`` folder of both laptops towards the other machine. Folder where the borg repository is located is: ``/backup``.

Our "favorite" Availability Zone (AZ) in eu-west-3 is "1a" / "eu-west-3a".

For RabbitMQ we run a separate host, based on AWS Ubuntu 20. It brings the ``rabbitmq-server`` package with it (version 3.8.2 on Erlang 22.2.7), and we'll install it with default settings, except for the following change: In the new file ``/etc/rabbitmq/rabbitmq.conf`` we enter the line

### Replicas

```

rm env.sh; cat replica.conf | ./refreshInstance auto-install-from-stdin

(Adjust the release accordingly, of course). (NOTE: During the first production days of the event we noticed that it was really a BAD IDEA to have all replicas use the same DB set-up, all writing to the MongoDB PRIMARY of the "live" replica set in eu-west-1. With tens of replicas running concurrently, this led to a massive block-up based on MongoDB not writing fast enough. This gave rise to a new application server AMI which now has a MongoDB set-up included, using "replica" as the MongoDB replica set name. Now, each replica hence can write into its own MongoDB instance, isolated from all others and scaling linearly.)

```

INSTALL_FROM_RELEASE=build-202306271444