3be5e5e965c1557c718c2c200ceaf0870db7e822
wiki/info/landscape/paris2024/olympic-setup.md
| ... | ... | @@ -58,6 +58,34 @@ replication: |
| 58 | 58 | |
| 59 | 59 | For "Paris 2024" we configured yet another MongoDB replica set that consisted only of the two on-site nodes and where we stored the backup copy of the ``security_service`` database. We should, however, be able to store the ``security_service`` DB backup in the same replica set of which the two local nodes with their MongoDB processes listening on ports ``10201/10202``. The ``security_service`` database is used as the target for a backup script for the ``security_service`` database. See below. We increased the priority of the ``sap-p1-1`` node from 1 to 2. |
| 60 | 60 | |
| 61 | +For log rotation, the following file must be created at ``/etc/logrotate.d/mongodb``: |
|
| 62 | + |
|
| 63 | +``` |
|
| 64 | +compress |
|
| 65 | +/var/log/mongodb/mongod.log |
|
| 66 | +{ |
|
| 67 | + rotate 5 |
|
| 68 | + weekly |
|
| 69 | + postrotate |
|
| 70 | + /usr/bin/killall -SIGUSR1 mongod |
|
| 71 | + endscript |
|
| 72 | +} |
|
| 73 | +``` |
|
| 74 | + |
|
| 75 | +and likewise, if a second MongoDB replica set is running producing, e.g., ``/var/log/mongodb/mongod-security-service.log`` then you need to add a second file, e.g., at ``/etc/logrotate.d/mongodb-security-service`` like this: |
|
| 76 | + |
|
| 77 | +``` |
|
| 78 | +compress |
|
| 79 | +/var/log/mongodb/mongod-security-service.log |
|
| 80 | +{ |
|
| 81 | + rotate 5 |
|
| 82 | + weekly |
|
| 83 | + postrotate |
|
| 84 | + /usr/bin/killall -SIGUSR1 mongod |
|
| 85 | + endscript |
|
| 86 | +} |
|
| 87 | +``` |
|
| 88 | + |
|
| 61 | 89 | ### User Accounts |
| 62 | 90 | |
| 63 | 91 | The essential user account on both laptops is ``sailing``. The account is intended to be used for running the Java VM that executes the SAP Sailing Analytics server software. The account is currently still protected by a password that our on-site team should know. On both laptops the ``sailing`` account has a password-less SSH key installed under ``/home/sailing/.ssh`` that is contained in the ``known_hosts`` file of ``paris-ssh.sapsailing.com`` as well as the mutually other P1 laptop. This way, all tunnels can easily be created once logged on to this ``sailing`` account. |
| ... | ... | @@ -90,10 +118,14 @@ TracTrac Dev Jorge (Linux) 10.1.3.228 10.8.0.135 |
| 90 | 118 | TracTrac Dev Chris (Linux) 10.1.3.233 10.8.0.136 |
| 91 | 119 | ``` |
| 92 | 120 | |
| 93 | -The OpenVPN connection is set up with the GUI of the Linux Desktop. Therefore the management is done through Network Manager. Network Manager has a CLI, ``nmcli``. With that more properties of connections can be modified. The ``connection.secondaries`` property defines the UUID of a connection that will be established as soon as the initial connection is working. With ``nmcli connection show`` you will get the list of connections with the corresponding UUIDs. For the Medemblik Event the OpenVPN connection to the A server is bound to the wired interface that is used with |
|
| 121 | +The OpenVPN connection is set up with the GUI of the Linux Desktop. Therefore the management is done through Network Manager. Network Manager has a CLI, ``nmcli``. With that more properties of connections can be modified. The ``connection.secondaries`` property defines the UUID of a connection that will be established as soon as the initial connection is working. With ``nmcli connection show`` you will get the list of connections with the corresponding UUIDs. For the Medemblik Event the OpenVPN connection to the A server is bound to the wired interface and made "persistent" (meaning it will retry connecting after being disconnected) that is used with |
|
| 94 | 122 | |
| 95 | 123 | ``` |
| 96 | 124 | sudo nmcli connection modify <Wired Connection 2> +connection.secondaries <UUID-of-OpenVPN-A> |
| 125 | +nmcli connection modify <Name-of-OpenVPN-A> connection.autoconnect-retries 0 |
|
| 126 | +nmcli connection modify <Name-of-OpenVPN-A> vpn.persistent yes |
|
| 127 | +nmcli connection modify <Name-of-OpenVPN-B> connection.autoconnect-retries 0 |
|
| 128 | +nmcli connection modify <Name-of-OpenVPN-B> vpn.persistent yes |
|
| 97 | 129 | ``` |
| 98 | 130 | |
| 99 | 131 | For the OpenVPN connections we have received two alternative configuration files together with keys and certificates for our server and work laptops, as well as the certificates for the OpenVPN server (``ca.crt``, ``dh.pem``, ``pfs.key``). The "A" configuration, e.g., provided in a file named ``st-soft-aws_A.ovpn``, looks like this: |
| ... | ... | @@ -364,6 +396,8 @@ The ``monitor-disk-usage`` script checks the partition holding ``/var/lib/mongod |
| 364 | 396 | |
| 365 | 397 | On ``sap-p1-2`` we run a script ``compare-secondary-to-primary-master`` every five minutes which basically does a ``compareServers -ael`` which uses the REST API for comparing server contents. If a difference is reported by the tool then an e-mail notification is sent out to the list of operators. |
| 366 | 398 | |
| 399 | +On ``paris-ssh.sapsailing.com`` we run ``monitor-on-site-servers`` every minute. Like other scripts it is installed as a symbolic link from ``/usr/local/bin`` to ``/root/code/configuration/on-site-scripts/paris2024/paris-ssh`` and that is called by a cron job from the ``root`` user's ``crontab`` file. The file checks that the primary and secondary master can be reached through HTTPS (localhost port 9443/9444, respectively), and the primary through HTTP, port 8888. This verifies that the SSH tunnels from the on-site laptops to the ``paris-ssh`` jump host are in place, up and running. Indirectly, this also verifies that the OpenVPN connections on the on-site laptops are working alright. |
|
| 400 | + |
|
| 367 | 401 | ### Time Synchronizing |
| 368 | 402 | Setup chronyd service on desktop machine, in order to regurlary connect via VPN and relay the time towards the two P1s. Added |
| 369 | 403 | ``` |