f0ea1a007d97170a152e34c36ae88661c2f6714d
wiki/info/landscape/creating-ec2-image-from-scratch.md
| ... | ... | @@ -1,178 +1,7 @@ |
| 1 | 1 | # Creating an Amazon AWS EC2 Image from Scratch |
| 2 | 2 | |
| 3 | -I started out with a clean "Amazon Linux AMI 2023" image from Amazon with a single 100GB root volume. The root/system volume I left as is, to start with. This requires having access to a user key that can be selected when launching the image. |
|
| 3 | +I started out with a clean "Amazon Linux 2" image from Amazon with a single 100GB root volume and the "Sailing Analytics App" security group. The root/system volume I left as is, to start with. This requires having access to a user key that can be selected when launching the image. Then I ran the script ``configuration/sailing_server_setup/setup-sailing-server.sh`` with the instance's external IP address as an argument. This installs everything needed, so in order to understand what happens during this process, review the script. In short, it installs a few packages using the `yum` package manager, downloads and installs the SAP JVM 8 in its latest version into ``/opt/sapjvm_8``, installs a few systemd service units that check for and then activate NVMe swap space where available and interpret the EC2 user data after boot. The MongoDB environment that is being installed is configured to be a replica set named ``replica``, but initialization is left to the ``sailing.service``. See the ``configuration/sailing`` script for the post-boot configuration, installed as a service (see ``configuration/sailing_server_setup/sailing.service``). |
|
| 4 | 4 | |
| 5 | -Add a ``sailing`` user / group. Under that user account, clone ``ssh://trac@sapsailing.com/home/trac/git`` to ``/home/sailing/code``. |
|
| 5 | +When the script finishes, you can shut down / stop the instance, create an AMI and tag the AMI as well as the root volume's snapshot, e.g., as "SAP Sailing Analytics 2.0" and "SAP Sailing Analytics 2.0 (Root)", respectively. |
|
| 6 | 6 | |
| 7 | -Under ``/usr/local/bin`` install the following: |
|
| 8 | -``` |
|
| 9 | -lrwxrwxrwx 1 root root 56 Oct 20 09:20 cp_root_mail_properties -> /home/sailing/code/configuration/cp_root_mail_properties |
|
| 10 | --rwxr-xr-x 1 root root 24707072 Jan 30 2022 docker-compose |
|
| 11 | -lrwxrwxrwx 1 root root 71 May 10 2021 getLatestImageOfType.sh -> /home/sailing/code/configuration/aws-automation/getLatestImageOfType.sh |
|
| 12 | -lrwxrwxrwx 1 root root 50 Mar 23 2021 launchhudsonslave -> /home/sailing/code/configuration/launchhudsonslave |
|
| 13 | -lrwxrwxrwx 1 root root 57 Mar 23 2021 launchhudsonslave-java11 -> /home/sailing/code/configuration/launchhudsonslave-java11 |
|
| 14 | -lrwxrwxrwx 1 root root 69 Jun 1 2019 mountnvmeswap -> /home/sailing/code/configuration/archive_instance_setup/mountnvmeswap |
|
| 15 | -lrwxrwxrwx 1 root root 78 Jan 27 2021 update_authorized_keys_for_landscape_managers -> /home/sailing/code/configuration/update_authorized_keys_for_landscape_managers |
|
| 16 | -lrwxrwxrwx 1 root root 89 Feb 4 2021 update_authorized_keys_for_landscape_managers_if_changed -> /home/sailing/code/configuration/update_authorized_keys_for_landscape_managers_if_changed |
|
| 17 | -``` |
|
| 18 | - |
|
| 19 | -Enable the EPEL repository by issuing `yum-config-manager --enable epel/x86_64` or `sudo amazon-linux-extras install epel -y`. |
|
| 20 | - |
|
| 21 | -I then did a `yum update` and added the following packages: |
|
| 22 | - |
|
| 23 | - - httpd |
|
| 24 | - - mod_proxy_html |
|
| 25 | - - tmux |
|
| 26 | - - nfs-utils |
|
| 27 | - - chrony |
|
| 28 | - - libstdc++48.i686 (for Android builds) |
|
| 29 | - - glibc.i686 (for Android builds) |
|
| 30 | - - libzip.i686 (for Android builds) |
|
| 31 | - - telnet |
|
| 32 | - - apachetop |
|
| 33 | - - goaccess |
|
| 34 | - - postfix (for sending e-mail, e.g., to invite competitors and buoy pingers) |
|
| 35 | - - tigervnc-server |
|
| 36 | - - WindowMaker |
|
| 37 | - - xterm |
|
| 38 | - - sendmail-cf |
|
| 39 | - |
|
| 40 | -I copied the JDK7/JDK8 installations, particularly the current sapjvm_8 VM, from an existing SL instance to /opt (using scp). |
|
| 41 | - |
|
| 42 | -In order to be able to connect to AWS DocumentDB instances, the corresponding certificate must be installed into the JVM's certificate store (2 separate commands): |
|
| 43 | - |
|
| 44 | -``` |
|
| 45 | - wget -O /tmp/rds.pem https://s3.amazonaws.com/rds-downloads/rds-combined-ca-bundle.pem |
|
| 46 | - /opt/sapjvm_8/bin/keytool -importcert -alias AWSRDS -file /tmp/rds.pem -keystore /opt/sapjvm_8/jre/lib/security/cacerts -noprompt -storepass changeit |
|
| 47 | -``` |
|
| 48 | - |
|
| 49 | -A latest MongoDB shell is installed by the following: |
|
| 50 | - |
|
| 51 | -``` |
|
| 52 | -cat << EOF >/etc/yum.repos.d/mongodb-org.4.4.repo |
|
| 53 | -[mongodb-org-4.4] |
|
| 54 | -name=MongoDB Repository |
|
| 55 | -baseurl=https://repo.mongodb.org/yum/amazon/2023/mongodb-org/4.4/x86_64/ |
|
| 56 | -gpgcheck=1 |
|
| 57 | -enabled=1 |
|
| 58 | -gpgkey=https://www.mongodb.org/static/pgp/server-4.4.asc |
|
| 59 | -EOF |
|
| 60 | - |
|
| 61 | -yum update |
|
| 62 | -yum install mongodb-org-server mongodb-org-shell mongodb-org-tools |
|
| 63 | -``` |
|
| 64 | - |
|
| 65 | -Then I created a mount point /home/sailing and copied the following lines from the /etc/fstab file from an existing SL instance: |
|
| 66 | - |
|
| 67 | -``` |
|
| 68 | -UUID=a1d96e53-233f-4e44-b865-c78b862df3b8 /home/sailing ext4 defaults,noatime,commit=30 0 0 |
|
| 69 | -UUID=7d7e68a3-27a1-49ef-908f-a6ebadcc55bb none swap sw 0 0 |
|
| 70 | - |
|
| 71 | -# Mount the Android SDK from the Build/Dev box; use a timeout of 10s (100ds) |
|
| 72 | -172.31.28.17:/home/hudson/android-sdk-linux /opt/android-sdk-linux nfs tcp,intr,timeo=100,retry=0 |
|
| 73 | -172.31.18.15:/var/log/old /var/log/old nfs tcp,intr,timeo=100,retry=0 |
|
| 74 | -``` |
|
| 75 | - |
|
| 76 | -This will mount the swap space partition as well as the /home/sailing partition, /var/log/old and the Android SDK stuff required for local builds. |
|
| 77 | -Do the following steps (until it says otherwise) without logging out in between them: |
|
| 78 | -In `/etc/ssh/sshd_config` I commented the line |
|
| 79 | - |
|
| 80 | -``` |
|
| 81 | -# Only allow root to run commands over ssh, no shell |
|
| 82 | -#PermitRootLogin forced-commands-only |
|
| 83 | -``` |
|
| 84 | - |
|
| 85 | -and added the lines |
|
| 86 | - |
|
| 87 | -``` |
|
| 88 | -PermitRootLogin without-password |
|
| 89 | -PermitRootLogin Yes |
|
| 90 | -MaxStartups 100 |
|
| 91 | -``` |
|
| 92 | - |
|
| 93 | - |
|
| 94 | -to allow root shell login, and allow for several concurrent SSH connections (up to 100) starting up around the |
|
| 95 | -same time. |
|
| 96 | - |
|
| 97 | -Furthermore, on recent AMIs, you may have to go to `/root/.ssh/authorized_keys` and remove the statements before the keys start, otherwise you might lock yourself out (because you can't access root but the new permissions block ec2-user access). If you are locked out, then you can use EC2 Instance Connect, which can be found by clicking on an instance and clicking connect. |
|
| 98 | - |
|
| 99 | -You may now _logout_. |
|
| 100 | - |
|
| 101 | -I linked /etc/init.d/sailing to /home/sailing/code/configuration/sailing and added the following links to it: |
|
| 102 | - |
|
| 103 | -``` |
|
| 104 | -rc0.d/K10sailing |
|
| 105 | -rc1.d/K10sailing |
|
| 106 | -rc2.d/S95sailing |
|
| 107 | -rc3.d/S95sailing |
|
| 108 | -rc4.d/S95sailing |
|
| 109 | -rc5.d/S95sailing |
|
| 110 | -rc6.d/K10sailing |
|
| 111 | -``` |
|
| 112 | - |
|
| 113 | -Linked /etc/profile.d/sailing.sh to /home/sailing/code/configuration/sailing.sh. As this contains a PATH entry for /opt/amazon and the new image has the Amazon scripts at /opt/aws, I aldo created a symbolic link from /opt/amazon to /opt/aws to let this same path configuration find those scripts under the old and the new images. |
|
| 114 | - |
|
| 115 | -Added the lines |
|
| 116 | - |
|
| 117 | -``` |
|
| 118 | -# number of connections the firewall can track |
|
| 119 | -net.ipv4.ip_conntrac_max = 131072 |
|
| 120 | -``` |
|
| 121 | - |
|
| 122 | -to `/etc/sysctl.conf` in order to increase the number of connections that are possible concurrently. |
|
| 123 | - |
|
| 124 | -Added the following two lines to `/etc/security/limits.conf`: |
|
| 125 | - |
|
| 126 | -``` |
|
| 127 | -* hard nproc unlimited |
|
| 128 | -* hard nofile 128000 |
|
| 129 | -``` |
|
| 130 | - |
|
| 131 | -This increases the maximum number of open files allowed from the default 1024 to a more appropriate 128k. |
|
| 132 | - |
|
| 133 | -Copied the httpd configuration files `/etc/httpd/conf/httpd.conf`, `/etc/httpd/conf.d/000-macros.conf` and the skeletal `/etc/httpd/conf.d/001-events.conf` from an existing server. Make sure the following lines are in httpd.conf: |
|
| 134 | - |
|
| 135 | -<pre> |
|
| 136 | - SetEnvIf X-Forwarded-For "^([0-9]*\.[0-9]*\.[0-9]*\.[0-9]*).*$" original_client_ip=$1 |
|
| 137 | - LogFormat "%v %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined |
|
| 138 | - LogFormat "%v %{original_client_ip}e %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" first_forwarded_for_ip |
|
| 139 | - CustomLog logs/access_log combined env=!original_client_ip |
|
| 140 | - CustomLog logs/access_log first_forwarded_for_ip env=original_client_ip |
|
| 141 | -</pre> |
|
| 142 | - |
|
| 143 | -They ensure that the original client IPs are logged also if the Apache server runs behind a reverse proxy or an ELB. See also [the section on log file analysis](/wiki/howto/development/log-file-analysis#log-file-analysis_log-file-types_apache-log-files). |
|
| 144 | - |
|
| 145 | -Copied /etc/logrotate.conf from an existing SL instance so that `/var/log/logrotate-target` is used to rotate logs to. |
|
| 146 | - |
|
| 147 | -Instead of having the `ANDROID_HOME` environment variable be set in `/etc/profile` as in the old instances, I moved this statement to the `sailing.sh` script in git at `configuration/sailing.sh` and linked to by `/etc/profile.d/sailing.sh`. For old instances this will set the variable redundantly, as they also have it set by a manually adjusted `/etc/profile`, but this shouldn't hurt. |
|
| 148 | - |
|
| 149 | -Had to fiddle a little with the JDK being used. The default installation has an OpenJDK installed, and the AWS tools depend on it. Therefore, it cannot just be removed. As a result, it's important that `env.sh` has the correct `JAVA_HOME` set (/opt/jdk1.8.0_45, in this case). Otherwise, the OSGi environment won't properly start up. |
|
| 150 | - |
|
| 151 | -For the ``root`` user create the symbolic link from ``/root/crontab`` to ``/home/sailing/code/configuration/crontab`` and run ``crontab crontab``. It adds the following crontab entry that is responsible for updating the SSH keys of the users with permission for landscape management in the ``/root/.ssh/authorized_keys`` file. |
|
| 152 | -``` |
|
| 153 | -* * * * * export PATH=/bin:/usr/bin:/usr/local/bin; sleep $(( $RANDOM * 60 / 32768 )); update_authorized_keys_for_landscape_managers_if_changed $( cat /root/ssh-key-reader.token ) https://security-service.sapsailing.com /root 2>&1 >>/var/log/sailing.err |
|
| 154 | -``` |
|
| 155 | -Make sure, a valid bearer token is installed in ``/root/ssh-key-reader.token``. |
|
| 156 | - |
|
| 157 | -To ensure that chronyd is started during the boot sequence, issued the command |
|
| 158 | - |
|
| 159 | -``` |
|
| 160 | -chkconfig chrony on |
|
| 161 | -``` |
|
| 162 | - |
|
| 163 | -which creates the necessary entries in the rc*.d directories. |
|
| 164 | - |
|
| 165 | -Update the file `/etc/postfix/main.cf` in order to set the server's sending hostname to `sapsailing.com` as follows: |
|
| 166 | -``` |
|
| 167 | - myhostname = sapsailing.com |
|
| 168 | -``` |
|
| 169 | - |
|
| 170 | -Adjust the /etc/sysconfig/vncservers settings to something like: |
|
| 171 | - |
|
| 172 | -``` |
|
| 173 | -VNCSERVERS="2:sailing" |
|
| 174 | -VNCSERVERARGS[2]="-geometry 1600x900" |
|
| 175 | -``` |
|
| 176 | - |
|
| 177 | -## Mail Relaying |
|
| 178 | -For setting up mail relaying towards central postfix server, have a look [here](https://wiki.sapsailing.com/wiki/info/landscape/mail-relaying) |
|
| 7 | +Compared to earlier versions of this image type, no mail infrastructure and no httpd reverse proxy is being configured. No NFS mounts are performed, and the instances resulting from this will not have everything required to *build* the solution, in particular no NFS mount of the Android SDK. |
|
| ... | ... | \ No newline at end of file |