9a090b1bfad8b19173da5368a83e3ae09e2d7000
configuration/hudson_instance_setup/setup-hudson-server.sh
| ... | ... | @@ -62,6 +62,10 @@ EOF |
| 62 | 62 | sudo cp /root/mail.properties /home/sailing/servers/DEV/configuration |
| 63 | 63 | sudo chown sailing /home/sailing/servers/DEV/configuration/mail.properties |
| 64 | 64 | sudo chgrp sailing /home/sailing/servers/DEV/configuration/mail.properties |
| 65 | + # Start the sailing.service with empty/no user data, so the next boot is recognized as a re-boot |
|
| 66 | + sudo systemctl start sailing.service |
|
| 67 | + sudo systemctl stop sailing.service |
|
| 68 | + sudo mount -a |
|
| 65 | 69 | else |
| 66 | 70 | echo "Not running on an AWS instance; refusing to run setup!" >&2 |
| 67 | 71 | echo "To prepare an instance running in AWS, provide its external IP as argument to this script." >&2 |
wiki/info/landscape/creating-ec2-image-for-hudson-from-scratch.md
| ... | ... | @@ -1,112 +1,55 @@ |
| 1 | 1 | # Setting up an image for the hudson.sapsailing.com server |
| 2 | 2 | |
| 3 | -This is an add-on to the regular EC2 image set-up described [here](https://wiki.sapsailing.com/wiki/info/landscape/creating-ec2-image-from-scratch). An Android SDK needs to be installed. |
|
| 4 | - |
|
| 5 | - |
|
| 6 | -* Create a ``hudson`` user/group |
|
| 7 | -* Make sure ``/home/hudson`` is a separate mount; probably just mount the existing volume of a previous installation |
|
| 8 | -* Install an Android SDK under ``/home/hudson/android-sdk-linux``; if you simply re-used an old ``/home/hudson`` mount this should already be in place. |
|
| 9 | -* Install Eclipse to ``/home/hudson/eclipse`` to allow sharing it in case a large AWS instance is needed, e.g., for heap dump analysis. |
|
| 10 | -* export ``/home/hudson/android-sdk-linux`` and ``/home/hudson/eclipse`` as follows in ``/etc/exports``: |
|
| 3 | +Like when setting up a regular sailing application server instance, start with a fresh Amazon Linux 2 image and create an instance with a 16GB root volume. Use the "Sailing Analytics Server" security group and something like a ``t3.small`` instance type for creating the image. Then, invoke the script |
|
| 11 | 4 | ``` |
| 12 | -/home/hudson/android-sdk-linux 172.31.0.0/16(rw,nohide,no_root_squash) |
|
| 13 | -/home/hudson/eclipse 172.31.0.0/16(rw,nohide,no_root_squash) |
|
| 5 | + configuration/hudson_instance_setup/setup-hudson-server.sh {external-IP-of-new-instance} |
|
| 14 | 6 | ``` |
| 7 | +This will first run the regular sailing server set-up which allows the instance to run the ``dev.sapsailing.com`` Sailing Analytics instance later. Then, the script will continue to obtain the Hudson WAR file from ``https://static.sapsailing.com/hudson.war.patched-with-mail-1.6.2`` and deploy it to ``/usr/lib/hudson``, obtain and install the default system-wide Hudson configuration, get adjusted dev server secrets from ``ssh://root@sapsailing.com/root/dev-secrets`` as well as ``mail.properties``, and install a ``hudson.service`` unit under ``/etc/systemd/system``. A ``hudson`` user is created, and its ``/home/hudson`` home directory is emptied so it can act as a mount point. A latest version of the SAP Sailing Analytics is installed to ``/home/sailing/servers/DEV``. |
|
| 15 | 8 | |
| 16 | -* Ensure you have EC2 / EBS snapshot backups for the volumes by tagging them as follows: ``WeeklySailingInfrastructureBackup=Yes`` for ``/`` and ``/home/hudson``. |
|
| 9 | +The ``/home/hudson/android-sdk-linux`` folder that is later expected to be mounted into the ``/home/hudson`` mount point is exported through NFS by appending a corresponding entry to ``/etc/exports``. |
|
| 17 | 10 | |
| 18 | -``/home/hudson/repo`` has the Hudson build repository. The Hudson WAR file is under ``/usr/lib/hudson/hudson.war`` which has to be taken from [here](https://static.sapsailing.com/hudson.war.patched-with-mail-1.6.2). ``/etc/init.d/hudson``, linked to from ``/etc/rc0.d/K29hudson``, ``/etc/rc1.d/K29hudson``, ``/etc/rc2.d/K29hudson``, ``/etc/rc3.d/S81hudson``, ``/etc/rc4.d/K29hudson``, ``/etc/rc5.d/S81hudson``, and ``/etc/rc6.d/K29hudson``, takes care of spinning up Hudson during instance re-boot. It can be obtained from [here](https://static.sapsailing.com/etc-init.d-hudson). Hudson systemwide configuration is under ``/etc/sysconfig/hudson``: |
|
| 11 | +When the script has finished, proceed as follows: |
|
| 12 | + |
|
| 13 | +* make a volume available that holds the ``/home/hudson`` content, including the ``android-sdk-linux`` folder. This can happen, e.g., by creating a snapshot of the existing ``/home/hudson`` volume of the running "Build/Dev/Test" server, or you may search the weekly backup snapshots for the latest version to start with. Make sure to create the volume in the same availability zone (AZ) as your instance is running in. |
|
| 14 | +* Based on how full the volume already is, consider re-sizing it by creating the volume larger than the snapshot. |
|
| 15 | +* Attach it. |
|
| 16 | +* In the instance, as ``root`` user, call ``dmesg``. This will show the new volume that just got attached, as well as its partition names. |
|
| 17 | +* If you chose the volume bigger than the snapshot, use ``resize2fs`` to resize accordingly. |
|
| 18 | +* Enter an ``/etc/fstab`` entry for the volume, e.g., like this: |
|
| 19 | +``` |
|
| 20 | +UUID={the-UUID-of-the-partition-to-mount} /home/hudson ext4 defaults,noatime,commit=30 0 0 |
|
| 21 | +``` |
|
| 22 | +To find out ``{the-UUID-of-the-partition-to-mount}``, use ``blkid``. |
|
| 23 | +* Mount with ``mount -a`` |
|
| 24 | +* Adjust ownerships in case the new instance's ``hudson`` user/group IDs have changed compared to the old instance: |
|
| 19 | 25 | ``` |
| 20 | -## Path: Development/Hudson |
|
| 21 | -## Description: Configuration for the Hudson continuous build server |
|
| 22 | -## Type: string |
|
| 23 | -## Default: "/var/lib/hudson" |
|
| 24 | -## ServiceRestart: hudson |
|
| 25 | -# |
|
| 26 | -# Directory where Hudson store its configuration and working |
|
| 27 | -# files (checkouts, build reports, artifacts, ...). |
|
| 28 | -# |
|
| 29 | -HUDSON_HOME="/home/hudson/repo" |
|
| 26 | + sudo chmod -R hudson:hudson /home/hudson |
|
| 27 | +``` |
|
| 28 | +* If you'd like to keep in sync with the latest version of a still running live Hudson environment, keep copying its ``/home/hudson`` contents with ``rsync -av root@dev.internal.sapsailing.com:/home/hudson/ /home/hudson/`` until you switch |
|
| 29 | +* Ensure you have EC2 / EBS snapshot backups for the volumes by tagging them as follows: ``WeeklySailingInfrastructureBackup=Yes`` for ``/`` and ``/home/hudson``. |
|
| 30 | 30 | |
| 31 | -## Type: string |
|
| 32 | -## Default: "" |
|
| 33 | -## ServiceRestart: hudson |
|
| 34 | -# |
|
| 35 | -# Java executable to run Hudson |
|
| 36 | -# When left empty, we'll try to find the suitable Java. |
|
| 37 | -# |
|
| 31 | +## In-Place Start-Up |
|
| 38 | 32 | |
| 39 | -HUDSON_JAVA_CMD="/opt/sapjvm_8/bin/java" |
|
| 40 | -# The following line choses JavaSE-1.7 |
|
| 41 | -#HUDSON_JAVA_CMD="/opt/jdk1.7.0_02/bin/java" |
|
| 42 | -# The following line choses JavaSE-1.8 |
|
| 43 | -#HUDSON_JAVA_CMD="/opt/jdk1.8.0_20/bin/java" |
|
| 33 | +You can then either use the instance right away by starting the two essential services, as follows: |
|
| 34 | +``` |
|
| 35 | + sudo systemctl start hudson.service |
|
| 36 | + sudo systemctl start sailing.service |
|
| 37 | +``` |
|
| 38 | +## Creating an AMI to Launch |
|
| 44 | 39 | |
| 45 | -## Type: string |
|
| 46 | -## Default: "hudson" |
|
| 47 | -## ServiceRestart: hudson |
|
| 48 | -# |
|
| 49 | -# Unix user account that runs the Hudson daemon |
|
| 50 | -# Be careful when you change this, as you need to update |
|
| 51 | -# permissions of $HUDSON_HOME and /var/log/hudson. |
|
| 52 | -# |
|
| 53 | -HUDSON_USER="hudson" |
|
| 40 | +Or you stop the instance, either from within using ``shutdown -h now`` or from the AWS Console, then create an image (AMI) that you can use to create a new instance at any time. Keep in mind, though, that keeping such an image around incurs cost for the relatively large ``/home/hudson`` volume's snapshot. As the volume is part of the weekly backup strategy anyhow, and due to the fair degree of automation during producing a new version of this instance type, this may not be necessary. |
|
| 54 | 41 | |
| 55 | -## Type: string |
|
| 56 | -## Default: "-Djava.awt.headless=true" |
|
| 57 | -## ServiceRestart: hudson |
|
| 58 | -# |
|
| 59 | -# Options to pass to java when running Hudson. |
|
| 60 | -# |
|
| 61 | -HUDSON_JAVA_OPTIONS="-Djava.awt.headless=true -Xmx2G -Dhudson.slaves.ChannelPinger.pingInterval=60 -Dhudson.slaves.ChannelPinger.pingIntervalSeconds=60 -Dhudson.slaves.ChannelPinger.pingTimeoutSeconds=60" |
|
| 42 | +The AMI should be labeled accordingly, and so should the snapshots. For allowing for automated image upgrades, ensure the AMI and snapshots follow a common naming and versioning pattern. Name your AMI something like "Build/Dev/Test x.y" with x and y being major and minor version numbers, such as 2.0. Then, your snapshots shall be named "Build/Dev/Test x.y ({volume-name})" where {volume-name} can be any human-readable identifier that lets you recognize the volume. Examples for {volume-name} may be "Root" or "Home" or "HudsonHome" or similar. |
|
| 62 | 43 | |
| 63 | -## Type: integer(0:65535) |
|
| 64 | -## Default: 8080 |
|
| 65 | -## ServiceRestart: hudson |
|
| 66 | -# |
|
| 67 | -# Port Hudson is listening on. |
|
| 68 | -# |
|
| 69 | -HUDSON_PORT="8080" |
|
| 44 | +Then terminate your instance used only for image creation and launch from the AMI, then, as in "In-Place Start-Up", place the instance into the "Hudson" target group and adjust the elastic IP to point to the new instance. |
|
| 70 | 45 | |
| 71 | -## Type: integer(1:9) |
|
| 72 | -## Default: 5 |
|
| 73 | -## ServiceRestart: hudson |
|
| 74 | -# |
|
| 75 | -# Debug level for logs -- the higher the value, the more verbose. |
|
| 76 | -# 5 is INFO. |
|
| 77 | -# |
|
| 78 | -HUDSON_DEBUG_LEVEL="5" |
|
| 46 | +## Steps to Perform to Activate the New Server |
|
| 79 | 47 | |
| 80 | -## Type: yesno |
|
| 81 | -## Default: no |
|
| 82 | -## ServiceRestart: hudson |
|
| 83 | -# |
|
| 84 | -# Whether to enable access logging or not. |
|
| 85 | -# |
|
| 86 | -HUDSON_ENABLE_ACCESS_LOG="no" |
|
| 48 | +* Replace the old "Build/Test/Dev" instance in the "Hudson" and "S-DEV" and "S-DEV-m" target groups with the new one |
|
| 49 | +* Switch the elastic IP ``52.17.217.83`` to the new instance, too |
|
| 50 | +* Change the ``dev.internal.sapsailing.com`` Route53 DNS entry from the old instance's internal IP address to the new instance's internal IP address |
|
| 87 | 51 | |
| 88 | -## Type: integer |
|
| 89 | -## Default: 100 |
|
| 90 | -## ServiceRestart: hudson |
|
| 91 | -# |
|
| 92 | -# Maximum number of HTTP worker threads. |
|
| 93 | -# |
|
| 94 | -HUDSON_HANDLER_MAX="100" |
|
| 95 | 52 | |
| 96 | -## Type: integer |
|
| 97 | -## Default: 20 |
|
| 98 | -## ServiceRestart: hudson |
|
| 99 | -# |
|
| 100 | -# Maximum number of idle HTTP worker threads. |
|
| 101 | -# |
|
| 102 | -HUDSON_HANDLER_IDLE="20" |
|
| 53 | +## Testing the New and Terminating the Old Instance |
|
| 103 | 54 | |
| 104 | -## Type: string |
|
| 105 | -## Default: "" |
|
| 106 | -## ServiceRestart: hudson |
|
| 107 | -# |
|
| 108 | -# Pass arbitrary arguments to Hudson. |
|
| 109 | -# Full option list: java -jar hudson.war --help |
|
| 110 | -# |
|
| 111 | -HUDSON_ARGS="" |
|
| 112 | -``` |
|
| 55 | +After a while of testing the new environment successfully you may choose to terminate the old instance. Make sure the large ``/home/hudson`` volume is deleted with it, and it not, delete it manually. |
|
| ... | ... | \ No newline at end of file |