Building, Deploying, Stopping and Starting Server Instances

Table of Contents

Running a Build

Builds are generally executed in either of the following two ways:

  • Check out a branch of your liking from git and run /configuration/buildAndUpdateProduct.sh build (check the various options of this script by invoking it with no arguments). The build results including the p2 product repository are then located in the git workspace.

  • Ensure that Hudson has a job for your branch; simply push the branch to the central git at sapsailing.com and let Hudson do the job. This is mostly good for knowing whether everything builds and tests ok. Only if you push the special "release" tag, Hudson will build a release and upload it to releases.sapsailing.com.

Deploying Build Results

When the build has been run using the buildAndUpdateProduct.sh script, the build results in the git workspace can be deployed to a server environment under ~/servers/<servername> using the command

<gitroot>/configuration/buildAndUpdateProduct.sh -s <servername> install

This will copy all necessary files, in particular the p2 product, to the server directory, including the start and stop scripts.

Again, check out the script's options for more and other possibilities including a remote deployment option and a hot deploy of individual bundles into a running server environment.

Deploying build results is generally also possible with a Hudson build, but it is not recommended because a user would need to log in to the Hudson server, know where which build workspace is located and then apply the deployment script there.

Working with Releases

Particularly when starting an EC2 instance, it is helpful to be able to do that using a well-known release of the product. When an EC2 instance starts, it has a version of the product built into the image and its disk snapshots from which the instance got initialized. This, however, is usually not up to date. To refresh it, you could run a build from a specific git commit, or you could install a release previously assembled using the release option of the buildAndUpdateProduct.sh script as follows:

<gitroot>/configuration/buildAndUpdateProduct.sh build
<gitroot>/configuration/buildAndUpdateProduct.sh -w trac@sapsailing.com -n <release-name> release

This will ask you for a comment about the release which goes into the release notes text file that accompanies the release. The build results are packed up into a .tar.gz file which is then uploaded to releases.sapsailing.com, using the name optionally provided using the -n parameter with the release action, or—as a default—the current timestamp for the release name.

A release can be downloaded and installed to a server by changing to the server's directory, e.g., ~/servers/server or whatever the sub-directory of the server installation is, and there executing the refreshInstance.sh script with the parameter install-release <release-name>. Afterwards, starting the instance works as after a local build.

A sample session could look like this:

$ ssh sailing@34.250.136.229
$ cd servers/ubilabs-test
$ ./refreshInstance.sh install-release build-201712210442
$ ./stop; ./start

Instead of building a release yourself you can let the build server to the job. There is a job that looks out for a git tag named release. If a new revision is found then an automatic release build is being executed. The result of that is persisted to http://releases.sapsailing.com/. You can create that tag as follows. Make sure that you're on the current master branch before executing the following commands.

$ git tag -f release
$ git push -f origin release:release

Give the build server some time (20-30 Minutes) until it will have the release ready.

Example with Hudson

Deploying jobs from hudson to releases.sapsailing.com is quite simple.

  • login as hudson@<build server>
  • go to /home/hudson/repo/jobs/{jobname}/workspace
  • execute ./configuration/buildAndUpdateProduct.sh -n build -w trac@sapsailing.com release for triggering the upload

Working with Environments

There exist a number of preconfigured environment configurations at releases.sapsailing.com/environments. Such an environment can be automagically applied to your instance by changing to the servers directory and then executing the refreshInstance.sh script with the parameter install-env <environment-name>. This will update your env.sh. Make sure to afterwards restart your server.

Starting, Administrating and Stopping a Java Instance

The product runs as a Java instance consisting largely of an Equinox OSGi server that load and runs various OSGi bundles constituting the product and that contains an embedded Jetty web server. It connects to a database, may serve as a master for replication through a messaging server, may be launched as a replica of some other master instance, may receive wind data in Expedition format on a specific port and can listen for telnet requests to administrate the OSGi server on a specific port. These and other properties are usually configured in a file called env.sh which has to be located in the server directory, e.g., in ~/servers/server, next to the start script. When the start script is executed, it first sources the env.sh file which sets the various properties which are then passed to the actual Java process, usually in the form of system properties.

After successfully having started a Java instance, it can be administrated through a telnet connection. The port on which the OSGi console listens for incoming connections can be configured in the env.sh file. Usually it defaults to port 14888. Therefore, a telnet localhost 14888 connects you to the OSGi console where an ss command will show you all bundles loaded. Once logged on to the OSGi console, a disconnect command will disconnect the telnet session from the OSGi console.

An exit command will terminate the Java instance after a confirmation. This will obviously stop all services provided by the instance, including all static and dynamic contents served by its web server. You should only trigger the exit command if you really know what you are doing!

Stopping a running server has—for your convenience—been wrapped into the stop script usually located in the server's directory. Simply executing it will use a telnet connection to the server's OSGi port and trigger an exit command automatically. See above for the "know what you're doing" part…

Automatic Java Instance Start-Up After Boot

When firing up an EC2 instance it can be convenient to not having to log on to have the EC2 instance run a Java instance automatically after it has completed its boot process. This is possible using so-called user data. The process of firing up an instance that either builds a certain git commit, installs and starts it after server boot or that downloads and installs a release and starts it is explained here.

App Build Process for iOS and Android

Our git and Project Portal structure seems to be somewhat unusual for the Final Assembly group. It is planned to move us to Xmake in the near future, although we don't know exactly which changes this will entail. Presumably, we'll become a little more flexible with regards to choosing branch names and branch namespace layouts.

The following descriptions are based on a situation where an Xmake migration has not yet taken place. Builds for iOS apps are performed using the "MiOS" infrastructure, builds for Android apps use "LeanDI."

MiOS Build for iOS Apps

For iOS and Android there are two different build processes in place. After changing from MiOS to xMake during July 2017, things have changed considerably. This is an attempt to summarize the technical steps necessary to get the build done and hand things over to the "Final Assembly" department for deployment to the stores.

The iOS app build is largely described by contents of a cfg/ directory which is symbolically linked to from the root of the git hierarchy and is actually located under ios/SAPTracker/cfg. It is independent on any Maven builds and therefore unrelated to any POM files (pom.xml) as used in the back-end OSGi and Android builds. Other than for the previous MiOS approach we can therefore build the contents of the master branch more or less directly. Another "sacrifice" we have to make to the iOS build is having to link src/ symbolically from the root of our git to ios/SAPTracker/src, but this seems adequate given the branch handling which is now much simplified over the previous Maven build where we had to separate the pom.xml contents between iOS build branches and the master branch.

Still, we use the branch central-ios-release-build to prepare for the build. We usually merge ubilabs--ios--develop into master and then master into central-ios-release-build. Builds off the latter branch can be tested using this Jenkins job. Run a Build with Parameters and enter central-ios-release-build as the TREEISH. The build should succeed and produce the .ipa file somewhere in the build's target output When the test build was successful, the branch contents need to be merged into the central git repository's branch fa/rel-1.x, e.g., fa/rel-1.0.

Final Assembly will need to be notified using a BCP ticket, such as a Handover ticket that can be created using this link or an update ticket as explained here in section "Requesting an update."

As a result of handling the request, Final Assembly will trigger builds on https://xmake-mobile-dev.wdf.sap.corp:8443/view/all/job/sapsailingprogram/job/sapsailingcapture-SP-REL-common_directshipment/44/.

xMake Build for Android Apps

We currently release the Android apps off a branch called android-xmake-release and referred to in the sequel as the "Android release branch."

The Android release branch holds the changes required to run a successful Android build in xMake. The actual build is still the Gradle build that xMake invokes like this: 

./gradlew -Pxmake -Pversion=1.4.116 -PcommonRepoURL=https://int.repositories.cloud.sap/artifactory/build-releases/

We have various branches on which Android mobile app features are being developed. Ultimately, they should be merged into the master branch from where they get merged into the Android release branch. A so-called "Customer release" build can be triggered for the Android release branch on a central Jenkins instance. To get a release build staged for releasing, the version numbers in a few descriptor files, particularly the gradle.properties files of each app sub-directory under /mobile, as well as cfg/files2sign.json need to be adjusted to the new version.

Also note that once released to Nexus / Artifactory, the next release build will have to use at least an incremented micro version, such as 1.4.1.

In order to get the Android release branch merged into the rel-x.y branch, the same process as explained above for the merge of iOS branches into fa/rel-x.y is required. In particular, this means that a Java Correction Workbench Correction Request needs to be created and the latest commit to be pushed to refs/for/rel-x.y needs to be amended so that it contains a valid Change-Id: and CR-Id: line. Then, the commit needs to be voted on. Other than for the MiOS branches under fa/rel-x.y it seems to be required that another committer votes with +2 to enable a merge. Once this has been done, the blue Submit button will appear on your change page (something like https://git.wdf.sap.corp/#/c/1436212/) that you can use to technically get the commit merged into the rel-x.y branch.

Many of the above steps can be automated by using the script configuration/releaseAndroidApps.sh. Call like this:

      ./configuration/releaseAndroidApps.sh

Additional options: 

  -m Disable upgrading the versionCode and versionName
  -g Disable the final git push operation to fa/rel-1.4
  -r The git remote; defaults to origin

This will checkout the android-xmake-release branch, merge the fa/arel-1.4 release branch into it, push the result again so Gerrit knows this commit / change already. Then, the version substitutions for all pom.xml and MANIFEST.MF files are carried out automatically by looking at the version currently declared in the root pom.xml. If that's a -SNAPSHOT version, only the -SNAPSHOT suffix is removed to obtain the new release version number. If no -SNAPSHOT suffix is found, the micro version is increased by one (e.g., 1.4.18 –> 1.4.19). Furthermore, the android:versionCode number is increased by one for each of the three apps in their AndroidManifest.xml, and the android:versionName which is assumed to have the format major.minor will have the minor version increased by one. The result is committed to git and the resulting commit is pushed to the fa/rel-1.4 branch.

Once this is done, a Customer release build can be started. Select "Build with Parameters" from the left, then select Stage as the build mode, then customer as the RELEASE_MODE and enter the fa/rel-x.y branch name in the "TREEISH" field, make sure the Maven version 3.1.1 is selected, select Tycho version 1.0.0, switch the "Next development version" to no change and click the "Build" button. If the build succeeds, Final Assembly should be able to promote the build artifact, such as the .apk files which should have been signed using the SAP certificate.

If the build fails due to missing Nexus artifacts, check out the document describing how to upload artifacts to Nexus. You will have to create a JIRA ticked in this process and upload the artifact and its pom.xml file to Nexus. Usually, these requests get handled within less than 48 hours. Good luck…