024c8e08332bf44bde385dc3e5c4a90663f3b55b
wiki/howto/onboarding.md
| ... | ... | @@ -1,17 +1,5 @@ |
| 1 | 1 | # OnBoarding Information |
| 2 | 2 | |
| 3 | -<!-- |
|
| 4 | -This Markdown document is designed to work with Gollum not with GitHub. Internal links to chapters have the following syntax in for Gollum: |
|
| 5 | - |
|
| 6 | -Chapter Hierachy: |
|
| 7 | -# chapter |
|
| 8 | -## new subchapter |
|
| 9 | -### chapter to reference |
|
| 10 | -## another chapter |
|
| 11 | - |
|
| 12 | -[linkText](#chapter_new-subchapter_chapter-to-reference) |
|
| 13 | - --> |
|
| 14 | - |
|
| 15 | 3 | This document describes the onboarding process for a new team member (developer) |
| 16 | 4 | |
| 17 | 5 | First of all, make sure you've looked at [http://www.amazon.de/Patterns-Elements-Reusable-Object-Oriented-Software/dp/0201633612](http://www.amazon.de/Patterns-Elements-Reusable-Object-Oriented-Software/dp/0201633612). That's a great book, and knowing at least some of it will help you a great deal finding your way around our solution. |
| ... | ... | @@ -21,29 +9,29 @@ First of all, make sure you've looked at [http://www.amazon.de/Patterns-Elements |
| 21 | 9 | ### Accounts |
| 22 | 10 | |
| 23 | 11 | 1. Git Account |
| 24 | - The primary Git repository for the project is hosted on Github (see [https://github.com/SAP/sailing-analytics](https://github.com/SAP/sailing-analytics)). To clone, use ``git@github.com:SAP/sailing-analytics.git``. To gain write access you have to become member of the [sailing-analytics-team](https://github.com/orgs/SAP/teams/sailing-analytics-team) organization. We still have a shadow repository around that, e.g., powers our Wiki at [https://wiki.sapsailing.com](https://wiki.sapsailing.com) and which lives at ``ssh://trac@sapsailing.com/home/trac/git``. |
|
| 25 | 12 | |
| 26 | - - For access to the external git at `ssh://trac@sapsailing.com/home/trac/git` please send your SSH public key to one of the project maintainers, requesting git access. Make sure to NOT generate the key using Putty. Putty keys don't work reliably under Linux and on Windows/Cygwin environments. Use ssh-keygen in a Cygwin or Linux or MacOS/X environment instead. For further instructions for generating an ssh-key see [GitHub](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent). |
|
| 13 | + - The primary Git repository for the project is hosted on Github (see [https://github.com/SAP/sailing-analytics](https://github.com/SAP/sailing-analytics)). To clone, use ``git@github.com:SAP/sailing-analytics.git``. To gain write access you have to become member of the [sailing-analytics-team](https://github.com/orgs/SAP/teams/sailing-analytics-team) organization. For that you need to [link your Github user to the Github SAP organization](https://wiki.one.int.sap/wiki/display/ospodocs/Self-Service+for+Joining+an+SAP+GitHub+Organization). We still have a shadow repository around that, e.g., powers our Wiki at [https://wiki.sapsailing.com](https://wiki.sapsailing.com) and which lives at ``ssh://trac@sapsailing.com/home/trac/git``. |
|
| 14 | + |
|
| 15 | + - In case you'd like to get access to the external git at `ssh://trac@sapsailing.com/home/trac/git` please send your SSH public key to one of the project maintainers, requesting git access. Make sure to NOT generate the key using Putty. Putty keys don't work reliably under Linux and on Windows/Cygwin environments. Use ssh-keygen in a Cygwin or Linux or MacOS/X environment instead. For further instructions for generating an ssh-key see [GitHub](https://docs.github.com/en/authentication/connecting-to-github-with-ssh/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent). |
|
| 27 | 16 | Note: If you want to use the ssh-key in the context of our solution, it can be an RSA or ED25519 format. Example for creating a key: `ssh-keygen -t ed25519 -b 512 -C "test@test.com"`. Make sure to set a non-empty password for your key. |
| 28 | 17 | |
| 29 | 18 | 2. Bugzilla |
| 30 | 19 | |
| 31 | 20 | - Create an account at https://bugzilla.sapsailing.com |
| 32 | - - Ask the Bugzilla administrator (axel.uhl@sap.com) to enable your account for editing bugs |
|
| 33 | - - Bugzilla url: [https://bugzilla.sapsailing.com](https://bugzilla.sapsailing.com) |
|
| 21 | + - Ask a Bugzilla administrator (e.g., axel.uhl@sap.com) to enable your account for editing bugs |
|
| 22 | + - Bugzilla URL: [https://bugzilla.sapsailing.com](https://bugzilla.sapsailing.com) |
|
| 34 | 23 | |
| 35 | 24 | 3. Wiki |
| 36 | 25 | |
| 37 | -We have so far decided against migrating our existing Gollum-based Wiki to Github's Wiki pages. Therefore, for the time being, you can either push changes to the ``wiki/`` folder to the ``main`` branch of the repository from where they will make it to the Gollum-provided Wiki, or our can request a Gollum account to be created for you. This will also allow you to view non-public pages through [https://wiki.sapsailing.com](https://wiki.sapsailing.com). |
|
| 38 | - |
|
| 39 | -For a Gollum Wiki account send a request to Axel Uhl or Simon Marcel Pamies that includes the SHA1 hash of your desired password. Obtain such an SHA1 hash for your password here: [http://www.sha1-online.com/](http://www.sha1-online.com/). |
|
| 26 | + - We have so far decided against migrating our existing Gollum-based Wiki to Github's Wiki pages. Therefore, for the time being, you can either push changes to the ``wiki/`` folder to the ``main`` branch of the repository from where they will make it to the Gollum-provided Wiki, or our can request a Gollum account to be created for you. This will also allow you to view non-public pages through [https://wiki.sapsailing.com](https://wiki.sapsailing.com). |
|
| 27 | + - For a Gollum Wiki account send a request to Axel Uhl or Simon Marcel Pamies that includes the SHA1 hash of your desired password. Obtain such an SHA1 hash for your password here: [http://www.sha1-online.com/](http://www.sha1-online.com/). |
|
| 28 | + - Once our Github repository is switched to a "public" repository, we will change our Gollum configuration such that at least read-only access is granted for all pages also for anonymous users. |
|
| 40 | 29 | |
| 41 | 30 | 4. Hudson |
| 42 | 31 | |
| 43 | 32 | - Request a [Hudson](https://hudson.sapsailing.com) user by sending e-mail to Axel Uhl or Simon Marcel Pamies. |
| 44 | 33 | |
| 45 | 34 | ### Installations |
| 46 | - |
|
| 47 | 35 | 1. Eclipse IDE for Eclipse Committers, version ["2025-03"](https://www.eclipse.org/downloads/packages/release/2025-03/r/eclipse-ide-eclipse-committers). If you are using a Mac and want to use SAPJVM, this has to be the 64 bit version. This is because SAPJVM is not available for Apple Silicon Macs, and Eclipse's OS architecture must match the JVM architecture. |
| 48 | 36 | 2. JDK 1.8 (Java SE 8), ideal is the SAPJVM 1.8: Go to [https://tools.eu1.hana.ondemand.com/#cloud](https://tools.eu1.hana.ondemand.com/#cloud), scroll down to `SAP JVM` select your operating System, extract the downloaded .zip into desired location (e.g. Windows `C:\Program Files\Java`. If you want to make this your default JDK, set the `JAVA_HOME` variable to it. In any case, set the `JAVA8_HOME` variable to it which is required by a few build scripts where certain steps currently are not yet compatible with newer JDK releases, such as our Android build process, keeping us on Gradle 6.0.1 for the time being which isn't Java 17-compatible. |
| 49 | 37 | 3. Git (e.g. Git for Windows v2.18), [http://git-scm.com](http://git-scm.com) / [https://git-for-windows.github.io](https://git-for-windows.github.io)still |
| ... | ... | @@ -71,14 +59,12 @@ For a Gollum Wiki account send a request to Axel Uhl or Simon Marcel Pamies that |
| 71 | 59 | |
| 72 | 60 | ### Further optional but recommended installations |
| 73 | 61 | |
| 74 | -1. Cygwin, [http://www.cygwin.com/](http://www.cygwin.com/) |
|
| 62 | +1. For Windows users, [Cygwin](http://www.cygwin.com/) or a [Git Bash](https://git-scm.com/downloads) may be useful for being able to run any Bash scripts. |
|
| 75 | 63 | Please note that when using one of the newer versions of Cygwin, your Cygwin home folder setting might differ from your Windows home folder. This will likely lead to problems when issuing certain commands. For troubleshooting, take a look at the following thread: [https://stackoverflow.com/questions/1494658/how-can-i-change-my-cygwin-home-folder-after-installation](https://stackoverflow.com/questions/1494658/how-can-i-change-my-cygwin-home-folder-after-installation) |
| 76 | 64 | 2. Eclipse Mylyn Bugzilla extension |
| 77 | 65 | 3. kdiff3 (git tool) |
| 78 | 66 | 4. Firebug (javascript & .css debugging, included in Firefox Developer Tools in newer versions of Firefox by default) |
| 79 | 67 | |
| 80 | - |
|
| 81 | - |
|
| 82 | 68 | ### Git repository configuration essentials |
| 83 | 69 | |
| 84 | 70 | The project has some configuration of line endings for specific file types in ".gitattributes". To make this work as intended, you need to ensure that the git attribute "core.autocrlf" is set to "false". This can be done by navigating to your local repository in a Bash/Git Bash/Cygwin instance and executing the command `git config core.autocrlf false`. |
| ... | ... | @@ -89,7 +75,7 @@ Depending on the location of your local repository, it's filepaths might be too |
| 89 | 75 | |
| 90 | 76 | ### Maven Setup |
| 91 | 77 | |
| 92 | -Copy the settings.xml (may be in $GIT_HOME/configuration/maven-settings.xml and $GIT_HOME/configuration/maven-settings.xml) **and** the toolchains.xml from the top-level git folder to your ~/.m2 directory. Adjust the proxy settings in settings.xml accordingly (suggested settings for corporate network inside). Set the paths inside of toolchains.xml to your JDKs depending on where you installed them (this is like setting the compiler for your IDE, but for Maven; This makes it possible to build with the same Maven configuration on every system). Make sure the mvn executable you installed above is in your path. |
|
| 78 | +Copy the settings.xml (may be in $GIT_HOME/configuration/maven-settings.xml and $GIT_HOME/configuration/maven-settings-proxy.xml) **and** the toolchains.xml from the top-level git folder to your ~/.m2 directory. Adjust the proxy settings in settings.xml accordingly (suggested settings for inside a corporate network requiring a HTTP proxy for access to external web). Set the paths inside of toolchains.xml to your JDKs depending on where you installed them (this is like setting the compiler for your IDE, but for Maven; This makes it possible to build with the same Maven configuration on every system). Make sure the mvn executable you installed above is in your path. |
|
| 93 | 79 | |
| 94 | 80 | ### Automatic Eclipse plugin installation |
| 95 | 81 | The necessary Eclipse plugins can be automatically installed into a newly unzipped version of ["2023-09"](https://www.eclipse.org/downloads/packages/release/2023-09/r/eclipse-ide-eclipse-committers) by using the `./configuration/pluginsForEclipse2025-03.p2f` file, found in the git repository cloned in _step 11_. To install the plugins open Eclipse and install Software Items from File. (File ⇒ Import ⇒ Install ⇒ Install Software from File). The description file is located at `/configuration/pluginsForEclipse2025-03.p2f`. |
| ... | ... | @@ -136,68 +122,39 @@ Go to Window ⇒ Preferences and change the following two settings: |
| 136 | 122 | - For Eclipse-based debugging of GWT web applications with SDBG, make sure that Chrome is set as your default browser: "General ⇒ Web Browser". If missing, add a profile for Chrome and specify "%URL%" as the parameter. |
| 137 | 123 | - Consider installing [https://marketplace.eclipse.org/content/protocol-buffer-editor](https://marketplace.eclipse.org/content/protocol-buffer-editor) for a Protocol Buffers (protobuf) editor |
| 138 | 124 | |
| 139 | -### Steps to build and run the Race Analysis Suite |
|
| 125 | +### Steps to build and run the Sailing Analytics |
|
| 140 | 126 | |
| 141 | -1. Check out the 'master' branch from the git repository. The 'master' branch is the main development branch. Please check that you start your work on this branch. |
|
| 127 | +1. Check out the ``main`` branch from the git repository. The ``main`` branch is the main development branch. Please check that you start your work based on this branch. |
|
| 142 | 128 | 2. Setup and configure Eclipse |
| 143 | 129 | - Import all Race Analysis projects from the `java/` subdirectory of the git main folder (make sure to import via the wizard [but without smart import] "Import ⇒ General ⇒ Projects from Folder or Archive" in Eclipse, and additionally make sure to scan for nested projects!) |
| 144 | - - Import all projects from the `mobile/` subdirectory of the git main folder; this in particular contains the race committee app projects |
|
| 145 | 130 | - In "Window ⇒ Preferences ⇒ Plug-in Development ⇒ Target Platform" set the Eclipse target platform to `Race Analysis Target` (located in com.sap.sailing.targetplatform/definitions//race-analysis-p2-remote.target) |
| 146 | 131 | - Wait until the target platform has been resolved completely |
| 147 | 132 | - Start a clean build (Project ⇒ Clean) |
| 148 | - |
|
| 149 | -3. On clear workspace additional steps should be performed once: |
|
| 133 | +3. To get a clean workspace, additional steps should be performed once: |
|
| 150 | 134 | 1. Run "GWT Dashboards SDM" launch configuration. After successful start, launch configuration can be stopped. |
| 151 | 135 | 2. Run "GWT Security SDM" launch configuration. After successful start, launch configuration can be stopped. |
| 152 | 136 | 3. Run "GWT xdStorage Sample SDM" launch configuration. After successful start, launch configuration can be stopped. |
| 153 | 137 | 4. Run the Race Analysis Suite |
| 154 | - 1. Start the MongoDB-Server |
|
| 138 | + 1. Ensure your local MongoDB Server is running; depending on your platform, maybe you can start your MongoDB using ``sudo systemctl start mongod``, or you may have to do something like: |
|
| 155 | 139 | 1. Create a folder for the mongoDB to store the data. For existing folders make sure they do not contain a `mongod.lock` file |
| 156 | 140 | 2. Open a terminal and navigate to the location of the MongoDB installation `cd /somePathTo MongoDBInstallation/mongodb/bin` |
| 157 | 141 | 3. Start the databse in the with the mongoDB Datafolder as db path: |
| 158 | 142 | `./mongod --dbpath /somePathTo/MongoDBDataDirectory` |
| 159 | 143 | 2. Run "GWT Sailing SDM" in the debug dropdown |
| 160 | - 3. Start the appropriate Eclipse launch configuration (e.g. 'Sailing Server (no Proxy)') You´ll find this in the debug dropdown |
|
| 144 | + 3. Start the appropriate Eclipse back-end launch configuration (in most cases 'Sailing Server (no Proxy)'). You´ll find this in the debug dropdown. |
|
| 161 | 145 | 5. Import races within the Race Analysis Suite |
| 162 | - - Choose "GWT Sailing SDM" in the "Development Mode" Tab and open "...AdminConsole.html..." (It is normal that the first try fails. Reload the page after the first try) |
|
| 146 | + - Choose "GWT Sailing SDM" in the "Development Mode" Tab and open "...AdminConsole.html...". This should open [http://127.0.0.1:8888/gwt/AdminConsole](http://127.0.0.1:8888/gwt/AdminConsole). (It is normal that the first try fails. Reload the page after the first try) |
|
| 163 | 147 | - Default Login: user "admin", password "admin" |
| 164 | 148 | - In the list on the left, click on "Connectors" |
| 165 | 149 | - For TracTrac Events: In the "TracTrac Connections" Form, fill in the JSON URL [http://germanmaster.traclive.dk/events/event_20120905_erEuropean/jsonservice.php](http://germanmaster.traclive.dk/events/event_20120905_erEuropean/jsonservice.php)(all other required information will be filled in automatically) |
| 166 | 150 | - Press "List Races" |
| 167 | - |
|
| 168 | - |
|
| 169 | 151 | 6. Further useful launch configurations |
| 170 | 152 | - Use SAP JVM Profiler. If you used the script above and installed the SAPJVM instead of the jdk, you can now open the profiling perspective by clicking on Window ⇒ Perspective ⇒ Open Perspective ⇒ Profiling) |
| 171 | 153 | - Debugging gwt: For further instructions please see [here](./development/super-dev-mode) |
| 172 | 154 | |
| 173 | 155 | If you want to use **breakpoints**, *avoid* clicking on the options in the Development Mode tab. Instead, within the _Debug Configurations_ menu, select the _Debug AdminConsole_ (found in the _Launch Browser_ subtab); change the browser search order, such that chrome is the leftmost; and then launch. This is necessary because SDBG is compatible with Chrome. Further, details of how GWT Super Dev Mode (SDM) works, can be found in the link above. |
| 174 | 156 | |
| 175 | - |
|
| 176 | -### Build for deployment |
|
| 177 | -Open a shell (preferrably a git bash or a cygwin bash), cd to the git workspace's root folder and issue "./configuration/buildAndUpdateProduct.sh build". This should build the software and run all the tests. If you want to avoid the tests being executed, use the -t option. If you only want to build one GWT permutation (Chrome/English), use the -b option. When inside the SAP VPN, add the -p option for proxy use. Run the build script without arguments to get usage hints. |
|
| 178 | - |
|
| 179 | - |
|
| 180 | - |
|
| 181 | -## Further hints |
|
| 182 | - |
|
| 183 | -If you are working with a linux-system and you get the error message `error while loading shared libraries: libz.so.1: cannot open shared object file: No such file or directory` try to install lib32z1 and lib32stdc++6. |
|
| 184 | - |
|
| 185 | -### Steps to consider for using other modules |
|
| 186 | - |
|
| 187 | -1. For Eclipse Build |
|
| 188 | - - MANIFEST.MF , add module names unter dependencies |
|
| 189 | - - \*.gwt.xml , add `<inherits name="-modulename-.-gwt.xml file name-" />` |
|
| 190 | - - In DebugConfigurations => Classpaths, Add Sourcefolder where classes are you want to user from the module |
|
| 191 | -2. For Maven Build |
|
| 192 | - - pom.xml , Add Dependency to used module ie. |
|
| 193 | - `<dependency>` |
|
| 194 | - `<groupId>com.sap.sailing</groupId>` |
|
| 195 | - `<artifactId>com.sap.sailing.domain.common</artifactId>` |
|
| 196 | - `<version>1.0.0-SNAPSHOT</version>` |
|
| 197 | - `<classifier>sources</classifier>` |
|
| 198 | - `</dependency>` |
|
| 199 | - |
|
| 200 | -### Using Android Studio for Development |
|
| 157 | +### Using Android Studio for App Development (Only if You're Working on Mobile Apps) |
|
| 201 | 158 | |
| 202 | 159 | The Android Apps can be built in Android Studio or gradle command line. Android Studio is built on top of IntelliJ IDEA, so it is possible to use IntelliJ IDEA as well. |
| 203 | 160 | |
| ... | ... | @@ -227,6 +184,26 @@ The Android Apps can be built in Android Studio or gradle command line. Android |
| 227 | 184 | |
| 228 | 185 | If git is not in the Path system environment variable, the gradle build will not work. |
| 229 | 186 | |
| 187 | +## Further hints |
|
| 188 | + |
|
| 189 | +### Build for deployment |
|
| 190 | +Open a shell (preferrably a git bash or a cygwin bash), cd to the git workspace's root folder and issue "./configuration/buildAndUpdateProduct.sh build". This should build the software and run all the tests. If you want to avoid the tests being executed, use the -t option. If you only want to build one GWT permutation (Chrome/English), use the -b option. When inside the SAP VPN, add the -p option for proxy use. Run the build script without arguments to get usage hints. |
|
| 191 | + |
|
| 192 | +### Steps to consider for using other GWT modules |
|
| 193 | + |
|
| 194 | +1. For Eclipse Build |
|
| 195 | + - MANIFEST.MF, add module names under dependencies |
|
| 196 | + - \*.gwt.xml , add `<inherits name="-modulename-.-gwt.xml file name-" />` |
|
| 197 | + - In DebugConfigurations => Classpaths, Add Sourcefolder where classes are you want to user from the module |
|
| 198 | +2. For Maven Build |
|
| 199 | + - pom.xml , Add Dependency to used module ie. |
|
| 200 | + `<dependency>` |
|
| 201 | + `<groupId>com.sap.sailing</groupId>` |
|
| 202 | + `<artifactId>com.sap.sailing.domain.common</artifactId>` |
|
| 203 | + `<version>1.0.0-SNAPSHOT</version>` |
|
| 204 | + `<classifier>sources</classifier>` |
|
| 205 | + `</dependency>` |
|
| 206 | + |
|
| 230 | 207 | ### Git usage troubleshooting |
| 231 | 208 | |
| 232 | 209 | There are some inconsistencies regarding line endings (unix vs windows) in our git repository. There is a configuration named ".gitattributes" committed to the root folder of the repository that helps to prevent inconsistencies of line endings when committing files. Files that existed before are partially using unix (LF) or windows (CRLF) line endings. When committing files, git will ensure unix line endings for e.g. \*.java files. This can lead to big diffs that hurt when trying to merge/diff. |
| ... | ... | @@ -255,17 +232,18 @@ Irritating behavior can occur on ARM-based processors, such as on a MacBook. The |
| 255 | 232 | In such cases it might help to set AWT to headless mode (`-Djava.awt.headless=true`, see [stackoverflow](https://stackoverflow.com/questions/13796611/imageio-read-with-mac for more information)). |
| 256 | 233 | Another struggle can be to install the JVM Profiler Plug-in on ARM based Eclipse. It seems to block the necessary Software while the Automatic Eclipse plugin installation. One way is to use the Eclipse x64 Installer. It will take more time for transformation, but you will be able to use the Profiler. |
| 257 | 234 | |
| 258 | -### GWT Browser Plugin |
|
| 235 | +### GWT Browser Plugin |
|
| 236 | +This applies only if you try to get old "GWT Dev Mode" to work, support for which has ended on very old Firefox version 24. |
|
| 259 | 237 | Install the GWT Browser Plugin for the GWT Development mode. As of 2016-08-31 Firefox is the only browser supporting the GWT plugin, you have to download Firefox version 24 for it to work. The Plugin can be found on this page: [https://code.google.com/archive/p/google-web-toolkit/downloads](https://code.google.com/archive/p/google-web-toolkit/downloads) |
| 260 | 238 | |
| 261 | 239 | ### Create Hudson Job |
| 262 | 240 | If you want a hudson job to run when you push your branch then you can run a script in `configuration` called . Run options for a branch titled `createHudsonJobForBug.sh`. For you bug branch titled `bug<bug number>`, create a build job, which will create a release, by running the script like so: `./createHudsonJobForBug.sh <bug number>`. |
| 263 | -If on Windows, you may need to disable any web shields in antivirus software, to allow `curl` to function. If on Mac, you may need to install gnu-sed. |
|
| 241 | +If on Windows, you may need to disable any web shields in antivirus software, to allow `curl` to function. If on Mac, you may need to install gnu-sed (``gsed``) via Homebrew. |
|
| 264 | 242 | |
| 265 | 243 | ### Issues when playing around with AWS |
| 266 | 244 | - The problem: **aws cli (used for aws ec2 describe-tags) hangs in eu-west-2** in all AZs on new instances I created, using a target group which permitted all outbound connections and inbound https, http and ssh connections. I tried permitting everything but that didn’t work. When I attached (at Axel’s suggestion) the Java Application with Reverse Proxy security group, it worked but — even if I duplicated this security group, and applied that copy instead — it still didn’t work. |
| 267 | 245 | Curl issue solution: it turns out that the network interface only permits certain outbound and inbounds from certain target groups. |
| 268 | -The path to the solution: On my instance in eu-west-2a, I ran aws --debug ec2 describe-tags (you may need to do aws configure first). This is much akin to verbose mode of other unix commands. I noticed it hang on a request to ec2.eu-west-2.amazonaws.com. If you do `dig -t any ec2.eu-west-2.amazonaws.com` you see 3 ip addresses, which — as you will see later — are IPs in each of the eu-west-2 availability zones. When I ran curl -v ec2.eu-west-2.amazonaws.com (the v flag is verbose), one of the IPs from dig was used (namely the one in eu-west-2a, where the instance resides) and it hangs. I then went to endpoints for the VPC and noticed a service for the service `com.amazonaws.eu-west-2.ec2`. It had the default security group, which turned out to only allow inbound rules from the default or Java Application with Reverse Proxy target group. |
|
| 246 | +The path to the solution: On my instance in eu-west-2a, I ran aws --debug ec2 describe-tags (you may need to do aws configure first). This is much akin to verbose mode of other unix commands. I noticed it hang on a request to ec2.eu-west-2.amazonaws.com. If you do `dig -t any ec2.eu-west-2.amazonaws.com` you see 3 ip addresses, which — as you will see later — are IPs in each of the eu-west-2 availability zones. When I ran curl -v ec2.eu-west-2.amazonaws.com (the v flag is verbose), one of the IPs from dig was used (namely the one in eu-west-2a, where the instance resides) and it hangs. I then went to endpoints for the VPC and noticed a service for the service `com.amazonaws.eu-west-2.ec2`. It had the default security group, which turned out to only allow inbound rules from the default or Java Application with Reverse Proxy target group. |
|
| 269 | 247 | - Problem: A load balancer's target group health checks fail. I was told the checks failed with 403 errors. |
| 270 | 248 | Solution: This was occurring because the website didn't have any content in the /var/www/html. Whilst a site was still served (namely the Apache test page) it does throw a 403 error. If you fill the directory with and index.html the test then passes and a 200 code is returned |
| 271 | 249 | - Problem: Target platform reload in Eclipse. Sometimes reloading via Window -> Plug-in Development -> Target Platform doesn't work, so open the target definition itself and try reloading there. Often a restart proves helpful after the reload. In addition, you can clean all projects and rebuild; then rebuild the individual projects that fail. Sometimes the errors are actually just warnings and you can try to run the GWT SDM (remember the other SDM's must be run if everything is brand new). Lastly, try clearing the plugins content found at `WORKSPACE_HOME/.metadata/.plugins/org.eclipse.pde.core/.bundle_pool/plugins`. |
wiki/info/landscape/development-environment.md
| ... | ... | @@ -2,33 +2,41 @@ |
| 2 | 2 | |
| 3 | 3 | [[_TOC_]] |
| 4 | 4 | |
| 5 | -## Git and Our Branches |
|
| 6 | -Our main Git repository lives at ssh://<user>@sapsailing.com/home/trac/git. For those working in the SAP corporate network and therefore unable to access the external sapsailing.com server using SSH, the repository contents are replicated on an hourly basis into ssh://dxxxxxx@git.wdf.sap.corp:29418/SAPSail/sapsailingcapture.git where dxxxxxx obviously is to be replaced by your D- or I- or C-user. You need to have an account at https://git.wdf.sap.corp:8080/ to be able to access this Git repository. |
|
| 5 | +Here, we describe the process for doing simple standard development for the Sailing Analytics project, with a focus on how we handle Git w.r.t. branches, Bugzilla, Hudson-CI, and Github Actions. Other development scenarios you'll find described in more depth [[here|wiki/info/landscape/typical-development-scenarios]]. |
|
| 7 | 6 | |
| 8 | -Small, obvious and non-disruptive developments are usually carried out immediately on our master branch. This branch is configured such that Maven can be used to run the tests inside the SAP corporate network. The master branch is never deployed onto the sapsailing.com server and hence has no corresponding /home/trac/servers/ subdirectory. |
|
| 7 | +## Git, Bugzilla, and Our Branches |
|
| 8 | +Our main Git repository lives at github.com/SAP/sailing-analytics. Its ``main`` branch is mirrored to ssh://trac@sapsailing.com/home/trac/git periodically. |
|
| 9 | 9 | |
| 10 | -If a change looks reasonably good on the master branch and related JUnit tests or manual UI tests have succeeded locally, it is permissible to merge the master branch into the dev branch and run a central build on sapsailing.com. The dev branch is configured to run the Maven tests with direct Internet access. It can therefore also be used to run the tests locally if connected to the public Internet. |
|
| 10 | +Small, minor, obvious and non-disruptive developments are usually carried out immediately on our ``main`` branch. |
|
| 11 | 11 | |
| 12 | -Ideally, the build should be run including the test cases. If the tests succeed , the branch can be installed and the corresponding server instance can be restarted. The branch can then also be promoted to the next level (dev-->test-->prod1/prod2). Note, that currently re-starting a server instance may require re-loading the races that were previously loaded, particularly for the prod1 and prod2 instances, because several externally-announced URLs point to them. |
|
| 12 | +Everything else should follow the pattern |
|
| 13 | +- create Bugzilla issue (e.g., issue #12345) |
|
| 14 | +- create branch for Bugzilla issue ``bug12345``, typically branching from latest ``main`` tip |
|
| 15 | +- create Hudson job for branch using ``configuration/createHudsonJobForBug.sh 12345`` |
|
| 16 | +- make your changes on your branch and commit and push regularly |
|
| 17 | +- pushing triggers the [release workflow](https://github.com/SAP/sailing-analytics/actions/workflows/release.yml) which runs a build with tests |
|
| 18 | +- when the workflow has finished, it triggers your Hudson job which collects the [build and test results](https://hudson.sapsailing.com/job/bug12345) |
|
| 19 | +- be verbose and document your changes, progress, hold-ups and problems on your Bugzilla issue |
|
| 20 | +- when build is "green," suggest your branch for review; so far we do this informally by assigning the Bugzilla issue to the reviewer and in a comment asking for review; in the future, we may want to use Github Pull Requests for this |
|
| 21 | +- after your branch has been merged into ``main``, disable your Hudson build job for your branch |
|
| 22 | +- the ``main`` branch will then build a new release that you can roll out into the production landscape |
|
| 23 | +- in case of changes to i18n-related message properties files, merge ``main`` into ``translation`` which triggers the translation process; the completed translations will arrive as pushes to the ``translations`` branch, triggering another ``release`` workflow, and---if successful---an automated merge into ``main`` with the corresponding build/release process again |
|
| 24 | +- a successful ``main`` build (still on Java 8) will lead to an automatic merge into one or more branches for newer Java releases (such as ``docker-24``) with the corresponding build/release process |
|
| 13 | 25 | |
| 14 | -We typically promote the changes to the next branch by also merging the current master branch into the next-"higher" branch. This should lead to equivalent results compared to merging the "previous" branch (e.g., dev) into the "next" branch (e.g., test). The branches differ largely in the configurations used for the servers, particularly the port assignments for the Jetty web server, the UDP ports used for listening for Expedition wind messages, and the queue names used for the replication based on RabbitMQ (see also Scale-Out through Replication). |
|
| 26 | +Be eager to equip your features and functions with tests. There should be enough examples to learn from. For UI testing, use Selenium (see the ``java/com.sap.sailing.selenium.test`` project). |
|
| 15 | 27 | |
| 16 | -## Eclipse Setup, Required Plug-Ins |
|
| 17 | -We use Eclipse as our development environment. Using a different IDE would make it hard to re-use the project configurations (.project and .classpath files which are specific to Eclipse) as well as the GWT plug-in which assists in locally compiling and refactoring the GWT code, in particular the RPC code. |
|
| 18 | - |
|
| 19 | -The recommended and tested Eclipse version is currently Indigo (3.7). Colleagues have reported that they succeeded with a Juno (3.8/4.x) set-up as well. |
|
| 20 | - |
|
| 21 | -To get started, install Eclipse with at least PDE, Git, GWT (use http://dl.google.com/eclipse/plugin/3.7 as update site) and JSP editing support enabled. Eclipse Maven support is not recommended as in many cases it has caused trouble with the local Eclipse build. |
|
| 22 | - |
|
| 23 | -## Target Platform |
|
| 24 | -After the Eclipse installation and importing all projects under java/ from Git, it is required to set the target platform in Eclipse (Window --> Preferences --> Plugin Development --> Target Platform). The project com.sap.sailing.targetplatform contains "Race Analysis Target (IDE)" as target platform definition. It uses a number of p2 update sites and defines the OSGi bundles that constitute the target platform for the application. If this is not set in Eclipse, the local build environment assumes that the developer wants to implement Eclipse plug-ins and offers the entire set of Eclipse plug-ins and only those as the target platform which doesn't make any sense for our application. |
|
| 28 | +### Exceptionally Building Without Running Tests, More/Fewer CPUs, and With Release |
|
| 29 | +Ideally, the build should be run including the test cases. However, for exceptional cases you can trigger a build using the ``release`` workflow in Github Actions manually and can choose to ignore tests, change the number of CPUs to use for the build, and run the build with an OSGi target platform built according to the specifications of the branch you're building from. |
|
| 25 | 30 | |
| 26 | -Major parts of our target platform are hosted on sapsailing.com as a p2 repository which makes it possible to have only one central target platform configuration used by everyone. The target platform can be re-built, e.g., after adding another bundle to it, using the script in com.sap.sailing.targetplatform/scripts. |
|
| 31 | +Furthermore, if you push your branch, say ``bug12345`` to ``releases/bug12345`` then the Github Actions build triggered by the push will also build and publish a release (currently published on [https://releases.sapsailing.com](https://releases.sapsailing.com)) named after your branch. You can use such as release, e.g., to deploy it to a staging server such as [https://dev.sapsailing.com](https://dev.sapsailing.com). |
|
| 27 | 32 | |
| 28 | -It then needs to be installed again (by using a tool like scp for instance) to the /home/trac/p2-repositories directory from where it is exposed as http://p2.sapsailing.com/p2/sailing/ by the Apache web server. After such a change, all developers need to reload the target platform into their Eclipse environment. |
|
| 33 | +## Eclipse Setup, Required Plug-Ins |
|
| 34 | +The Eclipse setup is explained in our [[Onboarding|wiki/howto/onboarding]] description. |
|
| 29 | 35 | |
| 30 | 36 | ## Maven Build and Tests |
| 31 | -We use Maven to build our software and run our JUnit tests. The global setting.xml file to install in everyone's ~/.m2 directory is checked into the top-level Git folder. The checked-in copy assumes the developer is using Maven inside the SAP corporate network. If not, uncomment the <proxy> tag in the settings.xml file. See also section Git and Our Branches for details on which branch is configured to work in which network setup. |
|
| 37 | +We recommend using a local Maven-based build only if you try to understand or reproduce issues with the Github Actions build. In most other cases you should be fine using Eclipse with its local build/run/debug cycle. |
|
| 38 | + |
|
| 39 | +If you still feel you want to run a local Maven build, make sure again (see also [[Onboarding|wiki/howto/onboarding]]) to get your ``maven-settings.xml`` and ``toolchains.xml`` files in ``~/.m2`` in good shape first. |
|
| 32 | 40 | |
| 33 | 41 | We have a top-level Maven pom.xml configuration file in the root folder of our Git workspace. It delegates to the pom.xml file in the java/ folder where all the bundle projects are defined. We rely on the Tycho Maven plug-in to build our OSGi bundles, also known as the "manifest-first approach." The key idea is to mainly develop using Eclipse means, including its OSGi manifest editing capabilities, and keep the Maven infrastructure as simple as possible, deriving component dependencies from the OSGi manifests. See the various pom.xml files in the projects to see the project-specific settings. By and large, a pom.xml file for a bundle needs to have the bundle name and version defined (we currently have most bundles at version 1.0.0.qualifier in the manifest or 1.0.0.SNAPSHOT in Maven), and whether the bundle is a test or non-test bundle, expressed as the packaging type which here can be one of eclipse-plugin or ecplise-test-plugin. |
| 34 | 42 | |
| ... | ... | @@ -52,18 +60,9 @@ When building on sapsailing.com you should stick with the buildAndUpdateProduct. |
| 52 | 60 | |
| 53 | 61 | All these build lines also creates a log file with all error messages, just in case the screen buffer is not sufficient to hold all scrolling error messages. |
| 54 | 62 | |
| 55 | -## Automated Builds using Hudson |
|
| 56 | - |
|
| 57 | -The project uses a Hudson build server installation that can be reached at [hudson.sapsailing.com](http://hudson.sapsailing.com). Please ask a project administrator for an account. This Hudson server builds all new commits pushed to the master branch, performs the JUnit tests and publishes the JUnit test results. New jobs for other branches can easily be created by copying from the SAPSailingAnalytics-master job and updating the git branch to be checked out for build. This way, you can create your own job for your own branch. Don't forget to set yourself as the e-mail recipient for failing builds. |
|
| 58 | - |
|
| 59 | -As a special feature, release builds can automatically be performed and published to [releases.sapsailing.com](http:///releases.sapsailing.com) by pushing the tag named "release" to the version that you want to release. This can be done using the following series of git commands: |
|
| 63 | +When you're done with your local Maven build, finally run "mvn clean" to clean up the artifacts produced by the Maven build. Without this, remnants and outputs from the Maven build may collide with the local Eclipse build, such as JAR files that ended up in projects' ``bin/`` folders. |
|
| 60 | 64 | |
| 61 | - git tag -f release |
|
| 62 | - git push origin release:release |
|
| 63 | - |
|
| 64 | -You can follow the build triggered by this [here](http://hudson.sapsailing.com/job/SAPSailingAnalytics-release/). |
|
| 65 | - |
|
| 66 | -### Plotting test results with the Measurement Plugin |
|
| 65 | +## Plotting test results with the Measurement Plugin |
|
| 67 | 66 | |
| 68 | 67 | By default the duration of each test is published and can be viewed in comparison with older builds. It is possible to publish other values using the Measurement Plugin, which reads them out of a `MeasurementXMLFile`. |
| 69 | 68 | |
| ... | ... | @@ -72,19 +71,3 @@ MeasurementXMLFile performanceReport = new MeasurementXMLFile("TEST-" + getClass |
| 72 | 71 | MeasurementCase performanceReportCase = performanceReport.addCase(getClass().getSimpleName()); |
| 73 | 72 | performanceReportCase.addMeasurement(new Measurement("MeasurementName", measurementValue)); |
| 74 | 73 | ``` |
| 75 | - |
|
| 76 | -## Product, Features and Target Platform |
|
| 77 | -The result of the build process is a p2 repository with a product consisting of a number of features. The product configuration is provided by the file raceanalysis.product in the com.sap.sailing.feature.p2build project. In its dependencies it defines the features of which it is built, which currently are com.sap.sailing.feature and com.sap.sailing.feature.runtime, each described in an equal-named bundle. The feature specified by com.sap.sailing.feature lists the bundles we develop ourselves as part of the project. The com.sap.sailing.feature.runtime feature lists those 3rd-party bundles from the target platform which are required by the product. |
|
| 78 | - |
|
| 79 | -The [target platform](#development-environment_target-platform) is defined in the various flavors for local and central environments in com.sap.sailing.targetplatform/definitions/*.target. It mainly uses Eclipse p2 repositories and our own p2 repository at http://p2.sapsailing.com/p2/sailing/ where we store those bundles required by our runtime which cannot be found as OSGi bundles in any other public p2 repository of which we are aware. |
|
| 80 | - |
|
| 81 | -This p2 repository at sapsailing.com can be re-built and correspondingly extended by the process explained [here](wiki/typical-development-scenarios#Adding-a-Bundle-to-the-Target-Platform). |
|
| 82 | - |
|
| 83 | -## External Libraries |
|
| 84 | - |
|
| 85 | -### Highcharts and jQuery |
|
| 86 | -We use the Highcharts library to present graphs to the user. These graphs are used on the RaceBoardPanel and (at the time of writing still under development) the PolarSheetsPanel. In the past, there were difficulties concerning the versions of the three interacting libraries: |
|
| 87 | - |
|
| 88 | -* The GWT Highcharts Wrapper – The source code can be found in our project and it’s slightly modified to match our scenario |
|
| 89 | -* The actual Highcharts Library |
|
| 90 | -* The jQuery Library |
|
| ... | ... | \ No newline at end of file |