bug5779: adjusted comments and docs regarding LaunchTemplates

- <p>Creates an application replica set with an auto-scaling group and corresponding launch configuration, load

- load balancer being used) is removed, too, and so are the launch configuration and auto-scaling group

- * auto scaling groups and launch configurations for auto-scaling the number of replicas; change the software version running on an application server

- machineImagesTable.getSelectionModel().getSelectedObject()));

- * download and unpack the new release but will not yet stop the master process. In parallel, an existing

- * launch configuration will be copied and updated with user data reflecting the new release to be used.

- * An existing auto-scaling group will then be updated to use the new launch configuration. The old launch

- * configuration will then be removed.<p>

- * Updates the AMI to use in the launch configurations of those of the {@code replicaSets} that have an auto-scaling group.

- * hence no launch configuration to update

- * The {@code replicaInstanceType} is used to configure the launch configuration used by the auto-scaling group

- * The {@code replicaInstanceType} is used to configure the launch configuration used by the auto-scaling group

- * download and unpack the new release but will not yet stop the master process. In parallel, an existing

- * launch configuration will be copied and updated with user data reflecting the new release to be used.

- * An existing auto-scaling group will then be updated to use the new launch configuration. The old launch

- * configuration will then be removed.<p>

- * Updates the AMI to use in the launch configurations of those of the {@code replicaSets} that have an auto-scaling group.

- * hence no launch configuration to update

- * If the {@code replicaSet} provided has one or more auto-scaling groups, their launch configuration is adjusted such that it

- * matches the {@code optionalInstanceType}. The existing replicas managed currently by the auto-scaling group are

- * replaced one by one with new instances with the new configuration. This happens by setting the auto-scaling

- * group's new minimum size to the current number of instances managed by the auto-scaling group plus one, then

- * waiting for the new instance to become available, and then terminating one of the old auto-scaling group managed

- * replicas, again waiting for the one next new replica to become ready, and so on, until the last old auto-scaling

- * replica has been stopped/terminated. Then, the auto-scaling group's minimum size is reset to what it was when

- * this method was called.

-import com.sap.sailing.landscape.procedures.CreateLaunchConfigurationAndAutoScalingGroup;

- * The {@code replicaInstanceType} is used to configure the launch configuration used by the auto-scaling group

- // construct a replica configuration which is used to produce the user data for the launch configuration used in an auto-scaling group

- final CreateLaunchConfigurationAndAutoScalingGroup.Builder<String, ?, SailingAnalyticsMetrics, SailingAnalyticsProcess<String>> createLaunchConfigurationAndAutoScalingGroupBuilder =

- CreateLaunchConfigurationAndAutoScalingGroup.builder(landscape, region, replicaSetName, createLoadBalancerMapping.getPublicTargetGroup());

- createLaunchConfigurationAndAutoScalingGroupBuilder

- minimumNumberOfReplicas.ifPresent(minNumberOfReplicas->createLaunchConfigurationAndAutoScalingGroupBuilder.setMinReplicas(minNumberOfReplicas));

- maximumNumberOfReplicas.ifPresent(maxNumberOfReplicas->createLaunchConfigurationAndAutoScalingGroupBuilder.setMaxReplicas(maxNumberOfReplicas));

- createLaunchConfigurationAndAutoScalingGroupBuilder.setImage(replicaMachineImage.get());

- createLaunchConfigurationAndAutoScalingGroupBuilder.setImage(

- createLaunchConfigurationAndAutoScalingGroupBuilder.setKeyName(optionalKeyName);

- createLaunchConfigurationAndAutoScalingGroupBuilder.build().run();

- " does not have an auto-scaling group configured, so no changes can be made to its launch configuration.");

-package com.sap.sailing.landscape.procedures;

-

-import java.util.Optional;

-

-import com.sap.sse.landscape.Region;

-import com.sap.sse.landscape.Release;

-import com.sap.sse.landscape.application.ApplicationProcessMetrics;

-import com.sap.sse.landscape.application.ApplicationReplicaSet;

-import com.sap.sse.landscape.aws.AmazonMachineImage;

-import com.sap.sse.landscape.aws.AwsApplicationProcess;

-import com.sap.sse.landscape.aws.AwsAutoScalingGroup;

-import com.sap.sse.landscape.aws.AwsLandscape;

-import com.sap.sse.landscape.aws.Tags;

-import com.sap.sse.landscape.aws.TargetGroup;

-import com.sap.sse.landscape.aws.orchestration.AwsApplicationConfiguration;

-import com.sap.sse.landscape.orchestration.AbstractProcedureImpl;

-import com.sap.sse.landscape.orchestration.Procedure;

-

-import software.amazon.awssdk.services.ec2.model.InstanceType;

-

-/**

- * For an {@link ApplicationReplicaSet} and a {@link TargetGroup} that represents the application replica set's public

- * target group creates an AWS EC2 Launch Configuration and a corresponding Auto-Scaling Group which by default produces

- * a minimum of one replica, a maximum of 30 replicas, scaling based on the number of requests per target which are

- * supposed to not exceed 15,000 per minute. An {@link AwsApplicationConfiguration} for the replica's config must be

- * provided and is expected to use the same {@link Release} as the master.

- * <p>

- *

- * The builder for the procedure requires the bearer token for replication permissions and the target group. Other

- * properties are optional.

- *

- * @author Axel Uhl (D043530)

- *

- * @param <ShardingKey>

- */

-public class CreateLaunchConfigurationAndAutoScalingGroup<ShardingKey, MetricsT extends ApplicationProcessMetrics, ProcessT extends AwsApplicationProcess<ShardingKey,MetricsT,ProcessT>>

-extends AbstractProcedureImpl<ShardingKey>

-implements Procedure<ShardingKey> {

- private static final int DEFAULT_MIN_REPLICAS = 1;

- private static final int DEFAULT_MAX_REPLICAS = 30;

-

- /**

- *

- * @author Axel Uhl (D043530)

- */

- public static interface Builder<ShardingKey, BuilderT extends Builder<ShardingKey, BuilderT, MetricsT, ProcessT>, MetricsT extends ApplicationProcessMetrics, ProcessT extends AwsApplicationProcess<ShardingKey,MetricsT,ProcessT>>

- extends com.sap.sse.landscape.orchestration.Procedure.Builder<BuilderT, CreateLaunchConfigurationAndAutoScalingGroup<ShardingKey, MetricsT, ProcessT>, ShardingKey> {

- BuilderT setKeyName(String keyName);

-

- BuilderT setReplicaConfiguration(AwsApplicationConfiguration<ShardingKey, MetricsT, ProcessT> applicationConfiguration);

-

- BuilderT setInstanceType(InstanceType instanceType);

-

- BuilderT setImage(AmazonMachineImage<ShardingKey> image);

-

- BuilderT setTags(Tags tags);

-

- /**

- * Set the minimum number of replicas. Defaults to 1 (see {@link CreateLaunchConfigurationAndAutoScalingGroup#DEFAULT_MIN_REPLICAS}).

- */

- BuilderT setMinReplicas(int minReplicas);

-

- /**

- * Set the maximum number of replicas. Defaults to 30 (see {@link CreateLaunchConfigurationAndAutoScalingGroup#DEFAULT_MAX_REPLICAS}).

- */

- BuilderT setMaxReplicas(int maxReplicas);

-

- /**

- * Defines the scaling threshold based on the number of requests per target per minute. Defaults to 15,000 (see

- * {@link AwsAutoScalingGroup#DEFAULT_MAX_REQUESTS_PER_TARGET}).

- */

- BuilderT setMaxRequestsPerTarget(int maxRequestsPerTarget);

- }

-

- protected static class BuilderImpl<ShardingKey, BuilderT extends Builder<ShardingKey, BuilderT, MetricsT, ProcessT>, MetricsT extends ApplicationProcessMetrics, ProcessT extends AwsApplicationProcess<ShardingKey,MetricsT,ProcessT>>

- extends AbstractProcedureImpl.BuilderImpl<BuilderT, CreateLaunchConfigurationAndAutoScalingGroup<ShardingKey, MetricsT, ProcessT>, ShardingKey>

- implements Builder<ShardingKey, BuilderT, MetricsT, ProcessT> {

- private final String replicaSetName;

- private final TargetGroup<ShardingKey> targetGroup;

- private final Region region;

- private String keyName;

- private AmazonMachineImage<ShardingKey> image;

- private InstanceType instanceType;

- private AwsApplicationConfiguration<ShardingKey, MetricsT, ProcessT> replicaConfiguration;

- private Optional<Tags> tags;

- private int minReplicas = DEFAULT_MIN_REPLICAS;

- private int maxReplicas = DEFAULT_MAX_REPLICAS;

- private int maxRequestsPerTarget = AwsAutoScalingGroup.DEFAULT_MAX_REQUESTS_PER_TARGET;

-

- public BuilderImpl(AwsLandscape<ShardingKey> landscape, Region region, String replicaSetName,

- TargetGroup<ShardingKey> targetGroup) {

- this.replicaSetName = replicaSetName;

- this.targetGroup = targetGroup;

- this.region = region;

- setLandscape(landscape);

- }

-

- @Override

- public BuilderT setKeyName(String keyName) {

- this.keyName = keyName;

- return self();

- }

-

- @Override

- public BuilderT setImage(AmazonMachineImage<ShardingKey> image) {

- this.image = image;

- return self();

- }

-

- @Override

- public BuilderT setInstanceType(InstanceType instanceType) {

- this.instanceType = instanceType;

- return self();

- }

-

- @Override

- public BuilderT setReplicaConfiguration(AwsApplicationConfiguration<ShardingKey, MetricsT, ProcessT> replicaConfiguration) {

- this.replicaConfiguration = replicaConfiguration;

- return self();

- }

-

- protected String getReplicaSetName() {

- return replicaSetName;

- }

-

- protected int getMinReplicas() {

- return minReplicas;

- }

-

- @Override

- public BuilderT setMinReplicas(int minReplicas) {

- this.minReplicas = minReplicas;

- return self();

- }

-

- protected int getMaxReplicas() {

- return maxReplicas;

- }

-

- @Override

- public BuilderT setMaxReplicas(int maxReplicas) {

- this.maxReplicas = maxReplicas;

- return self();

- }

-

- protected int getMaxRequestsPerTarget() {

- return maxRequestsPerTarget;

- }

-

- @Override

- public BuilderT setMaxRequestsPerTarget(int maxRequestsPerTarget) {

- this.maxRequestsPerTarget = maxRequestsPerTarget;

- return self();

- }

-

- protected TargetGroup<ShardingKey> getTargetGroup() {

- return targetGroup;

- }

-

- protected Region getRegion() {

- return region;

- }

-

- protected String getKeyName() {

- return keyName;

- }

-

- protected AmazonMachineImage<ShardingKey> getImage() {

- return image;

- }

-

- protected InstanceType getInstanceType() {

- return instanceType;

- }

-

- protected AwsApplicationConfiguration<ShardingKey, MetricsT, ProcessT> getReplicaConfiguration() {

- return replicaConfiguration;

- }

-

- protected Optional<Tags> getTags() {

- return tags;

- }

-

- @Override

- public BuilderT setTags(Tags tags) {

- this.tags = Optional.ofNullable(tags);

- return self();

- }

-

- @Override

- public CreateLaunchConfigurationAndAutoScalingGroup<ShardingKey, MetricsT, ProcessT> build() throws Exception {

- return new CreateLaunchConfigurationAndAutoScalingGroup<>(this);

- }

- }

-

- private final String replicaSetName;

- private final TargetGroup<ShardingKey> targetGroup;

- private final Region region;

- private final String keyName;

- private final InstanceType instanceType;

- private final String imageId;

- private final AwsApplicationConfiguration<ShardingKey, MetricsT, ProcessT> replicaConfiguration;

- private final Optional<Tags> tags;

- private final int minReplicas;

- private final int maxReplicas;

- private final int maxRequestsPerTarget;

-

- public static <ShardingKey, BuilderT extends Builder<ShardingKey, BuilderT, MetricsT, ProcessT>, MetricsT extends ApplicationProcessMetrics, ProcessT extends AwsApplicationProcess<ShardingKey,MetricsT,ProcessT>>

- Builder<ShardingKey, BuilderT, MetricsT, ProcessT> builder(

- AwsLandscape<ShardingKey> landscape, Region region,

- String replicaSetName,

- TargetGroup<ShardingKey> targetGroup) {

- return new BuilderImpl<>(landscape, region, replicaSetName, targetGroup);

- }

-

- protected <BuilderT extends Builder<ShardingKey, BuilderT, MetricsT, ProcessT>>

- CreateLaunchConfigurationAndAutoScalingGroup(BuilderImpl<ShardingKey, BuilderT, MetricsT, ProcessT> builder) {

- super(builder);

- this.region = builder.getRegion();

- this.replicaSetName = builder.getReplicaSetName();

- this.targetGroup = builder.getTargetGroup();

- this.keyName = builder.getKeyName();

- this.imageId = builder.getImage().getId();

- this.instanceType = builder.getInstanceType();

- this.replicaConfiguration = builder.getReplicaConfiguration();

- this.tags = builder.getTags();

- this.minReplicas = builder.getMinReplicas();

- this.maxReplicas = builder.getMaxReplicas();

- this.maxRequestsPerTarget = builder.getMaxRequestsPerTarget();

- }

-

- @Override

- public AwsLandscape<ShardingKey> getLandscape() {

- return (AwsLandscape<ShardingKey>) super.getLandscape();

- }

-

- @Override

- public void run() throws Exception {

- getLandscape().createLaunchTemplateAndAutoScalingGroup(region, replicaSetName, tags,

- targetGroup, keyName, instanceType, imageId, replicaConfiguration, minReplicas, maxReplicas, maxRequestsPerTarget);

- }

-}

- are required by each shard to ensure availability. They are created with the same launch configuration

- * filling the launch configuration's user data, including aspects such as the release and the image

- * {@link AwsApplicationReplicaSet#getAllAutoScalingGroups() All} auto-scaling groups of the replica set;

- * this is important because the method will eventually remove the old launch configuration used by those

- * auto-scaling groups, and this will fail if there is any other auto-scaling group still using it.

- * {@link AwsApplicationReplicaSet#getAllAutoScalingGroups() All} auto-scaling groups of the replica set;

- * this is important because the method will eventually remove the old launch configuration used by those

- * auto-scaling groups, and this will fail if there is any other auto-scaling group still using it.

- * {@link AwsApplicationReplicaSet#getAllAutoScalingGroups() All} auto-scaling groups of the replica set;

- * this is important because the method will eventually remove the old launch configuration used by those

- * auto-scaling groups, and this will fail if there is any other auto-scaling group still using it.

- return getEc2AsyncClient(getRegion(region)).describeLaunchTemplateVersionsPaginator(b->b.versions("$Default")).subscribe(response->

- .versions("$Default")).launchTemplateVersions().iterator().next();

- return CreateLaunchTemplateVersionRequest.builder().launchTemplateId(launchTemplateToCreateNewVersionFor.launchTemplateId()).sourceVersion("$Default");

- throw new IllegalArgumentException("At least one auto-scaling group must be provided for updating a launch configuration");

- logger.info("Creating launch configuration for replica set "+replicaSetName);

- .version("$Default").build());

- .launchTemplate(ltb->ltb.launchTemplateId(launchTemplateId).version("$Default"));

-- a launch configuration used by the auto-scaling group, named after the replica set with the release name appended, separated by a dash (-), e.g., "abc-build-202202142355"

-When an application replica set's replica processes are provided by the auto-scaling group, the corresponding launch configuration specifies the instance type used for the dedicated instances used to host the replica processes. If this instance type turns out to be inadequate for the situation, e.g., because the event hosted in the application replica set produces more CPU load than expected or produces more tracking data than assumed, the instance type can be changed for the launch configuration, with a rolling update being performed for all replica instances managed by the auto-scaling group.

-When the event or season or whatever you chose to assign an application replica set to has come to the end of its live workload with races and other updates no longer taking place, the application replica set is up for archiving. This will help save cost due to better resource utilization. The auto-scaling group, its launch configuration, its DNS record (unless the "dynamic" load balancing scenario in the default region eu-west-1 was used), its target groups and its load balancer listener rules can all be removed which frees up capacity for new or other live events. Furthermore, the processes can be stopped, freeing up memory and disk space on the instances they ran on, or even allowing for the termination of entire instances. Lastly, storage space in the ``live`` MongoDB replica set is more expensive than in the ``slow`` replica set that can be used for backup and archiving of DB content.

-Finally, if you ticked the "Remove archive replica set after successful verification", the application replica set will be completely removed by stopping its master and replica processes, removing all its load balancer rules, removing its two target groups, removing the auto-scaling group and the corresponding launch configuration and, if a DNS-based load balancer was used, removing its DNS record.

-This action is really only useful for application replica sets that were created for development, testing or debugging purposes. While its MongoDB database is left untouched, all other resources pertinent to the application replica set will be removed, including its load balancing rules, target groups, auto-scaling group, launch configuration, application processes and potentially the instances they ran on in case the processes were the last on their instance, and the optional DNS record.

-When upgrading the ``sailing-analytics-server`` AMI there is a good chance that the AMI you start with is used by one or more launch configurations that belong to auto-scaling groups and are used to launch new instances. Unfortunately, AWS doesn't keep you from removing old AMIs despite the fact that they are still referenced by one or more launch configurations that are in active use by their respective auto-scaling groups. So at some point you would want to upgrade those auto-scaling groups to use updated launch configurations which refer to the new AMI that results from the upgrade.

-After the AMI upgrade succeeds, you will see a pop-up dialog prompting you with a choice of whether you would like to update launch configurations for application replica sets that you selected in the table before upgrading the AMI, or in case you didn't pick any application replica sets suggesting all of them that currently use the AMI you just upgraded in their launch configuration. If you choose "OK" then all those auto-scaling groups will be updated so they point to new launch configurations copied from the previous ones, referencing the new AMI. The old launch configurations will be deleted. The names for the new launch configurations is constructed from the replica set name with the ID of the AMI appended to it. Note that running replicas are not affected by this.

-* while your master spins up, prepare a new Launch Configuration for the replicas that uses the new release; for this, find out which release your new master is running (e.g., `build-202012211912`), copy your existing Launch Configuration to one with a new name that reflects the new release, and edit the new Launch Configuration's "User Data" section, adjusting the `INSTALL_FROM_RELEASE` variable to the new release name. You find the "User Data" in the section "Additional configuration - optional" after expanding the "Advanced details" drop-down. Acknowledge your key at the bottom and save the new launch configuration.