wiki/info/landscape/olympic-setup.md
... ...
@@ -684,4 +684,34 @@ See the introduction of this main section. Synopsis:
684 684
-R release name; must be provided to select the release, e.g., build-202106040947
685 685
-t Instance type; defaults to
686 686
-s Skip release download
687
-</pre>
... ...
\ No newline at end of file
0
+</pre>
1
+
2
+## Log File Analysis
3
+
4
+Athena table definitions and queries have been provided in region ``eu-west-3`` (Paris) where we hosted our EU part during the event after a difficult start in ``eu-west-1`` with the single MongoDB live replica set not scaling well for all the replicas that were required in the region.
5
+
6
+The key to the Athena set-up is to have a table definition per bucket, with a dedicated S3 bucket per region where ALB logs were recorded. An example of a query based on the many tables the looks like this:
7
+<pre>
8
+ with union_table AS
9
+ (select *
10
+ from alb_logs_ap_northeast_1
11
+ union all
12
+ select *
13
+ from alb_logs_ap_southeast_2
14
+ union all
15
+ select *
16
+ from alb_logs_eu_west_3
17
+ union all
18
+ select *
19
+ from alb_logs_us_east_1
20
+ union all
21
+ select *
22
+ from alb_logs_us_west_1)
23
+ select date_trunc('day', parse_datetime(time,'yyyy-MM-dd''T''HH:mm:ss.SSSSSS''Z')), count(distinct concat(client_ip,user_agent))
24
+ from union_table
25
+ where (parse_datetime(time,'yyyy-MM-dd''T''HH:mm:ss.SSSSSS''Z')
26
+ between parse_datetime('2021-07-21-00:00:00','yyyy-MM-dd-HH:mm:ss')
27
+ and parse_datetime('2021-08-08-02:00:00','yyyy-MM-dd-HH:mm:ss'))
28
+ group by date_trunc('day', parse_datetime(time,'yyyy-MM-dd''T''HH:mm:ss.SSSSSS''Z'))
29
+</pre>
30
+It defines a ``union_table`` which unites all contents from all buckets scanned.
... ...
\ No newline at end of file