tangled.org
Overview#
This pr implements the first part of this proposal and introduces one new environment variable
- LogBucket, when set logs will be uploaded to the specified bucket.
+224
-36
Diff
round #0
+19
-12
go.mod
+19
-12
go.mod
···
7
7
github.com/alecthomas/assert/v2 v2.11.0
8
8
github.com/alecthomas/chroma/v2 v2.23.1
9
9
github.com/avast/retry-go/v4 v4.6.1
10
-
github.com/aws/aws-sdk-go-v2 v1.41.1
11
-
github.com/aws/aws-sdk-go-v2/credentials v1.19.9
12
-
github.com/aws/aws-sdk-go-v2/service/s3 v1.96.0
10
+
github.com/aws/aws-sdk-go-v2 v1.41.4
11
+
github.com/aws/aws-sdk-go-v2/credentials v1.19.12
12
+
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.1
13
13
github.com/blevesearch/bleve/v2 v2.5.3
14
14
github.com/bluekeyes/go-gitdiff v0.8.1
15
15
github.com/bluesky-social/indigo v0.0.0-20260220055544-bf41e2ee75ab
···
70
70
github.com/adrg/frontmatter v0.2.0 // indirect
71
71
github.com/alecthomas/repr v0.5.2 // indirect
72
72
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be // indirect
73
-
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4 // indirect
74
-
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.17 // indirect
75
-
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.17 // indirect
76
-
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.17 // indirect
77
-
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 // indirect
78
-
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.8 // indirect
79
-
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.17 // indirect
80
-
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.17 // indirect
81
-
github.com/aws/smithy-go v1.24.0 // indirect
73
+
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.7 // indirect
74
+
github.com/aws/aws-sdk-go-v2/config v1.32.12 // indirect
75
+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20 // indirect
76
+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.20 // indirect
77
+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.20 // indirect
78
+
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 // indirect
79
+
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.21 // indirect
80
+
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 // indirect
81
+
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.12 // indirect
82
+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.20 // indirect
83
+
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.20 // indirect
84
+
github.com/aws/aws-sdk-go-v2/service/signin v1.0.8 // indirect
85
+
github.com/aws/aws-sdk-go-v2/service/sso v1.30.13 // indirect
86
+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.17 // indirect
87
+
github.com/aws/aws-sdk-go-v2/service/sts v1.41.9 // indirect
88
+
github.com/aws/smithy-go v1.24.2 // indirect
82
89
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
83
90
github.com/aymerick/douceur v0.2.0 // indirect
84
91
github.com/beorn7/perks v1.0.1 // indirect
+38
go.sum
+38
go.sum
···
27
27
github.com/avast/retry-go/v4 v4.6.1/go.mod h1:V6oF8njAwxJ5gRo1Q7Cxab24xs5NCWZBeaHHBklR8mA=
28
28
github.com/aws/aws-sdk-go-v2 v1.41.1 h1:ABlyEARCDLN034NhxlRUSZr4l71mh+T5KAeGh6cerhU=
29
29
github.com/aws/aws-sdk-go-v2 v1.41.1/go.mod h1:MayyLB8y+buD9hZqkCW3kX1AKq07Y5pXxtgB+rRFhz0=
30
+
github.com/aws/aws-sdk-go-v2 v1.41.4 h1:10f50G7WyU02T56ox1wWXq+zTX9I1zxG46HYuG1hH/k=
31
+
github.com/aws/aws-sdk-go-v2 v1.41.4/go.mod h1:mwsPRE8ceUUpiTgF7QmQIJ7lgsKUPQOUl3o72QBrE1o=
30
32
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4 h1:489krEF9xIGkOaaX3CE/Be2uWjiXrkCH6gUX+bZA/BU=
31
33
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4/go.mod h1:IOAPF6oT9KCsceNTvvYMNHy0+kMF8akOjeDvPENWxp4=
34
+
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.7 h1:3kGOqnh1pPeddVa/E37XNTaWJ8W6vrbYV9lJEkCnhuY=
35
+
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.7/go.mod h1:lyw7GFp3qENLh7kwzf7iMzAxDn+NzjXEAGjKS2UOKqI=
36
+
github.com/aws/aws-sdk-go-v2/config v1.32.12 h1:O3csC7HUGn2895eNrLytOJQdoL2xyJy0iYXhoZ1OmP0=
37
+
github.com/aws/aws-sdk-go-v2/config v1.32.12/go.mod h1:96zTvoOFR4FURjI+/5wY1vc1ABceROO4lWgWJuxgy0g=
32
38
github.com/aws/aws-sdk-go-v2/credentials v1.19.9 h1:sWvTKsyrMlJGEuj/WgrwilpoJ6Xa1+KhIpGdzw7mMU8=
33
39
github.com/aws/aws-sdk-go-v2/credentials v1.19.9/go.mod h1:+J44MBhmfVY/lETFiKI+klz0Vym2aCmIjqgClMmW82w=
40
+
github.com/aws/aws-sdk-go-v2/credentials v1.19.12 h1:oqtA6v+y5fZg//tcTWahyN9PEn5eDU/Wpvc2+kJ4aY8=
41
+
github.com/aws/aws-sdk-go-v2/credentials v1.19.12/go.mod h1:U3R1RtSHx6NB0DvEQFGyf/0sbrpJrluENHdPy1j/3TE=
42
+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20 h1:zOgq3uezl5nznfoK3ODuqbhVg1JzAGDUhXOsU0IDCAo=
43
+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20/go.mod h1:z/MVwUARehy6GAg/yQ1GO2IMl0k++cu1ohP9zo887wE=
34
44
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.17 h1:xOLELNKGp2vsiteLsvLPwxC+mYmO6OZ8PYgiuPJzF8U=
35
45
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.17/go.mod h1:5M5CI3D12dNOtH3/mk6minaRwI2/37ifCURZISxA/IQ=
46
+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.20 h1:CNXO7mvgThFGqOFgbNAP2nol2qAWBOGfqR/7tQlvLmc=
47
+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.20/go.mod h1:oydPDJKcfMhgfcgBUZaG+toBbwy8yPWubJXBVERtI4o=
36
48
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.17 h1:WWLqlh79iO48yLkj1v3ISRNiv+3KdQoZ6JWyfcsyQik=
37
49
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.17/go.mod h1:EhG22vHRrvF8oXSTYStZhJc1aUgKtnJe+aOiFEV90cM=
50
+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.20 h1:tN6W/hg+pkM+tf9XDkWUbDEjGLb+raoBMFsTodcoYKw=
51
+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.20/go.mod h1:YJ898MhD067hSHA6xYCx5ts/jEd8BSOLtQDL3iZsvbc=
52
+
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 h1:qYQ4pzQ2Oz6WpQ8T3HvGHnZydA72MnLuFK9tJwmrbHw=
53
+
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6/go.mod h1:O3h0IK87yXci+kg6flUKzJnWeziQUKciKrLjcatSNcY=
38
54
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.17 h1:JqcdRG//czea7Ppjb+g/n4o8i/R50aTBHkA7vu0lK+k=
39
55
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.17/go.mod h1:CO+WeGmIdj/MlPel2KwID9Gt7CNq4M65HUfBW97liM0=
56
+
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.21 h1:SwGMTMLIlvDNyhMteQ6r8IJSBPlRdXX5d4idhIGbkXA=
57
+
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.21/go.mod h1:UUxgWxofmOdAMuqEsSppbDtGKLfR04HGsD0HXzvhI1k=
40
58
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 h1:0ryTNEdJbzUCEWkVXEXoqlXV72J5keC1GvILMOuD00E=
41
59
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4/go.mod h1:HQ4qwNZh32C3CBeO6iJLQlgtMzqeG17ziAA/3KDJFow=
60
+
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 h1:5EniKhLZe4xzL7a+fU3C2tfUN4nWIqlLesfrjkuPFTY=
61
+
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7/go.mod h1:x0nZssQ3qZSnIcePWLvcoFisRXJzcTVvYpAAdYX8+GI=
42
62
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.8 h1:Z5EiPIzXKewUQK0QTMkutjiaPVeVYXX7KIqhXu/0fXs=
43
63
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.8/go.mod h1:FsTpJtvC4U1fyDXk7c71XoDv3HlRm8V3NiYLeYLh5YE=
64
+
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.12 h1:qtJZ70afD3ISKWnoX3xB0J2otEqu3LqicRcDBqsj0hQ=
65
+
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.12/go.mod h1:v2pNpJbRNl4vEUWEh5ytQok0zACAKfdmKS51Hotc3pQ=
44
66
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.17 h1:RuNSMoozM8oXlgLG/n6WLaFGoea7/CddrCfIiSA+xdY=
45
67
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.17/go.mod h1:F2xxQ9TZz5gDWsclCtPQscGpP0VUOc8RqgFM3vDENmU=
68
+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.20 h1:2HvVAIq+YqgGotK6EkMf+KIEqTISmTYh5zLpYyeTo1Y=
69
+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.20/go.mod h1:V4X406Y666khGa8ghKmphma/7C0DAtEQYhkq9z4vpbk=
46
70
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.17 h1:bGeHBsGZx0Dvu/eJC0Lh9adJa3M1xREcndxLNZlve2U=
47
71
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.17/go.mod h1:dcW24lbU0CzHusTE8LLHhRLI42ejmINN8Lcr22bwh/g=
72
+
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.20 h1:siU1A6xjUZ2N8zjTHSXFhB9L/2OY8Dqs0xXiLjF30jA=
73
+
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.20/go.mod h1:4TLZCmVJDM3FOu5P5TJP0zOlu9zWgDWU7aUxWbr+rcw=
48
74
github.com/aws/aws-sdk-go-v2/service/s3 v1.96.0 h1:oeu8VPlOre74lBA/PMhxa5vewaMIMmILM+RraSyB8KA=
49
75
github.com/aws/aws-sdk-go-v2/service/s3 v1.96.0/go.mod h1:5jggDlZ2CLQhwJBiZJb4vfk4f0GxWdEDruWKEJ1xOdo=
76
+
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.1 h1:csi9NLpFZXb9fxY7rS1xVzgPRGMt7MSNWeQ6eo247kE=
77
+
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.1/go.mod h1:qXVal5H0ChqXP63t6jze5LmFalc7+ZE7wOdLtZ0LCP0=
78
+
github.com/aws/aws-sdk-go-v2/service/signin v1.0.8 h1:0GFOLzEbOyZABS3PhYfBIx2rNBACYcKty+XGkTgw1ow=
79
+
github.com/aws/aws-sdk-go-v2/service/signin v1.0.8/go.mod h1:LXypKvk85AROkKhOG6/YEcHFPoX+prKTowKnVdcaIxE=
80
+
github.com/aws/aws-sdk-go-v2/service/sso v1.30.13 h1:kiIDLZ005EcKomYYITtfsjn7dtOwHDOFy7IbPXKek2o=
81
+
github.com/aws/aws-sdk-go-v2/service/sso v1.30.13/go.mod h1:2h/xGEowcW/g38g06g3KpRWDlT+OTfxxI0o1KqayAB8=
82
+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.17 h1:jzKAXIlhZhJbnYwHbvUQZEB8KfgAEuG0dc08Bkda7NU=
83
+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.17/go.mod h1:Al9fFsXjv4KfbzQHGe6V4NZSZQXecFcvaIF4e70FoRA=
84
+
github.com/aws/aws-sdk-go-v2/service/sts v1.41.9 h1:Cng+OOwCHmFljXIxpEVXAGMnBia8MSU6Ch5i9PgBkcU=
85
+
github.com/aws/aws-sdk-go-v2/service/sts v1.41.9/go.mod h1:LrlIndBDdjA/EeXeyNBle+gyCwTlizzW5ycgWnvIxkk=
50
86
github.com/aws/smithy-go v1.24.0 h1:LpilSUItNPFr1eY85RYgTIg5eIEPtvFbskaFcmmIUnk=
51
87
github.com/aws/smithy-go v1.24.0/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0=
88
+
github.com/aws/smithy-go v1.24.2 h1:FzA3bu/nt/vDvmnkg+R8Xl46gmzEDam6mZ1hzmwXFng=
89
+
github.com/aws/smithy-go v1.24.2/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
52
90
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
53
91
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
54
92
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
+54
-24
nix/gomod2nix.toml
+54
-24
nix/gomod2nix.toml
···
7
7
[mod."github.com/Blank-Xu/sql-adapter"]
8
8
version = "v1.1.1"
9
9
hash = "sha256-9AiQhXoNPCiViV+p5aa3qGFkYU4rJNbADvNdYGq4GA4="
10
+
[mod."github.com/BurntSushi/toml"]
11
+
version = "v0.3.1"
12
+
hash = "sha256-Rqak1dE/Aj/+Kx1/pl3Hifgt+Q3OzuZ5fJR+/x3nTbo="
10
13
[mod."github.com/Microsoft/go-winio"]
11
14
version = "v0.6.2"
12
15
hash = "sha256-tVNWDUMILZbJvarcl/E7tpSnkn7urqgSHa2Eaka5vSU="
···
16
19
[mod."github.com/RoaringBitmap/roaring/v2"]
17
20
version = "v2.4.5"
18
21
hash = "sha256-igWY0S1PTolQkfctYcmVJioJyV1pk2V81X6o6BA1XQA="
22
+
[mod."github.com/adrg/frontmatter"]
23
+
version = "v0.2.0"
24
+
hash = "sha256-WJsVcdCpkIkjqUz5fJOFStZYwQlrcFzQ6+mZatZiimo="
19
25
[mod."github.com/alecthomas/assert/v2"]
20
26
version = "v2.11.0"
21
27
hash = "sha256-tDJCDKZ0R4qNA7hgMKWrpDyogt1802LCJDBCExxdqaU="
···
33
39
version = "v4.6.1"
34
40
hash = "sha256-PeZc8k4rDV64+k8nZt/oy1YNVbLevltXP3ZD1jf6Z6k="
35
41
[mod."github.com/aws/aws-sdk-go-v2"]
36
-
version = "v1.41.1"
37
-
hash = "sha256-umafTZB+cuy8+Kzpl2WrlygJO3tR3D2WkTC0eTY5G/g="
42
+
version = "v1.41.4"
43
+
hash = "sha256-k9xv4f8YPSzZ1yR3/zuyNDGenZKk0DD4lceL713yXtc="
38
44
[mod."github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream"]
39
-
version = "v1.7.4"
40
-
hash = "sha256-ZY/Jn1p0IgDe8MONhp0RFHZmRgTBZZ5ddqXlNWEo7Ys="
45
+
version = "v1.7.7"
46
+
hash = "sha256-5hw1g8s21Gx1Q3JtiaVM6q8qGt7DW1YO9U1dEEqRXfE="
47
+
[mod."github.com/aws/aws-sdk-go-v2/config"]
48
+
version = "v1.32.12"
49
+
hash = "sha256-aTkdSRe8KPmVZdsunU8j/hZQLhGw1ckKpLN/ryRBZM0="
41
50
[mod."github.com/aws/aws-sdk-go-v2/credentials"]
42
-
version = "v1.19.9"
43
-
hash = "sha256-eqM5BmetQ/MrxTwoUCqRpkm5frFAspuq+QWod+5wzrU="
51
+
version = "v1.19.12"
52
+
hash = "sha256-xEIT1ARA9RYrQtLZIus71E6niNHIOVM1J7mUnA5AhJQ="
53
+
[mod."github.com/aws/aws-sdk-go-v2/feature/ec2/imds"]
54
+
version = "v1.18.20"
55
+
hash = "sha256-dCTpdKZheVCSt+R+NnFOnlS0bCt4gPavlDh15Kl/sMQ="
44
56
[mod."github.com/aws/aws-sdk-go-v2/internal/configsources"]
45
-
version = "v1.4.17"
46
-
hash = "sha256-W+d0WDYBrVJxdfRSvoe3cvZYgIgUSVyXsVdQlMcIZvc="
57
+
version = "v1.4.20"
58
+
hash = "sha256-aATIk4oLd7aaV66ereBdjINLMDwmIHxu+NNsgKWH1t4="
47
59
[mod."github.com/aws/aws-sdk-go-v2/internal/endpoints/v2"]
48
-
version = "v2.7.17"
49
-
hash = "sha256-dKJx0+K1DKi2LJKQNwnTofZH4GWLDgY6/ZI2XR2oCGI="
60
+
version = "v2.7.20"
61
+
hash = "sha256-G6266uj64sgfDTJ9V1UY1sQs3UmryB0CFgxzmbjjChY="
62
+
[mod."github.com/aws/aws-sdk-go-v2/internal/ini"]
63
+
version = "v1.8.6"
64
+
hash = "sha256-oIRPqu99vnGINAWKnCEytpv7N0gRWO7S72tb1r8oxvk="
50
65
[mod."github.com/aws/aws-sdk-go-v2/internal/v4a"]
51
-
version = "v1.4.17"
52
-
hash = "sha256-PKwNn9nFf+7TqocgQprxP9r1Fs4IxwGLGd05MrsxmLg="
66
+
version = "v1.4.21"
67
+
hash = "sha256-Sq4kRaiIFkPqwYnKv1dsxplyX4duipUYGFFT5gzJtJM="
53
68
[mod."github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding"]
54
-
version = "v1.13.4"
55
-
hash = "sha256-Rm6czqOnOULP080D97WQQSqkBhmN6ei1qZaTa51SRj8="
69
+
version = "v1.13.7"
70
+
hash = "sha256-AfYJdpmnW01Bk/jfHATlNU6lddjqcigFkHw/zcT9WO4="
56
71
[mod."github.com/aws/aws-sdk-go-v2/service/internal/checksum"]
57
-
version = "v1.9.8"
58
-
hash = "sha256-DZhR0aqHrgAFBGSlnsSQ6XeAJ/q504RG/LBWhtQqRVg="
72
+
version = "v1.9.12"
73
+
hash = "sha256-gze5CLp6Cp3X1jdqawWlXG7LToH+L6L/79b8F/AgWIY="
59
74
[mod."github.com/aws/aws-sdk-go-v2/service/internal/presigned-url"]
60
-
version = "v1.13.17"
61
-
hash = "sha256-qS7Db9S/KQ24kvJdL8qH4gnoN116J1ezwOnbovCiFEI="
75
+
version = "v1.13.20"
76
+
hash = "sha256-a5TifKunIoqKd2uAceYh6F1LvMHMyEQcWvJf0sxKhPM="
62
77
[mod."github.com/aws/aws-sdk-go-v2/service/internal/s3shared"]
63
-
version = "v1.19.17"
64
-
hash = "sha256-x/Cb4j3HFlg1+U21YCAIhBnfpf1yehJU2Ss/PxamEMI="
78
+
version = "v1.19.20"
79
+
hash = "sha256-T+pXMFELor69MTIo67TIkcWWyK+wubweg0gS5I9Dp8A="
65
80
[mod."github.com/aws/aws-sdk-go-v2/service/s3"]
66
-
version = "v1.96.0"
67
-
hash = "sha256-lzAn2KHIkd742mDwEZqGEaIXlEfvltL6HdP+sEQ/8YA="
81
+
version = "v1.97.1"
82
+
hash = "sha256-J51tttUIPFVEnHddftY4CJXKe2WLLIQDhJ8eYM8jgts="
83
+
[mod."github.com/aws/aws-sdk-go-v2/service/signin"]
84
+
version = "v1.0.8"
85
+
hash = "sha256-o4pWg3yMZHxdI94x5Z6qbiRg7gpmzbpJnJWsR1BOc44="
86
+
[mod."github.com/aws/aws-sdk-go-v2/service/sso"]
87
+
version = "v1.30.13"
88
+
hash = "sha256-V277a0ikm/H0paIeDLtPGEyav2a69Kdb9d5bh+JLAeY="
89
+
[mod."github.com/aws/aws-sdk-go-v2/service/ssooidc"]
90
+
version = "v1.35.17"
91
+
hash = "sha256-r5V5DoCIR4yzN1Ttg+dIA85GVkWMPgeD6Zu0rWGqNJE="
92
+
[mod."github.com/aws/aws-sdk-go-v2/service/sts"]
93
+
version = "v1.41.9"
94
+
hash = "sha256-I15uxeoKxDURsZrEVDzCRtVIu/HE756M1Rt7PPpdZ7c="
68
95
[mod."github.com/aws/smithy-go"]
69
-
version = "v1.24.0"
70
-
hash = "sha256-ZPFhf2Yv3BQpUn3cN4wSnoO7uBki8oCisZxL6F09nnE="
96
+
version = "v1.24.2"
97
+
hash = "sha256-v0y+Lir61fgdCwdVoca5mK+FcGh9OD3cTEwHIfLytcI="
71
98
[mod."github.com/aymanbagabas/go-osc52/v2"]
72
99
version = "v2.0.1"
73
100
hash = "sha256-6Bp0jBZ6npvsYcKZGHHIUSVSTAMEyieweAX2YAKDjjg="
···
696
723
[mod."gopkg.in/warnings.v0"]
697
724
version = "v0.1.2"
698
725
hash = "sha256-ATVL9yEmgYbkJ1DkltDGRn/auGAjqGOfjQyBYyUo8s8="
726
+
[mod."gopkg.in/yaml.v2"]
727
+
version = "v2.4.0"
728
+
hash = "sha256-uVEGglIedjOIGZzHW4YwN1VoRSTK8o0eGZqzd+TNdd0="
699
729
[mod."gopkg.in/yaml.v3"]
700
730
version = "v3.0.1"
701
731
hash = "sha256-FqL9TKYJ0XkNwJFnq9j0VvJ5ZUU1RvH/52h/f5bkYAU="
+10
nix/modules/spindle.nix
+10
nix/modules/spindle.nix
···
109
109
default = "5m";
110
110
description = "Timeout for each step of a pipeline";
111
111
};
112
+
113
+
logBucket = mkOption {
114
+
type = types.str;
115
+
default = "tangled-logs";
116
+
description = "S3 bucket for workflow logs";
117
+
};
112
118
};
113
119
};
114
120
};
···
138
144
"SPINDLE_SERVER_SECRETS_OPENBAO_MOUNT=${cfg.server.secrets.openbao.mount}"
139
145
"SPINDLE_NIXERY_PIPELINES_NIXERY=${cfg.pipelines.nixery}"
140
146
"SPINDLE_NIXERY_PIPELINES_WORKFLOW_TIMEOUT=${cfg.pipelines.workflowTimeout}"
147
+
"SPINDLE_NIXERY_PIPELINES_LOG_BUCKET=${cfg.pipelines.logBucket}"
148
+
"AWS_ACCESS_KEY_ID=${builtins.getEnv "AWS_ACCESS_KEY_ID"}"
149
+
"AWS_SECRET_ACCESS_KEY=${builtins.getEnv "AWS_SECRET_ACCESS_KEY"}"
150
+
"AWS_REGION=${builtins.getEnv "AWS_REGION"}"
141
151
];
142
152
ExecStart = "${cfg.package}/bin/spindle";
143
153
Restart = "always";
+5
spindle/config/config.go
+5
spindle/config/config.go
···
41
41
WorkflowTimeout string `env:"WORKFLOW_TIMEOUT, default=5m"`
42
42
}
43
43
44
+
type S3 struct {
45
+
LogBucket string `env:"LOG_BUCKET"`
46
+
}
47
+
44
48
type Config struct {
45
49
Server Server `env:",prefix=SPINDLE_SERVER_"`
46
50
NixeryPipelines NixeryPipelines `env:",prefix=SPINDLE_NIXERY_PIPELINES_"`
51
+
S3 S3 `env:",prefix=SPINDLE_S3_"`
47
52
}
48
53
49
54
func Load(ctx context.Context) (*Config, error) {
+27
spindle/engine/engine.go
+27
spindle/engine/engine.go
···
3
3
import (
4
4
"context"
5
5
"errors"
6
+
"fmt"
6
7
"log/slog"
8
+
"path/filepath"
7
9
"sync"
8
10
9
11
securejoin "github.com/cyphar/filepath-securejoin"
···
50
52
Name: w.Name,
51
53
}
52
54
55
+
defer func() {
56
+
logBucket := cfg.S3.LogBucket
57
+
58
+
if logBucket != "" {
59
+
logFile := filepath.Join(cfg.Server.LogDir, fmt.Sprintf("%s.log", wid.String()))
60
+
if err := uploadWorkflowLogs(ctx, logFile, "tangled-demo"); err != nil {
61
+
l.Error("error uploading logs", "err", err)
62
+
}
63
+
}
64
+
}()
65
+
53
66
wfLogger, err := models.NewFileWorkflowLogger(cfg.Server.LogDir, wid, secretValues)
54
67
if err != nil {
55
68
l.Warn("failed to setup step logger; logs will not be persisted", "error", err)
···
131
144
wg.Wait()
132
145
l.Info("all workflows completed")
133
146
}
147
+
148
+
func uploadWorkflowLogs(ctx context.Context, logfile, bucket string) error {
149
+
s3, err := NewS3(bucket)
150
+
if err != nil {
151
+
return fmt.Errorf("error creating s3 client: %w", err)
152
+
}
153
+
154
+
name := filepath.Join(logfile)
155
+
if err := s3.WriteFile(ctx, name); err != nil {
156
+
return fmt.Errorf("error saving logs: %w", err)
157
+
}
158
+
159
+
return nil
160
+
}
+71
spindle/engine/s3.go
+71
spindle/engine/s3.go
···
1
+
package engine
2
+
3
+
import (
4
+
"context"
5
+
"fmt"
6
+
"io"
7
+
"os"
8
+
"path/filepath"
9
+
10
+
"github.com/aws/aws-sdk-go-v2/aws"
11
+
"github.com/aws/aws-sdk-go-v2/config"
12
+
"github.com/aws/aws-sdk-go-v2/service/s3"
13
+
)
14
+
15
+
type S3 struct {
16
+
bucket string
17
+
client *s3.Client
18
+
}
19
+
20
+
const BASE_S3_PATH = "spindle/workflows"
21
+
22
+
func NewS3(bucket string) (*S3, error) {
23
+
ctx := context.Background()
24
+
sdkConfig, err := config.LoadDefaultConfig(ctx)
25
+
26
+
if err != nil {
27
+
return nil, fmt.Errorf("error loading s3 config: %w", err)
28
+
}
29
+
s3Client := s3.NewFromConfig(sdkConfig)
30
+
31
+
return &S3{
32
+
bucket: bucket,
33
+
client: s3Client,
34
+
}, nil
35
+
}
36
+
37
+
func (s *S3) WriteFile(ctx context.Context, path string) error {
38
+
s3_key := fmt.Sprintf("%s/%s", BASE_S3_PATH, filepath.Base(path))
39
+
40
+
file, err := os.Open(path)
41
+
if err != nil {
42
+
return fmt.Errorf("error opening file %s: %w", path, err)
43
+
}
44
+
defer file.Close()
45
+
46
+
_, err = s.client.PutObject(ctx, &s3.PutObjectInput{
47
+
Bucket: &s.bucket,
48
+
Key: &s3_key,
49
+
Body: file,
50
+
})
51
+
52
+
if err != nil {
53
+
return fmt.Errorf("error writing to s3: %w", err)
54
+
}
55
+
56
+
return nil
57
+
}
58
+
59
+
func (s *S3) ReadFile(ctx context.Context, name string) ([]byte, error) {
60
+
res, err := s.client.GetObject(ctx, &s3.GetObjectInput{
61
+
Bucket: &s.bucket,
62
+
Key: aws.String(name),
63
+
})
64
+
65
+
if err != nil {
66
+
return nil, fmt.Errorf("error reading file %s: %w", name, err)
67
+
}
68
+
defer res.Body.Close()
69
+
70
+
return io.ReadAll(res.Body)
71
+
}
History
10 rounds
3 comments
jobala.tngl.sh
submitted
#9
2 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
no conflicts, ready to merge
expand 0 comments
jobala.tngl.sh
submitted
#8
10 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle/engine: remove unrelated changes
expand 0 comments
jobala.tngl.sh
submitted
#7
12 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle/engine: remove unrelated changes
spindle/engine: add overwritten changes
spindle/engine: fix typo
expand 0 comments
jobala.tngl.sh
submitted
#6
11 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle/engine: remove unrelated changes
spindle/engine: add overwritten changes
expand 0 comments
jobala.tngl.sh
submitted
#5
10 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle/engine: remove unrelated changes
expand 0 comments
jobala.tngl.sh
submitted
#4
9 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
expand 0 comments
jobala.tngl.sh
submitted
#3
8 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
expand 0 comments
jobala.tngl.sh
submitted
#2
7 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
expand 1 comment
jobala.tngl.sh
submitted
#1
expand 0 comments
jobala.tngl.sh
submitted
#0
1 commit
expand
collapse
spindle/engine: store workflow logs in s3
expand 2 comments
spindle/config/config.go:44-51 — this will result in SPINDLE_S3_LOG_BUCKET, but
nix/modules/spindle.nix:147 we reference SPINDLE_NIXERY_PIPELINES_LOG_BUCKET here.
spindle/engine/engine.go:60 — probably good to parameterize the bucket name from config as well!
spindle/engine/s3.go:20 — Go convention is camel case, so BaseS3Path is preferred here.
spindle/engine/s3.go:38 — also camel case here please: s3Key
spindle/engine/engine.go:154 — this is a no-op with just a single argument.
nix/modules/spindle.nix:148-150 — we should ideally use EnvironmentFile here. Refer to how secrets are passed through to the appview service, for example.
High level design concern: NewS3 creates a new S3 client for each workflow (on every defer). We should ideally initialize this upon spindle startup and passed through via the engine struct.
We should ideally initialize this upon spindle startup and passed through via the engine struct.
Doing this means that each engine will have to register a clean up task for uploading logs which is easy to forget. Doing it in spindle/engine means that new engines don't have to worry about how to upload logs.
sorry, but you will need to rebase with master before submitting!