tangled.org
Overview#
This pr implements the first part of this proposal and introduces one new environment variable
- LogBucket, when set logs will be uploaded to the specified bucket.
+235
-36
Diff
round #2
+19
-12
go.mod
+19
-12
go.mod
···
7
7
github.com/alecthomas/assert/v2 v2.11.0
8
8
github.com/alecthomas/chroma/v2 v2.23.1
9
9
github.com/avast/retry-go/v4 v4.6.1
10
-
github.com/aws/aws-sdk-go-v2 v1.41.1
11
-
github.com/aws/aws-sdk-go-v2/credentials v1.19.9
12
-
github.com/aws/aws-sdk-go-v2/service/s3 v1.96.0
10
+
github.com/aws/aws-sdk-go-v2 v1.41.4
11
+
github.com/aws/aws-sdk-go-v2/credentials v1.19.12
12
+
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.1
13
13
github.com/blevesearch/bleve/v2 v2.5.3
14
14
github.com/bluekeyes/go-gitdiff v0.8.1
15
15
github.com/bluesky-social/indigo v0.0.0-20260220055544-bf41e2ee75ab
···
70
70
github.com/adrg/frontmatter v0.2.0 // indirect
71
71
github.com/alecthomas/repr v0.5.2 // indirect
72
72
github.com/anmitsu/go-shlex v0.0.0-20200514113438-38f4b401e2be // indirect
73
-
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4 // indirect
74
-
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.17 // indirect
75
-
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.17 // indirect
76
-
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.17 // indirect
77
-
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 // indirect
78
-
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.8 // indirect
79
-
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.17 // indirect
80
-
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.17 // indirect
81
-
github.com/aws/smithy-go v1.24.0 // indirect
73
+
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.7 // indirect
74
+
github.com/aws/aws-sdk-go-v2/config v1.32.12 // indirect
75
+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20 // indirect
76
+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.20 // indirect
77
+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.20 // indirect
78
+
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 // indirect
79
+
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.21 // indirect
80
+
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 // indirect
81
+
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.12 // indirect
82
+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.20 // indirect
83
+
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.20 // indirect
84
+
github.com/aws/aws-sdk-go-v2/service/signin v1.0.8 // indirect
85
+
github.com/aws/aws-sdk-go-v2/service/sso v1.30.13 // indirect
86
+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.17 // indirect
87
+
github.com/aws/aws-sdk-go-v2/service/sts v1.41.9 // indirect
88
+
github.com/aws/smithy-go v1.24.2 // indirect
82
89
github.com/aymanbagabas/go-osc52/v2 v2.0.1 // indirect
83
90
github.com/aymerick/douceur v0.2.0 // indirect
84
91
github.com/beorn7/perks v1.0.1 // indirect
+38
go.sum
+38
go.sum
···
27
27
github.com/avast/retry-go/v4 v4.6.1/go.mod h1:V6oF8njAwxJ5gRo1Q7Cxab24xs5NCWZBeaHHBklR8mA=
28
28
github.com/aws/aws-sdk-go-v2 v1.41.1 h1:ABlyEARCDLN034NhxlRUSZr4l71mh+T5KAeGh6cerhU=
29
29
github.com/aws/aws-sdk-go-v2 v1.41.1/go.mod h1:MayyLB8y+buD9hZqkCW3kX1AKq07Y5pXxtgB+rRFhz0=
30
+
github.com/aws/aws-sdk-go-v2 v1.41.4 h1:10f50G7WyU02T56ox1wWXq+zTX9I1zxG46HYuG1hH/k=
31
+
github.com/aws/aws-sdk-go-v2 v1.41.4/go.mod h1:mwsPRE8ceUUpiTgF7QmQIJ7lgsKUPQOUl3o72QBrE1o=
30
32
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4 h1:489krEF9xIGkOaaX3CE/Be2uWjiXrkCH6gUX+bZA/BU=
31
33
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.4/go.mod h1:IOAPF6oT9KCsceNTvvYMNHy0+kMF8akOjeDvPENWxp4=
34
+
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.7 h1:3kGOqnh1pPeddVa/E37XNTaWJ8W6vrbYV9lJEkCnhuY=
35
+
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.7.7/go.mod h1:lyw7GFp3qENLh7kwzf7iMzAxDn+NzjXEAGjKS2UOKqI=
36
+
github.com/aws/aws-sdk-go-v2/config v1.32.12 h1:O3csC7HUGn2895eNrLytOJQdoL2xyJy0iYXhoZ1OmP0=
37
+
github.com/aws/aws-sdk-go-v2/config v1.32.12/go.mod h1:96zTvoOFR4FURjI+/5wY1vc1ABceROO4lWgWJuxgy0g=
32
38
github.com/aws/aws-sdk-go-v2/credentials v1.19.9 h1:sWvTKsyrMlJGEuj/WgrwilpoJ6Xa1+KhIpGdzw7mMU8=
33
39
github.com/aws/aws-sdk-go-v2/credentials v1.19.9/go.mod h1:+J44MBhmfVY/lETFiKI+klz0Vym2aCmIjqgClMmW82w=
40
+
github.com/aws/aws-sdk-go-v2/credentials v1.19.12 h1:oqtA6v+y5fZg//tcTWahyN9PEn5eDU/Wpvc2+kJ4aY8=
41
+
github.com/aws/aws-sdk-go-v2/credentials v1.19.12/go.mod h1:U3R1RtSHx6NB0DvEQFGyf/0sbrpJrluENHdPy1j/3TE=
42
+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20 h1:zOgq3uezl5nznfoK3ODuqbhVg1JzAGDUhXOsU0IDCAo=
43
+
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.18.20/go.mod h1:z/MVwUARehy6GAg/yQ1GO2IMl0k++cu1ohP9zo887wE=
34
44
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.17 h1:xOLELNKGp2vsiteLsvLPwxC+mYmO6OZ8PYgiuPJzF8U=
35
45
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.17/go.mod h1:5M5CI3D12dNOtH3/mk6minaRwI2/37ifCURZISxA/IQ=
46
+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.20 h1:CNXO7mvgThFGqOFgbNAP2nol2qAWBOGfqR/7tQlvLmc=
47
+
github.com/aws/aws-sdk-go-v2/internal/configsources v1.4.20/go.mod h1:oydPDJKcfMhgfcgBUZaG+toBbwy8yPWubJXBVERtI4o=
36
48
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.17 h1:WWLqlh79iO48yLkj1v3ISRNiv+3KdQoZ6JWyfcsyQik=
37
49
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.17/go.mod h1:EhG22vHRrvF8oXSTYStZhJc1aUgKtnJe+aOiFEV90cM=
50
+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.20 h1:tN6W/hg+pkM+tf9XDkWUbDEjGLb+raoBMFsTodcoYKw=
51
+
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.7.20/go.mod h1:YJ898MhD067hSHA6xYCx5ts/jEd8BSOLtQDL3iZsvbc=
52
+
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6 h1:qYQ4pzQ2Oz6WpQ8T3HvGHnZydA72MnLuFK9tJwmrbHw=
53
+
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.6/go.mod h1:O3h0IK87yXci+kg6flUKzJnWeziQUKciKrLjcatSNcY=
38
54
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.17 h1:JqcdRG//czea7Ppjb+g/n4o8i/R50aTBHkA7vu0lK+k=
39
55
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.17/go.mod h1:CO+WeGmIdj/MlPel2KwID9Gt7CNq4M65HUfBW97liM0=
56
+
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.21 h1:SwGMTMLIlvDNyhMteQ6r8IJSBPlRdXX5d4idhIGbkXA=
57
+
github.com/aws/aws-sdk-go-v2/internal/v4a v1.4.21/go.mod h1:UUxgWxofmOdAMuqEsSppbDtGKLfR04HGsD0HXzvhI1k=
40
58
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4 h1:0ryTNEdJbzUCEWkVXEXoqlXV72J5keC1GvILMOuD00E=
41
59
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.4/go.mod h1:HQ4qwNZh32C3CBeO6iJLQlgtMzqeG17ziAA/3KDJFow=
60
+
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7 h1:5EniKhLZe4xzL7a+fU3C2tfUN4nWIqlLesfrjkuPFTY=
61
+
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.13.7/go.mod h1:x0nZssQ3qZSnIcePWLvcoFisRXJzcTVvYpAAdYX8+GI=
42
62
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.8 h1:Z5EiPIzXKewUQK0QTMkutjiaPVeVYXX7KIqhXu/0fXs=
43
63
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.8/go.mod h1:FsTpJtvC4U1fyDXk7c71XoDv3HlRm8V3NiYLeYLh5YE=
64
+
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.12 h1:qtJZ70afD3ISKWnoX3xB0J2otEqu3LqicRcDBqsj0hQ=
65
+
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.9.12/go.mod h1:v2pNpJbRNl4vEUWEh5ytQok0zACAKfdmKS51Hotc3pQ=
44
66
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.17 h1:RuNSMoozM8oXlgLG/n6WLaFGoea7/CddrCfIiSA+xdY=
45
67
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.17/go.mod h1:F2xxQ9TZz5gDWsclCtPQscGpP0VUOc8RqgFM3vDENmU=
68
+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.20 h1:2HvVAIq+YqgGotK6EkMf+KIEqTISmTYh5zLpYyeTo1Y=
69
+
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.13.20/go.mod h1:V4X406Y666khGa8ghKmphma/7C0DAtEQYhkq9z4vpbk=
46
70
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.17 h1:bGeHBsGZx0Dvu/eJC0Lh9adJa3M1xREcndxLNZlve2U=
47
71
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.17/go.mod h1:dcW24lbU0CzHusTE8LLHhRLI42ejmINN8Lcr22bwh/g=
72
+
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.20 h1:siU1A6xjUZ2N8zjTHSXFhB9L/2OY8Dqs0xXiLjF30jA=
73
+
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.19.20/go.mod h1:4TLZCmVJDM3FOu5P5TJP0zOlu9zWgDWU7aUxWbr+rcw=
48
74
github.com/aws/aws-sdk-go-v2/service/s3 v1.96.0 h1:oeu8VPlOre74lBA/PMhxa5vewaMIMmILM+RraSyB8KA=
49
75
github.com/aws/aws-sdk-go-v2/service/s3 v1.96.0/go.mod h1:5jggDlZ2CLQhwJBiZJb4vfk4f0GxWdEDruWKEJ1xOdo=
76
+
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.1 h1:csi9NLpFZXb9fxY7rS1xVzgPRGMt7MSNWeQ6eo247kE=
77
+
github.com/aws/aws-sdk-go-v2/service/s3 v1.97.1/go.mod h1:qXVal5H0ChqXP63t6jze5LmFalc7+ZE7wOdLtZ0LCP0=
78
+
github.com/aws/aws-sdk-go-v2/service/signin v1.0.8 h1:0GFOLzEbOyZABS3PhYfBIx2rNBACYcKty+XGkTgw1ow=
79
+
github.com/aws/aws-sdk-go-v2/service/signin v1.0.8/go.mod h1:LXypKvk85AROkKhOG6/YEcHFPoX+prKTowKnVdcaIxE=
80
+
github.com/aws/aws-sdk-go-v2/service/sso v1.30.13 h1:kiIDLZ005EcKomYYITtfsjn7dtOwHDOFy7IbPXKek2o=
81
+
github.com/aws/aws-sdk-go-v2/service/sso v1.30.13/go.mod h1:2h/xGEowcW/g38g06g3KpRWDlT+OTfxxI0o1KqayAB8=
82
+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.17 h1:jzKAXIlhZhJbnYwHbvUQZEB8KfgAEuG0dc08Bkda7NU=
83
+
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.35.17/go.mod h1:Al9fFsXjv4KfbzQHGe6V4NZSZQXecFcvaIF4e70FoRA=
84
+
github.com/aws/aws-sdk-go-v2/service/sts v1.41.9 h1:Cng+OOwCHmFljXIxpEVXAGMnBia8MSU6Ch5i9PgBkcU=
85
+
github.com/aws/aws-sdk-go-v2/service/sts v1.41.9/go.mod h1:LrlIndBDdjA/EeXeyNBle+gyCwTlizzW5ycgWnvIxkk=
50
86
github.com/aws/smithy-go v1.24.0 h1:LpilSUItNPFr1eY85RYgTIg5eIEPtvFbskaFcmmIUnk=
51
87
github.com/aws/smithy-go v1.24.0/go.mod h1:LEj2LM3rBRQJxPZTB4KuzZkaZYnZPnvgIhb4pu07mx0=
88
+
github.com/aws/smithy-go v1.24.2 h1:FzA3bu/nt/vDvmnkg+R8Xl46gmzEDam6mZ1hzmwXFng=
89
+
github.com/aws/smithy-go v1.24.2/go.mod h1:YE2RhdIuDbA5E5bTdciG9KrW3+TiEONeUWCqxX9i1Fc=
52
90
github.com/aymanbagabas/go-osc52/v2 v2.0.1 h1:HwpRHbFMcZLEVr42D4p7XBqjyuxQH5SMiErDT4WkJ2k=
53
91
github.com/aymanbagabas/go-osc52/v2 v2.0.1/go.mod h1:uYgXzlJ7ZpABp8OJ+exZzJJhRNQ2ASbcXHWsFqH8hp8=
54
92
github.com/aymerick/douceur v0.2.0 h1:Mv+mAeH1Q+n9Fr+oyamOlAkUNPWPlA8PPGR0QAaYuPk=
+54
-24
nix/gomod2nix.toml
+54
-24
nix/gomod2nix.toml
···
7
7
[mod."github.com/Blank-Xu/sql-adapter"]
8
8
version = "v1.1.1"
9
9
hash = "sha256-9AiQhXoNPCiViV+p5aa3qGFkYU4rJNbADvNdYGq4GA4="
10
+
[mod."github.com/BurntSushi/toml"]
11
+
version = "v0.3.1"
12
+
hash = "sha256-Rqak1dE/Aj/+Kx1/pl3Hifgt+Q3OzuZ5fJR+/x3nTbo="
10
13
[mod."github.com/Microsoft/go-winio"]
11
14
version = "v0.6.2"
12
15
hash = "sha256-tVNWDUMILZbJvarcl/E7tpSnkn7urqgSHa2Eaka5vSU="
···
16
19
[mod."github.com/RoaringBitmap/roaring/v2"]
17
20
version = "v2.4.5"
18
21
hash = "sha256-igWY0S1PTolQkfctYcmVJioJyV1pk2V81X6o6BA1XQA="
22
+
[mod."github.com/adrg/frontmatter"]
23
+
version = "v0.2.0"
24
+
hash = "sha256-WJsVcdCpkIkjqUz5fJOFStZYwQlrcFzQ6+mZatZiimo="
19
25
[mod."github.com/alecthomas/assert/v2"]
20
26
version = "v2.11.0"
21
27
hash = "sha256-tDJCDKZ0R4qNA7hgMKWrpDyogt1802LCJDBCExxdqaU="
···
33
39
version = "v4.6.1"
34
40
hash = "sha256-PeZc8k4rDV64+k8nZt/oy1YNVbLevltXP3ZD1jf6Z6k="
35
41
[mod."github.com/aws/aws-sdk-go-v2"]
36
-
version = "v1.41.1"
37
-
hash = "sha256-umafTZB+cuy8+Kzpl2WrlygJO3tR3D2WkTC0eTY5G/g="
42
+
version = "v1.41.4"
43
+
hash = "sha256-k9xv4f8YPSzZ1yR3/zuyNDGenZKk0DD4lceL713yXtc="
38
44
[mod."github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream"]
39
-
version = "v1.7.4"
40
-
hash = "sha256-ZY/Jn1p0IgDe8MONhp0RFHZmRgTBZZ5ddqXlNWEo7Ys="
45
+
version = "v1.7.7"
46
+
hash = "sha256-5hw1g8s21Gx1Q3JtiaVM6q8qGt7DW1YO9U1dEEqRXfE="
47
+
[mod."github.com/aws/aws-sdk-go-v2/config"]
48
+
version = "v1.32.12"
49
+
hash = "sha256-aTkdSRe8KPmVZdsunU8j/hZQLhGw1ckKpLN/ryRBZM0="
41
50
[mod."github.com/aws/aws-sdk-go-v2/credentials"]
42
-
version = "v1.19.9"
43
-
hash = "sha256-eqM5BmetQ/MrxTwoUCqRpkm5frFAspuq+QWod+5wzrU="
51
+
version = "v1.19.12"
52
+
hash = "sha256-xEIT1ARA9RYrQtLZIus71E6niNHIOVM1J7mUnA5AhJQ="
53
+
[mod."github.com/aws/aws-sdk-go-v2/feature/ec2/imds"]
54
+
version = "v1.18.20"
55
+
hash = "sha256-dCTpdKZheVCSt+R+NnFOnlS0bCt4gPavlDh15Kl/sMQ="
44
56
[mod."github.com/aws/aws-sdk-go-v2/internal/configsources"]
45
-
version = "v1.4.17"
46
-
hash = "sha256-W+d0WDYBrVJxdfRSvoe3cvZYgIgUSVyXsVdQlMcIZvc="
57
+
version = "v1.4.20"
58
+
hash = "sha256-aATIk4oLd7aaV66ereBdjINLMDwmIHxu+NNsgKWH1t4="
47
59
[mod."github.com/aws/aws-sdk-go-v2/internal/endpoints/v2"]
48
-
version = "v2.7.17"
49
-
hash = "sha256-dKJx0+K1DKi2LJKQNwnTofZH4GWLDgY6/ZI2XR2oCGI="
60
+
version = "v2.7.20"
61
+
hash = "sha256-G6266uj64sgfDTJ9V1UY1sQs3UmryB0CFgxzmbjjChY="
62
+
[mod."github.com/aws/aws-sdk-go-v2/internal/ini"]
63
+
version = "v1.8.6"
64
+
hash = "sha256-oIRPqu99vnGINAWKnCEytpv7N0gRWO7S72tb1r8oxvk="
50
65
[mod."github.com/aws/aws-sdk-go-v2/internal/v4a"]
51
-
version = "v1.4.17"
52
-
hash = "sha256-PKwNn9nFf+7TqocgQprxP9r1Fs4IxwGLGd05MrsxmLg="
66
+
version = "v1.4.21"
67
+
hash = "sha256-Sq4kRaiIFkPqwYnKv1dsxplyX4duipUYGFFT5gzJtJM="
53
68
[mod."github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding"]
54
-
version = "v1.13.4"
55
-
hash = "sha256-Rm6czqOnOULP080D97WQQSqkBhmN6ei1qZaTa51SRj8="
69
+
version = "v1.13.7"
70
+
hash = "sha256-AfYJdpmnW01Bk/jfHATlNU6lddjqcigFkHw/zcT9WO4="
56
71
[mod."github.com/aws/aws-sdk-go-v2/service/internal/checksum"]
57
-
version = "v1.9.8"
58
-
hash = "sha256-DZhR0aqHrgAFBGSlnsSQ6XeAJ/q504RG/LBWhtQqRVg="
72
+
version = "v1.9.12"
73
+
hash = "sha256-gze5CLp6Cp3X1jdqawWlXG7LToH+L6L/79b8F/AgWIY="
59
74
[mod."github.com/aws/aws-sdk-go-v2/service/internal/presigned-url"]
60
-
version = "v1.13.17"
61
-
hash = "sha256-qS7Db9S/KQ24kvJdL8qH4gnoN116J1ezwOnbovCiFEI="
75
+
version = "v1.13.20"
76
+
hash = "sha256-a5TifKunIoqKd2uAceYh6F1LvMHMyEQcWvJf0sxKhPM="
62
77
[mod."github.com/aws/aws-sdk-go-v2/service/internal/s3shared"]
63
-
version = "v1.19.17"
64
-
hash = "sha256-x/Cb4j3HFlg1+U21YCAIhBnfpf1yehJU2Ss/PxamEMI="
78
+
version = "v1.19.20"
79
+
hash = "sha256-T+pXMFELor69MTIo67TIkcWWyK+wubweg0gS5I9Dp8A="
65
80
[mod."github.com/aws/aws-sdk-go-v2/service/s3"]
66
-
version = "v1.96.0"
67
-
hash = "sha256-lzAn2KHIkd742mDwEZqGEaIXlEfvltL6HdP+sEQ/8YA="
81
+
version = "v1.97.1"
82
+
hash = "sha256-J51tttUIPFVEnHddftY4CJXKe2WLLIQDhJ8eYM8jgts="
83
+
[mod."github.com/aws/aws-sdk-go-v2/service/signin"]
84
+
version = "v1.0.8"
85
+
hash = "sha256-o4pWg3yMZHxdI94x5Z6qbiRg7gpmzbpJnJWsR1BOc44="
86
+
[mod."github.com/aws/aws-sdk-go-v2/service/sso"]
87
+
version = "v1.30.13"
88
+
hash = "sha256-V277a0ikm/H0paIeDLtPGEyav2a69Kdb9d5bh+JLAeY="
89
+
[mod."github.com/aws/aws-sdk-go-v2/service/ssooidc"]
90
+
version = "v1.35.17"
91
+
hash = "sha256-r5V5DoCIR4yzN1Ttg+dIA85GVkWMPgeD6Zu0rWGqNJE="
92
+
[mod."github.com/aws/aws-sdk-go-v2/service/sts"]
93
+
version = "v1.41.9"
94
+
hash = "sha256-I15uxeoKxDURsZrEVDzCRtVIu/HE756M1Rt7PPpdZ7c="
68
95
[mod."github.com/aws/smithy-go"]
69
-
version = "v1.24.0"
70
-
hash = "sha256-ZPFhf2Yv3BQpUn3cN4wSnoO7uBki8oCisZxL6F09nnE="
96
+
version = "v1.24.2"
97
+
hash = "sha256-v0y+Lir61fgdCwdVoca5mK+FcGh9OD3cTEwHIfLytcI="
71
98
[mod."github.com/aymanbagabas/go-osc52/v2"]
72
99
version = "v2.0.1"
73
100
hash = "sha256-6Bp0jBZ6npvsYcKZGHHIUSVSTAMEyieweAX2YAKDjjg="
···
696
723
[mod."gopkg.in/warnings.v0"]
697
724
version = "v0.1.2"
698
725
hash = "sha256-ATVL9yEmgYbkJ1DkltDGRn/auGAjqGOfjQyBYyUo8s8="
726
+
[mod."gopkg.in/yaml.v2"]
727
+
version = "v2.4.0"
728
+
hash = "sha256-uVEGglIedjOIGZzHW4YwN1VoRSTK8o0eGZqzd+TNdd0="
699
729
[mod."gopkg.in/yaml.v3"]
700
730
version = "v3.0.1"
701
731
hash = "sha256-FqL9TKYJ0XkNwJFnq9j0VvJ5ZUU1RvH/52h/f5bkYAU="
+23
nix/modules/spindle.nix
+23
nix/modules/spindle.nix
···
109
109
default = "5m";
110
110
description = "Timeout for each step of a pipeline";
111
111
};
112
+
113
+
logBucket = mkOption {
114
+
type = types.str;
115
+
default = "tangled-logs";
116
+
description = "S3 bucket for workflow logs";
117
+
};
118
+
};
119
+
120
+
environmentFile = mkOption {
121
+
type = with types; nullOr path;
122
+
default = null;
123
+
example = "/etc/spindle.env";
124
+
description = ''
125
+
Additional environment file as defined in {manpage}`systemd.exec(5)`.
126
+
127
+
Sensitive secrets such as {env}`AWS_SECRET_ACCESS_KEY`,
128
+
{env}`AWS_ACCESS_KEY_ID`, {env}`AWS_REGION`
129
+
may be passed to the service
130
+
without making them world readable in the nix store.
131
+
'';
112
132
};
113
133
};
114
134
};
···
123
143
serviceConfig = {
124
144
LogsDirectory = "spindle";
125
145
StateDirectory = "spindle";
146
+
EnvironmentFile = mkIf (cfg.environmentFile != null) cfg.environmentFile;
147
+
126
148
Environment = [
127
149
"SPINDLE_SERVER_LISTEN_ADDR=${cfg.server.listenAddr}"
128
150
"SPINDLE_SERVER_DB_PATH=${cfg.server.dbPath}"
···
138
160
"SPINDLE_SERVER_SECRETS_OPENBAO_MOUNT=${cfg.server.secrets.openbao.mount}"
139
161
"SPINDLE_NIXERY_PIPELINES_NIXERY=${cfg.pipelines.nixery}"
140
162
"SPINDLE_NIXERY_PIPELINES_WORKFLOW_TIMEOUT=${cfg.pipelines.workflowTimeout}"
163
+
"SPINDLE_S3_LOG_BUCKET=${cfg.pipelines.logBucket}"
141
164
];
142
165
ExecStart = "${cfg.package}/bin/spindle";
143
166
Restart = "always";
+5
nix/vm.nix
+5
nix/vm.nix
···
118
118
};
119
119
services.tangled.spindle = {
120
120
enable = true;
121
+
environmentFile = "/var/lib/spindle/.env";
121
122
server = {
122
123
owner = envVar "TANGLED_VM_SPINDLE_OWNER";
123
124
hostname = envVarOr "TANGLED_VM_SPINDLE_HOST" "localhost:6555";
···
130
131
secrets = {
131
132
provider = "sqlite";
132
133
};
134
+
};
135
+
136
+
pipelines = {
137
+
logBucket = envVarOr "SPINDLE_S3_LOG_BUCKET" "";
133
138
};
134
139
};
135
140
services.postgresql = {
+5
spindle/config/config.go
+5
spindle/config/config.go
···
41
41
WorkflowTimeout string `env:"WORKFLOW_TIMEOUT, default=5m"`
42
42
}
43
43
44
+
type S3 struct {
45
+
LogBucket string `env:"LOG_BUCKET"`
46
+
}
47
+
44
48
type Config struct {
45
49
Server Server `env:",prefix=SPINDLE_SERVER_"`
46
50
NixeryPipelines NixeryPipelines `env:",prefix=SPINDLE_NIXERY_PIPELINES_"`
51
+
S3 S3 `env:",prefix=SPINDLE_S3_"`
47
52
}
48
53
49
54
func Load(ctx context.Context) (*Config, error) {
+16
spindle/engine/engine.go
+16
spindle/engine/engine.go
···
3
3
import (
4
4
"context"
5
5
"errors"
6
+
"fmt"
6
7
"log/slog"
8
+
"path/filepath"
7
9
"sync"
8
10
9
11
securejoin "github.com/cyphar/filepath-securejoin"
···
35
37
secretValues[i] = s.Value
36
38
}
37
39
40
+
s3, err := NewS3(cfg.S3.LogBucket)
41
+
if err != nil {
42
+
l.Error("error creating s3 client", "err", err)
43
+
}
44
+
38
45
var wg sync.WaitGroup
39
46
for eng, wfs := range pipeline.Workflows {
40
47
workflowTimeout := eng.WorkflowTimeout()
···
49
56
PipelineId: pipelineId,
50
57
Name: w.Name,
51
58
}
59
+
60
+
defer func() {
61
+
if s3 != nil {
62
+
logFile := filepath.Join(cfg.Server.LogDir, fmt.Sprintf("%s.log", wid.String()))
63
+
if err := s3.WriteFile(ctx, logFile); err != nil {
64
+
l.Error("error uploading logs", "err", err)
65
+
}
66
+
}
67
+
}()
52
68
53
69
wfLogger, err := models.NewFileWorkflowLogger(cfg.Server.LogDir, wid, secretValues)
54
70
if err != nil {
+75
spindle/engine/s3.go
+75
spindle/engine/s3.go
···
1
+
package engine
2
+
3
+
import (
4
+
"context"
5
+
"fmt"
6
+
"io"
7
+
"os"
8
+
"path/filepath"
9
+
10
+
"github.com/aws/aws-sdk-go-v2/aws"
11
+
"github.com/aws/aws-sdk-go-v2/config"
12
+
"github.com/aws/aws-sdk-go-v2/service/s3"
13
+
)
14
+
15
+
type S3 struct {
16
+
bucket string
17
+
client *s3.Client
18
+
}
19
+
20
+
const BaseS3Path = "spindle/workflows"
21
+
22
+
func NewS3(bucket string) (*S3, error) {
23
+
if bucket == "" {
24
+
return nil, fmt.Errorf("s3 bucket not provided")
25
+
}
26
+
27
+
ctx := context.Background()
28
+
sdkConfig, err := config.LoadDefaultConfig(ctx)
29
+
30
+
if err != nil {
31
+
return nil, fmt.Errorf("error loading s3 config: %w", err)
32
+
}
33
+
s3Client := s3.NewFromConfig(sdkConfig)
34
+
35
+
return &S3{
36
+
bucket: bucket,
37
+
client: s3Client,
38
+
}, nil
39
+
}
40
+
41
+
func (s *S3) WriteFile(ctx context.Context, path string) error {
42
+
s3Key := fmt.Sprintf("%s/%s", BaseS3Path, filepath.Base(path))
43
+
44
+
file, err := os.Open(path)
45
+
if err != nil {
46
+
return fmt.Errorf("error opening file %s: %w", path, err)
47
+
}
48
+
defer file.Close()
49
+
50
+
_, err = s.client.PutObject(ctx, &s3.PutObjectInput{
51
+
Bucket: &s.bucket,
52
+
Key: &s3Key,
53
+
Body: file,
54
+
})
55
+
56
+
if err != nil {
57
+
return fmt.Errorf("error writing to s3: %w", err)
58
+
}
59
+
60
+
return nil
61
+
}
62
+
63
+
func (s *S3) ReadFile(ctx context.Context, name string) ([]byte, error) {
64
+
res, err := s.client.GetObject(ctx, &s3.GetObjectInput{
65
+
Bucket: &s.bucket,
66
+
Key: aws.String(name),
67
+
})
68
+
69
+
if err != nil {
70
+
return nil, fmt.Errorf("error reading file %s: %w", name, err)
71
+
}
72
+
defer res.Body.Close()
73
+
74
+
return io.ReadAll(res.Body)
75
+
}
History
10 rounds
3 comments
jobala.tngl.sh
submitted
#9
2 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
no conflicts, ready to merge
expand 0 comments
jobala.tngl.sh
submitted
#8
10 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle/engine: remove unrelated changes
expand 0 comments
jobala.tngl.sh
submitted
#7
12 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle/engine: remove unrelated changes
spindle/engine: add overwritten changes
spindle/engine: fix typo
expand 0 comments
jobala.tngl.sh
submitted
#6
11 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle/engine: remove unrelated changes
spindle/engine: add overwritten changes
expand 0 comments
jobala.tngl.sh
submitted
#5
10 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
spindle/engine: remove unrelated changes
expand 0 comments
jobala.tngl.sh
submitted
#4
9 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
expand 0 comments
jobala.tngl.sh
submitted
#3
8 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
expand 0 comments
jobala.tngl.sh
submitted
#2
7 commits
expand
collapse
spindle/engine: store workflow logs in s3
spindle/engine: read secrets from environment file
nix,knotmirror: listen & sync local knots from local knotmirror
Signed-off-by: Seongmin Lee <git@boltless.me>
lexicons:
sync.requestCrawl and knot.subscribeRepos
Signed-off-by: Seongmin Lee <git@boltless.me>
nix,knotserver,knotmirror/xrpc:
sync.requestCrawl support
knotmirror ingests repo creation event from atproto relay through tap,
not from knotstream. Therefore, knotserver will request
`sync.requestCrawl` with optional new repo information to notify the
repo creation event.
Knot will call `sync.requestCrawl` on following cases:
- on startup
- when `/event` stream has failed
- on repo creation (mandatory)
Signed-off-by: Seongmin Lee <git@boltless.me>
ogre: rename appview/ogcard to ogre
And move it out of appview.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
ogre: request jpeg from avatar service
This plays nice with satori and doesn't require wacky conversions etc.
Signed-off-by: Anirudh Oppiliappan <anirudh@tangled.org>
expand 1 comment
jobala.tngl.sh
submitted
#1
expand 0 comments
jobala.tngl.sh
submitted
#0
1 commit
expand
collapse
spindle/engine: store workflow logs in s3
expand 2 comments
spindle/config/config.go:44-51 — this will result in SPINDLE_S3_LOG_BUCKET, but
nix/modules/spindle.nix:147 we reference SPINDLE_NIXERY_PIPELINES_LOG_BUCKET here.
spindle/engine/engine.go:60 — probably good to parameterize the bucket name from config as well!
spindle/engine/s3.go:20 — Go convention is camel case, so BaseS3Path is preferred here.
spindle/engine/s3.go:38 — also camel case here please: s3Key
spindle/engine/engine.go:154 — this is a no-op with just a single argument.
nix/modules/spindle.nix:148-150 — we should ideally use EnvironmentFile here. Refer to how secrets are passed through to the appview service, for example.
High level design concern: NewS3 creates a new S3 client for each workflow (on every defer). We should ideally initialize this upon spindle startup and passed through via the engine struct.
We should ideally initialize this upon spindle startup and passed through via the engine struct.
Doing this means that each engine will have to register a clean up task for uploading logs which is easy to forget. Doing it in spindle/engine means that new engines don't have to worry about how to upload logs.
sorry, but you will need to rebase with master before submitting!