loading up the forgejo repo on tangled to test page performance

fix: discard v25 secrets migrations errors instead of failing (#7251)

Failing the migration when a corrupted record is found is problematic because there is no transaction and the database may need to be restored from a backup to attempt the migration again, after deleting the corrupted records.

Each documented case of failed migration was resolved by removing the corrupted records. There is no instance of a failed migration that was caused by non corrupted record.

In the unlikely event of a false negative where a two_factor record is discarded although it is in use, the only consequence is that the user will have to enroll again. Detailed logs are displayed so the Forgejo admin can file a bug report if that happens.

Refs: https://codeberg.org/forgejo/forgejo/issues/6637

<!--start release-notes-assistant-->

## Release notes
<!--URL:https://codeberg.org/forgejo/forgejo-->
- Bug fixes
- [PR](https://codeberg.org/forgejo/forgejo/pulls/7251): <!--number 7251 --><!--line 0 --><!--description V2hlbiBtaWdyYXRpbmcgdG8gRm9yZ2VqbyB2MTAsIHRoZSBUT1RQIHNlY3JldHMgZm91bmQgdG8gYmUgY29ycnVwdGVkIGFyZSBub3cgdHJhbnNwYXJlbnRseSByZW1vdmVkIGZyb20gdGhlIGRhdGFiYXNlIGluc3RlYWQgb2YgZmFpbGluZyB0aGUgbWlncmF0aW9uLiBUT1RQIGlzIG5vIGxvbmdlciByZXF1aXJlZCB0byBsb2dpbiB3aXRoIHRoZSBhc3NvY2lhdGVkIHVzZXJzLiBUaGV5IHNob3VsZCBiZSBpbmZvcm1lZCBiZWNhdXNlIHRoZXkgd2lsbCBuZWVkIHRvIHZpc2l0IHRoZWlyIHNlY3VyaXR5IHNldHRpbmdzIGFuZCBjb25maWd1cmUgVE9UUCBhZ2Fpbi4gTm8gb3RoZXIgYWN0aW9uIGlzIHJlcXVpcmVkLg==-->When migrating to Forgejo v10, the TOTP secrets found to be corrupted are now transparently removed from the database instead of failing the migration. TOTP is no longer required to login with the associated users. They should be informed because they will need to visit their security settings and configure TOTP again. No other action is required.<!--description-->
<!--end release-notes-assistant-->

Reviewed-on: https://codeberg.org/forgejo/forgejo/pulls/7251
Reviewed-by: Gusted <gusted@noreply.codeberg.org>
Co-authored-by: Earl Warren <contact@earl-warren.org>
Co-committed-by: Earl Warren <contact@earl-warren.org>

authored by Earl Warren Earl Warren and committed by Earl Warren 83e186c0 d40ff8c1

Changed files
+39 -4
models
forgejo_migrations
migrations
fixtures
Test_MigrateTwoFactorToKeying
release-notes
+24 -3
models/forgejo_migrations/v25.go
··· 7 7 "context" 8 8 "crypto/md5" 9 9 "encoding/base64" 10 + "fmt" 10 11 11 12 "code.gitea.io/gitea/models/auth" 12 13 "code.gitea.io/gitea/models/db" 14 + "code.gitea.io/gitea/modules/log" 13 15 "code.gitea.io/gitea/modules/secret" 14 16 "code.gitea.io/gitea/modules/setting" 15 17 ··· 57 59 58 60 oldEncryptionKey := md5.Sum([]byte(setting.SecretKey)) 59 61 60 - return db.Iterate(context.Background(), nil, func(ctx context.Context, bean *auth.TwoFactor) error { 62 + messages := make([]string, 0, 100) 63 + ids := make([]int64, 0, 100) 64 + 65 + err = db.Iterate(context.Background(), nil, func(ctx context.Context, bean *auth.TwoFactor) error { 61 66 decodedStoredSecret, err := base64.StdEncoding.DecodeString(string(bean.Secret)) 62 67 if err != nil { 63 - return err 68 + messages = append(messages, fmt.Sprintf("two_factor.id=%d, two_factor.uid=%d: base64.StdEncoding.DecodeString: %v", bean.ID, bean.UID, err)) 69 + ids = append(ids, bean.ID) 70 + return nil 64 71 } 65 72 66 73 secretBytes, err := secret.AesDecrypt(oldEncryptionKey[:], decodedStoredSecret) 67 74 if err != nil { 68 - return err 75 + messages = append(messages, fmt.Sprintf("two_factor.id=%d, two_factor.uid=%d: secret.AesDecrypt: %v", bean.ID, bean.UID, err)) 76 + ids = append(ids, bean.ID) 77 + return nil 69 78 } 70 79 71 80 bean.SetSecret(string(secretBytes)) 72 81 _, err = db.GetEngine(ctx).Cols("secret").ID(bean.ID).Update(bean) 73 82 return err 74 83 }) 84 + if err == nil { 85 + if len(ids) > 0 { 86 + log.Error("Forgejo migration[25]: The following TOTP secrets were found to be corrupted and removed from the database. TOTP is no longer required to login with the associated users. They should be informed because they will need to visit their security settings and configure TOTP again. No other action is required. See https://codeberg.org/forgejo/forgejo/issues/6637 for more context on the various causes for such a corruption.") 87 + for _, message := range messages { 88 + log.Error("Forgejo migration[25]: %s", message) 89 + } 90 + 91 + _, err = db.GetEngine(context.Background()).In("id", ids).NoAutoCondition().NoAutoTime().Delete(&auth.TwoFactor{}) 92 + } 93 + } 94 + 95 + return err 75 96 }
+5 -1
models/forgejo_migrations/v25_test.go
··· 36 36 37 37 cnt, err := x.Table("two_factor").Count() 38 38 require.NoError(t, err) 39 - assert.EqualValues(t, 1, cnt) 39 + assert.EqualValues(t, 2, cnt) 40 40 41 41 require.NoError(t, MigrateTwoFactorToKeying(x)) 42 + 43 + cnt, err = x.Table("two_factor").Count() 44 + require.NoError(t, err) 45 + assert.EqualValues(t, 1, cnt) 42 46 43 47 var twofactor auth.TwoFactor 44 48 _, err = x.Table("two_factor").ID(1).Get(&twofactor)
+9
models/migrations/fixtures/Test_MigrateTwoFactorToKeying/two_factor.yml
··· 7 7 last_used_passcode: 8 8 created_unix: 1564253724 9 9 updated_unix: 1564253724 10 + - 11 + id: 2 12 + uid: 23 13 + secret: badbad 14 + scratch_salt: badbad 15 + scratch_hash: badbad 16 + last_used_passcode: 17 + created_unix: 1564253724 18 + updated_unix: 1564253724
+1
release-notes/7251.md
··· 1 + When migrating from a Forgejo version lower than v10, the TOTP secrets found to be corrupted are now transparently removed from the database instead of failing the migration. TOTP is no longer required to login with the associated users. They should be informed because they will need to visit their security settings and configure TOTP again. No other action is required.