tangled.org / core
Monorepo for Tangled tangled.org
fork atom

Implement tag-based filters for issues and pulls #1065

merged opened by octet-stream.net targeting master from octet-stream.net/core: thombles/search-filter-tags

Fixes issue #297

This changeset makes the search textfield on the "issues" and "pulls" pages much more powerful.

  • keyword and -keyword
  • "some phrase" and -"some phrase"
  • label:good-first-issue and -label:good-first-issue
  • author:octet-stream.net and -author:octet-stream.net
  • state:open, state:closed and state:merged
  • label:good-first-issue and label:documentation to require multiple labels

The open/closed/merged tab buttons now play a subordinate role to the textfield. When you first load the page it is populated with state:open to show the current state. If you click "closed" then this sends the current state of the query and on the server side the state tag will be replaced with state:closed for the re-rendered page. This works regardless of what other filters are currently configured and I believe I've made it all work equally with and without JS. If you wish you can manually select and delete the state: tag to see issues with different states in the same results listing.

If you are doing anything more advanced than a state: tag then bleve is used to perform the query.

This pull also fixes missing re-index triggers when labels are added/removed from issues and pulls.

Potentially controversial quirks:

  • The author: tag requires resolving the handle. This is fast if it's in the cache but a blocking operation if it's one we haven't seen before.
  • Clicking the "X" clears the textfield except for the state: tag. I feel that this is useful. The other logical option is to empty the field entirely so that we are looking at all issues in all states.
  • Issue and pull index mappings now contain a version that will be used to detect when the index needs to be rebuilt because the code has changed the schema.

Out of scope:

  • Any kind of UI to automate or guide the user to use the new tag types

Some screenshots to give you the idea...

Default view when you open the issues page:

Then you clear the textfield and press enter:

Label search:

Narrow it down with another label:

Label and a keyword together:

Labels

None yet.

assignee

None yet.

Participants 4
AT URI
at://did:plc:txurc6ueald5d7462bpvzdby/sh.tangled.repo.pull/3melcljdmh622
+1336 -227
Diff #3
unified split
+7
appview/indexer/bleve/query.go
··· 13 13 return q 14 14 } 15 15 16 + func MatchPhraseQuery(field, phrase, analyzer string) query.Query { 17 + q := bleve.NewMatchPhraseQuery(phrase) 18 + q.FieldVal = field 19 + q.Analyzer = analyzer 20 + return q 21 + } 22 + 16 23 func BoolFieldQuery(field string, val bool) query.Query { 17 24 q := bleve.NewBoolFieldQuery(val) 18 25 q.FieldVal = field
+83 -25
appview/indexer/issues/indexer.go
··· 29 29 issueIndexerDocType = "issueIndexerDocType" 30 30 31 31 unicodeNormalizeName = "uicodeNormalize" 32 + 33 + // Bump this when the index mapping changes to trigger a rebuild. 34 + issueIndexerVersion = 2 32 35 ) 33 36 34 37 type Indexer struct { ··· 84 87 85 88 docMapping.AddFieldMappingsAt("repo_at", keywordFieldMapping) 86 89 docMapping.AddFieldMappingsAt("is_open", boolFieldMapping) 90 + docMapping.AddFieldMappingsAt("author_did", keywordFieldMapping) 91 + docMapping.AddFieldMappingsAt("labels", keywordFieldMapping) 87 92 88 93 err := mapping.AddCustomTokenFilter(unicodeNormalizeName, map[string]any{ 89 94 "type": unicodenorm.Name, ··· 116 121 return false, errors.New("indexer is already initialized") 117 122 } 118 123 119 - indexer, err := openIndexer(ctx, ix.path) 124 + indexer, err := openIndexer(ctx, ix.path, issueIndexerVersion) 120 125 if err != nil { 121 126 return false, err 122 127 } ··· 133 138 if err != nil { 134 139 return false, err 135 140 } 141 + indexer.SetInternal([]byte("mapping_version"), []byte{byte(issueIndexerVersion)}) 136 142 137 143 ix.indexer = indexer 138 144 139 145 return false, nil 140 146 } 141 147 142 - func openIndexer(ctx context.Context, path string) (bleve.Index, error) { 148 + func openIndexer(ctx context.Context, path string, version int) (bleve.Index, error) { 143 149 l := tlog.FromContext(ctx) 144 150 indexer, err := bleve.Open(path) 145 151 if err != nil { ··· 149 155 } 150 156 return nil, nil 151 157 } 158 + 159 + storedVersion, _ := indexer.GetInternal([]byte("mapping_version")) 160 + if storedVersion == nil || int(storedVersion[0]) != version { 161 + l.Info("Indexer mapping version changed, deleting and rebuilding") 162 + indexer.Close() 163 + return nil, os.RemoveAll(path) 164 + } 165 + 152 166 return indexer, nil 153 167 } 154 168 ··· 168 182 return err 169 183 } 170 184 171 - // issueData data stored and will be indexed 172 185 type issueData struct { 173 - ID int64 `json:"id"` 174 - RepoAt string `json:"repo_at"` 175 - IssueID int `json:"issue_id"` 176 - Title string `json:"title"` 177 - Body string `json:"body"` 186 + ID int64 `json:"id"` 187 + RepoAt string `json:"repo_at"` 188 + IssueID int `json:"issue_id"` 189 + Title string `json:"title"` 190 + Body string `json:"body"` 191 + IsOpen bool `json:"is_open"` 192 + AuthorDid string `json:"author_did"` 193 + Labels []string `json:"labels"` 178 194 179 - IsOpen bool `json:"is_open"` 180 195 Comments []IssueCommentData `json:"comments"` 181 196 } 182 197 183 198 func makeIssueData(issue *models.Issue) *issueData { 184 199 return &issueData{ 185 - ID: issue.Id, 186 - RepoAt: issue.RepoAt.String(), 187 - IssueID: issue.IssueId, 188 - Title: issue.Title, 189 - Body: issue.Body, 190 - IsOpen: issue.Open, 200 + ID: issue.Id, 201 + RepoAt: issue.RepoAt.String(), 202 + IssueID: issue.IssueId, 203 + Title: issue.Title, 204 + Body: issue.Body, 205 + IsOpen: issue.Open, 206 + AuthorDid: issue.Did, 207 + Labels: issue.Labels.LabelNames(), 191 208 } 192 209 } 193 210 ··· 222 239 return ix.indexer.Delete(base36.Encode(issueId)) 223 240 } 224 241 225 - // Search searches for issues 226 242 func (ix *Indexer) Search(ctx context.Context, opts models.IssueSearchOptions) (*SearchResult, error) { 227 - var queries []query.Query 243 + var musts []query.Query 244 + var mustNots []query.Query 228 245 229 - if opts.Keyword != "" { 230 - queries = append(queries, bleve.NewDisjunctionQuery( 231 - bleveutil.MatchAndQuery("title", opts.Keyword, issueIndexerAnalyzer, 0), 232 - bleveutil.MatchAndQuery("body", opts.Keyword, issueIndexerAnalyzer, 0), 246 + for _, keyword := range opts.Keywords { 247 + musts = append(musts, bleve.NewDisjunctionQuery( 248 + bleveutil.MatchAndQuery("title", keyword, issueIndexerAnalyzer, 0), 249 + bleveutil.MatchAndQuery("body", keyword, issueIndexerAnalyzer, 0), 250 + )) 251 + } 252 + 253 + for _, phrase := range opts.Phrases { 254 + musts = append(musts, bleve.NewDisjunctionQuery( 255 + bleveutil.MatchPhraseQuery("title", phrase, issueIndexerAnalyzer), 256 + bleveutil.MatchPhraseQuery("body", phrase, issueIndexerAnalyzer), 257 + )) 258 + } 259 + 260 + for _, keyword := range opts.NegatedKeywords { 261 + mustNots = append(mustNots, bleve.NewDisjunctionQuery( 262 + bleveutil.MatchAndQuery("title", keyword, issueIndexerAnalyzer, 0), 263 + bleveutil.MatchAndQuery("body", keyword, issueIndexerAnalyzer, 0), 233 264 )) 234 265 } 235 - queries = append(queries, bleveutil.KeywordFieldQuery("repo_at", opts.RepoAt)) 236 - queries = append(queries, bleveutil.BoolFieldQuery("is_open", opts.IsOpen)) 237 - // TODO: append more queries 238 266 239 - var indexerQuery query.Query = bleve.NewConjunctionQuery(queries...) 267 + for _, phrase := range opts.NegatedPhrases { 268 + mustNots = append(mustNots, bleve.NewDisjunctionQuery( 269 + bleveutil.MatchPhraseQuery("title", phrase, issueIndexerAnalyzer), 270 + bleveutil.MatchPhraseQuery("body", phrase, issueIndexerAnalyzer), 271 + )) 272 + } 273 + 274 + musts = append(musts, bleveutil.KeywordFieldQuery("repo_at", opts.RepoAt)) 275 + if opts.IsOpen != nil { 276 + musts = append(musts, bleveutil.BoolFieldQuery("is_open", *opts.IsOpen)) 277 + } 278 + 279 + if opts.AuthorDid != "" { 280 + musts = append(musts, bleveutil.KeywordFieldQuery("author_did", opts.AuthorDid)) 281 + } 282 + 283 + for _, label := range opts.Labels { 284 + musts = append(musts, bleveutil.KeywordFieldQuery("labels", label)) 285 + } 286 + 287 + if opts.NegatedAuthorDid != "" { 288 + mustNots = append(mustNots, bleveutil.KeywordFieldQuery("author_did", opts.NegatedAuthorDid)) 289 + } 290 + 291 + for _, label := range opts.NegatedLabels { 292 + mustNots = append(mustNots, bleveutil.KeywordFieldQuery("labels", label)) 293 + } 294 + 295 + indexerQuery := bleve.NewBooleanQuery() 296 + indexerQuery.AddMust(musts...) 297 + indexerQuery.AddMustNot(mustNots...) 240 298 searchReq := bleve.NewSearchRequestOptions(indexerQuery, opts.Page.Limit, opts.Page.Offset, false) 241 299 res, err := ix.indexer.SearchInContext(ctx, searchReq) 242 300 if err != nil {
+264
appview/indexer/issues/indexer_test.go
··· 1 + package issues_indexer 2 + 3 + import ( 4 + "context" 5 + "os" 6 + "testing" 7 + 8 + "github.com/blevesearch/bleve/v2" 9 + "github.com/stretchr/testify/assert" 10 + "github.com/stretchr/testify/require" 11 + "tangled.org/core/appview/models" 12 + "tangled.org/core/appview/pagination" 13 + "tangled.org/core/appview/searchquery" 14 + ) 15 + 16 + func setupTestIndexer(t *testing.T) (*Indexer, func()) { 17 + t.Helper() 18 + 19 + tmpDir, err := os.MkdirTemp("", "issue_indexer_test") 20 + require.NoError(t, err) 21 + 22 + ix := NewIndexer(tmpDir) 23 + 24 + mapping, err := generateIssueIndexMapping() 25 + require.NoError(t, err) 26 + 27 + indexer, err := bleve.New(tmpDir, mapping) 28 + require.NoError(t, err) 29 + ix.indexer = indexer 30 + 31 + cleanup := func() { 32 + ix.indexer.Close() 33 + os.RemoveAll(tmpDir) 34 + } 35 + 36 + return ix, cleanup 37 + } 38 + 39 + func boolPtr(b bool) *bool { return &b } 40 + 41 + func makeLabelState(labels ...string) models.LabelState { 42 + state := models.NewLabelState() 43 + for _, label := range labels { 44 + state.Inner()[label] = make(map[string]struct{}) 45 + state.SetName(label, label) 46 + } 47 + return state 48 + } 49 + 50 + func TestSearchFilters(t *testing.T) { 51 + ix, cleanup := setupTestIndexer(t) 52 + defer cleanup() 53 + 54 + ctx := context.Background() 55 + 56 + err := ix.Index(ctx, 57 + models.Issue{Id: 1, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Fix login bug", Body: "Users cannot login", Open: true, Did: "did:plc:alice", Labels: makeLabelState("bug")}, 58 + models.Issue{Id: 2, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Add dark mode", Body: "Implement dark theme", Open: true, Did: "did:plc:bob", Labels: makeLabelState("feature")}, 59 + models.Issue{Id: 3, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Fix login timeout", Body: "Login takes too long", Open: false, Did: "did:plc:alice", Labels: makeLabelState("bug")}, 60 + ) 61 + require.NoError(t, err) 62 + 63 + opts := func() models.IssueSearchOptions { 64 + return models.IssueSearchOptions{ 65 + RepoAt: "at://did:plc:test/sh.tangled.repo/abc", 66 + IsOpen: boolPtr(true), 67 + Page: pagination.Page{Limit: 10}, 68 + } 69 + } 70 + 71 + // Keyword in title 72 + o := opts() 73 + o.Keywords = []string{"bug"} 74 + result, err := ix.Search(ctx, o) 75 + require.NoError(t, err) 76 + assert.Equal(t, uint64(1), result.Total) 77 + assert.Contains(t, result.Hits, int64(1)) 78 + 79 + // Keyword in body 80 + o = opts() 81 + o.Keywords = []string{"theme"} 82 + result, err = ix.Search(ctx, o) 83 + require.NoError(t, err) 84 + assert.Equal(t, uint64(1), result.Total) 85 + assert.Contains(t, result.Hits, int64(2)) 86 + 87 + // Phrase match 88 + o = opts() 89 + o.Phrases = []string{"login bug"} 90 + result, err = ix.Search(ctx, o) 91 + require.NoError(t, err) 92 + assert.Equal(t, uint64(1), result.Total) 93 + assert.Contains(t, result.Hits, int64(1)) 94 + 95 + // Author filter 96 + o = opts() 97 + o.AuthorDid = "did:plc:alice" 98 + result, err = ix.Search(ctx, o) 99 + require.NoError(t, err) 100 + assert.Equal(t, uint64(1), result.Total) 101 + assert.Contains(t, result.Hits, int64(1)) 102 + 103 + // Label filter 104 + o = opts() 105 + o.Labels = []string{"bug"} 106 + result, err = ix.Search(ctx, o) 107 + require.NoError(t, err) 108 + assert.Equal(t, uint64(1), result.Total) 109 + assert.Contains(t, result.Hits, int64(1)) 110 + 111 + // State filter (closed) 112 + o = opts() 113 + o.IsOpen = boolPtr(false) 114 + o.Labels = []string{"bug"} 115 + result, err = ix.Search(ctx, o) 116 + require.NoError(t, err) 117 + assert.Equal(t, uint64(1), result.Total) 118 + assert.Contains(t, result.Hits, int64(3)) 119 + 120 + // Combined: keyword + author + label 121 + o = opts() 122 + o.Keywords = []string{"login"} 123 + o.AuthorDid = "did:plc:alice" 124 + o.Labels = []string{"bug"} 125 + result, err = ix.Search(ctx, o) 126 + require.NoError(t, err) 127 + assert.Equal(t, uint64(1), result.Total) 128 + assert.Contains(t, result.Hits, int64(1)) 129 + } 130 + 131 + func TestSearchLabelAND(t *testing.T) { 132 + ix, cleanup := setupTestIndexer(t) 133 + defer cleanup() 134 + 135 + ctx := context.Background() 136 + 137 + err := ix.Index(ctx, 138 + models.Issue{Id: 1, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Issue 1", Body: "Body", Open: true, Did: "did:plc:alice", Labels: makeLabelState("bug")}, 139 + models.Issue{Id: 2, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Issue 2", Body: "Body", Open: true, Did: "did:plc:bob", Labels: makeLabelState("bug", "urgent")}, 140 + ) 141 + require.NoError(t, err) 142 + 143 + result, err := ix.Search(ctx, models.IssueSearchOptions{ 144 + RepoAt: "at://did:plc:test/sh.tangled.repo/abc", 145 + IsOpen: boolPtr(true), 146 + Labels: []string{"bug", "urgent"}, 147 + Page: pagination.Page{Limit: 10}, 148 + }) 149 + require.NoError(t, err) 150 + assert.Equal(t, uint64(1), result.Total) 151 + assert.Contains(t, result.Hits, int64(2)) 152 + } 153 + 154 + func TestSearchNegation(t *testing.T) { 155 + ix, cleanup := setupTestIndexer(t) 156 + defer cleanup() 157 + 158 + ctx := context.Background() 159 + 160 + err := ix.Index(ctx, 161 + models.Issue{Id: 1, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Fix login bug", Body: "Users cannot login", Open: true, Did: "did:plc:alice", Labels: makeLabelState("bug")}, 162 + models.Issue{Id: 2, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Add dark mode", Body: "Implement dark theme", Open: true, Did: "did:plc:bob", Labels: makeLabelState("feature")}, 163 + models.Issue{Id: 3, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Fix timeout bug", Body: "Timeout on save", Open: true, Did: "did:plc:alice", Labels: makeLabelState("bug", "urgent")}, 164 + ) 165 + require.NoError(t, err) 166 + 167 + opts := func() models.IssueSearchOptions { 168 + return models.IssueSearchOptions{ 169 + RepoAt: "at://did:plc:test/sh.tangled.repo/abc", 170 + IsOpen: boolPtr(true), 171 + Page: pagination.Page{Limit: 10}, 172 + } 173 + } 174 + 175 + // Negated label: exclude "bug", should only return issue 2 176 + o := opts() 177 + o.NegatedLabels = []string{"bug"} 178 + result, err := ix.Search(ctx, o) 179 + require.NoError(t, err) 180 + assert.Equal(t, uint64(1), result.Total) 181 + assert.Contains(t, result.Hits, int64(2)) 182 + 183 + // Negated keyword: exclude "login", should return issues 2 and 3 184 + o = opts() 185 + o.NegatedKeywords = []string{"login"} 186 + result, err = ix.Search(ctx, o) 187 + require.NoError(t, err) 188 + assert.Equal(t, uint64(2), result.Total) 189 + assert.Contains(t, result.Hits, int64(2)) 190 + assert.Contains(t, result.Hits, int64(3)) 191 + 192 + // Positive label + negated label: bug but not urgent 193 + o = opts() 194 + o.Labels = []string{"bug"} 195 + o.NegatedLabels = []string{"urgent"} 196 + result, err = ix.Search(ctx, o) 197 + require.NoError(t, err) 198 + assert.Equal(t, uint64(1), result.Total) 199 + assert.Contains(t, result.Hits, int64(1)) 200 + 201 + // Negated phrase 202 + o = opts() 203 + o.NegatedPhrases = []string{"dark theme"} 204 + result, err = ix.Search(ctx, o) 205 + require.NoError(t, err) 206 + assert.Equal(t, uint64(2), result.Total) 207 + assert.NotContains(t, result.Hits, int64(2)) 208 + } 209 + 210 + func TestSearchNegatedPhraseParsed(t *testing.T) { 211 + ix, cleanup := setupTestIndexer(t) 212 + defer cleanup() 213 + 214 + ctx := context.Background() 215 + 216 + err := ix.Index(ctx, 217 + models.Issue{Id: 1, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Fix login bug", Body: "Users cannot login", Open: true, Did: "did:plc:alice"}, 218 + models.Issue{Id: 2, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Add dark mode", Body: "Implement dark theme", Open: true, Did: "did:plc:bob"}, 219 + models.Issue{Id: 3, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Fix timeout bug", Body: "Timeout on save", Open: true, Did: "did:plc:alice"}, 220 + ) 221 + require.NoError(t, err) 222 + 223 + // Parse a query with a negated quoted phrase, as the handler would 224 + query := searchquery.Parse(`-"dark theme"`) 225 + var negatedPhrases []string 226 + for _, item := range query.Items() { 227 + if item.Kind == searchquery.KindQuoted && item.Negated { 228 + negatedPhrases = append(negatedPhrases, item.Value) 229 + } 230 + } 231 + require.Equal(t, []string{"dark theme"}, negatedPhrases) 232 + 233 + result, err := ix.Search(ctx, models.IssueSearchOptions{ 234 + RepoAt: "at://did:plc:test/sh.tangled.repo/abc", 235 + IsOpen: boolPtr(true), 236 + NegatedPhrases: negatedPhrases, 237 + Page: pagination.Page{Limit: 10}, 238 + }) 239 + require.NoError(t, err) 240 + assert.Equal(t, uint64(2), result.Total) 241 + assert.NotContains(t, result.Hits, int64(2)) 242 + } 243 + 244 + func TestSearchNoResults(t *testing.T) { 245 + ix, cleanup := setupTestIndexer(t) 246 + defer cleanup() 247 + 248 + ctx := context.Background() 249 + 250 + err := ix.Index(ctx, 251 + models.Issue{Id: 1, RepoAt: "at://did:plc:test/sh.tangled.repo/abc", Title: "Issue", Body: "Body", Open: true, Did: "did:plc:alice"}, 252 + ) 253 + require.NoError(t, err) 254 + 255 + result, err := ix.Search(ctx, models.IssueSearchOptions{ 256 + Keywords: []string{"nonexistent"}, 257 + RepoAt: "at://did:plc:test/sh.tangled.repo/abc", 258 + IsOpen: boolPtr(true), 259 + Page: pagination.Page{Limit: 10}, 260 + }) 261 + require.NoError(t, err) 262 + assert.Equal(t, uint64(0), result.Total) 263 + assert.Empty(t, result.Hits) 264 + }
+18
appview/indexer/notifier.go
··· 38 38 } 39 39 } 40 40 41 + func (ix *Indexer) NewIssueLabelOp(ctx context.Context, issue *models.Issue) { 42 + l := log.FromContext(ctx).With("notifier", "indexer", "issue", issue) 43 + l.Debug("reindexing issue after label change") 44 + err := ix.Issues.Index(ctx, *issue) 45 + if err != nil { 46 + l.Error("failed to index an issue", "err", err) 47 + } 48 + } 49 + 50 + func (ix *Indexer) NewPullLabelOp(ctx context.Context, pull *models.Pull) { 51 + l := log.FromContext(ctx).With("notifier", "indexer", "pull", pull) 52 + l.Debug("reindexing pull after label change") 53 + err := ix.Pulls.Index(ctx, pull) 54 + if err != nil { 55 + l.Error("failed to index a pr", "err", err) 56 + } 57 + } 58 + 41 59 func (ix *Indexer) NewPull(ctx context.Context, pull *models.Pull) { 42 60 l := log.FromContext(ctx).With("notifier", "indexer", "pull", pull) 43 61 l.Debug("indexing new pr")
+83 -24
appview/indexer/pulls/indexer.go
··· 28 28 pullIndexerDocType = "pullIndexerDocType" 29 29 30 30 unicodeNormalizeName = "uicodeNormalize" 31 + 32 + // Bump this when the index mapping changes to trigger a rebuild. 33 + pullIndexerVersion = 2 31 34 ) 32 35 33 36 type Indexer struct { ··· 79 82 80 83 docMapping.AddFieldMappingsAt("repo_at", keywordFieldMapping) 81 84 docMapping.AddFieldMappingsAt("state", keywordFieldMapping) 85 + docMapping.AddFieldMappingsAt("author_did", keywordFieldMapping) 86 + docMapping.AddFieldMappingsAt("labels", keywordFieldMapping) 82 87 83 88 err := mapping.AddCustomTokenFilter(unicodeNormalizeName, map[string]any{ 84 89 "type": unicodenorm.Name, ··· 111 116 return false, errors.New("indexer is already initialized") 112 117 } 113 118 114 - indexer, err := openIndexer(ctx, ix.path) 119 + indexer, err := openIndexer(ctx, ix.path, pullIndexerVersion) 115 120 if err != nil { 116 121 return false, err 117 122 } ··· 128 133 if err != nil { 129 134 return false, err 130 135 } 136 + indexer.SetInternal([]byte("mapping_version"), []byte{byte(pullIndexerVersion)}) 131 137 132 138 ix.indexer = indexer 133 139 134 140 return false, nil 135 141 } 136 142 137 - func openIndexer(ctx context.Context, path string) (bleve.Index, error) { 143 + func openIndexer(ctx context.Context, path string, version int) (bleve.Index, error) { 138 144 l := tlog.FromContext(ctx) 139 145 indexer, err := bleve.Open(path) 140 146 if err != nil { ··· 144 150 } 145 151 return nil, nil 146 152 } 153 + 154 + storedVersion, _ := indexer.GetInternal([]byte("mapping_version")) 155 + if storedVersion == nil || int(storedVersion[0]) != version { 156 + l.Info("Indexer mapping version changed, deleting and rebuilding") 157 + indexer.Close() 158 + return nil, os.RemoveAll(path) 159 + } 160 + 147 161 return indexer, nil 148 162 } 149 163 ··· 163 177 return err 164 178 } 165 179 166 - // pullData data stored and will be indexed 167 180 type pullData struct { 168 - ID int64 `json:"id"` 169 - RepoAt string `json:"repo_at"` 170 - PullID int `json:"pull_id"` 171 - Title string `json:"title"` 172 - Body string `json:"body"` 173 - State string `json:"state"` 181 + ID int64 `json:"id"` 182 + RepoAt string `json:"repo_at"` 183 + PullID int `json:"pull_id"` 184 + Title string `json:"title"` 185 + Body string `json:"body"` 186 + State string `json:"state"` 187 + AuthorDid string `json:"author_did"` 188 + Labels []string `json:"labels"` 174 189 175 190 Comments []pullCommentData `json:"comments"` 176 191 } 177 192 178 193 func makePullData(pull *models.Pull) *pullData { 179 194 return &pullData{ 180 - ID: int64(pull.ID), 181 - RepoAt: pull.RepoAt.String(), 182 - PullID: pull.PullId, 183 - Title: pull.Title, 184 - Body: pull.Body, 185 - State: pull.State.String(), 195 + ID: int64(pull.ID), 196 + RepoAt: pull.RepoAt.String(), 197 + PullID: pull.PullId, 198 + Title: pull.Title, 199 + Body: pull.Body, 200 + State: pull.State.String(), 201 + AuthorDid: pull.OwnerDid, 202 + Labels: pull.Labels.LabelNames(), 186 203 } 187 204 } 188 205 ··· 217 234 return ix.indexer.Delete(base36.Encode(pullID)) 218 235 } 219 236 220 - // Search searches for pulls 221 237 func (ix *Indexer) Search(ctx context.Context, opts models.PullSearchOptions) (*searchResult, error) { 222 - var queries []query.Query 238 + var musts []query.Query 239 + var mustNots []query.Query 223 240 224 241 // TODO(boltless): remove this after implementing pulls page pagination 225 242 limit := opts.Page.Limit ··· 227 244 limit = 500 228 245 } 229 246 230 - if opts.Keyword != "" { 231 - queries = append(queries, bleve.NewDisjunctionQuery( 232 - bleveutil.MatchAndQuery("title", opts.Keyword, pullIndexerAnalyzer, 0), 233 - bleveutil.MatchAndQuery("body", opts.Keyword, pullIndexerAnalyzer, 0), 247 + for _, keyword := range opts.Keywords { 248 + musts = append(musts, bleve.NewDisjunctionQuery( 249 + bleveutil.MatchAndQuery("title", keyword, pullIndexerAnalyzer, 0), 250 + bleveutil.MatchAndQuery("body", keyword, pullIndexerAnalyzer, 0), 251 + )) 252 + } 253 + 254 + for _, phrase := range opts.Phrases { 255 + musts = append(musts, bleve.NewDisjunctionQuery( 256 + bleveutil.MatchPhraseQuery("title", phrase, pullIndexerAnalyzer), 257 + bleveutil.MatchPhraseQuery("body", phrase, pullIndexerAnalyzer), 258 + )) 259 + } 260 + 261 + for _, keyword := range opts.NegatedKeywords { 262 + mustNots = append(mustNots, bleve.NewDisjunctionQuery( 263 + bleveutil.MatchAndQuery("title", keyword, pullIndexerAnalyzer, 0), 264 + bleveutil.MatchAndQuery("body", keyword, pullIndexerAnalyzer, 0), 265 + )) 266 + } 267 + 268 + for _, phrase := range opts.NegatedPhrases { 269 + mustNots = append(mustNots, bleve.NewDisjunctionQuery( 270 + bleveutil.MatchPhraseQuery("title", phrase, pullIndexerAnalyzer), 271 + bleveutil.MatchPhraseQuery("body", phrase, pullIndexerAnalyzer), 234 272 )) 235 273 } 236 - queries = append(queries, bleveutil.KeywordFieldQuery("repo_at", opts.RepoAt)) 237 - queries = append(queries, bleveutil.KeywordFieldQuery("state", opts.State.String())) 274 + 275 + musts = append(musts, bleveutil.KeywordFieldQuery("repo_at", opts.RepoAt)) 276 + if opts.State != nil { 277 + musts = append(musts, bleveutil.KeywordFieldQuery("state", opts.State.String())) 278 + } 279 + 280 + if opts.AuthorDid != "" { 281 + musts = append(musts, bleveutil.KeywordFieldQuery("author_did", opts.AuthorDid)) 282 + } 238 283 239 - var indexerQuery query.Query = bleve.NewConjunctionQuery(queries...) 284 + for _, label := range opts.Labels { 285 + musts = append(musts, bleveutil.KeywordFieldQuery("labels", label)) 286 + } 287 + 288 + if opts.NegatedAuthorDid != "" { 289 + mustNots = append(mustNots, bleveutil.KeywordFieldQuery("author_did", opts.NegatedAuthorDid)) 290 + } 291 + 292 + for _, label := range opts.NegatedLabels { 293 + mustNots = append(mustNots, bleveutil.KeywordFieldQuery("labels", label)) 294 + } 295 + 296 + indexerQuery := bleve.NewBooleanQuery() 297 + indexerQuery.AddMust(musts...) 298 + indexerQuery.AddMustNot(mustNots...) 240 299 searchReq := bleve.NewSearchRequestOptions(indexerQuery, limit, opts.Page.Offset, false) 241 300 res, err := ix.indexer.SearchInContext(ctx, searchReq) 242 301 if err != nil {
+135 -56
appview/issues/issues.go
··· 27 27 "tangled.org/core/appview/pages/repoinfo" 28 28 "tangled.org/core/appview/pagination" 29 29 "tangled.org/core/appview/reporesolver" 30 + "tangled.org/core/appview/searchquery" 30 31 "tangled.org/core/appview/validator" 31 32 "tangled.org/core/idresolver" 32 33 "tangled.org/core/orm" ··· 793 794 l := rp.logger.With("handler", "RepoIssues") 794 795 795 796 params := r.URL.Query() 796 - state := params.Get("state") 797 - isOpen := true 798 - switch state { 799 - case "open": 800 - isOpen = true 801 - case "closed": 802 - isOpen = false 803 - default: 804 - isOpen = true 805 - } 806 - 807 797 page := pagination.FromContext(r.Context()) 808 798 809 799 user := rp.oauth.GetMultiAccountUser(r) ··· 813 803 return 814 804 } 815 805 806 + query := searchquery.Parse(params.Get("q")) 807 + 808 + var isOpen *bool 809 + if urlState := params.Get("state"); urlState != "" { 810 + switch urlState { 811 + case "open": 812 + isOpen = ptrBool(true) 813 + case "closed": 814 + isOpen = ptrBool(false) 815 + } 816 + query.Set("state", urlState) 817 + } else if queryState := query.Get("state"); queryState != nil { 818 + switch *queryState { 819 + case "open": 820 + isOpen = ptrBool(true) 821 + case "closed": 822 + isOpen = ptrBool(false) 823 + } 824 + } else if _, hasQ := params["q"]; !hasQ { 825 + // no q param at all -- default to open 826 + isOpen = ptrBool(true) 827 + query.Set("state", "open") 828 + } 829 + 830 + var authorDid string 831 + if authorHandle := query.Get("author"); authorHandle != nil { 832 + identity, err := rp.idResolver.ResolveIdent(r.Context(), *authorHandle) 833 + if err != nil { 834 + l.Debug("failed to resolve author handle", "handle", *authorHandle, "err", err) 835 + } else { 836 + authorDid = identity.DID.String() 837 + } 838 + } 839 + 840 + var negatedAuthorDid string 841 + if negatedAuthors := query.GetAllNegated("author"); len(negatedAuthors) > 0 { 842 + identity, err := rp.idResolver.ResolveIdent(r.Context(), negatedAuthors[0]) 843 + if err != nil { 844 + l.Debug("failed to resolve negated author handle", "handle", negatedAuthors[0], "err", err) 845 + } else { 846 + negatedAuthorDid = identity.DID.String() 847 + } 848 + } 849 + 850 + labels := query.GetAll("label") 851 + negatedLabels := query.GetAllNegated("label") 852 + 853 + var keywords, negatedKeywords []string 854 + var phrases, negatedPhrases []string 855 + for _, item := range query.Items() { 856 + switch item.Kind { 857 + case searchquery.KindKeyword: 858 + if item.Negated { 859 + negatedKeywords = append(negatedKeywords, item.Value) 860 + } else { 861 + keywords = append(keywords, item.Value) 862 + } 863 + case searchquery.KindQuoted: 864 + if item.Negated { 865 + negatedPhrases = append(negatedPhrases, item.Value) 866 + } else { 867 + phrases = append(phrases, item.Value) 868 + } 869 + } 870 + } 871 + 872 + searchOpts := models.IssueSearchOptions{ 873 + Keywords: keywords, 874 + Phrases: phrases, 875 + RepoAt: f.RepoAt().String(), 876 + IsOpen: isOpen, 877 + AuthorDid: authorDid, 878 + Labels: labels, 879 + NegatedKeywords: negatedKeywords, 880 + NegatedPhrases: negatedPhrases, 881 + NegatedLabels: negatedLabels, 882 + NegatedAuthorDid: negatedAuthorDid, 883 + Page: page, 884 + } 885 + 816 886 totalIssues := 0 817 - if isOpen { 887 + if isOpen == nil { 888 + totalIssues = f.RepoStats.IssueCount.Open + f.RepoStats.IssueCount.Closed 889 + } else if *isOpen { 818 890 totalIssues = f.RepoStats.IssueCount.Open 819 891 } else { 820 892 totalIssues = f.RepoStats.IssueCount.Closed 821 893 } 822 894 823 - keyword := params.Get("q") 824 - 825 895 repoInfo := rp.repoResolver.GetRepoInfo(r, user) 826 896 827 897 var issues []models.Issue 828 - searchOpts := models.IssueSearchOptions{ 829 - Keyword: keyword, 830 - RepoAt: f.RepoAt().String(), 831 - IsOpen: isOpen, 832 - Page: page, 833 - } 834 - if keyword != "" { 898 + 899 + if searchOpts.HasSearchFilters() { 835 900 res, err := rp.indexer.Search(r.Context(), searchOpts) 836 901 if err != nil { 837 902 l.Error("failed to search for issues", "err", err) ··· 840 905 l.Debug("searched issues with indexer", "count", len(res.Hits)) 841 906 totalIssues = int(res.Total) 842 907 843 - // count matching issues in the opposite state to display correct counts 844 - countRes, err := rp.indexer.Search(r.Context(), models.IssueSearchOptions{ 845 - Keyword: keyword, RepoAt: f.RepoAt().String(), IsOpen: !isOpen, 846 - Page: pagination.Page{Limit: 1}, 847 - }) 848 - if err == nil { 849 - if isOpen { 850 - repoInfo.Stats.IssueCount.Open = int(res.Total) 851 - repoInfo.Stats.IssueCount.Closed = int(countRes.Total) 852 - } else { 853 - repoInfo.Stats.IssueCount.Closed = int(res.Total) 854 - repoInfo.Stats.IssueCount.Open = int(countRes.Total) 855 - } 908 + // update tab counts to reflect filtered results 909 + countOpts := searchOpts 910 + countOpts.Page = pagination.Page{Limit: 1} 911 + countOpts.IsOpen = ptrBool(true) 912 + if openRes, err := rp.indexer.Search(r.Context(), countOpts); err == nil { 913 + repoInfo.Stats.IssueCount.Open = int(openRes.Total) 914 + } 915 + countOpts.IsOpen = ptrBool(false) 916 + if closedRes, err := rp.indexer.Search(r.Context(), countOpts); err == nil { 917 + repoInfo.Stats.IssueCount.Closed = int(closedRes.Total) 856 918 } 857 919 858 - issues, err = db.GetIssues( 859 - rp.db, 860 - orm.FilterIn("id", res.Hits), 861 - ) 862 - if err != nil { 863 - l.Error("failed to get issues", "err", err) 864 - rp.pages.Notice(w, "issues", "Failed to load issues. Try again later.") 865 - return 920 + if len(res.Hits) > 0 { 921 + issues, err = db.GetIssues( 922 + rp.db, 923 + orm.FilterIn("id", res.Hits), 924 + ) 925 + if err != nil { 926 + l.Error("failed to get issues", "err", err) 927 + rp.pages.Notice(w, "issues", "Failed to load issues. Try again later.") 928 + return 929 + } 866 930 } 867 - 868 931 } else { 869 - openInt := 0 870 - if isOpen { 871 - openInt = 1 932 + filters := []orm.Filter{ 933 + orm.FilterEq("repo_at", f.RepoAt()), 934 + } 935 + if isOpen != nil { 936 + openInt := 0 937 + if *isOpen { 938 + openInt = 1 939 + } 940 + filters = append(filters, orm.FilterEq("open", openInt)) 872 941 } 873 942 issues, err = db.GetIssuesPaginated( 874 943 rp.db, 875 944 page, 876 - orm.FilterEq("repo_at", f.RepoAt()), 877 - orm.FilterEq("open", openInt), 945 + filters..., 878 946 ) 879 947 if err != nil { 880 948 l.Error("failed to get issues", "err", err) ··· 899 967 defs[l.AtUri().String()] = &l 900 968 } 901 969 970 + filterState := "" 971 + if isOpen != nil { 972 + if *isOpen { 973 + filterState = "open" 974 + } else { 975 + filterState = "closed" 976 + } 977 + } 978 + 902 979 rp.pages.RepoIssues(w, pages.RepoIssuesParams{ 903 - LoggedInUser: rp.oauth.GetMultiAccountUser(r), 904 - RepoInfo: repoInfo, 905 - Issues: issues, 906 - IssueCount: totalIssues, 907 - LabelDefs: defs, 908 - FilteringByOpen: isOpen, 909 - FilterQuery: keyword, 910 - Page: page, 980 + LoggedInUser: rp.oauth.GetMultiAccountUser(r), 981 + RepoInfo: repoInfo, 982 + Issues: issues, 983 + IssueCount: totalIssues, 984 + LabelDefs: defs, 985 + FilterState: filterState, 986 + FilterQuery: query.String(), 987 + Page: page, 911 988 }) 912 989 } 990 + 991 + func ptrBool(b bool) *bool { return &b } 913 992 914 993 func (rp *Issues) NewIssue(w http.ResponseWriter, r *http.Request) { 915 994 l := rp.logger.With("handler", "NewIssue")
+18
appview/labels/labels.go
··· 13 13 "tangled.org/core/appview/db" 14 14 "tangled.org/core/appview/middleware" 15 15 "tangled.org/core/appview/models" 16 + "tangled.org/core/appview/notify" 16 17 "tangled.org/core/appview/oauth" 17 18 "tangled.org/core/appview/pages" 18 19 "tangled.org/core/appview/validator" ··· 34 35 logger *slog.Logger 35 36 validator *validator.Validator 36 37 enforcer *rbac.Enforcer 38 + notifier notify.Notifier 37 39 } 38 40 39 41 func New( ··· 42 44 db *db.DB, 43 45 validator *validator.Validator, 44 46 enforcer *rbac.Enforcer, 47 + notifier notify.Notifier, 45 48 logger *slog.Logger, 46 49 ) *Labels { 47 50 return &Labels{ ··· 51 54 logger: logger, 52 55 validator: validator, 53 56 enforcer: enforcer, 57 + notifier: notifier, 54 58 } 55 59 } 56 60 ··· 244 248 245 249 // clear aturi when everything is successful 246 250 atUri = "" 251 + 252 + subject := syntax.ATURI(subjectUri) 253 + if subject.Collection() == tangled.RepoIssueNSID { 254 + issues, err := db.GetIssues(l.db, orm.FilterEq("at_uri", subjectUri)) 255 + if err == nil && len(issues) == 1 { 256 + l.notifier.NewIssueLabelOp(r.Context(), &issues[0]) 257 + } 258 + } 259 + if subject.Collection() == tangled.RepoPullNSID { 260 + pulls, err := db.GetPulls(l.db, orm.FilterEq("at_uri", subjectUri)) 261 + if err == nil && len(pulls) == 1 { 262 + l.notifier.NewPullLabelOp(r.Context(), pulls[0]) 263 + } 264 + } 247 265 248 266 l.pages.HxRefresh(w) 249 267 }
+21
appview/models/label.go
··· 291 291 292 292 type LabelState struct { 293 293 inner map[string]set 294 + names map[string]string 294 295 } 295 296 296 297 func NewLabelState() LabelState { 297 298 return LabelState{ 298 299 inner: make(map[string]set), 300 + names: make(map[string]string), 299 301 } 300 302 } 301 303 304 + func (s LabelState) LabelNames() []string { 305 + var result []string 306 + for key, valset := range s.inner { 307 + if valset == nil { 308 + continue 309 + } 310 + if name, ok := s.names[key]; ok { 311 + result = append(result, name) 312 + } 313 + } 314 + return result 315 + } 316 + 302 317 func (s LabelState) Inner() map[string]set { 303 318 return s.inner 319 + } 320 + 321 + func (s LabelState) SetName(key, name string) { 322 + s.names[key] = name 304 323 } 305 324 306 325 func (s LabelState) ContainsLabel(l string) bool { ··· 347 366 // this def was deleted, but an op exists, so we just skip over the op 348 367 return nil 349 368 } 369 + 370 + state.names[op.OperandKey] = def.Name 350 371 351 372 switch op.Operation { 352 373 case LabelOperationAdd:
+35 -17
appview/models/search.go
··· 3 3 import "tangled.org/core/appview/pagination" 4 4 5 5 type IssueSearchOptions struct { 6 - Keyword string 7 - RepoAt string 8 - IsOpen bool 6 + Keywords []string 7 + Phrases []string 8 + RepoAt string 9 + IsOpen *bool 10 + AuthorDid string 11 + Labels []string 12 + 13 + NegatedKeywords []string 14 + NegatedPhrases []string 15 + NegatedLabels []string 16 + NegatedAuthorDid string 9 17 10 18 Page pagination.Page 11 19 } 12 20 21 + func (o *IssueSearchOptions) HasSearchFilters() bool { 22 + return len(o.Keywords) > 0 || len(o.Phrases) > 0 || 23 + o.AuthorDid != "" || o.NegatedAuthorDid != "" || 24 + len(o.Labels) > 0 || len(o.NegatedLabels) > 0 || 25 + len(o.NegatedKeywords) > 0 || len(o.NegatedPhrases) > 0 26 + } 27 + 13 28 type PullSearchOptions struct { 14 - Keyword string 15 - RepoAt string 16 - State PullState 29 + Keywords []string 30 + Phrases []string 31 + RepoAt string 32 + State *PullState 33 + AuthorDid string 34 + Labels []string 35 + 36 + NegatedKeywords []string 37 + NegatedPhrases []string 38 + NegatedLabels []string 39 + NegatedAuthorDid string 17 40 18 41 Page pagination.Page 19 42 } 20 43 21 - // func (so *SearchOptions) ToFilters() []filter { 22 - // var filters []filter 23 - // if so.IsOpen != nil { 24 - // openValue := 0 25 - // if *so.IsOpen { 26 - // openValue = 1 27 - // } 28 - // filters = append(filters, FilterEq("open", openValue)) 29 - // } 30 - // return filters 31 - // } 44 + func (o *PullSearchOptions) HasSearchFilters() bool { 45 + return len(o.Keywords) > 0 || len(o.Phrases) > 0 || 46 + o.AuthorDid != "" || o.NegatedAuthorDid != "" || 47 + len(o.Labels) > 0 || len(o.NegatedLabels) > 0 || 48 + len(o.NegatedKeywords) > 0 || len(o.NegatedPhrases) > 0 49 + }
+4 -1
appview/notify/db/db.go
··· 206 206 // no-op for now 207 207 } 208 208 209 + func (n *databaseNotifier) NewIssueLabelOp(ctx context.Context, issue *models.Issue) {} 210 + func (n *databaseNotifier) NewPullLabelOp(ctx context.Context, pull *models.Pull) {} 211 + 209 212 func (n *databaseNotifier) NewFollow(ctx context.Context, follow *models.Follow) { 210 213 actorDid := syntax.DID(follow.UserDid) 211 214 recipients := sets.Singleton(syntax.DID(follow.SubjectDid)) ··· 375 378 recipients.Insert(syntax.DID(p)) 376 379 } 377 380 378 - entityType := "pull" 381 + entityType := "issue" 379 382 entityId := issue.AtUri().String() 380 383 repoId := &issue.Repo.Id 381 384 issueId := &issue.Id
+10
appview/notify/logging_notifier.go
··· 59 59 l.inner.DeleteIssue(ctx, issue) 60 60 } 61 61 62 + func (l *loggingNotifier) NewIssueLabelOp(ctx context.Context, issue *models.Issue) { 63 + ctx = tlog.IntoContext(ctx, tlog.SubLogger(l.logger, "NewIssueLabelOp")) 64 + l.inner.NewIssueLabelOp(ctx, issue) 65 + } 66 + 67 + func (l *loggingNotifier) NewPullLabelOp(ctx context.Context, pull *models.Pull) { 68 + ctx = tlog.IntoContext(ctx, tlog.SubLogger(l.logger, "NewPullLabelOp")) 69 + l.inner.NewPullLabelOp(ctx, pull) 70 + } 71 + 62 72 func (l *loggingNotifier) NewFollow(ctx context.Context, follow *models.Follow) { 63 73 ctx = tlog.IntoContext(ctx, tlog.SubLogger(l.logger, "NewFollow")) 64 74 l.inner.NewFollow(ctx, follow)
+8
appview/notify/merged_notifier.go
··· 58 58 m.fanout(func(n Notifier) { n.DeleteIssue(ctx, issue) }) 59 59 } 60 60 61 + func (m *mergedNotifier) NewIssueLabelOp(ctx context.Context, issue *models.Issue) { 62 + m.fanout(func(n Notifier) { n.NewIssueLabelOp(ctx, issue) }) 63 + } 64 + 65 + func (m *mergedNotifier) NewPullLabelOp(ctx context.Context, pull *models.Pull) { 66 + m.fanout(func(n Notifier) { n.NewPullLabelOp(ctx, pull) }) 67 + } 68 + 61 69 func (m *mergedNotifier) NewFollow(ctx context.Context, follow *models.Follow) { 62 70 m.fanout(func(n Notifier) { n.NewFollow(ctx, follow) }) 63 71 }
+6
appview/notify/notifier.go
··· 25 25 NewPullComment(ctx context.Context, comment *models.PullComment, mentions []syntax.DID) 26 26 NewPullState(ctx context.Context, actor syntax.DID, pull *models.Pull) 27 27 28 + NewIssueLabelOp(ctx context.Context, issue *models.Issue) 29 + NewPullLabelOp(ctx context.Context, pull *models.Pull) 30 + 28 31 UpdateProfile(ctx context.Context, profile *models.Profile) 29 32 30 33 NewString(ctx context.Context, s *models.String) ··· 47 50 } 48 51 func (m *BaseNotifier) NewIssueState(ctx context.Context, actor syntax.DID, issue *models.Issue) {} 49 52 func (m *BaseNotifier) DeleteIssue(ctx context.Context, issue *models.Issue) {} 53 + 54 + func (m *BaseNotifier) NewIssueLabelOp(ctx context.Context, issue *models.Issue) {} 55 + func (m *BaseNotifier) NewPullLabelOp(ctx context.Context, pull *models.Pull) {} 50 56 51 57 func (m *BaseNotifier) NewFollow(ctx context.Context, follow *models.Follow) {} 52 58 func (m *BaseNotifier) DeleteFollow(ctx context.Context, follow *models.Follow) {}
+10 -10
appview/pages/pages.go
··· 965 965 } 966 966 967 967 type RepoIssuesParams struct { 968 - LoggedInUser *oauth.MultiAccountUser 969 - RepoInfo repoinfo.RepoInfo 970 - Active string 971 - Issues []models.Issue 972 - IssueCount int 973 - LabelDefs map[string]*models.LabelDefinition 974 - Page pagination.Page 975 - FilteringByOpen bool 976 - FilterQuery string 968 + LoggedInUser *oauth.MultiAccountUser 969 + RepoInfo repoinfo.RepoInfo 970 + Active string 971 + Issues []models.Issue 972 + IssueCount int 973 + LabelDefs map[string]*models.LabelDefinition 974 + Page pagination.Page 975 + FilterState string 976 + FilterQuery string 977 977 } 978 978 979 979 func (p *Pages) RepoIssues(w io.Writer, params RepoIssuesParams) error { ··· 1103 1103 RepoInfo repoinfo.RepoInfo 1104 1104 Pulls []*models.Pull 1105 1105 Active string 1106 - FilteringBy models.PullState 1106 + FilterState string 1107 1107 FilterQuery string 1108 1108 Stacks map[string]models.Stack 1109 1109 Pipelines map[string]models.Pipeline
+23 -1
appview/pages/templates/fragments/tabSelector.html
··· 3 3 {{ $all := .Values }} 4 4 {{ $active := .Active }} 5 5 {{ $include := .Include }} 6 + {{ $form := .Form }} 6 7 <div class="flex justify-between divide-x divide-gray-200 dark:divide-gray-700 rounded border border-gray-200 dark:border-gray-700 overflow-hidden"> 7 8 {{ $activeTab := "bg-white dark:bg-gray-700 shadow-sm" }} 8 9 {{ $inactiveTab := "bg-gray-100 dark:bg-gray-800 shadow-inner" }} 9 10 {{ range $index, $value := $all }} 10 11 {{ $isActive := eq $value.Key $active }} 12 + {{ if $form }} 13 + <button type="submit" 14 + form="{{ $form }}" 15 + name="{{ $name }}" value="{{ $value.Key }}" 16 + hx-get="?{{ $name }}={{ $value.Key }}" 17 + hx-include="{{ $include }}" 18 + hx-push-url="true" 19 + hx-target="body" 20 + hx-on:htmx:config-request="if(!event.detail.parameters.q) delete event.detail.parameters.q" 21 + class="p-2 whitespace-nowrap flex justify-center items-center gap-2 text-sm w-full hover:no-underline text-center {{ if $isActive }} {{$activeTab }} {{ else }} {{ $inactiveTab }} {{ end }}"> 22 + {{ if $value.Icon }} 23 + {{ i $value.Icon "size-4" }} 24 + {{ end }} 25 + 26 + {{ with $value.Meta }} 27 + {{ . }} 28 + {{ end }} 29 + 30 + {{ $value.Value }} 31 + </button> 32 + {{ else }} 11 33 <a href="?{{ $name }}={{ $value.Key }}" 12 34 {{ if $include }} 13 35 hx-get="?{{ $name }}={{ $value.Key }}" ··· 27 49 28 50 {{ $value.Value }} 29 51 </a> 52 + {{ end }} 30 53 {{ end }} 31 54 </div> 32 55 {{ end }} 33 -
+10 -18
appview/pages/templates/repo/issues/issues.html
··· 8 8 {{ end }} 9 9 10 10 {{ define "repoContent" }} 11 - {{ $active := "closed" }} 12 - {{ if .FilteringByOpen }} 13 - {{ $active = "open" }} 14 - {{ end }} 11 + {{ $active := .FilterState }} 15 12 16 - {{ $open := 13 + {{ $open := 17 14 (dict 18 15 "Key" "open" 19 16 "Value" "open" 20 17 "Icon" "circle-dot" 21 18 "Meta" (string .RepoInfo.Stats.IssueCount.Open)) }} 22 - {{ $closed := 19 + {{ $closed := 23 20 (dict 24 21 "Key" "closed" 25 22 "Value" "closed" ··· 28 25 {{ $values := list $open $closed }} 29 26 30 27 <div class="grid gap-2 grid-cols-[auto_1fr_auto] grid-row-2"> 31 - <form class="flex relative col-span-3 sm:col-span-1 sm:col-start-2" method="GET"> 32 - <input type="hidden" name="state" value="{{ if .FilteringByOpen }}open{{ else }}closed{{ end }}"> 28 + <form id="search-form" class="flex relative col-span-3 sm:col-span-1 sm:col-start-2" method="GET"> 33 29 <div class="flex-1 flex relative"> 34 30 <input 35 31 id="search-q" ··· 40 36 placeholder="search issues..." 41 37 > 42 38 <a 43 - href="?state={{ if .FilteringByOpen }}open{{ else }}closed{{ end }}" 39 + {{ if $active }}href="?q=state:{{ $active }}"{{ else }}href="?"{{ end }} 44 40 class="absolute right-3 top-1/2 -translate-y-1/2 text-gray-400 hover:text-gray-600 dark:hover:text-gray-300 hidden peer-[:not(:placeholder-shown)]:block" 45 41 > 46 42 {{ i "x" "w-4 h-4" }} ··· 54 50 </button> 55 51 </form> 56 52 <div class="sm:row-start-1"> 57 - {{ template "fragments/tabSelector" (dict "Name" "state" "Values" $values "Active" $active "Include" "#search-q") }} 53 + {{ template "fragments/tabSelector" (dict "Name" "state" "Values" $values "Active" $active "Include" "#search-q" "Form" "search-form") }} 58 54 </div> 59 55 <a 60 56 href="/{{ .RepoInfo.FullName }}/issues/new" ··· 72 68 {{ template "repo/issues/fragments/issueListing" (dict "Issues" .Issues "RepoPrefix" .RepoInfo.FullName "LabelDefs" .LabelDefs) }} 73 69 </div> 74 70 {{if gt .IssueCount .Page.Limit }} 75 - {{ $state := "closed" }} 76 - {{ if .FilteringByOpen }} 77 - {{ $state = "open" }} 78 - {{ end }} 79 - {{ template "fragments/pagination" (dict 80 - "Page" .Page 81 - "TotalCount" .IssueCount 71 + {{ template "fragments/pagination" (dict 72 + "Page" .Page 73 + "TotalCount" .IssueCount 82 74 "BasePath" (printf "/%s/issues" .RepoInfo.FullName) 83 - "QueryParams" (queryParams "state" $state "q" .FilterQuery) 75 + "QueryParams" (queryParams "q" .FilterQuery) 84 76 ) }} 85 77 {{ end }} 86 78 {{ end }}
+11 -17
appview/pages/templates/repo/pulls/pulls.html
··· 8 8 {{ end }} 9 9 10 10 {{ define "repoContent" }} 11 - {{ $active := "closed" }} 12 - {{ if .FilteringBy.IsOpen }} 13 - {{ $active = "open" }} 14 - {{ else if .FilteringBy.IsMerged }} 15 - {{ $active = "merged" }} 16 - {{ end }} 17 - {{ $open := 11 + {{ $active := .FilterState }} 12 + {{ $open := 18 13 (dict 19 14 "Key" "open" 20 15 "Value" "open" 21 16 "Icon" "git-pull-request" 22 17 "Meta" (string .RepoInfo.Stats.PullCount.Open)) }} 23 - {{ $merged := 18 + {{ $merged := 24 19 (dict 25 20 "Key" "merged" 26 21 "Value" "merged" 27 22 "Icon" "git-merge" 28 23 "Meta" (string .RepoInfo.Stats.PullCount.Merged)) }} 29 - {{ $closed := 24 + {{ $closed := 30 25 (dict 31 26 "Key" "closed" 32 27 "Value" "closed" ··· 34 29 "Meta" (string .RepoInfo.Stats.PullCount.Closed)) }} 35 30 {{ $values := list $open $merged $closed }} 36 31 <div class="grid gap-2 grid-cols-[auto_1fr_auto] grid-row-2"> 37 - <form class="flex relative col-span-3 sm:col-span-1 sm:col-start-2" method="GET"> 38 - <input type="hidden" name="state" value="{{ .FilteringBy.String }}"> 32 + <form id="search-form" class="flex relative col-span-3 sm:col-span-1 sm:col-start-2" method="GET"> 39 33 <div class="flex-1 flex relative"> 40 34 <input 41 35 id="search-q" ··· 46 40 placeholder="search pulls..." 47 41 > 48 42 <a 49 - href="?state={{ .FilteringBy.String }}" 43 + {{ if $active }}href="?q=state:{{ $active }}"{{ else }}href="?"{{ end }} 50 44 class="absolute right-3 top-1/2 -translate-y-1/2 text-gray-400 hover:text-gray-600 dark:hover:text-gray-300 hidden peer-[:not(:placeholder-shown)]:block" 51 45 > 52 46 {{ i "x" "w-4 h-4" }} ··· 60 54 </button> 61 55 </form> 62 56 <div class="sm:row-start-1"> 63 - {{ template "fragments/tabSelector" (dict "Name" "state" "Values" $values "Active" $active "Include" "#search-q") }} 57 + {{ template "fragments/tabSelector" (dict "Name" "state" "Values" $values "Active" $active "Include" "#search-q" "Form" "search-form") }} 64 58 </div> 65 59 <a 66 60 href="/{{ .RepoInfo.FullName }}/pulls/new" ··· 162 156 {{ end }} 163 157 </div> 164 158 {{if gt .PullCount .Page.Limit }} 165 - {{ template "fragments/pagination" (dict 166 - "Page" .Page 167 - "TotalCount" .PullCount 159 + {{ template "fragments/pagination" (dict 160 + "Page" .Page 161 + "TotalCount" .PullCount 168 162 "BasePath" (printf "/%s/pulls" .RepoInfo.FullName) 169 - "QueryParams" (queryParams "state" .FilteringBy.String "q" .FilterQuery) 163 + "QueryParams" (queryParams "q" .FilterQuery) 170 164 ) }} 171 165 {{ end }} 172 166 {{ end }}
+133 -58
appview/pulls/pulls.go
··· 31 31 "tangled.org/core/appview/pages/repoinfo" 32 32 "tangled.org/core/appview/pagination" 33 33 "tangled.org/core/appview/reporesolver" 34 + "tangled.org/core/appview/searchquery" 34 35 "tangled.org/core/appview/validator" 35 36 "tangled.org/core/appview/xrpcclient" 36 37 "tangled.org/core/idresolver" ··· 524 525 525 526 user := s.oauth.GetMultiAccountUser(r) 526 527 params := r.URL.Query() 527 - 528 - state := models.PullOpen 529 - switch params.Get("state") { 530 - case "closed": 531 - state = models.PullClosed 532 - case "merged": 533 - state = models.PullMerged 534 - } 535 - 536 528 page := pagination.FromContext(r.Context()) 537 529 538 530 f, err := s.repoResolver.Resolve(r) ··· 541 533 return 542 534 } 543 535 544 - var totalPulls int 545 - switch state { 546 - case models.PullOpen: 547 - totalPulls = f.RepoStats.PullCount.Open 548 - case models.PullMerged: 549 - totalPulls = f.RepoStats.PullCount.Merged 550 - case models.PullClosed: 551 - totalPulls = f.RepoStats.PullCount.Closed 536 + query := searchquery.Parse(params.Get("q")) 537 + 538 + var state *models.PullState 539 + if urlState := params.Get("state"); urlState != "" { 540 + switch urlState { 541 + case "open": 542 + state = ptrPullState(models.PullOpen) 543 + case "closed": 544 + state = ptrPullState(models.PullClosed) 545 + case "merged": 546 + state = ptrPullState(models.PullMerged) 547 + } 548 + query.Set("state", urlState) 549 + } else if queryState := query.Get("state"); queryState != nil { 550 + switch *queryState { 551 + case "open": 552 + state = ptrPullState(models.PullOpen) 553 + case "closed": 554 + state = ptrPullState(models.PullClosed) 555 + case "merged": 556 + state = ptrPullState(models.PullMerged) 557 + } 558 + } else if _, hasQ := params["q"]; !hasQ { 559 + state = ptrPullState(models.PullOpen) 560 + query.Set("state", "open") 552 561 } 553 562 554 - keyword := params.Get("q") 563 + var authorDid string 564 + if authorHandle := query.Get("author"); authorHandle != nil { 565 + identity, err := s.idResolver.ResolveIdent(r.Context(), *authorHandle) 566 + if err != nil { 567 + l.Debug("failed to resolve author handle", "handle", *authorHandle, "err", err) 568 + } else { 569 + authorDid = identity.DID.String() 570 + } 571 + } 555 572 556 - repoInfo := s.repoResolver.GetRepoInfo(r, user) 573 + var negatedAuthorDid string 574 + if negatedAuthors := query.GetAllNegated("author"); len(negatedAuthors) > 0 { 575 + identity, err := s.idResolver.ResolveIdent(r.Context(), negatedAuthors[0]) 576 + if err != nil { 577 + l.Debug("failed to resolve negated author handle", "handle", negatedAuthors[0], "err", err) 578 + } else { 579 + negatedAuthorDid = identity.DID.String() 580 + } 581 + } 582 + 583 + labels := query.GetAll("label") 584 + negatedLabels := query.GetAllNegated("label") 585 + 586 + var keywords, negatedKeywords []string 587 + var phrases, negatedPhrases []string 588 + for _, item := range query.Items() { 589 + switch item.Kind { 590 + case searchquery.KindKeyword: 591 + if item.Negated { 592 + negatedKeywords = append(negatedKeywords, item.Value) 593 + } else { 594 + keywords = append(keywords, item.Value) 595 + } 596 + case searchquery.KindQuoted: 597 + if item.Negated { 598 + negatedPhrases = append(negatedPhrases, item.Value) 599 + } else { 600 + phrases = append(phrases, item.Value) 601 + } 602 + } 603 + } 557 604 558 - var pulls []*models.Pull 559 605 searchOpts := models.PullSearchOptions{ 560 - Keyword: keyword, 561 - RepoAt: f.RepoAt().String(), 562 - State: state, 563 - Page: page, 606 + Keywords: keywords, 607 + Phrases: phrases, 608 + RepoAt: f.RepoAt().String(), 609 + State: state, 610 + AuthorDid: authorDid, 611 + Labels: labels, 612 + NegatedKeywords: negatedKeywords, 613 + NegatedPhrases: negatedPhrases, 614 + NegatedLabels: negatedLabels, 615 + NegatedAuthorDid: negatedAuthorDid, 616 + Page: page, 564 617 } 565 - l.Debug("searching with", "searchOpts", searchOpts) 566 - if keyword != "" { 618 + 619 + var totalPulls int 620 + if state == nil { 621 + totalPulls = f.RepoStats.PullCount.Open + f.RepoStats.PullCount.Merged + f.RepoStats.PullCount.Closed 622 + } else { 623 + switch *state { 624 + case models.PullOpen: 625 + totalPulls = f.RepoStats.PullCount.Open 626 + case models.PullMerged: 627 + totalPulls = f.RepoStats.PullCount.Merged 628 + case models.PullClosed: 629 + totalPulls = f.RepoStats.PullCount.Closed 630 + } 631 + } 632 + 633 + repoInfo := s.repoResolver.GetRepoInfo(r, user) 634 + 635 + var pulls []*models.Pull 636 + 637 + if searchOpts.HasSearchFilters() { 567 638 res, err := s.indexer.Search(r.Context(), searchOpts) 568 639 if err != nil { 569 640 l.Error("failed to search for pulls", "err", err) ··· 572 643 totalPulls = int(res.Total) 573 644 l.Debug("searched pulls with indexer", "count", len(res.Hits)) 574 645 575 - // count matching pulls in the other states to display correct counts 576 - for _, other := range []models.PullState{models.PullOpen, models.PullMerged, models.PullClosed} { 577 - if other == state { 578 - continue 579 - } 580 - countRes, err := s.indexer.Search(r.Context(), models.PullSearchOptions{ 581 - Keyword: keyword, RepoAt: f.RepoAt().String(), State: other, 582 - Page: pagination.Page{Limit: 1}, 583 - }) 646 + // update tab counts to reflect filtered results 647 + countOpts := searchOpts 648 + countOpts.Page = pagination.Page{Limit: 1} 649 + for _, ps := range []models.PullState{models.PullOpen, models.PullMerged, models.PullClosed} { 650 + ps := ps 651 + countOpts.State = &ps 652 + countRes, err := s.indexer.Search(r.Context(), countOpts) 584 653 if err != nil { 585 654 continue 586 655 } 587 - switch other { 656 + switch ps { 588 657 case models.PullOpen: 589 658 repoInfo.Stats.PullCount.Open = int(countRes.Total) 590 659 case models.PullMerged: ··· 593 662 repoInfo.Stats.PullCount.Closed = int(countRes.Total) 594 663 } 595 664 } 596 - switch state { 597 - case models.PullOpen: 598 - repoInfo.Stats.PullCount.Open = int(res.Total) 599 - case models.PullMerged: 600 - repoInfo.Stats.PullCount.Merged = int(res.Total) 601 - case models.PullClosed: 602 - repoInfo.Stats.PullCount.Closed = int(res.Total) 603 - } 604 665 605 - pulls, err = db.GetPulls( 606 - s.db, 607 - orm.FilterIn("id", res.Hits), 608 - ) 609 - if err != nil { 610 - log.Println("failed to get pulls", err) 611 - s.pages.Notice(w, "pulls", "Failed to load pulls. Try again later.") 612 - return 666 + if len(res.Hits) > 0 { 667 + pulls, err = db.GetPulls( 668 + s.db, 669 + orm.FilterIn("id", res.Hits), 670 + ) 671 + if err != nil { 672 + l.Error("failed to get pulls", "err", err) 673 + s.pages.Notice(w, "pulls", "Failed to load pulls. Try again later.") 674 + return 675 + } 613 676 } 614 677 } else { 678 + filters := []orm.Filter{ 679 + orm.FilterEq("repo_at", f.RepoAt()), 680 + } 681 + if state != nil { 682 + filters = append(filters, orm.FilterEq("state", *state)) 683 + } 615 684 pulls, err = db.GetPullsPaginated( 616 685 s.db, 617 686 page, 618 - orm.FilterEq("repo_at", f.RepoAt()), 619 - orm.FilterEq("state", searchOpts.State), 687 + filters..., 620 688 ) 621 689 if err != nil { 622 - log.Println("failed to get pulls", err) 690 + l.Error("failed to get pulls", "err", err) 623 691 s.pages.Notice(w, "pulls", "Failed to load pulls. Try again later.") 624 692 return 625 693 } ··· 688 756 orm.FilterContains("scope", tangled.RepoPullNSID), 689 757 ) 690 758 if err != nil { 691 - log.Println("failed to fetch labels", err) 759 + l.Error("failed to fetch labels", "err", err) 692 760 s.pages.Error503(w) 693 761 return 694 762 } ··· 698 766 defs[l.AtUri().String()] = &l 699 767 } 700 768 769 + filterState := "" 770 + if state != nil { 771 + filterState = state.String() 772 + } 773 + 701 774 s.pages.RepoPulls(w, pages.RepoPullsParams{ 702 775 LoggedInUser: s.oauth.GetMultiAccountUser(r), 703 776 RepoInfo: repoInfo, 704 777 Pulls: pulls, 705 778 LabelDefs: defs, 706 - FilteringBy: state, 707 - FilterQuery: keyword, 779 + FilterState: filterState, 780 + FilterQuery: query.String(), 708 781 Stacks: stacks, 709 782 Pipelines: m, 710 783 Page: page, ··· 2485 2558 w.Close() 2486 2559 return &b 2487 2560 } 2561 + 2562 + func ptrPullState(s models.PullState) *models.PullState { return &s }
+204
appview/searchquery/searchquery.go
··· 1 + package searchquery 2 + 3 + import ( 4 + "strings" 5 + "unicode" 6 + ) 7 + 8 + type ItemKind int 9 + 10 + const ( 11 + KindKeyword ItemKind = iota 12 + KindQuoted 13 + KindTagValue 14 + ) 15 + 16 + type Item struct { 17 + Kind ItemKind 18 + Negated bool 19 + Raw string 20 + Key string 21 + Value string 22 + } 23 + 24 + type Query struct { 25 + items []Item 26 + } 27 + 28 + func Parse(input string) *Query { 29 + q := &Query{} 30 + runes := []rune(strings.TrimSpace(input)) 31 + if len(runes) == 0 { 32 + return q 33 + } 34 + 35 + i := 0 36 + for i < len(runes) { 37 + for i < len(runes) && unicode.IsSpace(runes[i]) { 38 + i++ 39 + } 40 + if i >= len(runes) { 41 + break 42 + } 43 + 44 + negated := false 45 + if runes[i] == '-' && i+1 < len(runes) && runes[i+1] == '"' { 46 + negated = true 47 + i++ // skip '-' 48 + } 49 + 50 + if runes[i] == '"' { 51 + start := i 52 + if negated { 53 + start-- // include the '-' in Raw 54 + } 55 + i++ // skip opening quote 56 + inner := i 57 + for i < len(runes) && runes[i] != '"' { 58 + if runes[i] == '\\' && i+1 < len(runes) { 59 + i++ 60 + } 61 + i++ 62 + } 63 + value := unescapeQuoted(runes[inner:i]) 64 + if i < len(runes) { 65 + i++ // skip closing quote 66 + } 67 + q.items = append(q.items, Item{ 68 + Kind: KindQuoted, 69 + Negated: negated, 70 + Raw: string(runes[start:i]), 71 + Value: value, 72 + }) 73 + continue 74 + } 75 + 76 + start := i 77 + for i < len(runes) && !unicode.IsSpace(runes[i]) && runes[i] != '"' { 78 + i++ 79 + } 80 + token := string(runes[start:i]) 81 + 82 + negated = false 83 + subject := token 84 + if len(subject) > 1 && subject[0] == '-' { 85 + negated = true 86 + subject = subject[1:] 87 + } 88 + 89 + colonIdx := strings.Index(subject, ":") 90 + if colonIdx > 0 { 91 + key := subject[:colonIdx] 92 + value := subject[colonIdx+1:] 93 + q.items = append(q.items, Item{ 94 + Kind: KindTagValue, 95 + Negated: negated, 96 + Raw: token, 97 + Key: key, 98 + Value: value, 99 + }) 100 + } else { 101 + q.items = append(q.items, Item{ 102 + Kind: KindKeyword, 103 + Negated: negated, 104 + Raw: token, 105 + Value: subject, 106 + }) 107 + } 108 + } 109 + 110 + return q 111 + } 112 + 113 + func unescapeQuoted(runes []rune) string { 114 + var b strings.Builder 115 + for i := 0; i < len(runes); i++ { 116 + if runes[i] == '\\' && i+1 < len(runes) { 117 + i++ 118 + } 119 + b.WriteRune(runes[i]) 120 + } 121 + return b.String() 122 + } 123 + 124 + func (q *Query) Items() []Item { 125 + return q.items 126 + } 127 + 128 + func (q *Query) Get(key string) *string { 129 + for i, item := range q.items { 130 + if item.Kind == KindTagValue && !item.Negated && item.Key == key { 131 + return &q.items[i].Value 132 + } 133 + } 134 + return nil 135 + } 136 + 137 + func (q *Query) GetAll(key string) []string { 138 + var result []string 139 + for _, item := range q.items { 140 + if item.Kind == KindTagValue && !item.Negated && item.Key == key { 141 + result = append(result, item.Value) 142 + } 143 + } 144 + return result 145 + } 146 + 147 + func (q *Query) GetAllNegated(key string) []string { 148 + var result []string 149 + for _, item := range q.items { 150 + if item.Kind == KindTagValue && item.Negated && item.Key == key { 151 + result = append(result, item.Value) 152 + } 153 + } 154 + return result 155 + } 156 + 157 + func (q *Query) Has(key string) bool { 158 + return q.Get(key) != nil 159 + } 160 + 161 + func (q *Query) Set(key, value string) { 162 + raw := key + ":" + value 163 + found := false 164 + newItems := make([]Item, 0, len(q.items)) 165 + 166 + for _, item := range q.items { 167 + if item.Kind == KindTagValue && !item.Negated && item.Key == key { 168 + if !found { 169 + newItems = append(newItems, Item{ 170 + Kind: KindTagValue, 171 + Raw: raw, 172 + Key: key, 173 + Value: value, 174 + }) 175 + found = true 176 + } 177 + } else { 178 + newItems = append(newItems, item) 179 + } 180 + } 181 + 182 + if !found { 183 + newItems = append(newItems, Item{ 184 + Kind: KindTagValue, 185 + Raw: raw, 186 + Key: key, 187 + Value: value, 188 + }) 189 + } 190 + 191 + q.items = newItems 192 + } 193 + 194 + func (q *Query) String() string { 195 + if len(q.items) == 0 { 196 + return "" 197 + } 198 + 199 + parts := make([]string, len(q.items)) 200 + for i, item := range q.items { 201 + parts[i] = item.Raw 202 + } 203 + return strings.Join(parts, " ") 204 + }
+252
appview/searchquery/searchquery_test.go
··· 1 + package searchquery 2 + 3 + import ( 4 + "testing" 5 + 6 + "github.com/stretchr/testify/assert" 7 + ) 8 + 9 + func TestParseMixed(t *testing.T) { 10 + q := Parse(`state:open bug "critical issue" label:good-first-issue fix`) 11 + items := q.Items() 12 + assert.Equal(t, 5, len(items)) 13 + 14 + assert.Equal(t, KindTagValue, items[0].Kind) 15 + assert.Equal(t, "state", items[0].Key) 16 + assert.Equal(t, "open", items[0].Value) 17 + 18 + assert.Equal(t, KindKeyword, items[1].Kind) 19 + assert.Equal(t, "bug", items[1].Raw) 20 + 21 + assert.Equal(t, KindQuoted, items[2].Kind) 22 + assert.Equal(t, `"critical issue"`, items[2].Raw) 23 + 24 + assert.Equal(t, KindTagValue, items[3].Kind) 25 + assert.Equal(t, "label", items[3].Key) 26 + assert.Equal(t, "good-first-issue", items[3].Value) 27 + 28 + assert.Equal(t, KindKeyword, items[4].Kind) 29 + 30 + assert.Equal(t, `state:open bug "critical issue" label:good-first-issue fix`, q.String()) 31 + } 32 + 33 + func TestGetSetLifecycle(t *testing.T) { 34 + q := Parse("label:bug state:open keyword label:feature label:urgent") 35 + 36 + // Get returns first match 37 + val := q.Get("state") 38 + assert.NotNil(t, val) 39 + assert.Equal(t, "open", *val) 40 + 41 + // Get returns nil for missing key 42 + assert.Nil(t, q.Get("author")) 43 + 44 + // Has 45 + assert.True(t, q.Has("state")) 46 + assert.False(t, q.Has("author")) 47 + 48 + // GetAll 49 + assert.Equal(t, []string{"bug", "feature", "urgent"}, q.GetAll("label")) 50 + assert.Equal(t, 0, len(q.GetAll("missing"))) 51 + 52 + // Set updates existing, preserving position 53 + q.Set("state", "closed") 54 + assert.Equal(t, "label:bug state:closed keyword label:feature label:urgent", q.String()) 55 + 56 + // Set deduplicates 57 + q.Set("label", "single") 58 + assert.Equal(t, "label:single state:closed keyword", q.String()) 59 + 60 + // Set appends new tag 61 + q.Set("author", "bob") 62 + assert.Equal(t, "label:single state:closed keyword author:bob", q.String()) 63 + } 64 + 65 + func TestParseEmpty(t *testing.T) { 66 + q := Parse(" ") 67 + assert.Equal(t, 0, len(q.Items())) 68 + assert.Equal(t, "", q.String()) 69 + } 70 + 71 + func TestParseUnclosedQuote(t *testing.T) { 72 + q := Parse(`"hello world`) 73 + items := q.Items() 74 + assert.Equal(t, 1, len(items)) 75 + assert.Equal(t, KindQuoted, items[0].Kind) 76 + assert.Equal(t, `"hello world`, items[0].Raw) 77 + assert.Equal(t, "hello world", items[0].Value) 78 + } 79 + 80 + func TestParseLeadingColon(t *testing.T) { 81 + q := Parse(":value") 82 + items := q.Items() 83 + assert.Equal(t, 1, len(items)) 84 + assert.Equal(t, KindKeyword, items[0].Kind) 85 + assert.Equal(t, ":value", items[0].Raw) 86 + } 87 + 88 + func TestParseColonInValue(t *testing.T) { 89 + q := Parse("key:value:with:colons") 90 + items := q.Items() 91 + assert.Equal(t, 1, len(items)) 92 + assert.Equal(t, "key", items[0].Key) 93 + assert.Equal(t, "value:with:colons", items[0].Value) 94 + } 95 + 96 + func TestParseEmptyValue(t *testing.T) { 97 + q := Parse("state:") 98 + items := q.Items() 99 + assert.Equal(t, 1, len(items)) 100 + assert.Equal(t, KindTagValue, items[0].Kind) 101 + assert.Equal(t, "state", items[0].Key) 102 + assert.Equal(t, "", items[0].Value) 103 + } 104 + 105 + func TestQuotedKeyValueIsNotTag(t *testing.T) { 106 + q := Parse(`"state:open"`) 107 + items := q.Items() 108 + assert.Equal(t, 1, len(items)) 109 + assert.Equal(t, KindQuoted, items[0].Kind) 110 + assert.Equal(t, "state:open", items[0].Value) 111 + assert.False(t, q.Has("state")) 112 + } 113 + 114 + func TestConsecutiveQuotes(t *testing.T) { 115 + q := Parse(`"one""two"`) 116 + items := q.Items() 117 + assert.Equal(t, 2, len(items)) 118 + assert.Equal(t, `"one"`, items[0].Raw) 119 + assert.Equal(t, `"two"`, items[1].Raw) 120 + } 121 + 122 + func TestEscapedQuotes(t *testing.T) { 123 + q := Parse(`"hello \"world\""`) 124 + items := q.Items() 125 + assert.Equal(t, 1, len(items)) 126 + assert.Equal(t, KindQuoted, items[0].Kind) 127 + assert.Equal(t, `"hello \"world\""`, items[0].Raw) 128 + assert.Equal(t, `hello "world"`, items[0].Value) 129 + } 130 + 131 + func TestEscapedBackslash(t *testing.T) { 132 + q := Parse(`"hello\\"`) 133 + items := q.Items() 134 + assert.Equal(t, 1, len(items)) 135 + assert.Equal(t, KindQuoted, items[0].Kind) 136 + assert.Equal(t, `hello\`, items[0].Value) 137 + } 138 + 139 + func TestNegatedTag(t *testing.T) { 140 + q := Parse("state:open -label:bug keyword -label:wontfix") 141 + items := q.Items() 142 + assert.Equal(t, 4, len(items)) 143 + 144 + assert.False(t, items[0].Negated) 145 + assert.Equal(t, "state", items[0].Key) 146 + 147 + assert.True(t, items[1].Negated) 148 + assert.Equal(t, KindTagValue, items[1].Kind) 149 + assert.Equal(t, "label", items[1].Key) 150 + assert.Equal(t, "bug", items[1].Value) 151 + assert.Equal(t, "-label:bug", items[1].Raw) 152 + 153 + assert.True(t, items[3].Negated) 154 + assert.Equal(t, "wontfix", items[3].Value) 155 + 156 + // Get/GetAll/Has skip negated tags 157 + assert.False(t, q.Has("label")) 158 + assert.Equal(t, 0, len(q.GetAll("label"))) 159 + 160 + // Set doesn't touch negated tags 161 + q.Set("label", "feature") 162 + assert.Equal(t, "state:open -label:bug keyword -label:wontfix label:feature", q.String()) 163 + } 164 + 165 + func TestNegatedBareWordIsKeyword(t *testing.T) { 166 + q := Parse("-keyword") 167 + items := q.Items() 168 + assert.Equal(t, 1, len(items)) 169 + assert.Equal(t, KindKeyword, items[0].Kind) 170 + assert.Equal(t, "-keyword", items[0].Raw) 171 + } 172 + 173 + func TestNegatedQuotedPhrase(t *testing.T) { 174 + q := Parse(`-"critical bug" state:open`) 175 + items := q.Items() 176 + assert.Equal(t, 2, len(items)) 177 + 178 + assert.Equal(t, KindQuoted, items[0].Kind) 179 + assert.True(t, items[0].Negated) 180 + assert.Equal(t, `-"critical bug"`, items[0].Raw) 181 + assert.Equal(t, "critical bug", items[0].Value) 182 + 183 + assert.Equal(t, KindTagValue, items[1].Kind) 184 + assert.Equal(t, "state", items[1].Key) 185 + } 186 + 187 + func TestNegatedQuotedPhraseAmongOthers(t *testing.T) { 188 + q := Parse(`"good phrase" -"bad phrase" keyword`) 189 + items := q.Items() 190 + assert.Equal(t, 3, len(items)) 191 + 192 + assert.Equal(t, KindQuoted, items[0].Kind) 193 + assert.False(t, items[0].Negated) 194 + assert.Equal(t, "good phrase", items[0].Value) 195 + 196 + assert.Equal(t, KindQuoted, items[1].Kind) 197 + assert.True(t, items[1].Negated) 198 + assert.Equal(t, "bad phrase", items[1].Value) 199 + 200 + assert.Equal(t, KindKeyword, items[2].Kind) 201 + } 202 + 203 + func TestWhitespaceNormalization(t *testing.T) { 204 + q := Parse(" state:open keyword ") 205 + assert.Equal(t, "state:open keyword", q.String()) 206 + } 207 + 208 + func TestParseConsecutiveColons(t *testing.T) { 209 + q := Parse("foo:::bar") 210 + items := q.Items() 211 + assert.Equal(t, 1, len(items)) 212 + assert.Equal(t, KindTagValue, items[0].Kind) 213 + assert.Equal(t, "foo", items[0].Key) 214 + assert.Equal(t, "::bar", items[0].Value) 215 + } 216 + 217 + func TestParseEmptyQuotes(t *testing.T) { 218 + q := Parse(`""`) 219 + items := q.Items() 220 + assert.Equal(t, 1, len(items)) 221 + assert.Equal(t, KindQuoted, items[0].Kind) 222 + assert.Equal(t, `""`, items[0].Raw) 223 + assert.Equal(t, "", items[0].Value) 224 + } 225 + 226 + func TestParseBareDash(t *testing.T) { 227 + q := Parse("-") 228 + items := q.Items() 229 + assert.Equal(t, 1, len(items)) 230 + assert.Equal(t, KindKeyword, items[0].Kind) 231 + assert.False(t, items[0].Negated) 232 + assert.Equal(t, "-", items[0].Raw) 233 + } 234 + 235 + func TestParseDashColon(t *testing.T) { 236 + q := Parse("-:value") 237 + items := q.Items() 238 + assert.Equal(t, 1, len(items)) 239 + assert.Equal(t, KindKeyword, items[0].Kind) 240 + assert.True(t, items[0].Negated) 241 + assert.Equal(t, ":value", items[0].Value) 242 + } 243 + 244 + func TestParseDoubleHyphen(t *testing.T) { 245 + q := Parse("--label:bug") 246 + items := q.Items() 247 + assert.Equal(t, 1, len(items)) 248 + assert.Equal(t, KindTagValue, items[0].Kind) 249 + assert.True(t, items[0].Negated) 250 + assert.Equal(t, "-label", items[0].Key) 251 + assert.Equal(t, "bug", items[0].Value) 252 + }
+1
appview/state/router.go
··· 340 340 s.db, 341 341 s.validator, 342 342 s.enforcer, 343 + s.notifier, 343 344 log.SubLogger(s.logger, "labels"), 344 345 ) 345 346 return ls.Router()

History

4 rounds 6 comments
sign up or login to add to the discussion
octet-stream.net submitted #3
interdiff
8 commits
expand
2ecf5ef1
appview/notify: fix entityType in NewIssueState notification
901ad387
appview/searchquery: add search query parser with negation support
337ba8ae
appview: add label name resolution to LabelState
54a563cb
appview: add search filters for issues and pulls
303e2fad
appview/indexer: force bleve index rebuild when mapping changes
6ce77b01
appview: re-index issues and pulls when labels are modified
9cdb9b52
appview: update tab counts to reflect search filters
bf6bfafb
appview/searchquery: test additional parser edge cases
expand 2 comments
octet-stream.net

Rebased on latest master

anirudh.fi

Thanks for your work on this!

pull request successfully merged
octet-stream.net submitted #2
diff interdiff
8 commits
expand
fbb955a9
appview/notify: fix entityType in NewIssueState notification
c33eb04d
appview/searchquery: add search query parser with negation support
21656a01
appview: add label name resolution to LabelState
787073bd
appview: add search filters for issues and pulls
b4b0dd23
appview/indexer: force bleve index rebuild when mapping changes
d2d9896e
appview: re-index issues and pulls when labels are modified
5ad31f33
appview: update tab counts to reflect search filters
124bee57
appview/searchquery: test additional parser edge cases
expand 0 comments
octet-stream.net submitted #1
diff interdiff
7 commits
expand
fbb955a9
appview/notify: fix entityType in NewIssueState notification
c33eb04d
appview/searchquery: add search query parser with negation support
21656a01
appview: add label name resolution to LabelState
787073bd
appview: add search filters for issues and pulls
b4b0dd23
appview/indexer: force bleve index rebuild when mapping changes
d2d9896e
appview: re-index issues and pulls when labels are modified
5ad31f33
appview: update tab counts to reflect search filters
expand 3 comments
octet-stream.net

Rebased on master to fix conflict. I dropped the changes from #1062 since it was incompatible with this branch and the pull counts are already correct.

boltless.me

Thank you for your amazing work! The implementation seems good to me.

Might be a nit pick, but I think we need more tests for malformed queries like state:foo, foo:::bar or "foo bar to ensure the behaviour is always consistent and work as expected.

Also note for other reviewers: this PR doesn't handle edits and thats on my fault not implementing reindex logic on updates. Will open an issue and self-assign myself.

octet-stream.net

I've added a handful of unit tests to the parser exploring edge cases, which I'll push as a revision in a moment. I'm satisfied with the current behaviour but writing out the tests is helpful to show that it's deliberate. 👍

Regarding your specific examples:

  • foo:::bar has a new test showing this will be interpreted as key foo with value ::bar. It could be argued that treating any number of colons as a separator is "nicer" but I don't expect it arise often.
  • "foo bar has an existing TestParseUnclosedQuote which shows that it will be auto-closed if it comes at the end of a query
  • state:foo is a different kettle of fish since it's downstream of the parser...

First, the behaviour. Currently the pulls and issues handlers try to match state against one of the specific words (open, closed, merged). If it doesn't match any then it will be ignored entirely, i.e. results will not be filtered by state after all. To me this seems more helpful than returning no results, even though that would be strictly correct. I'm open to alternative opinions here.

Second, testing. The logic I just described is spelt out in the issues and pulls handlers. It is technically possible to extract that into a helper function and unit test it but it would end up looking very trivial. Again, happy to explore testing further if this is desirable.

octet-stream.net submitted #0
diff
7 commits
expand
439d41be
appview/notify: fix entityType in NewIssueState notification
edf823b5
appview/searchquery: add search query parser with negation support
af261776
appview: add label name resolution to LabelState
b5a6d4cf
appview: add search filters for issues and pulls
f171e2c5
appview/indexer: force bleve index rebuild when mapping changes
64a1c37d
appview: re-index issues and pulls when labels are modified
c9ad56db
appview: update tab counts to reflect search filters
expand 1 comment
oppi.li

incredible work! will give this a test