Adopts ListenBrainz-inspired patterns to speed up imports from ~24h to
under 30 minutes for 49k scrobbles.
Phase 1 - track_lookup cache table:
- New migration (000006) adds persistent entity lookup cache
- Maps normalized (artist, track, album) → (artist_id, album_id, track_id)
- SubmitListen fast path: cache hit skips 18 DB queries → 2 queries
- Cache populated after entity resolution, invalidated on merge/delete
- Benefits both live scrobbles and imports
Phase 2 - SaveListensBatch:
- New batch listen insert using pgx CopyFrom → temp table → INSERT ON CONFLICT
- Thousands of inserts per second vs one-at-a-time
Phase 3 - BulkSubmitter:
- Reusable import accelerator for all importers
- Pre-deduplicates scrobbles by (artist, track, album) in memory
- Worker pool (4 goroutines) for parallel entity creation on cache miss
- Batch listen insertion via SaveListensBatch
Phase 4 - Migrate importers:
- Maloja, Spotify, LastFM, ListenBrainz importers use BulkSubmitter
- Koito importer left as-is (already fast with pre-resolved IDs)
Phase 5 - Skip image lookups during import:
- GetArtistImage/GetAlbumImage calls fully skipped when SkipCacheImage=true
- Background tasks (FetchMissingArtistImages/FetchMissingAlbumImages) backfill
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a listen arrives with no MBZ IDs and no album title (the common
multi-scrobbler/Last.fm case), search MusicBrainz by artist+track name
to resolve recording, release, and release group IDs. This unlocks
CoverArtArchive album art, proper album association, and duration data.
New file: internal/mbz/search.go
- SearchRecording() method with Lucene query escaping
- Confidence filter: case-insensitive exact match on title + artist credit
- Release selection: prefer Official status, then first available
- Uses existing rate-limited queue (1 req/sec)
Integration in catalog.go:
- Only triggers when RecordingMbzID, ReleaseMbzID, AND ReleaseTitle are
all missing — no impact on scrobbles that already have MBZ data
- Soft failure — search errors don't block the listen
- KOITO_DISABLE_MUSICBRAINZ handled automatically (MbzErrorCaller returns error)
Interface + mocks updated:
- SearchRecording added to MusicBrainzCaller interface
- MbzMockCaller: SearchResults map for test data
- MbzErrorCaller: returns error (existing pattern)
New tests:
- TestSubmitListen_SearchByName — mock search, verify album+duration resolved
- TestSubmitListen_SearchByNameNoMatch — verify graceful fallback
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* add dev branch container to workflow
* correctly set the default range of ActivityGrid
* fix: set name/short_name to koito (#61)
* fix dev container push workflow
* fix: race condition with using getComputedStyle primary color for dynamic activity grid darkening (#76)
* Fix race condition with using getComputedStyle primary color for dynamic activity grid darkening
Instead just use the color from the current theme directly. Tested works on initial load and theme changes.
Fixes https://github.com/gabehf/Koito/issues/75
* Rework theme provider to provide the actual Theme object throughtout the app, in addition to the name
Split name out of the Theme struct to simplify custom theme saving/reading
* fix: set first artist listed as primary by default (#81)
* feat: add server-side configuration with default theme (#90)
* docs: add example for usage of the main listenbrainz instance (#71)
* docs: add example for usage of the main listenbrainz instance
* Update scrobbler.md
---------
Co-authored-by: Gabe Farrell <90876006+gabehf@users.noreply.github.com>
* feat: add server-side cfg and default theme
* fix: repair custom theme
---------
Co-authored-by: m0d3rnX <jesper@posteo.de>
* docs: add default theme cfg option to docs
* feat: add ability to manually scrobble track (#91)
* feat: add button to manually scrobble from ui
* fix: ensure timestamp is in the past, log fix
* test: add integration test
* feat: add first listened to dates for media items (#92)
* fix: ensure error checks for ErrNoRows
* feat: add now playing endpoint and ui (#93)
* wip
* feat: add now playing
* fix: set default theme when config is not set
* feat: fetch images from subsonic server (#94)
* fix: useQuery instead of useEffect for now playing
* feat: custom artist separator regex (#95)
* Fix race condition with using getComputedStyle primary color for dynamic activity grid darkening
Instead just use the color from the current theme directly. Tested works on initial load and theme changes.
Fixes https://github.com/gabehf/Koito/issues/75
* Rework theme provider to provide the actual Theme object throughtout the app, in addition to the name
Split name out of the Theme struct to simplify custom theme saving/reading
* feat: add server-side configuration with default theme (#90)
* docs: add example for usage of the main listenbrainz instance (#71)
* docs: add example for usage of the main listenbrainz instance
* Update scrobbler.md
---------
Co-authored-by: Gabe Farrell <90876006+gabehf@users.noreply.github.com>
* feat: add server-side cfg and default theme
* fix: repair custom theme
---------
Co-authored-by: m0d3rnX <jesper@posteo.de>
* fix: rebase errors
---------
Co-authored-by: pet <128837728+againstpetra@users.noreply.github.com>
Co-authored-by: mlandry <mike.landry@gmail.com>
Co-authored-by: m0d3rnX <jesper@posteo.de>
* feat: search/merge items by id
* feat: update track duration using musicbrainz
* chore: changelog
* fix: make username updates case insensitive
* feat: add minutes listened to ui and fix image drop
* chore: changelog
* fix: embed db migrations (#37)
* feat: Add support for ARM in publish workflow (#51)
* chore: changelog
* docs: search by id and custom theme support
---------
Co-authored-by: potatoattack <lvl70nub@gmail.com>
Co-authored-by: Benjamin Jonard <benjaminjonard@users.noreply.github.com>
