yambs

module
v0.1.12-0...-549fd57 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: May 3, 2024 License: BSD-3-Clause

README

yambs (Yet Another MusicBrainz Seeder)

Build Status

yambs is a command-line program for seeding edits to the MusicBrainz music database.

It can simplify adding multiple standalone recordings: given a CSV or TSV file describing recordings, yambs can open the Add Standalone Recording page for each with various fields pre-filled. The Add Artist, Add Event, Add Label, and Add Work pages can be seeded in a similar manner.

yambs can also read key=value lines from text files to seed the Add Release page, and it can use Bandcamp, Qobuz, and Tidal album pages, Metal Archives artist and band pages, and local MP3 files to seed edits too.

There's a web frontend at yambs.erat.org.

Installation

To compile and install the yambs executable, install Go and run the following command:

go install ./cmd/yambs

If the account that you will use to run the yambs executable does not have access to the Google Cloud Translation API (used to detect releases' languages and scripts), you can supply the nogcp build tag to avoid attempting to connect to the service:

go install -tags nogcp ./cmd/yambs

Prebuilt executables are also available.

Usage

Usage: yambs [flag]... <FILE/URL>
Seeds MusicBrainz edits.

  -action value
    	Action to perform with seed URLs (open, print, serve, write) (default open)
  -addr string
    	Address to listen on for -action=serve (default "localhost:8999")
  -charset string
    	Charset for text input if not UTF-8 (IANA or MIME, e.g. "ISO-8859-1" or "latin1")
  -country string
    	Country code for querying Tidal API (ISO 3166, e.g. "US" or "DE"; "XW" for all)
  -fields string
    	Comma-separated fields for CSV/TSV columns (e.g. "artist,name,length")
  -format value
    	Format for text input (csv, keyval, tsv) (default tsv)
  -list-fields
    	Print available fields for -type and exit
  -modify-data value
    	Comma-separated modifications for online data (extract-artists, punctuation, remove-parens, split-artists)
  -server string
    	MusicBrainz server hostname (default "musicbrainz.org")
  -set value
    	Set a field for all entities (e.g. "edit_note=from https://www.example.org")
  -timeout duration
    	Timeout for generating edits (e.g. "30s" or "2m")
  -type value
    	Entity type for text or MP3 input (artist, event, label, recording, release, work)
  -verbose
    	Enable verbose logging
  -version
    	Print the version and exit

yambs reads the supplied file or URL (or stdin if no positional argument is supplied) and performs the action specified by the -action flag:

  • open: Open edits in a browser using a temporary file.
  • print: Write edit links to stdout (only possible for recordings).
  • serve: Open edits in a browser using a short-lived webserver launched at -addr (useful if you're running yambs in a container).
  • write: Write a webpage containing the edits to stdout.

If you supply a URL, yambs will fetch and parse it.

If you supply a filename, you should also pass the -type, -format, -fields, and -set flags to tell yambs how to interpret the file.

Examples

To add multiple non-album recordings for a single artist, you can run a command like the following:

yambs \
  -type recording \
  -format tsv \
  -fields name,length,edit_note \
  -set artist=7e84f845-ac16-41fe-9ff8-df12eb32af55 \
  -set url0_url=https://www.example.org/ \
  -set url0_type=255 \
  <recordings.tsv

with a recordings.tsv file like the following (with tab characters between the fields):

Song #1	4:35	info from https://example.org/song1.html
Song #2	53234.35	info from https://example.org/song2.html

The recordings' names, lengths, and edit notes will be read from the TSV file, and the -set artist=... flag sets all recordings' artist field to the specified artist.

Likewise, the -set url0_... flags add a URL relationship to each recording. seed/enums.go enumerates the different link types that can be specified between entities; 255 corresponds to LinkType_DownloadForFree_Recording_URL.


To edit existing recordings, specify their MBIDs via the mbid field:

yambs \
  -type recording \
  -format csv \
  -fields mbid,name \
  <recordings.csv

recordings.csv:

c55e74ff-bd7d-40ff-a591-c6993c59bda8,Sgt. Pepper’s Lonely Hearts Club Band
...

Note that this example uses the csv format rather than tsv.


More-complicated artist credits can also be assigned:

yambs \
  -type recording \
  -format tsv
  -fields ... \
  -set artist0_mbid=1a054dd8-c5fa-40b6-9397-61c26b0185d4 \
  -set artist0_credited=virt \
  -set 'artist0_join= & ' \
  -set artist1_name=Rush \
  ...

(Note that repeated fields are 0-indexed.)


The keyval format can be used to seed a single entity across multiple lines:

yambs -type release -format keyval <release.txt

release.txt:

title=Some Album
artist0_name=Some Artist
types=Album,Soundtrack
status=Official
packaging=Jewel Case
language=eng
script=Latn
event0_date=2021-05-15
event0_country=XW
medium0_format=CD
medium0_track0_title=First Track
medium0_track0_length=3:45.04
medium0_track1_title=Second Track
medium1_format=CD
medium1_track0_title=First Track on Second Disc
url0_url=https://www.example.org/
url0_type=75
edit_note=https://www.example.org

seed/enums.go shows that the url0_type=75 line corresponds to LinkType_DownloadForFree_Release_URL.


If you'd like to bulk-add LinkType_Published_Label_Release (ID 362) relationships between the existing label 02442aba and releases 43bcfb95 and a9d8b538, you can set the mbid field to edit the label and seed the new relationships:

yambs -type label -format keyval <label.txt

label.txt:

mbid=02442aba-cf00-445c-877e-f0eaa504d8c2
rel0_target=43bcfb95-f26c-4f8d-84f8-7b2ac5b8ab72
rel0_type=362
rel1_target=a9d8b538-c20a-4025-aea1-5530d616a20a
rel1_type=362

Pass the -list-fields flag to list all available fields for a given entity type:

yambs -type label     -list-fields
yambs -type recording -list-fields
yambs -type release   -list-fields
yambs -type work      -list-fields

Acceptable values for various fields are listed in seed/enums.go, which is automatically generated from t/sql/initial.sql in the musicbrainz-server repository.

A column in an input file can be assigned to multiple fields by supplying slash-separated field names. For example, -fields name,url0_url/edit_note,length maps the first column to field name, the second column to fields url0_url and edit_note, and the third column to field length.

A column can be skipped by passing an empty field name. For example, -fields name,,length maps the first column to field name, skips the second column, and maps the third column to length.


You can pass Bandcamp, Qobuz, or Tidal album URLs to seed release edits:

yambs https://austinwintory.bandcamp.com/album/journey
yambs https://www.qobuz.com/us-en/album/the-dark-side-of-the-moon-pink-floyd/xggxq5w5dmljb
yambs https://tidal.com/browse/album/55391786

The page that is opened will include a link to the album's highest-resolution cover art to make it easier to add in a followup edit.

If you pass a Bandcamp track URL that isn't part of an album, an edit to add it as a single will be created:

yambs https://caribouband.bandcamp.com/track/tin

You can pass the path to a local MP3 file to use it to seed a (single) release or standalone recording edit:

yambs \
  -type recording \
  -set artist=7e84f845-ac16-41fe-9ff8-df12eb32af55 \
  -set edit_note='from artist-provided MP3 at https://www.example.org/song.mp3' \
  /path/to/a/song.mp3

If the MP3 file contains embedded images, they will be extracted to temporary files so they can be added as cover art.


There's also a yambsd executable that exposes most of the same functionality through a webpage (with some limits to avoid abuse).

Why?

There are a bunch of MusicBrainz userscripts that run in the browser with the help of an extension like Tampermonkey to seed edits. They're well-tested, so why not just use them instead of writing a new thing?

Well, at first I was adding a bunch of standalone recordings that I'd downloaded from random musicians' homepages. I couldn't find any userscripts to help with that, since the main focus seems to be seeding releases from major websites. I ended up hacking together a shell script to generate URLs that would seed my edits, but I figured it'd be nice to have something more robust and convenient to use next time.

I had also been using the bandcamp_importer.user.js userscript to import releases from Bandcamp, but I'm nervous about using extensions like Tampermonkey that require permission to modify data on all sites. I'm not so worried about malice on the part of extension or userscript developers, but I have no idea about their security practices and I'm fearful of attackers compromising their computers and uploading malicious versions of their code.

I created a separate browser profile that I could use to run Tampermonkey without exposing any of my (non-MusicBrainz) credentials, but using it was a pain, so I decided to add Bandcamp support to this codebase as well since that's where I get most of my music.

Further reading

Directories

Path Synopsis
Package cache contains cache implementations.
Package cache contains cache implementations.
cmd
yambs
Package main implements a command-line program for generating seeded edits.
Package main implements a command-line program for generating seeded edits.
yambsd
Package main implements a web server for generating seeded edits.
Package main implements a web server for generating seeded edits.
Package mbdb contains functionality related to the MusicBrainz database.
Package mbdb contains functionality related to the MusicBrainz database.
Package render generates HTML pages listing edits.
Package render generates HTML pages listing edits.
res
Transpiles TypeScript files to JavaScript for the embedjs build flag.
Transpiles TypeScript files to JavaScript for the embedjs build flag.
Package seed generates URLs that pre-fill fields when adding entities to MusicBrainz.
Package seed generates URLs that pre-fill fields when adding entities to MusicBrainz.
gen
slog performs structured logging when running on App Engine.
slog performs structured logging when running on App Engine.
sources
mp3
Package mp3 generates seeded edits from metadata in MP3 files.
Package mp3 generates seeded edits from metadata in MP3 files.
online
Package online generates seeded edits from online music providers or databases.
Package online generates seeded edits from online music providers or databases.
online/bandcamp
Package bandcamp extracts information from Bandcamp pages.
Package bandcamp extracts information from Bandcamp pages.
online/internal
Package internal defines internal types and functions for online sources.
Package internal defines internal types and functions for online sources.
online/metalarchives
Package metalarchives extracts information from metal-archives.com pages.
Package metalarchives extracts information from metal-archives.com pages.
online/qobuz
Package qobuz extracts information from Qobuz pages.
Package qobuz extracts information from Qobuz pages.
online/tidal
Package tidal uses Tidal's API to seed edits.
Package tidal uses Tidal's API to seed edits.
text
Package text parses entities from textual input like CSV or TSV files.
Package text parses entities from textual input like CSV or TSV files.
Package strutil contains string-related utility functions.
Package strutil contains string-related utility functions.
Package web interacts with web pages.
Package web interacts with web pages.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL