JukeboxManager



Introduction

There exist a lot of application for managing a large collection of MP3 tracks. Most of them allow you not only to keep track which tracks are in your collection but also to perform operations on the collection itself. This might be needed because while downloading you might have had network problems which might have caused the MP3 file to by just slightly incomplete or you might have forgotton that you already downloaded a certain track and find that you have just copied it over again or you might have noticed that the tagging on a particular track is not what is should be because the original creator made a type-o or used the CDDB database which in its turn contained a type-O.

On the other hand there seem to be NO jukeboxManagers around that not only manage your collection but also allow you to load your favourite MP3 player device with a set of tracks. This is especially useful if your MP3 player happens to be a jukebox like the PJB100 or the Nomad.

Since these devices support large discs to store tracks, it takes quite some time to load these players with a significant set of tracks. This actually can take hours. Moreover, the discs are so large that they easily can contain multiple CDRom full of MP3 tracks. This causes the load process to become even more difficult since you need to insert the proper CD in the right time.

The JukeboxMgr, allows you to do just that : manage a large collection of tracks on multiple (CD)volumes and provide a 'batch' oriented load mechanism for your jukebox device.

Currently the JukeboxMgr only supports the PJB100 but it should be able to manage other jukeboxes as well.

Further Information

  1. Here is the homepage of the JukeboxMgr where you can find this document too (Thanx SourceForge)
  2. Here you find the Web interface to our CVS repository on sourceforge
  3. Here is the download page where you should be able to find the latest version

Design

    The following basic principles were the basis of the design of the JukeboxMgr

    be able to support many different jukebox-like MP3 devices

    be able to generate TableOfContents (TOC's) automatically

    be able to scan an arbitrarily structured database of MP3 files

    be able to update ANY information at ANY time even if the MP3 file is stored on a non-modifiable medium

    be able to show and modify tagging information

    support multiple medium readers (e.g multiple Cdroms or a combination of a CDRom and CDReader)

    Be able to add devices to the list of allowed devices (e.g. Add a Cdrom that is mounted over the network as a valid Cdrom device)

    Be able to find duplicates, find bad tracks

    Be able to manage albums as well as loose tracks

    Be able to set the playing sequence of albums in a track (especially useful for LIVE albums)\

Basic architeture

It is important to understand the different layers of data that the JukeboxMgr manages. First you have the MP3 filenames themselves. It is clear that there are many different formats possible. However the JukeboxMgr implements and understands 2 basic formats

CC - TT - AAAAA - TRTRTR and AAAAA - TRTRTR

where CC is the number of the CD in the album (see further)

where TT is the number of the track in the CD

where AAA is the name of the artist

where TRTRTR is the name of the track

If the filename corresponds to this format, the JukeboxMgr while scanning for MP3 tracks will automatically parse out the appropriate information.

The second source of information is the ID3 tag. When present, the JukeboxMgr, when it has located a MP3 track, will open it and find the tag information.

The third source of information, and the one that has the highest priority, is the Database. Actually it is not really a database but just an ascii file called Data in some directory. Besides other information, this file can contain value that overrule the filename and/or tag data. This might be needed whe the MP3 file is stored on a non-modifiable medium like a CDROM.

When performing updates on the tracks, the changed data is stored in the database. Sometimes, when your medium is capable to change the filename/ID3 tag information, you can push the changed data from the database to the lower level data stores (filename/ID3 tag). Once this pushing is sucessful, corresponding entry in the database is cleared out.

Basic structure of the database

The jukeboxMgr shows the track data in a structured way.

On the highest level you have the volumes. Volumes correspond to e.g a CDROM. Is allows you to assing a logical name to a set of tracks stored on some kind of medium.

Assigned to these volumes are media categories. Each volume has one and only one medium category. A medium category gives a logical name to a set of devices that have identical capabilities. Typical example is a CDROM and a CDWRITER. Both are capable to be used as a CDROM reader and typically belong to on Medium Category. However a remotely mounted CDROM (over NFS for instance) might also be a device instance in this category.

Finally, as almost descibed above you have the devices themselves. Devices are scanned automatically from the system configuration. This implies that each devices (even a remotely accessible CDROM) should be defined in the appropriate file (/etc/fstab is typical). However you are allowed to create directory entries as devices. These entries might be mount points for the automounter or might be just a normal directory on some harddisk.

Volumes contain artist. Artist are entries that correspond to real artists. However often you have also a loose set of tracks that really belong to no album. Therefore a special Artist entry "Various" is used. This special entry combines all odd tracks. Actually that name "Various" is configurable (but currently not through the UI)

Under artists we find the albums. For those tracks that are not belonging to an album the special album "Collection" is created. This also applies for the artist category 'Various'. Also this 'Collection' name can be configured.

The lowest level are tracks themselves

Managing Albums and artists

Because the JukeboxMgr cannot automatically figure out which tracks belong to a particular album, you MUST put tracks that belong to oneanother in a subdirectory with the name of the album. If tracks belonging to one album are not put in a special subdirectory, the JukeboxMgr cannot create an album entry.

The jukeboxMgr scans for MP3 in each subdirectory of a certain directory. However internally in the manager are a set of directory rules (currently NOT configurable) that determine how a MP3 track is added to the database. Based on that rule the manager will decide which track belongs to what album of which artist.

The following directory rules are applied
 

RL

Artistname / Albumname

Typical structure of an album of an artist

RD

Artistname / some directory

Typical structure of some loose tracks of an artist stored NEXT to normal albums

R

Artistname

Typical structure of an artist (with no real albums) but just the loose tracks stored under the aritst's directory

Just the tracks

Flat collection of tracks in the 'root' directory of the volume

D

Some directory

Tracks can be 1 deep

DD

Some directory / Some directory

2 deep

DDD

Some directory / Some directory / Some directory

Or even 3 deep


 
 

Basic operations

Configuring the Jukebox Manager

The most important option is the Database location.  This value should point to a directory in which the JukeboxMgr can store its data.  Hence this directory should be readable and probably somewhere in your HOME directory.




The next option, the Jukebox Devicefile is currently not used so you can skip it

Next you have the MP3 player.  This field should contain a command of an MP3 player that you want to use to test the quality of the tracks. This field should contain a '%s' where you want the track to be played to be inserted.  An example of this field might be '/usr/local/bin/kmpg %s'

Next you have the Activate Trace Mode option.  Checking this box will cause the JukeboxMgr to verbosely print statements on the parent console with information
about the actions it is performing.

The Autosave Every checkbox with its corresponding spinbox allow you to select if you want the changes that you have made to the MP3 Database to be stored in the
jukebox database location above and at what intervals you would like to do this

The Commit Every checkbox on the otherhand allows you to select if you want the changes that you have made to the MP3 databse to be stored in the tracks themselves (like the TAG information or the filename) and how frequently you want this to be performed

The Double Compare Length spinbox allows you to define the number of characters that will be compared when looking for double tracks in the database.

It is clear that if you do not specify a Database location, you cannot save anything.  if you specify no MP3 player you cannot play anything

Creating a device category

The first step in creating a volume is making sure that there is a category that corresponds to the media that this volume will be stored on. Typically this is a CDROM device or a harddisk.

Add a volume
Clicking on the 'configuration' tab and selecting the 'media' subtab should bring you to the image on the right.

There are two parts in this window. The top part allows you to add/delete and modify Categories. The lower part allows you to define devices that can handle media.

Normally the lower listview should have been populated with all physical devices that have been defined in your system files (/etc/fstab). This list is probably too extensive since it contains 'filesystems' that will never in a million years be able to store tracks (e.g. /proc). Besides the list of system devices it should also contain all the harddisk entries (i.e. Directories) that you have previously defined with the JukeboxMgr.

You cannot modify nor delete the system devices but you can do any operation on the harddisk entries. To do this, you select a name for the directory entry you are about to define and then select or type the physical location where this directory can be found. This location could actually be a mount point managed by the automounter.

The top part allows you to add/delete and modify categories. Remember categories are a logical grouping of devices that have similar capabilities like a CDROM and a CDWRITER.

First you select a name for the category and tell the system whether this category is removable and/or modifiable. The jukeboxMgr will use this information to manage the change of media effectively.

Once you have created a category you can select devices in the lower part and Add them to the category selected in the top part. The devices that have been added will show up below the corresponding category. You can add as many devices as you like.

Selecting a device and clicking on the delete button will remove the physical device from that category

Track management

Creating a volume


After you have defined the appropriate categories, it is time to define your first volume.

Creating a volume is as easy as selecting a name for it (spaces are allowed) and selecting, from the drop down combo box the category that applies for this volume.

The other fields (Capacity, max capacity, no of tracks, need to be commited, modifiable and mounted) are informational fields that are either calculated based on the actual content of the volume or derived from the underlying category.

Used capacity

The number of bytes that this volume actually occupies in KB

Max capacity

The maximal capacity of the media that are identified based on the category this volume corresponds to also in KB

No Of Tracks

As it says

Needs to be commited

There are changes in the database (changed tracknames for instance) that did not get pushed back to the volume yet. This field keeps the modifiability of the category into account. If the category is a CDROM, this field will never highlight

Removable

Whether or not the media in the category for this volume are removable or always present. Typically, CDROMS are removable but mounted directories (either through the system or through the automounter are not)

Modifiable

Checked if the media in the category are modibiable. CDROM's are not , but ZIP drives are

Mounted

Whether or not the medium that corresponds to this volume if physically accessible by the JukeboxMgr.



The ;New' button will only highlight if there is a name for the volume. Once the name is present, clicking New will actually create the volume. Besides creating the necessary data structures the JukeboxMgr will also scan the given directory (or medium) for tracks. It will detect all files ending with 'mp3' regardless of the case and try to detect what these tracks are based on the directory structure the tracks are found in (see Managing Albums and artist).

Based on that structure the JukeboxMgr will ask for confirmation of the directories it detects. It will also detect directories that are not album nor artist names. Since all albums MUST belong to an artist (even if that artist is a Various), when it detects an album and it does not know yet what the artist name is it will ask for a name. In that dialog box you will have the opportunity either to specify a real artist name or leave the field to the default value '-' in which case the JukeboxMgr will add the album to the special artist 'Various'

While scanning for tracks, a progress dialog will be shown.

After all scanning is performed successfully, the volume will be added to the track table which will be discussed in the next section

Track Table operations

The TrackTable is the main area of work in your MP3 database. It gives an overview of the content of ALL volumes, Artitst, Albums and tracks in the entire database and will allow you to perform a lot of operations on it.

What are the main informational fields in this part of the application.




Near the bottom of the window, you have the four checkboxes tagged with Volume, Artist, Album and Tracks. These checkboxes will allow you to remove information from the track table. A typical situation is that you wish only to see all the tracks (e.g to sort them and check for doubles). Then you uncheck Volume, Artist, Album and leave tracks checked. Then the TrackTable will be updated and will only show the tracks.

Just right to these four checkboxes is a blue label. The value in this label indicates the number of tracks in your entire track database.

Next, the green label, is the total playing time of all the tracks in the database.

The Orange label indicates whether the database is modified in memory and still needs to be saved. This can either be performed by hand (clicking right in the track table will popup a menu where you can save by hand) or automatically as specified in the AutoSave field (See Configuring the Jukebox Manager) 

The two buttons on the left allow you to collapse/Expand the tree in the track table.

Finally there is the track table itself. It contains detailed information of all tracks and will give you access to the main editing features of the JukeboxMgr. Double clicking a volume, album or track will bring you to the detailed editing windows of these items. Clicking right on an item in the track table will give you access to other operations (like saving, commiting, deleting, searching for doubles etc)

Some lines might be decorated by little icons. These icons are intended to give you a glance at the state of things. The following icons are currently available



This icon appears for these entries that have been and have not yet been commited to the volume (saving has nothing to do with it)



This icon appear for all entries if that entrie will be removed at the next available opportunity (i.e. Commit) and if the medium allows for physical changes. If that medium does not allow things to be changed the item will be filtered out from all other operations



This icon appears for all NON-track entries that have been modified but do NOT allow for changing.



This icon appears if the tag of that particular track is incomplete or missing.



Clicking right on entries in the track table will popup menus. It depends on the entry on which the click was performed, which items in the menu will be highlighted (From left to right, Album/Artist Menu, Track Menu, and Volume Menu)





Commit volume changes will (try to) commit ALL changes to ALL volumes. It will only do this if the volume is either mouted and/or modifiable.

Editing tracks

When you double click on a track entry, the top part of the main window will show the track editing widget.




This widget will allow you to change all aspects of the information of the track : Artist name, track name and track number (radio buttons on the right of the edit field show the names as they were extracted from the filename or the tag.

It also shows the track size and allows you to change to comment of the ID3 tag.

The Year checkbox (editable so you can actually type in any year) allows you also to select the decennium of the track

Clicking the Play button plays the track, clicking set will commit the changes to the internal (RAM) database and will cause the 'Mod' flag near the bottom of the track table to highlight. This means that the data is remembered but not yet saved in the database and not yet commited to the volume. So if you really want to change the tag you still need to commit the changes.

Editing albums






This widget will allow you to set the track sequence and other album related information. The set button wil again commit the changes. Albums can contain multiple CD's (like albums that consist of 2 or more CD's)

How do you set the sequence of the tracks ? It is a simple as clicking left or right on the listview's entries. Clicking left will increase only the track number, clicking right will increase the CD number and reset the track number.

First you press 'Reset' to reset the assignment process. Next you click RIGHT on the first track of the first CD (setting the values ot 01/01). Next you click LEFT for each consecutive track (setting the values to 01/02, 01/03 etc). Once the first CD is done you click again RIGHT on the first track of the SECOND CD and so on

If you make an error, pressing reset will reset everything. Actually nothing will be changed as long as you did not press SET.

Items that have been selected LEFT or RIGHT will be moved to the top of the listview so that the number of unordered tracks is reduced and you need to search through less items to find the one you need.

Using templates






When you click right on a track entry in the track table and select the "set as template" menu item, the above widget will be highlighted.

Each of the edit fields can be modified and are initially filled up with the tag information of the track you have clicked on.

On the right of the edit fields, you have 3 radio buttons that indicate what you want to do when applying the template. You have 3 choices. If you OVERRULE, you set the value of the tag of the selected entry to the corresponding template value regardless of the fact that the entry already has a value for it. If you choose to SET WHEN EMPTY, the value of the template will only be applied when the corresponding value of the selected track is empty or invalid. The LEAVE option finally will ignore the value of the template completely and and leave the corresponding track value as it is (even if it is invalid or missing)

Applying the template is performed by selecting tracks (multiple can be selected) and clicking right to select the 'Apply Template' menu entry.

Uploading the jukebox player

Overview






This window shows the global configuration parameters of your jukebox player. The window consists of 2 parts, one generic and one specific or the type of jukebox you are using. Currently only the PJB100 is supported.

The top part gives global information of the jukegbox. Currently you cannot change information yet.

PJB100 specific configuration

The bottom part allows you to define the parameters that will be used to generate the table of content of the jukebox. Several indices can be created. Tracks will only be copied once regardless of how many index entries you want to be created.

5 main set categories can be create.

Artists. A set is created for each artist that is found. Under this artist set 2 disc types can exists. The Album disc which means that for each album a different disc entry is created and a misc album that collects all the loose tracks (tracks not belonging to any album) for that artist are collected. If you check the Dup Album Tracks tag, each album track will also be added to the misc album so that the misc collection becomes a total overview of all tracks of that artist.

Year. You can create a special set called "_Yearbook_" in which tracks will be categorised according to the year in which the were released. If you check Dup Albums too, also the album tracks will be sorted this way in this set. If not it will only be the loose tracks. If a track has no valid year information it will be added in the disc "Unknown year"

Decennium. Is similar to year but the tracks will be grouped per decennium (60-ties, 70-ties etc) complete years (like 1985) will be converted to a decennium (80-ties). This is of course a crude algoritm but ...

Alpabet. Checking this entry will generate an alphabet listing based on the first character of the track and/or Album and/or artist (depending on the checked checkboxes) If you want to make a difference between Track called WHY and a artist called WHY you can optionally check the TAG entry checkbox which will add ALB, TRK or ART to the title.

The Categories checkbox finally, will categorize tracks according to the type of song (if available). Checking the optional subcheckboxes is similar to the equivalent once in the other categories.

Managing the jukebox content




This window contains all the information you need to be able to load and edit the jukebox' content in general.

The listview on the left side of the window reflects the actual content of the jukebox (when the JukeboxMgr is loaded) and will reflect the possible content of the jukebox (if all changes that have been requested are applied by clicking on the Apply Changes button)

The Listview in the lower right hand side of the window, contains the list of Volumes, Artists, Albums and tracks known in the entire database. In fact, the content is identicial to the track table except for the filtered out entries ( deleted entries).

You can add entries (you can also click on albums, artists or volumes) by selecting the relevant entries in the listview on the right and clicking the Add button. Entries that already exist will not be added again.

Alternatively you can also select a number of random albums (spinbox on the left) and/or a random set of loose tracks (tracks that do not belong to any album using the spinbox on the right) and click the Random button. Then the JukeboxMgr will add a random number of items to the jukebox' content. If a track/album exists anotherone will be selected.

Tracks that have been added to the jukebox but not yet have been transferred to the jukebox device are tagged with one of two possible icons



The track needs to be transferred but is in fact a link to a phyisical track



The track needs to be transferred and is a physical track

Transferring tracks

Once you have clicked the apply changes button, the JukeboxMgr will perform some calculations. It will first determine which volumes are going to provide tracks for the download.

Next it will order these volumes by increasing number of tracks that will be provided by that volume. This is because generally when you have selected things for download you are still there. So while you are there you can still change CDROMS. Since loading can really take long (Sometimes hours depending on the amount of items you have selected), the volumes that provide the largest number of tracks will be done while you are away.

The JukeboxMgr tries very hard to figure out if it can locate the volume automatically. If it determines that the needed volume is NOT mounted anywhere or the medium it is stored on is not moutable anywhere, it will ask you to insert the appropriate item. You can be sure that the JukeboxMgr checked all the alternative devices to make sure that none of them really contains the requested volume. (So if a CDROM and a CDWRITER are in the same category, the software has mounted and verified the CDROM as well as the CDWRITER for the presence of the requested volume)






The JukeboxMgr will show you what volume is needed and on which physical devices (i.e. Items that are belonging to the same category) are possible mount points for the medium. You then insert the proper medium, select the proper device (if there are alternatives like a CDROM and a CDWRITER) and click mount. Sometimes, depending on the speed of the device and others, the mount might fail. Click it again. If there is an error that says that the medium that has been detected is the wrong one, you probably did not insert the proper medium. The JukeboxMgr tries to identify the volume by trying to located the first known track.

Once that the mount is succesful, the JukeboxMgr will start downloading tracks






The above snapshot shows the status of the download job.

The entry 'Total tracks' shows how many tracks will be transferred for this job.

The entry Tracks Left shows how many items still need to be transferred in this job.

The Time left gives an indication of how long it will still take for the job to be complete.

The progress bar gives a visual feedback about the status of the complete job.

The Current Volume entry shows the volume that is currently being processed, while the Volume Tracks entry shows the number of tracks in this volume left to transfer. Again the progress bar gives a visual feedback of the volume's transfer state.

Next you have the current track that is being transferred and a visual feedback of the progress.

The Listview that follows next shows you the sequence in which the volumes will be processed. First column contains the total number of tracks for that volume. Next you have the volume's name and next either the name of the device category if the volume is still waiting to be processed or the device name on which the volume was or is being processed.

This listview gives you the opportunity to know how the JukeboxMgr will handle the job so you can prepare the media in advance and avoid having to wait for the mount prompt dialog to show. This way you can prepare in advance for a large job which saves you time.