iSmart Shuffle
By Daniel Blank
December 2007
-iTunes Smart Shuffle –
A weighted random shuffle for iTunes in Windows
Abstract
This program is a weighted random shuffle for iTunes. Using the play count and skip count of songs in your iTunes library, it establishes a baseline weight for each song. This weight is adjustable using the + / - buttons in the program.
Introduction
This program was envisioned after frustration with pure random shuffle, and the lack of an easy way to use the weighted random shuffle in iTunes. In iTunes, if you use the Party Shuffle, you can have it play higher rated songs more often. However, nobody sets their ratings by hand – it is very tedious – and ratings are really from 0-100, iTunes GUI offers 0-5. This program aims to solve that.
Methods
On first run, the user will need to clear the BPM values of the songs in their library. This is found in Options > Accurate Mode. Future development of the program will not need to use the BPM field in iTunes, but this version uses the BPM to store user votes (when they choose to play a song more or less often). Few people use the BPM field, and it is the field most suitable for this purpose. The program spawns a thread which goes through the entire library and sets the BPM to 50. 50 is the base for both the weights and the votes.
The next step for the first run is to run Reset Ratings > To Smart Shuffle First Run Setup. Here is where a lot of the magic of the program happens. All intensive loops are run in threads, such as these methods.
The program first runs through the library and calculates the average play and skip counts. Using this value, it runs through the library again and calculates the Standard Normal Distribution. This is expensive and there must be a better way to do this, but in the current implementation it is how it is done. Using the Standard Normal Distribution, it again runs through the library and computes the Standard Normal z-score for each track’s play and skip count (z is -3.0 to 3.0 where 0 is the mean, and 3.0 is 99.9 percentile.
Then it uses an estimation algorithm for Standard Normal Percentile distribution. This is borrowed from http://dotnetjunkies.com/WebLog/jekke/default.aspx . It is an approximation as a real value needs to be calculated by integration.
Public Shared Function StandardNormalCumulativeDistribution(ByVal x As Double) As Double
Dim Z As Double = 1 / Math.Sqrt(2.0 * System.Math.PI) * System.Math.Exp(-x ^ 2.0 / 2.0)
Dim p As Double = 0.2316419
Dim b1 As Double = 0.31938153
Dim b2 As Double = -0.356563782
Dim b3 As Double = 1.781477937
Dim b4 As Double = -1.821255978
Dim b5 As Double = 1.330274429
Dim t As Double = 1.0 / (1.0 + p * x)
Return 1.0 - Z * (b1 * t + b2 * t ^ 2.0 + b3 * t ^ 3.0 + b4 * t ^ 4.0 + b5 * t ^ 5.0)
End Function
What this returns, for 0 through 3.0, is a percentile 0.5 to .9999 or so. The way I use this is as follows. I convert a negative z score to positive (which is converted back to negative later). I then calculate the percentile from the StandardNormalCUmulativeDistribution function, subtract .5 from that result to have a range of 0 to .5, and then multiply by 2 (or -2 if a negative z-score) to have an approximate range of 0 to 1. If we are calculating play count weight, then the more the song is played the higher the rate. When calculating the skip weight, the more a song is skipped the lower the weight. The most a song weight can be affected in this way is +/- 20 (10 each for skip and play count). In the future I would like to include other metrics, such as artist and genre, in the weight as an option.
Once the song weights are calculated, the program populates an array of 20 upcoming songs, based on the rate. The algorithm for picking weighted random is as follows. It chooses a random number between 1 and 100. It then chooses a random track. If the random track has been played in the last 20 songs, or is an upcoming song, it chooses another song. (If the playlist has less than 40 songs, it will allow duplicates, otherwise it would just play in approximate rating order until no songs remain.) It subtracts its weight (rating) from the random number. If the number goes negative, then it plays the track. If it does not, it continues choosing random songs and subtracting their weight until a song causes the number to go negative. In this way, higher rated songs are more likely to cause the result to be negative, while a song with a rating of 0 will never be played. This is how the “Never play again” button works.
The array of 20 songs is about an hour of music, and manageable so that the user can choose to refresh the upcoming songs at any time, and skip ahead through the array quickly. It keeps a list of 20 played songs as well, to allow the user to go back. The reason for keeping an hour of music in either direction is so that a song can be replayed in the next hour. If it did not let any song which had been played, be played again, then over time, the currently playing song rating will decline due to the algorithm. Eventually, only the lowest rated songs will be left, which is not what we want.
The program chooses songs from the current playlist, if there is none it uses the main library. When a song ends it plays the next song in the upcoming list. It monitors some things by listening for COM events from iTunes, and other things it checks for by polling the player’s current state at ¾ second intervals.
Results
I found after using the program for some time that it was stable and worked as expected. There are features and improvements that can be made, but the program itself is usable as is.
The biggest problem is dealing with COM events from iTunes. If the program does not close properly, on the next run it will not hear COM events from iTunes unless iTunes is restarted. To work around this, instead of relying on COM events from iTunes, I poll iTunes for its status at regular intervals and make decisions based on this. If, however, iTunes has a popup message for the user, and my program tries to execute something, my program will crash if it cannot hear COM events from iTunes. This is because when iTunes has a popup box, it disallows COM commands, and unless my program can hear this, it will try to execute a command anyway and crash.