Stuff. Also, things.

Licensed listening based on the habits of pirates or lessons from sloppy item resolution

2012-06-29T10:07:00.001-07:00

So I spent Thursday and Friday a couple weeks ago at the Barcelona Music Hackday, part of the Sónar Music Festival. There were loads of excellent hacks (full list), including my own, Legalize It!. In this post I'm going to go into a bit more depth about the hack, lessons learned and teasers for things I might do.

Motivation

The core idea is a simple one – straightforward listening to things that are popular on Bittorrent (note that popular on Bittorrent is a slightly fuzzy concept, since Bittorrent is protocol for ad-hoc distribution, but we'll get back to that in a bit), without all the nastiness (and DNS blocking!) of looking at, say, The Pirate Bay's top music torrents (that's a proxy of tpb btw). And of course this removes any legal trouble that would be associated with gather and listening to music via those torrent charts.

How it all works

tl;dr - It's a torrent chart metadata-based content resolver, written in python and JS, you can fork the code.

Legalize It! has two parts, client and server. The server is a fairly simple pyramid webserver with two main tasks (it's deployed on heroku). The first involves fetching the torrent charts and resolving torrent release groups to legal streaming albums (Spotify, currently). This is simply a matter of fetching the daily torrent releasegroup chart from the Musicmetric API (full disclosure, they're my employer and I wrote most of the chart endpoints...) then walking through the top N items that look like albums and matching them to spotify albums. The matching is done through a fantastically naive string title + artist search on spotify's metadata API via the very useful Spotimeta python wrapper. This album resolution process has a simple web interface (if it returns an error try a refresh, heroku workers on the free tier sleep a bit too much), that I mostly built for testing, but can be quite useful without the commitment of installing the Spotify App. In addition to the human readable page, you can get the response back as JSON, which is handy on the client side.

The second job for the server is necessary to help select which songs from the top albums we'll be listening to. The final Spotify app will only select on song per album, so users can get a taste of every album in the top N without having to listen to N complete albums. But this should be done with care and grace to enhance the listening experience. Thankfully, with the assistance of The Echo Nest's searchable audio summary song-level features this is quite easy. Using their API we can search for a song by title and artist name (see a pattern of terrible id matching forming? What can I say, it's a hack.) and get back a set of descriptors that looks like this:

Once we have the set of song-level descriptors for every song in the neighbouring albums, it's simply a matter of minimizing the step size. I've done a bit of work on playlists before, and this seemed like a reasonable approach. While there are a number of approaches to step-size minimization, for this particular application we're doing a greedy sort of optimization that goes something like:

Select the song a in album A and the song b in album B, such that for a given audio descriptor (we'll use dancibility by default, but it could just as easily be a different measure, eg. tempo, loudness) the absolute value of the descriptor of a less the descriptor of b is minimized. That is to say we're looking for the song-pair from these two albums that are closest in terms of whatever descriptor is being used.
Select song c from album C such that the absolute difference from it's audio descriptor to song b's is similarly minimized.
Repeat (2) for remaining albums, using the last chosen song against the next album.

It's worth noting that this is not the globally optimal shortest path from the first album to the last album, but it comes with a tremendous advantage -- the first two songs are selected without any need to deal with the rest of the playlist, which can be worked on overtime. This allows for pseudo real time playlist creation, since we just need to know the next song before the current one is done playing.

To facilitate this algorithm the server has a hook that performs step (1) or (2) on on either a pair of albums or a song and album (specified as spotify URIs) can be accessed via a url that looks like

http://legalize-it.herokuapp.com/paired/[spotify URI of track or album]/[spotify URI of album]

For example, http://legalize-it.herokuapp.com/paired/spotify:album:6LBiuhK7PZKjVXyMfPxPoh/spotify:album:5nVUqrdkEMlWTm9sqjrYBt which returns a bit of JSON showing the closest (by dancability) song pair between My Beautiful Dark Twisted Fantasy by Kanye West and Up All Night by One Direction. This method also supports using any other audio summary feature being used for distance by adding it to the end of the URI. For example http://legalize-it.herokuapp.com/paired/spotify:album:6LBiuhK7PZKjVXyMfPxPoh/spotify:album:5nVUqrdkEMlWTm9sqjrYBt/tempo find the closest song pair from the same two records, but by tempo rather than by dancebility.

These two features are laced together in the client, which is a simple spotify app, that's basically just a small bit of jQuery that grabs the list of the top 25 albums then asynchronously gets the selected track for each album updating the playlist as each track is selected.

The Legalize It! app, as demo'd at the Music Hack Day

If you'd like to install the App, here are some instructions for installation in developer mode. I'll be cleaning up the UI to conform to Spotify's app guidelines and submitting the app to their App finder thing, but that will take a while.

Going Forward

While this is mostly complete for what it does there are a number of feature adds that I'll be slowly dealing with as time goes on. Highest among these is taking the same idea and applying it to different resolvers (maybe use tomahawk rather than picking one...). There are also some smaller feature adds to the app, things like autoplay once the first track has loaded and switching between different echonest summary features or musicmetric charts (eg. P2P release groups by acceleration).

Also, thanks very much to Spotify, who awarded me a prize for my hack!

So what do you think? Should we care about the taste of a bunch of peers on Bittorrent? Or am I doing it wrong?

Some brief thoughts on the London homebrew scene

2012-05-22T11:58:00.001-07:00

Not that homebrew, the beer one.

A few weeks back I was asked by The Strongroom bar to write up a few words on homebrewing in London for the lit at their London Beer Festival. Unfortunately, it had to be cut to make room for some write-ups from breweries added at the last minute, but it occurs to me that it might be interesting to others, so I'm throwing it up here. Without further ado:

---
The Rise and Rise of Homebrewing

Homebrewing in London is enjoying something of a comeback. Over the last few years, along with the rise (some might say return) of craft and artisanal beer in London, amateur and hobby brewing has been growing mightily. Like the current trends in professional craft brewing, these new hobbyists are focusing on quality and experimentation over matters of the bottom line. In London, this DIY brewing movement is focused in two clubs, each on different sides of the city and each with a different emphasis.

Meeting in East London is the London Amateur Brewers (LAB), of which I am a member. Until its recent closure, LAB met monthly at The Wenlock Arms in Hackney (for current meeting locations, consult the website). LAB has an open structure, operating without formal dues and a very small set of officers. The meetings consist of a short technical talk on some aspect of the brewing process or overview of a particular style of beer, followed by tastings of member’s beers. A typical meeting will involve tasting 8-12 beers, over an hour to hour and a half. These beers are typically very diverse in style, and at a single meeting you can encounter everything from a Best bitter to a new world IPA to a Belgian Saison. In addition to its regular meetings, LAB holds homebrewing competitions and festivals, the last of which was on 12 November 2011 in Wimbledon.

Across London in Durden Park, is the eponymous Durden Park Beer Circle. The Beer Circle is a more formal group then LAB, having both a formal membership process and thematic meetings, where the homebrew tastings are all keeping to a particular style, which will change from month to month. Additionally this group has something of a focus on understanding and preserving the historical beers of Britain. Over the years they have sought, archived, and tested many accurate recreations of style of beer long out of fashion. These recipes have been gathered into a book that group puts out, “Old British Beers and How To Make Them.” This book is an excellent resource in any homebrewer’s library. In fact, you can taste some (slightly modified versions) of the recipes in this book in action at some of London’s fine craft brewers, where it has served as inspiration for novel interpretations of classical local styles, most especially Porters and Stouts.

Think you might want to give homebrewing a try? It’s easier than you might think. Come to a meeting (if you aren’t in London, the Craft Brewers Association can point you in the right direction) or just simply give it a try in your kitchen. Aside from the previously linked webistes, information to get you started can be found at How to Brew, Homebrewing Stackexchange, The Homebrewer's Association, and Jim's Beer Kit, among others. Good luck and happy brewing!

Upcoming talks and travels

2011-10-17T02:37:00.001-07:00

I'm going to be up to a number of things that may be of interest to the readers of this (rather sparse) blog.

Quick summary

Co-chairing The Workshop on Music Recommendation and Discovery (WOMRAD) Oct 23rd, Chicago
Presenting some new Musicmetric API features during ISMIR, Oct 24-28 (demo on the 28th), Miami
Giving a talk about beer and critical tasting at Ignite London 5, Nov 8th, London

WOMRAD

Our workshop (other co-chairs are Amélie Anglade, Òscar Celma, Paul Lamere, and Brian McFee) will cover a diverse array of approaches and angles for music recommendation and discovery. The workshop is runs the full and is part of RecSys 2011, though I sadly can't stay for most of the conference aside from our workshop (see below). It should prove to be an interesting day of research. Are you planning on attending? Let us know.

ISMIR

I'll be dashing off from Chicago to Miami to attend ISMIR, and to present some new (not-yet-released) API features from Musicmetric. While things aren't quite live yet, I can say that in addition to our artist based endpoints, we'll be offering track-based endpoints soon as well, and aligning them with an I-bet-you-can-guess-which large public audio feature test set. More detail on this one to come.

Ignite London 5

Ignite is a series of lightning talks that have taken place in cities all over the world, unified not by a common theme, but a common format: all talks last five minutes, contain 20 slides, and the slides are automatically advanced every 15 seconds. The matching ethos of this structure is perhaps best seen in the Ignite slogan, "Enlighten us, but make it quick." I'll be speaking about beer, style and critical tasting in a talk titled "Ale or Lager and Other False Choices." Here's a brief description:

In a word, my talk is about beer. In a few more words, the driving narrative behind the talk is a crash course in beer styles and more generally, critical tasting. After an extraordinarily brief description of beer, broad ideas of style and the critical tasting process, the core of the talk will be made up of live lightning tastes of commercial examples of various styles of beer (one slide per style, 12 styles covered with one commercial example each). For coherence these tasting slides will grouped into broader styles, with aim toward width, rather than depth of coverage. The styles will be approximately based on those from the BJCP and the Brewer's Association.

I still haven't sorted out the exact spread of beers or how they will be grouped, though I'm leaning toward something simple and obviously ingredient tied (something like -- Lagers, ales:yeast driven, ales:malt forward, ales:hop heavy, with 3 beers each from a different recognized style in each). If anyone has any thoughts about style divisions or specific examples do let me know. If you'd like to go to ignite (and you know you would) the tickets will be available over this way later this week.

So, lots of things going on. Plus there's this other thing I've been working on.

Right, back to it.

A SXSW panel proposal - The Wisdom of Thieves: Meaning in P2P Behavior

2011-08-15T12:09:00.000-07:00

So I've submitted a proposal for SXSW interactive 2012 entitled "The Wisdom of Thieves: Meaning in P2P Behavior". If you're the sort of person that might be interested in that sort of thing you can comment and/or vote on it over here. The talk will basically be a tour of all the fun and exciting ways you can use BitTorrent data to make better (mostly music, but also TV, film, and app-store type things) applications, with data sources like this. Here's the abstract and questions:

The act of piracy is typically viewed as devaluing content - the track that wasn’t streamed, the video game that wasn’t purchased. However, peer-to-peer networks of piracy are rich descriptions of fans who are interested enough to find content. By observing these descriptions, artists can better understand their fan base; recommendation and discovery can be better tuned. In this talk we’ll explore the similarities between BitTorrent downloads and a number of other means of online interaction, such as likes, mentions, and scrobbles. We’ll show how interactions vary between popular artists and works versus those found in the long tail, whether they’re emerging artists or niche films. Our audience will leave with a utility belt of tools to leverage data about and around peer-to-peer sharing of music and video. This talk will use data available via the Semetric API and open source Python scripts, freely available for download prior to the talk.

Questions Answered:

How is peer-to-peer activity different from communities on Facebook, Twitter or Spotify?
Can you use location data and a torrent network to optimize a tour schedule?
Which countries should I syndicate my TV show in?
How can you use co-occurrence in piracy to recommend content?
Why should I consider the behavior of roving bands of thieves?

Also, my colleagues have panel proposals for SXSW music and film as well, go check them out here and here.

Doing ridiculous things with natural language processing

2011-05-16T04:50:00.000-07:00

Over this past weekend before last, while I jealously followed from afar the SF musichackday that I was unable to attend as I'm awaiting the result of a visa application, I started mucking about with the beta musicmetric api (full disclosure, they are my employer), in particular the sentiment analyzer.

So the first thing I put together was a bit of python to fetch the content of a tweet and use the mm_api to determine it's tone. This can be done quite simply (full source):

Which gives a number between 1 and 5, with 1 indicating the text is 'very negative' and 5 indicating the text is 'very positive' (gory details of the sentiment analyzer). While the sentiment analyzer is trained for larger chunks of text (500 word album/movie reviews and that sort of thing) it in fact does fairly well with tweets (though sarcasm is its downfall). So I thought I'd do something a bit silly and built a 'flamewar detector and troll finder' for conversations on twitter.

I've called the initial command line tool firealarm.

To gather the conversations, I'm just piggybacking on @jwheare's great tool exquisite tweets. Once a conversation is archived over on exquisite tweets, the cli can be pointed to it via the conversation's url. Each tweet in the conversation is pushed through the sentiment analyzer; the simple mean (µ) of all the sentiment scores is then dubiously used to determine if the conversation is a flamewar. If the sentiment is generally negative (µ < 3) it's a flamewar, if it's generally positive (µ>3) it's not a flamewar, and if it is exactly neutral (µ ==3) it's declared a tossup. The troll finder is equally straight forward (and equally dubious!). Across the sequence of tweets, the author of the tweet with the highest magnitude negative delta sentiment proceeding it is considered the troll. In the case of a tie the first occurance wins.

Here's an example (note that the linked-to example is a nasty, nasty flame war. If you offend easily, might want to skip it. Also, obviously, views expressed are not mine, etc.):

$ python firealarm.py http://www.exquisitetweets.com/collection/RodBegbie/402

This generates the following output:

A plot of the sentiments, with the maximum negative delta in red, looks like this:

A quick read of the tweets and you can see that the actual sentiment of the tweets is a bit more negative overall then the analyzer output, but this is good enough for binary classification and a fairly reasonable troll ID mechanism, albeit fairly naive.

The code is over at github if you want to have a look or run it yourself. If you want to run it, you'll need a musicmetric api key which takes about 30 seconds to get (apply here). Eventually I'm going to turn this into a web app, and when that happens I'll let everybody know. Also if you happen to find any really bad mislabels, let us know, as it helps us tune up our process.

Have fun being algorythmically judgemental!

Free Beer: A Plea for Open Data [About Beer]

2011-04-05T16:42:00.000-07:00

(I wrote most of this right after my viva, but got a bit sidetracked...)

Hey look, my first blog post about beer (or at least beer metadata).

So ~~yesterday~~ a few weeks ago, Tim Cowlishaw (@mistertim ) stated this on twitter:

To which I replied with this :

(Scraping is a way to get the info a human reads, say on a website into a format a computer program can read. More on why I'd want to do that in a minute...)

Which was followed by what I thought was a reasonable request:

Now at this point I had figured that was the end of it. Both rate beer or beer advocate ignored my requests for data a year or so ago, I was expecting the same this time. However, beer advocate responded via twitter:

(Note that the link to the tweet no longer resolves, because beeradvocate decided to delete this tweet a couple hours later. The screen capture was taken from my twitter client just after the deletion...)

Now, I hadn't been prepared for such knee-jerk nastiness regarding seemingly reasonable data requests and neither had Tim as he quickly push out this series of messages:

While I and others pushed some similar responses, Tim's summarize things really well: boo, disdain, technical critique. (after this both Tim and I appeared to be blocked from following beeradvocate...)

The crux of all this is that I (and it would appear others, but from here I speak only for myself) would love to have access to structured data (as a service or, better yet, as documents) about beer and the people who drink it.

I'd love to build browser-based applications that do cross-domain recommendation of say beer and music. But in order to do that I'd need data about people's taste, in beer and music. Lots of options to work with in the music domain. But beer? Machine readable beer data is harder to find.

Both ratebeer and beer advocate have a great deal of this data, it's just not (openly) machine readable. In ratebeer's case this is entirely crowdsourced and for beer advocate this is true for their community pages. There's a compelling case that crowdsourced data should be as open as possible, given that the data itself comes from the public at large. But beyond the moral case, opening your data means that the wide-world of evening and weekend software developers/architects/designers/whatevers (many have the same job during the day) will expand what is possible a site's data in a way that will benefit said site (like my half baked idea above). This, in essence, is the commercial argument for supporting open data and has been shown to be extremely effective in other domains (say, to pick one at random, music). And there is a simply massive spread of open data apis (again, both service and document) but barely any covering data about my favourite topic that isn't music, beer. So what do you say ratebeer or beeradvocate? How about some nice strucutured data?

note: I should mention that there are a couple sites that are beer related and open: untappd and beerspotr. Both are good sites, though neither is quite to the point of hitting critical mass in terms of data coverage and usefulness just yet. Either might at some point in the future, but ratebeer and beeradvocate already have, the data just isn't accessable.

Viva passed, corrections approved, blog barely updated...

2011-04-01T10:57:00.000-07:00

The last couple months have proven me to be a terrible blogger, as I haven't posted at all.

Anyway, that aside, I'm pleased to announce that I have passed my viva with minor corrections (back on march 2nd) and as of about an hour ago, had my submitted corrections approved, which means I'm totally done!

Hoorah!

So before I run off for a bit of celebratory drinking, I thought I'd post the soft copy in the the series of tubes (here's the full pdf) and here is a brief chapter-by-chapter summary:

Chapter 1: Introduction. We present the set of problems this thesis will address, through a discussion of relevant contexts, including changing patterns in music consumption and listening. The core terms are defined. Constraints imposed on this work are laid out along with our aims. Finally, we provide this outline to expose the structure of the document itself.
Chapter 2: Playlists and Program Direction. We survey the state of the art in playlist tools and playlist generation. A framework for types of playlists is presented. We then give a brief history of playlist creation. This is followed by a discussion of music similarity, the current state of the art and how playlist generation depends on music similarity. The re- mainder of the chapter covers a representative survey of all things playlist. This includes commercially available tools to make and manage playlists, research into playlist generation and analysis of playlists from a selection of available playlist generators. Having reviewed existing tools and gen- eration methods, we aim to demonstrate that a better understanding of song-to-song relationships than currently exists is a necessary underpin- ning for a robust playlist generation system, and this motivates much of the work in this thesis.
Chapter 3: Multimodal Social Network Analysis. We present an exten- sive analysis of a sample of a social network of musicians. First we analyse the network sample using standard complex network techniques to verify that it has similar properties to other web-derived complex networks. We then compute content-based pairwise dissimilarity values using the musical data associated with the network sample, and the relationship between those content-based distances and distances from network the- ory are explored. Following this exploration, hybrid graphs and distance measures are constructed and used to examine the community structure of the artist network. We close the chapter by presenting the results of these investigations and consider the recommendation and discovery applications these hybrid measures improve.
Chapter 4: Steerable Optimizing Self-Organized Radio. Using request radio shows as a base interactive model, we present the Steerable Opti- mizing Self-Organized Radio system as a prototypical music recommender system along side robust automatic playlist generation. This work builds directly on the hybrid models of similarity described in Chapter 3 through the creation of a web-based radio system that interacts with current lis- teners through the selection of periodic requests songs from a pool of nominees. We describe the interactive model behind the request system. The system itself is then described in detail. We detail the evaluation process, though note that the inability to rigorously compare playlists creates some difficulty for a complete study.
Chapter 5: A Method to Describe and Compare Playlists. In this chapter we survey current means of evaluating playlists. We present a means of comparing playlists in a reduced dimensional space through the use of aggregated tag clouds and topic models. To evaluate the fitness of this measure, we perform prototypical retrieval tasks on playlists taken from radio station logs gathered from Radio Paradise and Yes.com, using tags from Last.fm with the result showing better than random performance when using the query playlist’s station as ground truth, while failing to do so when using time of day as ground truth. We then discuss possible applications for this measurement technique as well as ways it might be improved.
Chapter 6: Conclusions. We discuss the findings of this thesis in their to- tality. After summarizing the conclusions we discuss possible future work and directions implied by these findings.

Enjoy!

(Also, if you find any deep hiding typos, I'd love to know about them. Not sending it to the printer/binder till Monday...)

woooo!

2010-12-21T16:29:00.000-08:00

I finally submitted my phd thesis. I'll post some bits of it over the next few weeks, and the whole thing after my viva. To start, here's the abstract:

---

It is not hyperbole to note that a revolution has occurred in the way that we as a society distribute data and information. This revolution has come about through the confluence of Web-related technologies and the approaching- universal adoption of internet connectivity. Add to this mix the normalised use of lossy compression in digital music and the uptick in digital music download and streaming services; the result is an environment where nearly anyone can listen to nearly any piece of music nearly anywhere. This is in many respects the pinnacle in music access and availability. Yet, a listener is now faced with a dilemma of choice. Without being familiar with the ever-expanding millions of songs available, how does a listener know what to listen to? If a near-complete collection of recorded music is available what does one listen to next? While the world of music distribution underwent a revolution, the ubiquitous access and availability it created brought new problems in recommendation and discovery.

In this thesis, a solution to these problems of recommendation and discovery is presented. We begin with an introduction to the core concepts around the playlist (i.e. sequential ordering of musical works). Next, we examine the history of the playlist as a recommendation technique, starting from before the invention of audio recording and moving through to modern automatic methods. This leads to an awareness that the creation of suitable playlists requires a high degree of knowledge of the relation between songs in a collection (e.g. song similarity). To better inform our base of knowledge of the relationships between songs we explore the use of social network analysis in combination with content-based music information retrieval. In an effort to show the promise of this more complex relational space, a fully automatic interactive radio system is proposed, using audio-content and social network data as a backbone. The implementation of the system is detailed. The creation of this system presents another problem in the area of evaluation. To that end, a novel distance metric between playlists is specified and tested. This distance method is then applied as a mean of evaluation to our interactive radio system. We then conclude with a discussion of what has been shown and what future work remains.

womrad live blog part the last

2010-09-26T08:42:00.000-07:00

last session: Long tail stuff:

Music Recommendation in the Personal Long Tail: Using a Social-based Analysis of a User's Long-Tailed Listening Behavior.

Kibeom Lee (presenting), Woon Seung Yeo and Kyogu Lee

focusing on popularity bias - referencing oscar's thesis work (Help! I'm stuck in the head)
Goal: keep the awesome of collaborative filtering but sort out popularity bias
the mystery of unpopular but 'loved' songs on last.fm -- shouldn't loved songs be played frequently... perhaps an area of music the user likes but doesn't venture very far into
'My tail is your head' - find the users who have a 'head' that overlaps with your 'tail' to draw recs from
personal story about how this idea came about -- one person's popularity bias is another person's novel rec.
refs oscar and paul's ISMIR 07 rec tutorial - this system is geared toward the top half of the user type pyramid
scraped last.fm to get more tracks per user (API gives 50/user scrape gives 500)
lots of tracks (about 9million)
eval by asking users how things worked out comparing recs from proposed algor v. trad model rate; used a 1-5 rating scale
promo'd the website in various ways, but not too much response
but, the limited response did show some improvement over traditional approach
overall - some improvement, much potential

Q how many users?: see above

Q so were your recs in the global head?:

sorta, mostly in the midsection

Music Recommendation and the Long Tail.

Mark Levy (presenting) and Klaas Bosteels

an overview of lit showing various rec bias especially the idea of positive feedback reinforcing the head (not this kind of bias though)
this work looks at 7 billion scrobbles all scrobbles from Jan - Mar this year (holy crap, that's some scale)
recs just from the last.fm radio
how do you define the long tail? use a fixed ref of overall artist ranks (number of listeners from last) + a fit model ~50-60k artists in the 'head'
looked at rec radio, non-rec radio, all music
the last.fm radio has less head bias then general listening, but only just
used an experimental cohort of listeners: new, active, but not insane spamming amounts of scrobbling. two subsets : radio users and not so much
this shows very little difference in the non-radio long tail listening among those who use last.fm radio v. those who don't
but: perhaps there's some demographic trouble
so split radio users into high users and low users
still no tail bias to speak of
perhaps from the fact that real systems only rec new tracks, mitigating reinforcement
so: built a simple item-based rec which limited candidates to the 'play direct-from-artist' scheme, not allowed to give artists with more than 10000 fans
deployed on playground.last.fm
eval based on a sample of the last.fm user traffic
effectively pushes curve out another order of magnitude
try online
[me: this is great!]

Q Do you see a problem, in terms of scholarship, with the fact that in practice you have access to all this data and the public does not?

well, hrm. how about being an intern

Q Does this make better recs?

Better, eh, interesting sure.

And WOMRAD done. feedback is elicited

afternoon papers

2010-09-26T07:13:00.000-07:00

content-based stuff now:

Content-based music recommendation based on user preference examples.

Dmitry Bogdanov, Martín Haro, Ferdinand Fuhrmann, Emilia Gómez and Perfecto Herrera

Dmitry presenting

Sim is not rec. need similarity
can we improve content based rec by merging pref data?
gmm + pref model
process:

ask user for small set of tracks that specify the user's preference by example
get bag of frames on these
SVMs to get sematics (probablistic)
in this semantic space, search for tracks

can search in a variety of ways (use of Pearson's correlation is taken from prev work)
for eval compare our method to a bunch of existing methods, content-based , contextual, random
some users did a test get pref set (varies form 19 to 178 tracks for a user) this takes a long time
get lots of tracks from all the methods, shuffle, stick in front of user ask lots of Qs per track
created three categories based on the evals: Hits, trusts, fails

Hits -user likes, is new
trusts - user likes, is not new
fail - no to all
unclear - the rest (18%)

A good system should provide many hits and some trusts avoiding fails
in the results, last.fm (via api) is very good for hits and trusts
everyone else was bad at trusts
the new method was best for non-last.fm with hits, but last.fm is different drawing set of music so they're better
proposed semantics offer an improvement over pure timbral features
but still inferior to industrial approaches, though this proposed work improves considerably, a good way to cold start perhaps

Q (oscar) I dont' understand the last.fm? why didn't you use for sim?

we tried, couldn't get enough info

(oscar follow up) low trust on the content, do you think it's tied to a lack of transparency?

maybe, but our definition of trust just meant user likes and knows.

Q() was the SEM-ALL about finding songs that are close to any or all?

any

UPDATE (~5pm):

Applying Constrained Clustering for Active Exploration of Music Collections.

Pedro Mercado and Hanna Lukashevich

Hannah is presenting

clustering can help you swim in the sea of data
users can fix incorrect clusters, positive feedback
system diagram:

similarity can be given considered as a graph, then you can do random walks, calc eigen values etc.
but, what if this user doesn't care about somethings? User pref based feature selection.
in the given space, you can then find distance (paper uses Pearson's but other dist could be used)
contraint the space (tricky math, see paper...)
eval: used the MIREX 04content description data
constraints from genre labels
using test train as an example: what's in contraint space, what isn't
mutual information, something else I didn't catch
some graphs showing that there's more awesome with presented method
when looking at outliers, things are less clear but still seem positive
[graphs are page 6 of the pdf, have a look for details]
to wrap up: ML approaches can improve recs at least with our simulated user...
our clustering methods are speedy, though scale is tricky but since our matrix sparse should be doable
Way better than random constraints
future work: stick constraints in feature selector, we did this, to appear in ICML, gives significant imporvement, but causes some trouble, read paper for detail [excellent ICML tease...]

-- coffee and demos now...

WOMRAD Afternoon live blog

2010-09-26T06:10:00.000-07:00

Afternoon live blog for WOMRAD. Intro in first post.

Afternoon session.

1500: Industrial panel

panelists:

Òscar Celma (BMAT, Spain), moderator
Tom Butcher (Microsoft Bing, US)
Mark Levy (last.fm, UK)
Michael S. Papish (Rovi Corporation) subing for Henri-Pierre Mousset (Yozik, France)
Gilad Shlang (Meemix, Israel)

Q (asked by OC): Do we need recommenders anymore? Are they relevant? (SFMusicTech quote about music rec only needed for people w/o friends)

TB: still valid, but personally more interested in discovery now...

ML: don't sell things, but tremendous effort in this direction, important to users, builds trust. Plus we compliment not replace social connections

MP: no need to draw lines, reinforces complimentary service idea. Many users may not need them, but perhaps that's not who these systems are for

GS: What's wrong with not having friends? Also, a tight group of friends may not have discovery, as a group. The removal of place opens more possibility to access long tail or different parts of the head. Perhaps more personalization than rec, but this a fine line

MP: the opposition of the individual v. group. If you listen to music without a community you loss the social experience.

GS: fair enough. some points about individual optimization of education. Group important, but also personal growth.

Q (asked by OC): Netflix prize. What is a good recommendation (in music)? How do we evaluate (in music)?

GS: a good rec will get people interested. wow factor. acknowledge you and surprise you at same time. music is short , which makes it easier to tune a rec profile. sharing implies liking, that's useful. tagging; more tags=more popular

(...small aside...)

ML: we run controlled experiments. quietly divide users to test different methods. Netflix fails in that it evaluates with data that's already been seen not new data.

MP: good rec means different things at different times. gives an example of a good rec that is not interesting: an artist you've previously bought releases a new album. not interesting but good. this would be bad for radio.

TB: in industry there are many ways to test. more purchase is different than more enjoyment.

ML: we at last would love for some theory to be developed for rec based on user logs.

OC: more data please

ML: you can always ask...

(this goes back and forth)

MP: yet there will never be good data. sparse data is hard, but makes your a better human (eat your green)

Q (OC) discuss user interfaces, user exerience etc. :

TB: pandora is a winner, don't ignore the interaction

ML: thesixtyone is great. interface v. playful good long tail, would love for a last.fm interaction, but discussing issues

MP: name checks Paul lamere, who he cites saying thesixtyone is an exotic rec, but MP thinks this is the way people normally use music, we should work to have systems that act like this. Need a toolbelt not one ubiquitous tool

GS:we tackle similar things. in B2B you need systems that complete clients' existing systems. If you over rec, you can scare people off, social dynamics

MP: think about the inverse rec - what should you not rec? Also from a UX stand point, to build trust, change recs over time. General to personal as a user interacts with a system

GS: different recs can be very personal

ML: last.fm takes a sort of opposite

-crosstalk-

MP: this is possible from last.fm's transparency

Q (OC) What do you want solved?

TB: we're hiring. Also, algorithms must scale or they're scalable

ML: how to merged datasources? how to use human-to-human recs

MP: see my keynote, exploit user psych, What are good Qs to ask users to build profiles

GS: more info for recs, params in audiofiles. map user params to extractable params Moving techniques to non-western musics. What about china and india? We should be serve them.

MP: is the sonic data really the key? I don't think so, too much effort in this direction sim is different than rec

GS: but sim is a good start

Q(OC) Do you use content-based features (y/n please)? ISMIR fun, glass ceiling, do you follow this work in academics, do you think it's solved?

GS: Yes. see last discussion core to our business, vital to start a relationship, move to social and such over time.

OC: what sorts of things do you use

GS: of things. 10 (does not list). aggressiveness very important.

ML: I come from the MIR community. we do content-based ID, have tried to intro content-based stuff and it's never been successful. But our hearts are in it. We have enough users that cold start doesn't matter. auto-tagging would be sweet though for the holes in our social data (musicological tags for instance). maybe youtube

TB: yes for the most part to ML's comments. content-based is too costly, tags and metadata are super effective

GS: what about the new company, that doesn't have lots of data. is you're just getting into the game. these people need results. can't tell them go gather data for a year and we'll sort it out

MP: item to item is very different than personalized

ML: check that P2P paper from ismir

Hannah from Franhauffer: we have clients (like film producers) no data,what then?

MP: exactly.

(Eugenio Tacchini): GS is the DNA all of it? really?

GS: no, not really. music DNA for rich space, but still need personalized info

Q (OC) if you were to hire a researcher (aside: research cannot program) what kind skills do you want (not resume skills, fancy skills)?

TB: domain experience, audio music, computer vision, breadth better than depth. production coding skill in some language

ML: we're hiring as well. If you don't want to code, probably won't work. CS skills really important. Big database skills. hadoop win. strong C++ and research also python. data and viz as well.

MP: we are hiring as well. growing r & d group. we have offices all over. we like building things. though we have room for research. we like solving problems. again broad. can you pivot. don't need a PhD to be useful.

GS: we're also hiring (that's 4 for 4) we're a start up. data analysis and mining. core CS + creative skills, willing to sweat.

OC: perhaps also adoptability

GS: yes. you're there to invent. plus we're in Tel Aviv and that's sweet

Q (claudio): What is the relationship your company has with musicians are they just a commodity?

OC: our missing speaker (Henri) does this

GS: I spoke with him, he thinks: for young musicians it's hard to reach your audience.

OC: BMAT does this with jamendo. when they type in 'Michael Jackson' what do you do?

MP: but don't sell recsys as a way to push new artists only. In a certain context, ie. neg search, but careful. Don't exploit users or artists

GS: but the state of the art pushes new bits

(from audience) What about piracy?

GS: it's not good.

TB: there are 2 view points: piracy increases consumption. otherside: do we now that?

OC: now we're over time sorry.

---

in light of the near transcript I just typed, I'm starting a new post for the afternoon talks.

Updated (5:33) : corrected questioner ID

A womrad live blog

2010-09-26T00:07:00.000-07:00

I'm in Barcelona today for the Workshop on Music Recommendation and Discovery (WOMRAD). The theme is 'Is Music Recommendation Broken? How Can We Fix It?'

I'm giving a talk at 11am ( in about 2 hours ) and I'll be doing some (mostly) live updates about the program...

Update (10:05am):

Keynote: "The Dark Art: Is Music Recommendation Science a Science?"

UPDATE ( 28 Sept 2010, 11:52am): Michael has posted the slides to his talk.

The view from outside, as his industrial has used and observed recs
Been there since the beginning (which appears to be about 2000)
Recommenders must combine humans and machines
understand both content and listeners, transparency, embrace emosocio aspects, optimize trust
What is science? Must be falsifiable (Popper) or Solvable, reproducible puzzles (gah, missed name)
Puzzle - understand the listeners preferences -- foundations (ISMIR 2001 resolution) - testable reusable
Lots of metrics though (too many?) (do we need a metric for metrics?)
MIREX (summary of AMS task) (haha it's automated, tell that to andy and mert) - very acoustically focused, not exactly recommendation similarity != recommendation
use of statistical measures across datasets e.g. Netflix prize -- but what about discovery? -- Netflix produces better numbers but does it produce better recommendations?
More holistic measures -- survey users about trust and satisfaction (Swearingen & Sinha) -- may miss UI issues -- practical 'business' metrics -- bottomline measurements -- does this remove the science?
appreciated history of MIR (from a rec POV) will stick pic here -- currently hitting 'Wall of Good Recs' since recs don't suck it's no harder to test
easy to test for bad recs -- hard to test for good recs
What if the emerging problems (like UI and trust) are no longer measurable
Is user preference too variable and unstable to be useful?
from science to art?
2 options:
1: focus on unsolved MIR: better encoding of preference (more socio-cultural research)
What are the limits of the avg listener (hey it's our playlist survey!)--playlist turing tests, understand artist v. album v. tracks -- can we build tools/games to expand this
listener profile -- can you quantify the sonic v. social preference -- add relevance layers to search and retrieval
2: adjourn to the Beach
Questions:
Mark Levy: Do you think you're too embarrassed about good engineering? What about controlled experiments by people like google/last? -- Move from science to engineering (this confuses me slightly ISMIR has alway been Engineering not Pure Science) It is fruitful but is it science.
Claudio: Can you speak a bit about your experience combining human knowledge vs. algorithms --- yes. what do you do with human knowledge? it's tricky. look for the ideal rec experience - sit around with your friends and play records: how do you scale that in a system? It's not about classification - humans are good at putting things together - train people to be qualitative assessors
Oscar: Since you used to be in college radio, how do you think this experience could inform playlist? Do you use playlisters? Well only a 1.5yr experience, but made me think about the groups of listeners. Name checks John Peel. What about presentation - In terms of what rovi does: Minimally we can stop making bad playlists: gives example then breaks - v. hard to differentiate btwn good - v. good - excellent
Me: what about bypassing order by selecting good sets:
(Eugenio Tacchini): how much is the expert transparency necessary? yes give justification but need to avoid the feeling of stereotyping, weird vague directions, not just look at this user but look at this part of this user.
tom butcher: Is music rec really a unique snowflake? - Every domain is unique. -- One thing: a bad user rec in music costs 2 minutes, a bad film rec costs you 2hrs music has a lower penalty cost for bad recs. Also diff in features will sonic features get you to pref, prob not in music [I think this is a think which may improve...]

(update 2 10:31am)

session 1:

Time Dependency

Rocking around the clock eight days a week: an exploration of temporal patterns of music listening -- [Perfecto Herrera], Zuriñe Resa and Mohamed Sordo

personal ex. showing diff between early day v. late night playlist
trying to link 2 concepts - Day- hour - (weather?) and Music track selection
few papers on this idea -- take things from Human Dynamics -- trying to enable playing music 'at the right moment' -- explore circular stats
Circular stats (eqs in paper at link) basically transform raw data by a perodicity (days, weeks)
Circular stats have analogous tools to trad stats - hyp tests for instance
Data for eval is full listening history of 992 unique last.fm users with artist/title + time of day (ToD) also got genre via track.getTopTags, keeping genre -- discarded users w/o enough data
scraped about half the data
attempt to make predictions - use two years of data to predict the ToD of play in next year
results: by day about 2.5x better than chance, by hour about 3-5x better than chance (move from half hour to hour tolerance doubles data
note that the figures are overall, some users are v. predictable in this way, some are not.
Concl: temporal patterns can be predicted - not just what but when. plugs the last.fm clocks
Q (dunno who asked): what about user to user offsets (eg. if a user gets up at 6 v 8am 8 am means something different)? Currently can't do this, need sensor data. Would be sweet if we could, though not tha tthe predictions are peruser, so this is to some effect already dealt with
Q (again, people say who you are): Method issue - when comp day v. hour there's a percentile diff in the err tolerance? Sure this could work look at baseline compare...
Q (Eugenio Tacchini): I tried this awhile ago, aggregated data, didn't find much spread do you think aggregation is the issue? yeah, must be specific to the user, right time + right user not just right user
Q (Klaas):do you think it would work with less data (can't wait 2 years)? Probably. This was a very conservative methodology, could probably get by with maybe three months. For this work we wanted lots of data to make things clear
Q (seriously ID guys): did you use a popularity filter? No. tested if pref for a genre is different than the average for that genre

Break time then my talk. no notes for my talk as I'm talking...

Update (12:16): I was without my machine for the social tag session, not just my talk. I'll get my hand notes in another post but for now here are the papers:

Update ( ):

next paper is being skipped since the author was unable to attend due to illness:

Survey of Music Recommendation Aids. Pirkka Åman and Lassi Liikkanen

Now joining the presentation already in progress by Audrey Laplante:

The Role People Play in Adolescents' Music Information Acquisition.

qualitative study of adolescents
'Did your music taste change significantly in the last three years?' Yes, whys: New boyfriend, New school therefore new friends, important discussion topic
"who in your 'gang' or group exert the most influence on others in terms of music?" -- 3 self-identified. Characteristics: highly invested in music, good comm, willing to share info. People who are opinion leaders want to stay opinion leaders, will invest heavily in effort
in other domains work shows that weak ties are more important then strong ties in finding new information works almost all the time -- for 2 participants weak ties important -- for others strong ties with significantly different social network are important -- music as vehicle for social interaction
strong ties have different roles -- not important for discover, but critical for legitimization of musical taste
similar and reliable social connections are critical
social network maps (pic forthcoming...)
unknown how common these results are (same survey) as yet unknown exact implications for recommenders
Q (unknown)- Weak ties v. Strong ties -- how do you define the difference?: not really about newness, but it's entirely possible with new detail
Q (claudio) - What kinds of systems are implied with this work? Not necessarily a different system for adolescents. tight connectivity is critical, perhaps the difference is that strong ties may become more critical
(claudio) - does the notion that music describes you change as you get older?: not really actually, adolescent are interested in individual uniqueness
Q (Mark L) are social networks online somewhat different?: yes and no. in facebook you can find relatives, but noise is a big problem. But trust is not known
I asked about using graph difference. Answer could work, also other automatic methods...

lunch now. I'll make a new post for the afternoon session.

updated again (5:14pm) Eugenio Tacchini is Italian not Finnish (oops)

Roomba Recon - A musichackday brain dump

2010-09-09T03:27:00.000-07:00

So this past weekend I attended the 2nd (annual?) London Music Hackday at the Guardian's offices at King's Cross. For the hack I created an algorithm that generates playlists between arbitrary start and end songs on soundcloud. It does this with almost no pre-indexing, allowing for playlists to cover the entire network and always use an up-to-date graph. It's (mostly) running live if you'd like to play with it.

Briefly, it performs a sort of bastardized A* search, bilaterially from both the start and the end song to form the playlist. There's a parameter to limit the length of the two playlist segments, by default this is 4 so the max playlistlength is 10 (2*4+2 for the end points).

The search algorithm collects social links of the artist corresponding to the given song. For each of these connections (you know, 'friends' or in soundcloud jargon 'followings') a determination of the cost of adding that song is calculated in the following way (for the half built from the start song):

where is the cost to add song m to list after song n, is some measure of distance from song n to song m and is the same measure of distance from song m to song e. Song e is the end song for the whole playlist. So basically the idea is that the cost of moving to a node is a ratio of how far away it is from where you were to how far it is to where you're trying to get. The whole thing is reversed for the other half, so the cost function makes it cheap to move toward the start song. If you simply want to randomly traverse social links the cost can be set to an arbitrary equal value (I used 1) for all links.

This leaves the matter of distance.

Starting with what I know best, I decided to try a content-based distance first. I should say that from the onset I figured this would be insanely slow, but none the less, I gave it a go. I implemented (available directly as well) a little object that will grab the echonest timbre features for any two soundcloud songs, summarize the features into a single multidimensional gaussian (mean and std) then take the cosine distance between the two tracks (other distance metrics could be computed as well, but cosine seemed reasonable). That takes something on the order of 45 seconds to do for every pair of tracks. When using it in the above playlister the whole thing would take maybe 4 hours (I think, I never actually let it complete). Clearly way too slow.

So taking inspiration from my about to be published work at WOMRAD, I thought some NLP could save the day. So the other distance measure I implemented (no direct access yet) is based on a tracks tags and comments. First I tokenize the comments and combine them with the tags to create a vector space model of a track's descriptive text. I then weighted everything using tfidf (the idf was populated with a random sample of tracks from across soundcloud that I gathered over the weekend, about 41,000 tracks in total. This is the only indexing that is done in advance). From the tfidf weighted terms in a vector space, I took the cosine distance. This is both quite quick and gives pretty good results.

Everything was built in python, the app is running in cherrypy, using numpy and scipy for the data handling and gensim for the tfidf related bits. Soundcloud and echonest interaction is all via their respective python wrappers. Also there's a more terse write up over at the musichackday wiki. I'll stick the code on my repository on github once it's cleaned up a bit (though that might be a little while as I seem to be rather busy with something at the moment...)

Right. Back to writing my thesis.

publications on playlists in ISMIR

2010-05-29T09:40:00.000-07:00

So for this year's ISMIR I'll be doing another tutorial. This one is entitled "Finding A Path Through The Jukebox – The Playlist Tutorial" and I'll be presenting it with Paul Lamere. As you may have guessed by the title it's all about playlists. So to frame some of my background work I thought I'd poke around the ISMIR proceedings to get a more complete idea of all of the papers that dealt with topic across the 10 years of proceedings (plus the just announced titles from this year).

First I did a simple title search using the tool at ismir.net. This shows that from 2000 - 2009 there were 14 papers with 'playlist' occurring somewhere in the title. Here they are over time:

Well, that doesn't show very much, just some interest, no trends or anything. So from there I took a look at the results of at the text search available from Rainer Typke's website. The full text search found some 123 papers mentioning playlist, certainly a few more than the title search. From there I wanted to see what the distribution of these papers was over time (as above), though this took a bit more work, as I couldn't sort out a means to export the search results... Anyway after a bit of counting I got this:

Well, now we're getting somewhere! Clearly there's an increasing number of papers discussing playlists at ISMIR. But wait, you say, this doesn't take into account the considerable expansion of the size of the conference over it's existence. So we can normalize to the number of papers per year that are known the the Cumulative ISMIR proceedings ( [35, 43, 62, 56, 108, 119, 99, 131, 111, 148] from 2000 - 2009 if anyone is interested). Below you can see both the title only and full paper search results normalized to the total number of papers:

The normalization didn't seem to change the trend much. But this leaves me wondering, what can be drawn from the the massive (and growing) disparity between title mentions and fulltext mentions? Obviously one would expect a higher number of hits, but a tenfold increase, seems very large. My first suspicion is that a great deal of this disparity comes from the fact that many papers at ISMIR that mention playlists are actually about something else (music similarity for instance) and then throw on a playlist as something of an afterthought. Perhaps this is an implicit acknowledgment of the great human-factor power of the playlist (as discussed in for instance this paper) or perhaps it's something else entirely.

Regardless of these finer points, it's clearly fair to say that there is a great deal of interest in playlist generation and analysis. If you're interested in these things, why not sign up for our tutorial?

facelift

2010-03-21T15:30:00.000-07:00

Hi blog readers. You may notice a slight change of scenery and the addition of some links just above the main text body. The links go the the other parts of my homepage and the color scheme shift is to keep everything consistent. I'm not much of a designer so I'll happily take any critique of the color scheme and such...

IEEE-THEMES --shameless self promotion--

2010-03-01T05:03:00.000-08:00

I'm going to be presenting work at IEEE-THEMES, a workshop collocated with ICASSP, on March 15th in Dallas, TX. The talk is associated with an article to be published in the august issue of Select Topics in Signal Processing, which is a special issue on signal processing and social networks. Here's the title/abstract (note: link is to a preprint, camera-ready isn't due till after the talk so paper may well change a touch...) :

Analysis and Exploitation of Musician Social Networks for Recommendation and Discovery

Abstract—This paper presents an extensive analysis of a sample of a social network of musicians. The network sample is first analyzed using standard complex network techniques to verify that it has similar properties to other web-derived complex networks. Content-based pairwise dissimilarity values between the musical data associated with the network sample are computed, and the relation- ship between those content-based distances and distances from network theory explored. Following this exploration, hybrid graphs and distance measures are constructed, and used to examine the community structure of the artist network. Finally, results of these investigations are presented and considered in the light of recommendation and discovery applications with these hybrid measures as their basis.

The paper mostly covers content that has been discussed elsewhere (much of it with Kurt Jacobson) refactored for a broader audience and with wider narratives in mind. That said there are some notable new findings in the paper as well. We have run another acoustic dissimilarity measure across the entire set (the 2009 MIREX entry in audio music similarity using marsyas) which for the most part confirms our earlier findings (that acoustic similarity and social similarity [mostly] aren't linearly correlated and that community genre labeling becomes more homogeneous [again, mostly] when using the audio sim as a weight). Additionally, we have broadened our comparison metrics to include an examination of the mutual information between the different dissimilarity sets. This also basically confirms our earlier findings, though mutual information provides a very satisfying level of nuance that is not possible from simply testing (using Pearsons) for linear correlation, especially given that our data is quite far from a normal distribution. So, if you're planning to be at ICASSP, I'd highly recommend IEEE-THEMES (the rest of the program looks to be very interesting as well...) and if you aren't going to be in Dallas, there are a few options for you.

If you're in London right now, you can come to Goldsmiths today at 4pm to rm 144 in the main building, where I'll be giving a trail run of the talk.
Slides (and perhaps some video) will be made available at some point (probably just after the talk is given).
IEEE is running a pay-to-watch live stream of THEMES, so there's that as well.

Generally, if you're going to be in Dallas fr0m March 15-19, much discussion can happen in person. Also, between now and then I'll be doing some traveling (tomorrow till 6 March I'll be at UIUC, then from there till the 14th of March I'll be in San Diego) so if any readers are interested in some in person discussion and our locations overlap, let me know and perhaps something can be arranged.

scipy and numpy from source, revisted

2010-02-11T07:41:00.000-08:00

A while back I posted some instructions for getting scipy and numpy mostly up and running from current svn checkouts under python 2.6 with mac os x 10.5.8. I updated to 10.6 sometime back and have been using the preinstalled version of numpy (1.2) for my array needs without any scipy with solid results. However, I needed to get at some scipy functionality (doing some mutual information analysis via pyentropy) so I thought I'd give the process a go with the newer OS version. I'm pleased to report that everything works and was relatively easy to install/build. Basically the old instructions still hold with a couple points.

It is necessary to update to a newer version of numpy, that you compile using the same fortran compiler you'll use with scipy.
If you're using the build of macpython that comes with 10.6 (which is py2.6) you'll need to add the option --install-lib=/Library/Python/2.6/site-packages/ to any commands using distutil to install (eg. setup.py install)

And that's about it. I used fresh check outs of scipy (r6233, v0.8.0.dev) and numpy(r8106, v1.5.0.dev), but the same versions I've had for a while of Sparse and gFortran (the details of which are in the old post). As bonus this seems to result in less unittest failures in scipy (now only 10!) for whatever that's worth.

MusicHackday: Stockholm

2010-01-27T07:19:00.000-08:00

So in a touch more than 48hrs I'll be hoping on a plane to go the Stockholm MusicHackday. It should be excellent, if the last one I went to is any judge. I'll be joined by fellow ISMS member Mike Jewell. The hack is being formulated, but may involve The World Bank's api and some yet to be determined sources of listener statistics. Also, somehow the echonest's api will be involved because I need to leave stockholm with one of these. We may need some further assistance to get something done in 24hrs, so if you're going to be at the hack and are looking for some folk to hack with drop a line in the comments...

A bit about playlists and similarity

2010-01-27T06:00:00.000-08:00

Sorry about the general radio silence of late. Many things going on, most of them interesting.

Lately I've been spending quite a bit of time considering various aspects of playlist generation and how they all fit together. Here are some of my lines of thought:

Evaluation of a playlist. How? Along which dimension? (Good v. Bad, Appropriate v. Offensive, Interesting v. Boring)
How do people in various functions create playlists? How does this process and its output compare to common (or state of the art) methods employed in automatic playlist construction. This is to say, are we doing it right? Are the correct questions even being asked?
What is the relationship between notions of music similarity (or pairwise relationship in the generic) and playlist construction?

While all these ideas are interrelated, for now I'm going to pick at point (3) a bit. I'm coming to believe this is central in understanding the other two points as well, at least to an extent. There are many ways to consider how two songs are related. In music informatics this similarity is almost always content-based, even if it isn't content derived. This can include methods based on timbral or harmonic features or most tags or similar labels (though these sometimes get away from content descriptors). This paints some kind of picture but leaves out something that can be critical to manual playlist construct as it is commonly understood (e.g. in radio or the creation of a 'mixtape'), socio-cultural context. In order to have the widest array of possible playlist constructions, it is necessary to have as complete an understanding of the relationship between member songs (not just neighbors...). Put another way, the complexity of your playlist is maximally bound by the complexity of your similarity measure.

M<=Cs

Where M is some not yet existant measure of the possible semantic complexity of a playlist and s is a similar measure of the semantic complexity of the similarity measure used in the construction of that playlist. C is our fudge factor constant. Now, obviously there are plenty of situations where complex structure isn't required. But if the goal is to make playlists for a wide range of functions and settings, it will be required some times.

In practice what this means is that you can make a bag of songs from a bag of features. However, imparting long form structure is at a minimum dependant on a much more complex understanding of the relationships (eg. sim) between songs (say from social networks or radio logs...)

Anyway, this is all a bit vague right now. I'm working on some better formalization, we'll see how that goes. Anyone have any thoughts?

Compiling bleeding edge SciPy on Mac OS X

2009-08-19T07:03:00.000-07:00

I do most of my number crunching computing task with SciPy these days, having basically kicked the matlab habit with the brief exception of occasional use of legacy libraries. SciPy is a joy to work with, but is a huge pain to build from source, in light of nasty dependancies (fortran things mostly) and some system specific hardware acceleration trickiness. Thankfully most users can download one of many prebuilt packages, perhaps the best being enthought's. If you've ever wanted to see what SciPy is all about, this is the easiest way to do so.

That said, one of the great things about using SciPy instead of matlab is that it's python. Except all the prebuilt binaries (to my knowledge anyway) use at newest python 2.5. Again, probably not a problem for most, but I use the nice socket library (amongst other things) that's been improved significantly in python 2.6. So for a while I had my SciPy python and my everything else python and every so often I would make another attempt building SciPy for py2.6 on my mac to integrate the two and every time it would defeat me.

Until yesterday.

So I'm going to attempt to fully document the process as I've now done it on 2 similar machines (home & lab) and now that I've figured out the tricky bits it seems fairly easy to reproduce. These instruction were followed on 1 - 2 year old intel based macs running 10.5.8. (Note these instructions don't touch on installing the other pieces of standard SciPy setup, ipython and pylab/matplotlib as I've never had much trouble getting these to build. I believe the easy_install process works for both, mostly)

(optional) If you don't want to build universal python modules remove "-arch ppc -arch i386" from the BASECFLAGS and the LDFLAGS in the python library Makefile, which should live somewhere around here: /Library/Frameworks/Python.framework/Versions/Current/lib/python2.6/config/Makefile
If you don't already have xCode 3.1.3 and the associated developer tools you need to get it for apple's custom build of gcc 4.2 (it's not the version that comes with most box copies of 10.5). Download and install a fresh copy of the Apple Developer Tools. You can get SciPy to compile with other variants of gcc 4.2 or greater (from MacPorts for instance) but they don't support apple specific options, which are very helpful in other situations.
Download and install gFortran as a patch to the apple gcc from att research. Why apple doesn't leave gfortran in gcc I don't know, but they don't and we need it. It's critical you use this fortran compiler as other variants of gfortran or g77 seem to cause errors.
Download and install UMFPACK and AMD from SuiteSparse. The easiest way I've gotten through this is to download the entire SuiteSparse and then do the following:
1. Modifiy the package wide config makefile found at SuiteSparse/UFconfig/UFconfig.mk by uncommenting the Macintosh options (currently lines 299 - 303)
2. In order to only compile the 2 packages we also need to modify the high level makefile (SuiteSparse/Makefile) by commenting out the references to the other packages under the default call (currently lines 10, 12-17, 19-24).
3. run make while in the SuiteSparse dir
4. because it would be too easy if SuiteSparse actually had an install routine, we have to install the just compiled libs ourselves. This is how I did it, though you can stick all these bits wherever you like as long as the python compiler and linker will see them:
  $sudo install UMFPACK/Include/* /usr/local/include/
  $sudo install AMD/Include/* /usr/local/include/
  $sudo install UMFPACK/Lib/* /usr/local/lib/
  $sudo install AMD/Lib/* /usr/local/lib/
  $sudo install UFconfig/UFconfig.h /usr/local/include/
Grab a bleeding edge copy of SciPy and NumPy via their subversion repositories.:
$svn co http://svn.scipy.org/svn/numpy/trunk numpy-from-svn
$svn co http://svn.scipy.org/svn/scipy/trunk scipy-from-svn
Build and install NumPy:
$cd numpy-from-svn
$sudo python setup.py build --fcompiler=gnu95 install
Test NumPy to make sure it's not broken (note that the tests need to be run out of the build directory):
$cd ..
$python -c "import numpy;numpy.test()"
Make sure numpy doesn't fail any of the tests (known fails and skips are okay) or the next bit may not work.
Similar to step 6, build and install SciPy:
$cd scipy-from-svn
$sudo python setup.py config_fc --fcompiler=gfortran install
similar to step 7, move out of the build directory and run the built in tests:
$cd ..
$python -c "import scipy;scipy.test()"
You're going to get some fails and maybe some errors. You're going to have to use your own judgement to as to whether these errors and fails are substantial. Most of the troubles I've encountered are trivial, things like a type being dtype('int32') instead of 'int32' which is actually the same and just needs to be updated to reflect newer numpy.

And now you have a nice SciPy build for whatever flavor of python you're working with on you Mac. Note that I have no idea how well this will work in anything other than python 2.6 on Mac OSX 10.5.8, though it will probably mostly work with other variants. Also, for completeness, I most recently compiled these versions: NumPy-r7303, SciPy-r5893. At some point I'm going to give it a go with python 3.x but that will be a whole new kind of pain I suspect. Anyway, if anyone uses these instructions and they don't quite work or you don't understand part of it, please let me know and I'll try to clarify or help as best I can. I'd really love to build a definitive set of instructions for building SciPy on a Mac, but I can only verify these instructions on my machines.

musichackday

2009-07-14T07:18:00.000-07:00

So I spent the weekend holed up in the Guardian offices at the musichackday. I went in with some perhaps overly ambition plans to generate playlists across the SoundCloud user graph, with song selection optimization done with features via theechonest. This might have been barely possible if I had been working with a couple other people of similar background, but circumstances led to me hacking mostly solo at this particular event.

In the end I spent a substantial amount of time beating the SoundCloud python wrapper into being more helpful for what I wanted it to do (which is perhaps not what it's envisioned use was, but hey, that's what hacks are for), namely walking the user (artist) space and creating a Complex Network so I can move the playlist generation tools we've created around myspace crawls over to SoundCloud.

So, to that end, I've created some bits of python that walk through the user graph on the SoundCloud and build a graph using iGraph. This code base is living over at a new github repository I've created called pySomethingClever. Included over there are diff files documenting the changes I made to official SoundCloud-api-wrapper, which will enable any willing victims to grab and run the hacky bits of code I have up.

Once I got the api wrapper in a place where it could do a bit of what I wanted I fired off a crawl. I got through about 4,000 users (of a complete user network of about 170k nodes for ~2.3% of the network) in SoundCloud's network before the presentations started on Sunday. To clarify slightly, the network contains all the users of SoundCloud, but only the outlinks (users a given user follows) from 4,000 nodes. This is to say I had a (mostly) complete vertex list and a very incomplete edge list. With the super great help of kurtj x this sampled network was pushed through the lanet k-core decomposition visualization to draw out some of the community structure and related forms of the sample graph. Here's that graph:

The size of each node is tied to the number of links (either direction) touching that node. The color and placement have to do with how critical the node is to the rest of the network maintaining its current state of connectedness.

Since the hack I've continued gather edges toward a complete representation of SoundCloud. I currently have the out link edges from more than 17,000 SoundCloud users (about 10% of the user base) and should have a full capture in the next few days. Below you can see the same visualization with the edges from 16,000 users (the graph is set to write every 2k):

As the crawl continues, my guess is the middle bits will continue to fill in, which would be expected if the SoundCloud behaves in the usual Power Law fashion (as most of The Internet's networks, social or otherwise, tend to).

It should be noted that these visualizations, while very interesting, are just the beginning of what is possible once the whole user network is captured. I'm going to be building some playlist generators and recommenders around this in the coming weeks. If things look good (and from here I'm quite excited) I'll push some of it to the ISMIR late breaking demos and possibly to AdMIRe. More to come!

The Semantic Web and Why You Should Care - a highly informal slidebook

2009-06-19T13:09:00.001-07:00

As you may have noticed, my posting has been super light (even for me) for the last few months. Sorry about that. Anyway, the reader may be interested in the presentation I gave at the IEEE/ACM Joint Conference on Digital Libraries, Workshop On Integrating Digital Library Content with Computational Tools and Services this morning (about 4 hours ago actually). These slides really require the presentation audio to make sense and if it was recorded (unsure if it was...) I'll edit the slide share to have the audio track as well. Regardless, here's the slidebook. Have fun.

The Semantic Web and Why You Should Care.

View more Microsoft Word documents from Ben Fields.

back to playlisting

2009-04-24T08:33:00.001-07:00

After a brief tangent into the wide world of mixing (of which I'll post some more in a bit from my proposed SMC paper in a bit) I'm back into playlist generation and related topics. Along those lines it occurs to me that readers of my little blog may be interested in perusing my recently completed (Dec 2008 actually) M.Phil to PhD upgrade document. For those of you unfamiliar with the British PhD system, PhD students start as a Master's of Philosophy by research student, then after about 24 months of independent research go through a process of summarizing and defending their work so far and what they intend to accomplish in the coming 18 - 24 months. The outcome of this process is one of three things:

Your work is deemed interesting, rigorous and sufficiently likely to succeed in the next couple of years. As a result, you upgrade to a PhD student and continue on with your research (this is what happened in my case)
You graduate at that point with a M.Phil.
You completely fail your upgrade process.

So that happened back in mid december. Here's the abstract from my upgrade:

A framework is described to consider various real world playlist use cases. Automatic playlist generation is introduced as a means to improve music recommendation. Literature in related topics is discussed.

A sample of the Myspace artist network is examined to investigate the relationship between social connectivity and audio-based similarity. Audio data from the Myspace artist pages is analyzed using well-established signal-based music information retrieval techniques. In addition to showing that the Myspace artist network exhibits many of the properties common to social networks, it is seen that there is an ambiguous relationship between audio-based similarity and the social connectivity. Further the Myspace sample is examined with the pairwise relational connectivity measure Minimum cut/Maximum flow. These values are then compared to a pairwise acoustic Earth Mover's Distance measure and the relationship is discussed. A means of constructing playlists using the maximum flow value to exploit both the social and acoustic distances is realized.

Two playlist generation methods are proposed for development and experimentation. The first is a direct extension of the myspace dataset analysis into a robust playlist system for interactive internet radio broadcast. The second is content based system which uses expert constructed playlists to construct transition models which can then be used on new material. This is followed by a discussion of evaluation needs and strategies.

If you're interested in reading the whole thing (comments welcome and encouraged) download the pdf.

On an unrelated note, I'll be a Yahoo Open Hackday 09 in Covent Garden in a couple weekends. It's free and I believe there are still some tickets if anyone is interested. It should be rad.

Conference alterations...

2009-04-16T05:23:00.000-07:00

So this paper I'm trying to put together on mixing algorithms just didn't quite come together in time for DAFx. Or more exactly, the actual research wasn't quite done till like yesterday. But it's not all bad. I'm submitting to Sound and Music Computing 2009 instead, which looks like it'll be an interesting conference. This also gives me a couple more much need days to hash out a few more thoughts on things.
Here's a teaser from the paper:

For the purpose of this discussion we will be dealing with various types of song transitions in order of temporal complexity, from the simplest to describe in time to the most complex.

Arbitrary Length Fixed Time Crossfade

Phrase-Aligned Start No Tempo Adjust

Phrase-Aligned Start, Running Beat Alignment

Phrase-Aligned Start, Phrase-Aligned Finish

Once I've got this draft done I'll post some more bits. In the meantime, anyone have any further ideas on core subdivisions of song to song transitions, from the perspective of time alignment?

working toward a rigorous definition of 'continuous mixing'

2009-03-27T08:54:00.000-07:00

So I have been thinking about this idea of continuous mixing, or generally speaking, more content aware automatic song to song transitions for some time now. To that end I'm aiming to put something together for DAfX on the topic. As this deadline for submission is fast approaching, my writing is generally focus in that direction. In the meantime however, I'll throw out a vague english definition of continuous mixing so that you the reader can get an idea of where I'm coming from on this topic, with some more rigorous definitions to follow.

continuous mixing: A song to song mixing technique where the goal is to obfuscate the transition between the two songs such that a casual listener cannot immediately pinpoint when the transition occurred. This can involve the use of beat matching/alignment, phrase matching/alignment, content aware equalization and other technical elements as well as sensible song selection with regard to harmony (i.e. key changes that make musical sense).

So that's my starting point. Anyone else have an opinion? What is continuous mixing mean to you? Bonus question! When is continuous mixing an appropriate transition technique in playlist presentation? I'm starting with modern electronic dance music, but these techniques can certainly apply to other musics (speed metal? maybe. free jazz? probably not...)