Data Mining, Down Under

Reducing Churn Through Social Network Analysis

Shane Butler — Tue, 23 Feb 2010 09:33:16 +0000

Earlier in the month local data miner Tim Manns presented at the Sydney Data Miners group. Tim spoke on some work been doing at Optus around using mobile call patterns to establish social networks and using these networks to reduce customer churn. Interestingly, there are also applications in many other areas, including data cleansing, for example, where one person has purchased two mobile phones and given one to their spouse or child. Using this analysis we can try to determine which account is likely to be the actual account holder and infer the details (such as age) of the other customer.

For a full write up of Tim’s work, check out James Taylor’s PAW 2009 summary or head over to Tim’s data mining blog.

PMML Tree Model to Code Converter

Shane Butler — Sat, 30 Jan 2010 00:33:58 +0000

Lately I’ve been trying to come up with a generic way to deploy models on any platform. So I’d like to share some early code that takes a PMML TreeModel and converts it to R code. The intention is to get the R code generation working right, then extend to support generation for other languages. Anyway, here it is (remember — early alpha, very rough still!!). Updates to follow soon!

AusDM 09 & Analytic Challenge

Shane Butler — Tue, 07 Jul 2009 13:47:52 +0000

Australian Data Mining conference (AusDM09) will be held in Melbourne next December and Dr Phil Brierley of Tiberius Data Mining has put out the call for proposals for an analytic challenge to accompany the conference. Competitions are quite popular in data mining circles and provide a good training ground for new practitioners to get access to real data and solve real problems. They also often have surprising results, such as the team who used laptop with 2GB RAM to beat IBM’s mighty clusters.

For businesses, this is a great opportunity to find out what is available by having others suggest new ideas and methods, or even to test your internally deployed models against the best of the best. So if you’re a business who has data, please consider being invloved! For further details, see the competition webpage.

PMML 4.0 Released

Shane Butler — Thu, 18 Jun 2009 09:50:37 +0000

The DMG has released a new version of the PMML open format for representing predictive models. The new version includes support for ensembles, new model types and more built in functions to name just a few of the enhancements. For a detailed summary, see the Zementis blog.

Ten Data Mining Mistakes to Avoid

Shane Butler — Fri, 15 May 2009 10:19:37 +0000

Some really good advice here from John Elder in a series of video tutorials on data mining mistakes to avoid. Tip #5, regarding contaminating the project with future data is a good one, although sometimes it can be quite tricky (if not impossible) to ‘rewind’ the data! I believe the video series is a part of the launch of The Handbook of Statistical Analysis and Data Mining Applications. You can watch part one below or head over to YouTube for the entire series.

RapidMiner to get dual GUIs

Shane Butler — Wed, 13 May 2009 16:55:44 +0000

A forum post by Ingo Mierswa of Rapid-I indicates the upcoming RapidMiner v5 will feature two GUIs: the existing tree-based designer and a new graph-based designer! I’m quite excited about this because I’ve personally found the existing UI a bit clunky. Details and screenshots over at the
user forum.

SAS hints at future R integration

Shane Butler — Tue, 17 Feb 2009 11:03:23 +0000

In more R news, it appears SAS isn’t as worried about airplane safety as originally thought, and has indicated they will include R support in an upcoming update to the SAS/IML product. For details see NYTimes & Adventures in Consulting.

R in the New York Times

Shane Butler — Thu, 08 Jan 2009 02:32:17 +0000

The New York Times has an interesting story on the increasing use of R for data analysis within academia and industry. Several large corporates are cited as having selected R over commercial conterparts such as S and SAS.

[via Slashdot]

Update: For more R news, see also Ajay Ohri’s interview with Dr Graham Williams, the author of Rattle.

RapidMiner 4.3 Released

Shane Butler — Fri, 28 Nov 2008 04:19:56 +0000

Rapid-I has released an new and improved version of the open source data mining suite RapidMiner (formely called YALE). I’ve been evaluating RapidMiner lately as a possible addition to my data mining toolbox. I’ve found the biggest hurdle in learning how to use it is probably the GUI. It is a tree-based GUI which I find much harder to understand than the graph-style approach used by many others. However RapidMiner is quite a powerful tool, and the Community Edition is free, so there is probably a lot of benefit in getting used to the strange GUI.

The built in tutorial is a really good way to get a grasp of the system and I highly recommend spending some time on this if you are interested in learning RapidMiner. I would also recommend a series of RapidMiner video turtorials over at Neural Market Trends that are worth checking out too.

RapidMiner 4.3 includes a 3D lift chart

SAS Forum (Australia) presentations available online

Shane Butler — Mon, 29 Sep 2008 23:30:52 +0000

The SAS Forum (Australia) was held in Sydney back in August. I was unable to attend but luckily the presentations have been put online. Here are some that I found interesting:

Make Sure Your Insight is Insightful: Analytical Marketing at NAB by Antony Ugoni (National Australia Bank)
Model Deployment and Management – The ATO Story by Warwick Graco (Australian Taxation Office)
Putting Cheques in Place to Identify Fraud by Dr Paul Bracewell (Offlode NZ) and Flavio Palaci (Marsh Australia)
Customer Value Creation Using Analytics by Arun VS (Satyam)
Analysing Performance and Tuning your SAS Application by Bill Gibson (SAS)