The second paper, Data Science Revealed: A Data-Driven Glimpse into the Burgeoning New Field (EMC Data Science Community), proposes a benchmark of the data science community (understand data mining and business intelligence practitioners). The study tries to differentiate data scientists and business intelligence people. For that, about 500 practitioners have been interviewed. I find their concluding paragraph very relevant:

“Once companies have brought in the right talent, they need to create an environment conducive to effective data science. That means building high-performing, cross-functional teams that include a variety of roles, including programmers, statisticians, and graphic designers, and aligning them to directly support interested business decision makers. They should also loosen restrictions on data in the enterprise, allowing employees to more freely run data-driven experiments. Finally, data scientists should be given free access to run experiments on data, without bureaucratic obstacles, so that they can rapidly translate their own intellectual curiosity into business results.”

Data Science Revealed: A Data-Driven Glimpse into the Burgeoning New Field

Intelligence for Everyone: Transforming Business Analytics Across the Enterprise

Data Mining, Refining, and Storage

As companies are able to more effectively store and retrieve information via online storage services, the relevance of data mining as the crucial first step towards intelligent business decisions will become an important tool that was previously unavailable for small and medium-sized businesses and may usher in a new era of success for those who can afford to make the change.

Online Storage: Addition by Subtraction

The effect on large and small companies of using services that provide online storage should be evaluated by looking at what the process takes away rather than what it necessarily adds.  More and more companies are finding themselves practically bombarded with the exponential growth of data that is available about their customers, provided in no small part by the increasing number of internet-connected sensors that people own—smart phones, tablets, laptops, to name just a few.

Just consider this snippet taken from a white paper produced by Globalknowledge, a leader in IT and business skills training:

“According to International Data Corporation (IDC), ‘The proliferation of devices, compliance, improved systems performance, online commerce and increased replication to secondary or backup sites is contributing to an annual doubling of the amount of information transmitted over the Internet.’”

Indeed, a veritable ocean of information has been created, and its unexplored depths contain the potential for game-changing discoveries.  Unfortunately for medium-sized businesses, or even larger businesses that find themselves underprepared, the challenges of collecting their relevant data efficiently and storing it securely are enough to keep them too busy to participate in this feeding frenzy of information.  That’s where the developing industry of online storage comes in.

Online storage as a service is a young industry, and as with any young industry there are still some very real customer concerns that need to be addressed before it can be widely accepted as a solution.  Specifically, those who wish to provide online storage to customers at a large scale will need to bridge the security gaps that are still associated with it; however, as those issues are resolved with time, purveyors of online storage services that position themselves to accommodate this massive influx of data from small to medium-sized businesses may find that they have customers knocking down their doors.

The introduction of online storage as a service to small and medium-sized businesses that are having difficulty managing their data has a more profound effect on their bottom lines than it might initially seem.  That’s because rather than spending too much time trying to manage data, companies can shift their primary focus toward understanding it, which is a monumental task unto itself.

If the process of local storage and management was a difficult burden, then the process of knowledge discovery required to understand it is a seemingly insurmountable one—how can any person possibly hope to make sense of petabytes of information?  The answer to this question, and the key first step toward vital business intelligence for curious companies, is data mining.  As an automated process, data mining can help companies identify patterns in their data that can be analyzed in detail and used in predictive or product-oriented capacities.

In a dark room, data mining is like emergency lights that come on and provide much needed guidance and direction.  Without the benefit of online storage to relieve the mounting pressure from data overload, though, small and medium-sized businesses are in danger of being too busy to see the light.

Eric Greenwood is a technophile whose interests span the range of data management, online storage, business intelligence and much more – Read more of his work at the blog Online Storage!

If you come from the business side (e.g. C-level), the book may be interesting but the explanations about predictive analytics are quite light and you won’t see all benefits of these techniques in the company. I know that the main focus of the book is not about teaching analytics. It seems also not to be about filling the gap between analytics and action. I’m thus a bit confused about the real objective of the book. It is also explaining concepts at a very high level of abstraction. It is thus not directly usable in practice.

The book is divided in three parts. In the first part, James explains what are DMS and why they are useful for the company. The second part focuses on building these DMS. The third part is about the enablers (people, processes and technology), i.e. the aspects that will allow such DMS to be a successful initiative. Personally, I found the book very interesting starting from chapter 6 (Design and Implement Decision Services). The topic of fraud detection and prevention is very well studied throughout the book.

A very strange choice has been made to repeat in full text the expression Decision Management Systems hundreds of times. It thus makes the reading sometimes a bit tiring. The simple use of the abbreviation DMS would have solved this issue. To conclude, I found the book interesting and well written. However, keep in mind that it is written with a very high level of abstraction. You will thus have a clear understanding of the domain, but no practical advices.

Decision Management Systems: A Practical Guide to Using Business Rules and Predictive Analytics

• Inside Data Mining: Blog written by the two authors of the excellent Data Mining Techniques in CRM, Antonios Chorianopoulos and Konstantinos Tsiptsis. The blog is about their book and data mining topics with application to Customer Relationship Management (CRM).
• Simplified Analytics: The blog by Sandeep Raut started in March 2011. It is about business analytics with various topics such as customer churn management, cross/up-selling, analytical tools and so on.

I’m sure these two blogs will add interesting content to your data mining readings.

• More than 67′000 visits
• Around 110′000 pageviews

Top 5 read content (in decreasing order):

1. Top 10 challenging problems in data mining
2. List of blogs
3. Top five articles in data mining
4. Standardization versus normalization
5. Data miners on twitter

I hope you had some nice reading on Data Mining Research in 2011. Next year will come with new interviews of data miners, data mining book reviews, case studies, current challenges, new blogs as well as a new project I will explain you soon!

Happy new year 2012!

Sandro Saitta

• Where to Begin with Predictive Analytics: James Taylor explains that a single successful project may be enough to justify the need for analytics in a company. In his article he explains that the best place to start analytics in a company is with operational decisions.

• Can Machines Think?: Robert Wright wrote this article 15 years ago and it’s very interesting to see what improvement we made since that time.

• The CEO Wants Analytics! Now What?: Imagine that your CEO just read the last business book on analytics. Now he wants analytics in his company. Meta Brown explains you what you can do to start.

Feel free to share your own recent readings.

The conference was a clear success! Very dynamic audience. Also very easy to interact with keynote speakers and BAQMaR organizing committee compared to larger conferences in the domain. The committee (six people) did an amazing work to make this event a success. Congratulations to the BAQMaR team! If you don’t know yet the BAQMaR association, feel free to visit their website (new version soon): www.baqmar.be

But what about association rule mining? If you use random sampling before an association rule algorithm, you may end up finding no rule. The reason is that association rule mining analyses the data as transactions. The idea is to find recurrent trends in a set of transactions that are usually continuous. Here is an example:

Transaction ID / product
112 / bread
112 / butter
112 / jam
113 / cheese
113 / bread
...

The issue with random sampling is that it will not take into account the continuous sequence of events. In the case of association rules, one should take a continuous subset of the data in order to get meaningful rules.

Do you have any other examples where random sampling can’t be used? Other issues with association rule mining? Feel free to comment this post.

Continuing with the presentations, the focus was on fraud detection with other interesting topics such as tax evasion, crime analytics and postal applications. To be noted also keynotes by Earl E. Devaney and Congressman Darrell Issa.

I think PAW Gov was successful thanks to the following factors: many experts, exciting topics, unique track, narrow place (and thus friendly discussions during breaks). My only recommendation for next year is to have a few more technical presentations. However, the event was clearly successful and I recommend it without any hesitation!

“In the last 5 years the rise of the internet and especially social media caused a boom in new opportunities we as researchers and analysts have: new data sources, alternative ways of collecting data & engaging marketing people with the consumer world. Next to that science moved on and showed us that consumer behavior is less rational than we think. A shocking fact that is putting question marks to everything we do.”

This year’s conference will bring together some of the brightest minds of the industry who found ‘creative ways’ to grasp the new opportunities or to overcome some of the issues we face today. Let’s take you on a journey to discover all the creativity that is out there!