There’s a large amount of untapped data sitting in CRM, ERP and other enterprise systems, ripe with possibilities. Big Data solutions like Hadoop will allow businesses and software vendors to put that data to use. Nucleus Research predicts we’ll soon see enterprise applications with embedded analytics, integration of role-based interfaces, and a push-model for information, which is already available with some collaborative applications such as Salesforce.com’s Chatter

The possibilities are endless. Many times today we see HDFS and other Hadoop-technologies used to store messages (think Facebook) or log/analytics files (think Google search) but ERP and accounting data can also be stored there and the business intelligence world is just starting to see the possibilities here. This is going to be fun!

What China does better than any place else in the world is to innovate by commercialization, as opposed to constant research and perfecting the theory, like the West. When the Chinese get an idea, they test it in the marketplace. They’re happy to do three to four rounds of commercialization to get an idea right, whereas in the West companies spend the same amount of time on research, testing, and validation before trying to take products to market. The electric vehicle is a good example. The Chinese view is that it’s not going to be perfect, and they’re not trying to make it perfect from day one. They’ve got a few more series of improvements to go, and they’ll work on them in parallel with finding out what the customer really likes and adapting to that. That’s an innovative way of doing innovation, something that the rest of the world is struggling to understand. In our business in China, if we don’t innovate through or with commercialization, we’re going to lag behind our competitors.

I try to take this approach when looking at a large project. If you don't focus on small wins and learning as you go, you end up getting lost in the big picture. Start with something is 60-70% to where you want to be and then go through an iterative process to get to 100%.

Instead of leading with new technology, the studios lead with litigation, legislation and lobbying. (Imagine if the $110 million/year spent on lobbying went to disruptive innovation.)

Great read on the history of the movie/music industry's fight with technology and how Congress has gotten pulled into the middle of it.

But there isn’t a clear path in increasing analytical capacity, and that’s what goes wanting.

This goes right along with something seen for years. IT for the sake of IT rather than IT investment for the sake of returning value to the business. There is little point in setting up a Hadoop cluster and storing 200 TB of data if there is no plan on using that 200 TB of data for a business use and you have the appropriate man-power (however little or much is required) to extract the value.

If everything you do needs to work on a three-year time horizon, then you’re competing against a lot of people. But if you’re willing to invest on a seven-year time horizon, you’re now competing against a fraction of those people, because very few companies are willing to do that. Just by lengthening the time horizon, you can engage in endeavors that you could never otherwise pursue. At Amazon we like things to work in five to seven years. We’re willing to plant seeds, let them grow—and we’re very stubborn. We say we’re stubborn on vision and flexible on details.

Eric Baldeschwieler, Keynote speaker at Apache Lucene Eurocon 2011 in Barcelona last week, predicts that 50% of the world’s data will be stored on Hadoop within 5 years

Interesting article on the key role search (Lucene/Solr) plays in a Big Data architecture.

Why does social intelligence emerge as the make-or-break leadership skill set? For one, leadership is the art of accomplishing goals through other people.

Something I find hard at times but I think I am getting better with as I gain more experience and insight into how others approach problems.

This is what the rest of the world has been waiting for. I am looking forward to using this device.

A survey by Citigroup showed the primary inhibiter to tablet purchases among consumers was the price, and most struggling would-be iPad manufacturers, like Motorola and Research in Motion, cannot afford shrink margins to undercut the iPad.

While the cost to manufacture the Amazon tablet is not yet known, Amazon itself is one of the few companies that can even afford to sell a device at a loss to firmly plant its platform. It is expecting $10 billion in revenue in the current quarter.

Yes, I will spend $250 that gets me 90-95% of what an iPad does at $500. I think there are many of us out there looking forward to this Amazon.

1) Make the commitment. You need to decide to be an entrepreneur and leave the traditional path behind. A good friend of mine, who has built a $400M company, once told me that choosing to be an entrepreneur, deciding to give up the seemingly safe employer-employee world, is like disengaging from the matrix. Once you have experienced reality, you never want to go back.

2) Find a problem, then solve it. There are two types of businesses --“pain” and “pleasure.” “Pain” businesses have a higher success rate because they address more urgent problems. Venture out to address a problem that is personal, that you care about, that you would love to solve.

3) Think big. You can never build what you cannot imagine, so to build a business that has big impact, you have to believe you can do it from the start. You come at the problem with a calm confidence. There is no reason to scream “We can do it!” because you just know it; it’s inevitable. Whatever is stopping you for really believing you can do what your business plan says you can do, dissect it, extract it, and throw it away.

4) You can’t do it alone. There are two types of people: big-idea people and get-it-done people. Very few of us can do both. Be honest with yourself, decide which type you are, then find (hire or partner with) someone who complements you.

5) You must do it alone. Entrepreneurial journeys are lonely. When I launched my consulting business I spent every day for three weeks alone at my computer, working an Excel database I had built of 300 prospective clients, sending 100 emails a day, trying to get my first project. Even after you’ve gotten your business going and have built a team, you will find yourself alone. Leadership, after all, is having no one else to ask.

6) Manage risk. Most ventures fail because they run out of money. So never forget “DROOM”--Don’t Run Out of Money. Live by that motto. Calculate what you will need in the worst-case scenario and make sure you have enough. I do this every week now for my business, and I have not hit a cash crunch since.

7) Learn to lead. You will reach a point at which you must shift from managing to leading. This means setting a vision and communicating it. It means making promises (“We will get there”) and keeping them.

8) Learn to sell. This is my favorite. All innovators are sales people. They are selling every day--to investors, to employees, to suppliers, to customers. By convincing everyone that the iPod will change the world, Steve Jobs makes it so. Yet no business school I know of teaches sales! Study the sales process by reading books like SPIN Selling. Get practice by, for example, volunteering to raise funds for your favorite cause. But most importantly, find your passion. If you really believe your product and company and you share it passionately, you will become a magnet.

9) Persist. If you are willing to persist through the tough times, you have a much better chance of succeeding. Develop stubbornness.

10) Play the game for life. Don’t just do this for the money; do it to realize who you are, to create the life you are here to live. One of my favorite quotes from the book reads “Entrepreneurship...isn’t just about solving a problem, building a venture, managing risk, or making money. It’s about having a positive impact on the world, making the most of the gifts you’ve been given, and realizing your full potential as a human being.”

Good stuff if you are an entrepreneur yourself or desire the entrepreneurship spirit in any other job.

The problem is that, in the world of big data, we don’t really know what value the data has when  it’s initially accepted from the array of sources available to us.

For instance unstructured data by itself many times takes on a whole new level of value to an organization when it is analyzed in context with transaction-based data.

RecordBreaker is a project that automatically turns your text-formatted data (logs, sensor readings, etc) into structured data, without any need to write parsers or extractors. In particular, RecordBreaker targets Avro as its output format. The project’s goal is to dramatically reduce the time spent preparing data for analysis, enabling more time for the analysis itself.

Hadoop’s HDFS is often used to store large amounts of text-formatted data: log files, sensor readings, transaction histories, etc. Much of this data is “near-structured”: the data has a format that’s obvious to a human observer, but is not made explicit in the file itself.

This looks like is would have potential in identifying and cataloging data files inside Hadoop.

Chart of Data Quality Maturity Levels for Big Data Analytics by Chris Bradley.