myNoSQL

memcached turns 10 years old

Tue, 21 May 2013 10:14:15 -0700

Ars Technica:

This week, memcached, a piece of software that prevents much of the Internet from melting down, turns 10 years old. Despite its age, memcached is still the go-to solution for many programmers and sysadmins managing heavy workloads. Without memcached, Ars Technica would likely be unable to serve this article to you at all.

According to a commenter on HN, memcached’s 10th birthday would actually be tomorrow (May 22nd). I’m pretty sure it will outlive today.

✚ I’ve left a comment myself on HN in which I express my admiration for obvious tools that have changed the face of software development (e.g. memcached, JUnit, etc.)

Original title and link: memcached turns 10 years old (NoSQL database©myNoSQL)

The MEAN Stack: MongoDB, ExpressJS, AngularJS and Node.js

Tue, 21 May 2013 07:33:00 -0700

The MEAN Stack: MongoDB, ExpressJS, AngularJS and Node.js:

MongoDB, ExpressJS, AngularJS and Node.js the MEAN stack or as the first commenter on the post called it: “the hipster stack”:

A few weeks ago, a friend of mine asked me for help with PostgreSQL. As someone who’s been blissfully SQL-free for a year, I was quite curious to find out why he wasn’t just using MongoDB instead.

It’s all roses on the way to MongoDB.

Original title and link: The MEAN Stack: MongoDB, ExpressJS, AngularJS and Node.js (NoSQL database©myNoSQL)

Apache Hive 0.11: Stinger Phase 1 Delivered

Tue, 21 May 2013 07:14:03 -0700

Apache Hive 0.11: Stinger Phase 1 Delivered:

Owen O’Malley on Hortonworks’ blog:

As representatives of this open, community led effort we are very proud to announce the first release of the new and improved Apache Hive, version 0.11. This substantial release embodies the work of a wide group of people from Microsoft, Facebook , Yahoo, SAP and others. Together we have addressed 386 JIRA tickets, of which there were 28 new features and 276 bug fixes. There were FIFTY-FIVE developers involved in this and I would like to thank every one of them.

This is indeed the power of open. But don’t forget that too much bragging might diminish it: keep repeating a word and its value will slowly vanish.

Original title and link: Apache Hive 0.11: Stinger Phase 1 Delivered (NoSQL database©myNoSQL)

6 Key Hardware Considerations for Deploying Hadoop in Your Environment

Tue, 21 May 2013 05:17:05 -0700

6 Key Hardware Considerations for Deploying Hadoop in Your Environment:

To deploy, configure, manage and scale Hadoop clusters in a way that optimizes performance and resource utilization there is a lot to consider.

The 6 aspects presented in the post: OS, MapReduce slots available across nodes, memory, storage, capacity, network. It would be a lot more useful to put these in some order based on the scenarios the Hadoop cluster will have to solve.

Original title and link: 6 Key Hardware Considerations for Deploying Hadoop in Your Environment (NoSQL database©myNoSQL)

10 questions to ask when hosting your database on AWS

Tue, 21 May 2013 04:49:15 -0700

10 questions to ask when hosting your database on AWS:

Dharshan Rangegowda, founder of Scalegrid, posted a list of 10 questions that should be answered before hosting your MongoDB on AWS. But these are generic enough to extend to any database-on-AWS solution. They cover aspects like HA, backup and restore, monitoring, and basic security. If you haven’t done this before, save them as a quick check list.

✚ Just because you set up HA and backups, it doesn’t mean they’ll actually work when you need them. Test them over and over again. Make it part of your regular procedures.

Original title and link: 10 questions to ask when hosting your database on AWS (NoSQL database©myNoSQL)

Hadoop, Security, and DataStax Enterprise

Mon, 20 May 2013 07:29:16 -0700

Hadoop, Security, and DataStax Enterprise:

But the eWeek article demonstrates that the same concerns [nb: about security] exist where Hadoop implementations are concerned. The article says: “It [Hadoop] was not written to support hardened security, compliance, encryption, policy enablement and risk management.”

The story goes like this: in the early days of NoSQL, when no NoSQL database had any sort of security features, people behind the projects answered: “it’s too early. we’re focusing on more important features. and you can still get around security by placing your database behind firewalls”. Today, when more and more NoSQL databases are adding security features, the story these same people are telling is quite different: “ohhh, security is critical. we don’t really see how you could run a database without these features”.

Security is always critical. And exactly the same can be said about maintaining a solid, coherent story of what you are telling your users.

Original title and link: Hadoop, Security, and DataStax Enterprise (NoSQL database©myNoSQL)

The Master-Slave Architecture of HBase

Mon, 20 May 2013 07:21:58 -0700

The Master-Slave Architecture of HBase:

Fantastic post by Matteo Bertozzi looking at HBase’s master-slave architecture:

At first glance, the Apache HBase architecture appears to follow a master/slave model where the master receives all the requests but the real work is done by the slaves. This is not actually the case, and in this article I will describe what tasks are in fact handled by the master and the slaves.

Original title and link: The Master-Slave Architecture of HBase (NoSQL database©myNoSQL)

Neo4j Blog: Reloading my Beergraph - using an in-graph-alcohol-percentage-index

Mon, 20 May 2013 06:55:46 -0700

Neo4j Blog: Reloading my Beergraph - using an in-graph-alcohol-percentage-index:

Rik Van Bruggen about data modeling in Neo4j:

One of the things that spurred the discussion was - probably not coincidentally - the AlcoholPercentage. Many people were expecting that to be a property of the Beerbrand - but instead in my beergraph, I had “pulled it out”. The main reason at the time was more coincidence than anything else, but when you think of it - it’s actually a fantastic thing to “pull things out” and normalise the data model much further than you probably would in a relational model. By making the alcoholpercentage a node of its own, it allowed me to do more interesting queries and pathfinding operations - which led to interesting beer recommendations. Which is what this is all about, right?

I can see where this is going, but I’m not sure I agree it’s the right approach. Basically in this case it works because the domain of the field is both discrete and small. Ideally, though, what you’d actually want is an index that could give you nodes that are “close-to-some value” (e.g.: “give me the beers in the 6.9-7.1 range”)

Original title and link: Neo4j Blog: Reloading my Beergraph - using an in-graph-alcohol-percentage-index (NoSQL database©myNoSQL)

One Good and one bad

Mon, 20 May 2013 04:30:24 -0700

For the beginning of the week, some good and bad news:

Tableau Software’s IPO is considered successful.

Derric Harris for GigaOm:

The company didn’t really need more capital to operate, Chabot said, but one of the primary drivers was to raise awareness of the company. It has about 12,000 customers, he said, but there are millions more possible users. As part of attracting them, the company is going to expand globally and is working to improve its reach across mobile devices, the cloud and the Mac operating system.
Bradford Stephens announces that Drawn to Scale closes. Drawn to Scale was building a SQL-on-HBase solution and according to the post it already had paying customers.

It seemed we had everything going for us — paid customers such as American Express, a major telecom, Flurry, and 4 others. Our technology worked brilliantly, we had a big hiring pipeline, and we had great media presence against our competitors who raised 10-100x more cash.

Yet five days before we signed term sheets for a big A round or sold the company, we started getting hit by a series of black swans — and we just didn’t have what we needed to recover.

I didn’t talk to Bradford Stephens, but I assume the black swans are all the recent big name announcements related to SQL-on-Hadoop.

✚ Bradford, I’m sad to learn that the Drawn to Scale adventure has ended. But an end is just a new beginning. Good luck!

Original title and link: One Good and one bad (NoSQL database©myNoSQL)

MySQL 5.6, InnoDB and fast storage: 240k QPS

Mon, 20 May 2013 04:27:35 -0700

MySQL 5.6, InnoDB and fast storage: 240k QPS:

Mark Callaghan runs some benchmarks against MySQL 5.6.11:

Using MySQL 5.6.11 and InnoDB with a few hacks the peak throughput was about 240,000 QPS and 210,000 block reads/second. The test server has 32 cores (16 physical cores, 32 logical cores with HT enabled). This is a great result that can probably be even better. Contention on fil_system->mutex was the bottleneck and I think that can be improved (see feature request #69276). I wonder if 400,000 block reads/second is possible?

Original title and link: MySQL 5.6, InnoDB and fast storage: 240k QPS (NoSQL database©myNoSQL)

MongoLab offers MongoDB on Google Cloud Platform

Wed, 15 May 2013 22:25:07 -0700

MongoLab offers MongoDB on Google Cloud Platform:

This was fast:

This week at Google I/O we are launching support for MongoLab‘s fifth cloud provider – Google Cloud Platform. You can now use MongoLab to provision and manage MongoDB deployments on Google Compute Engine (GCE)!

Good move for MongoLab and good win for MongoDB users. I’ve read a lot of good things about Google’s Cloud Platform.

Original title and link: MongoLab offers MongoDB on Google Cloud Platform (NoSQL database©myNoSQL)

Introducing Google Cloud Datastore

Wed, 15 May 2013 22:11:53 -0700

Introducing Google Cloud Datastore:

Urs Hölzle in a post summarizing some of the announcements at Google I/O:

Google Cloud Datastore is a fully managed and schemaless solution for storing non-relational data. Based on the popular App Engine High Replication Datastore, Cloud Datastore is a standalone service that features automatic scalability and high availability while still providing powerful capabilities such as ACID transactions, SQL-like queries, indexes and more.

I’m heading over to the project’s site to read more.

Original title and link: Introducing Google Cloud Datastore (NoSQL database©myNoSQL)

Hadoop for graphs - GraphLab picks up $6.75m from Madrona and NEA

Wed, 15 May 2013 05:14:56 -0700

Hadoop for graphs - GraphLab picks up $6.75m from Madrona and NEA:

Robin Wauters for TNW:

Seattle startup GraphLab claims it is building the “fastest machine-learning analytics engine for graph datasets”, based on the popular open-source distributed graph computation framework with the same name, and it has just raised capital to come through on its promise.

Good luck to GraphLab’s team.

✚ Here’s a short list of MapReduce implementations for graphs.

Original title and link: Hadoop for graphs - GraphLab picks up $6.75m from Madrona and NEA (NoSQL database©myNoSQL)

Hadoop, Moh's Law and Corollaries

Tue, 14 May 2013 21:05:16 -0700

Hadoop, Moh's Law and Corollaries:

Robert Novak’s proposes Moh’s law and Rob’s corollaries to Hadoop and Big Data:

Hadoop is hard.
Make sure your’re measuring what you think you’re measuring.
Make sure you’re measuring what you need to be measuring.

For the first, I’m somehow confident that Cloudera and Hortonworks and others will finally solve it over time. But for the latter you are the only responsible. Not even a SaaS can save you.

This is why big data is the sweet spot for SaaS ... and here are 5 reasons why it is Not

Tue, 14 May 2013 20:57:09 -0700

Derrick Harris in an article about some SEO-as-a-Service company I haven’t heard about:

People often ask me where the smart money is in big data. I often tell them that’s a foolish question, because I’m not an investor — but if I were, I’d look to software as a service.

There are two primary reasons why, the first of which is obvious: Companies are tired of managing applications and infrastructure, so something that optimizes a common task using techniques they don’t know on servers they don’t have to manage is probably compelling. It’s called cloud computing.

The other reason is that the big part of big data really is important if you want to get a really clear picture of what’s happening in any given space. While no single end-user company can (or likely would) address search-engine optimization, for example, by building a massive store comprised of data from hundreds or thousands of companies as well as the entire web, a cloud service dedicated to that specific task can.

These are obvious advantages of moving the responsibility to a third party service. But I don’t believe SaaS is the future of big data and here’s why big data is not the sweet spot of SaaS:

a SaaS solution is good at a particular job, but it’s rarely the case that particular job is answering all your company questions and reveal the insights in your data. SaaS solutions will tell you want they, not you, think is important about your data.
the promise of a SaaS solution to give you access to more aggregate data sounds wrong. Big data is mostly about your data and each customer will have access to their own slices. Indeed a SaaS solution could augment your data with open data or extra data you’d need to pay for.
transporting your data to each SaaS to answer every question your company has is extremely expensive. If possible.
the nature and form of the questions big data tries to answer is changing. SaaS services will not adapt as fast as you want to the range and depth you need.
having your data in different SaaS solutions is just equivalent to having it in different internal silos. Except you’d pay someone else to protect the silo. The costs of breaking these silos will be much, much higher, so long term you might actually find a real reason why you cannot analyze your data.

Big Data is about agility. It’s about experiments. It’s trial and error. SaaS is about none of these when speaking years and years of data.

Original title and link: This is why big data is the sweet spot for SaaS … and here are 5 reasons why it is Not (NoSQL database©myNoSQL)

Even web giants like Facebook and Yahoo generally aren’t dealing with big data

Mon, 13 May 2013 20:45:16 -0700

Even web giants like Facebook and Yahoo generally aren’t dealing with big data:

Even web giants like Facebook and Yahoo generally aren’t dealing with big data, and the application of Google-style tools is inappropriate.

Facebook and Yahoo run their own giant, in-house “clusters”—collections of powerful servers—for crunching data. The necessity of these clusters is one of the hallmarks of big data. After all, data isn’t all that “big” if you could chew through it on your PC at home. The necessity of breaking problems into many small parts, and processing each on a large array of computers, characterizes classic big data problems like Google’s need to compute the rank of every single web page on the planet.

But it appears that for both Facebook and Yahoo, those same clusters are unnecessary for many of the tasks which they’re handed.

I guess we need some sort of “big journalism” sooner rather than later.

Original title and link: Even web giants like Facebook and Yahoo generally aren’t dealing with big data (NoSQL database©myNoSQL)

What Open Source Hadoop Coming to Windows Means to IT

Mon, 13 May 2013 20:39:06 -0700

What Open Source Hadoop Coming to Windows Means to IT:

This will open up Hadoop to a large number of organizations that have no in- house Linux skills. Shaun Connolly, vice president of Corporate Strategy at Hortonworks, explains the thinking behind moving HDP to Windows in this way: “Essentially it’s a market-driven decision,” he says. “Hadoop is built for the scaleout commodity hardware market, and the commodity hardware market is 70% Windows by install base and expertise.”

Employees in Windows-only companies will be able to make use of Hadoop easily because Excel can be used as a business intelligence tool to view the results of Hadoop Big Data analysis (whether Hadoop is running on Windows or Linux). “Ideally we want Microsoft users to be oblivious to the fact that everything is coming from Hadoop,” says Connolly. “If end users can consume data without any learning curve, thanks to tools like Excel, then they get more value.”

Either the data or the logic above is not sound:

those Windows machines that make up the 70% of the market are probably running Excel
those 70% of the market Windows machines are not going to run Hadoop

Based on this sort of market-share decisions, tomorrow we should see Hadoop for iOS and Android and Nokia. Sometime soon Microsoft will release Excel for iOS and maybe Android.

MetLife uses MongoDB

Mon, 13 May 2013 20:05:00 -0700

MetLife uses MongoDB:

InformationWeek, in an article about MetLife migrating to MongoDB:

“We had 60 different teams working together as one group, and they were working nights and weekends not because they had to but because they were excited and wanted to,” says Gary Hoberman, MetLife’s senior VP and CIO of regional application development.

Just imagine how many nights and weekends and holidays these guys would put in if allowed to use an IDE. Like vim or emacs.

Bootstrapping Neo4j With Spring-Data...without XML

Mon, 13 May 2013 04:45:50 -0700

Bootstrapping Neo4j With Spring-Data...without XML:

The emphasis is on without XML:

With the maturing of Spring-Data I started porting all my personal projects to use Spring Data for bootstrapping.

Quite a bit of annotations needs, but I’d go with that instead of XML.

Cloudera Announces Cloudera Developer Kit, Enabling Developers to Build Hadoop Apps Faster

Mon, 13 May 2013 04:41:00 -0700

I didn’t know what to think of this announcement after reading the WSJ title . After checking the project GitHub page, I still don’t know what to make of it.

Original title and link: Cloudera Announces Cloudera Developer Kit, Enabling Developers to Build Hadoop Apps Faster (NoSQL database©myNoSQL)