Sphinx search engine news

Sphinx 2.0.3-Release

shodan — Thu, 29 Dec 2011 08:29:10 +0200

We are proud to announce availability of the Sphinx 2.0.3-release. This marks a milestone for Sphinx, not only because it's our first stable release since 0.9.9, but includes stable/battle-tested Real-Time Indexes, 64-bit MVA support, expression-based rankers, keywords dictionary plus many other features and improvements mentioned in the 2.0.2-beta release and described further in our changelog.

All the new features in version 2.0.3 are enterprise grade and fully covered by our support packages. From this point forward the 2.0 branch will receive priority bugfixes therefore we recommend upgrading to this version before filing any bugreports. In addition, we will push new features to the beta branch as soon as they are stable enough for real-world applications. But, if you are fascinated (like we are) with bright, shiny, and new things you can always check out our current development version.

On behalf of the Sphinx Team we would like to thank you for use of the Sphinx Engine and wish you a Happy New Year!

Sphinx 2.0.2-beta is out

shodan — Wed, 16 Nov 2011 18:27:38 +0200

Sphinx version 2.0.2-beta is now out, with over 30 new features, options, and other significant changes. It also marks a feature freeze of the 2.0.x branch, and 2.0.3-release is scheduled shortly.

The most important new features in 2.0.2-beta are the new MVA64 attribute type, dict=keywords and MVA support in RT indexes, expression-based ranker, ATTACH INDEX statement, and WHERE support in UPDATE statement.

Read on for a quick discussion of the biggest new ones and the planned release cycle, or just proceed to Downloads and grab it already!

MVA64 attributes are a further extension of our MVA feature. MVAs let you store a set of unsigned 32-bit values, MVA64 now enables signed 64-bit values in addition to that. This is useful to avoid CRC32 collisions when hashing string tags, or to encode extra auxiliary data into your MVA. They're supported in both disk and RT indexes.

Word start (prefix) searching in RT indexes is now supported through the dict=keywords feature. We deliberately chose to not implement prefix and infix pre-indexing from disk indexes in RT, because that would have been a huge memory hog. Substring (infix) searching support with dict=keywords is in development now, and schedule for 2.1.x branch.

MVA and index_exact_words in RT indexes are also supported now.

The new ATTACH INDEX statement is important because it lets you quickly convert from disk to RT index. That's right, you now can quickly batch index the main bulk of your data, then easily switch to RT, and keep updating the RT index. Right now, ATTACH only lets you convert the index once, and requires an empty RT target. However, enabling batch imports into existing, non-empty RT indexes is planned. That is why we chose to reserve ATTACH upfront instead of something like CONVERT.

UPDATE statement now supports full-blown conditions in WHERE, both with disk and RT indexes, too. So, say, updating prices on a 1000 rows from vendor X or just marking them for deletion just became 1000 times easier. Just like attributes UPDATEs themselves, this should also work in both disk and RT backends.

And last but not least, you can now create your own relevance formulas on the fly with the aid of the expression ranker. Previously, computing relevance values was limited to a number of built-in rankers, and changing the formula that combined various internal ranking factors (not to be confused with attributes) involved writing C++ code, and rebuilding and restarting searchd. From now on, you only need to specify a short and sweet formula, you can do on the fly on a per-query basis, and many more ranking factors are now available than we ever computed before. Amusingly, this is not even super slow: on my smallish 1,000,000-document test collection, emulating the default ranker with the expression based one was just 1.1x to 1.3x slower than the compiled C++ code.

Besides the new features, we have also been busy improving our internal testing, build, and release processes. And the next release, 2.0.3-release is now scheduled in approximately 1 month from now, as opposed to the long standing tradition of shipping it when it's done. We did a lot of testing for 2.0.2-beta, there are no known major issues, and the pre-existing functionality must be more stable than ever. However, our policy is not to tag it "release" until it received even more testing from the community. Thus, if you bump into anything in 2.0.2-beta, please be sure to report the issue! The earlier you report it, the sooner it gets fixed.

From there, the release plan is as follows. 2.0.x branch is now frozen and will only receive bug fixes. So after 2.0.3-release we are going to publish bugfix releases on regular basis. The specific intervals are going to be anywhere between 1 and 3 months, depending on the number and severity of reported and fixed issues.

Overall, we added a bunch of cool new stuff in 2.0.2, 2.0.x branch is now feature frozen and will be supported from here, there's more new exciting stuff coming in 2.1.x, and this post is getting long. So grab a new version, give it a whirl, and be sure to report any issues so we could work on them for the forthcoming 2.0.3. And thanks for choosing Sphinx!

Partner Page Live!

shodan — Sat, 10 Sep 2011 00:53:18 +0200

Today we are pleased to announce the launch of the Sphinx Partner Page. We are extremely excited to highlight those we work with to deliver complete search solutions. Our current Partners include Percona, Ivinco, SkySQL, Thinking/Flying Sphinx, and MariaDB. With these companies we endeavor to provide our customers and community with the services and products needed not only to implement Sphinx, but to provide seamless database and front-end support. If you are interested in joining our Partnership Program or working with one of our Partners please Contact Us.

Customers portal has been upgraded

shodan — Fri, 05 Aug 2011 17:08:19 +0200

Customers portal is now live and operational.

Thank you for the patience!

Vladimir Fedorkov,
Director of Professional Services

Customers portal upgrade

shodan — Fri, 05 Aug 2011 14:10:31 +0200

Sphinx Search customer portal news.

We're going to upgrade software on our customer portal in few hours from now. customers.sphinxsearch.com will be temporary unavailable today until approx 11:30am EST. We apologize for the inconvenience. If you are existing customer and you have an urgent request for us during customers.sphinxsearch.com unavailability - please contact me directly or use our emergency line.

Vladimir Fedorkov,
Director of Professional Services

MVA support is now available in Real-Time indexes!

shodan — Wed, 08 Jun 2011 15:19:10 +0200

We've been recently asked about MVA (Multi Value Attribute) support in Sphinx Real-Time indexes on NYPHP meetup.

As promised we're letting you know: MVA support is just committed to Sphinx code base!

If you'd like to be early adopter and help Sphinx community with testing - please feel free to download current trunk using:

svn checkout http://sphinxsearch.googlecode.com/svn/trunk

Share your experience with MVA attributes in RT indexes in our forum!

Sphinx Search Book is out!

shodan — Thu, 21 Apr 2011 12:31:01 +0200

A book about Sphinx Search Server from it's creator is now available!

You can get your own copy of "Introduction to Search with Sphinx" ebook right now (and/or pre-order a printed copy) from O'Reilly website.

It is also additionally available from Amazon.com

Enjoy!

Sphinx 1.10-beta is out

shodan — Mon, 19 Jul 2010 19:35:31 +0200

We're happy to release Sphinx 1.10-beta, with a new major version number that means, as promised, real-time indexes support.

In addition to RT indexes, it also brings preforked and threaded modes, string attributes, full SphinxQL support (with access to all searchd functions), advanced features such as hitless and blended indexing, optimized index format, and a lot more. A complete list of new 1.10-beta features is available in documentation.

How stable are RT indexes and can one use those in production? Pretty much, especially when backed with a Full or Premium support package to guarantee priority fixes and emergency support respectively. RT indexes are already deployed in production by several early adopters, and are reported to work well. Even though we've been focusing on functionality and stability rather than performance for this release, the latter is also reasonable.

And that's exactly what that "beta" tag is supposed to mean: while there are things missing from RT indexes, and might be bugs found in both RT and other new features, no major issues are however known. Plus the pre-existing features should be as stable as ever.

And yes, with SphinxQL, RT indexes, transactions, crash recovery and other goodies Sphinx now feels almost real close to a database. Except that our full-text searches are still fast!

Talking, blogging, coding, and meeting

shodan — Tue, 23 Mar 2010 01:11:31 +0200

My talk on Sphinx has been accepted for MySQL User Conference 2010, so we're hitting the conference again! The talk is called Sphinx: Full-text Search in 2010 and I will be covering all the shiny new features that we've been so busy adding lately. The talk is scheduled on Wednesday, April 14th and we'll also try to arrange a BOF session on search in general and Sphinx in particular on Tuesday, 13th.

We are not doing a booth this year, so if you'd like to meet either myself, Andrew Aksyonoff (technical honcho, fingers behind most of the code), or Richard Kelm (sales/customer support honcho, voice behind that 888 number), please do get in touch upfront. I'm staying for a few more days in SF Bay area so we're not really bound to meeting at conference site on the conference dates.

Speaking of which, that also might be a good occasion for a quick onsite consulting visit either in SF Bay or generally in the US. If you have any questions regarding this or would like to schedule a visit please feel free to call 888-333-1345 or email Richard via

And for those who can't attend MySQL UC and hear us talking, we're now blogging. News section on the frontpage isn't a really properly fitting format for everything that can be shared, which is probably one of the reasons it's updated so infrequently. So check out our new Sphinx blog for more things Sphinx: articles, tips, general musings, fun facts, you name it. The blog even comes prefilled.

Last but not least, despite the announcements, we're still mostly busy coding rather talking or blogging most of the time. And those activities tend to offset other ones (like documenting and packaging) lately. Which is, for instance, why real-time updates are in fact already available from public SVN trunk but not in a tarball or RPM form just yet.

So do get in touch. Whether it's attending a talk, following a blog, meeting at the conference, or volunteering to help us with something, we're looking forward to meeting you!

Sphinx 0.9.9-release is out!

shodan — Wed, 02 Dec 2009 16:12:03 +0200

Super-long incubation period from "rc2" to "release" tag did not, of course, result in any new features, because of a feature-freeze. We did add a few missing calls and features to APIs and SphinxSE and also a couple of optimizations but that's catching up, not really adding new stuff. On the other hand, looking back at 40+ bugs fixed in this release, I would still say that rc2 was pretty stable, as most of the bugs were either minor, or rare, or both.

Most importantly, index merging is now much more stable. Interesting that it's been around for quite a while, but many bug reports were only filed against 0.9.9 or even 0.9.10 (and then ported back). Perhaps most of you are only adopting it now. So note that 0.9.9-release is an officially recommended occasion to re-evaluate merging if you were having issues with it before.

Another major change worth a mention on the front page isn't directly related to the code, though. Old news to those dedicated people who follow our commits on Googlecode or #sphinxsearch on Twitter, and new news to everyone else, Sphinx now has official IANA assigned ports, 9312 for native API and 9306 for SphinxQL, so update your sphinx.conf and /etc/services. (On a side note, does anyone know what is this appman-server thing that owns port 3312? I'm still occasionally curious about that.)

Along with the new release today, we're also renewing our Consulting Services. We've added a number of services such as Installation Jumpstart that, in simple cases, can be performed within a fixed cost and time frame. So if you require assistance with upgrading (or switching) to 0.9.9-release then our services just got a few more options to consider. You'd expect us to have something for early adopters and, of course, we do. All our Support Packages will be discounted 10% off for those who purchase our consulting services by January 31, 2010. If you have any questions, comments, or suggestions regarding the services, feel free to email us at sales@sphinxsearch.com or call +1-888-333-1345.

And, while you're busy upgrading, we'll probably be busy building 1.x-alpha with incremental (real-time) updates for Christmas.