Site search

An up to date tutorial can be found here: http://www.jumbabox.com/2008/05/...

(from [19a89a66b5ab5900fd226feeac1c85731d7934cb]) Ignored Ferret index directory [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [52cf1ee6ac7486268d7ddc9d4986fee8cd97906a]) Removed unpublished content from search results [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [22258d77cbf4ec98fdf5f1bdbf647db7219f95cb]) Applied boosts to search fields [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [9af0deaea002938c1e35c7a8f8e56b69b3224c7f]) Small twekas to the search views [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [83fd3b61ac3ab5c0628fc31eda70487f59dcfba8]) Small improvements and fixes [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [1f93685483944750e697235eb813919fb002490b]) Removed search from navigation, it is not ready for public consumption [#11 Site search] http://github.com/k33l0r/hhlc/co...

In the long term, let's not use Ferret and AAF.

From http://blogs.nitobi.com/brian/20...

To make a long story short here’s a couple of quotes from Ezra Zygmuntowicz, of Engine Yard fame, who literally wrote the book on deploying rails applications:

Ferret is unstable in production. Segfaults, corrupted indexes galore. We’ve switched around 40 clients form ferret to sphinx and solved their problems this way. I will never use ferret again after all the problems I have seen it cause peoples production apps. Plus sphinx can reindex many many times faster then ferret and uses less cpu and memory as well.

Right about here you can hear me say in a western Canadian accent, “Sphinx eh..”. Ezra continues in a later post:

We have a bunch of clients using solr as well. In general it is more powerful then sphinx but a lot slower to reindex and querey. Also it uses 50 times the memory of sphinx. If you have a box or vm to put SOLR on by itself then it is a good option as well. but if sphinx can do everything you need from a a search indexer then it is a way better option cost wise.

[...]

The verdict: Sphinx rocks.

OK, one more choice for search: Xapian

It seems to be quite feature rich, and it supports more ddatabase systems than Sphinx. This is especially nice as it means that you don't need postgresql or mysql for development...

Locomotivation has a tutorial.

Other cool features:

• Ranked probabilistic search - important words get more weight than unimportant words, so the most relevant documents are more likely to come near the top of the results list.

• The indexer supplied can index HTML, PHP, PDF, PostScript, OpenOffice/StarOffice, OpenDocument, Microsoft Word/Excel/Powerpoint/Works, Word Perfect, AbiWord, RTF, DVI, Perl POD documentation, and plain text. Adding support for indexing other formats is easy where conversion filters are available. This indexer works using the filing system, but we also provide a script to allow the htdig web crawler to be hooked in, allowing remote sites to be searched using Omega.

• Written in C++. Perl bindings are available in the module Search::Xapian on CPAN. Java JNI bindings are included in the xapian-bindings module. We also support SWIG which can generate bindings for many languages. At present those for Python, PHP, TCL, C#, and Ruby are working.

• Highly portable - runs on Linux, Mac OS X, FreeBSD, NetBSD, OpenBSD, Solaris, HP-UX, Tru64, IRIX, and probably other Unix platforms, as well as Microsoft Windows.

• Xapian can suggest spelling corrections for user supplied queries. This is based on words which occur in the data being indexed, so works even for words which wouldn't be found in a dictionary (e.g. "xapian" would be suggested as a correct for "xapain").

• Synonyms are supported, both explicitly (e.g. "~cash") and as an automatic form of query expansion.

Just ran a quick test using the Locomotivation tutorial. Well, it seems to work.

(from [065ea8817a28c70074a30da510d570baa866f1b1]) Xapit initializer [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [0f4eb82fff94e6b1b67dbbb9edcac4c967ca33bc]) Added xapit index to ignore list [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [60e05101a3c55c3a1481330cede5749b7a83e4e0]) Final deferretting [#11 Site search] (or so I hope) http://github.com/k33l0r/hhlc/co...

(from [91babec37da63572ea151b2f3a789e5973b532fd]) Added Xapit to the readme [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [99e378a8e3e9a04ee85c9a49d4da5cbfae1b6957]) Added config option for stemming [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [e057a863fa7952d0b1ec85db8d2026ce0276532a]) Fixed news search method [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [c002ea39251f6ab001404630d37b443526359daf]) Added result count [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [be643954f35f77db2290f3ff131566356da7cab8]) Added result count and changed quotes to for the search terms [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [2f883a65d459d1b643bcfb5dccdbb90de698156f]) Removed format.xml from the Search controller [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [3f34ebec8cdbec9d6e6ace0cd29279b848b5e840]) Removed respond_tos from the search controller [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [06712c354de2640e7b7667d48a72fbd6e5bd61ef]) Added Search section to the readme [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [c9c32e8c16024e5df074887cd3a6b6ee72705925]) Added note to README [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [77cec3cfd545a93306139431adc7a324a75c5d1d]) Added meta tags to the search pages. [#11 Site search #73 Add Metadata to HEAD] http://github.com/k33l0r/hhlc/co...

(from [67a0145e04fc27a58fdf8801804819e56dc5768d]) Added setting to enable/disable search [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [fd126cf7b8c62f41dac9363c89d2a515d6365167]) Added OpenSearch support [#11 Site search] http://github.com/k33l0r/hhlc/co...

(from [675fd0fe1d22b6a87bd0f7df84b900e409aee714]) Don't load the Xapit plugin if search is disabled. [#11 Site search] http://github.com/jcxplorer/hhlc...

(from [82d7086d1ac2211ba9eebef55314eed5b073ce6a]) No xapit in models if search is disabled. [#11 Site search] http://github.com/jcxplorer/hhlc...

(from [8e7e4d4784b54f09d80c9712fea9c0a61ec6cfe8]) Tentative support for facets for articles [#11 Site search] http://github.com/k33l0r/hhlc/commit/8e7e4d4784b54f09d80c9712fea9c0...

(from [d41d0201fbc149b99c9413d2149b3710ecb1f44a]) Updated Xapit to the latest commit [#11 Site search] http://github.com/k33l0r/hhlc/commit/d41d0201fbc149b99c9413d2149b37...

Xapit-Sync should (hopefully) soon solve the search DB update problem: http://github.com/ryanb/xapit-sync/tree/master

(from [846035bfbf483347472ee33453866285b0cade51]) Updated Xapit to latest commit (v. 0.2.5) in preparation for xapit-sync [#11 Site search] http://github.com/k33l0r/hhlc/commit/846035bfbf483347472ee334538662...

Why don't we use the xapit gem instead?

Apparently it's better to use the plugin, according to Ryan.