Comments on: Handling a huge amount of fulltext searches http://blog.mostof.it/handling-a-huge-amount-of-fulltext-searches/ Where the rising ape meets the falling angel Thu, 01 Dec 2011 08:19:51 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: J. Digle http://blog.mostof.it/handling-a-huge-amount-of-fulltext-searches/#comment-5 J. Digle Thu, 28 May 2009 06:50:39 +0000 http://blog.iamnolegend.com/?p=3#comment-5 Try SOLR. Try SOLR.

]]> By: ochronus http://blog.mostof.it/handling-a-huge-amount-of-fulltext-searches/#comment-4 ochronus Fri, 22 May 2009 20:12:09 +0000 http://blog.iamnolegend.com/?p=3#comment-4 <a href="#comment-1346" rel="nofollow">@John Haugeland</a> Generally speaking, you're absolutely right, but consider development time. In about two hours I managed to put together a system which handles around 120 req/s with a concurrency of 50 on low-grade hardware. Of course I'm not talking about facebook or twitter size load :) @John Haugeland
Generally speaking, you’re absolutely right, but consider development time. In about two hours I managed to put together a system which handles around 120 req/s with a concurrency of 50 on low-grade hardware. Of course I’m not talking about facebook or twitter size load :)

]]> By: John Haugeland http://blog.mostof.it/handling-a-huge-amount-of-fulltext-searches/#comment-3 John Haugeland Fri, 22 May 2009 19:58:03 +0000 http://blog.iamnolegend.com/?p=3#comment-3 Generally speaking, the appropriate answer for something like this is a compiled language with efficient machine-near datastructures, such as C++ or Forth, since then you can just multihash the words and keep a large bucket histograph. If you're trying to handle a "huge" amount of searches, you shouldn't be stubbing yourself by working with general-case tools. You're likely to see two orders of magnitude improvement with a well chosen string hash and a non-generalized hash table. The C++ unordered_multimap in Boost is almost certainly what you're looking for. Generally speaking, the appropriate answer for something like this is a compiled language with efficient machine-near datastructures, such as C++ or Forth, since then you can just multihash the words and keep a large bucket histograph. If you’re trying to handle a “huge” amount of searches, you shouldn’t be stubbing yourself by working with general-case tools.

You’re likely to see two orders of magnitude improvement with a well chosen string hash and a non-generalized hash table. The C++ unordered_multimap in Boost is almost certainly what you’re looking for.

]]> By: Handling a huge amount of fulltext searches part 2 - the internals | blog@iamnolegend.com http://blog.mostof.it/handling-a-huge-amount-of-fulltext-searches/#comment-2 Handling a huge amount of fulltext searches part 2 - the internals | blog@iamnolegend.com Sat, 07 Feb 2009 06:20:14 +0000 http://blog.iamnolegend.com/?p=3#comment-2 [...] In part one, I generally drafted the ingredients needed for the system, now it’s time to have a deeper look at the internal workings, processes and connections. [...] [...] In part one, I generally drafted the ingredients needed for the system, now it’s time to have a deeper look at the internal workings, processes and connections. [...]

]]>