[HACK] Lucene search add-patch

Modules, Add-ons and custom code that's more than just a quick hack or Mod.
Martin
Site Admin
Site Admin
Posts: 1854
Joined: Wed Jun 17, 2009 6:30 pm
Location: South Yorkshire UK
Contact:

[HACK] Lucene search add-patch

Post by Martin »

Title: "Sphider Plus Search" Add-Patch
Author: Martin Smallridge (WebSnail)
License: MIT
Cost: TBA
Version: 0.9.1 (revision ??)
Compatible: 5.0.6 (tested)

I've called this an "Add-Patch" because it is essentially an add-on search system that patches the existing search system rather than replacing it. It's also been patched/stuck on an Interspire Site without any of the "nice" trappings such as admin integration. It is not intended as a permanent solution because I just need it to work but it does work :).

Components Used:
  • jQuery 3.1
  • jQuery Plugin: Thickbox
  • Zend Framework (Lucene port)
  • 85 cups of tea
  • 2 nervous breakdowns + a lot of swearing
  • A total rewrite
  • a lot of luck! ;)
Appearance:
The actual performance of the Hack revolves around use of the jQuery in conjunction with the Thickbox plugin to create an Ajax search results pop over window. This displays the results over the page and provides pagination...

This is all best demonstrated here and you use it by inputting a search term and then clicking the GO> button as normal.

Search functionality:
The search engine itself uses the Lucene module which was ported into PHP by the Zend Framework project. Like the (not ditched) Sphyder+ code it can handle a number of functions but primarily it is a spidering index system that spiders one or more sites to produce a search index. There are a number of complex features that are particularly useful but the main ones are:
  • Weighting which allows you to tweak results displayed according to where they appear (title, description, etc..)
  • Indexes the WHOLE site including your CMS content, FAQ/KB, etc... (not just your product line!)
  • It's very easy to adapt as I've written the whole indexing and search code from scratch
  • The Lucene module is a useful framework that's maintained.
  • ...and the list goes on...
What it doesn't do:
Many of the features Sphider+ had available were lost, including the ability to spider media files and more importantly (to me) it also lost the ability to index and include results from other sites. I'll be writing the latter functionality back in, over the next few weeks but the trade off was definitely worth it in return for losing the mess of Sphider code.

Availability:
I'll probably be releasing this as an "as is" mod' in a month or so, on a minimum contribution basis as well as on a "installed/supported" basis for anyone who wants it.
Darntoothysam
Confirmed
Confirmed
Posts: 33
Joined: Wed Jun 17, 2009 8:47 pm

Re: [HACK] Sphider Plus search engine patch on

Post by Darntoothysam »

OOO OOO Pick me!! Pick me!!

Can I get a quote from you on install service, your time, the module, ect ect. Seriously, we're chomping at the bit to get this bugger installed and working, and we want to support you for all the hard work you've been putting into this!

Let us know, you can hit us up on the website at www.darntoothysam.com and we have a live chat option available, you can email us at sales@darntoothysam.com *though we get over 150 emails a day, I don't recommend it), you can PM us here, or call us at 952-236-7243

Thanks Martin!
Martin
Site Admin
Site Admin
Posts: 1854
Joined: Wed Jun 17, 2009 6:30 pm
Location: South Yorkshire UK
Contact:

Re: [HACK] Sphider Plus search engine patch on

Post by Martin »

I'll figure something out once my brain has figured out which way is up... :) This weekend I have free to work on a few things although I do have a couple of to-do items that can't be bumped..

I'll suss things out and drop you a line once I'm clearer on the whole process.
Martin
Site Admin
Site Admin
Posts: 1854
Joined: Wed Jun 17, 2009 6:30 pm
Location: South Yorkshire UK
Contact:

Re: [HACK] Lucene search add-patch

Post by Martin »

Just a heads up as I realised I haven't kept this thread up to date...

I ditched Sphider+ months back now and have rewritten the code using the Zend Framework port of the Lucene search module (orig: in Java).

It's not pretty but it works... The original post has been updated to include the relevant information and once I'm done tweaking it I'll release it as a mod for those with a bit more PHP skill or install it for those who need/want it (for a fee naturally).

Cheers
:)
CharlieFoxtrot
Confirmed
Confirmed
Posts: 413
Joined: Sun Aug 09, 2009 1:23 pm

Re: [HACK] Lucene search add-patch

Post by CharlieFoxtrot »

Cool add-on!

Question... do you want the search results to open in a new window on purpose? (That's fine if you do... it's just a matter of personal preference.)

QUIRKY BEHAVIOR: If the search results are more than one page (in my case, I used the word "epson" for my search) I can easily page-forward through the results. AND... if I want to view previous pages, my browser's "Back" button works as expected. --- SO FAR SO GOOD.

BUT... if I'm on the first page of the search results, and I click my browser's "Back" button again... then the ISC "Page Not Found" error is displayed.

Great bit of coding... I hope my description of the quirky error helps you fine-tune it a bit. :-)
Martin
Site Admin
Site Admin
Posts: 1854
Joined: Wed Jun 17, 2009 6:30 pm
Location: South Yorkshire UK
Contact:

Re: [HACK] Lucene search add-patch

Post by Martin »

CharlieFoxtrot wrote:Question... do you want the search results to open in a new window on purpose? (That's fine if you do... it's just a matter of personal preference.)
Well, when I first started this it was one of those things that was just easier to code as a separate Jquery-Thickbox pop-up because then I didn't have to worry about incorporating it too closely with the ISC code. I probably could rewrite/extend the original search module to use the new approach and push everything out to templates/snippets/etc... but as it's "good enough" I'm leaving it as is. It's not like I could sell it to recoup the cost of doing the extra work so I have to be pragmatic.
QUIRKY BEHAVIOR: If the search results are more than one page (in my case, I used the word "epson" for my search) I can easily page-forward through the results. AND... if I want to view previous pages, my browser's "Back" button works as expected. --- SO FAR SO GOOD.

BUT... if I'm on the first page of the search results, and I click my browser's "Back" button again... then the ISC "Page Not Found" error is displayed.
Odd... I tried to recreate the problem and the only way that seems possible is if you started off with a 404 in the first place. I'll keep an eye out for it though as it could recur when I least expect it.. thanks for the info.
Great bit of coding... I hope my description of the quirky error helps you fine-tune it a bit. :-)
Thanks for the feedback... always helps :)
screwhead
Posts: 2
Joined: Thu Sep 03, 2009 11:05 am

Re: [HACK] Lucene search add-patch

Post by screwhead »

Hi Martin,

Would this hack be compatible with version 5.0.2 once it is officially released? We're very much interested to implement a better search on our site.

Thanks,

Hector
http://www.nahel.com
Martin
Site Admin
Site Admin
Posts: 1854
Joined: Wed Jun 17, 2009 6:30 pm
Location: South Yorkshire UK
Contact:

Re: [HACK] Lucene search add-patch

Post by Martin »

screwhead wrote:Hi Martin,

Would this hack be compatible with version 5.0.2 once it is officially released? We're very much interested to implement a better search on our site.

Thanks,

Hector
http://www.nahel.com
It should be as there's only one bit of code that gets altered in a single template file... Everything else is completely independent...

You would need shell access at the moment but I'll see about a version that doesn't in the future.
screwhead
Posts: 2
Joined: Thu Sep 03, 2009 11:05 am

Re: [HACK] Lucene search add-patch

Post by screwhead »

Hi again Martin,

Any ideas how much this will cost to implement? Also, is there a possibility of having the thumbnail included in the search result?

Hector :)
Martin
Site Admin
Site Admin
Posts: 1854
Joined: Wed Jun 17, 2009 6:30 pm
Location: South Yorkshire UK
Contact:

Re: [HACK] Lucene search add-patch

Post by Martin »

screwhead wrote:Any ideas how much this will cost to implement?
Tbh... I think there's some planned changes to the code coming in 5.1 so you might want to wait... I'm happy to release it as contribution-ware though so it's up to the end user.

At the moment it'll be about a week before I have time to finish off the code as a release because I'm rushing around a lot of schools sorting out server issues, etc... now that school is starting again.
Also, is there a possibility of having the thumbnail included in the search result?
Unlikely because unfortunately it's not just products that are being pulled out..

Depending on what happens with the 5.1 release I may opt to develop it further but in truth with a baby due in October I have my doubts..
Post Reply