In our latest update we pushed out a new version of our search engine which has resulted in faster and more relevant search results.

Back in the olden days when welovelocal was still a concept and we were planning the development, we looked at various “out of the box” search tools. There are quite a few available and in the end we decided that we would use the full-text functionality built into the database system, MySQL.

Whether this could be classed as “out of the box” or not is subject to debate but it is by no means a custom solution. Full-text is ideal for searching a lot of large textual database records having the database engine itself handle the search queries. It allows for some fancy things such as natural language search and query expansion where it does two passes to find words relevant to the first term. For it to work we need text for each business that allows it to be identified when the user enters keywords in the search - it is this text that forms part of the index that is searched on. To achieve this, we created keywords for every category meaning businesses within that category would inherit those particular keywords and that the full text index could be built on.

This worked alright without any tweaking but it was soon clear that we needed to make a lot of minor changes to the search code to get the most accurate results. This ranged from removing some more common keywords to hard coding some queries. In addition, we wanted to be able to allow you to search for a company name and have it match exactly or even offer partial matches.

We did not make our own search from the outset because for most things, it is always better to use something someone else has created if it is good enough. Often a dedicated “product” will be refined in more ways with more time and resources spent on it than can be done by someone creating their own solution. This is why, for example, you would not usually write your own blogging software and instead you would use one of the many tools available. It was also thought that in theory, full-text would be able to deliver the results we wanted.

It soon became clear that although it was providing some decent results some of the time, the code was becoming messy due to the hard coded keywords and the results returned by MySQL were not relevant enough. At the heart of this problem was the simple fact that full-text searches for individual words (unless you use boolean searching of course, but we could not force users to do that!) were being ranked in isolation with no consideration for phrase matches. If you searched for italian restaurant then it would find both words and give them equal relevancy ranking. Although this was fine for bigger searches that return a lot of results, searching for something like car repairs would show a lot of entries with “car” in the keywords.

So after much deliberation we decided to ditch full-text in favour of our own searching system which allows for the weighting of keyword matches not possible with full-text. This is combined with the company name exact and partial matching we had already coded to provide you with keyword and company search functionality.

Whilst we’re not as extensive as Google which uses over 200 variables to determine the ranking of results on the page, we factor in a number of different things to try and provide you with the best results for your search. As we get more data, we will be able to further refine the search and even personalise it to your tastes.

tags: ask, evolve your search, search, local search