Building Custom Search Engine with Algolia and CakePHP 3

Building Custom Search Engine with Algolia and CakePHP 3

Building a custom search engine for your application can be tricky. Beside designing an efficient data warehouse, robust search query. We also need a well-designed user interface.

Introduction Algolia

Algolia is web service providing hosted search engine. It takes way less time to build a custom search engine with Algolia. In this tutorial, we will build a simple yet powerful search engine with Algolia in CakePHP 3.

Algolia client

To communicate with Algolia web service. We first need a client class. To swapping out client class easily in the future, we prefer a interface at first.

Create src/Utility/Algolia/Client.php file with code shown as below:

<?php
 
namespace App\Utility\Algolia;
 
interface Client
{
    public function setIndexSettings(array $settings);
 
    public function saveObject(array $object);
 
    public function saveObjects(array $objects);
 
    public function deleteObject($objectId);
 
    public function getIndexSettings();
}

Algalia provides an official PHP SDK. We will use it to create our first client class. It is basically a wrapper class.

We will need to install the official PHP SDK beforehand. It can be done easily using composer.

Run command below to install Algolia PHP SDK:

composer require algolia/algoliasearch-client-php

Next Let's create src/Utility/Algolia/AlgoliaSdkClient.php file. Code shown as below is the wrapper class:

<?php
 
namespace App\Utility\Algolia;
 
 
use AlgoliaSearch\Index;
 
class AlgoliaSdkClient implements Client
{
    /**
     * @var Index
     */
    private $index;
 
    public function __construct($appId, $apiKey, $index)
    {
        $client = new \AlgoliaSearch\Client($appId, $apiKey);
        $this->index = $client->initIndex($index);
    }
 
    public function saveObject(array $object)
    {
        return $this->index->saveObject($object);
    }
 
    public function saveObjects(array $objects)
    {
        return $this->index->saveObjects($objects);
    }
 
    public function deleteObject($objectId)
    {
        return $this->index->deleteObject($objectId);
    }
 
    public function setIndexSettings(array $settings)
    {
        return $this->index->setSettings($settings);
    }
 
    public function getIndexSettings()
    {
        return $this->index->getSettings();
    }
 
}

Next we will create another client class which we can use for testing. Because data is stored in memory, it is much faster than AlgoliaSdkClient.

Create src/Utility/Algolia/InMemoryClient.php file. Implementation of InMemoryClient class is shown as below:

<?php
 
namespace App\Utility\Algolia;
 
 
class InMemoryClient implements Client
{
    public $objects = [];
 
    public $settings = [];
 
    public function getIndexSettings()
    {
        return $this->settings;
    }
 
    public function setIndexSettings(array $settings)
    {
        $this->settings = $settings;
    }
 
    public function saveObject(array $object)
    {
        array_push($this->objects, $object);
    }
 
    public function saveObjects(array $objects)
    {
        $this->objects = array_merge($this->objects, $objects);
    }
 
    public function deleteObject($objectId)
    {
        foreach($this->objects as $i=>$object) {
            if ($object['objectId'] == $objectId) {
                unset($this->objects[$i]);
            }
        }
    }
 
}

Syncing existing data

Now we have built our client classes for both testing and production. Three steps are involved to get our application running with Algolia. And they are:

Syncing existing data.
Tweaking ranking & relevance.
Implementing search.

In this section, we will go through steps of syncing existing data.

Setup credentials

First of all, we will need to have an account at Algolia. Algolia has a free plan, which is perfect for our tutorial purpose. You can sign up a Algolia account at https://www.algolia.com.

We need at least two API keys for this tutorial. One is Search-Only API Key for front-end purpose, the other is Admin API Key, which is used by backend script to sync our application data to Algolia data warehouse. You can generate both of them at https://www.algolia.com/licensing.

Application wise, we will store Algolia configure values at config/app_local.php. Since this config file is not tracked by Git, it is perfect for storing confidential information.

Create file config/app_local.php. Placing Algolia configure values into the file. Note you will need to change values to your own set:

<?php
 
return [
    'Algolia' => [
        'appId' => 'A388CTBZWA',
        'apiKey' => [
            'backend' => '00e5f1fa24276da528c10d633729d3fe',
            'search' => '204a6848b78ea7c3cc97702cd8a9869a',
        ],
        'index' => 'tutorial',
    ]
];

Import objects

If we are using Algolia for an application with existing data set, we need to import those data beforehand. To do that, we prefer to use Cake shell script.

The Run command as shown below to generate the ImportToAlgolia shell class;

bin/cake bake shell ImportToAlgolia

By best practice, we should also have a test.

Run command as shown below to generate a test for ImportToAlgolia class:

bin/cake bake test shell ImportToAlgoliaShell

The signature of ImportToAlgolia class is as shown below:

<?php
namespace App\Shell;
 
use App\Utility\Algolia\AlgoliaSdkClient;
use App\Utility\Algolia\Client;
use Cake\Console\Shell;
use Cake\Core\Configure;
 
/**
 * ImportToAlgolia shell command.
 */
class ImportToAlgoliaShell extends Shell
{
    private $algoliaClient = null;
 
    public function main()
    {
        
    }
 
    public function setAlgoliaClient(Client $client)
    {
         
    }
 
    /**
     * @return Client
     */
    public function getAlgoliaClient()
    {
        
    }
}

We would like address two interesting methods in the class above. They are setAlgoliaClient and getAlgoliaClient, this simple combination of setter&getter methods will allow us to set default Algolia client class and change to alternative at the same time.

The implementation is shown as below:

public function setAlgoliaClient(Client $client)
{
    $this->algoliaClient = $client;
}
 
/**
 * @return Client
 */
public function getAlgoliaClient()
{
    if (null == $this->algoliaClient) {
        $this->algoliaClient =  new AlgoliaSdkClient(
            Configure::read('Algolia.appId'),
            Configure::read('Algolia.apiKey.backend'),
            Configure::read('Algolia.index')
        );
    }
    return $this->algoliaClient;
}

By TDD standard, we should write test before we implement main method.

Our ImportToAlgoliaShellTest is as simple as shown below:

<?php
namespace App\Test\TestCase\Shell;
 
use App\Shell\ImportToAlgoliaShell;
use App\Utility\Algolia\InMemoryClient;
use Cake\TestSuite\TestCase;
 
/**
 * App\Shell\ImportToAlgoliaShell Test Case
 */
class ImportToAlgoliaShellTest extends TestCase
{
 
    /**
     * ConsoleIo mock
     *
     * @var \Cake\Console\ConsoleIo|\PHPUnit_Framework_MockObject_MockObject
     */
    public $io;
 
    /**
     * Test subject
     *
     * @var \App\Shell\ImportToAlgoliaShell
     */
    public $ImportToAlgolia;
 
    /**
     * setUp method
     *
     * @return void
     */
    public function setUp()
    {
        parent::setUp();
        $this->io = $this->getMock('Cake\Console\ConsoleIo');
        $this->ImportToAlgolia = new ImportToAlgoliaShell($this->io);
    }
 
    /**
     * tearDown method
     *
     * @return void
     */
    public function tearDown()
    {
        unset($this->ImportToAlgolia);
 
        parent::tearDown();
    }
 
    /**
     * Test main method
     *
     * @return void
     */
    public function testMain()
    {
        $inMemoryClient = new InMemoryClient();
 
        $this->ImportToAlgolia->setAlgoliaClient($inMemoryClient);
        $this->ImportToAlgolia->main();
 
        $this->assertEquals(2, count($inMemoryClient->objects));
        $this->assertEquals(['title', 'description'], $inMemoryClient->getIndexSettings()['attributesToIndex']);
    }
}

For demonstration purpose only, we will not import some static data to Algolia. In reality, our data should be retrieved from some kind of persistent layer, for example database.

At last, let's implement main method. It is pretty straightforward as shown below:

public function main()
{
    $this->out('Start importing to Algolia');
 
    // Get data from Database
    $objects = [
        [
            'objectID' => 1,
            'title' => 'Resumable file upload',
            'description' => 'If your application allows users to upload large files...',
            'comment_counts' => 2,
        ],
        [
            'objectID' => 2,
            'title' => 'PHP CRUD Tutorial',
            'description' => 'Creating CRUD grid is a very common task in web development...',
            'comment_counts' => 4,
        ],
    ];
 
    $this->getAlgoliaClient()->saveObjects($objects);
 
    $this->getAlgoliaClient()->setIndexSettings([
        'attributesToIndex' => array('title', 'description'),
        'customRanking' => array('desc(commen_counts)')
    ]);
 
    $this->out('Finish importing to Algolia');
}

The complete ImportToAlgoliaShell class is as shown below:

<?php
namespace App\Shell;
 
use App\Utility\Algolia\AlgoliaSdkClient;
use App\Utility\Algolia\Client;
use Cake\Console\Shell;
use Cake\Core\Configure;
 
/**
 * ImportToAlgolia shell command.
 */
class ImportToAlgoliaShell extends Shell
{
    private $algoliaClient = null;
 
    public function main()
    {
        $this->out('Start importing to Algolia');
 
        // Get data from Database
        $objects = [
            [
                'objectID' => 1,
                'title' => 'Resumable file upload',
                'description' => 'If your application allows users to upload large files...',
                'comment_counts' => 2,
            ],
            [
                'objectID' => 2,
                'title' => 'PHP CRUD Tutorial',
                'description' => 'Creating CRUD grid is a very common task in web development...',
                'comment_counts' => 4,
            ],
        ];
 
        $this->getAlgoliaClient()->saveObjects($objects);
 
        $this->getAlgoliaClient()->setIndexSettings([
            'attributesToIndex' => array('title', 'description'),
            'customRanking' => array('desc(commen_counts)')
        ]);
 
        $this->out('Finish importing to Algolia');
    }
 
    public function setAlgoliaClient(Client $client)
    {
        $this->algoliaClient = $client;
    }
 
    /**
     * @return Client
     */
    public function getAlgoliaClient()
    {
        if (null == $this->algoliaClient) {
            $this->algoliaClient =  new AlgoliaSdkClient(
                Configure::read('Algolia.appId'),
                Configure::read('Algolia.apiKey.backend'),
                Configure::read('Algolia.index')
            );
        }
        return $this->algoliaClient;
    }
}

Finally run command as shown below to import our existing data records:

bin/cake ImportToAlgolia

Add/update/delete object

By right, we need to add/update/delete objects in Algolia whenever we do in our application. This can be achieved conveniently using CakePHP callbacks.

The client methods to use are listed below. We will not cover this in this tutorial.

Add object: Client::saveObject().
Update object: Client::saveObject().
Delete object: Client::deleteObject().

Tweaking ranking & relevance

We have setup the initial indexing and ranking parameters in ImportToAlgolia shell script. However we can always tweak the ranking later and Algolia provides a convinennt way to do so.

Login to Algolia's website, we will find a RANKING under Indices panel. It provides a drag and drop interface for us to tweak the settings for the attributes to index and custom ranking.

Implementing search

Last step of this tutorial is to actually build the user interface. Algolia provides an out of box solution for building a very robust interface. It is named instantsearch.js.

The instantsearch.js default CSS file has no particular styling. We will copy the stylesheet of their demo page:

Download the stylesheet here and place it under webroot/css/style.css.
Download the images here and place them under webroot/img.

Link the resources in the header section of src/Template/Layout/default.ctp file:

<head>
...
 
<link rel="stylesheet" type="text/css" href="//cdn.jsdelivr.net/instantsearch.js/1/instantsearch.min.css" />
<script src="//cdn.jsdelivr.net/instantsearch.js/1/instantsearch.min.js"></script>
<?= $this->Html->css('style');?>
     
...
</head>

Create a dummy controller src/Controller/SearchesController.php with code as shown below:

<?php
namespace App\Controller;
 
 
class SearchesController extends AppController
{
 
   public function index()
   {
 
   }
}

Lastly create our view file src/Template/Searches/index.ctp. Below is the code for the view file:

<?php
use Cake\Core\Configure;
?>
 
<header>
    <a href="?" title="Home"><img src="img/instant_search_logo@2x.png"/></a>
    <div id="search-input"></div>
    <div id="search-input-icon"></div>
</header>
 
<main>
    <div id="right-column">
        <div id="hits"></div>
    </div>
</main>
 
 
<script type="text/html" id="hit-template">
    <div class="hit">
        <div class="hit-content">
            <h3 class="hit-title">{{title}}</h3>
            <p class="hit-description">{{{_highlightResult.description.value}}}</p>
        </div>
    </div>
</script>
 
<script type="text/html" id="no-results-template">
    <div id="no-results-message">
        <p>We didn't find any results for the search <em>"{{query}}"</em>.</p>
        <a href="?" class='clear-all'>Clear search</a>
    </div>
</script>
 
<script>
    function getTemplate(templateName) {
        return document.querySelector('#' + templateName + '-template').innerHTML;
    }
 
    var search = instantsearch({
        appId: '<?= Configure::read('Algolia.appId');?>',
        apiKey: '<?= Configure::read('Algolia.apiKey.search');?>',
        indexName: '<?= Configure::read('Algolia.index');?>',
        urlSync: { // optionnal, activate url sync if defined
            useHash: false
        }
    });
 
    // add a searchBox widget
    search.addWidget(
        instantsearch.widgets.searchBox({
            container: '#search-input',
            placeholder: 'Search for tutorials in Star Tutorial...'
        })
    );
 
    // add a hits widget
    search.addWidget(
        instantsearch.widgets.hits({
            container: '#hits',
            hitsPerPage: 10,
            templates: {
                item: getTemplate('hit'),
                empty: getTemplate('no-results')
            },
            autoHideContainer: true
        })
    );
 
    // start
    search.start();
</script>

The instantsearch.js uses a template engine called mustache.js. For detailed usage of instantsearch.js, check out its official manual.

The End

We have used Algolia in CakePHP 3 to build a simple custom search engine. With what you have learned from this tutorial, you can expand further to build something powerful.

#Building Custom Search Engine with Algolia and CakePHP 3

#Introduction Algolia

#Algolia client

#Syncing existing data

#Setup credentials

#Import objects

#Add/update/delete object

#Tweaking ranking & relevance

#Implementing search

#The End