Indexer Guide

The Sophora Indexer is connected to the Sophora Primary Server's ContentManager and receives notifications when structure nodes or documents are changed. How this applies to the connected search engine is defined through an indexer plugin.

The Indexer holds a priority update queue to handle state change events and to manage resulting document operations with the following priorities:

The indexer is separated into the projects com.subshell.sophora.Indexer and com.subshell.sophora.indexer.api. It is started by the class com.subshell.sophora.indexer.Indexer.

Directory Structure

As described in the Sophora Server Documentation it is recommended to use a certain directory structure for an installation of the Sophora Indexer.

The internal arrangement of files and directories within the tar.gz archive file, in which the indexer application is assembled, encourages the use of this structure.

----cms-directory
--------apps
------------...
------------com.subshell.sophora.indexer-2.3.0
------------...
------------sophora-indexer -> Symbolic link to com.subshell.sophora.indexer-2.3.0
------------...
--------indexer
------------config
----------------indexer.properties
------------groovy
------------logs
------------plugins
------------sophora-indexer.sh -> Symbolic link to ../apps/sophora-indexer/indexer.sh

apps – The directory apps contains the software components used in your Sophora environment. In this case the Sophora Indexer application is located in the subdirectory com.subshell.sophora.indexer-2.3. A symbolic link points to this diretory in order to enable an easy change of different versions of the indexer - e.g. in case of an update.

indexer - This folder is the workspace of the indexer installation. It contains the indexer's configuration file (in the subdirectory config), the log files (in the subdirectory logs), the plugin to use (in the subdirectory plugins), groovy scripts for providing field values, and a symbolic link to the start and stop script. Except for the logs directory, which is created automatically after the first execution, this directory including its subdirectories and the symbolic link must be created manually.

plugins - Place an indexer plugin jar of your choice in this directory. The plugin will then be made available for the indexer application at runtime. See the plugin mechanism section for more information.

Configuration

The following sections describe how to configure the Sophora Indexer for your needs.

Configuration file (sophora.properties)

The Indexer's behaviour is defined by a configuration file. This file is mandatory. At the start it is handed to the Indexer as VM argument. The syntax to specify the configuration file is as follows:

-Dsophora.properties=<path to properties file>

Properties from the configuration file overwrite those from the default configuration. To apply changes in this file the Indexer needs to be restarted.

Exemplary configuration

# Connection to the ContentManager
sophora.contentmanager.serviceUrl=rmi://localhost:1199/ContentManager
sophora.contentmanager.username=admin
sophora.contentmanager.password=admin
 
# JMX settings
sophora.indexer.jmx.registry.port=50
sophora.indexer.rmi.registry.port=5031
sophora.indexer.jmx.registry.username=admin
sophora.indexer.jmx.registry.password=secret
 
# Query for the synchronisation after restarting (inactive)
#sophora.replication.restartQuery=element(*, sophora-mix:document)[@sophora:id = 'test100']
 
# Starting date for the synchronisation
sophora.replication.restartDate=2015.03.27 12:00
sophora.subsearch.fullUpdate=false
 
# Selected connection
sophora.searchEngine.connection=dummySubsearch
 
# Search mixins and fields
sophora.indexer.searchMixinName=sophora-content-mix:searchable
sophora.indexer.unsearchableFieldName=sophora-content:unsearchable
 
# Directory for XSL files to transform property values
sophora.indexer.directory.xsl=c:/temp

Updating the Index on Startup

Mapping Document Properties to Index Fields of the Search Engine

The mapping of Sophora properties to index fields of the search engine is done in the siteAndMappingConfiguration.xml file. This file has to be created and placed next to the sophora.properties file in the same directory. If you apply changes to this configuration file, the Indexer needs to be restarted for changes to take effect.

The following example demonstrates all supported use cases and possible configurations. The XML scheme file can be downloaded here: indexer-configuration-1.0.0.xsd

Example of siteAndMappingConfiguration.xml

<?xml version="1.0" encoding="UTF-8"?>
<configuration xmlns="http://www.sophoracms.com/indexer-configuration/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://www.sophoracms.com/indexer-configuration/1.0 http://www.sophoracms.com/indexer-configuration/1.0/indexer-configuration-1.0.0.xsd">
     
    <!-- Assign index keys to sites and filters -->
    <indexes>   
        <index indexKey="indexKey1" isDefault="true">
            <sites>
                <!-- name is optional, id is required. Use the uuid of the structure node or the externalid of the structure node document to identify a structure node. -->
                <site name="sitename1" id="5c34195a-5574-4948-9b72-bc1df857fb8a" />
            </sites>
            <filter>
                <allowedNodeTypes>
                    <allowedNodeType>sophora-content-nt:story</allowedNodeType>
                </allowedNodeTypes>
                <requiredChannel>c0970f7e-85e6-412b-9e52-27073ca84e58</requiredChannel>
                <requiredProperty>sophora-content:topline</requiredProperty>
            </filter>
        </index>
        <index indexKey="indexKey2">
            <sites>
                <site name="sitename2" id="32dc2576-93b4-407d-a971-c2c6a437d7fc" />
                <site name="sitename3" id="b6d3cf76-b114-4e6c-95da-2033a413c08e" />
                <site name="aStructureNodeOfSite3" id="c7ad6486-b35b-4e16-9cec-c3b26332580d" />
                <site name="anotherStructureNodeOfSite3" id="05621d7b-585d-471d-8604-538c0b315880" />
            </sites>
            <filter>
                <allowedNodeTypes>
                    <allowedNodeType>sophora-content-nt:audio</allowedNodeType>
                    <allowedNodeType>sophora-content-nt:story</allowedNodeType>
                </allowedNodeTypes>
            </filter>
        </index>
    </indexes>
     
    <!-- Assign index key fields to Sophora document properties -->
    <mappings>
        <!-- The simpliest mapping is to assign a Sophora property to a search engine's index field -->
        <mapping key="sophoraid">
            <property>sophora:id</property>
        </mapping>
         
        <!-- You can set the format of date properties as they will appear in the search engine's index.
             For instance, if you want a property called "dateToSearch" to appear in the format "yyyy.MM.dd.HH.mm.ss",
             add the following lines to the configuration.
             In most cases the format "yyyy.MM.dd.HH.mm.ss" is used as default value. Exceptions are the properties
             "dateToSearch" and "publicationDate" which occure in "yyyy.MM.dd" by default. -->
        <mapping key="dateToSearch" format="yyyy.MM.dd.HH.mm.ss">
            <property>sophora:publicationDate</property>
        </mapping>
         
        <!-- Selectvalues: Without special configuration the selected key of a drop-down list is indexed.
             If the mapped property is configured as a select value, it is possible to write the label of the
             selected key into the index field. This is achieved by appending ".value" to the property name: -->
        <mapping key="selectedValue">
            <property>sophora:dropdownField.value</property>
        </mapping>
         
          <!-- raw property value: To map the property value without replacing all html/xml tags, you can append a ".rawValue" to the property name: -->
          <mapping key="rawPropertyValue">
          	<property>sophora:property.rawValue</property>
          </mapping>
          
        <!-- For properties containing XML data, an XSL transformation can be performed.
             The result of the transformation is then written into the index field.
             The XSL file name is given by the attribute "xsl".
             The file name is relative to the directory set in the property "sophora.indexer.directory.xsl" of the sophora.properties file. -->
        <mapping key="longitude" xsl="longitude.xsl">
            <property>sophora-content:map</property>
        </mapping>
         
        <!-- In order to write the content of a childnode into an index field use path expressions like these: -->
        <mapping key="teaserImageOverwrittenAlttext">
            <property>sophora-content:image/sophora-extension:alttext</property>
        </mapping>
        <mapping key="teaserImageUuid">
            <property>sophora-content:image/sophora:reference</property>
        </mapping>
        <mapping key="teaserCopytextImageUuid">
            <property>sophora-content:copytext/sophora-extension:paragraph/sophora-extension:paragraphimage[0]/sophora-extension:image[0]/sophora:reference</property>
        </mapping>
         
        <!-- If you want to insert multiple property values into a single index field, you can define these properties as a list.
             As you can see, each part of the property list can also be a path expression which refers to childnode values.
             Such a configuration will result in a single index field value, where the assigned values are separated by a white space character. -->
        <mapping key="sequence">
            <property>sophora-content:topline</property>
            <property>sophora-content:headline</property>
            <property>sophora-content:teasertext</property>
            <property>sophora-content:image/sophora-extension:alttext</property>
        </mapping>
         
        <!-- It is also possible to configure alternative properties, if the intented property of a document is empty. You can also define multiple alternatives.
             Example: If the 'sophora-content:date property' is not set, the value of 'sophora:publicationDate' is taken alternatively and so on: -->
        <mapping key="date" format="yyyy.MM.dd.HH.mm">
            <alternative>
                <property>sophora-content:date</property>
                <property>sophora:publicationDate</property>
                <property>sophora:modificationDate</property>
            </alternative>
        </mapping>
         
        <!-- Define multiple properties into a single index field, containing alternatives: -->
        <mapping key="sequenceWithAlternatives">
            <alternative>
                <property>sophora-content:topline</property>
                <property>sophora-content:headline</property>
            </alternative>
            <property>sophora-content:teasertext</property>
            <alternative>
                <property>sophora-content:date</property>
                <property>sophora:publicationDate</property>
                <property>sophora:modificationDate</property>
            </alternative>
        </mapping>
         
        <!-- If a property is not available for a document, it may be retrieved from the structure hierarchy. To do so, you can use the following expression.
             The example would be interpreted as follows: If the property "sophora:property" does not exist in the document that should be indexed,
             the indexer tries to retrieve it from the structure node documents of the parent structure nodes. -->
        <mapping key="field">
            <alternative>
                <property>sophora:property</property>
                <operation>sophora.indexer.getPropertyValueFromStructureHierarchy</operation>
            </alternative>
        </mapping>
         
        <!-- To write the UUIDs of the active channels of a document into a single index field,
             you need to configure the mapping property accordingly. The index field's content will be a space separated list of UUIDs like:
             e91c87d4-8e16-4e40-9d69-689548efe5ab f833f3f6-b894-4064-9d10-eadb244a52cf. The list includes the default structure hierarchy information;
             e.g. if no channels are defined for this document, it might inherit some from the parent nodes. -->
        <mapping key="channels">
            <operation>sophora.indexer.activeChannels</operation>
        </mapping>
              
        <!-- Another possibility to use information from a document's structure is to determine the structure nodes' UUIDs by defining the subsequent mapping:  -->
        <mapping key="structureNodes">
            <operation>sophora.indexer.generateStructurePathUuids</operation>
        </mapping>
         
 
        <!-- If you want to generate the URL of the document that is indexed, configure the following:  -->
        <mapping key="generatedUrl">
            <operation>sophora.indexer.generateUrl</operation>
        </mapping>
        <!-- Or alternatively, only generate an url if the document doesn't provide one: -->       
        <mapping key="url">
            <alternative>
                <property>sophora-content:url</property>
                <operation>sophora.indexer.generateUrl</operation>
            </alternative>
        </mapping>
 
        <!-- It is possible to provide values using classes defined in Groovy or Java. The content
             of the <operation>-element is the fully qualified classname of a class implementing
             the interface IFieldValueSource. -->
        <mapping key="groovyFoo">
            <operation>Foo</operation>
        </mapping>
        <mapping key="groovyAlternative">
            <alternative>
                <operation>Foo</operation>
                <operation>mycompany.Bar</operation>
                <operation>sophora.indexer.generateUrl</operation>
            </alternative>
        </mapping>
    </mappings>
</configuration>

Built-in field operations

Using Groovy scripts to implement custom operations

Operations providing field values can be implemented using Groovy scripts. The scripts must be located in the groovy-directory next to the config-directory. Each script must define a class that implements the interface com.subshell.sophora.indexer.source.IFieldValueSource. A custom operation implented by a script is referenced in the mapping using the fully qualified name of the class defined by the script.

The following example defines an operation which sets the value of the field "structureNodeName" to the name of the structure node, where the document to be indexed is located.

siteAndMappingConfiguration.xml

<mapping key="structureNodeName">    <operation>GetStructureNodeName</operation></mapping>

groovy/GetStructureNodeName.groovy

import com.subshell.sophora.api.content.INode;
import com.subshell.sophora.api.structure.StructureInfo
import com.subshell.sophora.client.ISophoraClient;
import com.subshell.sophora.indexer.api.IFieldValueSource;
 
class GetStructureNodeName implements IFieldValueSource {
    private ISophoraClient client;
     
    @Override
    public void setClient(ISophoraClient client) {
        this.client = client;
    }
     
    @Override
    public String getValue(INode document, String fieldName) {
        def structure = client.getStructureInfo(document.getString("sophora:structureNode"))
        return structure.getStructureNodeName()
    }
}

Deprecated: Mapping Document Properties to Index Fields of the Search Engine in the sophora.properties File

If the siteAndMappingConfiguration.xml file does not exist, the mapping of Sophora properties to index fields of the search engine is read from the sophora.properties file.

The following properties can be set in the sophora.properties file to configure the mapping:

Example mapping configuration in sophora.properties file (same configuration as in siteAndMappingConfiguration.xml explained above):

Mapping in sophora.properties

# -----------------------------
# Configure index keys to sites
# -----------------------------
sophora.indexer.sites=sitename1,sitename2,sitename3,structureNodeOfSite3,anotherStructureNodeOfSite3
 
sophora.indexer.site.sitename1.id=5c34195a-5574-4948-9b72-bc1df857fb8a
sophora.indexer.site.sitename1.indexkey=indexKey1
 
sophora.indexer.site.sitename2.id=32dc2576-93b4-407d-a971-c2c6a437d7fc
sophora.indexer.site.sitename2.indexkey=indexKey2
 
sophora.indexer.site.sitename3.id=b6d3cf76-b114-4e6c-95da-2033a413c08e
sophora.indexer.site.sitename3.indexkey=indexKey2
 
sophora.indexer.site.aStructureNodeOfSite3.id=c7ad6486-b35b-4e16-9cec-c3b26332580d
sophora.indexer.site.aStructureNodeOfSite3.indexkey=indexKey2
 
sophora.indexer.site.anotherStructurNodeOfSite3.id=05621d7b-585d-471d-8604-538c0b315880
sophora.indexer.site.anotherStructurNodeOfSite3.indexkey=indexKey2
 
# If the site UUID is empty
sophora.indexer.site.default.indexkey=indexKey1
 
# -----------------
# Configure filters
# -----------------
sophora.indexer.filter.indexKey1.allowedTypes=sophora-content-nt:story
sophora.indexer.filter.indexKey1.requiredChannel=c0970f7e-85e6-412b-9e52-27073ca84e58
sophora.indexer.filter.indexKey1.requiredProperty=sophora-content:topline
 
sophora.indexer.filter.indexKey2.allowedTypes=sophora-content-nt:audio,sophora-content-nt:story
 
# ---------------------------------------------------------
# Configure index key fields to Sophora document properties
# ---------------------------------------------------------
# Each mapping of Sophora properties to index fields of the search engine is done in the following way:
#  mapping.SEARCH_ENGINE_INDEXFIELD=SOPHRA_PROPERTY_EXPRESSION
 
# The simpliest mapping is to assign a Sophora property to a search engine's index field:
mapping.sophoraid=sophora:id
 
# You can set the format of date properties as they will appear in the search engine's index.
# For instance, if you want a property called "dateToSearch" to appear in the format "yyyy.MM.dd.HH.mm.ss",
# add the following in the configuration.
# In most cases the format "yyyy.MM.dd.HH.mm.ss" is used as default value. Exceptions are the properties
# "dateToSearch" and "publicationDate" which occure in "yyyy.MM.dd" by default.
mapping.dateToSearch=sophora:publicationDate
dateFormat.dateToSearch=yyyy.MM.dd.HH.mm.ss
 
# Selectvalues: Without special configuration the selected key of a drop-down list is indexed.
# If the mapped property is configured as a select value, it is possible to write the label of the
# selected key into the index field. This is achieved by appending ".value" to the property name:
mapping.selectedValue=sophora:dropdownField.value

# raw property value: To map the property value without replacing all html/xml tags, you can append a ".rawValue" to the property name: 
mapping.rawPropertyValue=sophora:property.rawValue
 
# For properties containing XML data, an XSL transformation can be performed.
# The result of the transformation is then written to the index field.
# The XSL file name is given in a property with the name "mapping.<field>.xsl".
# The file name is relative to the directory set in the property "sophora.indexer.directory.xsl".
sophora.indexer.directory.xsl=c:/temp
mapping.longitude=sophora-content:map
mapping.longitude.xsl=longitude.xsl
 
# In order to write the content of a childnode into an index field use a path expression like:
mapping.teaserImageOverwrittenAlttext=sophora-content:image/sophora-extension:alttext
mapping.teaserImageUuid=sophora-content:image/sophora:reference
mapping.teaserCopytextImageUuid=sophora-content:copytext/sophora-extension:paragraph/sophora-extension:paragraphimage[0]/sophora-extension:image[0]/sophora:reference
 
# If you want to insert multiple property values into a single index field, you can define these properties as list.
# As you can see each part of the property list can also be a path expression to refer to childnode values.
# Such a configuration will result in a single index field value where the assigned values are separated by a white space character.
mapping.sequence=sophora-content:topline,sophora-content:headline,sophora-content:teasertext,sophora-content:image/sophora-extension:alttext
 
# It is also possible to configure alternative properties, if the intented property of a document is empty.
# This is achieved using the delimiter "|". You can also define multiple alternatives.
# Example: If the sophora-content:date property is not set, the value of sophora:publicationDate is taken alternatively and so on:
mapping.date=sophora-content:date|sophora:publicationDate|sophora:modificationDate
dateFormat.date=yyyy.MM.dd.HH.mm
 
# NOTE: The combination of "," and "|" within one mapping expression is allowed. A combination of "/" and "|" on the contrary is prohibited.
# Define multiple properties into a single index field, containing alternatives:
mapping.sequenceWithAlternatives=sophora-content:topline|sophora-content:headline,sophora-content:teasertext,sophora-content:date|sophora:publicationDate|sophora:modificationDate
 
# If a property is not available for a document, it may be retrieved from the structure hierarchy. To do so add the following expression.
# The example would be interpreted as follows: If the property "sophora:property" does not exist in the document that should be indexed,
# the indexer tries to retrieve it from the structure node documents of the superior structure nodes.
mapping.field=sophora:property|sophora.indexer.getPropertyValueFromStructureHierarchy
 
# To write the UUIDs of the active channels of a document (including default structure hierarchy information;
# e.g. if no channels are defined for this document, it might inherit some from the superior nodes) into a single index field,
# you need to configure the mapping property accordingly. The index field's content will be a space separated list of UUIDs like:
# e91c87d4-8e16-4e40-9d69-689548efe5ab f833f3f6-b894-4064-9d10-eadb244a52cf
mapping.channels=sophora.indexer.activeChannels
 
# Another possibility to use information from the structure is to determine the structure nodes' UUIDs by defining the subsequent mapping:
mapping.structureNodes=sophora.indexer.generateStructurePathUuids
 
# If you want to generate the URL of the document that is indexed, configure the following:
mapping.generatedUrl=sophora.indexer.generateUrl
# Or alternatively, only generate an url if the document doesn't provide one:       
mapping.url=sophora-content:url|sophora.indexer.generateUrl

Generating URLs for Documents

If you want to generate the URL of the document that is indexed, using the sophora.indexer.generateUrl operation, the URL is generated from these parameters by default:

• URL configured for the site (e.g.http://www.sophoracms.com),
• name of the structure node from the structure path (e.g. "home"),
• the Sophora ID (e.g. "sophoraid100") and
• the extension ".html"

Example: http://www.sophoracms.com/home/sophoraid100.html

Using a Web Service

Instead of using the built-in algorithm, URLs for indexed documents can also be queried from a web service. To use this feature, you need to set the sophora.indexer.urlService.url property in the configuration to the URL of the web service. The web service will be given the UUID of the indexed document as the HTTP-GET parameter "uuid" and must return the URL of the document as plain text.

The indexer sets the following HTTP-GET parameters:

• uuid: The UUID of the document for which the web service should return the URL.
• modificationDate: The modification date of the document as milliseconds since the epoch (UTC). This is used for checking that the indexer and the web service have the same version of the document.

The following example shows the interaction between indexer and url-service for one document:

sophora.properties of indexer

sophora.indexer.urlService.url=http://mydomain.de/system/servlet/urlService.servlet

sophora.properties of web application

sophora.delivery.site.demosite.domain=http://mydomain.de

UUID of the indexed document: a2acc8f7-e2c3-4180-ada1-4fc4794453c9

Request by the indexer: http://mydomain.de/system/servlet/urlService.servlet?
uuid=a2acc8f7-e2c3-4180-ada1-4fc4794453c9&modificationDate=1319720134811

Response by the webservice: http://mydomain.de/demosite/news/news104.html

It is necessary to configure the service within your web application accordingly. Therefore you have to make sure the servlet is set up in the web.xml as follows.

web.xml

[...]
<servlet>
  <servlet-name>urlServlet</servlet-name>
  <servlet-class>com.subshell.sophora.delivery.servlet.UrlForIdServlet</servlet-class>
</servlet>
 
<servlet-mapping>
  <servlet-name>urlServlet</servlet-name>
  <url-pattern>/system/servlet/urlService.servlet</url-pattern>
</servlet-mapping>
[...]

Besides the UUID, the servlet may take additional parameters for URL creation:

• type: defines the template type to be included within the URL. Default: 'default'
• suffix: file suffix to use for the URL creation. Default: 'html'.
• modificationDate: The modification date of the document as milliseconds since the epoch (UTC). This is used for checking that the indexer and the servlet have the same version of the document. If the modification date differs, an error is returned.
• channel: The URL is created for the given channel. Default is the default channel. Do not use together with domainProperty.
• domainProperty: The domain may be optionally determined via a property, which is passed through this parameter to the servlet. The value of the property is read from the siteproperties. Do not use together with channel.

Whereas the uuid parameter is set by the indexer, all other parameters can be set for project specific adjustments. In this case a JSP template has to be called instead of invoking the servlet directly. This JSP sets the parameters and then redirects the call to the serlvet. See the following example for a JSP template which automatically sets the file suffix based on the mimetype of the binary data of a document.

indexer.jsp

<%@ page session="false" pageEncoding="utf-8" contentType="text/html; charset=UTF-8"%>
<%@ taglib uri="http://java.sun.com/jsp/jstl/core" prefix="c" %>
<%@ taglib tagdir="/WEB-INF/tags/sophora-commons" prefix="sc"%>
<%@ taglib uri="http://www.subshell.com/sophora/jsp" prefix="sophora" %>
 
<c:if test="${not empty param.uuid}">
    <sophora:getDocument var="document" uuid="${param.uuid}" />
 
    <%-- Suffix herausfinden und setzen --%>
    <c:choose>
        <c:when test="${not empty document.binarydata}">
            <c:set var="mimetype" value="${document.binarydata.mimeType}" />
            <c:if test="${not empty mimetype}">
                <sophora:getSuffixByMimeType var="suffix" mimeType="${mimetype}" />
            </c:if>
        </c:when>
        <c:when test="${document['jcr:primaryType'] eq 'sophora-extension-nt:image'}">
            <c:set var="suffix">jpeg</c:set>
        </c:when>
    </c:choose>
</c:if>
 
<c:set var="redirectUrl" >/system/servlet/urlService.servlet</c:set>
 
<c:redirect url="${redirectUrl}">
    <c:param name="uuid" value="${param.uuid}" />
    <c:param name="suffix" value="${suffix}" />
    <c:param name="type" value="${param.type}" />
</c:redirect>

If no URL is configured an error is returned. This is necessary due to the fact, that the generated URL always has to point to the live version (with the live domain) of the passed document.

Update Queue Database

The update queue database is a prioritized queue (like the update queue in the delivery). The queue contains the UUIDs of documents which have been sent to the index.

Actions like structure node changes are inserted to the queue with a lower priority than the processing of a single operation, for instance, changes to an individual document. This is because structure node changes effect all documents that are located at the node at hand and thus might take longer. Document set offline and document deleted events are not added to the update queue at all, because these actions are sent directly to the index.

In general, the queue does not contain large numbers of UUIDs, except for structure node changes or indexer is synchronizing.

To backup the indexer the update queue database is irrelevant. When you (re)start the indexer the sophora.replication.restartDate property should be set to a date which will ensure that all changes are synchronized since stoppage.

Indexer Plugin Mechanism

The Sophora Indexer uses a plugin mechanism to be able to connect to different search engines or to other external systems. Since version 1.33.1 the indexer itself only contains a dummy implementation which creates and writes to a log file. So in order to use the indexer to work with a specific search engine, a plugin has to be added and configured.

Add and configure an Indexer Plugin

First of all the library of the corresponding plugin has to be added to the indexer's plugins folder. This plugins folder must be created within the directory, where the config folder is located. Usually the plugin consists of one jar file, which contains the java code, necessary configuration files and all dependencies. Second of all the added plugin must be configured in the indexer's properties file. To achieve this you have to add the plugin specific configuration properties to the file. These properties are listed in the plugin's documentation. In addition you have to add the name of the plugin's bean which is defined in the indexerExtension.xml file of the plugin. The name has to be set to the property sophora.searchEngine.connection. When you restart the indexer the plugin will be used.

Please note that it is possible to use only one plugin at a time.

Create your own Plugin

To integrate another search engine or external system you have to set up a corresponding Java project. There, you have to put the library com.subshell.sophora.indexer.api into the build path.
Two classes are required that implement the interfaces com.subshell.sophora.indexer.api.ISearchEngine and com.subshell.sophora.indexer.api.ISearchEngineFactory respectively. The implementation of ISearchEngineFactory provides the method getEngine which returns an object of ISearchEngine. All necessary instantiations can be done here as well.

Spring Configuration

Within the build path of the newly created project there must be a directory called "spring". This folder needs to contain a XML file "indexerExtension.xml" (the names of the folder and the file must not be altered). This configuration file comprises a Spring bean definition that is an instance of com.subshell.sophora.indexer.api.ISearchEngineFactory, like the following example:

indexerExtension.xml

<beans xmlns="http://www.springframework.org/schema/beans"
    xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xmlns:context="http://www.springframework.org/schema/context"
    xsi:schemaLocation="
        http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
        http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd">
     
    <!-- enable annotation driven dependency injection -->
    <context:component-scan base-package="com.subshell.sophora.indexer.myindexerplugin"/>
     
    <bean id="myConnection" lazy-init="true" class="com.subshell.sophora.indexer.myindexerplugin.MyConnection">
        <property name="searchEngine" ref="mySearchEngine" />
    </bean>
     
</beans>

In this example only the bean defining an instance of ISearchEngineFactory is included. All other beans (like mySearchEngine) and their dependecies are configured using annotation driven dependency injection. The following code snippets show the MyConnection class, which makes use of Spring's dependency injection and an excerpt of the MySearchEngine class which is instantiated automatically using annotations.

MyConnection.java

public class MyConnection implements ISearchEngineFactory {
     
    private MySearchEngine mySearchEngine;
     
    @Override
    public ISearchEngine getSearchEngine(ISophoraClient client) {
        return mySearchEngine;
    }
 
    public void setSearchEngine(MySearchEngine mySearchEngine) {
        this.mySearchEngine = mySearchEngine;
    }
     
}

MySearchEngine.java

@Component
@Qualifier("mySearchEngine")
public class MySearchEngine implements ISearchEngine {
 
   [...]
 
}

}In addition to the interfaces ISearchEngine and ISearchEngineFactory the API provides an interface called ISiteIndexKeyProvider. A class implementing this interface is instantiated on startup and can be used within all plugins to retrieve the configured index keys for specified sites. You can get an instance of this class by using Spring's dependency injection. The name of the bean to inject is siteIndexKeyProvider.

Build your Plugin

Plugins should be build with and managed by Maven. Therefore the Indexer-API must be added as a dependency to the Maven project. The API itself brings some dependend libraries into the project, like Spring, Apache Commons, Sophora Client etc. On the one hand this makes it is easy to use those libraries for your own plugin, but on the other hand you have to be careful not to create conflicts when adding new dependencies.

Due to the fact that plugins should only consist of one jar file, it is recommended to use the Maven assembly plugin for building.

mvn package assembly:single

JMX Connection

To set up a JMX connection use the following pattern:

service:jmx:rmi://<host>:<sophora.indexer.jmx.registry.port>/jndi/rmi://<host>:<sophora.indexer.rmi.registry.port>/server

An example:

service:jmx:rmi://localhost:5030/jndi/rmi://localhost:5031/server

Username and password are read from the sophora.properties file (sophora.indexer.jmx.registry.username and sophora.indexer.jmx.registry.password), if configured.

The indexer provides the following operations: