Relevance Bench - SearchStax


Overview

The SearchStax® Relevance Bench provides a mechanism for experimenting with Solr relevancy using conditional boosts and functions that calculate boosts from index-field values. SearchStax Experiments Sets of boost settings, called experiments, can be contrasted to your current application, to one another, and to historical user actions (using simulations).

Professional Services Engagement

Some of the features described on this page require the participation of SearchStax Professional Services.

Contents of this page:

Solr Query Boosts

A "boost" is a multiplier applied to the partial-match score of a specific field. This lets the search designer declare that a match on one field is more important than a match on another field when ranking the search results.

For instance, an exact match on a part number or stockkeeping unit (SKU) should move that item to the top of the result list, but an exact match on a color should not.

Query boosting is a standard feature of Solr, and is usually applied through field-level notations added to the query URL. SearchStax lets us define sets of query boosts ("experiments") and apply them to a query through a single URL keyword parameter. (SearchStax translates boost settings into the form expected by the Solr Edismax query parser.)

SearchStax experiments support the following Solr boosts:

The boost settings are set up through the Relevance Bench, which is a one of the SearchStax Analytics pages.

Javascript Events Library

SearchStax Analytics features, including Relevance Bench experiments, depend on collecting user events from your running applications.

See Recording Events to learn how to capture user events on your web page. Event messages are sent to a SearchStax Analytics App for collation and display.

Creating Experiments

A set of SearchStax relevancy boosts is called an "experiment." To create an experiment:

  1. Open the appropriate SearchStax account.
  2. Click Analytics in the navigation column to expose the list of Analytics Apps.
  3. Click the appropriate Analytics App.
  4. On the Analytics App dashboard, click Experiments in the horizontal menu across the top of the display. This will expose the app's Relevance Bench.
  5. Click the New Experiment button on the Relevance Bench page to open the Experiment page.
  6. Configure and save the new experiment. (Details are presented below.)

To apply an experiment to a query:

  1. Set up a query URL applying an appropriate query to your Solr collection.
  2. Include "&model=" as part of the query URL.

For example:

https://ss123456-us-west-2-aws.searchstax.com/solr/ABdev/ABdeploy?location=90278&model=boost-freshness

Custom Query Parser

The SearchStax Search API requires a custom query parser
and a custom request handler, both configured in solrconfig.xml. The configuration depends on the fields in your schema.xml file.

Configure the query parser using the SearchstaxQParserPlugin:

<queryParser name="searchstax-parser" class="com.searchstax.searchapi.SearchstaxQParserPlugin" runtimeLib="true"/>

Configure the request handler, setting the defType to "searchstax-parser":

<requestHandler name="/customsearch" class="solr.SearchHandler">
  <lst name="defaults">
    <str name="defType">searchstax-parser</str>
  </lst>
</requestHandler>

Please contact SearchStax Professional Services for assistance configuring these custom components.

Experiment Fields

On the Relevance Bench page, click the New Experiment button. You will be offered the following fields.

Experiment Name (and Label)

Enter a name for the experiment. The system will downcase this name, hyphenate it, and use it as the Experiment Label. The label is the experiment name used in the query URL.

SearchStax API

Text Match Boosts

Enter zero or more attributes for text boosts.

SearchStax API

Numeric Match Boosts

Enter zero or more attributes for numeric boosts. Numeric boosts may be applied to numbers, locations, and dates. Both positive and inverse boosts may be configured.

SearchStax API

Matching Locations

SearchStax can provide location matching and boosting for documents that contain a Solr location field. If this is desired, a custom document ingestor reads ZIP codes from the input documents and converts them to location fields (containing latitude and longitude) as the documents are indexed.

SearchStax Professional Services will set up and configure the document ingestor per your requirements.

Once the location fields are in the index, you can apply a numeric boost to the closest locations.

Inverting Numeric Matches

A numeric match may be inverted by prepending the string "__inv__" to the attribute field name. For instance, a numeric boost on the field "listing_price" may be inverted by specifying "__inv__listing_price".

A numeric boost on a price promotes documents that have high prices. Inverting that boost promotes documents that have low prices.

Create, Update, Publish, and Promote

The Relevance Bench displays action buttons labeled "Create Experiment," "Publish," "Update & Publish," and "Promote," depending on whether you are creating or editing an experiment.

Configuration File Update

Caution: By publishing an experiment to a collection you are updating the configuration file for this collection. This will affect any collections that share this configuration file.

Boost Functions

SearchStax provides several functions that enhance your ability to boost query results. These functions are entered into an experiment's Attribute Label field:

SearchStax API

Promo Boost

__promoBoost(field:value)

The Promo Boost raises the relevance score of every document that contains the specified field:value pair.

__promoBoost(category:clown)

All documents containing category value "clown" will receive a boost.

Promo Match Boost

__promoMatchBoost(field1:value1 ... fieldN:valueN)

Boost fieldN:valueN when the previous field:value pairs are all matched in the user query.

__promoMatchBoost(location:90406,category:clown)

In this example, if the user searched for location:90406, then boost documents with category:clown.

Freshness Boost

__freshness(date,cutoff,decay)

The Freshness Boost raises a document's relevance as a function of a date field. Recent documents get a bigger boost.

__freshness(creation_date,30,0.1)

This example boosts documents that are less than 30 days old, reducing the boost by 1% per day to favor newer publications.

Ratings Boost

The Ratings Boost raises a document's relevance as a function of both the weighted mean of all ratings and the total number of ratings. The function strikes a balance between quality and quantity.

__ratings(mean,count,max,Q)

__ratings(ratings,review_count,5,50)

The implementation was based on the discussion shown here.

Random

The Random function randomizes the order of the items in the result set.

__random()

Run the same query multiple times. The result appear in a different order each time. This is useful when you want to display random results on the first page the user sees.

Questions?

Do not hesitate to contact the SearchStax Support Desk.