BoxLang 🚀 A New JVM Dynamic Language Learn More...

PresideCMS Extension: ElasticSearch

v7.0.1+0000080 Preside Extensions

ElasticSearch extension for Preside

This is an extension for the Preside Platform that provides APIs and a methodology for integrating full-text search with ElasticSearch.

Installation

From the webroot of your application:

box install preside-ext-elasticsearch

ElasticSearch engine versions

A single release of the extension supports multiple ElasticSearch engine versions. Set the version of the engine you are using, either with an environment variable:

ELASTICSEARCH_VERSION=1.7

or in your application's Config.cfc:

settings.elasticSearchConfig.engineVersion = "1.7";

The default is 6.3, although we recommend the version be explicitly set, to avoid potential upgrade issues. Currently supported versions:

Version Dialect Notes
1.7 Es1Dialect One mapping per document type. query_string searches. The fuzziness and prefixLength search arguments are ignored.
6.3 Es6Dialect A single doc mapping, with the document type stored in a type keyword field. multi_match searches.

The extension uses the dialect registered for the highest version that is less than or equal to the configured version, so 6.8 uses the 6.3 dialect. The admin search engine page shows the configured version next to the version reported by the server, and warns when their major versions differ.

Changing the engine version requires an application reload, followed by a rebuild of your search indexes.

Supporting a new ElasticSearch version

Everything that differs between engine versions lives in a "dialect" component in services/dialects/. To support a new version:

  1. Create a dialect that extends the closest existing dialect (e.g. Es7Dialect extends Es6Dialect), overriding only what has changed.
  2. Register it in config/Config.cfc (settings.elasticSearchConfig.dialects[ "7.10" ] = "Es7Dialect"). Applications can also register their own dialects using a full component path.
  3. Add specs for the new version to the unit tests.

Other settings

Setting (settings.elasticSearchConfig.*) Default Description
indexPageSize 100 Number of records sent to ElasticSearch in each batch when rebuilding an index.
searchPageSize 10 Default page size for search().
groupByRecordIdOnly false Group by the record ID instead of using Preside's autoGroupBy when fetching records for indexing.
indexExpiryUnit / indexExpiryMeasure d / 1 How long an index rebuild can run before it is considered to have timed out.

The extension also adds a scheduled task, Rebuild search indexes, which runs daily at 4am.

Documentation

Documentation is a work in progress. For now, we intend to use the Github Wiki and could do with all the help we can get. Contributions very welcome!

Versioning

We use SemVer for versioning. As of 7.0, the extension is universal and covers multiple versions of ElasticSearch. For the versions available, see the tags on this repository. Project releases can also be found and installed from Forgebox

License

This project is licensed under the GPLv2 License - see the LICENSE.txt file for details.

Authors

The project is maintained by The Pixl8 Group. The lead developer is Dominic Watson.

Code of conduct

We are a small, friendly and professional community. For the eradication of doubt, we publish a simple code of conduct and expect all contributors, users and passers-by to observe it.

Changelog

v7.0.1

v7.0.0

  • Combines the 1.x (ElasticSearch 1.7) and 6.x (ElasticSearch 6.3) releases into a single release

v6.3.14

  • Build fixes

v6.3.13

  • Change default indexing timeout to 1d and make configurable

v6.3.12

  • Ensure indexing fails and reports an error when there are errors returned from a batch indexing operation.

v6.3.11

  • Performance: do not fetchasbinary when making http calls to the ElasticSearch server - wastes java memory with encoding/decoding
  • New feature: allow bulk deletion of records

v6.3.10

  • Cleanup index name case sensitivity check

v6.3.9

  • Synonyms not working due to case sensitivity

v6.3.8

  • Address issue with enum renderer and indexed enum fields (where raw values wanted)

v6.3.7

  • Convert to Github actions

v6.3.6

  • Fix issue with multi-site configuration where index caches for page indexing were mixing up pages across sites

v6.3.5

  • Fix 10.14 compatibility issue with numeric fields attempting to get indexed with locale formatting (commas, etc.)

v6.3.3

  • Change DSL generation for updated API
  • Change default query to multi_match, add fuzziness
  • Add configurable prefixLength for fuzzy searches
  • Add new interception point: postElasticSearchGenerateDsl
  • Add option to specify highlight encoder
  • Remove truncated HTML entities from highlights

v6.3.2

  • #11 Respect @tenant when detecting site tenancy, not just siteFiltered=true

v6.3.1

  • Make all standard extension API calls and index metadata compatible with ElasticSearch 6.3
  • Make easier to use environment variables and default settings for connecting to ElasticSearch

v1.1.15

  • Fix issue with groupBy/autogroupBy for MSSQL
  • Fix build on trusty

v1.1.14

  • Fix compatibility issue with latest Lucee whose message/detail contents for database errors have switched

v1.1.13

  • Fix Travis test/build script

v1.1.12

  • Version number fix

v1.1.11

  • Do not run post/pre insert/update/delete data logic for objects that are not search enabled

v1.1.10

  • Re-enabling commented test

v1.1.9

  • Adding defensive code if the Object doesn't have a standard ID field

v1.1.8

  • Travis not picking up changes in repo? Changing test...

v1.1.7

  • Version number fix

v1.1.6

  • Better code for dealing with no query results

v1.1.5

  • Removing failing tests for now - need to debug. Main thing is all bar 3 are running and passing.
  • Simplifying the test runner.
  • Fixing how the tests are run for later versions of CommanBox - emulating what Preside does
  • Fixing up main box.json for extension

v1.1.4

  • Modifying so don't have to modify the arguments that "deleteData()" uses

v1.1.3

  • Fixing issue with preDeleteObjectData explicity taking an ID value passed and processing without considering any other filter parameters that might apply

v1.1.2

  • Adding option to avoid selecting from cache when selecting data to be indexed + bumping version number

v1.1.1

  • Exclude drafted page from being indexed

1.1.0

  • fixed bug in testsuite
  • added possibility to skip the single record indexing

v1.0.25

  • Version fix

v1.0.23

  • Corrected typo on variable name

v1.0.22

  • Make ensuring no physical index exists with our index name safer. Only run when necessary and then pause before adding alias
  • Ignore server.json for tests

v1.0.21

  • Catch missing index exceptions when attempting to delete an index

v1.0.20

  • Allow objects to be decorated with alternative handler method that defines where to get index data from

v1.0.19

  • Refactor tests for new logic

v1.0.18

  • Delete any existing indexes using the alias name before creating the alias.
  • Ensure records that change during a full reindex are reindexed again once the full reindex is complete

v1.0.17

  • Corrected tag number

v1.0.17

  • Update package location

v1.0.16

  • Remove the term suggestion function. Suggestion should be place together with search query

v1.0.15

  • Remove output="false" Ensures new page is indexed

v1.0.14

  • use correct repo URL
  • Merge changes that were pushed into closed source version of the repo

v1.0.13

  • fix bad json

v1.0.12

  • Ignore travis.yml in box package
  • Automate builds and publishing for the extension

v1.0.24

  • Add search suggestion result

v1.0.22

  • do not preside query cache for looking up hierarchical page data during indexing

v1.0.21

  • Fixing mapping configuratin for the site property to ensure it's not analyzed

v1.0.15-v1.0.20

  • Fixing tests

v1.0.13

  • Updating the ES search engine to index ALL Sites inclusive into one index

v1.0.12

  • update ES index when update data

v1.0.11

  • COONP-486 Ensures disable child page reindex option is added

v1.0.10

  • Do not version elasticsearch indexing status table

v1.0.11

  • updated box.json

v1.0.10

  • Adding a readme

v1.0.8

  • Make branch folder just the last part of branch name e.g. just v1.x.x of 'tags/v1.x.x'

v1.0.7

  • Removing unnecessary output=false from everything
  • Adding build scripts, etc. + fixing tests ready for extension to be built with jenkins

v1.0.5

  • Default to searchable=false for primary key

v1.0.3

  • Fix for use of discontinued use of properties as beans
  • Add non term filter

v1.0.2

  • Fixing borked layout in ES control page

v1.0.1

  • Initial release

Use It

$ box install preside-ext-elasticsearch

How do I do this?

Collaborators 0

No collaborators yet.

Links

   

Stats

  • {{ getFullDate("2016-10-14T07:45:13-05:00") }}
  • {{ getFullDate("2026-11-03T11:16:32-06:00") }}
  • 12,400
  • 220,068
Report Abuse