h1. Play Framework - Elastic Search Module

by *Felipe Oliveira*
"http://mashup.fm":http://mashup.fm
"http://geeks.aretotally.in":http://geeks.aretotally.in

Screencast
http://geeks.aretotally.in/play-elastic-search-module-new-0-0-5-release-with-screencast

Demo Source Code
https://github.com/mashup-fm/playframework-elasticsearch-0.0.5-demo

Searching is hard, just ask Google! Thankfully Lucene came about to provide incredible searching capabilities to the Java community. I have personally been using Lucene since 1999 and I am a huge fan, probably my very favorite Java project ever.

Lucene provides some great functionality but you still have a lot of work to do to integrate your app into Lucene; you need to map your models into a Lucene document, you need to write an indexer, you need to manage your indexes, cluster them, keep them in sync, write the search interface, etc. That’s a lot of work! To solve this problem there was a generation of frameworks built on top of Lucene, the most popular ones were Solr and Compass.

Compass and Solr are great frameworks, I have used them in production for very large deployments. I have built Foreclosure.com and Fannie Mae’s HomePath.com, Zillow.com’s Foreclosure Channel and an internal application to support HUD’s broker network for Florida, New York, New Jersey and Georgia entirely with that technology. There are still problems, it’s hard to cluster it, maintain real-time updates, backups, not have a single point of failure meaning failover which brings us to the next generation.

We finally come to Elastic Search from the awesome guys responsible for Compass! Elastic Search is the next generation of Compass, it’s a really great searching framework addressing a lot of the things that are so hard to deal with. It’s distributed, it supports Geo Spatial queries such bounding box searches, radius searches (around me type of search), backups on a slower data storages such as EC2, schema mapping, REST/JSON interface; like I said it’s really great stuff!

Elastic Search is a Distributed Search Solution based on Apache Lucene. 

The main advantages are: 
* Schema-Free  (NoSQL)
* REST API Support - Easy Integration
* Reliable/No Single Point of Failure (you won't lose any data or have any outage if a node goes broke, huge point when considered the next point also)
* Near Real-Time (pretty self explanatory)
* Async Write Behind for Long Term Storage (store data on a separate data center for example without affecting the performance of your application)
* Geospatial Support (forget about highly complex packages such as PostGIS)


Integrate Elastic Search in a Play! Framework Application. This module uses JPA events to notify Elastic Search of events of their own. In Local Mode it embeds a running Elastic Search instance (port 9200 by default), a good choice for development. In Client Mode it connects to an external instance of Elastic Search, that might be your setup in production.



h2. Prerequisites

Play! 1.2



h2. Install the module

Install the elasticsearch module from the modules repository:

bc. play install elasticsearch



h2. Enable the module

After installing the module, add the following to your conf/dependencies.yml to enable it (don't forget to run play dependencies):

bc. require:
	- play -> elasticsearch 0.2


h2. Configure the module

You need to configure the module by setting these properties in your application.conf. There are two ways to run your Play! app with Elastic Search, local mode or client mode. Local mode works well for development purposes, an Elastic Search instance will run on the same JVM as your Play! application automatically. You won't need to setup another service, etc. The second option is client mode which fits better in a production environment. The default option is local mode.

bc. # Option 1) Elastic Search (Local Model)
elasticsearch.local=true

bc. # Option 2) Elastic Search (Client Model)
elasticsearch.local=false
elasticsearch.client=mynode1:9200,mynode2:9200


h2. Usage

h3. @ElasticSearchable

You basically need to add annotation "@ElasticSearchable" to your Model class. It only works for JPA so far. If a model has that annotation our module we'll watching for JPA events for instances of that class and route messages to Elastic Search to make sure Elastic Search index has the latest information available on the database.

Example:

bc.. @ElasticSearchable
@Entity
public class Post extends Model {
 
    public String title;
    public Date postedAt;
    
}

p. It's possible to prevent fields from being included in the index by marking them with @ElasticSearchIgnore.

By default an index name is generated for you, however you can override this by setting the indexName field:

bc. @ElasticSearchable(indexName="my_index_name")

h3. @ElasticSearchEmbedded

p. Since 0.0.9 it's possible to embed properties by using "@ElasticSearchEmbedded".

bc. class Institution {
	public String name;
	public String type;
}

bc. @ElasticSearchable
class Education {
	...
	@ElasticSearchEmbedded
	Institution institution;
}

p. This will add institution.id and institution.name to education.
By default all fields are embedded, which can be overriden by specifying the fields to embed:

bc. @ElasticSearchable
class Education {
	...
	@ElasticSearchEmbedded(fields={"name"})
	Institution institution;
}

h2. Searching

h3. Simple Searching

bc. SearchResults<Post> list = ElasticSearch.search(QueryBuilders.fieldQuery("title", "what a search"), Post.class);
See "Elastic Search documentation":http://www.elasticsearch.org/guide/reference/java-api/search.html for more example.

h3. ElasticSearchController

The biggest change on this release is a nicer search interface. We are providing a very simple way to get started, inspired by Play!'s CRUD module.
Basically you need to create a controller class and extend ElasticSearchController. Use annotation @ElasticSearchController.For(YOURMODELCLASS.class) to tell our module what model you want to search on, here's an example:

bc.. @ElasticSearchController.For(ElasticSearchSampleModel.class)
public class ElasticSearchExample extends ElasticSearchController {

}

p. You should be able to search on http://localhost:9000/elasticSearchExample/index. If you want to customize the views, just create a directory ELASTIC_SEARCH under views and change whatever you need to change.

h3. Custom search

After you've created your XContentQueryBuilder object, you can search in two ways:

1) Call ElasticSearch.search()
2) Call ElasticSearch.query() and subsequently set query parameters (e.g. paging)

If you are searching for JPA entities and need referenced entities to be available (e.g. @ManyToOne or @OneToMany relationships), use 

bc. ElasticSearch.searchAndHydrate(...)

or call hydrate(true) on your Query:

bc. Query query = ElasticSearch.query(...);
query.hydrate(true)

This will take care of hydrating your entities so proxies and collections work.


h2. User Interface 

After you start your application (play run), you should have an admin interface automatically running on http://localhost:9000/es-admin/.


h2. Source Code

Fork it on Github "https://github.com/feliperazeek/playframework-elasticsearch":https://github.com/feliperazeek/playframework-elasticsearch.


h2. Roadmap

* Add different methods of notification (ElasticSearch River, AMQP, JMS)
* Adding support for non-Jpa models. 
* Parent/Child support
* Customizations for mapping, etc


h2. Credits

* Shay Banon for the great work with Elastic Search and Compass, I have been following his work for a few years, great stuff.
* Ben Birch ("https://github.com/mobz":https://github.com/mobz) for the work on the User Interface.


h2. Changelog

h3. Version 0.2

* Allow index name to be set in @ElasticSearchable

h3. Version 0.0.9

* Embedded property support through @ElasticSearchEmbedded
* Fixed a bug which prevented model create- and update-events from being processed
* ElasticSearchController now uses elastic search for searching

h3. Version 0.0.8

* JPA entity hydration support
* Query support
