Skip to main content

Nuances of Spatial data and operations Apache SOLR

SOLR is a scalable, distributed and powerful search and indexing solution. SOLR supports indexing spatial data and provides fast search capabilities on spatial data.
Unlike conventional spatial data solutions like PostGIS, indexing spatial data in SOLR provides high speed search using bounding box and range queries, commonly used in spatial data exploration and processing.
Indexing points and running bounding box queries is covered extensively in Apache SOLR documentation.

We will discuss some specific use cases here, described but not explained in the documentation.

1. Indexing Fences in SOLR: SOLR provides a convenient and powerful type, location_rpt which is an implementation of solr.SpatialRecursivePrefixTreeFieldType. location_rpt indexes POINT data, consisting of latitude and longitude. Points are sufficient to index locations that need to be monitored, and that can be done by using bbox or geofile queries as in the documentation. A typical use case in geospatial search is drawing fences around areas that need to be monitored. This requires storing, indexing and monitoring two dimensional areas. In order to do that, 2D shapes can be indexed using SpatialRecursivePrefixTreeFieldType (called RPT), by defining a custom type in the schema as below. This requires using spatial4j JtsSpatialContextFactory. JTS jar should be added to SOLR classpath in order to resolve the factory. Once the type is added, data can be indexed into fields defined of type fence_rpt, with 2D shapes defined as POLYGON("x1 y1,x2 y2,x3 y3,x4 y4")

<fieldType name="fence_rpt"   class="solr.SpatialRecursivePrefixTreeFieldType"           spatialContextFactory="com.spatial4j.core.context.jts.JtsSpatialContextFactory"           distErrPct="0.025"           maxDistErr="0.000009"           units="degrees"        />
In order to monitor fences, we can determine which of the indexed fences does a point intersect using fq=geo:Intersects("x y"). A common use case being, monitoring a person's movements and identifying which fences he get in and out of.

2.  Indexing non geo data in SOLR: location_rpt type, by default, indexes geographical data containing latitude, longitude. In order to index, search and process non geographic data, RPT type can be defined with geo=false attribute. When indexing non geographic points, world bounds need to be defined. These form the coordinate system for indexing data.
Non Geographic RPT types can be combined with JTS to support indexing 2D shapes.
This is useful in indoor geographic indexing and search. This can also be used, probably not intended, to build custom solutions where shapes need to be indexed and searched upon, for non location based use case too, like image analysis.

<fieldType name="nongeorpt"
class="solr.SpatialRecursivePrefixTreeFieldType"
geo="false"
worldBounds="ENVELOPE(0, 0, 1000, 1000)"
maxDistErr="0.001"
units="degrees" />


P.S: Food for thought: Motion controlled games use point in polygon to determine movement of controller and render the movement on screen. Motion controller applications can leverage indexing of points and polygons in custom reference frame for varied applications like hand movement tracking to move robotic mechanical arms.



Comments

Popular posts from this blog

Using JNDI managed JMS objects with Apache CAMEL

Apache CAMEL uses Spring JMS to work with JMS Queues or Topics. Evidently, we will need Spring to configure and use JMS capabilities provided by CAMEL. Details about how to implement JMS based routes using Apache CAMEL can be found in the CAMEL documentation. However, the documentation leaves a lot to be figured out. In a typical Java EE container, it is usually a good idea to abstract the underlying JMS resources by using JNDI. We can use the below configuration to achieve that. This configuration is tested in Websphere environment, but should work in any JEE container. Create a JMS queue connection factory in the JNDI registry. CAMEL configuration will be able to use only one queue connection factory, even if we have more than one. Create one or more JMS queue or topics, in the JNDI registry, as required. The above two steps are related to generic JNDI configuration for JMS resources. Now we come to the setup required for making these JMS resources work with CAMEL rout...

Catch hold of that Exception and hide that stacktrace!!!

E xceptions happen!!! Rules are to be followed, too. Time and again, Java developers are told the golden rule to catch specific exceptions and not to catch the generic Exception. The thought process behind that is, applications should not catch runtime exceptions. This is apt as runtime exceptions are an indicator of "bugs" in the code. However, blindly following rules, as always, can have unexpected consequences. If you are developing services that are to be exposed over the wire, it is always a good idea to break this rule and "catch that Exception". Instead, follow the below principles: Service methods should implement a generic Exception block, along with catching declared exceptions, thrown from inner layers of the code.  If needed, the service can throw another exception back to the client. What's important is that we create a new Exception instance to be thrown, along with relevant message for the client. The service can log stacktrace for the E...

Container ready spring boot application

Spring boot applications are now ubiquitous. The usual way to build one is to create an uber jar. At the same time, Docker allows us to build self reliant containers which are unaffected by the underlying server architecture or neighboring applications or their dependencies. Spring boot applications can also run in docker containers.  However running an uber jar inside a container fails to satisfy an important goal. That of high speed build and deployment. An uber jar is a heavy weight entity. That will make docker image heavy and slow to build. Here's a step by step solution, which leverages docker layer caching feature for faster builds with all spring boot goodness. We will use Maven to build a deployment structure for our docker image that allows fast deployments. Step 1: Create a spring boot application with only the SpringBootApplication and Configuration classes, such as one for REST configuration package scanning, one for JPA etc. Most often this wil...