Server setup

Download tar package of ealsticSearch and install:

aptitude update
aptitude install openjdk-7-jdk
wget https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.0.0/elasticsearch-2.0.0.tar.gz
tar zxf elasticsearch-2.0.0.tar.gz
cd elasticsearch-2.0.0
bin/elasticsearch

Or you can install it as a service with deb package

wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.3.deb
sudo dpkg -i elasticsearch-1.7.3.deb
sudo /etc/init.d/elasticsearch restart

Start in foreground: sudo /usr/share/elasticsearch/bin/elasticsearch --default.path.conf=/etc/elasticsearch/. If you omit the option "--default.path.conf", you'll get an error: org.elasticsearch.ElasticsearchException: Failed to load logging configuration.

When you want start the 2nd node in the same machine, always start it this way with the same command above.

Or run it in background: /usr/share/elasticsearch/bin/elasticsearch -d --default.path.conf=/etc/elasticsearch/;

Data manipulation in client

Create a new index:

curl -XPUT 'localhost:9200/blogs' -d '
{
   "settings" : {
      "number_of_shards" : 3,
      "number_of_replicas" : 1
   }
}'

List all indexes: curl 'localhost:9200/_cat/indices?v';

For a index named "test":

List all data in an index: curl -XGET 'http://localhost:9200/test/_search?pretty';

Remove an index: curl -XDELETE 'http://localhost:9200/test/', wild card can be used here: curl -XDELETE 'http://localhost:9200/logstash*/', or many indexes at once: curl -XDELETE 'http://localhost:9200/website,shakes*/', separated by comma.

Put the result as well-formatted json:

npm install -g json
curl 'localhost:9200/megacorp/employee/_search?pretty' | json

Notes

You can use curl 'localhost:9200/megacorp/employee/_search?pretty' | python -m json.tool in command line or :%!python -m json.tool in vim to format json string. But it will sort your keys, and maybe not friendly to Unicode characters.

Do NOT delete "mongodb_meta" index in ES, or you will get an error when starting mongo-connector:

AttributeError: 'IndicesClient' object has no attribute 'delete_mapping'

If you had deleted it, restart ES server to rebuild this index.

?petty sometimes will be invalid url.

Format json in vim: :%!json.

Ref:

http://stackoverflow.com/questions/352098/how-can-i-pretty-print-json

https://pascalprecht.github.io/2014/07/10/pretty-print-json-in-vim/

Security

Ref: Security for Elasticsearch.

Under directory "/home/es/apps/elasticsearch-2.0.0", run:

bin/plugin install elasticsearch/license/latest
bin/plugin install elasticsearch/shield/latest
bin/elasticsearch
bin/shield/esusers useradd es_admin -r admin
// input admin pwd: es@Newfairs2015

Elasticsearch authentication will affect kibana and mongo-connector, make things too complicated. So I decided to remove authentication. First stop elasticsearch server, then run:

bin/plugin --help
bin/plugin list
bin/plugin remove shield
bin/plugin remove license
bin/elasticsearch

Finally start mongo-connector and kibana.