1) Made calls Async from Sync which brought down time from 118 msec to 38 msec per call.
2) Increased the number of threads for Kafka ( also increasing the partitions). Existing system was not creating enough threads and as the work performed was IO bound there should have been more threads.
3) StackOverflow exceptions and Out of Memory errors in toString fixed with rope and Charsequence.
4) Elasticsearch enabled the snitch at driver level for equitable distribution of load.
5) Used Hashmap instead of ArrayList for contains in operation.
app soft nofile 16384
* hard memlock unlimited
elasticsearch hard memlock unlimited
root soft memlock unlimited
root hard memlock unlimited
#RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
elasticsearch soft RLIMIT_MEMLOCK 65536
elasticsearch hard RLIMIT_MEMLOCK 65536