Plugin conflict with Logstash > 2.0.0

[shalomvolchok]

When attempting to install this plugin, with the most recent version of logstash, I receive the following error. Are there any plans to update this to work with > 2.0.0?

logstash-filter-dynamodb (>= 0) java depends on
      logstash-core (< 2.0.0, >= 1.4.0) java

    logstash-core (= 2.1.1) java

[agaur-aws]

@shalomvolchok we currently don't have support for logstash > 2.0.0, however if you are able to add that support, we would definitely be open to a pull request from you.

[ahmedammar]

@shalomvolchok @agaur-aws

Attached is a patch that seems to work for me with logstash 2.1, I am using https://github.com/benjcooley/logstash-input-dynamodb.git since I can't run it without (missing jars).

Patch link removed, please see fork and pull request below

[raymolin]

@ahmedammar thanks for the patch! I will take a look into getting the repository up to date.

[shalomvolchok]

@ahmedammar Thanks for sharing the patch!

[ahmedammar]

I have rebased the patch, and shutdown seems functional.

[imewish]

@ahmedammar hi, i tried your patch it works fine. I could succesfully install the plugin. and i coud fetch the data from my dynamo db table by bin/logstash -f logsatsh-input-dyanmodb.conf from my logstash home directory. But when i move the config file to /etc/logstash/conf.d/ and restart the logstash services it gives me some error like this, "such file to load -- com/amazonaws/aws-java-sdk-elasticbeanstalk/1.10.11/aws-java-sdk-elasticbeanstalk-1.10.11 (LoadError)"". I guess the logstash deamon(run as logstash user) cant resolve/find the installed dependencies. other users lile root,ubuntu could easily resolve it and run the plugin smooth.
i tried to build and install the gem as logstash user(by changing the permission of root directoies, but its not a safe way ). it works fine. Any helps would be apreciated.

[ahmedammar]

I only ported the existing source to work with later logstash ... I'd be surprised if it worked before applying my patch ...

[imewish]

@ahmedammar i have used this files https://github.com/genesi/logstash-input-dynamodb/tree/b1be5b5e848d53e5e10a5b72740fc4bbf9c92e94 . and used the following steps,
jruby -S gem install bundler
bundle install
jruby -S gem build logstash-input-dynamodb.gemspec
bin/plugin install ../logstash-input-dynamodb/logstash-input-dynamodb-1.0.0-java.gem

and it works fine. My problem is with logstash daemon. The user logstash cant resolve the depndencies 😕 .

Do you have any idea ?

[ahmedammar]

I'd recommend doing all the steps you mentioned as the logstash user? Sorry I can't be of more assistance ...

[imewish]

I tried to build and install the gem as logstash user(by changing the permission of root directories, but its not a safe way ). it works fine. Need to find how install these gems globally

[teebu]

I think im experiencing the same problem:

bin/logstash -f logstash-simple.conf
fatal: Not a git repository (or any of the parent directories): .git
The error reported is:


        you might need to reinstall the gem which depends on the missing jar or in case there is Jars.lock then resolve the jars with `lock_jars` command

no such file to load -- com/amazonaws/aws-java-sdk-elasticbeanstalk/1.10.11/aws-java-sdk-elasticbeanstalk-1.10.11 (LoadError)

Works when I
sudo bin/logstash -f logstash-simple.conf

[imewish]

@teebu its because the gems are not available for your user its only avail for root. Try to build and install the plugin as ubuntu(any user you want). it should work. i couldnt find a better way to install gems globally

[teebu]

Not sure how to do it without sudo.

[ec2-user@radiatus logstash-input-dynamodb]$ jruby -S gem install bundler
Fetching: bundler-1.11.2.gem (100%)
ERROR:  While executing gem ... (Errno::EACCES)
    Permission denied - /home/ec2-user/.rvm/gems/jruby-1.7.19/wrappers/ruby

[imewish]

i have followed some wired way to make it work finally. You need to build and install the plugin as logstash user. bcoz the logstash service daemon runs as logstash user. otherwise you need to run the config file manually all the time. Please try these steps. not sure it woould work for you,

im using ubuntu15.10 wily.

1. Created a shell for logstash user(usermod –s /bin/bash/ logstash)
2. Login as logstash

sudo su

su - logstash

1. Allow permission to these folders for logstash user access,

/usr/bin/jruby,

1. Downloaded the binary from here,( https://github.com/genesi/logstash-input-dynamodb/archive/b1be5b5e848d53e5e10a5b72740fc4bbf9c92e94.zip)
2. In the downloaded folder,

Run this commands,

jruby -S bundle install

jruby -S gem build logstash-input-dynamodb.gemspec

jruby -S gem install logstash-input-dynamodb-1.0.0-java.gem
(This step stucks at fisrt time run (happened for me). Needed cancel and run again)

5.in logstash home directory (/opt/lohstash/),

Run,

bin/plugin install “path to logstash-input-dynamodb-1.0.0- java.gem”

it will install the logstash-input-dyanamodb plugin 

then Openen the /opt/logstash/Gemfile ,

Change the PATH of “gem logstash-input-dynamodb" to “path to logstash-input-dynamodb-1.0.0- java.gem”

run the config file again. if it works fine restart the logstash service

[marcosnils]

I'll update the logstash gem shortly in ruby gems so it can be installed directly without downloading it locally.