ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository #21

Sandeepgsn · 2016-07-08T10:15:54Z

While executing below command:
jruby -S gem install --local logstash-input-dynamodb-1.0.0-java.gem

Getting error:
ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository

marcosnils · 2016-07-11T11:49:07Z

@Sandeepgsn try using the GEM in rubygems https://rubygems.org/gems/logstash-input-dynamodb

sgarg-atlassian · 2016-07-21T14:59:15Z

@marcosnils I changed the version in the gemspec file from s.add_runtime_dependency "logstash-core", '>= 1.4.0', '< 2.0.0' to s.add_runtime_dependency "logstash-core", '>= 1.4.0', '< 2.3.4' and it worked. Do you see any problems with this?

marcosnils · 2016-07-21T15:06:31Z

@sgarg-symphy I'm not the owner of this repo, I just maintain forked versions of this library because seems like AWS dropped support for this. I've pushed the gems in ruby gems and the ones there work for me.

vamcyimmanni · 2016-07-28T09:59:33Z

Struggling with the same issue.
Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository.
How to get it to work?

@marcosnils I have tried using the gems from https://rubygems.org/gems/logstash-input-dynamodb. That didn't work either. If possible can you please list out the steps.

Thank you

marcosnils · 2016-07-28T10:36:23Z

@vamcyleonardo how are you installing the gem? and in which logstash version?

marcosnils · 2016-07-28T12:17:51Z

@vamcyleonardo just updated the logstash-input-dynamodb gem in rubygems. Try now.

vamcyimmanni · 2016-07-28T13:29:07Z

Thank you for your response. I will try it.

marcosnils · 2016-07-28T14:17:45Z

@vamcyleonardo don't clone that repo, install the gem from rubygems. Which logstash version are you using?

marcosnils · 2016-07-28T18:07:25Z

@vamcyleonardo use the gem from ruby gems. Don't clone and build from this repo.

marcosnils · 2016-07-28T18:13:34Z

gem install logstash-input-dynamodb:'> 2'
plugin install logstash-input-dynamodb

marcosnils · 2016-07-28T18:26:03Z

gem is there https://rubygems.org/gems/logstash-input-dynamodb. Try removing the :'> 2' from the end

marcosnils · 2016-07-28T18:38:05Z

maybe, give it a try

marcosnils · 2016-07-28T18:47:50Z

@vamcyleonardo it should work, that's how I build my logstash docker images. Maybe if you try a clean installation it works.

If you don't want to deal with all this 💩 you can just use our docker image https://hub.docker.com/r/mantika/logstash-dynamodb-streams/

marcosnils · 2016-07-28T19:00:29Z

U sure the credentials you're specifying have the correct access?

These are the policies you need

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "pointerTable0",
            "Effect": "Allow",
            "Action": [
                "dynamodb:BatchGetItem",
                "dynamodb:BatchWriteItem",
                "dynamodb:DeleteItem",
                "dynamodb:GetItem",
                "dynamodb:PutItem",
                "dynamodb:CreateTable",
                "dynamodb:Query",
                "dynamodb:Scan",
                "dynamodb:UpdateItem",
                "dynamodb:DescribeStream",
                "dynamodb:DescribeTable",
                "dynamodb:GetRecords",
                "dynamodb:GetShardIterator",
                "dynamodb:ListStreams"
            ],
            "Resource": [
                "<your arn>:table/*cptr"
            ]
        },{
            "Sid": "streamReading1",
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeStream",
                "dynamodb:DescribeTable",
                "dynamodb:Scan",
                "dynamodb:ListStreams",
                "dynamodb:GetShardIterator",
                "dynamodb:GetRecords"
            ],
            "Resource": [
                "<table ARN>",
                "<stream ARN>"
            ]
        }
    ]
}

marcosnils · 2016-07-28T19:03:47Z

Remember that you also need to enable the table stream in dynamoDB

mcolarusso · 2016-08-24T22:22:18Z

When running the docker image, i get "Couldn't find any input plugin named 'dynamodb'". Any idea why?

marcosnils · 2016-08-24T22:56:14Z

Which image are you running? V1 or v2?

sent from mobile

On Aug 24, 2016 7:22 PM, "Mike Colarusso" [email protected] wrote:

When running the docker image, i get "Couldn't find any input plugin named
'dynamodb'". Any idea why?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#21 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABgV2gm5_gQS-C6loNbtCBj9WIZAvHd0ks5qjMQagaJpZM4JH5q2
.

mcolarusso · 2016-08-24T22:59:56Z

I tried latest and 2

From: Marcos Nils
Reply-To: awslabs/logstash-input-dynamodb
Date: Wednesday, August 24, 2016 at 6:56 PM
To: awslabs/logstash-input-dynamodb
Cc: Michael Colarusso, Comment
Subject: Re: [awslabs/logstash-input-dynamodb] ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository (#21)

Which image are you running? V1 or v2?

sent from mobile

On Aug 24, 2016 7:22 PM, "Mike Colarusso" <[email protected]mailto:[email protected]> wrote:

When running the docker image, i get "Couldn't find any input plugin named
'dynamodb'". Any idea why?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#21 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABgV2gm5_gQS-C6loNbtCBj9WIZAvHd0ks5qjMQagaJpZM4JH5q2
.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com//issues/21#issuecomment-242233947, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AFVq4jXMtza36zX3kN9XvYtQD4sf7J8Fks5qjMwRgaJpZM4JH5q2.

marcosnils · 2016-08-24T23:45:34Z

@mcolarusso I've just updated the images, I believe those were outdated. Can you please update and try again?

mcolarusso · 2016-08-25T14:14:43Z

@marcosnils That seems to have fixed the issue.

feedtherich · 2016-08-31T21:39:58Z

What is this command "bin/plugin" ?
I don't see this anywhere, even after installing "gem install ..."

feedtherich · 2016-09-08T22:30:51Z

Hello, I tried using the mantika's Docker image and I had this problem:

Pipeline aborted due to error", :exception=>com.amazonaws.AmazonClientException: Unable to execute HTTP request: dynamodb.us-east-1.amazonaws.com, :backtrace=>["com.amazonaws.http.AmazonHttpClient.executeHelper(com/amazonaws/http/AmazonHttpClient.java:618)"

Any idea what's going on here?
Thanks!!

marcosnils · 2016-09-09T04:58:27Z

@feedtherich can u share your config?

feedtherich · 2016-09-09T17:54:00Z

maybe i'm doing this wrong. i'm new to docker. i am just running this script:

sudo docker run mantika/logstash-dynamodb-streams -e '
input { dynamodb {
        endpoint => "dynamodb.us-east-1.amazonaws.com"
        streams_endpoint => "streams.dynamodb.us-east-1.amazonaws.com"
        view_type => "new_and_old_images"
        aws_access_key_id => "<redacted for security>"
        aws_secret_access_key => "<redacted for security>"
        table_name => "<redacted for security>"
}   }
filter { dynamodb {} }
output { elasticsearch { host => localhost } stdout { } } '

marcosnils · 2016-09-10T22:43:39Z

@feedtherich if you're using the "latest" image you need to setup the output plugin as follows:

output {
elasticsearch {
hosts => ["elasticsearch"]
}
}

note that hosts is pluran and an array of values.

feedtherich · 2016-09-12T21:56:08Z

Thanks.
I still got the same error, however.
I think the thing I'm doing wrong is assuming everything is happening
automatically without me needing logstash setup locally.
Or, am I supposed to do something with one of the two Docker files?
I have no idea how to use them.

Thanks.

On Sat, Sep 10, 2016 at 3:43 PM, Marcos Nils [email protected]
wrote:

@feedtherich https://github.com/feedtherich if you're using the
"latest" image you need to setup the output plugin as follows:

output {
elasticsearch {
hosts => ["elasticsearch"]
}
}

note that hosts is pluran and an array of values.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#21 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AUYcgolTZ-5N5c9zrZzRnFvKkkCCbbzgks5qozKdgaJpZM4JH5q2
.

fast1234 · 2016-10-17T05:55:08Z

I am unable to install this plugin on my EC2 machine(Amazon linux). when I try this command:
jruby -S gem install --local logstash-input-dynamodb-1.0.0-java.gem
I am facing this error:
ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository
Please help

seriyvolk83 · 2019-10-17T20:37:10Z

I may be wrong, but
I think the issues are due to postfix "-java" the gems have in the repository. https://rubygems.org/downloads/logstash-core-2.3.1-java.gem.

All other packets look like https://rubygems.org/downloads/dynamodb-1.2.1.gem
i.g. nothing goes after version, only .gem.

marcosnils mentioned this issue Jul 21, 2016

Installation issues #17

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository #21

ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository #21

Sandeepgsn commented Jul 8, 2016

marcosnils commented Jul 11, 2016

sgarg-atlassian commented Jul 21, 2016

marcosnils commented Jul 21, 2016

vamcyimmanni commented Jul 28, 2016

marcosnils commented Jul 28, 2016 •

edited

Loading

marcosnils commented Jul 28, 2016

vamcyimmanni commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016 •

edited

Loading

marcosnils commented Jul 28, 2016 •

edited

Loading

marcosnils commented Jul 28, 2016

mcolarusso commented Aug 24, 2016

marcosnils commented Aug 24, 2016

mcolarusso commented Aug 24, 2016

marcosnils commented Aug 24, 2016

mcolarusso commented Aug 25, 2016 •

edited

Loading

feedtherich commented Aug 31, 2016

feedtherich commented Sep 8, 2016

marcosnils commented Sep 9, 2016

feedtherich commented Sep 9, 2016

marcosnils commented Sep 10, 2016

feedtherich commented Sep 12, 2016

fast1234 commented Oct 17, 2016

seriyvolk83 commented Oct 17, 2019

ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository #21

ERROR: Could not find a valid gem 'logstash-core' (< 2.0.0, >= 1.4.0) in any repository #21

Comments

Sandeepgsn commented Jul 8, 2016

marcosnils commented Jul 11, 2016

sgarg-atlassian commented Jul 21, 2016

marcosnils commented Jul 21, 2016

vamcyimmanni commented Jul 28, 2016

marcosnils commented Jul 28, 2016 • edited Loading

marcosnils commented Jul 28, 2016

vamcyimmanni commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016

marcosnils commented Jul 28, 2016 • edited Loading

marcosnils commented Jul 28, 2016 • edited Loading

marcosnils commented Jul 28, 2016

mcolarusso commented Aug 24, 2016

marcosnils commented Aug 24, 2016

mcolarusso commented Aug 24, 2016

marcosnils commented Aug 24, 2016

mcolarusso commented Aug 25, 2016 • edited Loading

feedtherich commented Aug 31, 2016

feedtherich commented Sep 8, 2016

marcosnils commented Sep 9, 2016

feedtherich commented Sep 9, 2016

marcosnils commented Sep 10, 2016

feedtherich commented Sep 12, 2016

fast1234 commented Oct 17, 2016

seriyvolk83 commented Oct 17, 2019

marcosnils commented Jul 28, 2016 •

edited

Loading

marcosnils commented Jul 28, 2016 •

edited

Loading

marcosnils commented Jul 28, 2016 •

edited

Loading

mcolarusso commented Aug 25, 2016 •

edited

Loading