Looks like a nightmare to get going with

[samuelgoldenbaum]

2 years old and not exactly straight forward, any alternatives?

[marcosnils]

Sorry, yes. I'd recommend using AWS Lambda to process the dynamo streams and send it to a ES instead of using this old and unsupported logstash plugin.

[mholford]

Would be nice if Amazon updated their documentation. They still list this as the recommended approach for loading streaming data from Dynamo -> ES

[danbovey]

You're right, this is ridiculous. And I'm all the way down this rabbit hole, all because DynamoDB can't do a lowercase text contains query.

Edit - to add something helpful to this. The alternative tutorial on the AWS blog that uses lambda is outdated as the blueprint no longer exists. But looks like somebody has written a python script here: https://github.com/vladhoncharenko/aws-dynamodb-to-elasticsearch/blob/master/scripts/dynamodb-to-es.py

[aetter]

Hi everyone, I recently removed the reference to this plugin from the Amazon Elasticsearch Service Developer Guide and added some simple sample code for stream processing:

https://docs.aws.amazon.com/elasticsearch-service/latest/developerguide/es-aws-integrations.html#es-aws-integrations-dynamodb-es

Very open to feedback. Feel free to open issues here:

https://github.com/awsdocs/amazon-elasticsearch-service-developer-guide/issues

[marcosnils]

+1 thanks

[scenterbar-sans]

+1 thanks

Did you follow through with the lambda solution? I feel like the solution in the docs doesn't account for failures or the inability to connect to es for any period of time. You would need to implement dead letter queues, and separate lambda functions for full table loads as opposed general streaming. @aetter if you have any recommendations I would certainly appreciate it. I just spoke with an elasticsearch subject matter expert from aws and his recommendation was to use logstash, which I agree seems much more robust. I would seriously appreciate any input or direction.

[mrichman]

I just spent 2 hours fighting with Java to get this working. Then I found this issue. I'm abandoning this effort. Someone should update the README to indicate this "solution" is hopelessly deprecated.