Using Sync Command Facing Timeout Issue & batch_size is not Working.

[kamrankhatti]

Hi -
When I run process command data migration works perfectly, but when I use sync command batch_size 10000 does not work (facing weird issue batch_size works for embedded tables but not for others) hence it causes TinyTds::Error: Adaptive Server connection timed out error in some tables which has bunch of records.

Also the migration process is very slow when we use sync as compare to process command, I think this same fix #38 should implement for sync as well. Need @hammady consideration over it.

Thanks
Kamran

[srizviOfficial]

@kamrankhatti Thanks for identifying this issue

@hammady @anlek

I am facing this issue, Process command & sync commands behaves differently.

I am able to write translation file which perfectly translate data from MSSQL SERVER to MongoDB via "process command"without much issue's there are some but we able to fix them.

But, we have requirement to deal with differential records (delta) as well. For that we are using Sync command to translate data but it has some issue which we need to fix. There are some timeout error other thing which IO observe is that its not migrating data in chunks instead its processing whole record at once which making issue's for us.

@hammady do you recommend any work around to work with or how I can deal this issue

[hammady]

If the process command was updated to work in batches after I merged the sync feature, I should be able to port this feature as well. @anlek please confirm.

[srizviOfficial]

@hammady That will be great help man

[anlek]

Hey guys,

I'm not sure, I think the sync command is not using the same commands internally to batch the data.
I think this is the part of the code that is not being batched.

@hammady Does that sound right to you?

[kamrankhatti]

One more thing I found in sync command it ignores all embedded tables, each time you run sync command it brings all data for embedded tables.

I have noticed it does not insert embedded table names in __mongify_sync_helper__ table thats why each time sync command brings all data for embedded tables.

@hammady had you tested sync for embedded tables?

[hammady]

Embedded tables are not supported by sync. I believe it is mentioned somewhere.

…

On Wed, Dec 14, 2016 at 3:19 PM Kamran Khatti ***@***.***> wrote: One more thing I found in sync it also ignores all embedded tables, each time you run sync command it brings all data for embedded tables. *Sync does not work for embedded tables*. @hammady <https://github.com/hammady> had you tested sync for embedded tables? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#126 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AA0PJ8mKEbgeNb5_7if9ocsuWm7h1Vhoks5rH97MgaJpZM4LHx6Z> .