shell command not finishing before next command is executed using await #58
Comments
|
Hmm, that's interesting. I do see one minor difference, with adb push
you're pushing to the mountpoint and not the full path. I guess there may
be some detection logic going on in regular adb. Could you try if your adb
push works if you specify the full path?
…On Sat, 10 Dec 2016 at 0:18 rsc-rchristensen ***@***.***> wrote:
A little background, I am using your library to push a firmware update
file to a custom android device. In order for the device to detect the
update file, it has to be in a particular block device at a particular
mount point. If I execute these shell commands in order, the push goes
through, the file ends up where it is supposed to be, and the device picks
up the firmware no problem. When I execute the same commands using your
library, the commands execute properly, however the file ends up in the
mount target folder and not the actual mounted device.
Working adb commands in ubuntu terminal:
adb shell mount /dev/block/mmcblk1p5 /mntpnt
adb push tmpUpdate.zip /mntpnt
adb shell mv /mntpnt/tmpUpdate.zip /mntpnt/update.zip
adb umount /mntpnt
When I do the same commands using your library (using Electron/babel await
if that makes a difference):
await client.shell(device.id, 'mount -t vfat /dev/block/mmcblk1p5 /mntpnt');
await client.shell(device.id, 'sync');
const pushFile = await client.push(device.id, updateFilePath, '/mntpnt/tmpFirmware.zip');
pushFile.on('progress', (stats) => {
store.dispatch(DEVICE_FIRMWARE_UPDATE_PROGRESS(device.id, { status: 'copyingFirmwareFile', progress: (stats.bytesTransferred / updateFiles.get(updateFile.fileName).get('total')) * 100 }));
});
pushFile.on('end', async () => {
store.dispatch(DEVICE_FIRMWARE_UPDATE_PROGRESS(device.id, { status: 'finishedCopyingFirmwareFile' }));
await client.shell(device.id, 'sync');
await client.shell(device.id, 'mv /mntpnt/tmpFirmware.zip /mntpnt/update.zip');
await client.shell(device.id, 'sync');
await client.shell(device.id, 'umount /mntpnt');
await client.shell(device.id, 'sync');
store.dispatch(DEVICE_FIRMWARE_UPDATE_PROGRESS(device.id, { status: 'waitingForReboot' }));
});
When I run the commands using adbkit, the device doesn't detect the
update, so I shell in, I can see the file in the /mntpnt directory, however
the block device is not mounted. However, if I shell in during the
transfer, I can see in mount that the device is mounted. Once I mount the
block device manually, the files disappear (due to the mount location
changing for the dir), and then once I unmount again, the files are back.
Any idea or direction would be greatly appreciated.
Notes:
shell sync commands added based on previous comments, does not work with
or without although does seem to improve the reliability of the mv command
(to rename the file) when sync is called after each shell command
I have also added console logs to the shell commands and the output seems
to be what I expect, it just seems like the client.push is what is not
performing the way I expect it to.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#58>, or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAB-_YGlrOiim_A-_8DJD9rr3dnMyYo6ks5rGXFBgaJpZM4LJChk>
.
|
|
mounting the block device then using this command: adb push update-bigupgrade-161129-signed.zip /mntpnt/tmpFirmware.zip results in the file being where it is supposed to be and picked up by our device |
|
Interesting note, I modified the client.push command to just be the mount point (excluding filename) and I got these errors (many more, truncated since they are all the same). the interesting thing though is, the update file was picked up by my device unexpectedly and the update was performed. That means the file copy was successful and placed the file in the correct location. Due to the input filename, the rename could not have succeeded, but the umount would have had to since if the block device is mounted, the update watcher service cannot check for an update file. |
|
Could you create a small file and do a packet capture of a regular push? If
you're on macOS, you can use CocoaPacketAnalyzer on the local interface
with the query set to 'tcp port 5037''. Don't monitor, just start and stop
when done. Then save and attach the pcap here.
…On Sat, 10 Dec 2016 at 0:32 rsc-rchristensen ***@***.***> wrote:
mounting the block device then using this command:
adb push update-bigupgrade-161129-signed.zip /mntpnt/tmpFirmware.zip
results in the file being where it is supposed to be and picked up by our
device
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#58 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAB-_SBdoksK0YdnBS7tlKXTEgZMx6e3ks5rGXR3gaJpZM4LJChk>
.
|
|
here you are, included two files, one where I pushed to the plain directory, the other where I pushed to the directory after I had run the mount command. |
|
There doesn't seem to be any difference. Could you do two more caps: One with your adb commands, including mount, the way it's supposed to be (but with a small test file to reduce noise) And another with the exact same thing using adbkit. You can get rid of the syncs as well, just keep it as similar as possible. |
|
Both should include the mount command. |
|
Ok, took me a bit of playing but I managed to get these files. wireshark_test_terminal is me manually doing the commands, wireshark_test_app is doing the same commands (mostly) from the app using the library. One thing I noticed is in the app one, it calls the commands for mv file then umount but then a few packets later gets the response that it can't mv the file because it doesn't exist. This makes me think that there is a race condition and maybe the adb push is happening before the volume mount is finished. The way I have this set up, are those commands supposed to happen one after another? note in the terminal version, I did not perform the move due to not wanting the system to pick up a file that isn't an update and think it is an update. i did confirm after both of these procedures that the same thing is happening, when i use the sdk, the file does not get put on the block device, when I do it from ubuntu terminal manually the file goes where it is supposed to. |
|
I haven't had time to check it out yet but there are two things. If it's a timing issue, I would recommend running a script that issues all the commands at once instead of running them manually (you'll never get the timing issue that way). Also, every adb invocation begins with an adb version check that takes a bit of time, so it may be less prone to appear there. Perhaps just run several commands in one adb shell and see if you can repro the timing issue that way. Also, try adding |
|
Sorry about the delayed response, I added a delay of 1 second after the mount and it did indeed resolve my issue with the file copy to the correct block device. It seems that when you run a shell command, the promise returns before the command has fully finished executing. Is this by design or could this be considered a bug? If it is intended, then we can consider the issue closed and I will find another way to poll that the task has finished to ensure consistency. |
|
I think it's a bug but perhaps you try to run the whole thing as one adb shell command (without adbkit) and see if the timing issue still occurs? I'm wondering if it's just that the mount needs a bit of time to think in any case. When you run the commands by hand, you're introducing a delay between each command, which might make it work even if the timing issue is always present. Even if you run several |
|
I guess the problem is I have to push a file in between the commands so it goes: adb shell mount |
|
Ok I think I just realized what's wrong. You're not consuming the output of the shell command. That would keep it alive. If you're not interested in the output, you can do |
|
in this case I'm not using bluebird, so I worked my code around to do this: this results in the correct actions, but there is at least a 20-30 second delay from the time of the mount return to the time that the file actually starts copying. any idea what could be causing that delay? If I just leave the 1 second delay in there, it works fine and goes fairly quick, although with another issue I was going to type up here in a little bit. |
rsc-rchristensen
changed the title
|
You don't need to use bluebird. The methods return promises. Simply add
that .then as I suggested or capture the result from await and await
readAll on the next line. That way your push won't proceed until all the
data has been read. You must do the same for your other shell calls as
well. Adb has issues with large number of simultaneous shell connections.
You can write your own wrapper method to consume and ignore the output if
you like. If you don't consume the output adb will eventually start reusing
things and you'll get incredibly difficult to explain errors later on.
I don't see anything obviously wrong with that code that would cause a
delay, but it's a bit messy right now due to the changes. I would suggest
posting a new snippet where all the shells are taken care of, and the
callback uses a promise. Also, it would be useful to see your console
output with timestamps. Perhaps also set DEBUG=adb* to see what it's doing
in more detail, too.
If this is on electron or whatever, try running it as a command line script
to see if there's any difference.
…On Fri, Dec 16, 2016 at 22:32 rsc- ***@***.***> wrote:
in this case I'm not using bluebird, so I worked my code around to do this:
console.log('mounting block device');
await client.shell(device.id, 'mount -t vfat /dev/block/mmcblk1p5 /tek2', async (err, result) => {
console.log(`mount complete: err: ${err}\nresult: ${result}`);
console.log('reading result');
const resultString = await adb.util.readAll(result);
console.log(`read result: ${resultString}`);
console.log('starting push of file');
const pushFile = await client.push(device.id, updateFilePath, '/tek2/tmpFirmware.zip');
pushFile.on('progress', (stats) => {
console.log('push file progress');
store.dispatch(DEVICE_FIRMWARE_UPDATE_PROGRESS(device.id, { status: 'copyingFirmwareFile', progress: (stats.bytesTransferred / updateFiles.get(updateFile.fileName).get('total')) * 100 }));
});
pushFile.on('end', async () => {
console.log('push file end');
store.dispatch(DEVICE_FIRMWARE_UPDATE_PROGRESS(device.id, { status: 'finishedCopyingFirmwareFile' }));
console.log('starting rename');
await client.shell(device.id, 'mv /tek2/tmpFirmware.zip /tek2/update.zip');
console.log('finished rename, unmounting');
await client.shell(device.id, 'umount /tek2');
console.log('finished unmount, waiting for reboot');
store.dispatch(DEVICE_FIRMWARE_UPDATE_PROGRESS(device.id, { status: 'waitingForReboot' }));
});
});
this results in the correct actions, but there is at least a 20-30 second
delay from the time of the mount return to the time that the file actually
starts copying. any idea what could be causing that delay? If I just leave
the 1 second delay in there, it works fine and goes fairly quick, although
with another issue I was going to type up here in a little bit.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#58 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAB-_Rj3XhlG2bqHaaXgL1Y3EYRo8TLYks5rIpLtgaJpZM4LJChk>
.
|
|
Will do as soon as I am able, it could be a bit due to the holidays. Feel free to close the ticket if you prefer until I am able to thoroughly test.
On Dec 16, 2016 10:44 AM, Simo Kinnunen <[email protected]> wrote:
You don't need to use bluebird. The methods return promises. Simply add
that .then as I suggested or capture the result from await and await
readAll on the next line. That way your push won't proceed until all the
data has been read. You must do the same for your other shell calls as
well. Adb has issues with large number of simultaneous shell connections.
You can write your own wrapper method to consume and ignore the output if
you like. If you don't consume the output adb will eventually start reusing
things and you'll get incredibly difficult to explain errors later on.
I don't see anything obviously wrong with that code that would cause a
delay, but it's a bit messy right now due to the changes. I would suggest
posting a new snippet where all the shells are taken care of, and the
callback uses a promise. Also, it would be useful to see your console
output with timestamps. Perhaps also set DEBUG=adb* to see what it's doing
in more detail, too.
If this is on electron or whatever, try running it as a command line script
to see if there's any difference.
On Fri, Dec 16, 2016 at 22:32 rsc- ***@***.***> wrote:
in this case I'm not using bluebird, so I worked my code around to do this:
console.log('mounting block device');
await client.shell(device.id, 'mount -t vfat /dev/block/mmcblk1p5 /tek2', async (err, result) => {
console.log(`mount complete: err: ${err}\nresult: ${result}`);
console.log('reading result');
const resultString = await adb.util.readAll(result);
console.log(`read result: ${resultString}`);
console.log('starting push of file');
const pushFile = await client.push(device.id, updateFilePath, '/tek2/tmpFirmware.zip');
pushFile.on('progress', (stats) => {
console.log('push file progress');
store.dispatch(DEVICE_FIRMWARE_UPDATE_PROGRESS(device.id, { status: 'copyingFirmwareFile', progress: (stats.bytesTransferred / updateFiles.get(updateFile.fileName).get('total')) * 100 }));
});
pushFile.on('end', async () => {
console.log('push file end');
store.dispatch(DEVICE_FIRMWARE_UPDATE_PROGRESS(device.id, { status: 'finishedCopyingFirmwareFile' }));
console.log('starting rename');
await client.shell(device.id, 'mv /tek2/tmpFirmware.zip /tek2/update.zip');
console.log('finished rename, unmounting');
await client.shell(device.id, 'umount /tek2');
console.log('finished unmount, waiting for reboot');
store.dispatch(DEVICE_FIRMWARE_UPDATE_PROGRESS(device.id, { status: 'waitingForReboot' }));
});
});
this results in the correct actions, but there is at least a 20-30 second
delay from the time of the mount return to the time that the file actually
starts copying. any idea what could be causing that delay? If I just leave
the 1 second delay in there, it works fine and goes fairly quick, although
with another issue I was going to type up here in a little bit.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#58 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AAB-_Rj3XhlG2bqHaaXgL1Y3EYRo8TLYks5rIpLtgaJpZM4LJChk>
.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub<#58 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AVNpRWWjX48K1qwIZ442fS8-9aQAs1XRks5rIrH1gaJpZM4LJChk>.
|
|
No reason to close it, but please remember to update when you can - so many leave ghost tickets laying around :( I don't really have any experience with mounting inside devices so getting this figured out would be great. |
A little background, I am using your library to push a firmware update file to a custom android device. In order for the device to detect the update file, it has to be in a particular block device at a particular mount point. If I execute these shell commands in order, the push goes through, the file ends up where it is supposed to be, and the device picks up the firmware no problem. When I execute the same commands using your library, the commands execute properly, however the file ends up in the mount target folder and not the actual mounted device.
Working adb commands in ubuntu terminal:
When I do the same commands using your library (using Electron/babel await if that makes a difference):
When I run the commands using adbkit, the device doesn't detect the update, so I shell in, I can see the file in the /mntpnt directory, however the block device is not mounted. However, if I shell in during the transfer, I can see in mount that the device is mounted. Once I mount the block device manually, the files disappear (due to the mount location changing for the dir), and then once I unmount again, the files are back. Any idea or direction would be greatly appreciated.
Notes:
shell sync commands added based on previous comments, does not work with or without although does seem to improve the reliability of the mv command (to rename the file) when sync is called after each shell command
I have also added console logs to the shell commands and the output seems to be what I expect, it just seems like the client.push is what is not performing the way I expect it to.
The text was updated successfully, but these errors were encountered: