Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PhantomJS junk processes #45

Open
mgolosova opened this issue Oct 9, 2017 · 0 comments
Open

PhantomJS junk processes #45

mgolosova opened this issue Oct 9, 2017 · 0 comments

Comments

@mgolosova
Copy link
Collaborator

When we stop Kafka Streams application, that runs a Dataflow stage working with CDS via Invenio client (which runs PhantomJS browser to interact with CDS), we are very likely to have some PhantomJS processes unstopped. After a number of run/stop/restart actions we can even run out of free slots for a new thread/process.

Killing all the PhantomJS helps, of course, but it is not to be a regular solution.

The issue is under investigation.

Where to look:

  • pyDKB: dataflow.cds (the modifies Invenio client)
  • Kafka Streams: everything connected to the application stop, shutdown etc
  • JAVA: java.lang.Process destroy() realization
  • NodeJS: the PhantomJS is run via node, so maybe there`s something?

...to be continued.

@mgolosova mgolosova self-assigned this Oct 9, 2017
mgolosova added a commit that referenced this issue Oct 9, 2017
When the application stops, it is very rare that it will shutdown
  all the subprocesses (see Issue #45).
mgolosova added a commit that referenced this issue Oct 28, 2017
There`s no guarantee that after `with...as` clause the main program
  is immediately goes to its end, so we need to propagate
  all the exceptions, including KeyboardInterrupt.

As we exit the `with...as` clause, we need to destroy the browser
  process. If there was an exception propagated, `__del__()` method
  is not called (at least before the end of the main program),
  so we need call `delete()` manually.

To ensure that phantomjs received the SIGTERM, we send it manually
  in the `delete()`. It doesn`t seem to help with Issue #45, however.
mgolosova added a commit that referenced this issue Oct 28, 2017
Another try to deal with Issue #45; doesn`t seem to work, though.
mgolosova added a commit that referenced this issue Oct 28, 2017
There`s no guarantee that after `with...as` clause the main program
  is immediately goes to its end, so we need to propagate
  all the exceptions, including KeyboardInterrupt.

As we exit the `with...as` clause, we need to destroy the browser
  process. If there was an exception propagated, `__del__()` method
  is not called (at least before the end of the main program),
  so we need call `delete()` manually.

To ensure that phantomjs received the SIGTERM, we send it manually
  in the `delete()`. It doesn`t seem to help with Issue #45, however.
mgolosova added a commit that referenced this issue Oct 28, 2017
Another try to deal with Issue #45; doesn`t seem to work, though.
mgolosova added a commit that referenced this issue Nov 1, 2017
When the application stops, it is very rare that it will shutdown
  all the subprocesses (see Issue #45).
mgolosova added a commit that referenced this issue Nov 1, 2017
When the application stops, it is very rare that it will shutdown
  all the subprocesses (see Issue #45).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant