Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

better error reporting needed #90

Open
individual-it opened this issue May 30, 2017 · 15 comments
Open

better error reporting needed #90

individual-it opened this issue May 30, 2017 · 15 comments

Comments

@individual-it
Copy link

I'm running tests with Mink on Saucelabs (Sauce Connect 4.4.6)
At some point tests with firefox started to fail with
Expected to be on "http://localhost/owncloud-core/index.php/login" but found "" instead (SensioLabs\Behat\PageObjectExtension\PageObject\Exception\UnexpectedPageException)

It took me a while and some network sniffing to find out that the Error Message send from Sauce Connect actually is:
The requested combination of browser, version and OS is unsupported by the requested Selenium version and would lead to a test failure. Please set a different Selenium version, or set none to get the default, working Selenium version for this platform: firefox 47.0. on Windows 2008 with Selenium 2.31.0

I had to set the selenium version. No idea why it did work automatically before. But a better error reporting on the console or log file would be good.
I did report this problem in MinkSelenium2Driver but was told the problem is in the PageObjectExtension, so I open this issue here

@jakzal
Copy link
Member

jakzal commented Jun 6, 2017

@individual-it is this message actually returned in a page body?

@individual-it
Copy link
Author

no, I only found the message looking into the network transfer and if I remember correctly it was in a HTTP return message. If you need I can try to reproduce it and send you a full network log

@jakzal
Copy link
Member

jakzal commented Jun 6, 2017

Would be great, thanks!

Currently I'm not sure if we can do anything about it, especially if the error is only shown in sauselabs logs (like locally some errors are shown by selenium only in its console). We might not be able to retrieve the actual message.

@individual-it
Copy link
Author

individual-it commented Jun 6, 2017

Without setting the selenium version is does not work.

very first HTTP request

POST /wd/hub/session HTTP/1.1
Authorization: Basic xxxxxxxxxxxxxxx=
Host: localhost:4445
Content-Type: application/json;charset=UTF-8
Accept: application/json;charset=UTF-8
Content-Length: 339

{"desiredCapabilities":{"browserName":"firefox","version":"47.0","platform":"","browserVersion":"9","browser":"firefox","name":" - ","deviceOrientation":"portrait","deviceType":"tablet","selenium-version":"2.31.0","tags":["localPC","PHP 7.1.5-1+0~20170522123046.25+jessie~1.gbpb8686b"],"maxDuration":"3600","ignoreZoomSetting":false}}

HTTP/1.1 404 Not Found
Transfer-Encoding: chunked
Date: Tue, 06 Jun 2017 15:02:21 GMT
Content-Type: text/plain
Server: monocle/0.38

123
The requested combination of browser, version and OS is unsupported by the requested Selenium version and would lead to a test failure. Please set a different Selenium version, or set none to get the default, working Selenium version for this platform: firefox 47.0. on  with Selenium 2.31.0
0

next request, actually trying to load the page:

POST /wd/hub/session/url HTTP/1.1
Authorization: Basic xxxxxxxxxxxxxxx=
Host: localhost:4445
Content-Type: application/json;charset=UTF-8
Accept: application/json;charset=UTF-8
Content-Length: 61

{"url":"http:\/\/localhost\/owncloud-core\/index.php\/login"}
HTTP/1.1 404 Not Found
Transfer-Encoding: chunked
Date: Tue, 06 Jun 2017 15:10:05 GMT
Content-Type: text/plain
Server: monocle/0.38

e6
ERROR The test with session id url has already finished, and can't receive further commands.
You can learn more at https://saucelabs.com/jobs/url
For help, please check https://wiki.saucelabs.com/display/DOCS/Common+Error+Messages
0

as you can see there is a 404 reply from saucelabs.
I wonder if the exception can be thrown already after the first 404 reply, and the error message being displayed

here is the full trace:

     SensioLabs\Behat\PageObjectExtension\PageObject\Exception\UnexpectedPageException: Expected to be on "http://localhost/owncloud-core/index.php/login" but found "" instead in /home/artur/www/owncloud-core/lib/composer/sensiolabs/behat-page-object-extension/src/SensioLabs/Behat/PageObjectExtension/PageObject/Page.php:217
      Stack trace:
      #0 /home/artur/www/owncloud-core/lib/composer/sensiolabs/behat-page-object-extension/src/SensioLabs/Behat/PageObjectExtension/PageObject/Page.php(186): SensioLabs\Behat\PageObjectExtension\PageObject\Page->verifyUrl(Array)
      #1 /home/artur/www/owncloud-core/lib/composer/sensiolabs/behat-page-object-extension/src/SensioLabs/Behat/PageObjectExtension/PageObject/Page.php(58): SensioLabs\Behat\PageObjectExtension\PageObject\Page->verify(Array)
      #2 /tmp/ProxyManagerGeneratedProxy__PM__PageLoginPageGeneratedb755cbf3bea1a48cfcd917422d91a2e8.php(134): SensioLabs\Behat\PageObjectExtension\PageObject\Page->open(Array)
      #3 /home/artur/www/owncloud-core/tests/ui/features/bootstrap/LoginContext.php(50): ProxyManagerGeneratedProxy\__PM__\Page\LoginPage\Generatedb755cbf3bea1a48cfcd917422d91a2e8->open()
      #4 [internal function]: LoginContext->iAmOnTheLoginPage()
      #5 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Call/Handler/RuntimeCallHandler.php(103): call_user_func_array(Array, Array)
      #6 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Call/Handler/RuntimeCallHandler.php(59): Behat\Testwork\Call\Handler\RuntimeCallHandler->executeCall(Object(Behat\Behat\Definition\Call\DefinitionCall))
      #7 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Call/CallCenter.php(140): Behat\Testwork\Call\Handler\RuntimeCallHandler->handleCall(Object(Behat\Behat\Definition\Call\DefinitionCall))
      #8 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Call/CallCenter.php(96): Behat\Testwork\Call\CallCenter->handleCall(Object(Behat\Behat\Definition\Call\DefinitionCall))
      #9 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/Tester/Runtime/RuntimeStepTester.php(125): Behat\Testwork\Call\CallCenter->makeCall(Object(Behat\Behat\Definition\Call\DefinitionCall))
      #10 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/Tester/Runtime/RuntimeStepTester.php(73): Behat\Behat\Tester\Runtime\RuntimeStepTester->testDefinition(Object(Behat\Behat\Context\Environment\InitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\StepNode), Object(Behat\Behat\Definition\SearchResult), false)
      #11 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/Hook/Tester/HookableStepTester.php(74): Behat\Behat\Tester\Runtime\RuntimeStepTester->test(Object(Behat\Behat\Context\Environment\InitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\StepNode), false)
      #12 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/EventDispatcher/Tester/TickingStepTester.php(59): Behat\Behat\Hook\Tester\HookableStepTester->test(Object(Behat\Behat\Context\Environment\InitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\StepNode), false)
      #13 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/EventDispatcher/Tester/EventDispatchingStepTester.php(73): Behat\Behat\EventDispatcher\Tester\TickingStepTester->test(Object(Behat\Behat\Context\Environment\InitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\StepNode), false)
      #14 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/Tester/StepContainerTester.php(59): Behat\Behat\EventDispatcher\Tester\EventDispatchingStepTester->test(Object(Behat\Behat\Context\Environment\InitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\StepNode), false)
      #15 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/Tester/Runtime/RuntimeScenarioTester.php(76): Behat\Behat\Tester\StepContainerTester->test(Object(Behat\Behat\Context\Environment\InitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\ScenarioNode), false)
      #16 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/Hook/Tester/HookableScenarioTester.php(74): Behat\Behat\Tester\Runtime\RuntimeScenarioTester->test(Object(Behat\Behat\Context\Environment\InitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\ScenarioNode), false)
      #17 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/EventDispatcher/Tester/EventDispatchingScenarioTester.php(103): Behat\Behat\Hook\Tester\HookableScenarioTester->test(Object(Behat\Behat\Context\Environment\InitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\ScenarioNode), false)
      #18 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/Tester/Runtime/IsolatingScenarioTester.php(69): Behat\Behat\EventDispatcher\Tester\EventDispatchingScenarioTester->test(Object(Behat\Behat\Context\Environment\InitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\ScenarioNode), false)
      #19 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/Tester/Runtime/RuntimeFeatureTester.php(84): Behat\Behat\Tester\Runtime\IsolatingScenarioTester->test(Object(Behat\Behat\Context\Environment\UninitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), Object(Behat\Gherkin\Node\ScenarioNode), false)
      #20 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/Hook/Tester/HookableFeatureTester.php(72): Behat\Behat\Tester\Runtime\RuntimeFeatureTester->test(Object(Behat\Behat\Context\Environment\UninitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), false)
      #21 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Behat/EventDispatcher/Tester/EventDispatchingFeatureTester.php(71): Behat\Behat\Hook\Tester\HookableFeatureTester->test(Object(Behat\Behat\Context\Environment\UninitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), false)
      #22 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Tester/Runtime/RuntimeSuiteTester.php(63): Behat\Behat\EventDispatcher\Tester\EventDispatchingFeatureTester->test(Object(Behat\Behat\Context\Environment\UninitializedContextEnvironment), Object(Behat\Gherkin\Node\FeatureNode), false)
      #23 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Hook/Tester/HookableSuiteTester.php(73): Behat\Testwork\Tester\Runtime\RuntimeSuiteTester->test(Object(Behat\Behat\Context\Environment\UninitializedContextEnvironment), Object(Behat\Testwork\Specification\GroupedSpecificationIterator), false)
      #24 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/EventDispatcher/Tester/EventDispatchingSuiteTester.php(72): Behat\Testwork\Hook\Tester\HookableSuiteTester->test(Object(Behat\Behat\Context\Environment\UninitializedContextEnvironment), Object(Behat\Testwork\Specification\GroupedSpecificationIterator), false)
      #25 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Tester/Runtime/RuntimeExercise.php(71): Behat\Testwork\EventDispatcher\Tester\EventDispatchingSuiteTester->test(Object(Behat\Behat\Context\Environment\UninitializedContextEnvironment), Object(Behat\Testwork\Specification\GroupedSpecificationIterator), false)
      #26 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/EventDispatcher/Tester/EventDispatchingExercise.php(70): Behat\Testwork\Tester\Runtime\RuntimeExercise->test(Array, false)
      #27 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Ordering/OrderedExercise.php(80): Behat\Testwork\EventDispatcher\Tester\EventDispatchingExercise->test(Array, false)
      #28 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Tester/Cli/ExerciseController.php(149): Behat\Testwork\Ordering\OrderedExercise->test(Array, false)
      #29 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Tester/Cli/ExerciseController.php(108): Behat\Testwork\Tester\Cli\ExerciseController->testSpecifications(Object(Symfony\Component\Console\Input\ArgvInput), Array)
      #30 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Cli/Command.php(63): Behat\Testwork\Tester\Cli\ExerciseController->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
      #31 /home/artur/www/owncloud-core/lib/composer/symfony/console/Command/Command.php(262): Behat\Testwork\Cli\Command->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
      #32 /home/artur/www/owncloud-core/lib/composer/symfony/console/Application.php(826): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
      #33 /home/artur/www/owncloud-core/lib/composer/symfony/console/Application.php(189): Symfony\Component\Console\Application->doRunCommand(Object(Behat\Testwork\Cli\Command), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
      #34 /home/artur/www/owncloud-core/lib/composer/behat/behat/src/Behat/Testwork/Cli/Application.php(124): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
      #35 /home/artur/www/owncloud-core/lib/composer/symfony/console/Application.php(120): Behat\Testwork\Cli\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
      #36 /home/artur/www/owncloud-core/lib/composer/behat/behat/bin/behat(34): Symfony\Component\Console\Application->run()
      #37 {main}

a request with selenium version set it does work

POST /wd/hub/session HTTP/1.1
Authorization: Basic xxxxxxxxxxxxxx=
Host: localhost:4445
Content-Type: application/json;charset=UTF-8
Accept: application/json;charset=UTF-8
Content-Length: 366

{"desiredCapabilities":{"browserName":"firefox","version":"47.0","platform":"","browserVersion":"9","browser":"firefox","name":" - ","deviceOrientation":"portrait","deviceType":"tablet","selenium-version":"2.31.0","tags":["localPC","PHP 7.1.5-1+0~20170522123046.25+jessie~1.gbpb8686b"],"seleniumVersion":"2.53.1","maxDuration":"3600","ignoreZoomSetting":false}}HTTP/1.1 200 OK
Transfer-Encoding: chunked
Date: Tue, 06 Jun 2017 15:06:17 GMT
Content-Type: application/json;charset=UTF-8
Server: monocle/0.38

258
{"status": 0, "sessionId": "7c71f1763ab945138191e3ae87079ace", "value": {"rotatable": false, "takesScreenshot": true, "acceptSslCerts": true, "cssSelectorsEnabled": true, "javascriptEnabled": true, "webdriver.remote.sessionid": "7c71f1763ab945138191e3ae87079ace", "databaseEnabled": true, "locationContextEnabled": true, "platform": "WINDOWS", "browserName": "firefox", "version": "47.0.1", "hasMetadata": true, "nativeEvents": false, "applicationCacheEnabled": true, "webStorageEnabled": true, "handlesAlerts": true}, "state": null, "hCode": 19076627, "class": "org.openqa.selenium.remote.Response"}
0
POST /wd/hub/session/7c71f1763ab945138191e3ae87079ace/url HTTP/1.1
Authorization: Basic xxxxxxxx=
Host: localhost:4445
Content-Type: application/json;charset=UTF-8
Accept: application/json;charset=UTF-8
Content-Length: 61

{"url":"http:\/\/localhost\/owncloud-core\/index.php\/login"}HTTP/1.1 200 OK
Transfer-Encoding: chunked
Date: Tue, 06 Jun 2017 15:06:25 GMT
Content-Type: application/json;charset=UTF-8
Server: monocle/0.38

a4
{"status": 0, "sessionId": "7c71f1763ab945138191e3ae87079ace", "value": null, "state": "success", "hCode": 19248061, "class": "org.openqa.selenium.remote.Response"}
0

@jakzal
Copy link
Member

jakzal commented Jun 6, 2017

This is useful. What's the status code of the first POST /wd/hub/session (notice the session id is missing, so mink doesn't seem to be validating it)?

To me it looks like starting the session fails but, mink doesn't raise an exception and carries on.

@individual-it
Copy link
Author

its HTTP/1.1 404 Not Found sorry the line-break was not in place.

yes some part of the code gets the 404 but does not react on it. Do you think its a Mink problem?

@jakzal
Copy link
Member

jakzal commented Jun 7, 2017

Perhaps it's a problem with https://github.com/instaclick/php-webdriver

@individual-it
Copy link
Author

what to do? report there?
I could open an issue there, but you might be able to give better comments as you are surly deeper into it

@jakzal
Copy link
Member

jakzal commented Jun 7, 2017

imo the best thing to do would be to reproduce the issue with webdriver alone, and then report it to them or possibly fix it :)

I only had a brief look, but looks like the session id is empty after calling session() on the webdriver, and I don't think such a situation should be allowed. I might be wrong though.

@bartonhammond
Copy link

@individual-it What tool did you use for network sniffing?

@individual-it
Copy link
Author

@bartonhammond it must have been wireshark, because that's what I'm always using

@bartonhammond
Copy link

@individual-it I installed wireshark and tried it. I don't see any HTTP requests. I capture events only when running the command line "vendor/bin/behat". As soon as the test fails, which is quick, I stop the capture. I guess I don't know what to look for. I'm suspecting that I'm not even making the request.

@jakzal
Copy link
Member

jakzal commented May 9, 2018

@bartonhammond the actuall request is not made via behat, but the webdriver if this helps

@aik099
Copy link

aik099 commented Mar 15, 2024

I've checked and the instaclick/php-webdriver package is now considering any 4xx HTTP response as an error and throws an exception using this code:

throw WebDriverException::factory(
    WebDriverException::CURL_EXEC,
    'Webdriver http error: ' . $httpCode . ', payload :' . substr($rawResult, 0, 1000)
);

@individual-it , now you should be getting a proper error, when Sauce Labs wasn't able to start the session due to the impossible os/browser combination given.

@jakzal
Copy link
Member

jakzal commented May 8, 2024

@aik099 if you wait long enough, the problem solves itself... 😂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants