INdexing stuck in middle

[kalpeshsomani007]

It always stuck on below eroor

Could not notify the webapp that project device:indexed=true,history=true was indexed: InboundJaxrsResponse{context=ClientResponse{method=PUT, uri=http://localhost:9999/source/api/v1/projects/device/indexed, status=404, reason=Not Found}}
below logs files attached

opengrok0.0.log.zip

[vladak]

You need to deploy the webapp first to the URL. HTTP 404 error means there is nothing on that URL.

[kalpeshsomani007]

@vladak webapp already deployed

[kalpeshsomani007]

Webapps is deployed

I have run the below command
opengrok-indexer -J=-Djava.util.logging.config.file=etc/logging.properties -a dist/lib/opengrok-1.8.2.jar -- -c /usr/local/bin/ctags -s src/aosp-caf/ -d data -H -P -S -G -W etc/configuration.xml -U http://localhost:9999/source

Most of the time it stuck on below line
WARNING: Failed to set last modified time on '/home/rshaik/Documents/workspace/opengrok/data/timestamp', used for timestamping the index database.
opengrok0.0.log.zip

[vladak]

If the webapp was deployed successfully, you would not be getting the HTTP 404 error when trying to access the URL on which it should be listening for requests (http://localhost:9999/source/ in your case). The above screenshot merely says that the application server is up, however does not say anything about deployed web applications, specifically the OpenGrok web app.

In my case, Tomcat is listening on port 8080 and when I access http://localhost:8080/manager/html (after properly configuring the admin user in Tomcat's tomcat-users.xml configuration and restarting it), it shows this:

As visible in the screenshot, in my case, the OpenGrok web app is deployed to URL http://localhost:8080/opengrok_web_war/

[vladak]

Anyhow, let me explain why the web app needs to be accessible for the indexer: when the indexer is running, it needs to change the state of the of the webapp, in particular the configuration generated during indexing needs to be set in the web application so that it knows what projects/repositories are present. These changes happen via by sending API requests to the web app URL. Also, during indexing, the indexer sends some API requests to the webapp to mark projects as indexed and also to refresh index searchers in the web app so that search results can reflect the latest state of the input data.

Now, it is perfectly fine to run the indexer without the web app being accessible, just do not use the -U indexer option. It really depends on what is your goal of the indexing.

[kalpeshsomani007]

Do i need to change the command ??
opengrok-indexer -J=-Djava.util.logging.config.file=etc/logging.properties -a dist/lib/opengrok-1.8.2.jar -- -c /usr/local/bin/ctags -s src/aosp-caf/ -d data -H -P -S -G -W etc/configuration.xml -U http://localhost:9999/source

THIS COMMAND WORKS PERFECTLY WHEN WE HAVE SHORT DIRECTORy. bUT iF WE HAVE LARGE DIRECTORY IT'S GETTING STUCK MOST OF THE TIME

LET'S SAY IN ABOVE LOGS INDEXING COMPLETED 28 DIRECTORY OUT OF 34. YOU CAN SEE FROM THE LOGS
SINGLE DIRECTORY IT'S DEPLOYING PERFECTLY.

CONFIGURATION WHICH I USE
TOMCAT -10
OPENGROK -1.8.2
CTAGE - UNIVERSAL CTAG

[vladak]

Huh, is your caps-lock key stuck or something ? Anyhow, the indexer options look good to me. Of course, it depends on what is the intention, in particular w.r.t. the -U option.

If the indexer seems to be stuck, then my advice would be to raise the log level (see https://github.com/oracle/opengrok/wiki/Debugging#web-application) and see what exactly is going on. Maybe also use the --progress indexer option.

[kalpeshsomani007]

yes caps lock stuck sorry for that.

I want to deploy the Whole android code to opengrok but in the middle of indexing it is getting stuck.

is there any command where if the file is taking time it should skip after some time & go on next file ??

[vladak]

See my previous pieces of advice.