Navigation tab is empty (ctags) #4185

shusterboris · 2023-02-25T17:35:55Z

shusterboris
Feb 25, 2023

Dear friends!
Click on Navigation tab shows empty window. It's work fine on the small source directory. Just I create full index, it stops working. Container's log contains follow error for each clicking of the tab:

24-Feb-2023 18:46:31.742 WARNING [http-nio-8080-exec-14] org.apache.jsp.list_jsp._jspService doCtags()
        java.lang.NullPointerException
                at org.apache.jsp.list_jsp._jspService(list_jsp.java:1227)
                at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
                at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:814)
                at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:456)
                at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:380)
                at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:328)
                at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:814)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:223)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.opengrok.web.CookieFilter.doFilter(CookieFilter.java:50)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.opengrok.web.AuthorizationFilter.doFilter(AuthorizationFilter.java:95)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.opengrok.web.StatisticsFilter.doFilter(StatisticsFilter.java:67)
                at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:185)
                at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:158)
                at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:177)
                at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
                at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
                at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:119)
                at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
                at org.apache.catalina.valves.RequestFilterValve.process(RequestFilterValve.java:378)
                at org.apache.catalina.valves.RemoteAddrValve.invoke(RemoteAddrValve.java:56)
                at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:690)
                at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
                at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:357)
                at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:400)
                at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
                at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:859)
                at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1734)
                at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
                at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
                at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
                at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                at java.base/java.lang.Thread.run(Thread.java:829)

OpenGrok 1.7.42, standard container's configuration
OS
Linux 78ffa7bbff33 4.18.0-425.10.1.el8_7.x86_64 #1 SMP Wed Dec 14 16:00:01 EST 2022 x86_64 x86_64 x86_64 GNU/Linux

JAVA
openjdk 11.0.17 2022-10-18
OpenJDK Runtime Environment Temurin-11.0.17+8 (build 11.0.17+8)
OpenJDK 64-Bit Server VM Temurin-11.0.17+8 (build 11.0.17+8, mixed mode, sharing)
root@78ffa7bbff33:/opengrok#

Tomcat 10

Do you have any idea? Thank you advance!

UPDATED: I added many extensions to exclude list in configuration.xml to avoid indexing of binaries, images, libraries and it's started working. But I don't understand real reason of the issue.

Answered by shusterboris

Feb 27, 2023

Hi! Thank you for short answer! I am using 1.7.42Respectfully, BorisОтправлено с телефона-------- Исходное сообщение --------От: Vladimir Kotal ***@***.***>Дата: пн, 27 февр. 2023 г., 10:46Кому: oracle/opengrok ***@***.***>Копия: Boris Shuster ***@***.***>, Author ***@***.***>Тема: Re: [oracle/opengrok] Navigation tab is empty (ctags) (Discussion #4185) What OpenGrok version are you running ? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

View full answer

shusterboris · 2023-02-27T09:19:43Z

shusterboris
Feb 27, 2023
Author

Hi! Thank you for short answer! I am using 1.7.42Respectfully, BorisОтправлено с телефона-------- Исходное сообщение --------От: Vladimir Kotal ***@***.***>Дата: пн, 27 февр. 2023 г., 10:46Кому: oracle/opengrok ***@***.***>Копия: Boris Shuster ***@***.***>, Author ***@***.***>Тема: Re: [oracle/opengrok] Navigation tab is empty (ctags) (Discussion #4185) What OpenGrok version are you running ? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

5 replies

vladak Feb 27, 2023
Maintainer

Looking at the expanded list.jsp for 1.7.42:

  1218                      if (g == AbstractAnalyzer.Genre.PLAIN) {
  1219                          Definitions defs = null;
  1220                          ObjectPool<Ctags> ctagsPool = cfg.getEnv().getIndexerParallelizer().getCtagsPool();
  1221                          int tries = 2;
  1222                          while (cfg.getEnv().isWebappCtags()) {
  1223                              Ctags ctags = ctagsPool.get();
  1224                              try {
  1225                                  ctags.setTabSize(project != null ?
  1226                                  project.getTabSize() : 0);
  1227                                  defs = ctags.doCtags(tempf.getPath());
  1228                                  break;
  1229                              } catch (InterruptedException ex) {

there are 2 ways how the null pointer exception can happen: either ctags object is null or the tempf File object is null.

This will only happen if the webapp configuration was set with so that isWebappCtags() on line 1222 produces true, i.e. setWebappCtags in the read-only configuration is set to true. I assume this is your case.

The ctags object returned from the pool on line 1223 has to go through a validator that makes sure it is not null:

opengrok/opengrok-indexer/src/main/java/org/opengrok/indexer/analysis/CtagsValidator.java

Lines 30 to 32 in aeef436

    
           public boolean isValid(Ctags ctags) { 
        
               return ctags != null && !ctags.isClosed(); 
        
           }

I think that makes the second possibility more probably, i.e. tempf is null. This can happen e.g. when the contents for given file at given revision cannot be retrieve or there is some other problem:

opengrok/opengrok-web/src/main/webapp/xref.jspf

Lines 57 to 77 in aeef436

    
           File tempf = null; 
        
           try { 
        
               if (rev.isBlank() || rev.equals(DUMMY_REVISION)) { 
        
                   in = new BufferedInputStream(new FileInputStream(resourceFile)); 
        
               } else { 
        
                   tempf = File.createTempFile("ogtags", basename); 
        
                   if (HistoryGuru.getInstance().getRevision(tempf, resourceFile.getParent(), basename, rev)) { 
        
                       in = new BufferedInputStream(new FileInputStream(tempf)); 
        
                   } else { 
        
                       tempf.delete(); 
        
                       tempf = null; 
        
                   } 
        
               } 
        
           } catch (Exception e) { 
        
               // fall through to error message 
        
               error = e.getMessage(); 
        
               if (tempf != null) { 
        
                   tempf.delete(); 
        
                   tempf = null; 
        
               } 
        
           }

The JSP code listed above should be more careful w.r.t. the tempf being null.

Clicking on the 'Navigate' link should not produce any HTTP request, it merely displays the information embedded in the page, I think. What is the URL that triggers the problem ?

shusterboris Feb 27, 2023
Author

The property WebappCtags in the read-only configuration is really was set to true. I will try to remove and reindex it.

vladak Feb 27, 2023
Maintainer

No need to reindex, just reflect the change in whatever configuration file is the webapp using and restart the webapp.

shusterboris Mar 2, 2023
Author

Thank you. The webAppCtags in read only configuration was the root of issue. Thank you for quick help

vladak Mar 2, 2023
Maintainer

Well, the root cause lies elsewhere, I believe. The webAppCtags set to false merely avoids the problem. 1.8.1 contains the check for the null doCtags() argument so webAppCtags should be possible to set to true once again in your case in that version.

This still does not provide an answer to the question why the temporary file was not created. My wild guess is that getRevision() failed for some reason. The webapp logs might contain some clues.

shusterboris · 2023-03-05T14:09:06Z

shusterboris
Mar 5, 2023
Author

So, reason is webAppCtags=true in read only configuration. So discussion can be closed. Thanks!

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Navigation tab is empty (ctags) #4185

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Replies: 2 comments 5 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Navigation tab is empty (ctags) #4185

Uh oh!

Uh oh!

shusterboris Feb 25, 2023

Replies: 2 comments · 5 replies

Uh oh!

shusterboris Feb 27, 2023 Author

Uh oh!

vladak Feb 27, 2023 Maintainer

Uh oh!

shusterboris Feb 27, 2023 Author

Uh oh!

vladak Feb 27, 2023 Maintainer

Uh oh!

shusterboris Mar 2, 2023 Author

Uh oh!

vladak Mar 2, 2023 Maintainer

Uh oh!

shusterboris Mar 5, 2023 Author

shusterboris
Feb 25, 2023

Replies: 2 comments 5 replies

shusterboris
Feb 27, 2023
Author

vladak Feb 27, 2023
Maintainer

shusterboris Feb 27, 2023
Author

vladak Feb 27, 2023
Maintainer

shusterboris Mar 2, 2023
Author

vladak Mar 2, 2023
Maintainer

shusterboris
Mar 5, 2023
Author