Error on large codebase #openrouter #deepseek

[arthurfanti]

I'm currently running the tool agains a codebase with 242 files (after filtering down with flags -i and -e), but I keep getting the same error:

$ python main.py --dir ~/code/ads-hub-advertiser-frontend -i "*.ts" "*.tsx" -e "node_modules/**/*" "*.spec.*"
Starting tutorial generation for: /Users/afanti/code/ads-hub-advertiser-frontend in English language
Crawling directory: /Users/afanti/code/ads-hub-advertiser-frontend...
Fetched 242 files.
Identifying abstractions using LLM...
Identifying abstractions using LLM...
Identifying abstractions using LLM...
Traceback (most recent call last):
  File "/Users/afanti/code/PocketFlow-Tutorial-Codebase-Knowledge/main.py", line 84, in <module>
    main()
  File "/Users/afanti/code/PocketFlow-Tutorial-Codebase-Knowledge/main.py", line 81, in main
    tutorial_flow.run(shared)
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 16, in run
    return self._run(shared)
           ^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 50, in _run
    def _run(self,shared): p=self.prep(shared); o=self._orch(shared); return self.post(shared,p,o)
                                                  ^^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 48, in _orch
    while curr: curr.set_params(p); last_action=curr._run(shared); curr=copy.copy(self.get_next_node(curr,last_action))
                                                ^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 13, in _run
    def _run(self,shared): p=self.prep(shared); e=self._exec(p); return self.post(shared,p,e)
                                                  ^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 33, in _exec
    if self.cur_retry==self.max_retries-1: return self.exec_fallback(prep_res,e)
                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 28, in exec_fallback
    def exec_fallback(self,prep_res,exc): raise exc
                                          ^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 31, in _exec
    try: return self.exec(prep_res)
                ^^^^^^^^^^^^^^^^^^^
  File "/Users/afanti/code/PocketFlow-Tutorial-Codebase-Knowledge/nodes.py", line 152, in exec
    yaml_str = response.strip().split("```yaml")[1].split("```")[0].strip()
               ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range
(3.11.6) 

I'm using openai module and openrouter url/key, like this: (I've tried different models here)

def call_llm(prompt, use_cache: bool = True):    
    from openai import OpenAI
    client = OpenAI(
        base_url="https://openrouter.ai/api/v1",
        api_key=os.environ.get("OPENAI_API_KEY", "your-api-key")
        )
    r = client.chat.completions.create(
        # model="tngtech/deepseek-r1t-chimera:free",
        # model="deepseek/deepseek-r1-zero:free",
        model="meta-llama/llama-4-maverick:free",
        messages=[{"role": "user", "content": prompt}],
        response_format={
            "type": "text"
        },
        reasoning_effort="medium",
        store=False
    )
    return r.choices[0].message.content

And I tried increasing the waiting interval like so:

identify_abstractions = IdentifyAbstractions(max_retries=3, wait=240)

Also, I've confirmed that running python utils/call_llm.py works fine:

$ python utils/call_llm.py                                                                                           
Making call...
Response: I'm doing well, thanks for asking! I'm a large language model, so I don't have feelings or emotions like humans do, but I'm functioning properly and ready to help with any questions or tasks you may have. How about you? How's your day going?
(3.11.6) 

Can anyone assist me with this?
Thanks!

[zachary62]

Hey @arthurfanti could you use a more capable model like gemini 2.5 pro?
This issue is caused because the model is not capable enough to follow the yaml instruction.

[arthurfanti]

Hi @zachary62 thanks for your quick response!
I tried with gemini 2.5 flash thinking (both pro and flash have the same context length of 1,05M) and got the following error:

$ python main.py --dir ~/code/ads-hub-advertiser-frontend -i "*.ts" "*.tsx" -e "node_modules/**/*" "*.spec.*"
Starting tutorial generation for: /Users/afanti/code/ads-hub-advertiser-frontend in English language
Crawling directory: /Users/afanti/code/ads-hub-advertiser-frontend...
Fetched 242 files.
Identifying abstractions using LLM...
Identified 8 abstractions.
Analyzing relationships using LLM...
Generated project summary and relationship details.
Determining chapter order using LLM...
Determined chapter order (indices): [0, 1, 5, 3, 2, 7, 4, 6]
Preparing to write 8 chapters...
Writing chapter 1 for: Nordic Application Structure
 using LLM...
Writing chapter 1 for: Nordic Application Structure
 using LLM...
Writing chapter 1 for: Nordic Application Structure
 using LLM...
Writing chapter 1 for: Nordic Application Structure
 using LLM...
Writing chapter 1 for: Nordic Application Structure
 using LLM...
Traceback (most recent call last):
  File "/Users/afanti/code/PocketFlow-Tutorial-Codebase-Knowledge/main.py", line 84, in <module>
    main()
  File "/Users/afanti/code/PocketFlow-Tutorial-Codebase-Knowledge/main.py", line 81, in main
    tutorial_flow.run(shared)
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 16, in run
    return self._run(shared)
           ^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 50, in _run
    def _run(self,shared): p=self.prep(shared); o=self._orch(shared); return self.post(shared,p,o)
                                                  ^^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 48, in _orch
    while curr: curr.set_params(p); last_action=curr._run(shared); curr=copy.copy(self.get_next_node(curr,last_action))
                                                ^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 13, in _run
    def _run(self,shared): p=self.prep(shared); e=self._exec(p); return self.post(shared,p,e)
                                                  ^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 37, in _exec
    def _exec(self,items): return [super(BatchNode,self)._exec(i) for i in (items or [])]
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 37, in <listcomp>
    def _exec(self,items): return [super(BatchNode,self)._exec(i) for i in (items or [])]
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 33, in _exec
    if self.cur_retry==self.max_retries-1: return self.exec_fallback(prep_res,e)
                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 28, in exec_fallback
    def exec_fallback(self,prep_res,exc): raise exc
                                          ^^^^^^^^^
  File "/Users/afanti/.pyenv/versions/3.11.6/lib/python3.11/site-packages/pocketflow/__init__.py", line 31, in _exec
    try: return self.exec(prep_res)
                ^^^^^^^^^^^^^^^^^^^
  File "/Users/afanti/code/PocketFlow-Tutorial-Codebase-Knowledge/nodes.py", line 597, in exec
    chapter_content = call_llm(prompt)
                      ^^^^^^^^^^^^^^^^
  File "/Users/afanti/code/PocketFlow-Tutorial-Codebase-Knowledge/utils/call_llm.py", line 124, in call_llm
    return r.choices[0].message.content
           ~~~~~~~~~^^^
TypeError: 'NoneType' object is not subscriptable
(3.11.6)

Do you think it is due the model?
update: it definitely was the model, I'm proving with others now and it is generating the chapters

[TheHawk3r]

@arthurfanti what other models have you used that work ? Any local ones ?

[toyosky]

@arthurfanti what other models have you used that work ? Any local ones ?

Have you finally solve this problem? how did you?

[TheHawk3r]

@arthurfanti here you go, but using it on large codebases will not work, too many tokens needed.
#13