Did anyone get this code to run and generate a new manuscript on a "self"seeded topic?

[Ramzee-S]

Did anyone get this code to run and generate a new manuscript on a user-seeded topic?

Is it enough to give it some ideas? And if yes how to do it?
Will it based on that generate the experiment.py and other files? Or do we need to specify the experiment.py and other files for each idea ourselves? A lot of things are not clear to me.

The readme states
_## Making your own Template

If there is an area of study you would like The AI Scientist to explore, it should be very easy to create your own templates. In general, follow the structure of the existing templates, which consists of:

experiment.py -- This is a single file where the 'meat' of the content is. It takes in an argument for out_dir, which is where it should create the folder and save the relevant information from the run.
plot.py -- This should take in the information from the run folders and create plots. The code should be clear and easy to edit.
prompt.json -- Put information about your template here.
seed_ideas.json -- Put example ideas here. You can also try to generate ideas without any examples, and then pick the best one or two to put here.
latex/template.tex -- We recommend using our latex folder, but be sure to replace the pre-loaded citations with ones that you would expect to be more relevant.

But this seems to require that we not only put example ideas, but for each example ideas, also need to prepare the experiment.py, the plot.py prompt.py. Is this really true, or am i misunderstanding something?

Any help would be appreciated.

[BradKML]

I had the same idea as well! Can't believe I wrote the idea elsewhere, and also it might be possible to get another agentic tool to do this kind of leg-work (e.g. OpenHands/Kevin) #141 (comment) SmartManoj/Kevin#136 (comment)

[tonyboutwell]

I put claude code on it and had it help me write that part. It seems to have worked great. I got 15 papers out of after running for about 48 hours. My wife is not going to be happy about the api cost on that... but anthropic will be. :) Ai Scientist did everything (it looks like) correctly except write out the PDF's correctly to finish it up. So after it was finished I had claude code find what was missing that caused the error and then it rendered out the final PDF's for me.

[Ramzee-S]

Hi, Thanks for the reply. Could you be maybe be a bit more specific? My guess now is that you wrote your research idea, to Claude, and then provided the experiment.py, plot.py and prompt.py as examples and then asked to modify it? Was that all, i mean will it then work. or is it then still a lot of tweaking. I mean it is even interesting to see what it can do. But eventually would like to use it as a tool, to save time, not as a tool. But i guess if you wrote 14 papers it is working. Roughly how much API cost per paper did it take?

[tonyboutwell]

I tried first with the web version of claude to help write the template files based on the examples they showed but it had lots of errors when I ran it. So I had the idea to try claude code on it. It runs on your code local and can see the whole code base which I think helped it understand the goal and how best to write the template. I gave it my research and it created the template files in seconds and we were off to the races. I would estimate it was about $8 to $10 per paper on the cost and that was using the latest claude api (it supported) which I think was claude 3.5. I can double check for sure when I get home from work tonight.