Hi, I have a question about the dataset configurations.
Does
data/meta-llama/Meta-Llama-3.1-8B-Instruct/banking77/None/True/random_biased_end_only_positive/1.0/False/False/1.0/100/data.pickle
correspond to the Naive + ICRL setting,
while
data/meta-llama/Meta-Llama-3.1-8B-Instruct/banking77/None/True/random_unbiased_only_positive/0.1/False/False/1.0/100/data.pickle
corresponds to the Stochastic ICRL setting?
I inspected both files and found that the second one matches Figure 2 in your paper. However, the first one only reaches about 43% when I plot it using the released data, whereas in your paper it appears to exceed 60%.
Could you clarify whether I am mapping the paths to the correct experimental settings?
Thanks!
Hi, I have a question about the dataset configurations.
Does
data/meta-llama/Meta-Llama-3.1-8B-Instruct/banking77/None/True/random_biased_end_only_positive/1.0/False/False/1.0/100/data.pickle
correspond to the Naive + ICRL setting,
while
data/meta-llama/Meta-Llama-3.1-8B-Instruct/banking77/None/True/random_unbiased_only_positive/0.1/False/False/1.0/100/data.pickle
corresponds to the Stochastic ICRL setting?
I inspected both files and found that the second one matches Figure 2 in your paper. However, the first one only reaches about 43% when I plot it using the released data, whereas in your paper it appears to exceed 60%.
Could you clarify whether I am mapping the paths to the correct experimental settings?
Thanks!