Matt groupby 1210 - #16

Open

mhk197 wants to merge 40 commits into

dev_mixtralfrom

matt-groupby-1210

Commits on Dec 12, 2024

Merge pull request #17 from hugolatendresse/fixed_aggregate
mhk197
authored
aef9ac7
try one hidden layer and see if outputs
Ubuntu
committed
ea439b8
made slightly more sense than prev but changing back
Ubuntu
committed
391ccda
num experts 1
Ubuntu
committed
4449e0b
num experts 1
Ubuntu
committed
28f719d
Merge remote-tracking branch 'origin/dev_mixtral' into matt-groupby-1210
hugolatendresse
committed
84df2b1
try fix
Ubuntu
committed
1315dd7
Merge branch 'matt-groupby-1210' of https://github.com/hugolatendresse/FlexFlow into matt-groupby-1210
Ubuntu
committed
60d5237
try without num_local_experts = 1
Ubuntu
committed
0a09ff6
try without num_local_experts = 1
Ubuntu
committed
3dde8ed
fixed
Ubuntu
committed
e9acdf4
fixed
Ubuntu
committed
2bc3777
try new experts
Ubuntu
committed
3c3c1db
debug groupby
Ubuntu
committed
84a450b
debug groupby
Ubuntu
committed
31ad4e9
trying this
Ubuntu
committed
a4eb58b
trying this
Ubuntu
committed
cb6d1e9
trying this
Ubuntu
committed
77e1e5a
pushed
Ubuntu
committed
24ef878
fixed arg counts magic number in aggregate
Ubuntu
committed
e0a14f5
try 2 experts now
Ubuntu
committed
59e9d84
failed with 2 experts, 2 tokens per expert. trying 2 experts, 1 token per expert.
Ubuntu
committed
28a2f9b
worked with n = 2, tok = 1. trying n = 3, tok = 2.
Ubuntu
committed
a693ae0
worked with n = 2, tok = 1. trying n = 3, tok = 2.
Ubuntu
committed
8788a02
maybe fixed cuda kernel?
Ubuntu
committed
7d5dc9e

Commits on Dec 13, 2024

changed incr_decodig
Ubuntu
committed
c810f20
num experts = 1, tok = 1
Ubuntu
committed
b678b26
k =1, n =2
Ubuntu
committed
38aa0c6
k =1, n =1
Ubuntu
committed
4a3000d
k =1, n =2
Ubuntu
committed
6e585a1
n = 3, k = 1
Ubuntu
committed
4eca725
n = 4, k = 1
Ubuntu
committed
2e8e24a
n = 5, k = 1
Ubuntu
committed
2513484
full model again
Ubuntu
committed
98a7f90
try 1 again
Ubuntu
committed
03c67c9
try 1 again
Ubuntu
committed
7b48a58
try 1 again
Ubuntu
committed
97a73b7
try 1 again
Ubuntu
committed
50e2883
try 1 again
Ubuntu
committed
26e874f

Commits on Dec 15, 2024

conflicts
hugolatendresse
committed
e4f87f4