Ranking translations by likelihood

[Gingeh]

Firstly, this project is awesome! I'm so glad I stumbled upon it today :3

I wonder if it would be feasible/desirable to implement a translation scoring system similar to the one used by https://nasin.nimi.li/ (implementation here).

Basically it walks each possible parse tree, counts likely nodes, and discounts unlikely nodes, and then reports the score as a percentage using a softmax.

As an example: "mi pana e kili tawa sina" is more likely to mean "I provide fruit to you" than "I provide your moving fruit".

[neverRare]

Thank you so much for the nice words! I'm proud of it even though it's kinda broken haha!

nasin.nimi.li actually doesn't score by likelihood as far as I know. If you type in "mi tu li pona", it will score "mi (li) tu li pona" more than "(mi tu) li pona". It simply scores more "interesting" constructions more, similar to what ilo Token does. Although yes ilo Token doesn't have a scoring system similar to nasin.nimi.li. ilo Token simply uses a priority system. That is something that I can add.

Quantifying likelihood is a difficult task. There are so many exceptions to consider: "mi X li Y" tends to be considered "mi (li) X li Y" but there is an exception being "mi tu li pona". Not to mention scoring every definition that the dictionary has. Not impossible but it's a huge undertaking.

I'm thinking the best way is somehow analyzing the semantics through machine-learning system, which is outside of my ability to be honest.

I'm open for improving the ranking system but for now I'm more worried about the completeness.