Skip to content

Commit

Permalink
Let library users bring their own stem function
Browse files Browse the repository at this point in the history
  • Loading branch information
duijf committed Aug 12, 2016
1 parent a95ea98 commit d279acd
Show file tree
Hide file tree
Showing 5 changed files with 9 additions and 12 deletions.
11 changes: 4 additions & 7 deletions lib/simple_bayes/trainer/token_stemmer.ex
Original file line number Diff line number Diff line change
@@ -1,18 +1,15 @@
defmodule SimpleBayes.Trainer.TokenStemmer do
@doc """
Stems the word, or passes the word through.
Stems the word using `stemmer`. Passes the word through if it is false.
## Examples
iex> SimpleBayes.Trainer.TokenStemmer.stem("buying", nil)
"buying"
iex> SimpleBayes.Trainer.TokenStemmer.stem("buying", false)
"buying"
iex> SimpleBayes.Trainer.TokenStemmer.stem("buying", true)
iex> SimpleBayes.Trainer.TokenStemmer.stem("buying", &Stemmer.stem/1)
"buy"
"""
def stem(word, true), do: Stemmer.stem(word)
def stem(word, _), do: word
def stem(word, stemmer) when is_function(stemmer), do: stemmer.(word)
def stem(word, _), do: word
end
4 changes: 2 additions & 2 deletions mix.exs
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,16 @@ defmodule SimpleBayes.Mixfile do
end

def application do
[applications: [:logger, :math, :decimal, :stemmer]]
[applications: [:logger, :math, :decimal]]
end

defp deps do
[
{:ex_doc, ">= 0.0.0", only: :dev},
{:faker, ">= 0.0.0", only: :test},
{:stemmer, "~> 1.0.0", only: :test},
{:math, ">= 0.0.0"},
{:decimal, ">= 0.0.0"},
{:stemmer, "~> 1.0.0"}
]
end

Expand Down
2 changes: 1 addition & 1 deletion test/simple_bayes/model/bernoulli_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ defmodule SimpleBayes.BernoulliTest do
end

test "stemming" do
result = SimpleBayes.init(model: :bernoulli, stem: true)
result = SimpleBayes.init(model: :bernoulli, stem: &Stemmer.stem/1)
|> SimpleBayes.train(:apple, "buying apple")
|> SimpleBayes.train(:banana, "buy banana")
|> SimpleBayes.classify("buy apple")
Expand Down
2 changes: 1 addition & 1 deletion test/simple_bayes/model/binarized_multinomial_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ defmodule SimpleBayes.BinarizedMultinomialTest do
end

test "stemming" do
result = SimpleBayes.init(model: :binarized_multinomial, stem: true)
result = SimpleBayes.init(model: :binarized_multinomial, stem: &Stemmer.stem/1)
|> SimpleBayes.train(:apple, "buying apple")
|> SimpleBayes.train(:banana, "buy banana")
|> SimpleBayes.classify("buy apple")
Expand Down
2 changes: 1 addition & 1 deletion test/simple_bayes/model/multinomial_test.exs
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ defmodule SimpleBayes.MultinomialTest do
end

test "stemming" do
result = SimpleBayes.init(stem: true)
result = SimpleBayes.init(stem: &Stemmer.stem/1)
|> SimpleBayes.train(:apple, "buying apple")
|> SimpleBayes.train(:banana, "buy banana")
|> SimpleBayes.classify("buy apple")
Expand Down

0 comments on commit d279acd

Please sign in to comment.