obsidian/общее заметки/Stemming.md

#ml #учеба data mining !

Main function to reduce word to it's original (root) form - stem (lemma).

Lovins stemmer

Rules only, remove suffixes, main problem is over removal of suffixes the, those, them - the, love - lov and so on.

Porter stemmer

Classify characters as vowels and consonant. Group sequential v and c and remove them by some rules (have rules, so still have some errors) Implementation in nltk.

Snowball stemmer

Updated porter with new rules, stop word and including other languages (including Russian) also have stop words list, such as  the, a, being, and the like. Also has implementation in nltk.

Source: https://www.ibm.com/think/topics/stemming?regionCode=us&languageCode=en&cm-history=us-en