initial vault

main/общее заметки/Topic modeling.md

@@ -0,0 +1,17 @@
+#ml #учеба
+
+Topic modeling - unsupervised, provides set of topics of docs. In some way modification of [[TF-IDF]] to deal with synonyms and polysemy.
+Important to use [[Stemming]] and [[Lemmatization]] to reduce number of unique words in dictionary of documents. 
+
+LSA (latent semantic analysis or latent semantic indexing) 
+Has to more matrixes Term x Term - number of documents, there terms co-ocur. Document x Document - number of terms each doc has in common 
+Document x term - number of terms in each doc 
+Use [[SVD (singular value decomposition)]] on term x doc Reduce dimensions and use cosine similarity of this resulting matrixes to measure resemblance of documents. 
+
+LDA (Latent Dirichlet allocation)
+probabilistic algorithm. Using doc x term matrixes (freq and co-ocurence) - generate table of probabilities for each keyword in each doc (topic distribution).
+While assigning topic to words it uses [[Gibbs sampling]]
+
+sources:
+https://maartengr.github.io/BERTopic/getting_started/ctfidf/ctfidf.html - bertopic lib
+https://huggingface.co/blog/bobxwu/fastopic