Sentence embedding

Sentence embedding is the collective name for a set of techniques in natural language processing (NLP) where sentences are mapped to vectors of real numbers [1] [2] [3] [4] [5] [6] [7] [8] .

Part of a series on
Machine learning
and
data mining

Application

Sentence embedding is used by the deep learning software libraries PyTorch[9] and TensorFlow[10]

Evaluation

A way of testing sentence encodings is to apply them on Sentences Involving Compositional Knowledge (SICK) corpus[11] for both entailment (SICK-E) and relatedness (SICK-R).

In [12] the best results are obtained using a BiLSTM network trained on the Stanford Natural Language Inference (SNLI) Corpus. The Pearson correlation coefficient for SICK-R is 0.885 and the result for SICK-E is 86.3. A slight improvement over previous scores is presented in [13]: SICK-R: 0.888 and SICK-E: 87.8 using a concatenation of bidirectional Gated recurrent unit.

gollark: Are you sure they can't be done manually?
gollark: Oh, neæt.
gollark: Ideatic idea: git.osmarks.tk™.
gollark: ↓ Palaiologos in IRL real life
gollark: `gcd(x,y²)=1`

See also

InferSent sentence embeddings and training code

Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning

References
  1. Paper Summary: Evaluation of sentence embeddings in downstream and linguistic probing tasks
  2. Oren Barkan, Noam Razin, Itzik Malkiel, Ori Katz, Avi Caciularu, Noam Koenigstein. "Scalable Attentive Sentence-Pair Modeling via Distilled Sentence Embedding". AAAI 2020; arxiv:1908.05161.
  3. The Current Best of Universal Word Embeddings and Sentence Embeddings
  4. Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope: “Universal Sentence Encoder”, 2018; arXiv:1803.11175.
  5. Ledell Wu, Adam Fisch, Sumit Chopra, Keith Adams, Antoine Bordes: “StarSpace: Embed All The Things!”, 2017; arXiv:1709.03856.
  6. Sanjeev Arora, Yingyu Liang, and Tengyu Ma. "A simple but tough-to-beat baseline for sentence embeddings.", 2016; openreview:SyK00v5xx.
  7. Mircea Trifan, Bogdan Ionescu, Cristian Gadea, and Dan Ionescu. "A graph digital signal processing method for semantic analysis." In Applied Computational Intelligence and Informatics (SACI), 2015 IEEE 10th Jubilee International Symposium on, pp. 187-192. IEEE, 2015; ieee:7208196.
  8. Pierpaolo Basile, Annalina Caputo, and Giovanni Semeraro. "A study on compositional semantics of words in distributional spaces." In Semantic Computing (ICSC), 2012 IEEE Sixth International Conference on, pp. 154-161. IEEE, 2012; ieee:6337099 .
  9. Microsoft. "distilled-sentence-embedding".
  10. Google. "universal-sentence-encoder". TensorFlow Hub. Retrieved 6 October 2018.
  11. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi, and Roberto Zamparelli. "A SICK cure for the evaluation of compositional distributional semantic models." In LREC, pp. 216-223. 2014 .
  12. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault: “Supervised Learning of Universal Sentence Representations from Natural Language Inference Data”, 2017; arXiv:1705.02364.
  13. Sandeep Subramanian, Adam Trischler, Yoshua Bengio: “Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning”, 2018; arXiv:1804.00079.
This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.