(Image source: https://arxiv.org/pdf/1804.07754.pdf)
3.3 Natural language inference (NLI):
is the task of determining whether a “hypothesis” is true (entailment), false (contradiction), or undetermined (neutral) given a “premise”.
(Image source: https://paperswithcode.com/task/natural-language-inference)
The intuition is that by doing such trainings, the model can capture the most informative features and discard noise from text. This output embedding in turn will be applied to wide variety of NLP tasks such as relatedness, clustering, paraphrase detection and text classification.
4.Results of UNIVERSAL SENTENCE ENCODER MODEL
To compare the performance between using transfer learning via pretrained word embedding and no using any transfer learning, Word Embedding Association Test (WEAT) was used. WEAT includes:
The results are summarized on the table as follows:
The results show that:
- Transfer learning from the Transformer based sentence encoder usually performs as good or better than transfer learning from the DAN encoder
- Models that make use of sentence-level transfer learning tend to perform better than models that only use word-level transfer.
- The best performance on most tasks is obtained by models that make use of both sentence and word level transfer
Mikolov, Tomas, et al. Efficient Estimation of Word Representations in Vector Space. Jan. 2013. arxiv.org, https://arxiv.org/abs/1301.3781v3.
Bojanowski, Piotr, et al. Enriching Word Vectors with Subword Information. July 2016. arxiv.org, https://arxiv.org/abs/1607.04606v2.
Vaswani, Ashish, et al. Attention Is All You Need. June 2017. arxiv.org, https://arxiv.org/abs/1706.03762v5.
Iyyer, Mohit, et al. “Deep Unordered Composition Rivals Syntactic Methods for Text Classification.” Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Association for Computational Linguistics, 2015, pp. 1681–91. ACLWeb, doi:10.3115/v1/P15-1162
Hiring Data Scientist / Engineer
We are looking for Data Scientist and Engineer.
Please check our Career Page.
AI / Data Science Project
Please check about experiences for Data Science Project
Vietnam AI / Data Science Lab
Please also visit Vietnam AI Lab