Content 此篇论文主要完成了:1.通过数学推导，找到并证明了限制RNN-based LMs的性能瓶颈之一——Softmax Bottleneck问题2.针对这个瓶颈，提...
发简信
IP属地：新疆
-
Breaking the Softmax Bottleneck:A High-rank RNN Language Model -
读万卷书，行万里路。<br><br>记录那些读过的书籍。
-
深度学习的Nan问题
相信很多人都遇到过训练一个deep model的过程中，loss突然变成了NaN。在这里对这个问题做一个总结。 一般来说，出现NaN有以下几种情况： 如果在迭代的100轮以内...
-
2018 · NAACL · BP · Deep Contextualized Word Representations
2018 · NAACL · BP · Deep Contextualized Word Representations 问题：以往的词向量表示的语义信息并不丰富，没有挖掘到...
-
2018 · NIPS · Learned in Translation: Contextualized Word Vectors
2018 · NIPS · Learned in Translation: Contextualized Word Vectors 想法来源：从CV中借鉴的想法，深层的神经网...
-
2018 · ICLR · A NEW METHOD OF REGION EMBEDDING FOR TEXT CLASSIFICATION
2018 · ICLR · A NEW METHOD OF REGION EMBEDDING FOR TEXT CLASSIFICATION 想法来源：为了挖掘深层的语法信息...
-
2018 · ICLR · SKIP RNN: LEARNING TO SKIP STATE UPDATES IN RECURRENT NEURAL NETWORKS
2018 · ICLR · SKIP RNN: LEARNING TO SKIP STATE UPDATES IN RECURRENT NEURAL NETWORKS 想法来...
-
2018· ACL · Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features
2018· ACL · Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Fea...
-
2018 · ACL · Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in Predictin...
2018 · ACL · Some of Them Can be Guessed! Exploring the Effect of Linguistic Context in...
-
2018 · ACL · LSTMs Exploit Linguistic Attributes of Data
2018 · ACL · LSTMs Exploit Linguistic Attributes of Data 想法来源：实验性质文章。主要是测试LSTM的记忆能力。 价值...
-
2018 · ACL · Exploring Semantic Properties of Sentence Embeddings
2018 · ACL · Exploring Semantic Properties of Sentence Embeddings 想法来源：探索性实验性质文章。 价值：可以...
-
2017 · ICLR · EFFICIENT VECTOR REPRESENTATION FOR DOCUMENTS THROUGH CORRUPTION
2017 · ICLR · EFFICIENT VECTOR REPRESENTATION FOR DOCUMENTS THROUGH CORRUPTION 想法来源：文档表...
-
2018 · ACL · What you can cram into a single $&!#* vector-Probing sentence embeddings for linguis...
2018 · ACL · What you can cram into a single $&!#* vector-Probing sentence embeddings f...
-
2017 · EMNLP · Supervised Learning of Universal Sentence Representations from Natural Language In...
2017 · EMNLP · Supervised Learning of Universal Sentence Representations from Natural L...
-
2017 · ICLR · A STRUCTURED SELF-ATTENTIVE SENTENCE EMBEDDING
2017 · ICLR · A STRUCTURED SELF-ATTENTIVE SENTENCE EMBEDDING · 想法来源：监督任务+self-attention...
-
2016 · ICLR · IMPROVING PERFORMANCE OF RECURRENT NEURAL NETWORK WITH RELU NONLINEARITY
2016 · ICLR · IMPROVING PERFORMANCE OF RECURRENT NEURAL NETWORK WITH RELU NONLINEARITY ...
-
2018 · ICLR · AN EFFICIENT FRAMEWORK FOR LEARNING SENTENCE REPRESENTATIONS
2018 · ICLR · AN EFFICIENT FRAMEWORK FOR LEARNING SENTENCE REPRESENTATIONS · 想法来源：文章从分布...
-
2017 · Arxiv · Discourse-Based Objectives for Fast Unsupervised Sentence Representation Learning ·
2017 · Arxiv · Discourse-Based Objectives for Fast Unsupervised Sentence Representation...
-
2017 · TACL · Enriching Word Vectors with Subword Information
想法来源：很多的词向量表达都是将词作为单一体来看待的，没有考虑参数的共享。从感知角度来说，就是失去了原有的内部结构。 价值：a. 学习到词与词形态学方面的信息，词与词交集越多...
-
EMNLP · 2017 · Mimicking Word Embeddings using Subword RNNs
EMNLP · 2017 · Mimicking Word Embeddings using Subword RNNs · 想法来源：subword 价值：利用了subwor...
