自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 SRL(Semantic Role Labeling)

自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 SRL(Semantic Role Labeling)

# Gavin大咖金句

Gavin：理论上将Transformer能够更好的处理一切以“set of units”存在的数据，而计算机视觉、语音、自然语言处理等属于这种类型的数据，所以理论上讲Transformer会在接下来数十年对这些领域形成主导性的统治力。


Gavin：A feedforward network with a single layer is sufficient to represent any function, but the layer may be infeasibly large and may fail to learn and generalize correctly. — Ian Goodfellow, DLB


Gavin：Transformer是人工智能领域的新一代的引擎，本质是研究结构关系、工业界实践的核心是基于Transformer实现万物皆流。

Gavin：Non-linearity是Transformer的魔法


深入理解Transformer自然语言处理 SRL(Semantic Role Labeling)

以下的示例包含四个动词“drink" ：

"John wanted to drink tea, Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice."

在SRL.ipynb运行第3个例子

!echo '{"sentence": "John wanted to drink tea, Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice."}' | \ allennlp predict https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz -

运行结果如下：

2020-12-20 09:08:27,582 - INFO - transformers.file_utils - PyTorch version 1.5.1 available. 2020-12-20 09:08:27.767124: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.10.1 2020-12-20 09:08:29,592 - INFO - transformers.file_utils - TensorFlow version 2.4.0 available. 2020-12-20 09:08:30,797 - INFO - allennlp.common.file_utils - checking cache for https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz at /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 2020-12-20 09:08:30,797 - INFO - allennlp.common.file_utils - waiting to acquire lock on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 2020-12-20 09:08:30,799 - INFO - filelock - Lock 140584440236464 acquired on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724.lock 2020-12-20 09:08:30,799 - INFO - allennlp.common.file_utils - cache of https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz is up-to-date 2020-12-20 09:08:30,799 - INFO - filelock - Lock 140584440236464 released on /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724.lock 2020-12-20 09:08:30,799 - INFO - allennlp.models.archival - loading archive file https://storage.googleapis.com/allennlp-public-models/bert-base-srl-2020.03.24.tar.gz from cache at /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 2020-12-20 09:08:30,799 - INFO - allennlp.models.archival - extracting archive file /root/.allennlp/cache/e20d5b792a8d456a1a61da245d1856d4b7778efe69ac3c30759af61940aa0f42.f72523a9682cb1f5ad3ecf834075fe53a1c25a6bcbf4b40c11e13b7f426a4724 to temp dir /tmp/tmpse7z902p 2020-12-20 09:08:35,061 - INFO - allennlp.common.params - type = from_instances 2020-12-20 09:08:35,061 - INFO - allennlp.data.vocabulary - Loading token dictionary from /tmp/tmpse7z902p/vocabulary. 2020-12-20 09:08:35,062 - INFO - filelock - Lock 140584442130328 acquired on /tmp/tmpse7z902p/vocabulary/.lock 2020-12-20 09:08:35,089 - INFO - filelock - Lock 140584442130328 released on /tmp/tmpse7z902p/vocabulary/.lock 2020-12-20 09:08:35,089 - INFO - allennlp.common.params - model.type = srl_bert 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.regularizer = None 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.bert_model = bert-base-uncased 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.embedding_dropout = 0.1 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.initializer = <allennlp.nn.initializers.InitializerApplicator object at 0x7fdc5d9b87b8> 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.label_smoothing = None 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.ignore_span_metric = False 2020-12-20 09:08:35,090 - INFO - allennlp.common.params - model.srl_eval_path = /usr/local/lib/python3.6/dist-packages/allennlp_models/structured_prediction/tools/srl-eval.pl 2020-12-20 09:08:35,400 - INFO - transformers.configuration_utils - loading configuration file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-config.json from cache at /root/.cache/torch/transformers/4dad0251492946e18ac39290fcfe91b89d370fee250efe9521476438fe8ca185.7156163d5fdc189c3016baca0775ffce230789d7fa2a42ef516483e4ca884517 2020-12-20 09:08:35,400 - INFO - transformers.configuration_utils - Model config BertConfig { "architectures": [ "BertForMaskedLM" ], "attention_probs_dropout_prob": 0.1, "hidden_act": "gelu", "hidden_dropout_prob": 0.1, "hidden_size": 768, "initializer_range": 0.02, "intermediate_size": 3072, "layer_norm_eps": 1e-12, "max_position_embeddings": 512, "model_type": "bert", "num_attention_heads": 12, "num_hidden_layers": 12, "pad_token_id": 0, "type_vocab_size": 2, "vocab_size": 30522 } 2020-12-20 09:08:35,598 - INFO - transformers.modeling_utils - loading weights file https://cdn.huggingface.co/bert-base-uncased-pytorch_model.bin from cache at /root/.cache/torch/transformers/f2ee78bdd635b758cc0a12352586868bef80e47401abe4c4fcc3832421e7338b.36ca03ab34a1a5d5fa7bc3d03d55c4fa650fed07220e2eeebc06ce58d0e9a157 2020-12-20 09:08:38,288 - INFO - allennlp.nn.initializers - Initializing parameters 2020-12-20 09:08:38,289 - INFO - allennlp.nn.initializers - Done initializing parameters; the following parameters are using their default initialization from their code 2020-12-20 09:08:38,289 - INFO - allennlp.nn.initializers - bert_model.embeddings.LayerNorm.bias 2020-12-20 09:08:38,289 - INFO - allennlp.nn.initializers - bert_model.embeddings.LayerNorm.weight 2020-12-20 09:08:38,289 - INFO - allennlp.nn.initializers - bert_model.embeddings.position_embeddings.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.embeddings.token_type_embeddings.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.embeddings.word_embeddings.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.output.LayerNorm.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.output.LayerNorm.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.output.dense.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.output.dense.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.key.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.key.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.query.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.query.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.value.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.attention.self.value.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.intermediate.dense.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.intermediate.dense.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.output.LayerNorm.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.output.LayerNorm.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.output.dense.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.0.output.dense.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.output.LayerNorm.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.output.LayerNorm.weight 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.output.dense.bias 2020-12-20 09:08:38,290 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.output.dense.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.key.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.key.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.query.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.query.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.value.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.attention.self.value.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.intermediate.dense.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.intermediate.dense.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.output.LayerNorm.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.output.LayerNorm.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.output.dense.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.1.output.dense.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.output.LayerNorm.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.output.LayerNorm.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.output.dense.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.output.dense.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.key.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.key.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.query.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.query.weight 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.value.bias 2020-12-20 09:08:38,291 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.attention.self.value.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.intermediate.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.intermediate.dense.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.output.LayerNorm.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.output.LayerNorm.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.output.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.10.output.dense.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.output.LayerNorm.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.output.LayerNorm.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.output.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.output.dense.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.key.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.key.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.query.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.query.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.value.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.attention.self.value.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.intermediate.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.intermediate.dense.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.output.LayerNorm.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.output.LayerNorm.weight 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.output.dense.bias 2020-12-20 09:08:38,292 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.11.output.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.output.LayerNorm.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.output.LayerNorm.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.output.dense.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.output.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.key.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.key.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.query.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.query.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.value.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.attention.self.value.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.intermediate.dense.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.intermediate.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.output.LayerNorm.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.output.LayerNorm.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.output.dense.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.2.output.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.output.LayerNorm.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.output.LayerNorm.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.output.dense.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.output.dense.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.key.bias 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.key.weight 2020-12-20 09:08:38,293 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.query.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.query.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.value.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.attention.self.value.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.intermediate.dense.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.intermediate.dense.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.output.LayerNorm.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.output.LayerNorm.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.output.dense.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.3.output.dense.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.output.LayerNorm.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.output.LayerNorm.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.output.dense.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.output.dense.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.key.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.key.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.query.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.query.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.value.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.attention.self.value.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.intermediate.dense.bias 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.intermediate.dense.weight 2020-12-20 09:08:38,294 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.output.LayerNorm.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.output.LayerNorm.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.output.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.4.output.dense.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.output.LayerNorm.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.output.LayerNorm.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.output.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.output.dense.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.key.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.key.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.query.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.query.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.value.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.attention.self.value.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.intermediate.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.intermediate.dense.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.output.LayerNorm.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.output.LayerNorm.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.output.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.5.output.dense.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.output.LayerNorm.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.output.LayerNorm.weight 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.output.dense.bias 2020-12-20 09:08:38,295 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.output.dense.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.key.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.key.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.query.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.query.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.value.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.attention.self.value.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.intermediate.dense.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.intermediate.dense.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.output.LayerNorm.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.output.LayerNorm.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.output.dense.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.6.output.dense.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.output.LayerNorm.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.output.LayerNorm.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.output.dense.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.output.dense.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.key.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.key.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.query.bias 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.query.weight 2020-12-20 09:08:38,296 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.value.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.attention.self.value.weight 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.intermediate.dense.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.intermediate.dense.weight 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.output.LayerNorm.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.output.LayerNorm.weight 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.output.dense.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.7.output.dense.weight 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.output.LayerNorm.bias 2020-12-20 09:08:38,374 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.output.LayerNorm.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.output.dense.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.output.dense.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.key.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.key.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.query.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.query.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.value.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.attention.self.value.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.intermediate.dense.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.intermediate.dense.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.output.LayerNorm.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.output.LayerNorm.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.output.dense.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.8.output.dense.weight 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.output.LayerNorm.bias 2020-12-20 09:08:38,375 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.output.LayerNorm.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.output.dense.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.output.dense.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.key.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.key.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.query.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.query.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.value.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.attention.self.value.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.intermediate.dense.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.intermediate.dense.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.output.LayerNorm.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.output.LayerNorm.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.output.dense.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.encoder.layer.9.output.dense.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.pooler.dense.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - bert_model.pooler.dense.weight 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - tag_projection_layer.bias 2020-12-20 09:08:38,376 - INFO - allennlp.nn.initializers - tag_projection_layer.weight 2020-12-20 09:08:38,830 - INFO - allennlp.common.params - dataset_reader.type = srl 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.lazy = False 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.cache_directory = None 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.max_instances = None 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.manual_distributed_sharding = False 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.manual_multi_process_sharding = False 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.token_indexers = None 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.domain_identifier = None 2020-12-20 09:08:38,831 - INFO - allennlp.common.params - dataset_reader.bert_model_name = bert-base-uncased 2020-12-20 09:08:39,125 - INFO - transformers.tokenization_utils - loading file https://s3.amazonaws.com/models.huggingface.co/bert/bert-base-uncased-vocab.txt from cache at /root/.cache/torch/transformers/26bc1ad6c0ac742e9b52263248f6d0f00068293b33709fae12320c0e35ccfbbb.542ce4285a40d23a559526243235df47c5f75c197f04f37d1a0c124c32c9a084 input 0: {"sentence": "John wanted to drink tea, Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice."} prediction: {"verbs": [{"verb": "wanted", "description": "[ARG0: John] [V: wanted] [ARG1: to drink tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "drink", "description": "[ARG0: John] wanted to [V: drink] [ARG1: tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["B-ARG0", "O", "O", "B-V", "B-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "likes", "description": "John wanted to drink tea , [ARG0: Mary] [V: likes] [ARG1: to drink coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "drink", "description": "John wanted to drink tea , [ARG0: Mary] likes to [V: drink] [ARG1: coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "B-ARG0", "O", "O", "B-V", "B-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "drank", "description": "John wanted to drink tea , Mary likes to drink coffee but [ARG0: Karim] [V: drank] [ARG1: some cool water and Faiza] would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O"]}, {"verb": "would", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank some cool water and Faiza [V: would] [ARGM-DIS: like] to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-V", "B-ARGM-DIS", "O", "O", "O", "O", "O"]}, {"verb": "like", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] [ARGM-MOD: would] [V: like] [ARG1: to drink tomato juice] .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "B-ARGM-MOD", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O"]}, {"verb": "drink", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] would like to [V: drink] [ARG1: tomato juice] .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "O", "O", "O", "B-V", "B-ARG1", "I-ARG1", "O"]}], "words": ["John", "wanted", "to", "drink", "tea", ",", "Mary", "likes", "to", "drink", "coffee", "but", "Karim", "drank", "some", "cool", "water", "and", "Faiza", "would", "like", "to", "drink", "tomato", "juice", "."]} 2020-12-20 09:08:40,852 - INFO - allennlp.models.archival - removing temporary unarchived model dir at /tmp/tmpse7z902p

Transformer模型找到了自己的出路，如以下包含动词的原始输出摘录所示：

input 0: { "sentence": "John wanted to drink tea, Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice." } prediction: { "verbs": [{ "verb": "wanted", "description": "[ARG0: John] [V: wanted] [ARG1: to drink tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "drink", "description": "[ARG0: John] wanted to [V: drink] [ARG1: tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["B-ARG0", "O", "O", "B-V", "B-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "likes", "description": "John wanted to drink tea , [ARG0: Mary] [V: likes] [ARG1: to drink coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "drink", "description": "John wanted to drink tea , [ARG0: Mary] likes to [V: drink] [ARG1: coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "B-ARG0", "O", "O", "B-V", "B-ARG1", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "drank", "description": "John wanted to drink tea , Mary likes to drink coffee but [ARG0: Karim] [V: drank] [ARG1: some cool water and Faiza] would like to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O", "O", "O", "O", "O", "O", "O"] }, { "verb": "would", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank some cool water and Faiza [V: would] [ARGM-DIS: like] to drink tomato juice .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-V", "B-ARGM-DIS", "O", "O", "O", "O", "O"] }, { "verb": "like", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] [ARGM-MOD: would] [V: like] [ARG1: to drink tomato juice] .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "B-ARGM-MOD", "B-V", "B-ARG1", "I-ARG1", "I-ARG1", "I-ARG1", "O"] }, { "verb": "drink", "description": "John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] would like to [V: drink] [ARG1: tomato juice] .", "tags": ["O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "O", "B-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "I-ARG0", "O", "O", "O", "B-V", "B-ARG1", "I-ARG1", "O"] }], "words": ["John", "wanted", "to", "drink", "tea", ",", "Mary", "likes", "to", "drink", "coffee", "but", "Karim", "drank", "some", "cool", "water", "and", "Faiza", "would", "like", "to", "drink", "tomato", "juice", "."] }

当我们在AllenNLP在线界面上运行这个句子时，我们获得了几个可视化页面，将检查其中两个，第一个是完美的，它识别出动词“wanted"，并做出正确的关联：

然而，当它识别出动词“drank"时，它将“and Faiza”作为参数

这句话的意思是“Karim drank some cool water.”作为“drank”的参数，“and Faiza”的存在是有争议的。

该问题影响到“Faiza would like to drink tomato juice”：


“some cool water and”不是like的参数，只有“Faiza”是“like”的参数，

使用AllenNLP获得的文本输出证实了问题

wanted: [ARG0: John] [V: wanted] [ARG1: to drink tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .

drink: [ARG0: John] wanted to [V: drink] [ARG1: tea] , Mary likes to drink coffee but Karim drank some cool water and Faiza would like to drink tomato juice .

likes: John wanted to drink tea , [ARG0: Mary] [V: likes] [ARG1: to drink coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .

drink: John wanted to drink tea , [ARG0: Mary] likes to [V: drink] [ARG1: coffee] but Karim drank some cool water and Faiza would like to drink tomato juice .

drank: John wanted to drink tea , Mary likes to drink coffee but [ARG0:Karim] [V: drank] [ARG1: some cool water and Faiza] would like to drink tomato juice .

would: John wanted to drink tea , Mary likes to drink coffee but Karim drank some cool water and Faiza [V: would] [ARGM-DIS: like] to drink tomato juice .


like: John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] [ARGM-MOD: would] [V: like] [ARG1: to drink tomato juice] .

drink: John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] would like to [V: drink] [ARG1: tomato juice] .

输出有点模糊不清，例如，我们可以看到动词“like”的一个论点是“Karim drank some cool water and Faiza”，这令人困惑：

like: John wanted to drink tea , Mary likes to drink coffee but Karim drank [ARG0: some cool water and Faiza] [ARGM-MOD: would] [V: like] [ARG1: to drink tomato juice] .

我们发现基于BERT的transformer 在基本样本上产生了相对较好的结果，接下来让我们试试更难的。


# 星空智能对话机器人系列博客

- [NLP星空智能对话机器人系列：第二次星空智能对话机器人Zoom线上演示安排](https://blog.csdn.net/duan_zhihua/article/details/119973461?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：StarSpace: Embed All The Things](https://blog.csdn.net/duan_zhihua/article/details/120069995?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：Facebook StarSpace框架初体验](https://blog.csdn.net/duan_zhihua/article/details/120117492?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：Facebook StarSpace框架案例数据加载](https://blog.csdn.net/duan_zhihua/article/details/120121272?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 多头注意力架构-通过Python实例计算Q, K, V](https://blog.csdn.net/duan_zhihua/article/details/120230267?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 多头注意力架构 Q K V注意力评分](https://blog.csdn.net/duan_zhihua/article/details/120232738?spm=1001.2014.3001.5502)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 多头注意力架构 Concatenation of the output of the heads](https://blog.csdn.net/duan_zhihua/article/details/120234517?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 位置编码（positional_encoding）](https://blog.csdn.net/duan_zhihua/article/details/120256529?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 位置编码案例应用](https://blog.csdn.net/duan_zhihua/article/details/120262856?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 基于BERT模型微调实现句子分类](https://blog.csdn.net/duan_zhihua/article/details/120316808?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：第4、5、6、7次星空智能对话机器人Zoom线上演示安排](https://blog.csdn.net/duan_zhihua/article/details/120382427?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT](https://blog.csdn.net/duan_zhihua/article/details/120373424?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT ByteLevelBPETokenizer](https://blog.csdn.net/duan_zhihua/article/details/120384508?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT Initializing model](https://blog.csdn.net/duan_zhihua/article/details/120392929?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT Exploring the parameters](https://blog.csdn.net/duan_zhihua/article/details/120399404?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT Initializing the trainer](https://duanzhihua.blog.csdn.net/article/details/120405419)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 KantaiBERT Language modeling with FillMaskPipeline](https://blog.csdn.net/duan_zhihua/article/details/120405815?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 GLUE CoLA](https://blog.csdn.net/duan_zhihua/article/details/120422365?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 GLUE SST-2](https://blog.csdn.net/duan_zhihua/article/details/120441864?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 GLUE MRPC](https://blog.csdn.net/duan_zhihua/article/details/120443327?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 GLUE Winograd schemas and NER](https://blog.csdn.net/duan_zhihua/article/details/120473728?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Workshop on Machine Translation (WMT)](https://duanzhihua.blog.csdn.net/article/details/120495610)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 WMT数据处理](https://blog.csdn.net/duan_zhihua/article/details/120516215?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 BLEU smoothing](https://blog.csdn.net/duan_zhihua/article/details/120565514?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Translations with Trax](https://blog.csdn.net/duan_zhihua/article/details/120572401?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：第五次星空智能对话机器人线上演示](https://blog.csdn.net/duan_zhihua/article/details/120583102?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 BertViz](https://blog.csdn.net/duan_zhihua/article/details/120607908?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 The Reformer](https://blog.csdn.net/duan_zhihua/article/details/120610502?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：重磅信息：10月份一共会安排6次星空对话的机器人线上演示交流](https://blog.csdn.net/duan_zhihua/article/details/120615817?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Pattern-Exploiting Training (PET)](https://blog.csdn.net/duan_zhihua/article/details/120615259?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 The philosophy of Pattern-Exploiting Training (PET)](https://blog.csdn.net/duan_zhihua/article/details/120618581?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 It‘s time to make a decision](https://blog.csdn.net/duan_zhihua/article/details/120625482?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Text completion with GPT-2](https://blog.csdn.net/duan_zhihua/article/details/120629841?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Text completion with GPT-2 step3-5](https://blog.csdn.net/duan_zhihua/article/details/120632856?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Text completion with GPT-2 step 6-8](https://blog.csdn.net/duan_zhihua/article/details/120640746?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Text completion with GPT-2 step 9](https://blog.csdn.net/duan_zhihua/article/details/120658569?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Training a GPT-2 language model](https://blog.csdn.net/duan_zhihua/article/details/120678518?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：论文学习 Do Transformers Really Perform Bad for Graph Representation](https://blog.csdn.net/duan_zhihua/article/details/120688601?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Training a GPT-2 language model Steps 2 to 6](https://blog.csdn.net/duan_zhihua/article/details/120700685?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Training a GPT-2 language model Steps 7 to 9](https://blog.csdn.net/duan_zhihua/article/details/120710206?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Training a GPT-2 language model Steps 10](https://blog.csdn.net/duan_zhihua/article/details/120720980?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 T5-large transformer model](https://blog.csdn.net/duan_zhihua/article/details/120751601?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Architecture of the T5 model](https://blog.csdn.net/duan_zhihua/article/details/120761331?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Summarizing documents with T5-large](https://blog.csdn.net/duan_zhihua/article/details/120770732?spm=1001.2014.3001.5502)

- [NLP星空智能对话机器人系列：业界动态信息分享 自然语言处理中的小样本学习](https://blog.csdn.net/duan_zhihua/article/details/120772216?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：业界动态信息分享 Transformer 的性能瓶颈分析与优化](https://blog.csdn.net/duan_zhihua/article/details/120790068?spm=1001.2014.3001.5501)

- [自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Matching datasets and tokenizers](https://blog.csdn.net/duan_zhihua/article/details/120793814?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Noisy relationships](https://blog.csdn.net/duan_zhihua/article/details/120816733?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Standard NLP tasks with specific vocabulary](https://blog.csdn.net/duan_zhihua/article/details/120835204?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 T5 Bill of Rights Sample](https://blog.csdn.net/duan_zhihua/article/details/120903287?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：论文解读 How Good is Your Tokenizer? (你的词元分析器有多好？多语言模型的单语性能研究)](https://blog.csdn.net/duan_zhihua/article/details/120949735?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列： Faiss Tutorial](https://blog.csdn.net/duan_zhihua/article/details/121053075?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 Semantic Role Labeling (SRL).](https://blog.csdn.net/duan_zhihua/article/details/121006165?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：Gavin大咖 Transformer 101问答集 第一问](https://blog.csdn.net/duan_zhihua/article/details/121059773?spm=1001.2014.3001.5501)

- [NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 SRL  Sample 1](https://blog.csdn.net/duan_zhihua/article/details/121097719?spm=1001.2014.3001.5501)

- [自然语言处理NLP星空智能对话机器人系列：深入理解Transformer自然语言处理 SRL(Semantic Role Labeling)  Sample 2](https://zhuanlan.zhihu.com/p/428883118)

©著作权归作者所有,转载或内容合作请联系作者
  • 人面猴
    序言：七十年代末，一起剥皮案震惊了整个滨河市，随后出现的几起案子，更是在滨河造成了极大的恐慌，老刑警刘岩，带你破解...
    沈念sama阅读 203,324评论 5赞 476
  • 死咒
    序言：滨河连续发生了三起死亡事件，死亡现场离奇诡异，居然都是意外死亡，警方通过查阅死者的电脑和手机，发现死者居然都...
    沈念sama阅读 85,303评论 2赞 381
  • 救了他两次的神仙让他今天三更去死
    文/潘晓璐 我一进店门，熙熙楼的掌柜王于贵愁眉苦脸地迎上来，“玉大人，你说我怎么就摊上这事。” “怎么了？”我有些...
    开封第一讲书人阅读 150,192评论 0赞 337
  • 道士缉凶录：失踪的卖姜人
    文/不坏的土叔 我叫张陵，是天一观的道长。 经常有香客问我，道长，这世上最难降的妖魔是什么？ 我笑而不...
    开封第一讲书人阅读 54,555评论 1赞 273
  • 港岛之恋（遗憾婚礼）
    正文 为了忘掉前任，我火速办了婚礼，结果婚礼上，老公的妹妹穿的比我还像新娘。我一直安慰自己，他们只是感情好，可当我...
    茶点故事阅读 63,569评论 5赞 365
  • 恶毒庶女顶嫁案：这布局不是一般人想出来的
    文/花漫 我一把揭开白布。 她就那样静静地躺着，像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上，一...
    开封第一讲书人阅读 48,566评论 1赞 281
  • 城市分裂传说
    那天，我揣着相机与录音，去河边找鬼。 笑死，一个胖子当着我的面吹牛，可吹牛的内容都是我干的。 我是一名探鬼主播，决...
    沈念sama阅读 37,927评论 3赞 395
  • 双鸳鸯连环套：你想象不到人心有多黑
    文/苍兰香墨 我猛地睁开眼，长吁一口气：“原来是场噩梦啊……” “哼！你这毒妇竟也来了？” 一声冷哼从身侧响起，我...
    开封第一讲书人阅读 36,583评论 0赞 257
  • 万荣杀人案实录
    序言：老挝万荣一对情侣失踪，失踪者是张志新（化名）和其女友刘颖，没想到半个月后，有当地人在树林里发现了一具尸体，经...
    沈念sama阅读 40,827评论 1赞 297
  • 护林员之死
    正文 独居荒郊野岭守林人离奇死亡，尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 35,590评论 2赞 320
  • 白月光启示录
    正文 我和宋清朗相恋三年，在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 37,669评论 1赞 329
  • 活死人
    序言：一个原本活蹦乱跳的男人离奇死亡，死状恐怖，灵堂内的尸体忽然破棺而出，到底是诈尸还是另有隐情，我是刑警宁泽，带...
    沈念sama阅读 33,365评论 4赞 318
  • 日本核电站爆炸内幕
    正文 年R本政府宣布，位于F岛的核电站，受9级特大地震影响，放射性物质发生泄漏。R本人自食恶果不足惜，却给世界环境...
    茶点故事阅读 38,941评论 3赞 307
  • 男人毒药：我在死后第九天来索命
    文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹，春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 29,928评论 0赞 19
  • 一桩弑父案，背后竟有这般阴谋
    文/苍兰香墨 我抬头看了看天上的太阳。三九已至，却和暖如春，着一层夹袄步出监牢的瞬间，已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 31,159评论 1赞 259
  • 情欲美人皮
    我被黑心中介骗来泰国打工， 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留，地道东北人。 一个月前我还...
    沈念sama阅读 42,880评论 2赞 349
  • 代替公主和亲
    正文 我出身青楼，却偏偏与公主长得像，于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子，可洞房花烛夜当晚...
    茶点故事阅读 42,399评论 2赞 342

推荐阅读更多精彩内容

  • 2019-11-28 173
    16宿命：用概率思维提高你的胜算 以前的我是风险厌恶者，不喜欢去冒险，但是人生放弃了冒险，也就放弃了无数的可能。 ...
    yichen大刀阅读 6,030评论 0赞 4
  • 2019-11-28三数灯谜分析
    公元：2019年11月28日19时42分农历：二零一九年 十一月 初三日 戌时干支：己亥乙亥己巳甲戌当月节气：立冬...
    石放阅读 6,870评论 0赞 2
  • 断舍离第四天2019.11.28
    今天上午陪老妈看病，下午健身房跑步，晚上想想今天还没有断舍离，马上做，衣架和旁边的的布衣架，一看乱乱，又想想自己是...
    影子3623253阅读 2,903评论 1赞 8