Keras 、Tensorflow建立lstm模型资料

官方文档

I personally love coding in Python, but with Theano, you have to write low-level neural networks. So combine Keras with Theano and it is very easy. However I am currently using Torch now (very similar to Keras) as installations are the simplest and I don’t use any of CNN or LSTM.

This examples gives a very good understanding of the implementation of Dynamic RNN in tensorflow. These code can be extended to create neural stack machine, neural turing machine, RNN-EMM in tensorflow.

这两个之间推荐使用TensorFlow,因为都是基于Python的符号运算库,TensorFlow显然支持更好,Google也比高校有更多的人力投入。Theano的主要开发者现在都在Google,可以想见将来的工程资源上也会更偏向于TF一些。
Theano的一个优势在于代码是在计算时生成并编译的,所以理论上可以达到更高的速度(不需要运行时的polymorphism,而且如果写得好的话可以fuse kernel),但是因为是学术实现,没有花大精力在优化上面,所以实际速度并不占优势。另外现在大家都高度依赖于第三方库比如说cudnn,所以比较速度已经是上个时代的事情了,不必太在意。

另外吐槽一下,TensorFlow的分布式计算不是最快的,单机使用CPU作reduction,多机用基于socket的RPC而不是更快的RDMA,主要的原因是TF现有框架的抽象对于跨设备的通讯不是很友好(最近开始有一些重新设计的倾向,待考)。

在分布式上百度美研的解决方案要好得多,虽然没有开源,但是Bryan Cantazaro很愿意分享技术细节,混NVidia machine learning summit的大家应该都知道百度在高性能分布式深度学习上的造诣的确很深,作为竞争对手也必须要赞一下。另外,我没有拿百度的钱,求不黑。
作者:贾扬清
链接http://www.zhihu.com/question/41907061/answer/99405271

Theano 和 tensorflow的速度都差不多的(慢),然而一看tensorflow的文档就知道是个大公司的成熟产品,api文档详细,结构和抽象都很合理。
再看看Theano (以及mxnet),一看就是几个做科研的写的,api文档写得非常随意,真是难读。
然而我还是选择了mxnet,毕竟速度快内存占用小,对于用自己打游戏的电脑来跑模型的人来说还是挺重要的。
作者:小格波
链接http://www.zhihu.com/question/41907061/answer/99391201

Paste_Image.png
Paste_Image.png
Paste_Image.png

Both pieces of code run at approximately the same speed. In terms of accuracy, as you can see on the following screenshots, They give quite comparable results, with slightly better results for Keras.

Are you using TF 0.8+? The main reason that TF is sometimes slower than theano is that it is newer. It's currently much faster than after release but there's still room for optimization. This takes time, though.

Predicting sequences of vectors (regression) in Keras using RNN - LSTM

A complete guide to using Keras as part of a TensorFlow workflow


Paste_Image.png

The purpose of Keras is to be a model-level framework, providing a set of "Lego blocks" for building Deep Learning models in a fast and straightforward way. Among Deep Learning frameworks, Keras is resolutely high up on the ladder of abstraction.
Tensorflow 和 theano在keras中 性能比较

Paste_Image.png

In this tutorial, we will answer some common questions about autoencoders, and we will cover code examples of the following models:
a simple autoencoder based on a fully-connected layer
a sparse autoencoder
a deep fully-connected autoencoder
a deep convolutional autoencoder
an image denoising model
a sequence-to-sequence autoencoder
a variational autoencoder ...

A complete guide to using Keras as part of a TensorFlow workflow,If TensorFlow is your primary framework, and you are looking for a simple & high-level model definition interface to make your life easier, this tutorial is for you.

发布

讨论

讨论

讨论

讨论 细致
I already played with it quite a bit Kobol.But it's very slow to train. Being able to use GPUs is much more efficient to test models faster.
Before I started machine learning I was a node hacker so I feel your pain. Karpathy wrote some pretty cool convnet stuff for javascript which I'd encourage you to check out if you have the time. http://cs.stanford.edu/people/karpathy/convnetjs/

论坛讨论
Now, if I'm reading this right, given a list of 10 sequences [0,1,2,3,4,5,6,7,8,9] split into batches [0,1,2,3,4] and [5,6,7,8,9], does it mean that the model expects the couples of sequences (0,5), (1,6), (2,7), etc., to be one after the other? It seems quite an unnatural thing to do, I'd expect the model to treat sequence 5 as the follow-up of sequence 4. Am I missing something important here?

博客


Paste_Image.png

A simple sin?It's an easy signal, easy to train with low error.
A Keras tutorial actually uses a sin for their example

基本概念以及理解、DNN深度神经网络 的原理 以及 使用、RNN循环网络原理以及使用、DLSTM 双向长短期记忆网络 、RSNN深度残差网络、RCNN 区域卷积网络(视频侦测分析人脸侦测区域检测 )

Run-time Performance test of RNN and Streamlined RNN.

I'm trying to follow the Deep Autoencoder Keras example. I'm getting a dimension mismatch exception, but for the life of me, I can't figure out why. It works when I use only one encoded dimension, but not when I stack them.

The Advantage of using Theano for Developing Neural Networks
Various approaches have been considered for the initial assignment of network weights. One method is theXavier algorithm, which balances initial weight assignments such that attenuation or unstable signal growth does not occur initially in convolutional neural networks (CNNs) [2]. The weights in this method are assigned within a uniform distribution having bounding values determined by network properties. In recurring networks, additional mechanisms must be introduced in order to prevent signal attenuation. Memory elements can be positioned in the network, where they effectively sustain signal strength at each stage of recurrence. Two memory models are the Long Short Term Memory (LSTM) [3], and the Gated Recurring Unit (GRU) [4]. The GRU is simpler in structure compared to the LSTM and has been demonstrated to perform better under certain circumstances. The LSTM model, however, has been shown to produce the best network performance given more training time, and a certain constant initial bias parameter value.
The Keras Framework
Since we will be using an NVIDIA Tesla K80 GPU card, we want to examine a network which has sufficient complexity, such that using a GPU provides some practical benefit. Simple models, or smaller components of larger networks, such as Perceptrons, Autoencoders (Restricted Boltzmann Machines), or Multi-layer Perceptrons (MLPs), do not contain enough neurons and connecting weights to require the use of GPUs. These smaller networks can instead be solved on a CPU within reasonable time. Larger networks, inspired by biological models, such as LeNet[5], AlexNet[6], GoogLeNet[7], and other deep network architectures, do require GPUs in order to decrease compute time to a practical range. Modern neural networks designed in order to do image classification, or Natural Language Processing (NLP), require a GPU [8].
Building a Movie Review Sentiment Classifier using Keras and Theano Deep Learning Frameworks
代码
Building an Image Classifier Using Keras and Theano Deep Learning Frameworks
代码
Benchmarking Results for Modified LeNet

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 206,482评论 6 481
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 88,377评论 2 382
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 152,762评论 0 342
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 55,273评论 1 279
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 64,289评论 5 373
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 49,046评论 1 285
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 38,351评论 3 400
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 36,988评论 0 259
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 43,476评论 1 300
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 35,948评论 2 324
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 38,064评论 1 333
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 33,712评论 4 323
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 39,261评论 3 307
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 30,264评论 0 19
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 31,486评论 1 262
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 45,511评论 2 354
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 42,802评论 2 345

推荐阅读更多精彩内容