Large-scale Linear Models with TensorFlow#

TensorFlow下的大规模线性模型#

The tf.learn API provides (among other things) a rich set of tools for working with linear models in TensorFlow. This document provides an overview of those tools. It explains:

what a linear model is.
why you might want to use a linear model.
how tf.learn makes it easy to build linear models in TensorFlow.
how you can use tf.learn to combine linear models with deep learning to get the advantages of both.

在Tensorflow里，tf.learn API 提供一个丰富的用于线性模型的工具集。这篇文档提供一个关于这些工具的概述，解释了：

什么是线性模型。
为什么你可能会想使用线性模型。
如果在 Tensorflow 内使 tf.learn 方便的建立线性模型。
如何使用 tf.learn 将线性模型和深度学习进行结合以获得其优势。

Read this overview to decide whether the tf.learn linear model tools might be useful to you. Then do the Linear Models tutorial to give it a try. This overview uses code samples from the tutorial, but the tutorial walks through the code in greater detail.

阅读这篇概述以决定 tf.learn 线性模型工具是否对你有用。然后尝试一下线性模型教程。这篇概述会使用该教程的示例代码，并详细介绍这些代码。

To understand this overview it will help to have some familiarity with basic machine learning concepts, and also with tf.learn.

理解这篇概述，它将有助于你熟悉基本的机器学习的概念和 tf.learn.

Contents

What is a linear model?##

什么是线性模型？##

A linear model uses a single weighted sum of features to make a prediction. For example, if you have data on age, years of education, and weekly hours of work for a population, you can learn weights for each of those numbers so that their weighted sum estimates a person's salary. You can also use linear models for classification.

一个线性模型使用特征的单个加权和来进行预测。例如，如果你有一些人口的年龄，受教育程度和每周工作时间的数据。你可以学习到每个参数，并使其的和来估计一个人的薪水。你也同样可以使用线性模型来进行分类。

Some linear models transform the weighted sum into a more convenient form. For example, logistic regression plugs the weighted sum into the logistic function to turn the output into a value between 0 and 1. But you still just have one weight for each input feature.

一些线性模型会将加权和转换为一个更方便的形式。例如，logistic 回归 注入加权和进 logistic函数中以将输出转换为0和1之间的值。但是你依然需要为每一个输入的特征提供一个权重。

Why would you want to use a linear model?##

为什么你想使用线性模型？##

Why would you want to use so simple a model when recent research has demonstrated the power of more complex neural networks with many layers?

为什么你要在最近的研究展示了多层复杂的神经网络的（强大）能力的情况下，使用一个如此简单的模型？

Linear models:

train quickly, compared to deep neural nets.
can work well on very large feature sets.
can be trained with algorithms that don't require a lot of fiddling with learning rates, etc.
can be interpreted and debugged more easily than neural nets. You can examine the weights assigned to each feature to figure out what's having the biggest impact on a prediction.
provide an excellent starting point for learning about machine learning.
are widely used in industry.

线性模型:

相比于深度神经网络有着更快的训练速度。
能够在大量特征集下表现优秀。
能够使用不需要大量的学习速率的算法进行训练。
比起神经网络，容易进行解释和调试。你可以为每个特征分配权重以获得其对于预测有着多大的影响。
为学习机器学习提供了一个极好的起步。
广泛应用于工业。

How does tf.learn help you build linear models?##

tf.learn如何帮助你建立线性模型？###

You can build a linear model from scratch in TensorFlow without the help of a special API. But tf.learn provides some tools that make it easier to build effective large-scale linear models.

在Tensorflow下，你能够在不借助一些特别的API的情况下，从头开始建立一个线性模型。并且 tf.learn 提供一些工具使得能够的建立一个大规模线性模型。

Feature columns and transformations###

特征列和转换###

Much of the work of designing a linear model consists of transforming raw data into suitable input features. tf.learn uses the FeatureColumn abstraction to enable these transformations.

设计一个线性模型有许多工作，包括转换原始数据成一个适合的输入特征。tf.learn 使用 FeatureColumn 来抽象使用这些变换。

A FeatureColumn represents a single feature in your data. A FeatureColumn may represent a quantity like 'height', or it may represent a category like 'eye_color' where the value is drawn from a set of discrete possibilities like {'blue', 'brown', 'green'}.

一个FeatureColumn 在你的数据中表示一个单个特征。一个 FeatyreColumn 可能会表示成 '高度' 的数量，或者它可能表示成一个 '眼睛颜色' 的目录并且其值可能会被绘制成一个像{'蓝色', '棕色', '绿色'}的离散可能性值。

In the case of both continuous features like 'height' and categorical features like 'eye_color', a single value in the data might get transformed into a sequence of numbers before it is input into the model. The FeatureColumn abstraction lets you manipulate the feature as a single semantic unit in spite of this fact. You can specify transformations and select features to include without dealing with specific indices in the tensors you feed into the model.

在这两种连续特征，像 '高度' 和像 '眼睛颜色' 的 ‘分类特征’ 情况下，单个值在输入这个模型前可能会被转换成一个数字序列。抽象出的FeatureColumn 使得你能够像操作语义一样来操作这些特征。你可以在不需要处理输入模型的张量中的特定指数的情况下，指定转换方式并选择要进行转换的特征。

Sparse columns####

稀疏列####

Categorical features in linear models are typically translated into a sparse vector in which each possible value has a corresponding index or id. For example, if there are only three possible eye colors you can represent 'eye_color' as a length 3 vector: 'brown' would become [1, 0, 0], 'blue' would become [0, 1, 0] and 'green' would become [0, 0, 1]. These vectors are called "sparse" because they may be very long, with many zeros, when the set of possible values is very large (such as all English words).

分类特征在线性模型通常被转化成一个稀疏向量，并且其对应值都有一个对应的序列或者id。例如，如果对于眼睛颜色只存在三种可能性的情况下，你可以表示‘眼睛颜色’为一个长度为3的向量：'棕色'应为[1, 0, 0], '蓝色'应为[0, 1, 0]然后 '绿色' 为 [0, 0, 1]。当可能值的集合非常大时（如所有英语单词），这些向量被称为“”稀疏向量”，因为他们可能很长但是存在很多0值。

While you don't need to use sparse columns to use tf.learn linear models, one of the strengths of linear models is their ability to deal with large sparse vectors. Sparse features are a primary use case for the tf.learn linear model tools.

当你不需要使用稀疏来使用 tf.learn 的线性模型，线性模型的其中一个优势是他有能力去处理大规模的稀疏向量。稀疏特征是 tf.learn 线性模型工具的一个主要用例。

Encoding sparse columns#####

编码稀疏列#####

FeatureColumn handles the conversion of categorical values into vectors automatically, with code like this:
eye_color = tf.contrib.layers.sparse_column_with_keys( column_name="eye_color", keys=["blue", "brown", "green"])

稀疏列主要用于自动将分类值转化为向量值，其代码像：
eye_color = tf.contrib.layers.sparse_column_with_keys( column_name="eye_color", keys=["blue", "brown", "green"])

where eye_color is the name of a column in your source data.

eye_color是你的源数据中一列的名称。

You can also generate FeatureColumns for categorical features for which you don't know all possible values. For this case you would use sparse_column_with_hash_bucket(), which uses a hash function to assign indices to feature values.
education = tf.contrib.layers.sparse_column_with_hash_bucket("education", hash_bucket_size=1000)

当你不知道所有的可能值时，你同样能够为分类特征生成FeatureColumns。在这种情况下你应该使用sparse_column_with_hash_bucket()，其使用hash函数去为特征值分类序列。
education = tf.contrib.layers.sparse_column_with_hash_bucket("education", hash_bucket_size=1000)

Feature Crosses#####

交叉特征#####

Because linear models assign independent weights to separate features, they can't learn the relative importance of specific combinations of feature values. If you have a feature 'favorite_sport' and a feature 'home_city' and you're trying to predict whether a person likes to wear red, your linear model won't be able to learn that baseball fans from St. Louis especially like to wear red.

因为线性模型是给每个单独的特征分配独立的权重，所以他们无法学习特征在特定组合的重要性。如果你有一个特征 '最喜欢的运动' 和特征 '居住城市'，然后并且你试图想要推断一个人在什么情况下会穿红色的衣服。你的线性模型将没有不会学习到来自St.Louis的棒球粉丝是很喜欢穿红色系的衣服。

You can get around this limitation by creating a new feature 'favorite_sport_x_home_city'. The value of this feature for a given person is just the concatenation of the values of the two source features: 'baseball_x_stlouis', for example. This sort of combination feature is called a feature cross.

你可以通过创建一个新的叫‘最喜欢的运动x居住城市’ 特征来克服这个限制。这个特征的值对一个给定的人（的数据）来说只是关联了两个源特征的值。例如，这种组合特征被称为交叉特征。

The crossed_column() method makes it easy to set up feature crosses:
sport = tf.contrib.layers.sparse_column_with_hash_bucket("sport", hash_bucket_size=1000) city = tf.contrib.layers.sparse_column_with_hash_bucket("city", hash_bucket_size=1000) sport_x_city = tf.contrib.layers.crossed_column([sport, city], hash_bucket_size=int(1e4))

crossed_column()函数能够很容易就建立出交叉特征。

Continuous columns####

连续列####

You can specify a continuous feature like so:
age = tf.contrib.layers.real_valued_column("age")

你像这样可以指定一个连续特征：

Although, as a single real number, a continuous feature can often be input directly into the model, tf.learn offers useful transformations for this sort of column as well.

尽管对于一个实数，一个连续的特征一般是能够直接输入到模型中的。tf.learn也提供对连续列进行转换。

Bucketization####

桶化####

Bucketization turns a continuous column into a categorical column. This transformation lets you use continuous features in feature crosses, or learn cases where specific value ranges have particular importance.

桶化转换一个连续列为分类列。这种转换会让你在交叉特征中使用连续特征，或者是学习对于特定的范围值能够造成的重大影响。

Bucketization divides the range of possible values into subranges called buckets:
age_buckets = tf.contrib.layers.bucketized_column( age, boundaries=[18, 25, 30, 35, 40, 45, 50, 55, 60, 65])

桶化会将可能值的范围划分为一系列小的子范围，这样的子范围被称之为桶：

The bucket into which a value falls becomes the categorical label for that value.

~~其桶内新添加的值将成为该值的分类标签~~

Input function####

输入函数####

FeatureColumns provide a specification for the input data for your model, indicating how to represent and transform the data. But they do not provide the data itself. You provide the data through an input function.

FeatureColumns为你的模型输入提供了一个规范格式，表面如何去表示和转换这些数据。但是其本身不会提供这些数据。你需要通过一个输入函数来提供这些数据。

The input function must return a dictionary of tensors. Each key corresponds to the name of a FeatureColumn. Each key's value is a tensor containing the values of that feature for all data instances. See Building Input Functions with tf.contrib.learn for a more comprehensive look at input functions, and input_fn in the linear models tutorial code for an example implementation of an input function.

这个输入函数必须返回一个张量的目录。每一个键值对应一个FeatureColumn的名字。每一个键值对应的值是一个包含所有数据实例中某类特征值的张量。详情可见使用tf.contrib.learn建立输入函数以便更全面的了解输入函数，并且在线性模型教程代码中有引入了一个输入函数的实例。

The input function is passed to the fit() and evaluate() calls that initiate training and testing, as described in the next section.

输入函数通过调用fit()和evaluate()来发起训练和测试，将会在下一节介绍他们。

Linear estimators###

线性估计器###

tf.learn's estimator classes provide a unified training and evaluation harness for regression and classification models. They take care of the details of the training and evaluation loops and allow the user to focus on model inputs and architecture.

tf.learn的估计器类为回归和分类模型提供了一个统一的训练和评估方法。他们主要处理训练和评估循环的细节及其允许用户能够更专注于模型输入与结构。

To build a linear estimator, you can use either the tf.contrib.learn.LinearClassifier estimator or the tf.contrib.learn.LinearRegressor estimator, for classification and regression respectively.

为了建立一个线性估计器，你可以使用分别用于分类和回归的tf.contrib.learn.LinearClassifier估计器或tf.contrib.learn.LinearRegressor估计器。

As with all tf.learn estimators, to run the estimator you just:

Instantiate the estimator class. For the two linear estimator classes, you pass a list of FeatureColumns to the constructor.
Call the estimator's fit() method to train it.
Call the estimator's evaluate() method to see how it does.
For example:
e = tf.contrib.learn.LinearClassifier(feature_columns=[ native_country, education, occupation, workclass, marital_status, race, age_buckets, education_x_occupation, age_buckets_x_race_x_occupation], model_dir=YOUR_MODEL_DIRECTORY)
e.fit(input_fn=input_fn_train, steps=200)
# Evaluate for one step (one pass through the test data).
results = e.evaluate(input_fn=input_fn_test, steps=1)
# Print the stats for the evaluation.for key in sorted(results):
print "%s: %s" % (key, results[key])

所有的tf.learn估计器，只需要以下几步就可以运行：

实例化估计器类。对于这两种估计器类，需要传递一个FeatureColumns列表到其构造器中。
调用估计器的fit()方法去训练它。
调用估计器的evaluate()方法去观察其运行。
例如：

Wide and deep learning##

广度和深度学习##

The tf.learn API also provides an estimator class that lets you jointly train a linear model and a deep neural network. This novel approach combines the ability of linear models to "memorize" key features with the generalization ability of neural nets. Use tf.contrib.learn.DNNLinearCombinedClassifier to create this sort of "wide and deep" model:
e = tf.contrib.learn.DNNLinearCombinedClassifier( model_dir=YOUR_MODEL_DIR, linear_feature_columns=wide_columns, dnn_feature_columns=deep_columns, dnn_hidden_units=[100, 50])

tf.learn API同样提供一个估计器类使得让你联合训练一个线性模型和一个深度神经网络。这种新颖的方法结合了线性模型“记忆”关键特征与神经网络的泛化能力。使用tf.contrib.learn.DNNLinearCombinedClassifier去创建这种“广度且深度”的模型：

For more information, see the Wide and Deep Learning tutorial.

欲了解更多信息，可以查看这个广度和深度学习教程.

（翻译）TensorFlow下的大规模线性模型