Deming Regression

这里展示如何用TensorFlow求解线性戴明回归。

=+y=Ax+b

我们用iris数据集,特别是:

y = Sepal Length 且 x = Petal Width。

戴明回归Deming regression也称为total least squares, 其中我们最小化从预测线到实际点(x,y)的最短的距离。最小二乘线性回归最小化与预测线的垂直距离，戴明回归最小化与预测线的总的距离，这种回归线最小化y值和x值的误差。我们从加载必要的库开始。

#List3-40

import matplotlib.pyplot as plt

import numpy as np

import tensorflow as tf

from sklearn import datasets

from tensorflow.python.framework import ops

ops.reset_default_graph()

#tf.set_random_seed(42)

np.random.seed(42)

Load the data

iris.data = [(Sepal Length, Sepal Width, Petal Length, Petal Width)]

iris = datasets.load_iris()

x_vals = np.array([x[3] for x in iris.data]) # Petal Width

y_vals = np.array([y[0] for y in iris.data]) # Sepal Length

定义模型，损失函数，变量的梯度。

对于戴明损失,我们想要计算:

它告诉我们点(x,y)到预测线的最短距离, ⋅+A⋅x+b.

def model(x,w,b):

Declare model operations

model_output = tf.add(tf.matmul(x, w), b)

return model_output

def loss1(x,y,w,b):

Declare Deming loss function

deming_numerator = tf.abs(tf.subtract(tf.add(tf.matmul(x, w), b), y))

deming_denominator = tf.sqrt(tf.add(tf.square(w),1))

loss = tf.reduce_mean(tf.truediv(deming_numerator, deming_denominator))

return loss

def grad1(x,y,w,b):

with tf.GradientTape() as tape:

loss_1 = loss1(x,y,w,b)

return tape.gradient(loss_1,[w,b])

Declare batch size

batch_size = 125

learning_rate = 0.25 # Will not converge with learning rate at 0.4

iterations = 50

Create variables for linear regression

w1 = tf.Variable(tf.random.normal(shape=[1,1]),tf.float32)

b1 = tf.Variable(tf.random.normal(shape=[1,1]),tf.float32)

optimizer = tf.optimizers.Adam(learning_rate)

Training loop

loss_vec = []

for i in range(5000):

rand_index = np.random.choice(len(x_vals), size=batch_size)

rand_x = np.transpose([x_vals[rand_index]])

rand_y = np.transpose([y_vals[rand_index]])

x=tf.cast(rand_x,tf.float32)

y=tf.cast(rand_y,tf.float32)

grads1=grad1(x,y,w1,b1)

optimizer.apply_gradients(zip(grads1,[w1,b1]))

#sess.run(train_step, feed_dict={x_data: rand_x, y_target: rand_y})

temp_loss1 = loss1(x, y,w1,b1).numpy()

#sess.run(loss, feed_dict={x_data: rand_x, y_target: rand_y})

loss_vec.append(temp_loss1)

if (i+1)%25==0:

print('Step #' + str(i+1) + ' A = ' + str(w1.numpy()) + ' b = ' + str(b1.numpy()))

print('Loss = ' + str(temp_loss1))

Get the optimal coefficients

slope\] = w1.numpy() \[y_intercept\] = b1.numpy() # Get best fit line best_fit1 = \[

for i in x_vals:

best_fit1.append(slope*i+y_intercept)

Plot the result

plt.plot(x_vals, y_vals, 'o', label='Data Points')

plt.plot(x_vals, best_fit1, 'r-', label='Best fit line', linewidth=3)

plt.legend(loc='upper left')

plt.title('Sepal Length vs Petal Width')

plt.xlabel('Petal Width')

plt.ylabel('Sepal Length')

plt.show()

Plot loss over time

plt.plot(loss_vec, 'k-')

plt.title('L1 Loss per Generation')

plt.xlabel('Generation')

plt.ylabel('L1 Loss')