（二）神经网络入门之Logistic回归（分类问题）

xiaoxiao2021-02-28  13

（一）神经网络入门之线性回归Logistic分类函数（二）神经网络入门之Logistic回归（分类问题）（三）神经网络入门之隐藏层设计Softmax分类函数（四）神经网络入门之矢量化（五）神经网络入门之构建多层网络

Logistic回归（分类问题）

import numpy as np import matplotlib.pyplot as plt from matplotlib.colors import colorConverter, ListedColormap from matplotlib import cm

定义类分布

# Define and generate the samples nb_of_samples_per_class = 20 # The number of sample in each class red_mean = [-1,0] # The mean of the red class blue_mean = [1,0] # The mean of the blue class std_dev = 1.2 # standard deviation of both classes # Generate samples from both classes x_red = np.random.randn(nb_of_samples_per_class, 2) * std_dev + red_mean x_blue = np.random.randn(nb_of_samples_per_class, 2) * std_dev + blue_mean # Merge samples in set of input variables x, and corresponding set of output variables t X = np.vstack((x_red, x_blue)) t = np.vstack((np.zeros((nb_of_samples_per_class,1)), np.ones((nb_of_samples_per_class,1)))) # Plot both classes on the x1, x2 plane plt.plot(x_red[:,0], x_red[:,1], 'ro', label='class red') plt.plot(x_blue[:,0], x_blue[:,1], 'bo', label='class blue') plt.grid() plt.legend(loc=2) plt.xlabel('$x_1$', fontsize=15) plt.ylabel('$x_2$', fontsize=15) plt.axis([-4, 4, -4, 4]) plt.title('red vs. blue classes in the input space') plt.show()

Logistic函数和交叉熵损失函数

交叉熵损失函数

logistic(z)函数实现了Logistic函数，cost(y, t)函数实现了损失函数，nn(x, w)实现了神经网络的输出结果，nn_predict(x, w)实现了神经网络的预测结果。

# Define the logistic function def logistic(z): return 1 / (1 + np.exp(-z)) # Define the neural network function y = 1 / (1 + numpy.exp(-x*w)) def nn(x, w): return logistic(x.dot(w.T)) # Define the neural network prediction function that only returns # 1 or 0 depending on the predicted class def nn_predict(x,w): return np.around(nn(x,w)) # Define the cost function def cost(y, t): return - np.sum(np.multiply(t, np.log(y)) + np.multiply((1-t), np.log(1-y))) # Plot the cost in function of the weights # Define a vector of weights for which we want to plot the cost nb_of_ws = 100 # compute the cost nb_of_ws times in each dimension ws1 = np.linspace(-5, 5, num=nb_of_ws) # weight 1 ws2 = np.linspace(-5, 5, num=nb_of_ws) # weight 2 ws_x, ws_y = np.meshgrid(ws1, ws2) # generate grid cost_ws = np.zeros((nb_of_ws, nb_of_ws)) # initialize cost matrix # Fill the cost matrix for each combination of weights for i in range(nb_of_ws): for j in range(nb_of_ws): cost_ws[i,j] = cost(nn(X, np.asmatrix([ws_x[i,j], ws_y[i,j]])) , t) # Plot the cost function surface plt.contourf(ws_x, ws_y, cost_ws, 20, cmap=cm.pink) cbar = plt.colorbar() cbar.ax.set_ylabel('$\\xi$', fontsize=15) plt.xlabel('$w_1$', fontsize=15) plt.ylabel('$w_2$', fontsize=15) plt.title('Cost function surface') plt.grid() plt.show()

梯度下降优化损失函数

gradient(w, x, t)函数实现了梯度∂ξ/∂w，delta_w(w_k, x, t, learning_rate)函数实现了Δw。

# define the gradient function. def gradient(w, x, t): return (nn(x, w) - t).T * x # define the update function delta w which returns the # delta w for each weight in a vector def delta_w(w_k, x, t, learning_rate): return learning_rate * gradient(w_k, x, t)
梯度下降更新

# Set the initial weight parameter w = np.asmatrix([-4, -2]) # Set the learning rate learning_rate = 0.05 # Start the gradient descent updates and plot the iterations nb_of_iterations = 10 # Number of gradient descent updates w_iter = [w] # List to store the weight values over the iterations for i in range(nb_of_iterations): dw = delta_w(w, X, t, learning_rate) # Get the delta w update w = w-dw # Update the weights w_iter.append(w) # Store the weights for plotting # Plot the first weight updates on the error surface # Plot the error surface plt.contourf(ws_x, ws_y, cost_ws, 20, alpha=0.9, cmap=cm.pink) cbar = plt.colorbar() cbar.ax.set_ylabel('cost') # Plot the updates for i in range(1, 4): w1 = w_iter[i-1] w2 = w_iter[i] # Plot the weight-cost value and the line that represents the update plt.plot(w1[0,0], w1[0,1], 'bo') # Plot the weight cost value plt.plot([w1[0,0], w2[0,0]], [w1[0,1], w2[0,1]], 'b-') plt.text(w1[0,0]-0.2, w1[0,1]+0.4, '$w({})$'.format(i), color='b') w1 = w_iter[3] # Plot the last weight plt.plot(w1[0,0], w1[0,1], 'bo') plt.text(w1[0,0]-0.2, w1[0,1]+0.4, '$w({})$'.format(4), color='b') # Show figure plt.xlabel('$w_1$', fontsize=15) plt.ylabel('$w_2$', fontsize=15) plt.title('Gradient descent updates on cost surface') plt.grid() plt.show()

训练结果可视化

# Plot the resulting decision boundary # Generate a grid over the input space to plot the color of the # classification at that grid point nb_of_xs = 200 xs1 = np.linspace(-4, 4, num=nb_of_xs) xs2 = np.linspace(-4, 4, num=nb_of_xs) xx, yy = np.meshgrid(xs1, xs2) # create the grid # Initialize and fill the classification plane classification_plane = np.zeros((nb_of_xs, nb_of_xs)) for i in range(nb_of_xs): for j in range(nb_of_xs): classification_plane[i,j] = nn_predict(np.asmatrix([xx[i,j], yy[i,j]]) , w) # Create a color map to show the classification colors of each grid point cmap = ListedColormap([ colorConverter.to_rgba('r', alpha=0.30), colorConverter.to_rgba('b', alpha=0.30)]) # Plot the classification plane with decision boundary and input samples plt.contourf(xx, yy, classification_plane, cmap=cmap) plt.plot(x_red[:,0], x_red[:,1], 'ro', label='target red') plt.plot(x_blue[:,0], x_blue[:,1], 'bo', label='target blue') plt.grid() plt.legend(loc=2) plt.xlabel('$x_1$', fontsize=15) plt.ylabel('$x_2$', fontsize=15) plt.title('red vs. blue classification boundary') plt.show()

CoderPai 是一个专注于算法实战的平台，从基础的算法到人工智能算法都有设计。如果你对算法实战感兴趣，请快快关注我们吧。加入AI实战微信群，AI实战QQ群，ACM算法微信群，ACM算法QQ群。长按或者扫描如下二维码，关注 “CoderPai” 微信号（coderpai）