[PYTHON] PRML implementation Chapter 3 Linear basis function model

PRML implementation Chapter 3 Linear basis function model

I wrote a script to find the maximum likelihood estimation solution.

script.py

import numpy as np
from numpy.linalg import inv
import matplotlib.pyplot as plt

M = 10 #Model complexity
N = 20 #The number of data
L = 10 #Number of datasets

ary_mu = np.linspace(0,1,M)

def phi(ary_mu, x):
    ret = np.array([np.exp(-(x-mu)**2/(2*0.1**2)) for mu in ary_mu])
    ret[0] = 1
    return ret

for x in range(L):
    x_train = np.linspace(0,1,N)
    y_train = np.sin(2*np.pi*x_train) + np.random.normal(0,0.3,N)

    #Design matrix(3.Type 16)Calculation
    PHI = np.array([phi(ary_mu,x) for x in x_train]).reshape(N,M)

    #Maximum likelihood estimation solution(3.Type 15)Calculation
    w_ml = inv(PHI.transpose().dot(PHI)).dot(PHI.transpose()).dot(y_train)
    #Use below for regularization
    # l = 0.3
    # w_ml = inv(l*np.identity(M)+PHI.transpose().dot(PHI)).dot(PHI.transpose()).dot(y_train)

    xx = np.linspace(0,1,100)
    y = [phi(ary_mu,x).dot(w_ml) for x in xx]

    plt.plot(xx,y)

plt.show()

 sin(2*\pi*x) + N(0,0.3)

I generated the data with and displayed the result for each dataset.

When not regularizing

When regularization is performed

is. Somehow the effect of regularization has come out.