В настоящее время у меня проблемы с использованием оптимизатора Hyperas в моей сети с несколькими входами..
Вот как я это реализовал:
def data():
X_train, Y_train = next(train_generator())
X_test, Y_test = next(test_generator())
datagen = ImageDataGenerator()
train_list = []
for input in X_train:
train_list.append(datagen.fit(input))
return datagen, train_list, Y_train, X_test, Y_test
Я использую data_generator, так как все данные не могут содержаться в оперативной памяти.
Основываясь на примере данных, которые они сделали, я сделал это:
def fws(datagen, X_train, Y_train, X_test, Y_test):
# Input shape: (batch_size,40,45,3)
# Output shape: (1,15,50)
# Number of units in conv_feature_map = splitd
filter_size = 8
pooling_size = 28
stride_step = 2
pool_splits = ((splits - pooling_size)+1)/2
temp_list = []
sun_temp_list = []
conv_featur_map = []
pool_feature_map = []
print "Printing shapes"
list_of_input = [Input(shape = (window_height,total_frames_with_deltas,3)) for i in range(splits)]
# Convolution
shared_conv = Conv2D(filters = 150, kernel_size = (filter_size,45), activation='relu')
for i in range(splits):
conv_featur_map.append(shared_conv(list_of_input[i]))
# Pooling
input = Concatenate()(conv_featur_map)
input = Reshape((splits,-1))(input)
pooled = MaxPooling1D(pool_size = pooling_size, strides = stride_step)(input)
# fc
dense1 = Dense(units = 1000, activation = 'relu', name = "dense_1")(pooled)
dense2 = Dense(units = 1000, activation = 'relu', name = "dense_2")(dense1)
dense3 = Dense(units = 50 , activation = 'softmax', name = "dense_3")(dense2)
model = Model(inputs = list_of_input , outputs = dense3)
sgd = keras.optimizers.SGD(lr = {{uniform(0, 1)}}, decay = {{uniform(0, 1)}}, momentum = {{uniform(0, 1)}}, nesterov = True)
model.compile(loss="categorical_crossentropy", optimizer=sgd , metrics = [metrics.categorical_accuracy])
hist_current = model.fit_generator(datagen.flow(X_train, Y_train),
steps_per_epoch=32,
epochs = 1000,
verbose = 1,
validation_data = (X_test, Y_test),
validation_steps=32,
pickle_safe = True,
workers = 4)
score, acc = model.evaluate(X_test, Y_test, verbose=0)
return {'loss': -acc, 'status': STATUS_OK, 'model': model}
Особенностью этой сети является то, что она принимает несколько входов. Я мог бы сделать так, чтобы он брал только один вход и использовал лямбда-слой для его разделения, но так как разделение довольно утомительно, я решил сохранить его разделенным и подать его в разделенном виде, создав, таким образом, 33 входа. В остальном сеть довольно стандартная.
(Визуализация сети)
if __name__ == '__main__':
datagen, X_train, Y_train, X_test, Y_test = data()
best_run, best_model = optim.minimize(model=fws,
data=data,
algo=tpe.suggest,
max_evals=5,
trials=Trials())
print("Evalutation of best performing model:")
print(best_model.evaluate(X_test, Y_test))
Это где я начинаю оптимизацию, а также где я получаю сообщение об ошибке:
Traceback (most recent call last):
File "keras_cnn_phoneme_original_fit_generator_hyperas.py", line 211, in <module>
trials=Trials())
File "/usr/local/lib/python2.7/dist-packages/hyperas/optim.py", line 43, in minimize
notebook_name=notebook_name, verbose=verbose)
File "/usr/local/lib/python2.7/dist-packages/hyperas/optim.py", line 63, in base_minimizer
model_str = get_hyperopt_model_string(model, data,functions,notebook_name, verbose, stack)
File "/usr/local/lib/python2.7/dist-packages/hyperas/optim.py", line 130, in get_hyperopt_model_string
imports = extract_imports(cleaned_source, verbose)
File "/usr/local/lib/python2.7/dist-packages/hyperas/utils.py", line 44, in extract_imports
import_parser.visit(tree)
File "/usr/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/usr/lib/python2.7/ast.py", line 249, in generic_visit
self.visit(item)
File "/usr/lib/python2.7/ast.py", line 241, in visit
return visitor(node)
File "/usr/local/lib/python2.7/dist-packages/hyperas/utils.py", line 14, in visit_Import
if (self._import_asnames(node.names)!=''):
File "/usr/local/lib/python2.7/dist-packages/hyperas/utils.py", line 36, in _import_asnames
return ''.join(asname)
TypeError: sequence item 0: expected string, NoneType found
Я не уверен, как следует интерпретировать эту ошибку. Это ошибка реализации или ошибка в библиотеке? Я не знаю...
Минимальный рабочий пример:
import numpy as np
import re
from keras.utils import np_utils
from keras import metrics
import keras
from keras.models import Sequential
from keras.optimizers import SGD
import scipy
from keras.layers.core import Dense, Activation, Lambda, Reshape,Flatten
from keras.layers import Conv1D,Conv2D,MaxPooling2D, MaxPooling1D, Reshape
#from keras.utils.visualize_util import plot
from keras.utils import np_utils
from keras.models import Model
from keras.layers import Input, Dense
from keras.layers import Dropout
from keras import backend as K
from keras.layers.merge import Concatenate
from keras.models import load_model
from keras.utils import plot_model
from keras.preprocessing.image import ImageDataGenerator
import math
import random
from keras.callbacks import ModelCheckpoint
import tensorflow as tf
from hyperopt import Trials, STATUS_OK, tpe
from hyperas import optim
from hyperas.distributions import uniform
def train_generator():
while True:
train_input = np.random.randint(100,size=(1,33,8,45,3))
train_input_list = np.split(train_input,33,axis=1)
for i in range(len(train_input_list)):
train_input_list[i] = train_input_list[i].reshape(1,8,45,3)
train_output = np.random.randint(100,size=(1,3,50))
yield (train_input_list, train_output)
def test_generator():
while True:
test_input = np.random.randint(100,size=(1,33,8,45,3))
test_input_list = np.split(test_input,33,axis=1)
for i in range(len(test_input_list)):
test_input_list[i] = test_input_list[i].reshape(1,8,45,3)
test_output = np.random.randint(100,size=(1,3,50))
yield (test_input_list, test_output)
def data():
X_train, Y_train = next(train_generator())
X_test, Y_test = next(test_generator())
datagen = ImageDataGenerator()
train_list = []
for input in X_train:
train_list.append(datagen.fit(input))
return datagen, train_list, Y_train, X_test, Y_test
def fws(datagen, X_train, Y_train, X_test, Y_test):
# Input shape: (batch_size,40,45,3)
# Output shape: (1,15,50)
# Number of units in conv_feature_map = splitd
filter_size = 8
pooling_size = 28
stride_step = 2
pool_splits = ((splits - pooling_size)+1)/2
temp_list = []
sun_temp_list = []
conv_featur_map = []
pool_feature_map = []
print "Printing shapes"
list_of_input = [Input(shape = (8,45,3)) for i in range(33)]
# Convolution
shared_conv = Conv2D(filters = 150, kernel_size = (filter_size,45), activation='relu')
for i in range(splits):
conv_featur_map.append(shared_conv(list_of_input[i]))
# Pooling
input = Concatenate()(conv_featur_map)
input = Reshape((splits,-1))(input)
pooled = MaxPooling1D(pool_size = pooling_size, strides = stride_step)(input)
#reshape = Reshape((3,-1))(pooled)
#fc
dense1 = Dense(units = 1000, activation = 'relu', name = "dense_1")(pooled)
dense2 = Dense(units = 1000, activation = 'relu', name = "dense_2")(dense1)
dense3 = Dense(units = 50 , activation = 'softmax', name = "dense_3")(dense2)
model = Model(inputs = list_of_input , outputs = dense3)
sgd = keras.optimizers.SGD(lr = {{uniform(0, 1)}}, decay = {{uniform(0, 1)}}, momentum = {{uniform(0, 1)}}, nesterov = True)
model.compile(loss="categorical_crossentropy", optimizer=sgd , metrics = [metrics.categorical_accuracy])
hist_current = model.fit_generator(datagen.flow(X_train, Y_train),
steps_per_epoch=32,
epochs = 1000,
verbose = 1,
validation_data = (X_test, Y_test),
validation_steps=32,
pickle_safe = True,
workers = 4)
score, acc = model.evaluate(X_test, Y_test, verbose=0)
return {'loss': -acc, 'status': STATUS_OK, 'model': model}
if __name__ == '__main__':
datagen, X_train, Y_train, X_test, Y_test = data()
best_run, best_model = optim.minimize(model=fws,
data=data,
algo=tpe.suggest,
max_evals=5,
trials=Trials())
print("Evalutation of best performing model:")
print(best_model.evaluate(X_test, Y_test))