Intro to Deep Learning

Trevor Bedford (@trvrb)
27 Jun 2018
VIDD Faculty Retreat
Union, WA

480k ratings on 18k movies

Winning algorithm was a hand-tuned ensemble of matrix factorization, neighborhood models with temporal dynamics, restricted Boltzmann machines and gradient boosted decision trees

Entire model using Keras deep learning library

movie_count = 17771
user_count = 2649430
model_left = Sequential()
model_left.add(Embedding(movie_count, 60, input_length=1))
model_right = Sequential()
model_right.add(Embedding(user_count, 20, input_length=1))
model = Sequential()
model.add(Merge([model_left, model_right], mode='concat'))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('sigmoid'))
model.add(Dense(64))
model.add(Activation('sigmoid'))
model.add(Dense(64))
model.add(Activation('sigmoid'))
model.add(Dense(1))
model.compile(loss='mean_squared_error')
	

Toy example at playground.tensorflow.org

Research focuses on clever construction of network structures fit to the problem domain

Images processing uses convolutional networks

Features build on features

Production deep learning networks are huge, power lies in their ability to scale with increasing data

Imagenet database of 14M images and 20k categories

There has been massive progress in AI tasks enabled by deep learning

Imagenet recognition performance

Software libraries highly advanced and user friendly

       

Moving towards fully GUI tools like lobe.ai

Seeing use in a large number of different domains

• Google Search
• Language translation
• Prediction of chaotic system behavior
• DNA-protein binding, enhancer, regulatory motif prediction
• Prediction of protein function from sequence