-
Notifications
You must be signed in to change notification settings - Fork 11
Expand file tree
/
Copy pathREADME
More file actions
148 lines (107 loc) · 7.52 KB
/
Copy pathREADME
File metadata and controls
148 lines (107 loc) · 7.52 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
# Copyright 2015 Yin Zheng, Yu-Jin Zhang, Hugo Larochelle. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification, are
# permitted provided that the following conditions are met:
#
# 1. Redistributions of source code must retain the above copyright notice, this list of
# conditions and the following disclaimer.
#
# 2. Redistributions in binary form must reproduce the above copyright notice, this list
# of conditions and the following disclaimer in the documentation and/or other materials
# provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY Yin Zheng, Yu-Jin Zhang, Hugo Larochelle ``AS IS'' AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF
# MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL
# Yin Zheng, Yu-Jin Zhang, Hugo Larochelle OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT,
# INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
# TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
# THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
# The views and conclusions contained in the software and documentation are those of the
# authors and should not be interpreted as representing official policies, either expressed
# or implied, of Yin Zheng, Yu-Jin Zhang, Hugo Larochelle.
'''
@reference: A Deep and Autoregressive Approach for Topic Modeling of Multimodal Data, IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI)
Project page: https://sites.google.com/site/zhengyin1126/home/supdeepdocnade
@Authors: Yin Zheng, Received Ph.D from Tsinghua University, 2015. Homepage: https://sites.google.com/site/zhengyin1126/
Yu-Jin Zhang, Tsinghua University,
Hugo Larochelle, University of Sherbrooke and Twitter. Homepage: http://www.dmi.usherb.ca/~larocheh/index_en.html
@contact: Yin Zheng, yzheng3xg@gmail.com
'''
We provide:
1. SupDeepDocNADE.py : Classes of DeepDocNADE and SupDeepDocNADE, which is used by run_pretrain_DeepDocNADE.py and run_SupDeepDocNADE.py
2. run_pretrain_DeepDocNADE.py : Python script to pretrain DeepDocNADE model on MIR Flickr Unlabeled data
3. run_SupDeepDocNADE.py : Python script to train and test SupDeepDocNADE mdoel.
4. gen_dataset_labeled.py: Python script to generate the labeled dateset files needed by the model
5. gen_dataset_unlabeled.py: Python script to generate the unlabeled dateset files needed by the model
6. MIR_Flickr_Theano_lab.py: Python script to load labeled dataset and provide interface to SupDeepDocNADE model
7. MIR_Flickr_Theano_Unlab.py: Python script to load unlabeled dataset and provide interface to DeepDocNADE model
=====================================================================================================================
0. Install liblinear (http://www.csie.ntu.edu.tw/~cjlin/liblinear/) and put liblinear into the PYTHONPATH
NOTE: make sure that you can use liblinear in python. To test whether you config liblinear successfully, you could
use "from liblinearutil import *" in python.
1. Generate the dataset:
1) Download the dataset from Nitsh's homepage: http://www.cs.toronto.edu/~nitish/multimodal/index.html
2) Run "python gen_dataset_labeled.py" to generate the labeled dataset.
a) Read the comments in the script about how to use it.
b) create a file IN THE SAME PATH OF THE LABELED DATASET named "meta.txt" with the following lines in the meta.txt
voc_size:2000
n_regions:14
text_voc_size:2000
global_feat_size:1857
n_classes:38
c) Create a file IN THE SAME PATH OF THE LABELED DATASET named "sizes.txt" with the following content:
10000
5000
10000
3) Run "python gen_dataset_unlabeled.py" to generate the unlabeled dataset.
a) Read the comments in the script about how to use it.
b) create a file IN THE SAME PATH OF THE UNLABELED DATASET with the name meta.txt, the content is:
voc_size:2000
n_regions:14
text_voc_size:2000
global_feat_size:1857
dataset_split:50
c) create a file IN THE SAME PATH OF THE UNLABELED DATASET named "sizes.txt" with the following content:
975000
2. Run run_pretrain_DeepDocNADE.py to pretrain DeepDocNADE model on unlabeled data:
python run_pretrain_DeepDocNADE.py n_pretrain pre_learning_rate hidden_size activation_function dropout_rate model_file_dir unlab_dataset_dir batch_size anno_weight polyakexp_weight model_init
The parameters of the script is as follows:
n_pretrain : number of iterations
pre_learning_rate : learning rate of the pretraining
hidden_size : the hidden size of the model, e.g. 2048_2048_2048 is a 3 hidden layers model with 2048 units each layer
activation_function : the activation function of the hidden layers, "sigmoid, relu or tanh"
dropout_rate: the dropout rate for each hidden layer, e.g. "0.5_0.5_0.5" means dropout rate is 0.5 for each layer
model_file_dir: path to save the pretrained model
unlab_dataset_dir: the path to the unlabeled dataset
batch_size: the batch size
anno_weight: annotation weight
polyakexp_weight: polyak weight
model_init: path to the saved pretrain model, which is used to continue_training based on the trained model. It could be NULL if no model saved before.
One example of the scipt is:
python run_pretrain_DeepDocNADE.py 6000 0.03 2048_2048_2048 reclin 0.5_0.5_0.5 PATH_TO_SAVE_THE_MODEL PATH_TO_UNLABELED_DATA 500 12000 std 0.9995 PATH_TO_SAVED_MODEL
3. Run run_SupDeepDocNADE.py to train SupDeepDocNADE model on labeled data, which could be trained from scratch or initialize from pretrained model on unlabeled data :
python run_SupDeepDocNADE.py folder_ID use_pretrain max_iter look_ahead hidden_size learning_rate unsup_weight activation_function Linear_minC, Linear_maxC, dropout_rate uniresult_dir Pretrain_model_name lab_dataset_dir batch_size anno_weight polyakexp_weight
The parameters are as follows:
folder_ID : ID of the dataset ( 1 to 5 )
use_pretrain: Whether use pretrained model or training from randomly initialized parameters (True or False)
max_iter: number of max iterations
look_ahead: early stop if number of iterations without improvement exceed the number of look ahead
hidden_size: the hidden size of the model, e.g. 2048_2048_2048 is a 3 hidden layers model with 2048 units each layer
learning_rate: learning rate of training process
unsup_weight: the weight of the unsupervised part
activation_function: the activation function of the hidden layers, "sigmoid, relu or tanh"
Linear_minC: the minimum value of C for linear SVM (in log_2 space)
Linear_maxC: the max value of C for linear SVM (in log_2 space)
dropout_rate: the dropout rate for each hidden layer, e.g. "0.5_0.5_0.5" means dropout rate is 0.5 for each layer
uniresult_dir: the path to save the results
Pretrain_model_name: the path and the name of the pretrained model if you set use_pretrain True. Otherwise, any string
lab_dataset_dir: the path to the labeled dataset
batch_size : the mini-batch size
anno_weight: the weight of the annotation words
polyakexp_weight: polyak weight
One example of the script is:
python run_SupDeepDocNADE.py 1 True 20000 300 2048_2048_2048 0.01 0.25 reclin -17 10 0.5_0.5_0.5 PATH_TO_SAVE_RESULTS PATH_NAME_OF_PRETRAINED_MODEL PATH_TO_LABELED_DATA 500 12000 0.9995