neural network - Backward pass in Caffe Python Layer is not called/working? -


i unsuccessfully trying implement simple loss layer in python using caffe. reference, found several layers implemented in python, including here, here , here.

starting euclideanlosslayer provided caffe documentation/examples, not able working , startd debugging. using simple testlayer:

def setup(self, bottom, top):     """     checks correct number of bottom inputs.      :param bottom: bottom inputs     :type bottom: [numpy.ndarray]     :param top: top outputs     :type top: [numpy.ndarray]     """      print 'setup'  def reshape(self, bottom, top):     """     make sure involved blobs have right dimension.      :param bottom: bottom inputs     :type bottom: caffe._caffe.rawblobvec     :param top: top outputs     :type top: caffe._caffe.rawblobvec     """      print 'reshape'     top[0].reshape(bottom[0].data.shape[0], bottom[0].data.shape[1], bottom[0].data.shape[2], bottom[0].data.shape[3])  def forward(self, bottom, top):     """     forward propagation.      :param bottom: bottom inputs     :type bottom: caffe._caffe.rawblobvec     :param top: top outputs     :type top: caffe._caffe.rawblobvec     """      print 'forward'     top[0].data[...] = bottom[0].data  def backward(self, top, propagate_down, bottom):     """     backward pass.      :param bottom: bottom inputs     :type bottom: caffe._caffe.rawblobvec     :param propagate_down:     :type propagate_down:     :param top: top outputs     :type top: caffe._caffe.rawblobvec     """      print 'backward'     bottom[0].diff[...] = top[0].diff[...]

i not able python layer working. learning task rather simple, merely trying predict whether real-valued number positive or negative. corresponding data generated follows , written lmdbs:

n = 10000 n_train = int(0.8*n)  images = [] labels = []  n in range(n):                 image = (numpy.random.rand(1, 1, 1)*2 - 1).astype(numpy.float)     label = int(numpy.sign(image))      images.append(image)     labels.append(label)

writing data lmdb should correct tests mnist dataset provided caffe show no problems. network defined follows:

 net.data, net.labels = caffe.layers.data(batch_size = batch_size, backend = caffe.params.data.lmdb,                                                  source = lmdb_path, ntop = 2)  net.fc1 = caffe.layers.python(net.data, python_param = dict(module = 'tools.layers', layer = 'testlayer'))  net.score = caffe.layers.tanh(net.fc1)  net.loss = caffe.layers.euclideanloss(net.score, net.labels)

solving done manually using:

for iteration in range(iterations):     solver.step(step)

the corresponding prototxt files below:

solver.prototxt:

weight_decay: 0.0005 test_net: "tests/test.prototxt" snapshot_prefix: "tests/snapshot_" max_iter: 1000 stepsize: 1000 base_lr: 0.01 snapshot: 0 gamma: 0.01 solver_mode: cpu train_net: "tests/train.prototxt" test_iter: 0 test_initialization: false lr_policy: "step" momentum: 0.9 display: 100 test_interval: 100000

train.prototxt:

layer {   name: "data"   type: "data"   top: "data"   top: "labels"   data_param {     source: "tests/train_lmdb"     batch_size: 64     backend: lmdb   } } layer {   name: "fc1"   type: "python"   bottom: "data"   top: "fc1"   python_param {     module: "tools.layers"     layer: "testlayer"   } } layer {   name: "score"   type: "tanh"   bottom: "fc1"   top: "score" } layer {   name: "loss"   type: "euclideanloss"   bottom: "score"   bottom: "labels"   top: "loss" }

test.prototxt:

layer {   name: "data"   type: "data"   top: "data"   top: "labels"   data_param {     source: "tests/test_lmdb"     batch_size: 64     backend: lmdb   } } layer {   name: "fc1"   type: "python"   bottom: "data"   top: "fc1"   python_param {     module: "tools.layers"     layer: "testlayer"   } } layer {   name: "score"   type: "tanh"   bottom: "fc1"   top: "score" } layer {   name: "loss"   type: "euclideanloss"   bottom: "score"   bottom: "labels"   top: "loss" }

i tried tracking down, adding debug messages in backward , foward methods of testlayer, forward methods gets called during solving (note no testing performed, calls can related ot solving). added debug messages in python_layer.hpp:

virtual void forward_cpu(const vector<blob<dtype>*>& bottom,     const vector<blob<dtype>*>& top) {   log(info) << "cpp forward";   self_.attr("forward")(bottom, top); } virtual void backward_cpu(const vector<blob<dtype>*>& top,     const vector<bool>& propagate_down, const vector<blob<dtype>*>& bottom) {   log(info) << "cpp backward";   self_.attr("backward")(top, propagate_down, bottom); }

again, forward pass executed. when remove backward method in testlayer, solving still works. when removing forward method, error thrown forward not implemented. expect same backward, seems backward pass not executed @ all. switching regular layers , adding debug messages, works expected.

i have feeling missing simple or fundamental, not able resolve problem several days now. helps or hints appreciated.

thanks!

in addition erik b.'s answer, can force caffe backprob specifying

force_backward: true

in net prototxt.
see comments in caffe.proto more information.


Comments

Post a Comment

Popular posts from this blog

sql server - Cannot query correctly (MSSQL - PHP - JSON) -

i have little problem on project. , turns out query result returns nothing when value of first name or last name has 'ñ' in it. here's code config.php: <?php ini_set('mssql.charset', 'utf-8'); header('content-type: text/html; charset=utf-8'); $servername = "192.168.1.21"; /* ip add of db server */ $connection = array("database" => "db", "uid" => "?", "pwd" => "***"); $conn = sqlsrv_connect($servername, $connection); if(!$conn){ echo "connection not established."; die(print_r(sqlsrv_errors(), true)); } ?> myquery: $idnumber = $_post["idnum"]; $response = array(); $query2 = "select clname, cfname, cmname, cqualifier student cidno = '$idnumber'"; try{ $stmt2 = sqlsrv_query($conn, $query2); } catch(pdoexeption $ex){ $response["success"] = 0; $response["message"] = "d...
Read more

php - trouble displaying mysqli database results in correct order -

i have table in database stores events , date expire. want display events on website display current events first expired events after? i have tried following sql $sql = "select noticeid, notice, noticedesc, noticeimg, noticedate, expirydate tbl_notices group noticeid ,expired order expirydate asc"; but returns expired results first due expirydate being ordered asc in query. my tbl_notice structure looks this noticeid => int(4) notice => varchar(100) noticedesc => text noticedate => timestamp noticeimg => varchar(40) expirydate => datetime expired => int(1) 0 current or 1 expired my php code is $sql = "select noticeid, notice, noticedesc, noticeimg, noticedate, expirydate tbl_notices group noticeid ,expired order expirydate asc"; $result = mysqli_query($conn,$sql); if ($result->num_rows > 0) { // output data of each row while($row = $result->fetch_assoc()) { ...
Read more

C++ Linked List -

i trying create linked list download txt file , use linked list handle file line line. when handling downloaded linked list operations performed on such text editor would. encountering problems however. seems "node(string value)" section of code has wrong though original node() declaration no arguments passes. unable figure out quite is. node.h class node { public: node(); node(string value); void setnext(node *nextnode); // allows user set "next" pointer of node points friend class linkedlist; private: string data; // data box node* next; // pointer box }; node.cpp # include <string> # include "node.h" using namespace std; node::node() { data = ""; next = null; } node::node(string value) { data = value; next = null; } void node::setnext(node *nextnode) // allows user set "next" pointer of node points { this->next = nextnode; } your #include <string> ...
Read more