多标签使用两个不同的LMDB

问题描述

我是新的在caffe框架，我想使用caffe实施与多标签的培训。我使用两个LMDB来分别保存数据和标签。数据LMDB的尺寸为Nx1xHxW，而标签LMDB的尺寸为Nx1x1x3。标签是浮动数据。

I am new in caffe framework and I would like to use caffe to implement the training with multi-label. I use two LMDB to save data and labels, respectively. The data LMDB is of dimension Nx1xHxW while the label LMDB is of dimension Nx1x1x3. Labels are float data.

文本文件如下：

5911 3
train/train_data/4224.bmp        13         0        12
train/train_data/3625.bmp        11         3         7
...                              ...

我使用C ++创建LMDB。我的main.cpp：

I use C++ to create LMDB. My main.cpp:

#include <algorithm>
#include <fstream>  // NOLINT(readability/streams)
#include <string>
#include <utility>
#include <vector>
#include <QImage>

#include "boost/scoped_ptr.hpp"
#include "gflags/gflags.h"
#include "glog/logging.h"

#include "caffe/proto/caffe.pb.h"
#include "caffe/util/db.hpp"
#include "caffe/util/format.hpp"
#include "caffe/util/rng.hpp"

#include <boost/filesystem.hpp>
#include <iomanip>
#include <iostream>  // NOLINT(readability/streams)
#include <string>

#include "google/protobuf/message.h"

#include "caffe/common.hpp"
#include "caffe/proto/caffe.pb.h"
#include "caffe/util/format.hpp"

#ifndef CAFFE_TMP_DIR_RETRIES
#define CAFFE_TMP_DIR_RETRIES 100
#endif

using namespace caffe;  // NOLINT(build/namespaces)
using std::pair;
using boost::scoped_ptr;

const char *dat_lab="/home/mul/caffe-master/examples/2D_3D/new/info/train.data";
string data_db="/home/mul/caffe-master/examples/2D_3D/new/2D_3D_data_leveldb";
string label_db="/home/mul/caffe-master/examples/2D_3D/new/2D_3D_label_leveldb";
string root="/home/mul/caffe-master/examples/2D_3D/new/";
string path;

int main()
{

    //Create data DB
    scoped_ptr<db::DB> dat_db(db::GetDB("leveldb"));
    dat_db->Open(data_db, db::NEW);
    scoped_ptr<db::Transaction> dat_txn(dat_db->NewTransaction());

    //Create label DB
    scoped_ptr<db::DB> lab_db(db::GetDB("leveldb"));
    lab_db->Open(label_db, db::NEW);
    scoped_ptr<db::Transaction> lab_txn(lab_db->NewTransaction());

    //Storing to db
    Datum dat_datum,lab_datum;
    int count=0;

    std::ifstream infile(dat_lab);
    std::string filename;
    const char *dataname;
    int dataNum;
    int labelcount;
    QImage img;
    infile>>dataNum>>labelcount;
    LOG(INFO) << "A total of " << dataNum<< " images.";

    for (int line_id = 0; line_id < dataNum; ++line_id)
    {
        infile>>filename;
        path=root+filename;
        dataname=path.c_str();
        img.load(dataname);

        dat_datum.set_channels(1);
        dat_datum.set_height(img.height());
        dat_datum.set_width(img.width());
        dat_datum.clear_data();
        dat_datum.clear_float_data();

        int datum_channels = dat_datum.channels();
        int datum_height = dat_datum.height();
        int datum_width = dat_datum.width();
        int datum_size = datum_channels * datum_height * datum_width;
        std::string buffer(datum_size, ' ');
        const uchar* ptr = img.bits();
        int img_index = 0;
        for (int h = 0; h < datum_height; ++h)
        {

            for (int w = 0; w < datum_width; ++w)
            {
                for (int c = 0; c < datum_channels; ++c)
                {
                    int datum_index = (c * datum_height + h) * datum_width + w;
                    buffer[datum_index] = static_cast<char>(ptr[img_index++]);
                }
            }
        }
        dat_datum.set_data(buffer);


        lab_datum.set_channels(labelcount);
        lab_datum.set_height(1);
        lab_datum.set_width(1);
        lab_datum.clear_data();
        lab_datum.clear_float_data();
        for(int i=0;i<labelcount;++i)
        {
            float mid;
            infile>>mid;
            lab_datum.add_float_data(mid);
        }

        // sequential
        string key_str = caffe::format_int(line_id, 8);

        // Put in db
        string out;
        CHECK(dat_datum.SerializeToString(&out));
        dat_txn->Put(key_str, out);
        CHECK(lab_datum.SerializeToString(&out));
        lab_txn->Put(key_str, out);

        if (++count % 1000 == 0)
        {
            // Commit db
            dat_txn->Commit();
            dat_txn.reset(dat_db->NewTransaction());
            lab_txn->Commit();
            lab_txn.reset(lab_db->NewTransaction());
            LOG(INFO) << "Processed " << count << " files.";
        }
    }
    // write the last batch
    if (count % 1000 != 0)
    {
        dat_txn->Commit();
        lab_txn->Commit();
        LOG(INFO) << "Processed " << count << " files.";
    }
    return 0;
}

可以成功创建两个LMDB。但是当我使用caffe来实现两个LMDB的训练，结果总是错误。损耗层为EUCLIDEAN_LOSS，损失不能下降。我不知道是否可以创建两个LMDB的代码是错误的。谁能帮我 ？非常感谢。

Two LMDB can be created successfully. But when I use caffe to implement the training with two LMDB, the result is always wrong. The loss layer is EUCLIDEAN_LOSS and the loss can not descent. I don't know whether the code which can create two LMDB is wrong. Who can help me ? Thanks whatever.

推荐答案

作为一个整体，上面的代码是确定的，但你应该注意到：

As a whole, your code above is Ok but you should notice that:

1. 您的.cpp是创建LEVELDB而不是LMDB，当然这不是导致您的问题的原因，任一类型都可以。


2. 您的代码生成的标签 LMDB 属于维度 Nx3x1x1 ，而不是Nx1x1x3 strong> NumberxChannelxWidthxHeight ）。


3. 在使用minibatch SGD的学习任务中，据我所知，为了获得更优化的模型，将数据随机转移是非常有用的。我不知道你是否注意到了这一点。但至少你的cpp没有洗掉你的train.data。


4. 最重要的是，导致你的问题在这里最可能撒谎的原因网络中的数据层，它读取您的数据和标签lmdb / leveldb文件，因为您将标签分配给datum的float数据，而caffe中的DataLayer实际上不会读取float数据（仅当您使用自定义数据层）。所以，也请上传您的prototxt文件，定义您的网络。

1. Your .cpp is to create LEVELDB instead of LMDB, of course this is not the reason that caused your problem, either type is ok.
2. The "label LMDB" generated by your code is of dimension Nx3x1x1 instead of Nx1x1x3(NumberxChannelxWidthxHeight).
3. In a learning task using minibatch SGD, as far as I know, it is very useful to shuffle your data for training to get more optimized model. I'm not sure whether you noticed this. But at least your cpp didn't shuffle your "train.data".
4. Most important of all, the reason that caused your problem here most possibly lied in the data layer in your network which read your data and label lmdb/leveldb files, because you assigned the labels to the float data of datum and the DataLayer in caffe in fact does not read the float data(Only if you used your self-defined data layer). So please also upload your prototxt file which defines your network. Thus we can find out what the problem really was.

最后，我添加了一个MultiTaskData图层从多重任务训练的数据读取多标签，您可以进行简单的修改将其添加到您的咖啡并使用如下：

At last, I added a "MultiTaskData" layer MultiTaskDataLayer to read multi-labels from datum for multi-task training and you can make simple modifications to add it to your caffe and use like this:

    name: "AgeNet"
    layer {
        name: "Age"
        type: "MultiTaskData"
        top: "data"
        top: "age_label"
        top: "gender_label"
        data_param { 
            source: "age_gender_classification_0_60p_train_leveldb"   
            batch_size: 60 
            task_num: 2
            label_dimension: 1
            label_dimension: 1
        }
        transform_param {
            scale: 0.00390625
            crop_size: 60
            mirror: true
        }
        include:{ phase: TRAIN }
    }
    layer { 
        name: "cls_age" 
        type: "InnerProduct"
        bottom: "data"  
        top: "cls_age" 
        param {
            lr_mult: 1
            decay_mult: 1
        }
        param {
            lr_mult: 2
            decay_mult: 0
        }
        inner_product_param {
            num_output: 7
            weight_filler {
            type: "xavier"
            }    
        bias_filler {      
            type: "constant"
            }  
        }
    }
    layer {  
        name: "age_loss"  
        type: "SoftmaxWithLoss"  
        bottom: "cls_age" 
        bottom: "age_label"
        top: "age_loss"
        include:{ phase: TRAIN }
    }
    layer { 
        name: "cls_gender" 
        type: "InnerProduct"
        bottom: "data"  
        top: "cls_gender" 
        param {
            lr_mult: 1
            decay_mult: 1
        }
        param {
            lr_mult: 2
            decay_mult: 0
        }
        inner_product_param {
            num_output: 2
            weight_filler {
                type: "xavier"
            }    
            bias_filler {      
                type: "constant"
            }  
        }
    }
    layer {  
        name: "gender_loss"  
        type: "SoftmaxWithLoss"  
        bottom: "cls_gender" 
        bottom: "gender_label"
        top: "gender_loss"
        include:{ phase: TRAIN }
    }

这篇关于多标签使用两个不同的LMDB的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！