我正在尝试序列化以下类型的对象std::unordered_map<std::vector<Card>, std::unordered_map<InfosetHistory, Node>>& nodeMapCard是一个结构，InfosetHistory和Node是使用其他一些结构作为成员变量的类。我已经为所有需要它的类创建了serialize函数。例如，这是Card的代码:
struct Card {
int rank;
int suit;
...
template<class Archive>
void serialize(Archive& ar, const unsigned int version) {
ar & rank;
ar & suit;
}
};
我按如下方式序列化nodeMap:
std::ofstream file("Strategy" + std::to_string(iteration) + ".bin");
boost::archive::binary_oarchive archive(file);
archive << nodeMap;
file.close();
然后像这样分别反序列化(当前选择反序列化“Strategy0.bin”):
std::ifstream ifs("Strategy0.bin");
std::unordered_map<std::vector<Card>, std::unordered_map<InfosetHistory, Node>> nodeMap;
if (ifs.good()) {
boost::archive::binary_iarchive ia(ifs);
ia >> nodeMap;
}
当我运行该程序以创建并序列化nodeMap时，我始终能够进行序列化而没有任何问题。分别创建了.bin文件，它们的大小似乎适合于我希望它们存储的数据。
但是，当我运行程序以反序列化nodeMap时，如果nodeMap不那么大，则不会有问题，但是如果它很大，则会出现以下错误:
terminate called after throwing an instance of 'boost::archive::archive_exception'
what(): input stream error
我认为这实际上并不是因为nodeMap很大，更多是因为代码创建某项的可能性以某种方式导致了问题，并且添加的项越多，出现问题的可能性就越大。我在Boost文档中读到，由于未初始化的数据，可能会产生这种错误。我不相信我有未初始化的数据，但是我不确定如何确保这一点。
通常，我不确定如何调试此类问题。任何帮助，将不胜感激。
注意:我非常努力地创建了一个最小的可复制示例，但是我创建的所有示例均未产生问题。仅当我在程序中创建此类对象并添加成千上万的条目时，我才遇到此类问题。
编辑
@sehe要求更多代码。这是与序列化对象相关的类和结构的所有相关部分:
https://pastebin.com/xPE8h8a3
请注意，这些类和结构位于单独的文件中。
InfosetHistory和DrawAction在InfosetHistory.h中声明，Node在Node.h中声明，Card在GRUtil.h中声明。@sehe还提到我没有提到上面序列化的类型。要序列化的类型是对我要序列化的对象的引用:
std::unordered_map<std::vector<Card>, std::unordered_map<InfosetHistory, Node>>& nodeMap。编辑2
我设法使用下面提供的代码@sehe创建了一个最小的可复制示例。使用我的代码，我创建了一个知道会产生反序列化错误的NodeMap，然后将所有数据打印到一个名为“StrategyWritten0.txt”的文本文件中。在这个可重现的示例中，我输入了该文本文件中的所有数据以创建NodeMap，序列化所得NodeMap中的所有数据，然后尝试反序列化NodeMap。我得到以下输出:
Successfully created Node Map
Serialized Node Map
terminate called after throwing an instance of 'boost::archive::archive_exception'
what(): input stream error
这是文件:
https://drive.google.com/file/d/1y4FLgi7f-XWJ-igRq_tItK-pXFDEUbhg/view?usp=sharing
这是代码:
https://pastebin.com/FebQqssx
最佳答案
更新
经过数天的努力(在VM上安装MingW并调试了详细的细节)后，我将其范围缩小到在关联容器的第25个元素之后发生的某些特殊情况。不久之后，我had the brainwave:
是。 Windows线尾对几个人来说毁了好几天。再次。
我总是在我的存档文件上写std::ios::binary标志，但是在这种情况下，我没有发现它们丢失了。
添加它们将解决问题:
void writeBinaryStrategy(std::string const& fname, NodeMap const& nodeMap) {
std::ofstream f(fname, std::ios::binary);
boost::archive::binary_oarchive archive(f);
archive << nodeMap;
}
NodeMap readBinaryStrategy(std::string const& fname) {
NodeMap nodeMap;
std::ifstream ifs(fname, std::ios::binary);
boost::archive::binary_iarchive ia(ifs);
ia >> nodeMap;
return nodeMap;
}
在研究细节的过程中，我提出了一个严格的往返测试器应用程序。代码是Coliru上的~unlive~，因此我在文件中添加了一个Gist:
test.cpp#include <iostream>
#include <boost/archive/binary_oarchive.hpp>
#include <boost/archive/binary_iarchive.hpp>
#include <boost/serialization/serialization.hpp>
#include <boost/serialization/unordered_map.hpp>
#include <boost/serialization/map.hpp>
#include <boost/serialization/vector.hpp>
#include <boost/serialization/array.hpp>
#include <boost/functional.hpp>
#include <boost/container_hash/hash.hpp>
using boost::hash_value;
struct Card {
int rank, suit;
Card(int rank = 0, int suit = 0) : rank(rank), suit(suit) {}
template<class Archive>
void serialize(Archive& ar, unsigned /*version*/) {
ar & rank;
ar & suit;
}
friend size_t hash_value(Card const& c) {
auto v = hash_value(c.rank);
boost::hash_combine(v, hash_value(c.suit));
return v;
}
auto tied() const { return std::tie(rank, suit); }
bool operator<(Card const& rhs) const { return tied() < rhs.tied(); }
bool operator==(Card const& rhs) const { return tied() == rhs.tied(); }
};
struct DrawAction {
bool fromDeck;
Card card;
explicit DrawAction(bool fromDeck = false, Card card = {})
: fromDeck(fromDeck), card(card) {}
template<class Archive>
void serialize(Archive& ar, unsigned /*version*/) {
ar & fromDeck;
ar & card;
}
friend size_t hash_value(DrawAction const& da) {
auto v = hash_value(da.fromDeck);
boost::hash_combine(v, hash_value(da.card));
return v;
}
auto tied() const { return std::tie(fromDeck, card); }
bool operator<(DrawAction const& rhs) const { return tied() < rhs.tied(); }
bool operator==(DrawAction const& rhs) const { return tied() == rhs.tied(); }
};
using Cards = std::vector<Card>;
using Hand = std::array<Card, 10>;
using Draws = std::vector<DrawAction>;
class InfosetHistory {
public:
Card initialDiscardPile;
Hand initialHand;
Draws playerDrawActions;
Cards playerDiscardActions;
Draws opponentDrawActions;
Cards opponentDiscardActions;
InfosetHistory(
Card initialDiscardPile = {},
Hand hand = {},
Draws playerDrawActions = {},
Cards playerDiscardActions = {},
Draws opponentDrawActions = {},
Cards opponentDiscardActions = {}
) : initialDiscardPile(initialDiscardPile),
initialHand(std::move(hand)),
playerDrawActions(std::move(playerDrawActions)),
playerDiscardActions(std::move(playerDiscardActions)),
opponentDrawActions(std::move(opponentDrawActions)),
opponentDiscardActions(std::move(opponentDiscardActions)) {}
template<class Archive>
void serialize(Archive& ar, const unsigned int /*version*/) {
ar & initialDiscardPile & initialHand
& playerDrawActions & playerDiscardActions
& opponentDrawActions & opponentDiscardActions;
}
friend size_t hash_value(InfosetHistory const& ish) {
auto v = hash_value(ish.initialDiscardPile);
auto combine = [&v](auto& range) { boost::hash_range(v, begin(range), end(range)); };
combine(ish.initialHand);
combine(ish.playerDrawActions);
combine(ish.playerDiscardActions);
combine(ish.opponentDrawActions);
combine(ish.opponentDiscardActions);
return v;
}
auto tied() const { return std::tie(initialDiscardPile, initialHand,
playerDrawActions, playerDiscardActions, opponentDrawActions,
opponentDiscardActions); }
bool operator<(InfosetHistory const& rhs) const { return tied() < rhs.tied(); }
bool operator==(InfosetHistory const& rhs) const { return tied() == rhs.tied(); }
};
class Node {
public:
Cards allowedActions;
unsigned int NUM_ACTIONS{};
std::vector<double> regretSum;
std::vector<double> strategySum;
unsigned char phase{};
explicit Node(std::vector<Card> allowedActions = {},
unsigned int NUM_ACTIONS = 0,
std::vector<double> regretSum = {},
std::vector<double> strategySum = {},
unsigned char phase = 0
) : allowedActions(std::move(allowedActions)),
NUM_ACTIONS(NUM_ACTIONS),
regretSum(std::move(regretSum)),
strategySum(std::move(strategySum)),
phase(phase) {}
template<class Archive>
void serialize(Archive& ar, unsigned /*version*/) {
ar & allowedActions
& NUM_ACTIONS
& regretSum & strategySum & phase;
}
auto tied() const { return std::tie(allowedActions, NUM_ACTIONS, regretSum, strategySum, phase); }
bool operator<(Node const& rhs) const { return tied() < rhs.tied(); }
bool operator==(Node const& rhs) const { return tied() == rhs.tied(); }
};
#include <map>
#include <fstream>
#if defined(ORDERED_MAP)
template<typename K, typename V>
using htable = std::map<K, V>;
template <typename K, typename V>
static inline bool check_insert(htable<K, V>& t, K k, V v) {
return t.emplace(std::move(k), std::move(v)).second;
}
#elif defined(UNORDERED_MAP)
template<typename K, typename V>
using htable = std::unordered_map<K, V, boost::hash<K> >;
template <typename K, typename V>
static inline bool check_insert(htable<K, V>& t, K k, V v) {
return t.emplace(std::move(k), std::move(v)).second;
}
#elif defined(INPUT_PRESERVING) // retain exact input order
#include <boost/multi_index_container.hpp>
#include <boost/multi_index/sequenced_index.hpp>
#include <boost/multi_index/hashed_index.hpp>
#include <boost/multi_index/member.hpp>
namespace bmi = boost::multi_index;
template<typename K, typename V, typename P = std::pair<K, V> >
using htable = bmi::multi_index_container<
P,
bmi::indexed_by<
bmi::sequenced<>,
bmi::hashed_unique<bmi::member<P, K, &P::first>, boost::hash<K> >
>
>;
template <typename K, typename V>
static inline bool check_insert(htable<K, V>& t, K k, V v) {
return t.insert(t.end(), std::make_pair(std::move(k), std::move(v))).second;
}
#endif
using NodeMap = htable<Hand, htable<InfosetHistory, Node>>;
NodeMap readTextStrategy(std::string const& fname);
NodeMap readBinaryStrategy(std::string const& fname);
void writeTextStrategy(std::string const& fname, NodeMap const& nodeMap);
void writeBinaryStrategy(std::string const& fname, NodeMap const& nodeMap);
int main() {
auto const original = readTextStrategy("StrategyWritten0.txt");
NodeMap bin = original, txt;
for (int i = 1; i<5; ++i) {
auto const fname = "roundtrip" + std::to_string(i);
writeBinaryStrategy(fname + ".bin", bin);
writeTextStrategy(fname + ".txt", bin);
bin = readBinaryStrategy(fname + ".bin");
txt = readTextStrategy(fname + ".txt");
std::cout << "text roundtrip " << i << " is " << (txt == original?"equal":"different") << "\n";
std::cout << "bin roundtrip " << i << " is " << (bin == original?"equal":"different") << "\n";
}
}
void writeBinaryStrategy(std::string const& fname, NodeMap const& nodeMap) {
std::ofstream f(fname, std::ios::binary);
boost::archive::binary_oarchive archive(f);
archive << nodeMap;
}
NodeMap readBinaryStrategy(std::string const& fname) {
NodeMap nodeMap;
std::ifstream ifs(fname, std::ios::binary);
boost::archive::binary_iarchive ia(ifs);
ia >> nodeMap;
return nodeMap;
}
#include <iomanip>
#include <boost/lexical_cast.hpp> // full precision see https://stackoverflow.com/a/48085309/85371
namespace TextSerialization {
#if defined(__MINGW32__) || defined(WIN32)
static constexpr char const* CRLF = "\n";
#else
static constexpr char const* CRLF = "\r\n";
#endif
static inline std::ostream& operator<<(std::ostream& os, Card const& c) {
return os << c.rank << CRLF << c.suit;
}
static inline std::ostream& operator<<(std::ostream& os, DrawAction const& da) {
return os << da.fromDeck << CRLF << da.card;
}
template <typename T, size_t N>
static inline std::ostream& operator<<(std::ostream& os, std::array<T, N> const& from) {
auto n = N;
for (auto& el : from)
os << el << (--n?CRLF:"");
return os;
}
template <typename... T>
static inline std::ostream& operator<<(std::ostream& os, std::vector<T...> const& from) {
os << from.size();
for (auto& el : from)
os << CRLF << el;
return os;
}
template <typename K, typename V>
static inline std::ostream& operator<<(std::ostream& os, htable<K, V> const& from) {
auto n = from.size();
os << n;
for (auto& [k,v] : from)
os << CRLF << k << CRLF << v;
return os;
}
static inline std::ostream& operator<<(std::ostream& os, InfosetHistory const& ish) {
return os
<< ish.initialHand << CRLF << ish.initialDiscardPile << CRLF
<< ish.playerDrawActions << CRLF << ish.playerDiscardActions << CRLF
<< ish.opponentDrawActions << CRLF << ish.opponentDiscardActions;
}
static inline std::ostream& operator<<(std::ostream& os, Node const& n) {
assert(n.NUM_ACTIONS == n.regretSum.size());
assert(n.NUM_ACTIONS == n.strategySum.size());
os << n.allowedActions << CRLF
<< n.NUM_ACTIONS << CRLF;
for (auto& v: {n.regretSum, n.strategySum})
for (auto& el: v)
os << boost::lexical_cast<std::string>(el) << CRLF;
return os << n.phase;
}
}
namespace TextDeserialization {
template <typename Cont>
static inline void read_n(std::istream& is, size_t n, Cont& into) {
while (n--)
is >> *into.emplace(into.end());
}
static inline std::istream& operator>>(std::istream& is, Card& c) {
return is >> c.rank >> c.suit;
}
static inline std::istream& operator>>(std::istream& is, DrawAction& da) {
return is >> da.fromDeck >> da.card;
}
template <typename T, size_t N>
static inline std::istream& operator>>(std::istream& is, std::array<T, N>& into) {
for (auto& el : into)
is >> el;
return is;
}
template <typename... T>
static inline std::istream& operator>>(std::istream& is, std::vector<T...>& into) {
size_t n;
is >> n;
read_n(is, n, into);
return is;
}
template <typename K, typename V>
static inline std::istream& operator>>(std::istream& is, htable<K, V>& into) {
size_t n;
is >> n;
K k; V v;
while (n--) {
if (is >> k >> v && !check_insert(into, std::move(k), std::move(v)))
throw std::range_error("duplicate key");
}
return is;
}
static inline std::istream& operator>>(std::istream& is, InfosetHistory& ish) {
return is
>> ish.initialHand >> ish.initialDiscardPile
>> ish.playerDrawActions >> ish.playerDiscardActions
>> ish.opponentDrawActions >> ish.opponentDiscardActions;
}
static inline std::istream& operator>>(std::istream& is, Node& n) {
is >> n.allowedActions;
is >> n.NUM_ACTIONS;
read_n(is, n.NUM_ACTIONS, n.regretSum);
read_n(is, n.NUM_ACTIONS, n.strategySum);
return is >> n.phase;
}
}
void writeTextStrategy(std::string const& fname, NodeMap const& nodeMap) {
using namespace TextSerialization;
std::ofstream os(fname);
os << nodeMap << CRLF;
}
NodeMap readTextStrategy(std::string const& fname) {
using namespace TextDeserialization;
std::ifstream is(fname);
NodeMap nodeMap;
is >> nodeMap;
return nodeMap;
}
CMakeLists.txtPROJECT(work)
CMAKE_MINIMUM_REQUIRED(VERSION 3.16)
SET(CMAKE_CXX_STANDARD 17)
SET(CMAKE_CXX_FLAGS "-g -O0 -L .")
LINK_DIRECTORIES("C:\\boost\\lib")
LINK_LIBRARIES(boost_serialization-mgw8-mt-d-x64-1_73)
INCLUDE_DIRECTORIES("C:\\boost\\include\\boost-1_73")
ADD_EXECUTABLE(ordered test.cpp)
TARGET_COMPILE_DEFINITIONS(ordered PRIVATE "-DORDERED_MAP")
ADD_EXECUTABLE(unordered test.cpp)
TARGET_COMPILE_DEFINITIONS(unordered PRIVATE "-DUNORDERED_MAP")
ADD_EXECUTABLE(multi-index test.cpp)
TARGET_COMPILE_DEFINITIONS(multi-index PRIVATE "-DINPUT_PRESERVING")
建立:
mkdir -p build && (cd build; cmake ..; cmake --build .)
在使用输入顺序保留数据结构(
multi-index和-DINPUT_PRESERVING)进行测试时:./build/multi-index.exe
md5sum roundtrip*.txt StrategyWritten0.txt
版画
text roundtrip 1 is equal
bin roundtrip 1 is equal
text roundtrip 2 is equal
bin roundtrip 2 is equal
text roundtrip 3 is equal
bin roundtrip 3 is equal
text roundtrip 4 is equal
bin roundtrip 4 is equal
d62dd8fe217595f2e069eabf54de479a *roundtrip1.txt
d62dd8fe217595f2e069eabf54de479a *roundtrip2.txt
d62dd8fe217595f2e069eabf54de479a *roundtrip3.txt
d62dd8fe217595f2e069eabf54de479a *roundtrip4.txt
d62dd8fe217595f2e069eabf54de479a *StrategyWritten0.txt
原始答案
这可能会导致存档不完整，这可能是您的问题。您在存档析构函数运行之前关闭文件。更好:
我按如下方式序列化nodeMap:
{
std::ofstream file("Strategy" + std::to_string(iteration) + ".bin");
boost::archive::binary_oarchive archive(file);
archive << nodeMap;
}
这样，存档的析构函数将在文件的析构函数之前运行(作为奖励，您不必手动关闭文件)。
(析构函数以相反的顺序运行)。
其他注意事项
关于c++ - 如何调试Boost Archive异常输入流错误，我们在Stack Overflow上找到一个类似的问题：https://stackoverflow.com/questions/62225730/