Saving and loading packed_trie_map¶
There are times when you need to save the state of a packed_trie_map
instance to a file, or an in-memory buffer, and load it back later. Doing that
is now possible by using the save_state() and
load_state() member methods of the
packed_trie_map class.
First, let’s define the type of use:
using map_type = mdds::packed_trie_map<std::string, int>;
As with the previous examples, we will use std::string as the key type and
int as the value type. In this example, we are going to use the world’s
largest cities and their 2018 populations as the data to store
in the container.
The following code defines the entries:
std::vector<map_type::entry> entries =
{
{ MDDS_ASCII("Ahmedabad"), 7681000 },
{ MDDS_ASCII("Alexandria"), 5086000 },
{ MDDS_ASCII("Atlanta"), 5572000 },
{ MDDS_ASCII("Baghdad"), 6812000 },
{ MDDS_ASCII("Bangalore"), 11440000 },
{ MDDS_ASCII("Bangkok"), 10156000 },
{ MDDS_ASCII("Barcelona"), 5494000 },
{ MDDS_ASCII("Beijing"), 19618000 },
{ MDDS_ASCII("Belo Horizonte"), 5972000 },
{ MDDS_ASCII("Bogota"), 10574000 },
{ MDDS_ASCII("Buenos Aires"), 14967000 },
{ MDDS_ASCII("Cairo"), 20076000 },
{ MDDS_ASCII("Chengdu"), 8813000 },
{ MDDS_ASCII("Chennai"), 10456000 },
{ MDDS_ASCII("Chicago"), 8864000 },
{ MDDS_ASCII("Chongqing"), 14838000 },
{ MDDS_ASCII("Dalian"), 5300000 },
{ MDDS_ASCII("Dallas"), 6099000 },
{ MDDS_ASCII("Dar es Salaam"), 6048000 },
{ MDDS_ASCII("Delhi"), 28514000 },
{ MDDS_ASCII("Dhaka"), 19578000 },
{ MDDS_ASCII("Dongguan"), 7360000 },
{ MDDS_ASCII("Foshan"), 7236000 },
{ MDDS_ASCII("Fukuoka"), 5551000 },
{ MDDS_ASCII("Guadalajara"), 5023000 },
{ MDDS_ASCII("Guangzhou"), 12638000 },
{ MDDS_ASCII("Hangzhou"), 7236000 },
{ MDDS_ASCII("Harbin"), 6115000 },
{ MDDS_ASCII("Ho Chi Minh City"), 8145000 },
{ MDDS_ASCII("Hong Kong"), 7429000 },
{ MDDS_ASCII("Houston"), 6115000 },
{ MDDS_ASCII("Hyderabad"), 9482000 },
{ MDDS_ASCII("Istanbul"), 14751000 },
{ MDDS_ASCII("Jakarta"), 10517000 },
{ MDDS_ASCII("Jinan"), 5052000 },
{ MDDS_ASCII("Johannesburg"), 5486000 },
{ MDDS_ASCII("Karachi"), 15400000 },
{ MDDS_ASCII("Khartoum"), 5534000 },
{ MDDS_ASCII("Kinshasa"), 13171000 },
{ MDDS_ASCII("Kolkata"), 14681000 },
{ MDDS_ASCII("Kuala Lumpur"), 7564000 },
{ MDDS_ASCII("Lagos"), 13463000 },
{ MDDS_ASCII("Lahore"), 11738000 },
{ MDDS_ASCII("Lima"), 10391000 },
{ MDDS_ASCII("London"), 9046000 },
{ MDDS_ASCII("Los Angeles"), 12458000 },
{ MDDS_ASCII("Luanda"), 7774000 },
{ MDDS_ASCII("Madrid"), 6497000 },
{ MDDS_ASCII("Manila"), 13482000 },
{ MDDS_ASCII("Mexico City"), 21581000 },
{ MDDS_ASCII("Miami"), 6036000 },
{ MDDS_ASCII("Moscow"), 12410000 },
{ MDDS_ASCII("Mumbai"), 19980000 },
{ MDDS_ASCII("Nagoya"), 9507000 },
{ MDDS_ASCII("Nanjing"), 8245000 },
{ MDDS_ASCII("New York City"), 18819000 },
{ MDDS_ASCII("Osaka"), 19281000 },
{ MDDS_ASCII("Paris"), 10901000 },
{ MDDS_ASCII("Philadelphia"), 5695000 },
{ MDDS_ASCII("Pune"), 6276000 },
{ MDDS_ASCII("Qingdao"), 5381000 },
{ MDDS_ASCII("Rio de Janeiro"), 13293000 },
{ MDDS_ASCII("Riyadh"), 6907000 },
{ MDDS_ASCII("Saint Petersburg"), 5383000 },
{ MDDS_ASCII("Santiago"), 6680000 },
{ MDDS_ASCII("Sao Paulo"), 21650000 },
{ MDDS_ASCII("Seoul"), 9963000 },
{ MDDS_ASCII("Shanghai"), 25582000 },
{ MDDS_ASCII("Shenyang"), 6921000 },
{ MDDS_ASCII("Shenzhen"), 11908000 },
{ MDDS_ASCII("Singapore"), 5792000 },
{ MDDS_ASCII("Surat"), 6564000 },
{ MDDS_ASCII("Suzhou"), 6339000 },
{ MDDS_ASCII("Tehran"), 8896000 },
{ MDDS_ASCII("Tianjin"), 13215000 },
{ MDDS_ASCII("Tokyo"), 37400068 },
{ MDDS_ASCII("Toronto"), 6082000 },
{ MDDS_ASCII("Washington, D.C."), 5207000 },
{ MDDS_ASCII("Wuhan"), 8176000 },
{ MDDS_ASCII("Xi'an"), 7444000 },
{ MDDS_ASCII("Yangon"), 5157000 },
};
It’s a bit long as it contains entries for 81 cities. We are then going to
create an instance of the packed_trie_map class directly:
map_type cities(entries.data(), entries.size());
Let’s print the size of the container to make sure the container has been successfully populated:
std::cout << "Number of cities: " << cities.size() << std::endl;
You will see the following output:
Number of cities: 81
if the container has been successfully populated. Now, let’s run a prefix search on names beginning with an ‘S’:
std::cout << "Cities that begin with 'S':" << std::endl;
auto results = cities.prefix_search("S");
for (const auto& city : results)
std::cout << " * " << city.first << ": " << city.second << std::endl;
to make sure you get the following ten cities and their populations as the output:
Cities that begin with 'S':
* Saint Petersburg: 5383000
* Santiago: 6680000
* Sao Paulo: 21650000
* Seoul: 9963000
* Shanghai: 25582000
* Shenyang: 6921000
* Shenzhen: 11908000
* Singapore: 5792000
* Surat: 6564000
* Suzhou: 6339000
So far so good. Next, we will use the save_state()
method to dump the internal state of this container to a file named cities.bin:
std::ofstream outfile("cities.bin", std::ios::binary);
cities.save_state(outfile);
This will create a file named cities.bin which contains a binary blob
representing the content of this container in the current working directory.
Run the ls -l cities.bin command to make sure the file has been created:
-rw-r--r-- 1 kohei kohei 17713 Jun 20 12:49 cities.bin
Now that the state of the container has been fully serialized to a file, let’s
work on restoring its content in another, brand-new instance of
packed_trie_map.
map_type cities_loaded;
std::ifstream infile("cities.bin", std::ios::binary);
cities_loaded.load_state(infile);
Here, we used the load_state() method to
restore the state from the file we have previously created. Let’s make sure
that this new instance has content equivalent to that of the original:
std::cout << "Equal to the original? " << std::boolalpha << (cities == cities_loaded) << std::endl;
If you see the following output:
Equal to the original? true
then this new instance has equivalent contant as the original one. Let’s also make sure that it contains the same number of entries as the original:
std::cout << "Number of cities: " << cities_loaded.size() << std::endl;
Hopefully you will see the following output:
Number of cities: 81
Lastly, let’s run on this new instance the same prefix search we did on the original instance, to make sure we still get the same results:
std::cout << "Cities that begin with 'S':" << std::endl;
auto results = cities_loaded.prefix_search("S");
for (const auto& city : results)
std::cout << " * " << city.first << ": " << city.second << std::endl;
You should see the following output:
Cities that begin with 'S':
* Saint Petersburg: 5383000
* Santiago: 6680000
* Sao Paulo: 21650000
* Seoul: 9963000
* Shanghai: 25582000
* Shenyang: 6921000
* Shenzhen: 11908000
* Singapore: 5792000
* Surat: 6564000
* Suzhou: 6339000
which is the same output we saw in the first prefix search.
The complete source code for this example is found here.