PLF C++ Library - plf::colony - MSVC benchmarks

Last updated 8-2-2017 v3.87

Test setup
Raw performance benchmarks (against standard containers)
Comparitive performance benchmarks (against modified standard containers)
Low-modification scenario test
High-modification scenario test
Overall performance conclusions

Test machine setup

The test setup is a Haswell-class Xeon (E3-1241) on an Intel motherboard, 8GB ram, running MSVC 2015 update 3 x64 as compiler. OS is a stripped-back Windows 7 x64 SP1 installation with most services, background tasks (including explorer) and all networking disabled. Build settings are "/O2 /Oi /Ot /Oy /GL". Lastly, because MSVC's deque implementation is not good, we've skipped the heavy-referencing tests for MSVC, and packed_deque in the comparitive tests (this's been replaced with std::list in case anyone's interested in the performance of that). In order to not duplicate effort and conclusions, commentary is left to the GCC results (with the exception of some small overall conclusions).

The source code for the benchmarks can be found in the colony page downloads section.

Raw performance tests

Insertion Performance

Click images or hover over to see results at linear scale instead

Erase Performance

Click images or hover over to see results at linear scale instead

Post-erasure Iteration Performance

Click images or hover over to see results at linear scale instead

Comparative performance tests

Insertion

Click images or hover over to see results at linear scale instead

Erasure

Click images or hover over to see results at linear scale instead

Post-erasure Iteration

Click images or hover over to see results at linear scale instead

'Real-world' scenario testing - low modification

Click images or hover over to see results at linear scale instead

Performance results

Memory results

'Real-world' scenario testing - high modification

Performance results

Click images or hover over to see results at linear scale instead

Memory results

Overall Performance Conclusions

For the most part MSVC's results are similar to GCC's, with the exception of deque, which performs poorly, and vector, which slightly outperform's GCC's vector for iteration and erase performance, but underperforms by comparison for insertion performance. The end result is much the same, although we cannot compare packed_deque results. std::list performs admirably under the modification tests, at least for less than 500 elements, but at higher numbers of elements it's performance becomes worse.

Using a boolean skipfield in combination with a vector or deque is of course a no-go, due to it's poor iteration performance once the number of erasures increases. Similarly, using an erasable iteration field as was used with indexed_vector and pointer_deque results in both wasteful memory consumption and poor performance once the number of modifications or elements becomes too high, but may be more-than-adequate at low-modification levels without too much differential in terms of memory usage.

In short, where both the rate of modification is >= 10% of all elements per 3600 frames and the number of elements is >= 9000, and where unordered, often-inserted/erased content is required, use a colony.