Home

PLF C++ Library - plf::colony - MSVC benchmarks

Last updated 8-2-2017 v3.87

  1. Test setup
  2. Raw performance benchmarks (against standard containers)
  3. Comparitive performance benchmarks (against modified standard containers)
  4. Low-modification scenario test
  5. High-modification scenario test
  6. Overall performance conclusions

Test machine setup

The test setup is a Haswell-class Xeon (E3-1241) on an Intel motherboard, 16GB ECC ram, running MSVC 2015 update 3 x64 as compiler. OS is a stripped-back Windows 7 x64 SP1 installation with most services, background tasks (including explorer) and all networking disabled. Build settings are "/O2 /Oi /Ot /Oy /GL". The source code for the benchmarks can be found in the colony page downloads section, as can further details about the test designs. Lastly, because MSVC's deque implementation is not good, we've skipped the heavy-referencing tests for MSVC, and packed_deque in the comparitive tests (this's been replaced with std::list in case anyone's interested in the performance of that). In order to not duplicate effort and conclusions, commentary is left for the GCC results (with the exception of some small overall conclusions).

Raw performance tests

Insertion Performance

Click images or hover over to see results at linear scale instead

test result graph
test result graph
test result graph
test result graph
test result graph

Erase Performance

Click images or hover over to see results at linear scale instead

test result graph
test result graph
test result graph
test result graph
test result graph

Post-erasure Iteration Performance

Click images or hover over to see results at linear scale instead

test result graph
test result graph
test result graph
test result graph
test result graph

Comparative performance tests

Insertion

Click images or hover over to see results at linear scale instead

test result graph

Erasure

Click images or hover over to see results at linear scale instead

test result graph
test result graph
test result graph

Post-erasure Iteration

Click images or hover over to see results at linear scale instead

test result graph
test result graph
test result graph
test result graph

'Real-world' scenario testing - low modification

Click images or hover over to see results at linear scale instead

Performance results

test result graph
test result graph
test result graph

Memory results

test result graph
test result graph
test result graph

'Real-world' scenario testing - high modification

Performance results

Click images or hover over to see results at linear scale instead

test result graph
test result graph
test result graph

Memory results

test result graph
test result graph
test result graph

Overall Performance Conclusions

For the most part MSV's results are similar to GCC's, with the exception of deque, which performs poorly, and vector, which slightly outperform's GCC's vector for iteration and erase performance, but underperforms by comparison for insertion performance. The end result is much the same, although we cannot compare packed_deque results. std::list performs admirably under the modification tests, at least for less than 500 elements, but at higher numbers of elements it's performance becomes worse.

Using a boolean skipfield in combination with a vector or deque is of course a no-go, due to it's poor iteration performance once the number of erasures increases. Similarly, using an erasable iteration field as was used with indexed_vector and pointer_deque results in both wasteful memory consumption and poor performance once the number of modifications or elements becomes too high, but may be more-than-adequate at low-modification levels without too much differential in terms of memory usage.

In short, where both the rate of modification is >= 10% of all elements per 3600 frames and the number of elements is >= 9000, and where unordered, often-inserted/erased content is required, use a colony.

Contact: footer
plf:: library and this page Copyright (c) 2017, Matthew Bentley