Incrementing value from multiple threads

Hey all, I was wondering about incrementing a value (an int) from different threads and its potential race conditions. More specifically, do I need to use atomic functions to ensure that no two threads attempt to increment this counter at the same time or do I even need to be worried about this?

Atomics will make sure that threads that simultaneously increment the value don’t step on each other. (e.g., thread one will not read 13 at the same time thread 2 reads 13 and they both write back 14, instead of 15 which is what you get with two increments.) Atomics will not make sure they don’t simultaneously increment the value. You do need to worry about that if your algorithm depends on the threads doing different things to different pieces of data.

As a follow-up, does anybody know roughly how expensive these atomic operations are in the underlying hardware? Are they relatively cheap (comparable to a regular add/increment/etc.), or expensive enough to only be used when absolutely necessary?

The cost depends on the hardware, unfortunately. My understanding is that in Nvidia’s Fermi they are “a lot cheaper” than what they were before, which was incredibly expensive. In general they should be several times slower than a non-atomic operation. You should be able to measure this quite easily, but my guess would be between 5 and 50 times slower.