Yes, it's feasible for single-thread performance to lag behind multi-threaded execution.
Yes, it's feasible for single-thread performance to lag behind multi-threaded execution.
I'm a student who has written code to run several copies of an engineering simulation at the same time using multiple processes. I consistently see faster completion times when running more simulations concurrently. I expect the average time to actually decrease as I increase concurrency, but for 1 to 10 it seems to improve. The simulations are very similar in inputs and outputs. My system is an older Intel Xeon server with two Xeon x5670 processors (6 cores each) and hyper-threading, giving a total of 24 threads. I'm not interested in the software details; I've verified the basics and just want to know if hardware could explain this behavior.
Edited February 5, 2022 by floopygiraffe
I think scaling between two Numa nodes becomes challenging because RAM allocated to one CPU is also used by the other. This explains the declining performance you noticed around 12 threads. In fact, I experienced similar issues using comparable hardware in a SQL database environment where threads were distributed across two nodes. With help from @leadeater, I was able to improve performance by reducing the number of threads assigned to the VM.
You're working with two CPUs, which is important. Without a clear understanding of what the software is doing, pinpointing the problem becomes difficult. Once you surpass a certain thread count, you're likely encountering core-to-core delays managed by the north bridge chipset. Secondly, past 12 threads you enter HyperThreading space. Although HT was useful, it wasn't as polished in 2010. It worked well for gaming tests but sometimes hurt productivity tests, especially with Intel's recent improvements. Lastly, heavy memory usage can lead to running out of RAM and switching to the page file, which severely impacts speed. Overall, more details would help clarify the situation.
Additional specifics show the simulation includes 5 to 6 phases, each mainly made up of single-threaded tasks. Your feedback raises concerns about whether these phases are distributing workload across different cores, potentially dealing with core-to-core delays. Would it be more likely for a process to switch to another CPU when many threads are active? It’s unclear what might be causing this in the software, but I suspect there could be a hardware-related reason. I’m not sure if this is significant, though—I’m hoping there’s a clear explanation on the side of the hardware. The software runs Python and C on Ubuntu.
@floopygiraffe Are we dealing with a baremetal OS setup? Meaning no hypervisor or virtual machine layer above? I’d suggest using numactl and directing each process to a distinct NUMA node, switching between them as needed. numactl -N 0 -m 0 [run process] [process arguments] https://cvw.cac.cornell.edu/hybrid/numactl http://www.hpc.acad.bg/numactl/ This approach helps manage memory access patterns.
If you’re seeing single-threaded execution, it’s likely intentional—each simulation runs independently without parallel threads. You should verify whether your setup truly supports concurrency or if the environment is intentionally limited.
Two key considerations stand out: CPU power and hyperthreading. The amount of power each core consumes influences its frequency per core, while hyperthreading allows one thread to share resources with another. Memory bandwidth also plays a role; running many processes can strain bandwidth, potentially lowering performance beyond what the hardware can handle.
In practice, enabling Hyperthreading is beneficial only if your workloads can leverage multiple threads without bottlenecks. However, excessive threading can cause contention and reduce efficiency. CPU package power matters too—each core needs sufficient energy to maintain its target frequency. If you run many processes, the system may hit limits, causing slower per-thread performance.
Another point is that disabling Hyperthreading (HT/SMT) in BIOS and running single threads per core is often recommended for HPC applications. This avoids unnecessary thread contention and helps maintain stability.
Ultimately, the ideal configuration depends on your specific workload. Testing with numactl to control process placement can help you identify the best balance. Keep an eye on execution times across different group sizes—performance may peak at around 10 processes before dropping off. Your intuition is correct; understanding these factors will guide you toward better results.
Core to core, many components are locked in place. Some delays in unblocking cores exist. Ideally, a design should support more than one thread per instance. (My view is that 16-core CPUs are typical for modern gaming setups.) We must consider how Ubuntu manages processes and threads. Is the testing software tuned for the OS or vice versa? Would it be better to use Windows Server or standard desktop apps like Windows 11, which handle multi-threading more effectively than older systems such as W7 that struggle with core parking? I have several ideas on how to evaluate performance per single thread. But when you go beyond one thread or instance, you’re essentially multitasking—measuring how long it takes to run many instances of the testing software using different methods to cut execution time. It’s clear that as resources increase (like loading multiple cores), performance per core tends to drop unless you adjust accordingly. This is why processors have a maximum advertised single-core boost frequency for efficiency. To boost throughput, more processes can run, but if the frequency drops significantly, individual core speed decreases. My main question remains: how to accurately assess single-core performance?
Thank you to all who replied! I think this has a lot to do with what I'm seeing, and made me think to log core utilization and core clock during the simulation. The simulation process jumps around between 2 or 3 cores quite a few times before returning results. There's a loop in the simulation that something like "gather initial conditions from engineering model - solve for dispatch solution - send dispatch solution to engineering model - update intial conditions" and repeat until the time horizon is complete. The "solve" and "update" activities are intensive tasks, but the "gather" and send" activities are not. I think the less intensive activities are giving the CPU enough time to return to a lower clock speed before having to ramp back up when they get one of the more intensive activities. Looking at a plot of core clock, it is bouncing up and down quite a bit but averages pretty low when running 1 simulation concurrently. But if you run say 10 concurrently core clock is still all over the place but averages much higher. So while I think the processes are still moving around on the cores, whatever core its landing on is already clocked pretty high and so there's a time saving. Anyway this is a good enough explanation for me, and all of the replies got me there. Thank you!