The serial fraction (20%) caps your theoretical maximum speedup at 5.00x, no matter how many cores you add.
Parallel efficiency is only 41.7%. Most cores are idle waiting on the serial section. Reducing the serial fraction would help far more than adding cores.
At 8 cores you have used 67% of the theoretical maximum. Doubling cores gives diminishing gains.
Next stepTo close the gap to the theoretical limit, focus on reducing the serial fraction of the code rather than adding more processors.