There are some fundamental limitations to running parallel code irrespective of the architectureOne is Amdahl’s Law, which states that as you keep splitting your task into parallel threads, you can only go as fast as the slowest part of your task that has to be completed in order, no matter how many computations you can run concurrently The second law, the name of which I’ve forgotten, states that as you add more parallel computing units or threads for your task, the portion of your code that has to be completed in order begins to grow – that is, the task of managing and organizing the computation and collating the results takes more computing time When you take Amdahl’s law alone, you get diminishing returns, but every additional thread or computational unit still speeds things up somewhat When you take the whole bureaucracy of parallel computation into account, you get negative returns Trying to add too many computing units or threads on the job will actually turn out slower
2 Comments
There are some fundamental limitations to running parallel code irrespective of the architectureOne is Amdahl’s Law, which states that as you keep splitting your task into parallel threads, you can only go as fast as the slowest part of your task that has to be completed in order, no matter how many computations you can run concurrently The second law, the name of which I’ve forgotten, states that as you add more parallel computing units or threads for your task, the portion of your code that has to be completed in order begins to grow – that is, the task of managing and organizing the computation and collating the results takes more computing time When you take Amdahl’s law alone, you get diminishing returns, but every additional thread or computational unit still speeds things up somewhat When you take the whole bureaucracy of parallel computation into account, you get negative returns Trying to add too many computing units or threads on the job will actually turn out slower
Major thanks for the articleMuch thanks again Really Great
Leave A Comment