Parallel algorithm

A parallel algorithm divides a computation among processing elements that execute concurrently, coordinating their work to produce a result.

Task parallelism: A form of parallelism in which distinct tasks execute concurrently, often on different data or functions. It describes algorithms that divide work by assigning different operations to concurrent workers.
Shared-memory multiprocessing: A computing architecture in which multiple processors access a common address space. Algorithms on this architecture can coordinate through shared data rather than explicit messages.
Parallel sorting: The design of sorting algorithms that use multiple processors to order data concurrently. Sorting exposes the tradeoffs among parallel work, communication, and synchronization.
Sequential algorithm: An algorithm whose operations follow a single ordered sequence of execution. It provides the baseline for assessing the speedup and overhead of parallel execution.
Data parallelism: A form of parallelism in which the same operation is applied simultaneously to multiple data elements. It is a common way parallel algorithms partition large datasets.
Distributed-memory: A computing architecture in which each processor has private memory and communicates with others by exchanging data. It determines how an algorithm partitions data and pays for communication.
Parallel prefix sum: A parallel computation of cumulative results for every prefix of an input sequence. It is a reusable primitive for building many parallel algorithms.
Concurrent algorithm: An algorithm structured around multiple tasks whose executions can overlap, whether or not they run simultaneously. Concurrency allows overlapping progress, while parallelism specifically requires simultaneous execution.
Parallel computing: The use of multiple computing resources to perform parts of a computation at the same time. Parallel algorithms are the software methods used within this broader computing approach.
Flynn's taxonomy: A classification of computer architectures by the number of instruction and data streams they process. Its categories distinguish architectures that support different styles of parallel execution.

Linked from 5 pages