Kinds of parallelism

multiple data items vs multiple operations. Flynn categorizations looks at streams of data and instructions, leaving control flow alone... one can form four categories. Single or multiple instruction streams or single or multiple data streams. Ex: SISD (Von Neuman), MIMD (modern multiprocessors), SIMD (Vector computers), and MISD (maybe research but typically not useful)

interconnection network

usually denoted as a switch. Can be a serial bus or N^2 fully connected network. Concurrency of the system is directly related to concurrency of this network

conditions for parallelism

functional parallelism - computational units syncrhonize in specific ways or times that limit complexity... vectorization good for dense matrixes, sparse or irregularly patterned data is better suited to MIMD or possibly SISD if cache benefits cant be realized.