1. komponenta

Lecture type	Total
Lectures	30
Exercises	15

* Load is given in academic hour (1 academic hour = 45 minutes)

COURSE AIMS AND OBJECTIVES: In the initial period of the development of computer technology, each new generation of computers was significantly faster than the previous one. This trend ended around 2005, and since then increasing the efficiency of programs has been possible mainly by using parallel execution on multiple processors, i.e. processor cores. Thus, parallel programming becomes a central topic of all computing curricula. Parallelism appears in a computer system at several levels: within each individual microprocessor core (vector operations), within a multi-core processor, due to the presence of accelerators in the system (for example, graphics cards) and finally in computer clusters. Each type of parallelism within the circuit corresponds to a specific type of development of parallel programs and corresponding support libraries. The objective of this course is to familiarize students with the basic models of parallel code development and related software tools. Students need to master vector code generation, parallel programming with threads, parallel programming with message passing, and programming of heterogeneous systems that (today) include a central processing unit and a graphics card. The software tools that will be used in the area of vector, multi-threaded and multi-processor programming are quite stable (OpenMP, C++ programming threads, MPI and the like), which is not the case with programming on graphics cards and other accelerators. There we have the possibility of using CUDA, HIP or OpenCL programming languages and OpenMP, OpenACC, SYCL programming interfaces, as well as programming models at a higher level of abstraction, such as Kokkos and Raja. The objectives of the course in this part will change depending on the positioning of these tools in the market.

COURSE DESCRIPTION AND SYLLABUS:
1. Vector programming for superscalar processors. Autovectorization through the compiler, use of intrinsic commands and special program libraries.
2. Parallel programming with shared memory. Use of OpenMP directives and C++ system of program threads/POSIX threads in C language. Synchronization of program threads, implementation of shared data structures and parallel algorithms. Memory model of OpenMP system and programming language C++. Atomic operations and lock-free programming.
3. Parallel programming with distributed memory. Application of MPI system, individual and collective operations, parallel input and output, topologies.
4. Programming of parallel algorithms on graphics cards (and possibly other accelerators). Programming using directives (OpenAcc and OpenMP) and application of CUDA/OpenCL/SYCL programming languages/environments.

1. Parallel and High Performance Computing, R.Robey, Y. Zamora, Manning, 2021.
2. Using MPI, treće izdanje, W. Gropp, E. Lusk, A. Skjellum, MIT, 2014.
3. C++ Concurrency in Action, drugo izdanje, A. Williams, Manning, 2019.
4. The Art of Writing Efficient Programs, F.G. Pikus, Packt, 2021.
5. Parallel Programming with MPI, P.S. Pacheco, Morgan Kaufmann Publishers, 1997.
6. Using OpenMP, B. Chapman, G. Jost, R. Van der Pas, MIT, 2008.
7. Programming Massively Parallel Processors, A Hands-on Approach, treće izdanje, D.B. Kirk, Wen-mei W. Hwu, Morgan Kaufman, 2017.
8. OpenCL in Action, M. Scarpino, Manning, 2012.
9. Dokumentacija biblioteke NVIDIA CUDA, https://docs.nvidia.com/cuda/.
10. Specifikacija programskog sučelja OpenACC, https://www.openacc.org.
11. Specifikacija programskog sučelja OpenMP, https://www.openmp.org.
12. Dokumentacija C++ programskog modela Kokkos, https://github.com/kokkos/kokkos-tutorials.
13. Dokumentacija softverske biblioteke RAJA, https://raja.readthedocs.io/en/develop/.

Izborni modul D - Znanstveno računanje, 1. godina - Regular study - Computer Science and Mathematics
Ostali izborni predmeti - Regular study - Computer Science and Mathematics

Izborni modul D - Znanstveno računanje, 1. godina - Regular study - Computer Science and Mathematics
Ostali izborni predmeti - Regular study - Computer Science and Mathematics