Contact
- Name: Andreas Ellison
- Email: aellison@student.ethz.ch
-
Discord: HumbleCatcher#1527
(I prefer this over email, don't hesitate to ask me questions π)
Resources
-
Documentation for
java.util.concurrent
- PVW script (not official)
Part 1
- Terminology
- Official Java tutorials on concurrency
-
Java
bytecode instruction set. This is only interesting if you want to inspect the bytecode of a
compiled Java program. This can be done with
javap -c path/to/MyClass.class
on the command line or online at https://godbolt.org/. - List of Java Thread states and their meaning
Part 2
-
Overall,
"The Art of Multiprocessor Programming"
is a very very good resource for this part of the course. It is very
well written and covers almost all topics of the lecture in depth.
- Chapter 2 (p. 21): Covers the mutual exclusion algorithms from the lecture, with proofs.
- Chapter 7, 7.1-7.4 (p. 141): Covers TAS, TATAS and exponential backoff locks.
- Chapter 8 (p. 177): Covers monitors, semaphores and the readers-writers lock, but a bit differently to the lecture.
- Chapter 9 (p. 195): The content on lock granularity from the lecture is based on this chapter in the book and follows it very closely. The book is a nice reference here especially for implementations of the algorithms.
- Chapter 10, 10.5-10.6 (p. 230): Covers the implementation of a lock-free unbounded queue and the ABA problem, using the queue as an example.
- Chapter 3, 3.1-3.6 (p. 45): Covers sequential consistency and linearizability (and quiescent consistency, but this is not exam relevant). The proofs starting from 3.6.2 were not covered in the lecture (and are hence of course not exam relevant). Good to know: the book always lists the linearization points of code given later chapters.
- Chapter 5, excluding 5.1.1 and 5.5-7 (p. 99): Covers consensus. Proofs like for theorem 5.2.1 or 5.4.1 are not exam relevant.
- Chapter 18 (p. 417): Covers transactional memory. Parts of the chapter go a bit more in depth into implementation details that were not covered in the lecture.
-
Resources on the Java memory model:
- Short summary of happens-before from the Java documentation.
- Official paper explaining the memory model. It should be understandable. Alternatively, you can read the official specification, which is more compact but has less explanations. If you choose the specification, I can recommend reading up to and including section 17.4.5.
- A nice talk on the Java memory model.
- Chapter 16.1 (p. 337) of the book "Java Concurrency in Practice" has an explanation of the Java memory model.
- An introductory blog article on memory models in general.
-
Resources on MPI:
- The MPI standard.
-
If you want to understand the difference between
synchronous/asynchron or blocking/non-blocking better, I can
recommend sections 3.4 and 3.7 of the
standard
(there the "standard" communication mode for
send
is what we call asynchronous). - Documentation of the different MPI operations. The descriptions should be helpful, e.g. here is the one for MPI_Allgather.
- Blog articles on collective communication and scatter, gather and allgather.
FAQ
Exercise sessions
Week 2
- "Derivation" for why we need a scheduler, processes and threads
-
Part two, chapters 3 and 4 of
this
book are about processes and threads, if you want to read more about
the topic.
Note: Having a deep understanding of processes and threads in an OS is not exam relevant for this course. - Demo (Eclipse project) of summing up the values in an array with Java threads
- Quiz
Week 3
- Demo from class
- Quiz
-
Official (well written!)
Java tutorial
on synchronization (introduces the need for
synchronized
similar to the lecture) -
Documentation for
java.util.concurrent.atomic
- Java specification of wait and notify . It turns out that the internal list of waiting threads is called the wait set and not the wait list.
Week 4
- My notes on parallelism vs. concurrency and pipelining
- HS20 wait and notify task: code template and link to exam
- Plots shown in
class and the
code
used to generate them (first run the Java program, then run the
script
plot.py
). Please let me know if you find any mistakes in the code to generate the data!
The first few people to email me a solution to the following challenge before
- 22.03.2023 4 PM for the Wednesday group
- 24.03.2023 10 AM for the Friday group
get a bar of chocolate. π«
This challenge was successfully solved by
jusdai
and jpotempa
. Congrats!
the pipeline is balanced
βΊ
no stage is
longer than the first
Note: A pipeline is balanced if all iterations have the same latency. Solution: See notes for week 5.
Week 5
Week 6
- My notes on Amdahl's and Gustafson's laws. The explanation in the exercise session slides (on Moodle) is also good.
- Kahoot and short explanations for some of the pipelining questions.
The following resources are interesting if you're curious about the history behind Amdahl's and Gustafson's "laws" and why they exist.
Week 7
Please fill out the
feedback form!
-
Exercises we solved in class:
- Task graphs (FS21, Moodle link)
- Fork/Join (FS19)
- Kahoot
Some advanced resources, beyond the scope of the course:
- Paper on how the Fork/Join framework is implemented in Java on a high level
- Chapter 16 of this book is a much more advanced resource on task parallelism, if you want to have a peek at what more there is to the topic (e.g. it discusses the most relevant theoretical results and gives an implementation of a work-stealing scheduler in Java).
- A theoretical paper on an algorithm for work-stealing scheduling which was referenced in the lecture slides
Week 8
- The livelock example from class.
- Demo code of a CAS lock and the plots I showed in class. The
- The Java program with unexpected behavior that I showed at the end.
- There are links to some useful resources on the current topics at the top of the page (under "Part 2").
Week 9
- There are links to some useful resources on the current topics at the top of the page (under "Part 2").
- Proofs showing that the Filter lock satisfies mutual exclusion and starvation freedom can be found on page 30 of "The Art of Multiprocessor Programming". The same can be found for the bakery lock later on.
Week 10
Types of exercises that might come up in the exam
Disclaimer: This list is not guaranteed to be complete and is only
meant to give you an idea of what has been asked on previous exams.
Locks
Usually there are not too many question on this topic.- true/false questions of which lock has which properties (fairness, starvation free)
- find bug in lock code (violation of mutual exclusion or deadlock freedom)
- draw state space diagram and/or read off correctness properties
- reproduce Peterson/Filter/Bakery lock
- prove correctness of Peterson lock or similar (but not Filter or Bakery)
Monitors, semaphores, barriers
- semaphore implementation (mostly with monitors)
- (never seen rendezvous with semaphores in an exam)
- barrier implementation (mostly with monitors)
- (only seen a task on implementing a barrier with semaphores once in FS21, 8b)
- fill out some program using monitors (similar to wait/notify exercises, maybe with lock conditions)
What we did in class
-
Barriers:
- Code
for a non-reusable barrier using
wait/notify
. - Exercises on correctness of barrier implementations: FS19, 11a, HS14 3c (there was no time for this during the session)
-
Code for
first
(more recommended for exam) and
second
(more elegant, but less convenient for the exam because of the use
of
ThreadLocal
) implementation of a reentrant barrier usingwait/nofity
.
- Code
for a non-reusable barrier using
- Kahoot
Week 11
- Kahoot
- Up to date solution for why the Peterson lock works in Java.
- I can very much recommend the book chapter on "lock granularity" from "The Art of Multiprocessor Programming" (see resources above) for further reference on the topic. It has code for most of the operations, so it's especially useful if you're stuck with this week's assignment.
Week 12
- The explanation for race condition vs. bad interleaving vs. data race can be found in the new "FAQ" section above.
- Quiz with slides. Credit to HansjΓΆrg Schraubenzieher for the questions.
-
Little
derivation
of
enq
from the lock-free unbounded queue implementation of the lecture. (I didn't show this on Wednesday.)
Week 13
- For an example of how the ABA problem can happen with the lock-free unbounded queue, see chapter 10.6 of the book
- My notes on sequential consistency and linearizability. I am more than happy to answer questions about my explanations, e.g. if it is unclear how everything connects to the way it is presented in the lecture/book. Also the book covers the topic way back in chapter 3 (3.1 - 3.6) and if you want to read deeper into the topic, this is the paper that initially introduced the concept. (You may recognize one of the authors.)
Week 14
- I added an explanation of linearization points to the "FAQ" section above.
- Check the resources at the top for where to read more about consensus and transactional memory in the book and for some resources on MPI.
- Quiz (with explanations for the consensus questions)
Exam tips for consensus and MPI
Disclaimer: The following tips are only my thoughts as a student.
The lecture makes no such promises and according to them everything that
was covered in the lecture is exam relevant (unless stated otherwise).
-
Consensus:
- I have a strong feeling that the head TA wants to ask this in the exam. ;)
-
I can think of three types of questions on this topic: short
questions on the definitions, identifying whether given code is
a correct consensus protocol or making a statement about the
consensus number of an object. For the latter, I only see two
options:
-
Proving that an object
x
has consensus numberβ₯y(orβ). Then you must provide an algorithm solvingy-thread consensus using any number of instances ofx
(or for theβcase, provide an algorithm solvingn-thread consensus for arbitraryn). -
Proving that an object
x
has consensus number at mostz. This would involve proving that it is impossible to implement(z+1)-thread consensus withx
. With the content of the lecture, the only option I see is to use the theorems that were shown. For example, if we know thatx
was implemented using atomic registers, then it cannot solve consensus for more than one thread. Proofs that do not rely on such results require some concepts that were not covered in the lecture. (An example is the proof for why atomic registers cannot solve 2-thread consensus in "The Art of Multiprocessor Programming" (p. 103).)
-
Proving that an object
- MPI: Since the topic was only covered in short time in the lecture and no exercises were provided on the topic, I would not expect coding exercises with MPI in the exam (again, cannot promise). But that does not exclude reading and understanding MPI code. I would make sure to remember what the collective operations do, as this has been asked before.