Multi-Threading

    By default, Julia starts up with a single thread of execution. This can be verified by using the command Threads.nthreads():

    The number of execution threads is controlled either by using the /--threads command line argument or by using the environment variable. When both are specified, then -t/--threads takes precedence.

    The number of threads can either be specified as an integer (--threads=4) or as auto (--threads=auto), where auto sets the number of threads to the number of local CPU threads.

    Julia 1.5

    The -t/--threads command line argument requires at least Julia 1.5. In older versions you must use the environment variable instead.

    Julia 1.7

    Using auto as value of the environment variable JULIA_NUM_THREADS requires at least Julia 1.7. In older versions, this value is ignored.

    Lets start Julia with 4 threads:

    1. $ julia --threads 4

    Let’s verify there are 4 threads at our disposal.

    1. julia> Threads.nthreads()
    2. 4

    But we are currently on the master thread. To check, we use the function Threads.threadid

    1. julia> Threads.threadid()
    2. 1

    Note

    If you prefer to use the environment variable you can set it as follows in Bash (Linux/macOS):

    1. export JULIA_NUM_THREADS=4

    C shell on Linux/macOS, CMD on Windows:

    Powershell on Windows:

    1. $env:JULIA_NUM_THREADS=4

    Note

    The number of threads specified with -t/--threads is propagated to worker processes that are spawned using the -p/--procs or --machine-file command line options. For example, julia -p2 -t2 spawns 1 main process with 2 worker processes, and all three processes have 2 threads enabled. For more fine grained control over worker threads use and pass -t/--threads as exeflags.

    You are entirely responsible for ensuring that your program is data-race free, and nothing promised here can be assumed if you do not observe that requirement. The observed results may be highly unintuitive.

    The best way to ensure this is to acquire a lock around any access to data that can be observed from multiple threads. For example, in most cases you should use the following code pattern:

    1. julia> lock(lk) do
    2.            use(a)
    3.        end
    5. julia> begin
    6.            lock(lk)
    7.            try
    8.                use(a)
    9.            finally
    10.                unlock(lk)
    11.            end
    12.        end

    where lk is a lock (e.g. ReentrantLock()) and a data.

    Additionally, Julia is not memory safe in the presence of a data race. Be very careful about reading any data if another thread might write to it! Instead, always use the lock pattern above when changing data (such as assigning to a global or closure variable) accessed by other threads.

    1. Thread 1:
    2. global b = false
    3. global a = rand()
    4. global b = true
    6. Thread 2:
    7. while !b; end
    8. bad_read1(a) # it is NOT safe to access `a` here!
    10. Thread 3:
    11. while !@isdefined(a); end
    12. bad_read2(a) # it is NOT safe to access `a` here

    Let’s work a simple example using our native threads. Let us create an array of zeros:

    1. julia> a = zeros(10)
    2. 10-element Vector{Float64}:
    3.  0.0
    5.  0.0
    6.  0.0
    7.  0.0
    8.  0.0
    9.  0.0
    10.  0.0
    11.  0.0

    Let us operate on this array simultaneously using 4 threads. We’ll have each thread write its thread ID into each location.

    Julia supports parallel loops using the Threads.@threads macro. This macro is affixed in front of a for loop to indicate to Julia that the loop is a multi-threaded region:

    The iteration space is split among the threads, after which each thread writes its thread ID to its assigned locations:

    1. julia> a
    2. 10-element Vector{Float64}:
    3.  1.0
    4.  1.0
    5.  1.0
    6.  2.0
    7.  2.0
    8.  2.0
    9.  3.0
    10.  3.0
    11.  4.0
    12.  4.0

    Note that does not have an optional reduction parameter like @distributed.

    Julia supports accessing and modifying values atomically, that is, in a thread-safe way to avoid race conditions. A value (which must be of a primitive type) can be wrapped as to indicate it must be accessed in this way. Here we can see an example:

    1. julia> i = Threads.Atomic{Int}(0);
    3. julia> ids = zeros(4);
    5. julia> old_is = zeros(4);
    7. julia> Threads.@threads for id in 1:4
    8.            old_is[id] = Threads.atomic_add!(i, id)
    9.            ids[id] = id
    10.        end
    12. julia> old_is
    13. 4-element Vector{Float64}:
    14.  0.0
    15.  1.0
    16.  7.0
    17.  3.0
    19. julia> i[]
    20.  10
    22. julia> ids
    23. 4-element Vector{Float64}:
    24.  1.0
    25.  2.0
    26.  3.0
    27.  4.0

    Had we tried to do the addition without the atomic tag, we might have gotten the wrong answer due to a race condition. An example of what would happen if we didn’t avoid the race:

    1. julia> using Base.Threads
    3. 4
    5. julia> acc = Ref(0)
    6. Base.RefValue{Int64}(0)
    8. julia> @threads for i in 1:1000
    10.        end
    12. julia> acc[]
    13. 926
    15. julia> acc = Atomic{Int64}(0)
    16. Atomic{Int64}(0)
    18. julia> @threads for i in 1:1000
    19.           atomic_add!(acc, 1)
    20.        end
    22. julia> acc[]
    23. 1000

    We can also use atomics on a more granular level using the @atomic, , and @atomicreplace macros.

    Any field in a struct declaration can be decorated with @atomic, and then any write must be marked with @atomic also, and must use one of the defined atomic orderings (:monotonic, :acquire, :release, :acquire_release, or :sequentially_consistent). Any read of an atomic field can also be annotated with an atomic ordering constraint, or will be done with monotonic (relaxed) ordering if unspecified.

    Julia 1.7

    Per-field atomics requires at least Julia 1.7.

    Side effects and mutable function arguments

    When using multi-threading we have to be careful when using functions that are not as we might get a wrong answer. For instance functions that have a name ending with ! by convention modify their arguments and thus are not pure.

    External libraries, such as those called via ccall, pose a problem for Julia’s task-based I/O mechanism. If a C library performs a blocking operation, that prevents the Julia scheduler from executing any other tasks until the call returns. (Exceptions are calls into custom C code that call back into Julia, which may then yield, or C code that calls jl_yield(), the C equivalent of .)

    The @threadcall macro provides a way to avoid stalling execution in such a scenario. It schedules a C function for execution in a separate thread. A threadpool with a default size of 4 is used for this. The size of the threadpool is controlled via environment variable UV_THREADPOOL_SIZE. While waiting for a free thread, and during function execution once a thread is available, the requesting task (on the main Julia event loop) yields to other tasks. Note that @threadcall does not return until the execution is complete. From a user point of view, it is therefore a blocking call like other Julia APIs.

    It is very important that the called function does not call back into Julia, as it will segfault.

    @threadcall may be removed/changed in future versions of Julia.

    Caveats

    At this time, most operations in the Julia runtime and standard libraries can be used in a thread-safe manner, if the user code is data-race free. However, in some areas work on stabilizing thread support is ongoing. Multi-threaded programming has many inherent difficulties, and if a program using threads exhibits unusual or undesirable behavior (e.g. crashes or mysterious results), thread interactions should typically be suspected first.

    There are a few specific limitations and warnings to be aware of when using threads in Julia:

    • Base collection types require manual locking if used simultaneously by multiple threads where at least one thread modifies the collection (common examples include push! on arrays, or inserting items into a Dict).
    • The schedule used by @spawn is nondeterministic and should not be relied on.
    • Compute-bound, non-memory-allocating tasks can prevent garbage collection from running in other threads that are allocating memory. In these cases it may be necessary to insert a manual call to GC.safepoint() to allow GC to run. This limitation will be removed in the future.
    • Avoid running top-level operations, e.g. include, or eval of type, method, and module definitions in parallel.
    • Be aware that finalizers registered by a library may break if threads are enabled. This may require some transitional work across the ecosystem before threading can be widely adopted with confidence. See the next section for further details.

    Safe use of Finalizers

    Because finalizers can interrupt any code, they must be very careful in how they interact with any global state. Unfortunately, the main reason that finalizers are used is to update global state (a pure function is generally rather pointless as a finalizer). This leads us to a bit of a conundrum. There are a few approaches to dealing with this problem:

    1. When single-threaded, code could call the internal jl_gc_enable_finalizers C function to prevent finalizers from being scheduled inside a critical region. Internally, this is used inside some functions (such as our C locks) to prevent recursion when doing certain operations (incremental package loading, codegen, etc.). The combination of a lock and this flag can be used to make finalizers safe.

    2. A second strategy, employed by Base in a couple places, is to explicitly delay a finalizer until it may be able to acquire its lock non-recursively. The following example demonstrates how this strategy could be applied to Distributed.finalize_ref:

      1. function finalize_ref(r::AbstractRemoteRef)
      2.     if r.where > 0 # Check if the finalizer is already run
      3.         if islocked(client_refs) || !trylock(client_refs)
      4.             # delay finalizer for later if we aren't free to acquire the lock
      5.             finalizer(finalize_ref, r)
      6.             return nothing
      7.         end
      8.         try # `lock` should always be followed by `try`
      9.             if r.where > 0 # Must check again here
      10.                 # Do actual cleanup here
      11.                 r.where = 0
      12.             end
      13.         finally
      14.             unlock(client_refs)
      15.         end
      16.     end
      17.     nothing
      18. end