Proper maintenance and care of multi-threading locks

    Below are all of the locks that exist in the system and the mechanisms for using them that avoid the potential for deadlocks (no Ostrich algorithm allowed here):

    The following are definitely leaf locks (level 1), and must not try to acquire any other lock:

    • safepoint

      Note that this lock is acquired implicitly by and JL_UNLOCK. use the _NOGC variants to avoid that for level 1 locks.

      While holding this lock, the code must not do any allocation or hit any safepoints. Note that there are safepoints when doing allocation, enabling / disabling GC, entering / restoring exception frames, and taking / releasing locks.

    • shared_map

    • finalizers

    • pagealloc

    • gcpermlock

    • flisp

    • jlinstackwalk (Win32)

      flisp itself is already threadsafe, this lock only protects the jl_ast_context_list_t pool

    The following is a leaf lock (level 2), and only acquires level 1 locks (safepoint) internally:

    • typecache

    The following is a level 3 lock, which can only acquire level 1 or level 2 locks internally:

    • Method->writelock

    The following is a level 4 lock, which can only recurse to acquire level 1, 2, or 3 locks:

    No Julia code may be called while holding a lock above this point.

    The following are a level 6 lock, which can only recurse to acquire locks at lower levels:

    • codegen
    • jlmodulesmutex

    • typeinf

      this one is perhaps one of the most tricky ones, since type-inference can be invoked from many points

      currently the lock is merged with the codegen lock, since they call each other recursively

    The following lock synchronizes IO operation. Be aware that doing any I/O (for example, printing warning messages or debug information) while holding any other lock listed above may result in pernicious and hard-to-find deadlocks. BE VERY CAREFUL!

    • iolock

    • Individual ThreadSynchronizers locks

      this may continue to be held after releasing the iolock, or acquired without it, but be very careful to never attempt to acquire the iolock while holding it

    The following is the root lock, meaning no other lock shall be held when trying to acquire it:

    The following locks are broken:

    • toplevel

      doesn’t exist right now

      fix: create it

    • Module->lock

      This is vulnerable to deadlocks since it can’t be certain it is acquired in sequence. Some operations (such as import_module) are missing a lock.

    • loading.jl: require and register_root_module

      This file potentially has numerous problems.

      fix: needs locks

    These data structures each need locks due to being shared mutable global state. It is the inverse list for the above lock priority list. This list does not include level 1 leaf resources due to their simplicity.

    MethodTable modifications (def, cache, kwsorter type) : MethodTable->writelock

    Type declarations : toplevel lock

    Type application : typecache lock

    Global variable tables : Module->lock

    Module serializer : toplevel lock

    JIT & type-inference : codegen lock

    MethodInstance/CodeInstance updates : Method->writelock, codegen lock

    • These are set at construction and immutable:

      • specTypes
      • sparam_vals

    • These are set by jl_type_infer (while holding codegen lock):

      • cache
      • rettype
      • inferred

    LLVMContext : codegen lock

    • roots array (serializer and codegen)
    • invoke / specializations / tfunc modifications