Proper maintenance and care of multi-threading locks

    Below are all of the locks that exist in the system and the mechanisms for using them that avoid the potential for deadlocks (no Ostrich algorithm allowed here):

    The following are definitely leaf locks (level 1), and must not try to acquire any other lock:

    • safepoint

      Note that this lock is acquired implicitly by and JL_UNLOCK. use the _NOGC variants to avoid that for level 1 locks.

      While holding this lock, the code must not do any allocation or hit any safepoints. Note that there are safepoints when doing allocation, enabling / disabling GC, entering / restoring exception frames, and taking / releasing locks.

    • shared_map

    • finalizers

    • pagealloc

    • gcpermlock

    • flisp

      flisp itself is already threadsafe, this lock only protects the jl_ast_context_list_t pool

    The following is a leaf lock (level 2), and only acquires level 1 locks (safepoint) internally:

    The following is a level 3 lock, which can only acquire level 1 or level 2 locks internally:

    • Method->writelock
    • MethodTable->writelock

    No Julia code may be called while holding a lock above this point.

    The following is a level 6 lock, which can only recurse to acquire locks at lower levels:

    • codegen

    The following is an almost root lock (level end-1), meaning only the root look may be held when trying to acquire it:

    The following is the root lock, meaning no other lock shall be held when trying to acquire it:

    • toplevel

      this should be held while attempting a top-level action (such as making a new type or defining a new method): trying to obtain this lock inside a staged function will cause a deadlock condition!

      additionally, it's unclear if any code can safely run in parallel with an arbitrary toplevel expression, so it may require all threads to get to a safepoint first

    The following locks are broken:

    • toplevel

    MethodTable modifications (def, cache, kwsorter type) : MethodTable->writelock

    Type declarations : toplevel lock

    Type application : typecache lock

    Module serializer : toplevel lock

    JIT & type-inference : codegen lock

    MethodInstance updates : codegen lock

    • These fields are generally lazy initialized, using the test-and-test-and-set pattern.

    • These are set at construction and immutable:

      • specTypes
      • sparam_vals
      • def

    • These are set by jl_type_infer (while holding codegen lock):

      • inferred
      • these can also be reset, see jl_set_lambda_rettype for that logic as it needs to keep in sync

    • inInference flag:

      • optimization to quickly avoid recurring into jl_type_infer while it is already running
      • actual state (of setting inferred, then fptr) is protected by codegen lock

    • Function pointers (jlcall_api and , unspecialized_ducttape):

      • these transition once, from NULL to a value, while the codegen lock is held

    • Code-generator cache (the contents of functionObjectsDecls):

      • these can transition multiple times, but only while the codegen lock is held
      • it is valid to use old version of this, or block for new versions of this, so races are benign, as long as the code is careful not to reference other data in the method instance (such as rettype) and assume it is coordinated, unless also holding the codegen lock

    • compile_traced flag:

      • unknown

    Method : Method->writelock

    • invoke / specializations / tfunc modifications