Horizontal Sharding

    Defines a rudimental ‘horizontal sharding’ system which allows a Session to distribute queries and persistence operations across multiple databases.

    For a usage example, see the Horizontal Sharding example included in the source distribution.

    Deep Alchemy

    The horizontal sharding extension is an advanced feature, involving a complex statement -> database interaction as well as use of semi-public APIs for non-trivial cases. Simpler approaches to refering to multiple database “shards”, most commonly using a distinct per “shard”, should always be considered first before using this more complex and less-production-tested system.

    class sqlalchemy.ext.horizontal_shard.ShardedSession

    Members

    , connection_callable(),

    Class signature

    class sqlalchemy.ext.horizontal_shard.ShardedSession ()

    • method sqlalchemy.ext.horizontal_shard.ShardedSession.__init__(shard_chooser: ShardChooser, identity_chooser: Optional[IdentityChooser] = None, execute_chooser: Optional[Callable[[ORMExecuteState], Iterable[Any]]] = None, shards: Optional[Dict[str, Any]] = None, query_cls: Type[Query[_T]] = <class ‘sqlalchemy.ext.horizontal_shard.ShardedQuery’>, *, id_chooser: Optional[Callable[[Query[_T], Iterable[_T]], Iterable[Any]]] = None, query_chooser: Optional[Callable[[Executable], Iterable[Any]]] = None, **kwargs: Any) → None

      Construct a ShardedSession.

      • Parameters:

        • shard_chooser – A callable which, passed a Mapper, a mapped instance, and possibly a SQL clause, returns a shard ID. This id may be based off of the attributes present within the object, or on some round-robin scheme. If the scheme is based on a selection, it should set whatever state on the instance to mark it in the future as participating in that shard.

        • identity_chooser –

          A callable, passed a Mapper and primary key argument, which should return a list of shard ids where this primary key might reside.

        • execute_chooser –

          For a given , returns the list of shard_ids where the query should be issued. Results from all shards returned will be combined together into a single listing.

        • shards – A dictionary of string shard names to Engine objects.

    • method get_bind(mapper: Optional[_EntityBindKey[_O]] = None, *, shard_id: Optional[ShardIdentifier] = None, instance: Optional[Any] = None, clause: Optional[ClauseElement] = None, **kw: Any) → _SessionBind

      Return a “bind” to which this is bound.

      The “bind” is usually an instance of Engine, except in the case where the has been explicitly bound directly to a Connection.

      For a multiply-bound or unbound , the or clause arguments are used to determine the appropriate bind to return.

      Note that the “mapper” argument is usually present when Session.get_bind() is called via an ORM operation such as a , each individual INSERT/UPDATE/DELETE operation within a Session.flush(), call, etc.

      The order of resolution is:

      1. if mapper given and is present, locate a bind based first on the mapper in use, then on the mapped class in use, then on any base classes that are present in the __mro__ of the mapped class, from more specific superclasses to more general.

      2. if clause given and Session.binds is present, locate a bind based on Table objects found in the given clause present in .

      3. if Session.binds is present, return that.

      4. if clause given, attempt to return a bind linked to the ultimately associated with the clause.

      5. if mapper given, attempt to return a bind linked to the MetaData ultimately associated with the or other selectable to which the mapper is mapped.

      6. No bind can be found, UnboundExecutionError is raised.

      Note that the method can be overridden on a user-defined subclass of Session to provide any kind of bind resolution scheme. See the example at .

    class sqlalchemy.ext.horizontal_shard.set_shard_id

    a loader option for statements to apply a specific shard id to the primary query as well as for additional relationship and column loaders.

    The set_shard_id option may be applied using the method of any executable statement:

    Above, the statement when invoked will limit to the “shard1” shard identifier for the primary query as well as for all relationship and column loading strategies, including eager loaders such as selectinload(), deferred column loaders like , and the lazy relationship loader lazyload().

    New in version 2.0.0.

    Members

    , propagate_to_loaders

    Class signature

    class (sqlalchemy.orm.ORMOption)

    • method sqlalchemy.ext.horizontal_shard.set_shard_id.__init__(shard_id: str, propagate_to_loaders: bool = True)

      Construct a option.

      • Parameters:

        • shard_id – shard identifier

        • propagate_to_loaders – if left at its default of , the shard option will take place for lazy loaders such as lazyload() and ; if False, the option will not be propagated to loaded objects. Note that defer() always limits to the shard_id of the parent row in any case, so the parameter only has a net effect on the behavior of the strategy.

    • attribute sqlalchemy.ext.horizontal_shard.set_shard_id.propagate_to_loaders

      if True, indicate this option should be carried along to “secondary” SELECT statements that occur for relationship lazy loaders as well as attribute load / refresh operations.

    class sqlalchemy.ext.horizontal_shard.ShardedQuery

    Query class used with .

    Legacy Feature

    The ShardedQuery is a subclass of the legacy class. The ShardedSession now supports 2.0 style execution via the ShardedSession.execute() method.

    Members

    Class signature

    class sqlalchemy.ext.horizontal_shard.ShardedQuery ()