Build

    • Proxy is developed in JAVA, and JDK version 1.8 or later is recommended.
    • Data migration adopts the cluster mode, and ZooKeeper is currently supported as the registry.
    1. Run the following command to compile the ShardingSphere-Proxy binary package:

    Release package：

    • /shardingsphere-distribution/shardingsphere-proxy-distribution/target/apache-shardingsphere-${latest.release.version}-shardingsphere-proxy-bin.tar.gz

    Or you can get the installation package through the

    1. Decompress the proxy release package and modify the configuration file . Please refer to proxy startup guide for details.

    2. Modify the configuration file conf/server.yaml. Please refer to for details.

    Currently, mode must be Cluster, and the corresponding registry must be started in advance.

    Configuration sample:

    1. mode:
    2.   type: Cluster
    3.   repository:
    4.     type: ZooKeeper
    5.     props:
    6.       namespace: governance_ds
    7.       server-lists: localhost:2181
    8.       retryIntervalMilliseconds: 500
    9.       timeToLiveSeconds: 60
    10.       maxRetries: 3
    11.       operationTimeoutMilliseconds: 500
    1. Introduce JDBC driver.

    Proxy has included JDBC driver of PostgreSQL.

    If you are migrating to a heterogeneous database, then you could use more types of database, e.g. Oracle. Introduce JDBC driver as above too.

    1. Start ShardingSphere-Proxy:
    1. sh bin/start.sh
    1. View the proxy log logs/stdout.log. If you see the following statements:
    1. [INFO ] [main] o.a.s.p.frontend.ShardingSphereProxy - ShardingSphere-Proxy start success

    The startup will have been successful.

    1. Configure and migrate on demand.

    7.1. Query configuration.

    The default configuration is as follows.

    1. +--------------------------------------------------------------+--------------------------------------+------------------------------------------------------+
    2. | read                                                         | write                                | stream_channel                                       |
    3. +--------------------------------------------------------------+--------------------------------------+------------------------------------------------------+
    4. +--------------------------------------------------------------+--------------------------------------+------------------------------------------------------+

    7.2. New configuration (Optional).

    A default value is available if there is no configuration.

    A completely configured DistSQL is as follows.

    1. CREATE MIGRATION PROCESS CONFIGURATION (
    2. READ(
    3.   WORKER_THREAD=40,
    4.   BATCH_SIZE=1000,
    5.   SHARDING_SIZE=10000000,
    7. ),
    8. WRITE(
    9.   WORKER_THREAD=40,
    10.   BATCH_SIZE=1000,
    11.   RATE_LIMITER (TYPE(NAME='TPS',PROPERTIES('tps'='2000')))
    12. ),
    13. STREAM_CHANNEL (TYPE(NAME='MEMORY',PROPERTIES('block-queue-size'='10000')))
    14. );
    1. CREATE MIGRATION PROCESS CONFIGURATION (
    2. READ( -- Data reading configuration. If it is not configured, part of the parameters will take effect by default.
    3.   WORKER_THREAD=40, -- Obtain the thread pool size of all the data from the source side. If it is not configured, the default value is used.
    4.   BATCH_SIZE=1000, -- The maximum number of records returned by a query operation. If it is not configured, the default value is used.
    5.   SHARDING_SIZE=10000000, -- Sharding size of all the data. If it is not configured, the default value is used.
    6.   RATE_LIMITER ( -- Traffic limit algorithm. If it is not configured, traffic is not limited.
    7.   TYPE( -- Algorithm type. Option: QPS
    8.   NAME='QPS',
    9.   PROPERTIES( -- Algorithm property
    10.   'qps'='500'
    11.   )))
    12. ),
    13. WRITE( -- Data writing configuration. If it is not configured, part of the parameters will take effect by default.
    14.   WORKER_THREAD=40, -- The size of the thread pool on which data is written into the target side. If it is not configured, the default value is used.
    15.   RATE_LIMITER ( -- Traffic limit algorithm. If it is not configured, traffic is not limited.
    16.   TYPE( -- Algorithm type. Option: TPS
    17.   NAME='TPS',
    18.   PROPERTIES( -- Algorithm property.
    19.   'tps'='2000'
    21. ),
    22. STREAM_CHANNEL ( -- Data channel. It connects producers and consumers, used for reading and writing procedures. If it is not configured, the MEMORY type is used by default.
    23. TYPE( -- Algorithm type. Option: MEMORY
    24. NAME='MEMORY',
    25. PROPERTIES( -- Algorithm property
    26. 'block-queue-size'='10000' -- Property: blocking queue size.
    27. )))
    28. );

    DistSQL sample: configure READ for traffic limit.

    Configure data reading for traffic limit. Other configurations use default values.

    7.3. Modify configuration.

    ALTER MIGRATION PROCESS CONFIGURATION, and its internal structure is the same as that of CREATE MIGRATION PROCESS CONFIGURATION.

    DistSQL sample: modify traffic limit parameter

    1. ALTER MIGRATION PROCESS CONFIGURATION (
    2. READ(
    3.   RATE_LIMITER (TYPE(NAME='QPS',PROPERTIES('qps'='1000')))
    4. )
    5. );
    6. ---
    7. ALTER MIGRATION PROCESS CONFIGURATION (
    8. READ(
    9.   RATE_LIMITER (TYPE(NAME='QPS',PROPERTIES('qps'='1000')))
    10. ), WRITE(
    11.   RATE_LIMITER (TYPE(NAME='TPS',PROPERTIES('tps'='1000')))
    12. )
    13. );

    7.4. Clear configuration.

    DistSQL sample: clear the configuration of READ and restore it to the default value.

    1. DROP MIGRATION PROCESS CONFIGURATION '/READ';
    1. DROP MIGRATION PROCESS CONFIGURATION '/READ/RATE_LIMITER';