DataGen SQL Connector

The DataGen connector allows for creating tables based on in-memory data generation. This is useful when developing queries locally without access to external systems such as Kafka. Tables can include Computed Column syntax which allows for flexible record generation.

By default, a DataGen table will create an unbounded number of rows with a random value for each column. For variable sized types, char/varchar/string/array/map/multiset, the length can be specified. Additionally, a total number of rows can be specified, resulting in a bounded table.

Time types are always the local machines current system time.

CREATE TABLE Orders (
    order_number BIGINT,
    buyer        ROW<first_name STRING, last_name STRING>,
) WITH (...)
-- create a bounded mock table
    'connector' = 'datagen',
    'number-of-rows' = '10'
)
LIKE Orders (EXCLUDING ALL)