Initializing New Segments

The first time you run gpexpand with a valid input file it creates the expansion schema and sets the distribution policy for all tables to DISTRIBUTED RANDOMLY. After these steps are completed, running gpexpand detects if the expansion schema has been created and, if so, performs table redistribution.

Creating an Input File for System Expansion
Rolling Back a Failed Expansion Setup

Parent topic:

To begin expansion, gpexpand requires an input file containing information about the new segments and hosts. If you run gpexpand without specifying an input file, the utility displays an interactive interview that collects the required information and automatically creates an input file.

If you create the input file using the interactive interview, you may specify a file with a list of expansion hosts in the interview prompt. If your platform or command shell limits the length of the host list, specifying the hosts with -f may be mandatory.

Before you run gpexpand to create an input file in interactive mode, ensure you know:

The number of new hosts (or a hosts file)
The mirroring strategy used in existing hosts, if any
The number of segments to add per host, if any

The utility automatically generates an input file based on this information, dbid, content ID, and data directory values stored in gp_segment_configuration, and saves the file in the current directory.

To create an input file in interactive mode

Log in on the master host as the user who will run your Greenplum Database system; for example, gpadmin.
Run gpexpand. The utility displays messages about how to prepare for an expansion operation, and it prompts you to quit or continue.

Optionally, specify a hosts file using -f. For example:
At the prompt, select Y to continue.
Unless you specified a hosts file using -f, you are prompted to enter hostnames. Enter a comma separated list of the hostnames of the new expansion hosts. Do not include interface hostnames. For example:
```
> sdw4, sdw5, sdw6, sdw7
```
To add segments to existing hosts only, enter a blank line at this prompt. Do not specify localhost or any existing host name.
Enter the mirroring strategy used in your system, if any. Options are spread|grouped|none. The default setting is grouped.
Enter the number of new primary segments to add, if any. By default, new hosts are initialized with the same number of primary segments as existing hosts. Increase segments per host by entering a number greater than zero. The number you enter will be the number of additional segments initialized on all hosts. For example, if existing hosts currently have two segments each, entering a value of 2 initializes two more segments on existing hosts, and four segments on new hosts.
If you are adding new primary segments, enter the new primary data directory root for the new segments. Do not specify the actual data directory name, which is created automatically by based on the existing data directory names.

For example, if your existing data directories are as follows:
```
/gpdata/primary/gp0
/gpdata/primary/gp1
```
then enter the following (one at each prompt) to specify the data directories for two new primary segments:
```
/gpdata/primary
```
When the initialization runs, the utility creates the new directories gp2 and gp3 under /gpdata/primary.
If you are adding new mirror segments, enter the new mirror data directory root for the new segments. Do not specify the data directory name; it is created automatically by gpexpand based on the existing data directory names.

For example, if your existing data directories are as follows:

enter the following (one at each prompt) to specify the data directories for two new mirror segments:
```
/gpdata/mirror
/gpdata/mirror
```
When the initialization runs, the utility will create the new directories gp2 and gp3 under /gpdata/mirror.

These primary and mirror root directories for new segments must exist on the hosts, and the user running gpexpand must have permissions to create directories in them.

After you have entered all required information, the utility generates an input file and saves it in the current directory. For example:
```
gpexpand_inputfile_yyyymmdd<_145134>
```
If the Greenplum cluster is configured with tablespaces, the utility automatically generates an additional tablespace mapping file. This file is required for later parsing by the utility so make sure it is present before proceeding with the next step. For example:
```
gpexpand_inputfile_yyyymmdd<_145134>.ts
```

Use the interactive interview process to create your own input file unless your expansion scenario has atypical needs.

For example:

sdw5:sdw5-1:50011:/gpdata/primary/gp9:11:9:p:53011
sdw5:sdw5-2:50012:/gpdata/primary/gp10:12:10:p:53011
sdw5:sdw5-2:60011:/gpdata/mirror/gp9:13:9:m:63011

For each new segment, this format of expansion input file requires the following:

After you have created an input file, run gpexpand to initialize new segments. The utility automatically stops Greenplum Database segment initialization and restarts the system when the process finishes.

Log in on the master host as the user who will run your Greenplum Database system; for example, gpadmin.
Run the gpexpand utility, specifying the input file with -i. Optionally, use -D to specify the database in which to create the expansion schema. For example:
The utility detects if an expansion schema exists for the Greenplum Database system. If a schema exists, remove it with gpexpand -c before you start a new expansion operation. See Removing the Expansion Schema.

When the new segments are initialized and the expansion schema is created, the utility prints a success message and exits.

When the initialization process completes, you can connect to Greenplum Database and view the expansion schema. The schema resides in the database you specified with -D or in the database specified by the PGDATABASE environment variable. For more information, see .

After segment initialization is complete, redistribute the tables to balance existing data over the new segments.

At any time, you can check the state of cluster expansion by running the gpstate utility with the -x flag:

$ gpstate -x

If the expansion schema exists in the postgres database, gpstate -x reports on the progress of the expansion. During the first expansion phase, gpstate reports on the progress of new segment initialization. During the second phase, reports on the progress of table redistribution, and whether redistribution is paused or active.

You can also query the expansion schema to see expansion status. See Monitoring Table Redistribution for more information.

You can roll back an expansion setup operation (adding segment instances and segment hosts) only if the operation fails.

Roll back the failed expansion with the following command: