Apache Doris 发布流程

    发布的大致流程如下：

    1. 环境准备
    2. 发布准备
      1. 社区发起 DISCUSS 并与社区交流具体发布计划
      2. 创建分支用于发布
      3. 清理 issue
      4. 将必要的 Patch 合并到发布的分支
    3. 社区发布投票流程
      1. 将 tag 打包，签名并上传到Apache Dev svn 仓库
      2. 在 发起投票
      3. 投票通过后，在Doris社区发 Result 邮件
      4. 在 Incubator 社区 发起新一轮投票
      5. 发 Result 邮件到 general@incubator.apache.org
    4. 完成工作
      1. 上传签名的软件包到 ，并生成相关链接
      2. 准备 release note 并发 Announce 邮件到 general@incubator.apache.org
      3. 在 Doris 官网和 github 发布下载链接

    如果这是你第一次发布，那么你需要在你的环境中准备如下工具

    1. release signing https://www.apache.org/dev/release-signing.html
    2. gpg
    3. svn https://www.apache.org/dev/openpgp.html

    Release manager 在发布前需要先生成自己的签名公钥，并上传到公钥服务器，之后就可以用这个公钥对准备发布的软件包进行签名。 如果在里已经存在了你的KEY，那么你可以跳过这个步骤了。

    签名软件 GnuPG 的安装配置

    GnuPG

    1991年，程序员 Phil Zimmermann 为了避开政府监视，开发了加密软件PGP。这个软件非常好用，迅速流传开来，成了许多程序员的必备工具。但是，它是商业软件，不能自由使用。所以，自由软件基金会决定，开发一个PGP的替代品，取名为GnuPG。这就是GPG的由来。

    安装配置

    CentOS 安装命令：

    安装完成后，默认配置文件 gpg.conf 会放在 home 目录下。

    如果不存在这个目录或文件，可以直接创建一个空文件。 编辑gpg.conf, 修改或者增加 keyserver 配置：

    1. keyserver hkp://keys.gnupg.net

    Apache 签名推荐 SHA512， 可以通过配置 gpg 完成。 编辑gpg.conf, 增加下面的三行：

    1. personal-digest-preferences SHA512
    2. cert-digest-algo SHA512
    3. default-preference-list SHA512 SHA384 SHA256 SHA224 AES256 AES192 AES CAST5 ZLIB BZIP2 ZIP Uncompressed

    生成新的签名

    准备签名

    推荐的生成新签名的设置：

    这里必须通过 SecureCRT 等终端直接登录用户账户，不能通过 su - user 或者 ssh 转，否则密码输入 box 会显示不出来而报错。

    先看下 gpg 的 version 以及是否支持 SHA512.

    1. $ gpg --version
    2. gpg (GnuPG) 2.0.22
    3. libgcrypt 1.5.3
    4. Copyright (C) 2013 Free Software Foundation, Inc.
    5. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    6. This is free software: you are free to change and redistribute it.
    7. There is NO WARRANTY, to the extent permitted by law.
    9. Home: ~/.gnupg
    10. Supported algorithms:
    11. Pubkey: RSA, ?, ?, ELG, DSA
    12. Cipher: IDEA, 3DES, CAST5, BLOWFISH, AES, AES192, AES256, TWOFISH,
    13.         CAMELLIA128, CAMELLIA192, CAMELLIA256
    14. Hash: MD5, SHA1, RIPEMD160, SHA256, SHA384, SHA512, SHA224
    15. Compression: Uncompressed, ZIP, ZLIB, BZIP2
    生成新的签名
    1. $ gpg --gen-key
    2. gpg (GnuPG) 2.0.22; Copyright (C) 2013 Free Software Foundation, Inc.
    3. This is free software: you are free to change and redistribute it.
    4. There is NO WARRANTY, to the extent permitted by law.
    6. Please select what kind of key you want:
    7.    (1) RSA and RSA (default)
    8.    (2) DSA and Elgamal
    9.    (3) DSA (sign only)
    10.    (4) RSA (sign only)
    11. Your selection? 1
    12. RSA keys may be between 1024 and 4096 bits long.
    13. What keysize do you want? (2048) 4096
    14. Requested keysize is 4096 bits
    15. Please specify how long the key should be valid.
    16.          0 = key does not expire
    17.       <n>  = key expires in n days
    18.       <n>w = key expires in n weeks
    19.       <n>m = key expires in n months
    20.       <n>y = key expires in n years
    21. Key is valid for? (0)
    22. Key does not expire at all
    23. Is this correct? (y/N) y
    25. GnuPG needs to construct a user ID to identify your key.
    27. Real name: xxx
    28. Name must be at least 5 characters long
    29. Real name: xxx-yyy
    30. Email address: xxx@apache.org
    31. Comment: xxx's key
    32. You selected this USER-ID:
    33.     "xxx-yyy (xxx's key) <xxx@apache.org>"
    35. Change (N)ame, (C)omment, (E)mail or (O)kay/(Q)uit? o

    其中 Real name 需保持和 id.apache.org 中显示的 id 一致。 Email address 为 apache 的邮箱。

    查看和输出

    第一行显示公钥文件名（pubring.gpg），第二行显示公钥特征（4096位，Hash字符串和生成时间），第三行显示”用户ID”，第四行显示私钥特征。

    1. $ gpg --list-keys
    2. /home/lide/.gnupg/pubring.gpg
    3. -----------------------------
    4. pub   4096R/33DBF2E0 2018-12-06
    5. uid                  xxx-yyy  (xxx's key) <xxx@apache.org>
    6. sub   4096R/0E8182E6 2018-12-06

    其中 xxx-yyy 就是用户ID。

    1. gpg --armor --output public-key.txt --export [用户ID]
    1. $ gpg --armor --output public-key.txt --export xxx-yyy
    2. $ cat public-key.txt
    3. -----BEGIN PGP PUBLIC KEY BLOCK-----
    4. Version: GnuPG v2.0.22 (GNU/Linux)
    6. mQINBFwJEQ0BEACwqLluHfjBqD/RWZ4uoYxNYHlIzZvbvxAlwS2mn53BirLIU/G3
    7. 9opMWNplvmK+3+gNlRlFpiZ7EvHsF/YJOAP59HmI2Z...

    上传签名公钥

    公钥服务器是网络上专门储存用户公钥的服务器。send-keys 参数可以将公钥上传到服务器。

    其中 xxxx 为上一步 --list-keys 结果中 pub 后面的字符串，如上为：33DBF2E0

    也可以通过网站Apache Doris 发布流程 - 图4上传上述 public-key.txt 的内容：

    上传成功之后，可以通过查询这个，输入 0x33DBF2E0 查询：

    生成 fingerprint 并上传到 apache 用户信息中

    由于公钥服务器没有检查机制，任何人都可以用你的名义上传公钥，所以没有办法保证服务器上的公钥的可靠性。通常，你可以在网站上公布一个公钥指纹，让其他人核对下载到的公钥是否为真。

    fingerprint参数生成公钥指纹：

    1. gpg --fingerprint [用户ID]
    1. $ gpg --fingerprint xxx-yyy
    2. pub   4096R/33DBF2E0 2018-12-06
    3.       Key fingerprint = 07AA E690 B01D 1A4B 469B  0BEF 5E29 CE39 33DB F2E0
    4. sub   4096R/0E8182E6 2018-12-06

    将上面的 fingerprint （即 07AA E690 B01D 1A4B 469B 0BEF 5E29 CE39 33DB F2E0）粘贴到自己的用户信息中：

    OpenPGP Public Key Primary Fingerprint:

    生成 keys

    1. svn co //dist.apache.org/repos/dist/dev/incubator/doris/
    2. # edit doris/KEY file
    3. gpg --list-sigs [用户 ID] >> doris/KEYS
    4. gpg --armor --export [用户 ID] >> doris/KEYS
    5. svn ci --username $ASF_USERNAME --password "$ASF_PASSWORD" -m"Update KEYS"

    在社区发起 DISCUSS

    如果觉得已经修复了很多bug，开发了比较重要的 feature，任何 IPMC 成员都可以发起 DISCUSS 讨论发布新版本。 可以发起一个标题为 [DISCUSS] x.y.z release 的邮件，在社区内部进行讨论，说明已经修复了哪些bug，开发了哪些 features。 如果 DISCUSS 邮件得到大家支持就可以进行下一步。

    准备分支

    发布前需要先新建一个分支。例如：

    1. $ git checkout -b branch-0.9

    这个分支要进行比较充分的测试，使得功能可用，bug收敛，重要bug都得到修复。 这个过程需要等待社区，看看是否有必要的patch需要在这个版本合入，如果有，需要把它 cherry-pick 到发布分支。

    清理issue

    将属于这个版本的所有 issue 都过一遍，关闭已经完成的，如果没法完成的，推迟到更晚的版本。

    在发布等待过程中，可能会有比较重要的Patch合入，如果社区有人说要有重要的Bug需要合入，那么 Release Manager 需要评估并将重要的Patch合入到发布分支中。

    打 tag

    当上述分支已经比较稳定后，就可以在此分支上打 tag。 记得在创建 tag 时，修改 gensrc/script/gen_build_version.sh 中的 build_version 变量。如 build_version="0.10.0-release"

    例如：

    1. $ git checkout branch-0.9
    2. $ git tag -a 0.9.0-rc01 -m "0.9.0 release candidate 01"
    3. $ git push origin 0.9.0-rc01
    4. Counting objects: 1, done.
    5. Writing objects: 100% (1/1), 165 bytes | 0 bytes/s, done.
    6. Total 1 (delta 0), reused 0 (delta 0)
    7. To git@github.com:apache/incubator-doris.git
    8.  * [new tag]         0.9.0-rc01 -> 0.9.0-rc01
    10. $ git tag

    打包、签名上传

    如下步骤，也需要通过 SecureCRT 等终端直接登录用户账户，不能通过 su - user 或者 ssh 转，否则密码输入 box 会显示不出来而报错。

    1. $ git checkout 0.9.0-rc01
    3. $ git archive --format=tar 0.9.0-rc01 --prefix=apache-doris-0.9.0-incubating-src/ | gzip > apache-doris-0.9.0-incubating-src.tar.gz
    5. $ gpg -u xxx@apache.org --armor --output apache-doris-0.9.0-incubating-src.tar.gz.asc --detach-sign apache-doris-0.9.0-incubating-src.tar.gz
    7. $ gpg --verify apache-doris-0.9.0-incubating-src.tar.gz.asc apache-doris-0.9.0-incubating-src.tar.gz
    9. $ sha512sum apache-doris-0.9.0-incubating-src.tar.gz > apache-doris-0.9.0-incubating-src.tar.gz.sha512
    11. $ sha512sum --check apache-doris-0.9.0-incubating-src.tar.gz.sha512

    然后将打包的内容上传到svn仓库中，首先下载 svn 库：

    1. svn co https://dist.apache.org/repos/dist/dev/incubator/doris/

    将之前得到的全部文件组织成以下svn路径

    1. ./doris/
    3. |   |-- apache-doris-0.11.0-incubating-src.tar.gz
    4. |   |-- apache-doris-0.11.0-incubating-src.tar.gz.asc
    5. |   `-- apache-doris-0.11.0-incubating-src.tar.gz.sha512
    6. `-- KEYS

    上传这些文件

    发社区投票邮件

    [VOTE] Release Apache Doris 0.9.0-incubating-rc01

    1. Hi all,
    3. Please review and vote on Apache Doris 0.9.0-incubating-rc01 release.
    5. The release candidate has been tagged in GitHub as 0.9.0-rc01, available
    6. here:
    7. https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01
    9. Release Notes are here:
    10. https://github.com/apache/incubator-doris/issues/1891
    12. Thanks to everyone who has contributed to this release.
    14. The artifacts (source, signature and checksum) corresponding to this release
    15. candidate can be found here:
    16. https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc1/
    18. This has been signed with PGP key 33DBF2E0, corresponding to
    19. lide@apache.org.
    20. KEYS file is available here:
    21. https://dist.apache.org/repos/dist/dev/incubator/doris/KEYS
    22. It is also listed here:
    23. https://people.apache.org/keys/committer/lide.asc
    25. To verify and build, you can refer to following wiki:
    26. https://github.com/apache/incubator-doris/wiki/How-to-verify-Apache-Release
    27. https://wiki.apache.org/incubator/IncubatorReleaseChecklist
    29. The vote will be open for at least 72 hours.
    30. [ ] +1 Approve the release
    31. [ ] +0 No opinion
    32. [ ] -1 Do not release this package because ...
    34. Best Regards,
    35. xxx
    37. ----
    38. DISCLAIMER-WIP: 
    39. Apache Doris is an effort undergoing incubation at The Apache Software Foundation (ASF), 
    40. sponsored by the Apache Incubator. Incubation is required of all newly accepted projects 
    41. until a further review indicates that the infrastructure, communications, and decision 
    42. making process have stabilized in a manner consistent with other successful ASF projects. 
    43. While incubation status is not necessarily a reflection of the completeness or stability 
    44. of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
    46. Some of the incubating project’s releases may not be fully compliant with ASF policy. For 
    47. example, releases may have incomplete or un-reviewed licensing conditions. What follows is 
    48. a list of known issues the project is currently aware of (note that this list, by definition, 
    49. is likely to be incomplete): 
    51.  * Releases may have incomplete licensing conditions
    53. If you are planning to incorporate this work into your product/project, please be aware that
    54. you will need to conduct a thorough licensing review to determine the overall implications of 
    55. including this work. For the current status of this project through the Apache Incubator 
    56. visit: https://incubator.apache.org/projects/doris.html

    [Result][VOTE] Release Apache Doris 0.9.0-incubating-rc01

    1. Thanks to everyone, and this vote is now closed.
    3. It has passed with 4 +1 (binding) votes and no 0 or -1 votes.
    5. Binding:
    6. +1 Zhao Chun
    7. +1 xxx
    8. +1 Li Chaoyong
    9. +1 Mingyu Chen
    11. Best Regards,
    12. xxx

    发邮件到 general@incubator.apache.org 进行投票

    1. Hi all,
    3. Please review and vote on Apache Doris 0.9.0-incubating-rc01 release.
    5. Apache Doris is an MPP-based interactive SQL data warehousing for reporting and analysis.
    7. The Apache Doris community has voted on and approved this release:
    8. https://lists.apache.org/thread.html/d70f7c8a8ae448bf6680a15914646005c6483564464cfa15f4ddc2fc@%3Cdev.doris.apache.org%3E
    10. The vote result email thread:
    11. https://lists.apache.org/thread.html/64d229f0ba15d66adc83306bc8d7b7ccd5910ecb7e842718ce6a61da@%3Cdev.doris.apache.org%3E
    13. The release candidate has been tagged in GitHub as 0.9.0-rc01, available here:
    14. https://github.com/apache/incubator-doris/releases/tag/0.9.0-rc01
    16. There is no CHANGE LOG file because this is the first release of Apache Doris.
    17. Thanks to everyone who has contributed to this release, and there is a simple release notes can be found here:
    18. https://github.com/apache/incubator-doris/issues/406
    20. https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc01/
    22. This has been signed with PGP key 33DBF2E0, corresponding to lide@apache.org.
    23. KEYS file is available here:
    24. https://dist.apache.org/repos/dist/dev/incubator/doris/KEYS
    25. It is also listed here:
    26. https://people.apache.org/keys/committer/lide.asc
    28. The vote will be open for at least 72 hours.
    29. [ ] +1 Approve the release
    30. [ ] +0 No opinion
    31. [ ] -1 Do not release this package because ...
    33. To verify and build, you can refer to following instruction:
    37. Step1: Pull the docker image with Doris building environment
    38. $ docker pull apachedoris/doris-dev:build-env
    39. You can check it by listing images, its size is about 3.28GB.
    41. Step2: Run the Docker image
    42. You can run image directly:
    43. $ docker run -it apachedoris/doris-dev:build-env
    45. Step3: Download Doris source
    46. Now you should in docker environment, and you can download Doris source package.
    47. (If you have downloaded source and it is not in image, you can map its path to image in Step2.)
    48. $ wget https://dist.apache.org/repos/dist/dev/incubator/doris/0.9/0.9.0-rc01/apache-doris-0.9.0.rc01-incubating-src.tar.gz
    50. Step4: Build Doris
    51. Now you can decompress and enter Doris source path and build Doris.
    52. $ tar zxvf apache-doris-0.9.0.rc01-incubating-src.tar.gz
    53. $ cd apache-doris-0.9.0.rc01-incubating-src
    54. $ sh build.sh
    56. Best Regards,
    57. xxx
    59. ----
    60. DISCLAIMER-WIP: 
    61. Apache Doris is an effort undergoing incubation at The Apache Software Foundation (ASF), 
    62. sponsored by the Apache Incubator. Incubation is required of all newly accepted projects 
    63. until a further review indicates that the infrastructure, communications, and decision 
    64. making process have stabilized in a manner consistent with other successful ASF projects. 
    65. While incubation status is not necessarily a reflection of the completeness or stability 
    66. of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
    68. Some of the incubating project’s releases may not be fully compliant with ASF policy. For 
    69. example, releases may have incomplete or un-reviewed licensing conditions. What follows is 
    70. a list of known issues the project is currently aware of (note that this list, by definition, 
    71. is likely to be incomplete): 
    73.  * Releases may have incomplete licensing conditions
    75. If you are planning to incorporate this work into your product/project, please be aware that
    76. you will need to conduct a thorough licensing review to determine the overall implications of 
    77. including this work. For the current status of this project through the Apache Incubator 
    78. visit: https://incubator.apache.org/projects/doris.html

    邮件的 thread 连接可以在这里找到：

    https://lists.apache.org/list.html?dev@doris.apache.org

    发 Result 邮件到 general@incubator.apache.org

    [RESULT][VOTE] Release Apache Doris 0.9.0-incubating-rc01

    1. Hi,
    3. Thanks to everyone, and the vote for releasing Apache Doris 0.9.0-incubating-rc01 is now closed.
    5. It has passed with 4 +1 (binding) votes and no 0 or -1 votes.
    7. Binding:
    8. +1 Willem Jiang
    9. +1 Justin Mclean
    10. +1 ShaoFeng Shi
    11. +1 Makoto Yui
    13. The vote thread:
    14. https://lists.apache.org/thread.html/da05fdd8d84e35de527f27200b5690d7811a1e97d419d1ea66562130@%3Cgeneral.incubator.apache.org%3E
    16. Best Regards,
    17. xxx

    上传 package 到 release

    当正式发布投票成功后，先发[Result]邮件，然后就准备 release package。 将之前在dev下发布的对应rc文件夹下的源码包、签名文件和hash文件拷贝到另一个目录 0.9.0-incubating，注意文件名字中不要rcxx (可以rename，但不要重新计算签名，hash可以重新计算，结果不会变)

    第一次发布的话 KEYS 文件也需要拷贝过来。然后add到svn release 下。

    2. https://dist.apache.org/repos/dist/release/incubator/doris/0.9.0-incubating/
    4. 最终能在 apache 官网看到：
    5. http://www.apache.org/dist/incubator/doris/0.9.0-incubating/

    Title:

    1. [ANNOUNCE] Apache Doris (incubating) 0.9.0 Release

    发送邮件组：

    1. general@incubator.apache.org <general@incubator.apache.org>
    2. dev@doris.apache.org <dev@doris.apache.org>

    邮件正文：

    1. Hi All,
    3. We are pleased to announce the release of Apache Doris 0.9.0-incubating.
    5. Apache Doris (incubating) is an MPP-based interactive SQL data warehousing for reporting and analysis.
    7. The release is available at:
    8. http://doris.apache.org/downloads.html
    10. Thanks to everyone who has contributed to this release, and the release note can be found here:
    11. https://github.com/apache/incubator-doris/releases
    13. Best Regards,
    15. On behalf of the Doris team,
    16. xxx
    18. ----
    19. DISCLAIMER-WIP: 
    20. Apache Doris is an effort undergoing incubation at The Apache Software Foundation (ASF), 
    21. sponsored by the Apache Incubator. Incubation is required of all newly accepted projects 
    22. until a further review indicates that the infrastructure, communications, and decision 
    23. making process have stabilized in a manner consistent with other successful ASF projects. 
    24. While incubation status is not necessarily a reflection of the completeness or stability 
    25. of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
    27. Some of the incubating project’s releases may not be fully compliant with ASF policy. For 
    28. example, releases may have incomplete or un-reviewed licensing conditions. What follows is 
    29. a list of known issues the project is currently aware of (note that this list, by definition, 
    30. is likely to be incomplete): 
    32.  * Releases may have incomplete licensing conditions
    34. If you are planning to incorporate this work into your product/project, please be aware that
    35. you will need to conduct a thorough licensing review to determine the overall implications of 
    36. including this work. For the current status of this project through the Apache Incubator 
    37. visit: https://incubator.apache.org/projects/doris.html

    在 Doris 官网和 github 发布链接

    创建下载链接

    下载链接： http://www.apache.org/dyn/closer.cgi?filename=incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz&action=download

    wget —trust-server-names ““

    原始位置: https://www.apache.org/dist/incubator/doris/0.9.0-incubating/

    源码包（source package）: http://www.apache.org/dyn/closer.cgi/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz

    ASC:

    sha512: http://archive.apache.org/dist/incubator/doris/0.9.0-incubating/apache-doris-0.9.0-incubating-src.tar.gz.sha512

    KEYS:

    refer to: http://www.apache.org/dev/release-download-pages#closerApache Doris 发布流程 - 图6

    准备 release note

    需要修改如下两个地方：

    2、Doris 官网下载页面

    1. http://doris.apache.org/downloads.html