Apache Syncope – Release Process

Introduction
Prerequisites
- GIT
- GPG
- Apache Maven
- ASF settings
Release steps
Recovering from a vetoed release

Introduction

The Syncope team uses the Apache Nexus repository for releasing SNAPSHOT and release artifacts.
More details on releasing Maven artifacts at The Apache Software Foundation.

Prerequisites

GIT

Install GIT; binary packages for various platforms are available.

GPG

Install/Configure GPG - The artifacts that are deployed to the ASF central repository need to be signed. To do this you will need to have a public and private keypair. There is a very good guide that will walk you though this.

Apache Maven

Install Apache Maven 3.0.3 or higher; we strongly encourage our committers to install the latest Apache Maven version available.

Maven allows you to encrypt your servers' passwords. We highly recommend that you follow this guide to set your master password and use it to encrypt your ASF password in the next section.

ASF settings

Using the instructions from the previous step encrypt your Apache Nexus repository password and add the following servers to your ~/.m2/settings.xml file. You may already have other servers in this file, if not just create the file.

<?xml version="1.0" encoding="UTF-8"?>
<settings>
  ...
  <servers>
    <server>
      <id>apache.snapshots.https</id>
      <username>{put your ASF username here}</username>
      <password>{put your encrypted password here}</password>
    </server>
    <server>
      <id>apache.releases.https</id>
      <username>{put your ASF username here}</username>
      <password>{put your encrypted password here}</password>
    </server>
  </servers>
  ...
  <profiles>
    <profile>
      <id>apache</id>
      <activation>
        <activeByDefault>false</activeByDefault>
      </activation>
      <properties>
        <mavenExecutorId>forked-path</mavenExecutorId>
        <gpg.keyname>your-gpg-keyname</gpg.keyname>
        <!-- optional -->
        <gpg.passphrase>your-gpg-passphrase</gpg.passphrase>
      </properties>
    </profile>
  </profiles>
  ...
</settings>

Release steps

In the following, read

$VERSION: the version being released
$SYNCOPE_RELEASE_DIR: the working directory containing the GIT clone repository for the branch under release (1_2_X or 2_0_X)
$SYNCOPE_SITE_DIR: the working directory containing the GIT clone repository for the branch for site management (2_0_X)

Verify DBMSes and Java EE containers

This is an optional step.
These verifications take some time and have quite strong environment requirements. However, it is of fundamental importance to go through these at least once for every major release.

For more information please take a look at build instructions.

Prepare the source for release

Clean up JIRA so the Fix Version in issues resolved since the last release includes this release version correctly. Also, transition any Resolved issues to the Closed state.
From $SYNCOPE_RELEASE_DIR, update the CHANGES file, based on the text release reports from JIRA, then commit and push:
```
git commit -m "Updating CHANGES for release" CHANGES
git push
```
From $SYNCOPE_RELEASE_DIR, update
1. ide/eclipse/bundles/org.apache.syncope.ide.eclipse.plugin/META-INF/MANIFEST.MF
2. ide/eclipse/bundles/org.apache.syncope.ide.eclipse.tests/META-INF/MANIFEST.MF
3. ide/eclipse/releng/org.apache.syncope.ide.eclipse.site/category.xml
with $VERSION, then commit and push:
```
git commit -a -m "Updating Eclipse files for release"
git push
```
From $SYNCOPE_SITE_DIR, update the src/site/xdoc/downloads.xml site page by changing the relevant URLs, then commit and push:
```
git commit -m "Updating downloads site page for release" src/site/xdoc/downloads.xml
git push
```

Prepare the release

Perform the steps below from $SYNCOPE_RELEASE_DIR.

Do a dry run of the release:prepare step.
```
mvn -P apache-release release:prepare -DdryRun=true
```
The dry run will not commit any changes and gives you the opportunity to verify that the release process will complete as expected.
If you cancel a release:prepare before it updates the pom.xml versions, then use the release:clean goal to just remove the extra files that were created.
Verify that the release process completed as expected:
1. The release plugin will create pom.xml.tag files which contain the changes that would have been committed. The only differences between pom.xml.tag and its corresponding pom.xml file should be the version number.
2. If other formatting changes have been made you should review the changes and then commit them:
```
git commit -a -m "Fixing formatting for release"
```
3. Assuming the .tag files look OK you may proceed and do any other validation you feel necessary. The following list may be helpful:
  1. Check release.properties and make sure that the scm properties have the right version. Sometimes the scm location can be the previous version not the next version.
  2. Verify signatures: On Un*x platforms the following command can be executed:
```
for file in `find . -type f -iname '*.asc'`
do
  gpg --verify ${file} 
done
```
    You'll need to look at the output to ensure it contains only good signatures:
```
gpg: Good signature from ...
gpg: Signature made ...
```
4. Once any failures or required updates have been committed to GIT, rollback the release prepare files:
```
mvn -P apache-release release:rollback
```
Run the release:prepare step for real this time. You'll be prompted for the same version information and optionally your GPG passphrase again.
```
mvn -P apache-release release:prepare
```
Backup (zip or tar) your local release candidate directory in case you need to rollback the release after the next step is performed.
```
cd ..
tar -czf $VERSION.tar.gz $VERSION/
cd $VERSION
```

Perform the release

Staging artifacts

svn co https://dist.apache.org/repos/dist/dev/syncope syncope-dist-dev
cd syncope-dist-dev              
mkdir $VERSION                  
cd $VERSION
                                    
cp $SYNCOPE_RELEASE_DIR/target/syncope-*-source-release.zip .
cp $SYNCOPE_RELEASE_DIR/target/syncope-*-source-release.zip.asc .
cp $SYNCOPE_RELEASE_DIR/standalone/target/syncope-standalone-*-distribution.zip .
cp $SYNCOPE_RELEASE_DIR/standalone/target/syncope-standalone-*-distribution.zip.asc .
cp $SYNCOPE_RELEASE_DIR/installer/target/syncope-installer-*-uber.jar .
cp $SYNCOPE_RELEASE_DIR/installer/target/syncope-installer-*-uber.jar.asc .
cp $SYNCOPE_RELEASE_DIR/deb/core/target/apache-syncope-*.deb .           
cp $SYNCOPE_RELEASE_DIR/deb/core/target/apache-syncope-*.deb.asc .
cp $SYNCOPE_RELEASE_DIR/deb/console/target/apache-syncope-console-*.deb .
cp $SYNCOPE_RELEASE_DIR/deb/console/target/apache-syncope-console-*.deb.asc .
# only for 2_0_X
cp $SYNCOPE_RELEASE_DIR/deb/enduser/target/apache-syncope-enduser-*.deb .
cp $SYNCOPE_RELEASE_DIR/deb/enduser/target/apache-syncope-enduser-*.deb.asc .
cp $SYNCOPE_RELEASE_DIR/client/cli/target/syncope-client-cli-*.zip .
cp $SYNCOPE_RELEASE_DIR/client/cli/target/syncope-client-cli-*.zip.asc .
cp $SYNCOPE_RELEASE_DIR/ide/eclipse/releng/org.apache.syncope.ide.eclipse.site/target/org.apache.syncope.ide.eclipse.site-*.zip .
cp $SYNCOPE_RELEASE_DIR/ide/eclipse/releng/org.apache.syncope.ide.eclipse.site/target/org.apache.syncope.ide.eclipse.site-*.zip.asc .

for file in `find . -type f -iname '*.asc'`
do
  gpg --verify ${file} 
done
          
for file in `find . -name '*.deb' -or -name '*.jar' -or -name '*.zip'`; do
 openssl md5 $file | sed 's/.*= //' > $file.md5
 openssl sha1 $file | sed 's/.*= //' > $file.sha1
done
              
cd ..
svn add $VERSION
svn commit -m "Staging artifacts for $VERSION vote"

Staging site

unzip $SYNCOPE_RELEASE_DIR/target/syncope-$VERSION-source-release.zip
cd syncope-$VERSION        
mvn -PskipTests,all

# if releasing from 1_2_X              
mvn javadoc:aggregate
cd common
mvn -P offline-doc

svn checkout https://svn.apache.org/repos/asf/syncope/site/ site

cd $SYNCOPE_SITE_DIR
mvn -P site -Dsite.deploymentBaseUrl=file:///<absolute path to/site>/$VERSION

cd <absolute path to/site>/$VERSION

# if releasing from 1_2_X              
cp -r <absolute path to syncope-$VERSION above>/target/site/apidocs apidocs/1.2              
cp -r <absolute path to syncope-$VERSION above>/common/target/generated/wadl/ rest/1.2
rm -rf apidocs/2.0
rm -rf rest/2.0
cd ..
svn add $VERSION
svn copy apidocs/2.0 $VERSION/apidocs/
svn copy rest/2.0 $VERSION/rest/

# else if releasing from 2_0_X
cd ..
svn add $VERSION
svn copy apidocs/1.2 $VERSION/apidocs/
svn copy rest/1.2 $VERSION/rest/
              
svn commit -m "Staging site for release"

From $SYNCOPE_RELEASE_DIR execute (this step will create a maven staging repository):
```
mvn -P apache-release release:perform [-Duser.name=<your_apache_uid>]
```
If your local OS userid doesn't match your Apache userid, then you'll have to also override the value provided by the OS to Maven for the site-deploy step to work. This is known to work for Linux, but not for Mac and unknown for Windows.
1. Verify the staged artifacts in the Nexus repository:
  1. https://repository.apache.org/index.html
  2. Enterprise --> Staging
  3. Staging tab --> Name column --> org.apache.syncope
  4. Navigate through the artifact tree and make sure that all binary, javadoc, sources, and tests jars, as well as pomss, ... have .asc (GPG signature) and .md5 files
    The syncope-$VERSION-source-release.zip should likewise have signature and checksum files.
2. Close the Nexus staging repo:
  1. https://repository.apache.org/index.html
  2. Enterprise --> Staging
  3. Staging tab --> Name column --> org.apache.syncope
  4. Right click on the open org.apache.syncope-XXX staging repo and select Close.
Set the updated version for next development iteration (as provided above to release:prepare) in
1. ide/eclipse/bundles/org.apache.syncope.ide.eclipse.plugin/META-INF/MANIFEST.MF
2. ide/eclipse/bundles/org.apache.syncope.ide.eclipse.tests/META-INF/MANIFEST.MF
3. ide/eclipse/releng/org.apache.syncope.ide.eclipse.site/category.xml
4. ide/eclipse/bundles/org.apache.syncope.ide.eclipse.tests/pom.xml
then commit and push:
```
git commit -a -m "Updating Eclipse files for next development iteration"
git push
```

Vote the release

Create a VOTE email thread on syncope-dev to record votes as replies, e.g.:

To: dev@syncope.apache.org
Subject: [VOTE] Apache Syncope $VERSION

I've created a $VERSION release, with the following artifacts up for a vote:

GIT source tag (XXXXXXXXXXXXXXXX):
https://git-wip-us.apache.org/repos/asf?p=syncope.git;a=tag;h=XXXXXXXXXXXXXXXX

List of changes:
https://git-wip-us.apache.org/repos/asf?p=syncope.git;a=blob_plain;f=CHANGES;h=0a7fb550234f89f4ad315272cf934f2e050250f7;hb=acbc816bdd6a75b27d8fb12b46905fe07e2cd977              

Staging artifacts:
https://dist.apache.org/repos/dist/dev/syncope/$VERSION

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachesyncope-YYY/

Staging site:
http://syncope.apache.org/$VERSION/

PGP release keys (signed using ABCDEFG):
http://www.apache.org/dist/syncope/KEYS

Vote will be open for 72 hours.

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Create a DISCUSS email thread on syncope-dev for any vote questions, e.g.:

To: dev@syncope.apache.org
Subject: [DISCUSS] Apache Syncope $VERSION

Discussion thread for vote on $VERSION release, with GIT source tag (XXXXXXXXXXXXXXXX).

For more information on the release process, check out http://www.apache.org/dev/release.html

Some of the things to check before voting are:
- does "mvn apache-rat:check" pass on the source
- can you build the contents of source release zip and GIT tag
- do all of the staged jars/wars/zips contain the required LICENSE and NOTICE files
- are all of the staged jars/wars/zips signed and the signature verifiable
- is the signing key in the project's KEYS file and on a public server (i.e. http://www.apache.org/dist/syncope/)

Perform a review of the release and cast your vote; more details on Apache releases.
A -1 vote does not necessarily mean that the vote must be redone, however it is usually a good idea to rollback the release if a -1 vote is received (see "Recovering from a vetoed release").

After the vote has been open for at least 72 hours, has at least three +1 PMC votes and no -1 votes, then post the results to the vote thread:

To: dev@syncope.apache.org
Subject: [RESULT] [VOTE] Apache Syncope $VERSION

Hi all,
after 72 hours, the vote for Syncope $VERSION [1] *passes*
with ... PMC + ... non-PMC votes.

+1 (PMC / binding)
* ...

+1 (non binding)
* ... (or <none>)

0
* ... (or <none>)

-1
* ... (or <none>)

Thanks to everyone participating.

I will now copy this release to Syncope' dist directory and promote the artifacts to the central Maven repository.

Best regards.

[1] <link to syncope-dev ML archives for the related [VOTE] thread>

Finalize the release

Promote the staged Nexus artifacts:
1. https://repository.apache.org/index.html
2. Enterprise --> Staging
3. Staging tab --> Name column --> org.apache.syncope
4. Right click on the closed org.apache.syncope-XXX staging repo and select Release.

Add the distribution artifacts to the distribution area

svn mv -m "Moving the voted release artifacts to dist/release" \
https://dist.apache.org/repos/dist/dev/syncope/$VERSION
https://dist.apache.org/repos/dist/release/syncope/

svn co https://dist.apache.org/repos/dist/release/syncope syncope-dist-release
cd syncope-dist-release
svn rm <any older release artifact (if present)>
svn commit -m "Cleaning up older releases"

Add appropriate release notes to Releases wiki page based on the HTML release reports from JIRA
(if releasing from 1_2_X) Update the Create new project wiki page to point to the new release artifacts
Update the JIRA versions page to mark the version as Released, and set the date to the date that the release was approved. You may also need to make a new release entry for the next release.

Promote the staging site

svn co https://svn.apache.org/repos/asf/syncope/site/
cd site
svn rm *.html apidocs rest css images img js fonts docs
svn mv $VERSION/* .
svn rm $VERSION
svn commit -m "Promoting the staging site"

Announce the release

After the mirrors have had time to update (24 hours to be on the safe side), make an announcement about the release on the user, dev, and announce@apache.org lists as per the Apache Announcement Mailing Lists page

Recovering from a vetoed release

Reply to the initial vote email prepending [CANCELED] to the original subject.
Rollback the version upgrades in trunk by either:
1. restore the $VERSION.tar.gz and run
```
mvn -P apache-release release:rollback
```
2. or manually revert the version numbers in the branch under release to the prior version and commit
Delete the GIT tag created by the release:perform step:
```
git tag -d $VERSION && git push origin
```

Delete the staging site:

svn remove https://svn.apache.org/repos/asf/syncope/site/$VERSION -m "Deleting staging site from rolled back release"

Drop the Nexus staging repo:
1. https://repository.apache.org/index.html
2. Enterprise --> Staging
3. Staging tab --> Name column --> org.apache.syncope
4. Right click on the closed org.apache.syncope-XXX staging repo and select Drop.
Make the required updates that caused the vote to be canceled.
Spin another release attempt!