This page is divided in four sections. The first one lists the tasks that must be performed before creating a package. The second section includes a list of tasks to get a valid package. The third one to get the package published and the release announced. The last one lists and suggests some possible cleanup tasks to be done after releasing.
Note
Please note that this is not a complete list of tasks. Please feel free to improve it.
Before starting the release process it is necessary to perform some previous tasks.
We need to give localizers enough time to localize Pootle. They need time to do the actual translation and to feedback on any errors that they might encounter.
First upload the new translations:
Create the new templates:
$ git clone git@github.com:translate/pootle.git pootle-translations
$ cd pootle-translations
$ make pot
Upload the templates to Pootle for translation.
Update current translations against templates either on Pootle or in code and commits these updated files to Git.
Announce the new translations on the following channels:
A string freeze would normally run between an RC and a final version. We want to give a string freeze at least 2-4 weeks before a release. They must be announced, explicitly stating the duration, on the translate-announce@lists.sourceforge.net and the translate-pootle@lists.sourceforge.net mailing lists.
Note
If we do have a string freeze break then announce it to people. The string freeze breaks usually are only allowed to fix mistakes on the translatable strings.
The first steps are to create and validate a package for the next release.
We work from a clean checkout to ensure that everything you are adding to the build is what is in the repository and doesn’t contain any of your uncommitted changes. It also ensures that someone else could replicate your process.
$ git clone git@github.com:translate/pootle.git pootle-release
$ cd pootle-release
$ git submodule update --init
It is extremely advisable to create a Pull Request with the changes necessary for the release. This means that it is best to create a working branch to clearly separate those changes from master branch, which can be altered while doing the release, and ease rebasing if needed.
$ git remote add my-fork git@github.com:my-fork/pootle.git
$ git fetch my-fork
$ git checkout -b releasing
Update the minimum version number for the requirements in:
requirements/
pootle/checks.py
docs/server/requirements.rst
Make sure version numbers displayed on documentation examples match the latest requirements on the above files.
Update any copyright dates in docs/conf.py:copyright
and anywhere else
that needs fixing.
$ git grep 2013 # Should pick up anything that should be examined
Create ~/.pootle/pootle_build.conf
with the following content:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
"""Configuration file to build Pootle.
Must be placed in ~/.pootle/pootle_build.conf
"""
# Django now requires to set some secret key to be set.
SECRET_KEY = '__BuildingPootle_1234567890__'
# Silence some checks so the build output is cleaner.
SILENCED_SYSTEM_CHECKS = [
'pootle.W004', # Pootle requires a working mail server
'pootle.W006', # sqlite database backend is unsupported
'pootle.W010', # DEFAULT_FROM_EMAIL has default setting
'pootle.W011', # POOTLE_CONTACT_EMAIL has default setting
]
The quality checks descriptions are kept as a static HTML page that has to be regenerated in order to ensure the descriptions match the currently available quality checks.
$ mkvirtualenv build-checks-templates
(build-checks-templates)$ pip install --upgrade setuptools pip
(build-checks-templates)$ pip install -r requirements/build.txt
(build-checks-templates)$ export POOTLE_SETTINGS=~/.pootle/pootle_build.conf
(build-checks-templates)$ DJANGO_SETTINGS_MODULE=pootle.settings ./setup.py build_checks_templates
(build-checks-templates)$ deactivate
$ unset POOTLE_SETTINGS
$ rmvirtualenv build-checks-templates
Update the translations from the Pootle server
Download all translations
$ make get-translations
Update pootle/locale/LINGUAS
to list the languages we would like to
ship. While we package all PO files, this is an indication of which ones we
want packagers to use. The requirement is roughly 80% translated with no
obvious variable errors. Languages with a small userbase can be included.
$ make linguas
Check the output and make any adjustments such as adding back languages that don’t quite make the target but you wish to ship.
Build translations to check for errors:
$ ./setup.py build_mo # Build all LINGUAS enabled languages
$ ./setup.py build_mo --check # Not all of these are errors
We create our release notes in reStructured Text, since we use that elsewhere and since it can be rendered well in some of our key sites.
First we need to create a log of changes in Pootle, which is done generically like this:
$ git log $previous_version..HEAD > docs/releases/$version.rst
Or a more specific example:
$ git log 2.5.0..HEAD > docs/releases/2.5.1.rst
Edit this file. You can use the commits as a guide to build up the release notes. You should remove all log messages before the release.
Note
Since the release notes will be used in places that allow linking we use links within the notes. These should link back to products websites (Virtaal, Pootle, etc), references to Translate and possibly bug numbers, etc.
Read for grammar and spelling errors.
Note
When writing the notes please remember:
We create a list of contributors using this command:
$ git log 2.5.0..HEAD --format='%aN, ' | awk '{arr[$0]++} END{for (i in arr){print arr[i], i;}}' | sort -rn | cut -d\ -f2-
Bump the version for each of the apps so the caches are caned after upgrade:
apps.py
for each of the apps that have a versionUpdate the version number in:
pootle/__init__.py:VERSION
docs/server/installation.rst
and
docs/server/upgrading.rst
The version tuple should follow the pattern:
(major, minor, micro, candidate, extra)
E.g.
(1, 10, 0, 'final', 0)
(2, 7, 0 'alpha', 1)
When in development we use ‘alpha’ with extra
of 0. The first release of a
minor
version will always have a micro
of .0
. So 2.6.0
and
never just 2.6
.
Most likely your system will provide a nodejs version older than the one that is required. nvm is a tool that allows to quickly install and switch nodejs versions.
Follow the nvm installation instructions.
Building is the first step to testing that things work. From your clean checkout run:
$ mkvirtualenv build-pootle-release
(build-pootle-release)$ nvm install stable
(build-pootle-release)$ pip install --upgrade setuptools pip
(build-pootle-release)$ pip install -r requirements/build.txt
(build-pootle-release)$ pip install -e .[dev]
(build-pootle-release)$ export PYTHONPATH="${PYTHONPATH}:`pwd`"
(build-pootle-release)$ export POOTLE_SETTINGS=~/.pootle/pootle_build.conf
(build-pootle-release)$ ./setup.py build_mo # Build all LINGUAS enabled languages
(build-pootle-release)$ ./setup.py build_mo --all # If we are shipping an RC
(build-pootle-release)$ make build
(build-pootle-release)$ deactivate
$ unset POOTLE_SETTINGS
This will create a tarball in dist/
which you can use for further
testing.
Note
We use a clean checkout just to make sure that no inadvertant changes make it into the release.
The easiest way to test is in a virtualenv. You can test the installation of the new release using:
$ mkvirtualenv test-pootle-release
(test-pootle-release)$ pip install --upgrade setuptools pip
(test-pootle-release)$ pip install dist/Pootle-$version.tar.bz2
(test-pootle-release)$ pip install mysqlclient
(test-pootle-release)$ pootle init
You can then proceed with other tests such as checking:
Documentation is available in the package
Assets are available in the package
Quick SQLite installation check:
(test-pootle-release)$ pootle migrate
(test-pootle-release)$ pootle initdb
(test-pootle-release)$ pootle runserver --insecure
(test-pootle-release)$ # Browse to localhost:8000
MySQL installation check:
Create a blank database on MySQL:
mysql -u $db_user -p$db_password -e 'CREATE DATABASE `test-mysql-pootle` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;'
Change the database settings in the settings file created by
pootle init
(by default ~/.pootle/pootle.conf
)
to use this new MySQL database
Run the following:
(test-pootle-release)$ pootle migrate
(test-pootle-release)$ pootle initdb
(test-pootle-release)$ pootle runserver --insecure
(test-pootle-release)$ # Browse to localhost:8000
Drop the MySQL database you have created:
mysql -u $db_user -p$db_password -e 'DROP DATABASE `test-mysql-pootle`;'
MySQL upgrade check:
Download a database dump from Pootle Test Data repository
Create a blank database on MySQL:
mysql -u $db_user -p$db_password -e 'CREATE DATABASE `test-mysql-pootle` DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci;'
Import the database dump into the MySQL database:
mysql -u $db_user -p$db_password test-mysql-pootle < $db_dump_file
Run the following:
(test-pootle-release)$ pootle migrate
(test-pootle-release)$ pootle runserver --insecure
(test-pootle-release)$ # Browse to localhost:8000
Drop the MySQL database you have created:
mysql -u $db_user -p$db_password -e 'DROP DATABASE `test-mysql-pootle`;'
Check that the instructions in the Installation guide are correct
Check that the instructions in the Upgrade guide are correct
Check that the instructions in the Hacking guide are correct
Meta information about the package is correct. This is stored in
setup.py
, to see some options to display meta-data use:
$ ./setup.py --help
Now you can try some options like:
$ ./setup.py --name
$ ./setup.py --version
$ ./setup.py --author
$ ./setup.py --author-email
$ ./setup.py --url
$ ./setup.py --license
$ ./setup.py --description
$ ./setup.py --long-description
$ ./setup.py --classifiers
The actual long description is taken from /README.rst
with some
tweaking for releasing.
Finally clean your test environment:
(test-pootle-release)$ deactivate
$ rmvirtualenv test-pootle-release
Once we have a valid package it is necessary to publish it and announce the release.
Before merging the changes you must create a Pull Request to ensure that all tests and checks pass:
$ git push my-fork releasing
Note
Of course you must wait until all automatic checks pass.
You should branch only when you are releasing the first stable version of a new version series, since betas and release candidates can be developed in the master branch. To branch do:
$ git checkout -b stable/2.8.x
$ git push origin stable/2.8.x
If you branch you will want to update the README.rst
file so that it
points correctly to branched versions of badges and documentation. Review and
test the actual links created, you don’t need to commit everything.
$ workon build-pootle-release
(build-pootle-release)$ ./setup.py update_readme -w
(build-pootle-release)$ git diff README.rst
(build-pootle-release)$ git commit README.rst -m "Adjust README to branch"
(build-pootle-release)$ deactivate
Also if you branch you will want to limit requires.io to requirements in the branch. To do so check how it was done in this commit:
$ nano .requires.yml
$ git add .requires.yml
$ git commit -m "Requirements: Limit requires.io to branch requirements"
You should only tag once you are happy with your release as there are some things that are difficult to undo:
$ git tag -a 2.8.0 -m "Tag version 2.8.0"
$ git push --tags
We need a tagged release or branch before we can do this. The docs are published on Read The Docs.
Use the admin pages to flag a version that should be published. When we have
branched the stable release we use the branch rather then the tag i.e.
stable/2.5.x
rather than 2.5.0
as that allows any fixes of
documentation for the 2.5
releases to be immediately available.
Change all references to docs in the Pootle code to point to the branched version as apposed to the latest version.
Deactivate documentation that is no longer applicable.
Note
You need a username and password on Python Package Index (PyPI) and have rights to the project before you can proceed with this step.
These can be stored in $HOME/.pypirc
and will contain your username
and password. Check Create a PyPI account
for more details.
Run the following to publish the package on PyPI:
$ workon build-pootle-release
(build-pootle-release)$ pip install --upgrade pyopenssl ndg-httpsclient pyasn1 twine
(build-pootle-release)$ twine upload dist/Pootle-*
(build-pootle-release)$ deactivate
$ rmvirtualenv build-pootle-release
Do the following to create the release:
We use github pages for the website. First we need to checkout the pages:
$ git checkout gh-pages
_posts/
add a new release posting. Use the same text used for the
Github release description,
including the link to the full release notes.$version
as needed. See _config.yml
and
git grep $old_releaseLet people know that there is a new version:
Announce on mailing lists using plain text emails using the same text (adjusting what needs to be adjusted) used for the Github release description:
Warning
This has to be explicitly reviewed and approved by Dwayne so we don’t repeat the same email over and over.
Adjust the Pootle channel notice.
Use /topic [new topic]
to change the topic. It is easier if you copy the
previous topic and adjust it.
Email important users
Tweet about it
Update Pootle’s Wikipedia page
These are tasks not directly related to the releasing, but that are nevertheless completely necessary.
If this new release is a stable one, bump the version in master
to
{N+1}-alpha1
. The places to be changed are the same ones listed in
Up version numbers. This prevents anyone
using master
being confused with a stable release and we can easily check
if they are using master
or stable
.
After updating the release notes for the about to be released version, it is
necessary to add new release notes for the next release, tagged as dev
.
Some possible cleanup tasks:
pootle-release
checkout.master
.$version
docs rather then latest
?