Automated testing is widely used to identify regressions in libraries and applications in different environments and with varying configuration.
No matter whether you employ [Test-Driven Development], have a set of acceptance tests from [Behaviour-Driven Development], or use a suite of functional tests to ensure your application works as you expect, the goal of automated testing is to quickly provide engineers with reliable information on potential problems.
Continous Integration (CI) is the practice of continuously (and automatically) testing every change a developer makes. So automated tests become an integral part of the development process providing direct feedback on changes made.
The Need For Testing
To get the highest quality of feedback from automated tests, the environment the tests run in should match the target environment as closely as possible. For an application, this means that you should ideally share the same provisioning code (e.g. Chef, Puppet, Ansible, etc.) between your production and testing environment. You should also ensure that the versions of installed software matches. While for libraries, you should try to cover all supported platforms with as many different configurations as possible.
Using the Lock File for Tests
Davey Shafik’s article on [Composer’s Lock File] explains the typical usage of composer
update. The key takeaway is that developers should run
composer update manually to explicitly update individual dependencies while
composer install should be used in automated processes. This principle includes automated test environments.
composer install with a lock file will reproduce the
vendor directory in exactly the way it is in your production environment. This greatly reduces the risk of overlooking application bugs introduced through minor patch release updates of dependencies.
A library usually doesn’t have a single target platform that you could try to reproduce closely in your tests. Nevertheless, following the same principle and checking your
composer.lock file into the library’s repository has some advantages:
- When your automated tests fail, you can be certain that the failure does not result from installing a dependency with a different version than the one you used locally
- When users run your tests on their platform and submit a bug report, you can be certain the issue is a result of the platform and not of the installed versions of dependencies
- You explicitly document the precise versions you are testing against in your
composer.lockfile and your repository’s version history documents when this changes
When developing a library, you may explicitly want to support and test multiple versions of a dependency. For example, Symfony bundles usually support multiple versions of the Symfony framework.
Rather than trying to detect bugs randomly by having some users use different versions because there is no lock file in the repository, you can explicitly swap out these dependencies for a different version using
composer require some/dependency:1.3.4.
You can then do this for a predefined list of versions that you would like to run your test suite against. The require command updates the
composer.json file and runs
composer update some/dependency for you. But even in this case, the rest of the lock file remains unaltered, giving you all the advantages listed above.
A lot of libraries currently use
composer require --no-update, as described above, and then call
composer install. This only has an effect because these libraries fail to provide a lock file, so that
composer install actually has to run a full
composer update which gives you no certainty about which versions get installed at all.
Composer also offers a new `--prefer-lowest` option on `composer update` which will attempt to install the lowest version of all dependencies that still match all version constraints you have specified in your `composer.json`. This flag can be helpful in ensuring that your library actually works with all your minimum requirements.
## Improving Build Performance
It is difficult to integrate continuous builds into your development process if the build process takes a long time and doesn't deliver direct feedback. Hence, the performance of individual builds is important to the successful use of CI. Fortunately, using the right options when running Composer can improve performance a lot.
Apart from the advantages already discussed, installing dependencies from a lock file is significantly faster than running `composer update`. All package metadata is stored in the lock file itself, so `composer install` can immediately start downloading the actual dependencies instead of loading metadata and calculating dependency information.
Composer keeps a local cache. The cache consists of two main components: copies of downloaded files which can be reused and avoid unnecessary waiting time while dependencies are downloaded. And copies of package metadata that needs to be loaded from repositories like Packagist to calculate dependencies. This cache needs to remain between builds to be effective.
How sharing of data between builds can be accomplished depends on the tool you use to run your automated tests. On [Travis CI], a popular CI service for open source PHP libraries, you can [configure caching] to keep composer metadata. Another option is the [Vagrant Cachier] plugin, which can be used to share the Composer cache with your vagrant boxes, improving performance on reprovisioning.
By default, composer runs in dev mode. Dev mode means that dependencies listed in `require-dev` will be installed, but it also means that Composer will prefer cloning repositories over downloading archives. This can be helpful during development to directly make and commit changes within your vendor directory. However, during testing you can use `--prefer-dist` to download package archives, which is usually faster than cloning entire repositories.
You may prefer to use `--no-dev` for your automated tests to more closely match the production environment, but that would mean you cannot rely on testing tools to be installed from your `require-dev` dependency list. At the same time, `require-dev` packages do not influence the dependency resolution of packages in `require`. Even if you install `require-dev` packages in your test environment, all other dependencies will match the exact versions of your production environment.
Lastly, the `--optimize-autoloader` option creates a faster version of the Composer autoloader, which takes longer to generate but results in faster class to file lookups. This option is always helpful in production environments, and may improve the overall performance of automated tests as well.
Continous Integration of projects using Composer can be improved through consistent use of a `composer.lock` file—even for libraries. Carefully reviewing the different options available for `composer install` allows you to speed up build times and provide developers with more direct feedback on builds. Additionally, you can use `composer require` to explicitly test various versions of dependencies without modifying the rest of your `composer.lock`.
P.S. What did you think of this post? Got any questions or tips to share? Let us know by sharing a comment.