|
|
Linux and Open Source News for 29th December 2007
|
Distro Watch
|
|
Source: LinuxTracker.org Category: sidux Size: 1.69 GB Status: 3 seeders and 7 leechers Added: 2007-12-29 21:02:57
Source: LinuxTracker.org Category: Mepis Size: 309.35 MB Status: 7 seeders and 14 leechers Added: 2007-12-29 18:11:44
Source: LinuxTracker.org Category: Ubuntu Size: 1.64 GB Status: 1 seeders and 139 leechers Added: 2007-12-29 15:03:16
Source: LinuxTracker.org Category: Ubuntu Size: 1.73 GB Status: 1 seeders and 14 leechers Added: 2007-12-29 14:24:11
Source: LinuxTracker.org Category: Ubuntu Size: 1.73 GB Status: no seeders and 9 leechers Added: 2007-12-29 11:08:40
Source: LinuxTracker.org Category: PCLinuxOS Size: 3.65 GB Status: 5 seeders and 130 leechers Added: 2007-12-29 09:04:37
Source: freebsd Ken Smith has announced the availability of the first release candidate for FreeBSD 7.0: "FreeBSD 7.0-RC1 available. The ports team has gotten the release package sets built for most of the architectures (sparc64 is still a long way off) so we have begun including the pre-built packages on .
Source: pclinuxos The PCLinuxOS development community has announced the release of PCLinuxOS "GNOME 2.21.2", a remastered, community edition of PCLinuxOS containing the latest beta version of GNOME 2.22: "Ken Dotson has announced the release of PCLinuxOS GNOME edition 2.21.2. Featuring kernel 2.6.22.15, GNOME 2.21.2, GNOME office applications, Firefox 2.0.0.11, Frostwire, .
Source: grml Michael Prokop has announced the first release candidate for grml 1.1, a Debian-based live CD containing a large collection of GNU/Linux software and custom scripts for system administrators: "The first release candidate of grml 1.1 is available. New features: several improved, updated and extended configuration files; new grml-scripts .
Source: debian Martin Schultze has announced the availability of the seventh revision of Debian GNU/Linux 3.1 "Sarge", the project's legacy (or "old stable") branch: "The Debian project is pleased to announce the seventh update of its old stable distribution Debian GNU/Linux 3.1. This is the first time we update the .
|
|
News for nerds, stuff that matters
|
|
Source: Slashdot: Linux drewmoney writes "Misconceptions about what 'open source software' means has made elements of the US Defense Department reluctant to deploy in a live environment. DoD proponents of shared-source projects are now working to reverse this trend by educating IT decision-makers and demonstrating OSS usefulness. 'The cost of cleaning up a "network spill" that introduces classified material on an unclassified network is running about US$11,000 per incident on the Navy/Marine Corps Intranet (NMCI), so the free Secure Save tool could produce monetary savings for the Navy. Additionally, it would cover more file formats than the costly commercial redaction product currently available on the NMCI.'"Read more of this story at Slashdot.
Source: Slashdot: Linux News.com is reporting that the next version of Ubuntu will see KDE unsupported, but only for the time being. Because of the dramatic changeover from KDE 3.5 to 4.0, Ubuntu sponsor Canonical is unwilling to initially support the popular Linux GUI. Gnome will still be supported, and the company expects to return support to kubuntu soon. "Developer interest is focused on KDE 4.0, but it's not mature enough yet to use in the next KDE-based variation of Ubuntu, called Kubuntu, Scott James Remnant, leader of the Ubuntu Desktop team, said in an explanation to a Kubuntu mailing list. But most Kubuntu developers adding features "upstream" of today's products are focused on KDE 4.0, meaning that it's risky to release a long-term support version based on 3.5."Read more of this story at Slashdot.
Source: Slashdot: Linux christian.einfeldt writes "Computers and handheld devices running default GNU Linux or Unix OSes have swept Amazon's 'best of' list for 2007, according BusinessWire.com for 28 December 2007. Best selling computer? The Nokia Internet Tablet PC, running Linux. Best reviewed computer? The Apple MacBook Pro notebook PC. Most wished for computer? Asus Eee 4G-Galaxy 7-inch PC mobile Internet device, which comes with Xandros Linux pre-installed. And last, but not least, the most frequently gifted computer: The Apple MacBook notebook PC."Read more of this story at Slashdot.
|
|
The O'Reilly Network ONLamp Articles and Weblogs
|
|
Source: ONLamp.com Just before I started my job at the BBC, one of our developers committed code which reduced our test suite run time from an hour and twenty minutes down to twenty-two minutes. One of my first tasks was to improve that. However, improving performance begs the old question of “cpu or developer performance?” Both are equally important, but I’ll just talk about making the tests run faster. Right now, it looks like we’re on track to get our test suite to run in under ten minutes. Here’s how we did this.
Don’t Recreate the Database
The biggest win, of course, was to reduce the 80 minute runtime to 22 minutes. The problem was that many tests would set up and tear down the entire test database. While this is, in theory, a clean solution, the trade off is that it’s very slow. The solution was to divide tables into ’static’ tables whose data does not change or changes infrequently (ISO country codes, for example) and ‘dynamic’ tables which change constantly (customer orders). Each test program which uses the database creates a database handle and when the handle goes out of scope it does the following:
Disable foreign keys
Truncate all dynamic tables
Enable foreign keys
Scream if any static tables have changed
That was the bulk of the savings and for many slow test suites, this could be enough. However, we still had a 22 minute test run and we were targeting 10 minutes. With five developers, every time they each run a test suite, that’s an extra hour of waiting for information.
Don’t Duplicate Tests
The next step was examining our tests. I noticed that we had many tests in the form of test/yaml pairs and they looked like this:
t/unit/customer.t
t/unit/customer.yml
t/unit/order.t
t/unit/order.yml
Each of the .t files would look like this:
use Our::Test::API;
Our::Test::API-run;
Or this:
use Our::Test::System;
Our::Test::System-run;
Each of those classes (we had several, but different names from what’s here) inherited from Our::Test and used the name of the test program to determine the name of the YAML file. I deleted all of them and made one test program. I turned Our::Test into a factory which returned the correct class for the YAML being used.
use Our::Test;
foreach my $yml (get_yaml_tests()) {
Our::Test-new($yml)-run;
}
The test suite now ran in 16 minutes.
How did that save six minutes? Well, instead of invoking the perl interpreter for a bunch of test files and reloading all of the related classes every time, perl was invoked once and each class was loaded only once. This is similar to how Perl’s Test::Class can dramatically improve a test suite’s performance.
When factoring out duplicate code like this, you’ll also often discover that you’re factorinng out duplicate tests. Running the same test more than once is often meaningless.
Don’t Reload Anything
Now that the obvious performance issues were out of the way, I need to figure out how to get squeeze out an extra six minutes of performance. I had thought a lot about using Test::Class because it’s an excellent tool, but this meant rewriting a bunch of working tests and our performance wasn’t bad enough that I could really justify that. So I started writing Test::Aggregate. This doe something very similar to what the YAML tests did but in a more generic manner.
The idea is to slowly move your tests, one by one (this eases debugging) into a separate test directory and write a driver test like the following:
use Test::Aggregate;
my $tests = Test::Aggregate-new( {
dirs = 'aggtests',
set_filenames = 1,
} );
$tests-run;
What this does, effectively, is concatenate all of the tests together into one program and run them all at once. Of course, there’s a lot of magic under the hood to make this work, but just moving a few test programs over to this gained us an extra two minutes. One experimental branch had the test suite completing in under nine minutes (with a couple of strange failures). Not bad for being at 80 minutes a couple of months ago, eh?
Sometimes It’s OK to Fail
For diehard testers like myself, that sounds like heresy, but it’s true for a very limited set of circumstances. For example, we have one very slow test which loads every module and verifies that it uses strictures. While this is an important thing to do, disabling strictures does not automatically mean your software is broken. These tests are very slow and we’re likely to move them into a separate directory which only gets run during integration.
We also have tests which verify that our documentation is complete and properly formatted. If these tests fail, it doesn’t mean that the software is broken. These tests can also be moved into a separate ‘integration’ directory. While developers should still run all tests before checking in their code, it’s more important that they run those tests which really check that the software is doing what it should.
Using this and previous techniques, it’s possible we’ll have our core test suite running in under seven minutes.
Why Is This Important?
Why is a fast running test suite so important? At one company I worked with, the test suite took an hour and a half to run and developers would often check in code when they verified that their tests passed. Running the entire test suite took so long that they skipped it. More than once bugs got into production because no one was running the entire suite!
The faster your test suite runs, the more likely developers will run it. Proper test management is key to developing good software and this is one of the many areas developers should focus on.
In a later post, I hope to document ways that developers can develop tests easier (it will be far more Perl-specific, though). Just as a faster test suite is more likely to be run, easy-to-write tests are more likely to get written.
|
|
