The moment we started working in Feather, we got the feeling it was much faster than ActiveCollab 4.
So we thought it would be fun to see how Feather compares, performance-wise, to the previous version. To do that, we made a quick experiment.
Designing the Selenium Test
We have put together a quick test scenario: open a new project and make a task in it. Then run the scenarios in both Feather and ActiveCollab 4, and measure how long it takes to complete.
To remove the human error factor, we used Selenium IDE (not to be confused with Selenium Webdriver) - a web browser automation and test tool, available only as a Firefox extension. It doesn't require any programming experience so anyone web-savvy can use it.
When you open the extension window, you press the record button and perform actions in the browser (eg. going to a website, typing, clicking buttons and links). Selenium then records those actions as pieces of code. After recording, you can press playback and all the actions you recorded will be performed again. For more information, visit Selenium IDE documentation page.
Our Selenium tests looked like this:
We ran each test 10 times, put the time records in a spreadsheet, and did a simple statistical analysis.
On average, Feather was 18.4% faster than ActiveCollab 4 - at best 28.8% faster, and at worst 10.6% faster.
To open a project and task in it, ActiveCollab 4 needed 6.3 seconds and Feather did it in 5.3 seconds.
Here are the full results:
Raw vs. Perceived Performance
The test itself was very basic, designed to quickly gauge the performance.
We thought Feather would be much faster than 18%, but it was just psychology. Both versions operate pretty fast - but Feather wins out in one regard: you don't feel it takes any time.
Even though AC4 operated fast (technically) - it felt slower due to its UI complexity. Feather, on the other hand, had close performance but it just felt fast and light. The key to this phenomenon is the difference between Raw vs. Perceived performance.
When you want to open a task in AC4, a lot of places want your attention, leading to cognitive overload. That's not the case with Feather - there's no clutter and making a task is more intuitive. If you know you that tasks are project-based, you go to a project and you're greeted with a task form right in front of you saying + Add a Task.
While it may load and perform actions faster than AC4, it's the ease of use we were really aiming at.