If you are a user of Seapine Software’s TestTrack® tool, you are most likely using some of its basic workflow data capture, display, and reporting features to manage your software development projects. If you are willing to be adventurous, you can go beyond the basics and configure and customize TestTrack’s native workflow and time-tracking fields and also create new user-defined custom fields. This can provide you with powerful new metrics to help you better manage these same software projects. You can also take advantage of TestTrack’s Live Chart reporting feature to graphically display this information in a number of formats. Let’s look at some possibilities available to you.
Time-Tracking
TestTrack workflow events may be configured to include time-tracking fields, such as the estimated time needed to work on a TestTrack item, as well as the actual time spent working on that item. These items may be Tickets, Test Cases, Test Runs, Requirements, or Requirements Documents. To configure a workflow event to include a time-tracking field, select Tools>Administration>Workflow>Events, select the Type of TestTrack item (ex: Tickets), then select the Event to customize (ex: Customer Verify). On the Details tab for this event, for Time Tracking, select to display this time tracking field using the sum of hours from all events of this type, then click OK. You may now display this time-tracking field in TestTrack reports.
You can also configure which time-tracking fields represent the total time you estimated you would need to work a TestTrack item and which fields represent the total actual time you spent working on that item. To do this, select Tools>Administration>Project Options, then under Time Tracking select the TestTrack item to configure (ex: Tickets). Select which time-tracking fields represent Actual Hours fields and which ones represent Estimated Hours fields, and then click OK. You can now display the estimated and actual effort hours for these TestTrack items in TestTrack reports, as shown below. You can also display the average value for these items in these reports.
The following Live Chart report shows that actual time spent on one of Sprint 7’s User Stories was equal to, or slightly less than, the estimated time. The average estimate to do work, 1.8 hours, slightly exceeded the average actual time to do the work, 1.3 hours.
Example: Estimated vs Actual Effort to Deliver User Stories in a Sprint
Now you can define a simple TestTrack customized metric to track how closely the estimated effort to do work compares to the actual time it took to do that work. We will define it as follows:
Estimation Factor = (Estimated Hours) / (Actual Hours).
An Estimation Factor value greater than 1 indicates that the estimated time to do work was greater than the actual time it took. Conversely, a value less than 1 indicates that the actual effort took less than estimated.
To set up this custom field in your TestTrack project, select Tools>Administration>Custom Fields. On the Setup Custom Fields screen, select the Type of TestTrack item for which you want to create this field. For this example we will select Tickets. You will see a list of Active Fields representing any previously-created and activated custom fields. Select Add. On the Add Custom Fields screen you will define “Estimation Factor” as a Calculated Numeric field, and then select Edit to enter the calculated custom field formulas to define it. After entering your customized formula, select OK on the Add Custom Fields and Setup Custom Fields windows. TestTrack will ask you if you want to recalculate all calculated fields. Select Recalculate now and then select OK.
TestTrack calculated custom field formulas use the ECMAScript language, which is a subset of JavaScript. If you are not familiar with ECMAScript, refer to the ECMA-262 Standard Language Specification for information. You may also find descriptions of these formulas in Appendix F: Calculated Custom Field Functions, in the TestTrack User Guide (Version 2013.1 or later), or in TestTrack 2014 online Help topic: Reference: Calculated Custom Field Functions.
Example: TestTrack custom field definition for Estimation Factor metric
Now that you have built this metric, you can use it in TestTrack reports to assess how well your effort estimations are compared to the actual efforts expended. You can quickly create a Live Chart (or other) report with this metric and apply filters to the data sources to further refine and target the report for your project management information needs.
The following Live Chart report shows the Estimation Factors for all User Stories for all Sprints for one project, as well as the average value for this metric for the displayed data. You can also drill down to look at this metric for individual Sprints simply by creating and applying Sprint-specific filters.
The average Estimation Factor for this project was approximately 1.5, indicating the estimates to do the work were about 50% greater than the actual time expended. For User Stories where the Estimation Factor value is very different than 1, some root cause analysis may be in order. In particular, why was the estimated effort for User Story 507 17 times larger than the actual effort?
Example: Estimation Factors for all User Stories in a project
We have looked at just one way you can customize TestTrack to help you better manage your software projects. You now have metrics to monitor and manage project effort and duration in terms of estimated and actual effort, and you have seen how you can create effective Live Chart reports to display this information for review and assessment.
There is much more you can do to tailor TestTrack to better serve your project management information needs. In future articles we will look at how you can apply time-tracking metrics to Agile projects, and how you can characterize your software projects in terms of their associated original work and any subsequent rework.