![\"trace](\"https:\/\/www.jamasoftware.com\/media\/2023\/02\/2024-10-23research-note-trace-score.jpg\")
<\/h2>\n<\/h2>\nOne of the main causes of rework, delays, and cost overruns in product development is the creation of new requirements late in the process. This is a well-known risk in product development, but what management practices can empirically be shown to reduce this known risk?<\/p>\n
Using our proprietary database of metadata from over 50,000 complex product development projects, we were able to determine that the Trace Score is an empirical method to reduce late requirements. In fact, teams that maintain a high Trace Score reduce the burden late requirements have on their project by 67% compared to teams with low trace scores.<\/p>\n
Jama Software has the world\u2019s largest, live dataset of engineering process performance with over 50,000 engineering projects updated and growing continuously. Leveraging this dataset, it is now possible to determine empirically which management practices improve the performance of the product development process. To learn more about our benchmarking, please review our Traceability Benchmarking Report<\/a>.<\/p>\n In this analysis we will explore three key questions:<\/p>\n For the purpose of this analysis, we define \u201clate requirements\u201d as those requirements created after the completion of a project\u2019s requirement decomposition phase which we estimate as spanning the middle 50% of all requirement creation activity (creation and refinement). To illustrate what late requirements look like, we show two actual projects below with requirement activity plotted over time.<\/p>\n <\/p>\n In the Timely Project, requirement creation occurs in a defined requirement decomposition phase to form a necessary and sufficient set of requirements, with very few requirements being added after the fact (e.g. in fig (a), only 1.3% of requirements created late). In the Late Project\u2019, requirement creation bleeds into future phases of the project, leading to a significant amount of late requirements (e.g. in fig (b), 9.2% of requirements are created late).<\/p>\n We can measure the outsized burden late requirements have on project teams, which we have illustrated for our two projects below. We define late requirement burden as the total number of requirement activities (creation and refinement) attributed to late requirements as a percentage of all requirement activity.<\/p>\n In the Timely Project, minimal late requirements enable better forecasting of project completion and limit the rework and cost brought on by late requirements (e.g. in fig (c), late requirements only create an additional 8% burden).<\/p>\n In the Late Project, the high volume of late requirements makes it much harder to forecast project completion as the scope of the project is constantly changing, and project teams need to accommodate the late requirements (e.g. in fig (d), late requirements contribute an additional 31% burden).<\/p>\n Unsurprisingly, this additional burden of late requirements has an impact during testing for requirement validation. In our actual project examples, the Late Project has a test failure rate over 3x that of the Timely Project.<\/p>\n A core theorem of Systems Engineering is that maintaining high requirement traceability from the start of a project reduces the risk of late requirements and negative product outcomes. With our project dataset we can now test this theorem empirically. We define traceability as a measure of a project\u2019s \u2018expected\u2019 traceability that has actually been established and calculate the Trace Score as follows:(1)<\/p>\n For our example projects, the Timely Project achieved a Trace Score over 6X that of the Late Project; suggesting that maintaining a high Trace Score throughout the project reduces the risk of late requirements.<\/p>\n To further determine if Trace Score correlates to late requirements, we divided our dataset of projects into quartiles based on their Trace Scores (Quartile 1 = bottom 25% trace score, Quartile 4 = top 25% trace score) and then compared the distribution of \u2018Late Requirements Burden\u2019 across these quartile groups. What we found is that projects within the bottom traceability quartile had a median Late Requirements Burden 3x greater than those in the top traceability quartile. In other words, the evidence supports that projects managed with higher traceability generally experience less risk from late requirements.<\/p>\n Our analysis has shown that late requirements negatively impact projects and that managing projects through a Trace Score is the only empirical way to reduce the risk of late requirements. Below you can see how one can measure the Trace Score over time as a project progresses to ensure system engineering best practices are being followed. A low or falling Trace Score can quickly identify areas to address to reduce the risk of late requirements.<\/p>\n Here you can see how managing the Trace Score directly as the project is underway would have identified the risk early in the Late Project.<\/p>\n To learn more about achieving Live Traceability\u2122<\/a> on your projects, please reach out for a consultation.<\/a><\/p>\n <\/p>\n <\/p>\nThe Empirical Questions<\/h3>\n
\n
What are late requirements?<\/h3>\n
Requirement Creation Over Time<\/h6>\n
![\"\"](\"https:\/\/www.jamasoftware.com\/media\/2023\/02\/Requirement-Chart-814x1030.png\")
<\/p>\n
\nRELATED: <\/strong>Requirements Traceability Benchmark<\/a><\/span><\/span><\/h4>\n
\nHow do late requirements negatively impact projects?<\/h3>\n
Impact of Late Requirements on Project Team Activity Burden<\/h6>\n
![\"\"](\"https:\/\/www.jamasoftware.com\/media\/2023\/02\/Impact-Chart-762x1030.png\")
<\/p>\n![\"percentage](\"https:\/\/www.jamasoftware.com\/media\/2023\/02\/Rate-percentage-chart-1030x137.png\")
<\/p>\n
\nRELATED: <\/strong>Unlocking The Power of Live Traceability with Jama Connect<\/a><\/span><\/span><\/h4>\n
\nDoes maintaining a high Trace Score reduce the risk of late requirements?<\/h3>\n
![\"established](\"https:\/\/www.jamasoftware.com\/media\/2023\/02\/established-over-expected-1030x175.png\")
<\/p>\n![\"traceability](\"https:\/\/www.jamasoftware.com\/media\/2023\/02\/Traceability-Score-1030x144.png\")
<\/p>\nRecommendation<\/h3>\n
![\"Benchmark](\"https:\/\/www.jamasoftware.com\/media\/2023\/02\/Benchmark-Score-Chart-1030x577.png\")
<\/p>\nInterested in learning more? Download the entire Research Notes: Trace Score datasheet<\/span> HERE.<\/u><\/a><\/span><\/h3>\n
\n<\/div>\n