Automated builds, tests, and quality gates are key to software quality – report

When it comes to software quality, the top three practices cited for production readiness include automated building and deploying, automated tests, and automated quality gates for every merge request, according to the JetBrains Qodana State of Software Quality Report 2024. In addition the report found that implementing automated quality gates for each merge request, using feature flags for controlled rollouts, and manual testing for dedicated QA/testing teams were the biggest challenges.

Released February 4, the JetBrains report includes findings from a survey of 808 developers, QA engineers, and other technical professionals done in July and August 2024. JetBrains, which produces the Qodana code quality platform, found that there was a clear shift toward more rigorous practices as products evolve from launch to maturity. This shift highlights the increasing complexity of maintaining code quality as companies grow, the report said.

Asked which software development processes led to production readiness, 64% of respondents said automated building and deploying was “very effective,” followed by 63% for automated tests and 55% for automated quality gates for every merge request. Asked which practices they desired but were lacking for assessing production readiness, 27% said they would implement automated quality gates if they had more resources and time, while 21% expressed the desire to use feature flags and 19% reported a need for manual testing by a dedicated QA/testing team. JetBrains also found that companies typically implement four to six mandatory checks as part of the pre-production quality assurance process. Chief among these was passing all unit tests written for the existing code.

The report also offers recommendations for making the adoption of new practices manageable. These include focusing on new and changed code only and starting with low-risk areas of newly introduced products or processes. Organizations also should choose tools that are readily available to developers.

Other findings in the JetBrains Qodana State of Software Quality Report 2024:

Common foes of software quality include technical challenges and organizational and process issues.

For pre-production readiness, data security and privacy issues were considered a “must fix” by 60% of respondents.

39% of respondents use manual testing by a dedicated team member and find it effective. 

39% of respondents said manual testing required too much manual effort or workaround solutions — more than any other practice.

61% of teams use automated tests and find them effective in overall software quality assurance.