Does Staff Expertise Affect Software Quality?

Software Quality

Many years ago, Capers Jones, the software metrics guru, analyzed his database of thousands of software projects for the key factors affecting “real” software quality.  “Real” software quality relates to how the software actually performed and how robust in the field.   His list in priority order was:

  1. Programmer Application (domain) Experience
  2. Programmer Technical Experience
  3. Reuse
  4. Structured Methods
  5. Stable Requirements
  6. Private Office Space
  7. Careful Reviews / Inspections
  8. Experienced Management
  9. Realistic Schedule Targets

Can we learn from his research?  Does this fit current models of development and field quality?  Do modern development and test tools affect “real” software quality?

One clear takeaway in my view is the value of training software development and test staff.  While our process may expect system engineers and safety engineers to create solid and correct inputs to software development, you just cannot duplicate the value of a software developer having domain experience.  Their ability to question, constructively criticize, and otherwise scrutinize the requirements, design, and performance of the software early during development is irreplaceable.  Clearly the company that has software developers with domain experience will produce higher quality and likely safer software than the company that does not.

So how does one obtain domain expertise if they do not have it when you hire them?  Get them into the field!  Allow software developers to spend time in customer service and complaint handling roles.  Create realistic user environments onsite and have the software developers perform user roles in simulated circumstances.  Ensure that the software developers participate in product level and system level risk analysis so that they understand and have a healthy concern for how the device can hurt people or the environment.  For perspective, read Nancy Leveson’s excellent analysis of the Therac-25 accident.

Why are requirements so important?  Could it be that “working software” cannot tell us everything about the software?  Requirements provide a natural language description of the what we expect the software to do.  For efficiency, perhaps we do not require as much description of the software where its behavior is obvious and clear from use.  But certainly behaviors related to safety and essential functionality should be “agreed upon” in natural language.  Read our post on a checklist of quality attributes for requirements.

About the author

Brian Pate helps medical device companies achieve efficient and FDA regulatory compliant product development to produce higher quality and clinically valued software. He began his career in clinical research in 1985 with the Department of Anesthesiology at UAB developing closed-loop control systems for the automated delivery of gases and control. In 1990, he made the switch from university research to the medical device industry designing control systems, communication interfaces, user interface, and other software for real-time embedded systems and clinical information systems, working for medical device companies including Johnson & Johnson, Baxter Healthcare, and GE Medical. Today, he is a Partner and the General Manager of Crisis Prevention and Recovery LLC (dba SoftwareCPR®), a general-purpose regulatory consulting firm that is recognized globally for their expertise with standards and national regulations pertaining to medical device, mobile medical app, and HealthIT software. He has taught the AAMI/FDA course on Software Regulation to FDA Reviewers at FDA and is currently the lead faculty for the public version of that course taught annually along with FDA staff. Brian served on the AAMI/FDA TIR working group that created AAMI TIR32 Guidance on the application of ISO 14971 to Software (later superseded by IEC 80002-1). He later served on the original AAMI/FDA working group that created the AAMI TIR45-2012 TIR Guidance on the use of Agile practices in the development of medical device software and is currently the co-chair leading the creation of the 2nd edition of TIR45. He has served as faculty for all offerings of the AAMI/FDA Compliant Use of Agile Methods public course. Brian also served as an instructor for the AAMI Design Controls course. He is also a member of the Underwriters’ Laboratories Standards Technical Panel 5500, Remote Software Updates. He now serves as a member of the AAMI Software Committee.

Cybersecurity Review

Our cybersecurity experts are NESSUS Pro Licensed and can quickly remediate cybersecurity deficiencies with your medical device or digital health software.  Planning, requirements, validation, and submissions – we can assist with all.

Interested in having a conversation?  Email us to arrange a Zoom meeting or call us at +1 781-721-2921.

Corporate Office

15148 Springview St
Tampa, FL 33624
Partners located in the US (CA, FL, MA, MN), Canada, and Italy.