Quality Assurance of Machine Learning Applications

Nowadays ML engineering and development is carried out and implemented in almost every organisation to a certain extent - in a very analog way, in the same way that software development has taken place throughout the years. ML software however, has particular characteristics which cause it to differ significantly from traditional company software development:

  • Models are based on historical data, while the data itself are constantly changing.
  • The quality of the data is taken as a given, without appropriate testing and validation.
  • Summaries of performance (accuracy, precision, F1 result) for particular data sets provide no guarantee of generalised performance for data from the real world.
  • Attempts at integration are generally underestimated and often simply entail the insertion of models into downstream processes, with the result that inflated or failed integrations have to be manually manipulated.
  • Constant further development of tools and techniques in the ML ecosystem results in a multitude of moving parts.

The aim of the joint project is to support SMEs with the special ML software development lifecycle (ML-SDLC) and the significant quality indicators which result from this. Five SMEs will work with three universites of applied sciences to develop evaluation of data quality with regard to representative coverage of the characterisation, as well as evaluation of the quality of the KI models gained during the learning process. This will ensure the product risk of the manufacturers' KI-based products and guarantee the customer quantified performance with regard to KI decisions.

Project partner:

Offenburg University, Karlsruhe University of Applied Sciences
Competion IT GmbH, Inferics GmbH, C.R.S. iiMotion GmbH, tepcon GmbH, schrempp edv GmbH

Funding

Ministry of Science, Research and the Arts, Baden-Württemberg

Contact details