Deciding whether an open source software (OSS) meets the required standards for adoption in terms of quality, maturity, activity of development and user support is not a straightforward process as it involves exploring various sources of information including its source code repositories to identify how actively the code is developed, which programming languages are used, how well the code is commented, whether there are unit tests etc., communication channels such as newsgroups, forums and mailing lists to identify whether user questions are answered in a timely and satisfactory manner, to estimate the number of experts and users of the software, its bug tracking system to identify whether the software has many open bugs and at which rate bugs are fixed, and other relevant metadata such as the number of downloads, the license(s) under which it is made available, its release history etc.
This task becomes even more challenging when one needs to discover and compare several OSS projects that offer software of similar functionality (e.g. there are more than 20 open source XML parsers for the Java programming language) and make an evidence-based decision on which one should be selected for the task at hand. Moreover, even when a decision has been made for the adoption of a particular OSS product, decision makers need to be able to monitor whether the OSS project continues to be healthy, actively developed and adequately supported throughout the lifecycle of the software development project in which it is used, in order to identify and mitigate any risks emerging from a decline in the quality indicators of the project in a timely manner.
Previous work in the field of OSS analysis and measurement has mainly concentrated on analysing the source code behind OSS software to calculate quality indicators and metrics.
OSSMETER aims to extend the scope and effectiveness of OSS analysis and measurement with novel contributions on language-agnostic and language-specific methods for source code analysis, combined with state-of-the-art Natural Language Processing (NLP) and text mining techniques such as question/answer extraction, sentiment analysis and thread clustering to analyse and integrate relevant information extracted from communication channels (newsgroups, forums, mailing lists), and bug tracking systems supporting OSS projects, in order to provide a more comprehensive picture of the quality indicators of OSS projects and facilitate better evidence-based decision making and monitoring.
OSSMETER also aims at providing metamodels for capturing the meta-information relevant to OSS projects (e.g. types and details of source code repositories, communication channels and bug tracking systems, types of licences, number of downloads etc.), and effective quality indicators, in a rigorous and consistent manner that will enable direct comparison between OSS projects. These contributions will be integrated in the form of an extensible cloud-based platform through which users can register, discover and compare OSS projects, but which can also be extended in order to support quality analysis and monitoring of proprietary software development projects.
To achieve its aims, OSSMETER brings together experts in domain modelling, source code analysis, text mining, open source development and software engineering, with industrial partners from diverse business domains, and under the guidance of an advisory board of internationally recognised experts in the field of OSS development and analysis. The new platform will be a highly valuable supporting tool for:
- Developers and Project Managers who are responsible for deciding on the adoption of OSS, as it will enable them to make decisions on hard facts and uniform quality indicators;
- Developers of OSS as it will enable them to monitor the quality of the OSS projects they contribute to, promote the OSS they contribute to using independently-calculated and trustworthy quality indicators, and identify related projects for establishing synergies;
- Funding Bodies that are funding ICT projects which produce OSS, as it will allow them to monitor the quality and assess the impact of the produced software even after the end of the projects.
OSSMETER aims at building on top of the state-of-the-art in OSS source code analysis, but also at extending OSS project measurement facilities to cover the aforementioned aspects. OSSMETER also aims at integrating all these quality measurement tools and techniques in the context of an open source extensible platform.
Key technology advances from the OSSMETER project include:
- Comprehensive domain modelling for the domain of open source software development; identification and formal representation of the meta-information that needs to be captured in order to extract meaningful quality metrics and indicators for OSS projects;
- Extraction of quality metrics by analysing aspects related to the source code and the development team behind an OSS project;
- Extraction of quality metrics related to the communication channels, and bug tracking facilities of OSS projects using Natural Language Processing and text mining techniques;
- Development of an extensible cloud-based platform that can monitor and incrementally analyse a large number of OSS projects, and a web-application to present their related quality metrics in an intuitive manner that aids decision making.
The automated OSS monitoring and measurement platform will enable software developers, project managers, OSS providers and funding bodies to measure the quality, dissemination and user participation in OSS software developed.
