Navigating the License Compliance Landscape
The Significance of Scanning and Monitoring for License Compliances in Organisation
In the dynamic realm of software development, understanding and managing licenses are paramount for fostering innovation while mitigating risks. License scanning and monitoring have emerged as essential practices for an organization of any size, ensuring legal compliance and steering clear of potential pitfalls. In this article, we’ll spotlight the specific detection of Non-permissive/Copyleft Licenses (such as AGPL) and unravel how an organisation benefits from detecting non-compliant licenses at the early build stage and restricting such libraries to production.
The Crucial Role of License Scanning and Monitoring
Imagine a software project as a complex puzzle, with each piece represented by different components and dependencies. Licenses act as the rules governing how these pieces can be used, modified, and distributed. A comprehensive understanding of the licensing landscape is vital in the enterprise landscape, where collaboration is key.
License scanning involves scrutinising the licenses of all components within a software project, ensuring alignment with the enterprise’s legal and business requirements. Monitoring goes a step further, providing ongoing oversight to catch any license changes that might occur during the project’s lifecycle.
Different Types of Software Licenses
Permissive Licenses
Permissive licenses are more liberal in allowing users to do what they want with the software, with fewer restrictions on how the code can be used, modified, and distributed.
Key Features
- Imposes minimal restrictions on how the software can be used, modified, and redistributed.
- Allows for greater flexibility in combining the code with proprietary software.
- Places fewer obligations on developers who use the code in their projects.
Examples: MIT License, BSD License, Apache License.
Copyleft License
Copyleft is a general method for making a program (or other work) free and requiring all modified and extended versions of the program to be free as well.
Key Features of Copyleft License:
- Requires that any modified or derived works are also distributed under the same copyleft license.
- Ensures that subsequent versions of the software remain open source and freely available.
- Strives to protect and preserve the openness of the software ecosystem.
Example: GNU General Public License (GPL), Affero General Public License (AGPL).
The severity of using an Affero General Public License (AGPL) is notable, as evidenced by Google issuing a comprehensive company-wide advisory cautioning against the adoption of this license. The warning can be reviewed here.
Automating License Scanning: A Jenkins-Powered Journey
In our pursuit of license compliance excellence, we’ve engineered a Jenkins pipeline that goes beyond the ordinary. This powerhouse not only accommodates various project types but also effortlessly scans licenses within the repository and branch of your service. Let’s explore how this tailored solution simplifies the intricate process of license detection.
Crafting the Jenkins Pipeline: A Closer Look
Our Jenkins pipeline is designed to meticulously scan a service repository, compiling a detailed list of licenses in use. To streamline this process, we leverage License Finder — an open-source license scanner. We opted for this solution among several alternatives due to its versatility in supporting multiple project types and its proven reliability.
Utilising this tool, we create a custom image for our Jenkins pipeline, pre-loaded with the necessary prerequisites for License Finder to seamlessly operate alongside other essential requirements.
FROM <licious-repository>/licious-jenkin-agent:<tag>
USER root
RUN apt-get update && apt-get install -y \
net-tools \
wget \
curl \
unzip
RUN apt install python3 -y && apt install -y python3-pip
RUN apt install -y python3-hvac
# install nodejs
RUN apt-get install -y nodejs npm
# install yarn
RUN curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | apt-key add - && \
echo "deb https://dl.yarnpkg.com/debian/ stable main" | tee /etc/apt/sources.list.d/yarn.list && \
apt -q update && apt install -y yarn && rm -rf /var/lib/apt/lists/*
# install bower
RUN npm install -g bower && \
echo '{ "allow_root": true }' > /root/.bowerrc
# install pnpm
RUN npm install -g pnpm && \
pnpm version
#install jdk 12
RUN curl -L -o openjdk12.tar.gz https://download.java.net/java/GA/jdk12.0.2/e482c34c86bd4bf8b56c0b35558996b9/10/GPL/openjdk-12.0.2_linux-x64_bin.tar.gz && \
tar xvf openjdk12.tar.gz && \
rm openjdk12.tar.gz && \
mv jdk-12.0.2 /opt/ && \
rm /opt/jdk-12.0.2/lib/src.zip
ENV JAVA_HOME=/opt/jdk-12.0.2
ENV PATH=$PATH:$JAVA_HOME/bin
RUN java -version
# install maven
RUN curl -O https://archive.apache.org/dist/maven/maven-3/$MAVEN_VERSION/binaries/apache-maven-$MAVEN_VERSION-bin.tar.gz && \
tar -xf apache-maven-$MAVEN_VERSION-bin.tar.gz; rm -rf apache-maven-$MAVEN_VERSION-bin.tar.gz && \
mv apache-maven-$MAVEN_VERSION /usr/local/lib/maven && \
ln -s /usr/local/lib/maven/bin/mvn /usr/local/bin/mvn
# install gradle
RUN curl -L -o gradle.zip https://services.gradle.org/distributions/gradle-$GRADLE_VERSION-bin.zip && \
unzip -q gradle.zip && \
rm gradle.zip && \
mv gradle-$GRADLE_VERSION /root/gradle
ENV PATH=/root/gradle/bin:$PATH
#install go
WORKDIR /go
RUN wget https://storage.googleapis.com/golang/go$GO_LANG_VERSION.linux-amd64.tar.gz -O go.tar.gz && tar --strip-components=1 -xf go.tar.gz && rm -f go.tar.gz
ENV GOROOT /go
ENV PATH=$PATH:/go/bin
# install license-finder
# https://github.com/pivotal/LicenseFinder
RUN apt-get update && apt-get install -y ruby
RUN gem install license_finderRegardless of the complexity of the service, the pipeline diligently examines all licenses in use. This unified approach ensures that every project type is covered, leaving no room for oversight.
Once the licenses are scanned, the results are transformed into a CSV format. This strategic conversion serves a dual purpose — it streamlines the data for easier analysis and, more importantly, prepares it for effortless integration with Elasticsearch.
stage('Service Repository Checkout') {
checkout([$class: 'GitSCM', branches: [[name: "${Branch}"]], userRemoteConfigs: [[credentialsId: 'Credentials', url: 'Service-Repository-URL']]])
sh " license_finder --format csv > ${Service_Name}.csv "
}
stage('Upload CSV Report To Elasticsearch') {
def fileName = "${Service_Name}.csv"
def elasticsearchURL = "Elasticsearch URL"
sh """
curl --location '$elasticsearchURL' \
--form 'gitUrl=Service-Repository-URL' \
--form 'branchName=${Branch}' \
--form 'buildTag=${BUILD_NUMBER}' \
--form 'file=@$fileName'
"""
}Elasticsearch Integration: Elevating Visibility
The real magic unfolds when we seamlessly integrate Jenkins with Elasticsearch. The scan results are sent to Elasticsearch, creating a centralised repository that serves as a comprehensive dashboard for license monitoring. This integration empowers us to have a real-time, bird’s-eye view of our services and their associated licenses.
public ResponseEntity<String> uploadCsvFile() {
elasticSearchService.saveCsvData(gitUrl, branchName, buildTag, file);
return ResponseEntity.ok("CSV file uploaded and saved to Elasticsearch!");
}Elasticsearch acts as the backbone, storing our license data in a structured and easily accessible format. Kibana, the front-end maestro, transforms this data into visually rich dashboards. Each service, with its distinct license landscape, is vividly represented, providing a comprehensive view of licenses in use.
Setting Filters for Non-Compliant License Detection in Kibana
The real magic happens when we dive into Kibana’s filtering capabilities. With a few clicks, we configure filters that hone in on services utilizing Copyleft/AGPL/Non-Compliant licenses. Whether it’s a specific project type or an entire branch, Kibana’s flexibility allows us to tailor filters to our unique needs.
Setting Alerts in Kibana
The final step is seamlessly integrating alerting into the heart of the monitoring and detection process using Kibana. First, we define a custom index pattern in Kibana that corresponds to the Elasticsearch index where the license data is stored. Using Kibana Query Language (KQL), we craft a precise search query to filter out services associated with the AGPL license.
Once the query is optimized, we set up a Watcher in Elasticsearch. The Watcher is configured to periodically execute the defined query against the indexed data. The Watcher triggers an alert if the query returns results indicating AGPL license usage.
For the alert action, we configured Kibana to send notifications through various channels, such as email, Slack, or any other supported method, ensuring that relevant stakeholders are promptly informed of potential licensing issues.
Conclusion
In conclusion, as enterprises navigate the intricate web of software licenses, adopting a proactive approach to scanning and monitoring is crucial. Understanding the nuances of licenses, especially the distinctive AGPL, enables organisations to balance innovation, collaboration, and compliance in their software development endeavours. The ability to monitor, detect, and visualize licenses in real-time ensures that our services align with licensing requirements. As we continue to navigate the licensing landscape, this innovative approach stands as a testament to the power of automation in ensuring compliance and transparency in software development.
References
https://fossa.com/blog/all-about-permissive-licenses/
