Posted on

Azure DevOps Pipelines – how secret are secret variables?

Azure Pipelines supports storing secret variables within the project, either through variable groups or as secret variables.

This is a convenient place to store all those database connection strings and access tokens you need to allow access to external services like JFrog Artifactory or deploy to Azure services such as a Kubernetes cluster or CosmosDB.

Simply enter the secret value, check the padlock and everything is safe right? Well depends what you mean by ‘safe’!

Not so fast kiddo

Although Azure Pipelines take pains to obscure the echoing of secrets to the pipeline console and even prevents secret being made available by default to pipeline scripts this does not mean the secrets can’t be still be , how shall we say, obtained.

A pipeline developer who has access to the azure-pipelines.yml can quite easily grab the secret value, echo it to a file, publish that file as a pipeline artifact and then simply download the file from the pipeline console once run.

What do you mean – ‘developers can see the production passwords’?

Well, if you use variable groups to store secret variables for each environment you deploy to, then YES!

Now this may be fine if only admins have access to run the pipelines, but if your azure-pipelines.yml file is embedded within your application source code this in theory means an application developer could change the pipeline definition to reveal production secrets.

So how do I prevent this happening in our team?

Luckily the Azure Pipelines security onion does have a good selection of layers to peel back.

  1. Force azure-pipelines.yml to extend a ‘master’ template which restricts what tasks and scripts can be run by the child script using a mechanism similar to inheritance. Use a ‘required template’ check to ensure only sanctioned templates that extend ‘master’ can be run.
  2. Don’t put your azure-pipelines.yml in your app source code, instead store it within separate protected repo and pull in a reference to your app repo.
  3. Use permissions on variable groups to allow access to pipeline admin level roles excluding developer roles. However this only works in devs are not allowed to deploy code to higher environments.
  4. Separate the CI phase from the CD phase. This is similar to the previous technique and is in fact how pre yaml Azure Pipelines structures it’s builds. You could argue that the CI phase can be run freely by developers including deploying to a dev environment but the CD phase should only be accessible more privileged user that can promote the deployment through the various environments to production.
  5. Finally, don’t use pipeline library secret variables. Instead use Service Connections, but a problem here is that not all services support service connections e.g. Azure Databricks.

Security should always be concern number zero with any production system and a CD Pipeline that holds the keys to so many precious castles is a core component to protect.

Posted on

Do your dependencies leave you open to attack?

According to the 2015 Verizon Data Breach Investigations Report (DBIR). 98% of attacks are opportunistic in nature, and aimed at easy targets. The report also found that more than 70% of attacks exploited known vulnerabilities that had patches available.

The recent breach at Equifax was caused by a known vulnerability in the popular Struts web framework library, when uploading files. It took Equifax at least two weeks after the attack to discover the data breach and this was almost four months after the exploit had been made public. Automated alerting on known exploits could have prevented this catastrophic security hole.

This post shows an automated way to check your third party library dependencies to ensure your site does not become a victim to these opportunistic attacks.

We will use the dependency checker provided by OWASP. This example shows integration with a Maven build where the check is run against every build during the verify stage. The first run will take a while as it has to download the entire vulnerability database. Subsequent runs will have this cached and so will run much faster.

Maven dependency include:

         <dependency>
            <groupId>org.owasp</groupId>
            <artifactId>dependency-check-maven</artifactId>
            <version>${org.owasp.dependency-check-maven.version}</version>
            <scope>test</scope>
        </dependency>
Maven plugin configuration:
             <plugin>
                <groupId>org.owasp</groupId>
                <artifactId>dependency-check-maven</artifactId>
                <version>3.1.2</version>
                <configuration>
                    <cveValidForHours>12</cveValidForHours>
                    <failBuildOnCVSS>8</failBuildOnCVSS>
                </configuration>
                <executions>
                    <execution>
                        <phase>verify</phase>
                        <goals>
                            <goal>check</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>

Maven command to run:

mvn org.owasp:dependency-check-maven:check

Reference

https://www.triology.de/en/blog-entries/automatic-checks-for-vulnerabilities-in-java-project-dependencies

https://jeremylong.github.io/DependencyCheck/dependency-check-maven/index.html

https://www.owasp.org/index.php/OWASP_Dependency_Check