Azure, Security

Create Issues in Azure DevOps via Snyk API

Snyk is a great tool for scanning your code and containers for vulnerabilities. Snyk is constantly evolving and adding new features and integrations so if you haven’t checked out the Snyk website, I highly recommend you do so. There also is a free tier for you to get started.

One of the features is Jira Integration, this allows you to create a Jira Issue from within Snyk. If you use Jira then I can see a benefit for this but what if you use Azure DevOps or you want to automate the issue creation.

This post goes though using an Azure Logic App to create an issue in Azure DevOps when a new issue is discovered (Note: the process works for Jira too, just use the Azure Logic App Jira connector).

To use the Snyk API you will need to be on the Business Plan or above (at the time of writing), this then allows the ability to add a webhook to receive events.


So the flow of the Logic App is something like this

On enable of the Logic App it will register as a webhook with your Snyk account and on disable will unregister the webhook.

Let’s build the logic app, step one create a new Logic App in Azure

Once that has been created, select a blank logic app and find the HTTP Webhook trigger

As detailed in the Snyk API Documentation set the subscribe method to POST and the URI for web hooks with your organization Id (this can be found on your Snyk account under Org Settings -> General)

Add the subscribe body that includes a secret defined by you and the url of the logic app (for the URL select the Callback Url in dynamic content)

Now set the unsubscribe method to DELETE and use an expression for the URI and leave the Body as blank

concat('https://snyk.io/api/v1/org/<your org id>/webhooks/',triggerOutputs().subscribe.body.id)

Now add new parameters for Subscribe – Headers and Unsubscribe – Headers

Authorization in both headers should be set to your API token (this can be found on your Snyk account Account Settings -> API Token)

When registering the application Snyk sends a Ping event which is determined by the X-Snyk-Event header, we really don’t want to run the rest of the workflow when this happens so we can add a condition to terminate

Select New Step then find Control and select it

Then select Condition

For the value use an expression to get the X-Snyk-Event header

triggerOutputs()?['headers']?['X-Snyk-Event']

and then make the condition check it contains the word ping (the actual value is ping and the version e.g. ping/v0)

Now in the True side add an action to Terminate and Set it to Cancelled

Now if the message is anything other than a ping then we want to continue processing the response. We will want to validate that the message coming in is from Snyk and is intended for us as it will have created a signature using our custom secret and added to the header under X-Hub-Signature. To perform this validation we can use an Azure Function.

You can create an Azure Function via the Portal, I suggest you use Linux as the OS

Using Visual Studio Code with the Functions Runtime installed you can create and deploy the following function. If you are not sure how to do this take a look at the Microsoft Docs they are really helpful

I named the function ValidateRequest and used some code from the Snyk API documentation to perform the validation and return either OK (200) or Bad Request (400)

const crypto = require('crypto');
module.exports = async function (context, req) {
     
    context.log('JavaScript HTTP trigger function processed a request.');
    const secret = req.headers['x-logicapp-secret'];
    const hubsignature = req.headers['x-hub-signature'];
    const hmac = crypto.createHmac('sha256', secret);
    const buffer = JSON.stringify(req.body);
    hmac.update(buffer, 'utf8');
    const signature = `sha256=${hmac.digest('hex')}`;    
    const isValid = signature === hubsignature;
   
    context.res = {
        status: isValid ? 200 : 400,
        body: isValid
    };    
}

Now the Function is deployed we can add the next step to the Logic App.

Select the function app we created previously

And select the function we deployed previously

Now we need to pass the payload from the webhook into our ValidateRequest function

Add additional parameters for method and header

Set the method to POST and switch the headers to text mode

Then add the following expression to add the request headers and one with your secret

addProperty(triggerOutputs()['headers'], 'X-LogicApp-Secret', '<your secret>')

If the check is successful then the next step is to parse the json and loop through the new issues

For the Content add the payload as you did previously for the validate functionand the Schema add the following

{
    "properties": {
        "group": {
            "properties": {},
            "type": "object"
        },
        "newIssues": {
            "type": "array"
        },
        "org": {
            "properties": {},
            "type": "object"
        },
        "project": {
            "properties": {},
            "type": "object"
        },
        "removedIssues": {
            "type": "array"
        }
    },
    "type": "object"
}

Next we need to loop through the new issues, add a new action using the control For each and add newIssues

For this I am only interested in the high severity issues so we need to add another condition using an expression for the value and is equal to high (Note: I renamed the condition to Severity)

items('For_each')?['issueData']?['severity']

Now if the severity is high then create work item using the built-in Azure DevOps connector

This will ask you to sign in

Once signed in you can set the details of the Azure DevOps organization, project and work item type

To add the details from the issue as the title and description use the following expressions

items('For_each')?['issueData']?['title']
items('For_each')?['issueData']?['description']

And that is the Logic App complete, a created work item with the title and description fields from the payload looks like this

Perhaps the formatting could do with some work but the information is there and the workflow works.

Although I used Azure DevOps as the output, there is a Jira connector that will allow creation of an issue in Jira

Once you are happy everything is running there is one last step, securing the secrets inside the logic app so they can only be seen by the designer

For the webhook select the settings and turn on Secure Inputs and Secure Outputs

and for the ValidateRequest function turn on at least Secure Inputs.

I find Azure Logic Apps a great way to connect systems together for these types of workflows because it has so many connectors.

I hope it helps others integrate Snyk with their workflows and can’t wait to see what other features the API will provide in the future.

Azure, Azure Functions

Migrate Azure Functions from .NET Core 3.1 to .NET 5

Migrating Azure Functions v3 from .NET Core 3.1 to .NET 5 (isolated) requires a number of changes. In this post I am going to walk through steps I went through to upgrade an Azure Function.

The Microsoft Docs and the examples on Microsoft’s GitHub are well worth looking at as they give more details about what has changed.

The function that I am going to convert is a very simple one, it uses a Timer Trigger to check for messages every hour, a Blob to track when messages were last processed and add the messages to an Event Grid.

Here is the function code:

using Microsoft.Azure.EventGrid.Models;
using Microsoft.Azure.Storage.Blob;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Extensions.EventGrid;
using Microsoft.Extensions.Logging;
using System;
using System.IO;
using System.Linq;
using System.Text.Json;
using System.Threading.Tasks;
using JsonSerializer = System.Text.Json.JsonSerializer;

namespace MessagePoller
{
    public class MessageFunction
    {
        private readonly IMessageProcessor _messageProcessor;

        public MessageFunction(IMessageProcessor messageProcessor)
        {
            _messageProcessor = messageProcessor;
        }

        [FunctionName("MessageFunction")]
        public async Task RunAsync([TimerTrigger("0 0 * * * *")] TimerInfo timerInfo,
            [Blob("tracking/trackingdate.json", FileAccess.ReadWrite, Connection = "StorageConnection")] ICloudBlob trackingBlob,
            [EventGrid(TopicEndpointUri = "MessageEventGridEndpoint", TopicKeySetting = "MessageEventGridTopicKey")] IAsyncCollector<EventGridEvent> outputEvents,
            ILogger log)
        {
            log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");

            await using var stream = await trackingBlob.OpenReadAsync().ConfigureAwait(false);
            var tracking = await JsonSerializer.DeserializeAsync<TrackingObject>(stream, new JsonSerializerOptions
            {
                PropertyNameCaseInsensitive = true
            }).ConfigureAwait(false);

            var messages = await _messageProcessor.GetMessagesAsync(tracking).ConfigureAwait(false);
            await foreach (var message in messages.ToAsyncEnumerable())
            {
                var currentEvent = new EventGridEvent
                {
                    Id = Guid.NewGuid().ToString("D"),
                    Subject = "Message event",
                    EventTime = DateTime.UtcNow,
                    EventType = "Message",
                    Data = message,
                    DataVersion = "1"
                };

                await outputEvents.AddAsync(currentEvent).ConfigureAwait(false);
            }

            var bytes = JsonSerializer.SerializeToUtf8Bytes(new TrackingObject { ModificationDate = DateTime.UtcNow }, new JsonSerializerOptions
            {
                WriteIndented = true,
                PropertyNamingPolicy = JsonNamingPolicy.CamelCase
            });

            await trackingBlob.UploadFromByteArrayAsync(bytes, 0, bytes.Length).ConfigureAwait(false);
        }
    }
}

Step 1 – Update Project Files

The current .NET Core 3.1 project file looks like this:

<Project Sdk="Microsoft.NET.Sdk">
	<PropertyGroup>
		<TargetFramework>netcoreapp3.1</TargetFramework>
		<AzureFunctionsVersion>v3</AzureFunctionsVersion>
	</PropertyGroup>
	<ItemGroup>
		<PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="3.1.16" />
		<PackageReference Include="Microsoft.Azure.Functions.Extensions" Version="1.1.0" />
		<PackageReference Include="Microsoft.NET.Sdk.Functions" Version="3.0.13" />
		<PackageReference Include="Microsoft.Azure.EventGrid" Version="3.2.0" />
		<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.EventGrid" Version="2.1.0" />
		<PackageReference Include="Microsoft.Azure.WebJobs.Extensions.Storage" Version="4.0.4" />
        <PackageReference Include="System.Linq.Async" Version="5.0.0" />
	</ItemGroup>
	<ItemGroup>
		<None Update="host.json">
			<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
		</None>
		<None Update="local.settings.json">
			<CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
			<CopyToPublishDirectory>Never</CopyToPublishDirectory>
		</None>
	</ItemGroup>
</Project>

In order to update it to use .NET 5 we need to:

  • Update the target framework to net5.0
  • Add the OutputType of Exe
  • Upgrade Microsoft.Extensions.DependencyInjection to 5.x
  • Remove Packages
    • Microsoft.Azure.Functions.Extensions
    • Microsoft.NET.Sdk.Functions
    • Microsoft.Azure.WebJobs.Extensions.EventGrid
    • Microsoft.Azure.WebJobs.Extensions.Storage
  • Add Packages
    • Microsoft.Azure.Functions.Worker
    • Microsoft.Azure.Functions.Worker.Sdk
    • Microsoft.Azure.Functions.Worker.Extensions.EventGrid
    • Microsoft.Azure.Functions.Worker.Extensions.Storage
    • Microsoft.Azure.Functions.Worker.Extensions.Timer

After changing those, the csproj file now looks like this:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <AzureFunctionsVersion>v3</AzureFunctionsVersion>
    <OutputType>Exe</OutputType>
  </PropertyGroup>
  <ItemGroup>
  <PackageReference Include="Microsoft.Azure.Functions.Worker" Version="1.3.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="1.0.3" />
    <PackageReference Include="Microsoft.Extensions.DependencyInjection" Version="5.0.1" />
    <PackageReference Include="Microsoft.Azure.EventGrid" Version="3.2.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.EventGrid" Version="2.1.0" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Storage" Version="4.0.4" />
    <PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.Timer" Version="4.1.0" />
    <PackageReference Include="System.Linq.Async" Version="5.0.0" />
  </ItemGroup>
  <ItemGroup>
    <None Update="host.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
    </None>
    <None Update="local.settings.json">
      <CopyToOutputDirectory>PreserveNewest</CopyToOutputDirectory>
      <CopyToPublishDirectory>Never</CopyToPublishDirectory>
    </None>
  </ItemGroup>
</Project>

NOTE: Microsoft.Azure.Functions.Worker.Extensions.Timer has been added as it is now a separate package

Step 2 – Create Entry point

Firstly for .NET 5 (isolated) we need a Program.cs file to work as the entry point

using Microsoft.Extensions.Hosting;

namespace MessagePoller
{
    public class Program
    {
        public static void Main()
        {
            var host = new HostBuilder()
                .ConfigureFunctionsWorkerDefaults()
                .Build();

            host.Run();
        }
    }
}

The function uses Dependency Injection and the dependencies are currently in Startup.cs and look like this:

using Microsoft.Azure.Functions.Extensions.DependencyInjection;
using Microsoft.Extensions.DependencyInjection;

[assembly: FunctionsStartup(typeof(MessagePoller.Startup))]

namespace MessagePoller
{
    public class Startup : FunctionsStartup
    {
        public override void Configure(IFunctionsHostBuilder builder)
        {
           builder.Services.AddTransient<IMessageProcessor, MessageProcessor>();
        }
    }
}

This configuration needs moving to Program.cs in ConfigureServices:

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;

namespace MessagePoller
{
    public class Program
    {
        public static void Main()
        {
            var host = new HostBuilder()
                .ConfigureFunctionsWorkerDefaults()
                .ConfigureServices(s =>
                {
                    s.AddTransient<IMessageProcessor, MessageProcessor>();
                })
                .Build();

            host.Run();
        }
    }
}

Now Startup.cs can be removed as it is no longer required.

Step 3 – Update Local Settings

The local.settings.json needs updating to change the Functions Runtime to dotnet-isolated

“FUNCTIONS_WORKER_RUNTIME”: “dotnet”

becomes

“FUNCTIONS_WORKER_RUNTIME”: “dotnet-isolated”

Step 4 – Code Changes

The first thing to change is “FunctionName” to “Function”

e.g.

[FunctionName("MessageFunction")]

becomes

[Function("MessageFunction")]

Next thing to change is ILogger, ILogger is no longer passed to the function method FunctionContext is and therefore we need to get the ILogger instance from the context.

e.g.

[Function("MessageFunction")]
public async Task<MessageOutputType> RunAsync([TimerTrigger("0 0 * * * *")] TimerInfo timerInfo,
       ...,
       FunctionContext executionContext)
{
    // Call to get the logger
    var log = executionContext.GetLogger("MessageFunction");
    ...
}

Now on to the next the function itself. The code uses ICloudBlob and IAsyncCollector<EventGridEvent> which are not supported.

Note from the Microsoft Docs on limitations of binding classes

The code also uses Blob as an Input and Output Binding which would need to now be split to use BlobInput and BlobOutput.

So if we change the current function declaration

[FunctionName("MessageFunction")]
public async Task RunAsync([TimerTrigger("0 0 * * * *")] TimerInfo timerInfo,
       [Blob("tracking/trackingdate.json", FileAccess.ReadWrite, Connection = "StorageConnection")] ICloudBlob trackingBlob,
       [EventGrid(TopicEndpointUri = "MessageEventGridEndpoint", TopicKeySetting = "MessageEventGridTopicKey")] IAsyncCollector<EventGridEvent> outputEvents,
       ILogger log)
{
    ...
}

to add the outputs, which would be Blob and EventGrid and then change ICloudBlob to something supported like the class object to deserialize to, we would end up with this:

[Function("MessageFunction")]
[EventGridOutput(TopicEndpointUri = "MessageEventGridEndpoint", TopicKeySetting = "MessageEventGridTopicKey")]
[BlobOutput("tracking/trackingdate.json")]
public async Task RunAsync([TimerTrigger("0 0 * * * *")] TimerInfo timerInfo,
       [BlobInput("tracking/trackingdate.json", Connection = "StorageConnection")] TrackingObject tracking,
       FunctionContext executionContext)
{
    ...
}

This of course will not work as you can only have one output binding and we now need a return type.

To solve this we need to create a custom return type which contains both the output bindings and then use this as the return type.

public class MessageOutputType
{
    [EventGridOutput(TopicEndpointUri = "MessageEventGridEndpoint", TopicKeySetting = "MessageEventGridTopicKey")]
    public IList<EventGridEvent> Events { get; set; }

    [BlobOutput("tracking/trackingdate.json")]
    public byte[] Tracking { get; set; }
}
[Function("MessageFunction")]
public async Task<MessageOutputType> RunAsync([TimerTrigger("0 0 * * * *")] TimerInfo timerInfo,
       [BlobInput("tracking/trackingdate.json", Connection = "StorageConnection")] TrackingObject tracking,
       FunctionContext executionContext)
{
    ...
}

Now all the code has been converted over the final result looks like this:

using Microsoft.Azure.EventGrid.Models;
using Microsoft.Azure.Functions.Worker;
using Microsoft.Extensions.Logging;
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace MessagePoller
{
    public class MessageFunction
    {
        private readonly IMessageProcessor _messageProcessor;

        public MessageFunction(IMessageProcessor messageProcessor)
        {
            _messageProcessor = messageProcessor;
        }

        [Function("MessageFunction")]
        public async Task<MessageOutputType> RunAsync([TimerTrigger("0 0 * * * *")] TimerInfo timerInfo,
            [BlobInput("tracking/trackingdate.json", Connection = "StorageConnection")] TrackingObject trackingBlob,
            FunctionContext executionContext)
        {
            var log = executionContext.GetLogger("MessageFunction");
            log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");

            var messages = await _messageProcessor.GetMessagesAsync(trackingBlob).ConfigureAwait(false);

            var eventMessages = await messages.ToAsyncEnumerable()
                .Select(message => new EventGridEvent
                {
                    Id = Guid.NewGuid().ToString("D"),
                    Subject = "Message event",
                    EventTime = DateTime.UtcNow,
                    EventType = "Message",
                    Data = message,
                    DataVersion = "1"
                })
                .ToListAsync().ConfigureAwait(false);

            return new MessageOutputType
            {
                Events = eventMessages,
                Tracking = new TrackingObject { ModificationDate = DateTime.UtcNow }
            };
        }
    }

    public class MessageOutputType
    {
        [EventGridOutput(TopicEndpointUri = "MessageEventGridEndpoint", TopicKeySetting = "MessageEventGridTopicKey")]
        public IList<EventGridEvent> Events { get; set; }

        [BlobOutput("tracking/trackingdate.json")]
        public TrackingObject Tracking { get; set; }
    }
}

Conclusion

Upgrading this particular function was surprisingly easy despite needing multiple output bindings, the Microsoft Docs and GitHub examples were extremely helpful in understanding the changes for the particular binding.

I have to say I like the multiple return type as well as the binding attributes being explicit about if they are Input or Output.

I deployed my new function from Visual Studio 2019 and worked as expected, now time to update the IaC (Infrastructure as Code) and deploy using my pipeline, hopefully there will be no surprises there.

I hope functions with other bindings are as easy to convert and can’t wait to see what other changes are added to Azure Functions in the future with the Isolated process, .NET 6 and Functions v4.

Happy Azure Functions upgrading!!