JavaScript – HoNoSoFt https://blog.honosoft.com Blog & Roll Tue, 09 Jul 2019 12:49:35 +0000 en-CA hourly 1 https://wordpress.org/?v=6.8.12 https://blog.honosoft.com/wp-content/uploads/2018/06/logo.png JavaScript – HoNoSoFt https://blog.honosoft.com 32 32 #Slack |> App |> Smee |> My Local App |> #Slack Reply : How to interact when you’re behind Firewall/Proxy https://blog.honosoft.com/2019/05/24/slack-app-smee-my-local-app-slack-reply-how-to-interact-when-youre-behind-firewall-proxy/?utm_source=rss&utm_medium=rss&utm_campaign=slack-app-smee-my-local-app-slack-reply-how-to-interact-when-youre-behind-firewall-proxy https://blog.honosoft.com/2019/05/24/slack-app-smee-my-local-app-slack-reply-how-to-interact-when-youre-behind-firewall-proxy/#respond Fri, 24 May 2019 12:48:27 +0000 https://blog.honosoft.com/?p=562 Continue Reading]]> Introduction

All I wanted to do is simply have a way to connect to my local server on the network and then when receiving a specific request from #Slack. Once received act on it, do a some background work or send reports. This server is obviously not exposed on the internet and reside behind a firewall/proxy. For that reason, I had to be a bit more creative and think out of the box.

In the past I resolved this by using a WebSocket to an Heroku WebSite. It works but it was not perfect. So, I started to look around for a better way to stream a service where I a simple Webhook would be forwarded. In that manner, I could watch a service where it send me back the data. I recently came across Smee.io which use the Server Side Events (SSE) but as a streaming service on the HTTP port. Following is a story of how to make all that working together.

Before going further, I would have loved a solution like
ngrok, but it’s not possible behind a proxy/firewall such as where I work.

Step 1. Grab a new Smee URL

To get the URI, it’s deadly simple. Simply go on Smee.io and then click on create the big button in order to create a new Webhook. Keep the Webhook URL for the next step.

Step 2. Configure the #Slack App/Bot

In that App, we will setup a bot, a hook and a bot for commands.

Configure the App

Open your #Slack page and add an App and if you don’t know how, simply go on https://api.slack.com/ and click on Start Building button and fill the form. It should look like this:

Once you have created your app, you will need to enable “Bot” and give him a name, and then go into the Create a new command page and then add a new command like the following:

Install your app

For that, you go for example in OAuth & Permissions and click install app To Workspace. (Note, you might also have to add a permission File.Upload.User)

You can see the beginning of the token. The 4 first characters indicate what kind of token it is. (Documentation in slack documentation exists for more details)

Step 3. Keep it simple with a sample app in NodeJS or Dotnet Core

Due to some missing implementation in the original repository/server, I’ve added what’s missing in my fork within GitHub. I then use my Heroku account to generate automatically my Docker container with the service. In the future, the issues should be fixed (PR currently waiting to be approved).

The main issue with the Smee.IO website is that it does not yet forward the content if it’s using x-www-form-urlencoded. If you use my branch, for now you will be ok. You can even use it for building your docker service on heroku.

Sample receiving data using NodeJS (index.js + smee-client & express packages)

const SmeeClient = require('smee-client')

/// Smee client (websocket)
const smee = new SmeeClient({
  source: 'https://smee.io/5sFasdasdukH5VQidvpGv', // In case Smee is not updated, please use my repository + an heroku deployment. It is that easy.
  target: 'http://localhost:3000/events',
  logger: console
})
const events = smee.start()
/// WebServer
const express = require('express');
const app = express();
const router = express.Router();

app.use(express.json()) // Instead of body-parser
app.use(express.urlencoded()) // Instead of body-parser
app.use(function(req, res, next) { // Allow CORS if you want.
  res.header("Access-Control-Allow-Origin", "*");
  res.header("Access-Control-Allow-Headers", "Origin, X-Requested-With, Content-Type, Accept");
  next();
});

console.log(app)

// from top level path e.g. localhost:3000, this response will be sent
app.get('/', (request, response) => response.send('Hello World'));
app.use('/events', router);

router.post('/', (request, response, next) => {
  console.log(request.body) // Raw data
  response.end();
});

app.listen(3000, () => console.log('Listening on port 3000'));

Sample receiving data using Dotnet Core

NuGet Package: Smee.IO.Client (That library simply do the SSE streaming for smee.io event types for you. So, if you want to use it for different SSE messages types, you will have to adapt the code).

The following code is also available in my Github repo (Demo).

using System;
using System.Threading;
using System.Threading.Tasks;
using Newtonsoft.Json;

namespace Smee.IO.Client.ConsoleDemo
{
    class Program
    {
        private static CancellationTokenSource source;

        static async Task Main(string[] args)
        {
            source = new CancellationTokenSource();
            var token = source.Token;

            // This could also be done automatically, but since we expect some developer to test, let's
            // keep it this way.
            Console.WriteLine("Hi there, please get a valid url on https://smee.io/new (or your own server. In case of this demo, I will use my own instance)");
            Console.Write("Please enter a valid URI because no validation will be made: ");
            var smeeUri = new Uri(Console.ReadLine());

            Console.ForegroundColor = ConsoleColor.DarkRed;
            Console.WriteLine(" > Hit CTRL-C in order to stop everything.");
            Console.WriteLine();
            Console.ResetColor();

            var smeeCli = new SmeeClient(smeeUri);
            smeeCli.OnConnect += (sender, a) => Console.WriteLine($"Connected to Smee.io ({smeeUri}){Environment.NewLine}");
            smeeCli.OnDisconnect += (sender, a) => Console.WriteLine($"Disconnected from Smee.io ({smeeUri}){Environment.NewLine}");
            smeeCli.OnMessage += (sender, smeeEvent) =>
            {
                Console.Write("Message received: ");
                Console.ForegroundColor = ConsoleColor.DarkYellow;
                Console.Write(JsonConvert.SerializeObject(smeeEvent)); // This is a typed object.
                Console.ResetColor();
                Console.WriteLine();
                Console.WriteLine();
            };
            smeeCli.OnPing += (sender, a) => Console.WriteLine($"Ping from Smee{Environment.NewLine}");
            smeeCli.OnError += (sender, e) => Console.WriteLine($"Error was raised (Disconnect/Anything else: {e.Message}{Environment.NewLine}");

            Console.CancelKeyPress += (sender, eventArgs) =>
            {
                source.Cancel();
                eventArgs.Cancel = true;
            };

            await smeeCli.StartAsync(token);
            Console.WriteLine("Finish executing. Thank you!");
        }
    }
}

Example of what you can see in Smee.io console

Slack Webhook call (Command)

The command in the previous capture is using a different command, don’t worry. Anything will do.

If the payload is empty, it’s because the content type is application/x-www-form-urlencoded (Current limitation on Smee.io). If you use my branch, as said earlier, you will have the proper data.

We can receive, now let’s send

For this part, you have a few option. I will be using the Dotnet implementation from now on. However, know that you can either use a library or use raw HTTP command using the Slack API. Basically, you will do either HTTP post to a specific URL or to the response_url. The response_url is not something that stays forever. As of now, it stays 30 minutes alive, after that, normally, it won’t be usable. It can be useful to make a small dialog with options that needs to be selected. Here we will now focus to send a simple message back to say “Message received!”.

In Dotnet, but also in NodeJS, some people already have written some libraries (e.g.: SlackAPI). It’s up to you to use them. The following code will be using a raw lazy HTTP post. For that, let’s replace smeeCli.OnMessage += ... by the following (not fully completed, but you get the idea):

smeeCli.OnMessage += (sender, smeeEvent) =>
{
    IncomingMessage msgBody = null;
    if (e.Data.Body != null)
    {
        msgBody = (e.Data.Body as JObject)?.ToObject<IncomingMessage>();
    }

    // If it's for command listener, you could use a strategy pattern.
    // Some code is not in previous block, but you can easily find out what to put in order to make
    // it work.
    _slackHttpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", _slackConfiguration.BotOAuthToken);
    var msg = new
    {
        token = _slackConfiguration.BotOAuthToken,
        channel = msgBody.UserId,
        // response_type = "ephemeral", // See doc.
        text = "Message received!"
    };
    var content = new StringContent(JsonConvert.SerializeObject(msg), Encoding.UTF8, "application/json");
    var result = await _slackHttpClient.PostAsync("chat.postMessage", content).ConfigureAwait(false);
    // https://api.slack.com/methods/im.open/test (open)
    // send message to specific channel...
    // https://api.slack.com/methods/im.close/test (open)
    // Conversation could be created, but for now let's consider no conversation.
    // e.g.: https://api.slack.com/dialogs#select_elements <== create a select box...
};

What’s next?

  • Basically, if you wish, you could also link yourself with Luis.AI, TensorFlow or anything in order to make a Chat Bot more alive and more intelligent.
  • Create some command to interact with your servers
  • Link your slack account with some of your service in order to be notified when some jobs are completed for example.
  • Link/Sync/Send stuff between your #Slack and your MS Team
  • …the sky is the limit…

The limits are actually your limits and how secure you want your stuff to be (as usual).

Conclusion

I hope you’ve enjoyed that article ;). I don’t have much time recently, so it’s hard to write more often.

]]>
https://blog.honosoft.com/2019/05/24/slack-app-smee-my-local-app-slack-reply-how-to-interact-when-youre-behind-firewall-proxy/feed/ 0
Why would you do rapid prototyping with SQLite? https://blog.honosoft.com/2019/03/05/why-would-you-do-rapid-prototyping-with-sqlite/?utm_source=rss&utm_medium=rss&utm_campaign=why-would-you-do-rapid-prototyping-with-sqlite https://blog.honosoft.com/2019/03/05/why-would-you-do-rapid-prototyping-with-sqlite/#respond Tue, 05 Mar 2019 12:25:20 +0000 https://blog.honosoft.com/?p=440 Continue Reading]]> As someone who likes to do some hack and automate pretty much my daily routine, recently I changed a bit how I was making those prototype by using a super light no-dependent DB. I use it for caching my data and avoid hitting the official servers too often and give a better performance overall (UX).

In the past, I used a lot NodeJs, but found out that some things were missing in order to be really effective. Especially when you have to touch the Rest API’s (Express or alike), it’s becoming almost a hell for managing data contract, generating the Swagger API and etc. For most of my scenarios where I want something that can scaffold and be ready in one hour, I now tend to use more Dotnet Core. Since it now runs pretty much everywhere and that we can scaffold super quickly API’s and UI (creating some templates).

In fact, in order to become more efficient, I started to build a template project available on GitHub and also NuGet. It’s also the template I most likely use for the articles on this site. For information, the template project use Dotnet Core 2+ (2.2 as o now) and contains 2 projects template. One that is the most basic web application you could have with VueJS and the other one which contains all the ready to start pretty much anything (Swagger, VueJS, VeeValidate, etc.).

Why VueJS and not Angular or any others? That’s a legitimate question. I personally like to use VueJS for it’s simplicity, it’s easy to debug and to finish, it’s really easy to use (great documentation). Then added to that, I use instead of the mastodon of Bootstrap the super lightweight Picnic CSS. It contains all the thing I usually require (grid alike feature, buttons, tabs, etc.). Yes, NodeJS is not gone, it’s still part of the deal. I use WebPack (hot reloading) in order to have a nice lightweight front-end deployed in “production” cases. The hot-reload become really handy when you want to see your changes live.

To debug your API’s using REST calls become easy and you know how the pipeline goes and security is also part of the deal with Kestrel (Http/s).

Regarding the API’s call from the front-end, my personal choice is Axios since it’s also easy and in case I would build UT, the library is simple to mock.

Why Dotnet Core 2.2 (or higher)?

Dotnet Core is using the power of the C# and all the business logic. In case I have to use a database, instead of connecting to a big database, I will simply use EntityFramework Core code first approach with SQLite. Also, Dotnet Core create a simple way to have a well defined swagger API, creates API’s through controllers and also give possibilities to easily integrate with existing Authentication mechanisms. The syntax and data contracts are part of the language and for anyone will understand them. The serialization is usually using the camelCase, but in case you need something different, you simply have to change the data contract factory (Newtonsoft.Json library) to suits your needs.

SQLite?

I used to use SQL Express, PostgreSQL, etc. However, to be doing rapid prototyping without installing any DBMS, SQLite give you the power of a database in a single small file (Don’t forget to VACUUM ;)). I use the latest version where you have foreign keys and cascade delete availability. The only drawback when using SQLite is to access the data with a visualization tool. I recently found one tool which was worth using. It’s not so big to install and you can drag and drop your database in the tool for lazy people ;). The tool in question is SQLite Studio.

When doing prototyping, you want to take a minimum of time to setup an environment, and if you have to deploy let’s say on a Raspberry PI or docker. You might not want to have to install anything else than your application (self-contained). SQLite files are created wherever your project wants it to be. That way, you simply deploy your application. At the startup of the application, the migrations happens and everything is ready. In worst case, or best case, you simply download or upload the database from/to your PC.

]]>
https://blog.honosoft.com/2019/03/05/why-would-you-do-rapid-prototyping-with-sqlite/feed/ 0
Canvas Chart ➡️ Snapshot ➡️ Manipulate ➡️ Send it to your confluence page (or elsewhere) https://blog.honosoft.com/2019/01/11/canvas-chart-%e2%9e%a1%ef%b8%8f-snapshot-%e2%9e%a1%ef%b8%8f-manipulate-%e2%9e%a1%ef%b8%8f-send-it-to-your-confluence-page-or-elsewhere/?utm_source=rss&utm_medium=rss&utm_campaign=canvas-chart-%25e2%259e%25a1%25ef%25b8%258f-snapshot-%25e2%259e%25a1%25ef%25b8%258f-manipulate-%25e2%259e%25a1%25ef%25b8%258f-send-it-to-your-confluence-page-or-elsewhere https://blog.honosoft.com/2019/01/11/canvas-chart-%e2%9e%a1%ef%b8%8f-snapshot-%e2%9e%a1%ef%b8%8f-manipulate-%e2%9e%a1%ef%b8%8f-send-it-to-your-confluence-page-or-elsewhere/#respond Fri, 11 Jan 2019 12:39:37 +0000 https://blog.honosoft.com/?p=426 Continue Reading]]> Today, let’s create a connection between one of your app using ChartJs + Confluence and send or synchronize a chart. The flow with Confluence server used will be with the basic authentication. In the case you want to build a real plugin/extension to Confluence, please follow the proper guide about it. In this blog post, the basic authentication is simply a mean to an end for quick prototyping.

A complete example of a standalone project can be found on Github (https://github.com/Nordes/HoNoSoFt.PushChartToConfluence.Sample).

Introduction

Like I said, we will have an application serving some charts using the Canvas (HTML5) technology. We will then resize, take a snapshot and then transmit this file to our backend server. Then it will connect to confluence, look at the existing attachments and if it exists it will update using a comment or in the case where the file does not exists, we will be posting a new file. It is nice to update the file, but why not add the file (image) to the page if it is not already there. 🙂

Proposal using a chart

Note that we’re going to add the chart, but the same flow could be used to also update some comment/text. As long as you are familiar with html syntax, you should be able to do something good.

Pre-requisites before starting

  1. If you already have a confluence server or follow the official installation guide from Confluence, skip to #3 (Create a small application)
  2. For the docker users, let’s go towards the evaluation “just because”
    1. docker pull atlassian/confluence-server:latest
      1. About 700mo
    2. While it download, you can start by requesting a KEY
      1. https://my.atlassian.com/license/evaluation
      2. Select a Confluence server license and keep that page open until the docker image is ready.
    3. Start the docker image
      • Command: docker run -v $volume$:/var/atlassian/application-data/confluence --name="confluence" -d -p 8090:8090 -p 8091:8091 atlassian/confluence-server
        • $volume$: C:/demo/your/confluence/home
      • Open site: http://localhost:8090/
    4. During installation, fetch the Server ID and input it in the evaluation page.
      1. Follow the instructions
        1. Select trial
        2. Get the Server ID
        3. Go on your atlassian page already open and paste it
        4. Generate your key
        5. Copy/paste your key in the form within the confluence page
        6. Click next (wait a little) and voilà
        7. Setup a user or two and then let’s start
  3. Create a small application with ChartJs (or use the one built in the previous post )

Back-End

In the back-end, we will first be receiving the image to at least test our local upload between front-end and back-end. After, we will start integrating the Confluence connexion.

Dotnet ImagesController: Receive Images (Part 1)

Within Dotnet Core, we can receive files or forms controls using the IFormCollection. I don’t think it is common to receive such thing, but when talking about receiving files, this comes handy. The client (JavaScript) will then create a collection and send it using a multipart/form-data. In there we can find many things, however what we will only using here are the files (((IFormCollection)MyFormCollection).Files).

Let’s create a controller named ImagesController we will receive the files as stream and then copy those files to a temporary folders. If you already played around with file streaming you can probably skip this part.

[Route("api/[controller]")]
[ApiController]
public ImagesController : ControllerBase {
    // .. some constructor stuff

    /// <summary>
    /// This will save the image in your local "temp" folder.
    /// </summary>
    [HttpPost]
    [ProducesResponseType((int)HttpStatusCode.Created)]
    public async Task<IActionResult> PostAsync(IFormCollection formCollection)
    {
        var files = formCollection.Files;
        long size = files.Sum(f => f.Length);
        List<string> fileList = new List<string>();

        foreach (var formFile in files)
        {
            // full path to file in temp location
            var filePath = Path.GetTempFileName();
            if (formFile.Length > 0)
            {
                using (var stream = new FileStream(filePath, FileMode.Create))
                {
                    // Save to file... We could remove the await, have a List<Task<..>> and then do a Task.WhenAll(myList)
                    await formFile.CopyToAsync(stream);
                    fileList.Add(filePath);
                }
            }
        }

        // Example of processed uploaded files returning details of the new file + original request.
        // You shouldn't rely on (or) trust the FileName property without validation.
        return Ok(new { count = files.Count, size, fileList });
    }
}

This is named Part 1 since we will come back in that code in order to add some code to send to confluence.

Front-End

Do the snapshot

From JavaScript, we have two methods that are quite handy to take snapshot of a Canvas. The first one is “toBlob(…)” and the second is “toDataUrl(…)“. While “toBlob” is not supported everywhere, I find it more useful for the demo, otherwise feel free to use “toDataUrl” for all browsers supports except Edge and then convert that base64 into a binary data (image/png). In case you go towards the “toDataUrl” method, don’t forget that you will have to transform the data once it’s server side for something readable (binary) as an image/png.

You will see in the next sub-section how to start from a ChartJs chart which you’d like to push to your back-end having a resolution of 1200px wide. Don’t forget that if you use an adaptive screen UX, it will not be displayed as 1200px until you do a snapshot for a fraction of a second.

How does the resizing work?

As you already know, a ChartJs canvas does not resize automatically at your wishes and if you want a snapshot, you are required to hack your way. Next, we are going to have the following flow:

Interesting part in the JS, I will skip the trivial step of generating a chart since you’re I suspect that you’re already able to do so.

// Some code (I used VueJs, but other language should look alike)

snapshot: function () {
  // Resize to desired size, bigger it is, more heavy will be the blob file.
  this.$refs.chartContainer.style.width = '1200px'
  this.chart.resize()
  var ctx = this

  // Do a snapshot
  this.$refs.chart.toBlob(function (blob) {
    // Resize back to original size
    ctx.$refs.chartContainer.style.width = ""
    ctx.chart.resize()

    // Prepare the form post
    // The file name used will be the chart title, but depending on your case, you
    // might want to have something more precise based on parameter (hash, encoding, something). 
    var filename = `${ctx.chartConfig.options.title.text}.png`
    var data = new FormData()
    data.append('file', blob, filename)

    const config = {
        headers: { 'content-type': 'multipart/form-data' }
    }

    // Post the file to your backend
    ctx.$http.post('/api/images', data, config)
    // Later in the article it should become as
    // ctx.$http.post('/api/images/confluence/{desiredPageId}', data, config)
  }, "image/png", 0.95);
}

// Some code

As you can see, I resize to 1200px => snapshot (toBlob) => resize back to original => send the file as multipart/form-data to the api.

The file should normally be created within your temp folder and the exact location will come back through the API. It is also part of the current data contract. Now that you have that, you can consider yourself ready for the next step, which is to transfer that buffered binary data directly to confluence using their API’s.

Back-end Part 2 – Use confluence API’s

Now that we know we can receive file and save it, we now simply need to use the Confluence API’s from Atlassian. The related documentation for that is:

API’s that we’re going to use for this demo are:

  • [GET] api/content/{pageId}/child/attachment?filename={formFile.FileName}&expand=version
    • Search/Retrieve the existing attachment. In case it does not exists, it will sends back an empty array.
  • [POST] api/content/{pageId}/child/attachment/{attachmentData.Id}/data
    • Update in case of existing attachment
  • [POST] api/content/{pageId}/child/attachment
    • Create the attachment resource if it was not already existing
  • [GET] api/content/{pageId}?expand=version,body.storage
    • Retrieve the details on the current page, especially the body specified in readable/editable way (storage). The version is also mandatory when you want to update the page.
  • [PUT] api/content/{pageId}
    • Api used in order to update the confluence page.

Update the ImagesController to forward to Confluence

There’s maybe more code than required in the controller. The proper approach would be to use a IDataProvider injected (for UT) and then implement the provider using the IHttpClientFactory. That way, all would be testable and also it would also put the logic where it should be. However, let’s put all for now in one place and please adapt for your needs.

// some code in the controller

        /// <summary>
        /// Receive 1 or more images to be sent to Confluence server.
        /// </summary>
        /// <param name="formCollection">The form data (only files are being consumed)</param>
        /// <param name="pageId">The confluence page Id</param>
        /// <remarks>
        /// More details can be found at https://developer.atlassian.com/server/confluence/confluence-rest-api-examples/
        /// </remarks>
        [HttpPost("confluence/{pageId}")]
        [ProducesResponseType((int)HttpStatusCode.Created)]
        public async Task<IActionResult> PostToConfluence(IFormCollection formCollection, int pageId)
        {
            var files = formCollection.Files;
            long size = files.Sum(f => f.Length);
            List<string> fileList = new List<string>();
            var forwardAttachmentTasks = new List<Task<FileTransferResult>>();

            foreach (var formFile in files)
            {
                if (formFile.Length > 0)
                {
                    forwardAttachmentTasks.Add(ForwardFileToConfluence(pageId, formFile));
                }
            }

            // In case we had multiple tasks at the same time.
            await Task.WhenAll(forwardAttachmentTasks.ToArray()).ConfigureAwait(false);
            await UpdatePage(pageId, forwardAttachmentTasks).ConfigureAwait(false);

            return StatusCode((int)HttpStatusCode.InternalServerError, new { count = files.Count, size, fileList });
        }

        private async Task UpdatePage(int pageId, List<Task<FileTransferResult>> forwardTasks)
        {
            if (forwardTasks.Any())
            {
                // Update the page
                var pageContentResult = await _confluenceHttpClient.GetAsync($"content/{pageId}?expand=version,body.storage");
                var pageContentData = JsonConvert.DeserializeObject<Models.Confluence.Content.ContentStorage>(await pageContentResult.Content.ReadAsStringAsync());

                string newContent = string.Empty;
                foreach (var sentAttachmentTask in forwardTasks)
                {
                    var sentAttachment = await sentAttachmentTask;
                    if (sentAttachment != null && sentAttachment.Results.Any())
                    {
                        // Add the file if not present on the page.
                        var fileName = sentAttachment.Results.First().Title;
                        if (pageContentData.Body.Storage.Value.IndexOf($"<ri:attachment ri:filename=\"{fileName}\" />") == -1)
                        {
                            newContent += $"<h2>You've just pushed: {fileName}</h2><p><ac:image><ri:attachment ri:filename=\"{fileName}\" /></ac:image></p>";
                        }
                        // Else: Nothing to do, it's already on the page somewhere.
                    }
                }

                // Update object (camelCase mandatory, so use a proper serializer in real life scenario)
                var updateQuery = new
                {
                    id = pageContentData.Id,
                    title = pageContentData.Title,
                    status = pageContentData.Status,
                    type = pageContentData.Type,
                    version = new { number = pageContentData.Version.Number + 1 },
                    body = new
                    {
                        storage = new
                        {
                            value = pageContentData.Body.Storage.Value + newContent,
                            representation = "storage"
                        }
                    }
                };

                var result = await _confluenceHttpClient.PutAsJsonAsync($"content/{pageId}", updateQuery);
            }
        }

        private async Task<FileTransferResult> ForwardFileToConfluence(int pageId, IFormFile formFile)
        {
            // Start getting if attachment exists
            var getIfAttachmentExists = _confluenceHttpClient.GetAsync($"content/{pageId}/child/attachment?filename={formFile.FileName}&expand=version").ConfigureAwait(false);
            // While previous request goes on, let's get the file.
            byte[] data;
            using (var br = new BinaryReader(formFile.OpenReadStream()))
            {
                data = br.ReadBytes((int)formFile.OpenReadStream().Length);
            }

            ByteArrayContent bytes = new ByteArrayContent(data);
            MultipartFormDataContent multipartContent = new MultipartFormDataContent();
            multipartContent.Add(bytes, "file", formFile.FileName);

            var attachmentRequestData = await getIfAttachmentExists;
            if (attachmentRequestData.IsSuccessStatusCode && attachmentRequestData.StatusCode == HttpStatusCode.OK)
            {
                // Page exists and no errors...
                var attachmentRequestContent = await attachmentRequestData.Content.ReadAsStringAsync().ConfigureAwait(false);
                var attachmentData = JsonConvert.DeserializeObject<FileSearch>(attachmentRequestContent);

                HttpResponseMessage putAttachmentResponse;
                // Update existing data.
                if (attachmentData.Size == 1)
                {
                    multipartContent.Add(new StringContent($"Automatic update/upload from TestApplication ;)."), "comment");
                    putAttachmentResponse = await _confluenceHttpClient.PostAsync(
                        $"content/{pageId}/child/attachment/{attachmentData.Results.First().Id}/data",
                        multipartContent);

                    // Result is 1 "item"
                    var content = await putAttachmentResponse.Content.ReadAsStringAsync().ConfigureAwait(false);
                    var result = JsonConvert.DeserializeObject<Models.Confluence.Result>(content);

                    return new FileTransferResult() { Results = new Models.Confluence.Result[] { result }, Size = 1 };
                }
                else
                {
                    // Create the attachment
                    multipartContent.Add(new StringContent($"Automatic upload from TestApplication ;)."), "comment");
                    putAttachmentResponse = await _confluenceHttpClient.PostAsync(
                        $"content/{pageId}/child/attachment",
                        multipartContent);

                    // Result is a list of item.
                    var content = await putAttachmentResponse.Content.ReadAsStringAsync().ConfigureAwait(false);
                    return JsonConvert.DeserializeObject<FileTransferResult>(content);
                }
            }

            return default(FileTransferResult);
        }

// some code in the controller

It’s a lot of code. By reading it, it should be really easy to understand. During the file upload to Atlassian Confluence, we add a comment (updated or created) and the version gets updated. That way you could show the changes over time.

What else you could do?

  • Transform the backend in order to have a provider injected
  • Create UT
  • Create IT
  • Change the token used in order to use the official Atlassian flow for plugins
  • Automate everything back-end by creating a scheduled job (not tested, but should be feasible)
    1. Selenium
    2. Image docker + chromium
    3. Execute the javascript using chrome Webdriver
    4. Send the image using a job.
  • Send a SnapShot (not necessarily from Charts) while building your backend app in a pipeline.

Conclusion

Thank you for reading and I hope you have learned something today, or at least enjoyed this article.

]]>
https://blog.honosoft.com/2019/01/11/canvas-chart-%e2%9e%a1%ef%b8%8f-snapshot-%e2%9e%a1%ef%b8%8f-manipulate-%e2%9e%a1%ef%b8%8f-send-it-to-your-confluence-page-or-elsewhere/feed/ 0
Create plugins in Dotnet Core or what we call Managed Extensibility Framework https://blog.honosoft.com/2018/11/29/create-plugins-in-dotnet-core-or-what-we-call-managed-extensibility-framework/?utm_source=rss&utm_medium=rss&utm_campaign=create-plugins-in-dotnet-core-or-what-we-call-managed-extensibility-framework https://blog.honosoft.com/2018/11/29/create-plugins-in-dotnet-core-or-what-we-call-managed-extensibility-framework/#respond Thu, 29 Nov 2018 13:27:41 +0000 https://blog.honosoft.com/?p=352 Continue Reading]]> In Java we have the Service Loader that can add plugin(s) within your application at startup. It basically means you can extend your application with a set of new features. This same service loader can also at some extent add those plugins at runtime by detecting new files within the plugin folder.

If we look what we have in Dotnet world, we have something very similar that have been existing for a long time. It’s still rare to see such system being used. It will more likely already exists without having anyone really having to maintain it. Since in Dotnet Core 2.1, the namespaces changed a bit, it makes it interesting to explore with you. I guess that if you are on this post, you are most likely a technical guy. I assume that you’ve already done some research on what it is and that you’d like an example (step by step) to really show how easy it is to build. In this article we’ll cover only partially the Manageed Extensibility Framework (MEF).

Edit: For Dotnet Core 3, there’s a new feature, so follow this link to MSDN.

What is the Managed Extensibility Framework (A.k.a.: MEF)?

Often you will hear about the acronym MEF instead of its long name “Managed Extensibility Framework”.  In the past we were also talking in parallel about the MAF which stands for Managed Add-in Framework (See details here). MAF still exists within the Dotnet Framework with Windows Client (WPF for example), however in Dotnet Core you it is not available “yet”.

In the introduction, I stated that if you’ve used Java, you know the main idea about it. For the others, the general idea is, that when, you or a third party, have finished to create your plugin you drop your JAR in the plugins folder.  In C#/Dotnet world, MEF and MAF are using the same principle, whereas instead of dropping a Jar you drop your “Assemblies”.

Let’s go back to the question, what is MEF? It’s a contracts based extensibility managed by imports and exports within your programs or libraries. You will then make an extensible application by implementing interfaces and then in your main application you will have a provider to provide the implemented contracts (interfaces) implemented. The Assemblies that you want to import in your main application require the exports. Those exports are going to be discovered through “Reflection“.

Let’s take the famous calculator as an example where you could extends the functionalities (plus, minus, division, etc.). Each of those functionalities could be a provided within different “Assemblies” loaded at the startup of your application. There would be no need of project references to those assemblies within your main application. So if you add or remove “Assemblies”, functionalities are going to be added or removed. The magic comes from the exports you would discover. For the calculator example, you could have a “Name” (Square Root) for what it is and the official “Operator” (√) for the discovered feature. You would then have an implementation of the operator receiving “Number 1, …” producing a new result (Contract within an interface shared between all the libraries, a.k.a. interfaces). The Imports are, as stated previously, most likely be in your main executable (WebApi, Console, etc.).

How does it works?

As stated above, the main program will do your discovery of shared contracts API’s. Those Exports/Imports are being assigned using attributes on your classes. Those attributes, to name only a few, will look like [import], [ImportMany], [ExportMetadata(“something”, “here”)], [Export(typeof(IYourInterface))], etc.. As you can see, the import will import only one instance while ImportMany will create an IEnumerable. Those attributes can be found within the Nuget package “System.Composition.AttributedModel“. Keep that package in your mind since it will become handy in a later stage.

If you wonder why the ExportMetadata can become something really cool. I would say that you could have some filtering that could come from there regarding your application. That could also help in order to give some capabilities about your module (Extensibility).

Practice time – Let’s build a simple App (WebApi/SPA)

Of course, we’re not going to be building the famous calculator sample. It’s all over the place already. I don’t say that the calculator is a not a great example, but my point here is not to copy what already exists. In this “exercise”, you will be scratching the surface of the Extensibility. In a real-world scenario, you could have logs that need to strips out content based on rules that are either common or specific. Let’s say, the customer numbers are not the same from customer to customer as such that each customer requires to have different custom rules mixed with general ones. That one could be done using simply a regex, but let’s say it goes to something really complex like the passport numbers. In that case, you could implement custom rules for each customer (per country) and simply drop those rules within the plugin folder. That way, you could even request the customer to build the plugin themselves and if you thrust them enough then simply allow them to update their plugin through a portal (upload, push, and either restart your service or then load at runtime the updated plugin).

The current exercice is to simply create a simple Web Application written in Dotnet Core 2.1 using a basic UI querying the API. It will then show the result in raw of what plugin have discovered.

Scaffold a simple WebApi and Projects

In previous posts, I said that I like to use my own template since it’s quite a good starter. It’s compact and fast enough for anyone with basic knowledge of Dotnet Core and VueJs. The UX is using, instead of the common Bootstrap 4, something lighter called Picnic CSS. You can download/update the template using > dotnet new -i HoNoSoFt.DotNet.Web.Spa.ProjectTemplates(Documentation here). This exercise can also be accomplished within a Console application. However, I think the most common scenario will be around the usage of a WebApplication and then extend it somehow. Let’s create the structure of the project (Command line, it’s easier):

> mkdir Sample.Mef
> cd Sample.Mef
Sample.Mef> dotnet new vuejs-picnic -o Sample.Mef.Web
Sample.Mef> dotnet new classlib -o Sample.Mef.Api
Sample.Mef> dotnet new classlib -o Sample.Mef.GiveMeOne
Sample.Mef> dotnet new classlib -o Sample.Mef.GiveMeFive
Sample.Mef> dotnet new sln
Sample.Mef> dotnet sln add .\Sample.Mef.Web\Sample.Mef.Web.csproj
Sample.Mef> dotnet sln add .\Sample.Mef.Api\Sample.Mef.Api.csproj
Sample.Mef> dotnet sln add .\Sample.Mef.GiveMeOne\Sample.Mef.GiveMeOne.csproj
Sample.Mef> dotnet sln add .\Sample.Mef.GiveMeFive\Sample.Mef.GiveMeFive.csproj
Sample.Mef> cd Sample.Mef.Web
Sample.Mef\Sample.Mef.Web> npm install
Sample.Mef\Sample.Mef.Web> dotnet run

Normally after completing the previous command, you should have the Web Project starting. The URL of the project will be displayed within the console or if you are in Visual Studio, it will open the start page by itself.

For the next part of the blog I will be using Visual Studio Community, but feel free to use VS Code.

Exercise: Structure Application Tree

In the following screen capture, some renaming have already taken place (1) and as indicated in the image, some files needs to be removed (2) or modified (3).

Not marked on the picture:

  • Remove IWeatherProvider.cs 
  • Update ServiceCollectionExtensions.cs (remove the injection of the IWeatherService).

Implement the contract in GiveMeOne & GiveMeFive

Some project will require the reference in order to be able to use IGiveNumber interface. Those relation are shown within one of the previous section (UML like)

  • Add the reference of Sample.Mef.Api project to
    • Sample.Mef.GiveMeOne
    • Sample.Mef.GiveMeFive
    • Sample.Mef.Web

The interface “IGiveNumber.cs” look like:

namespace Sample.Mef.Api
{
    public interface IGiveNumber
    {
        int GiveInt();
    }
}

This contract is going to be implemented within “GiveOne.cs” and “GiveFive.cs”

using Sample.Mef.Api;

namespace Sample.Mef.GiveMeOne
{
    public class GiveOne : IGiveNumber
    {
        public int GiveInt()
        {
            return 1;
        }
    }
}
using Sample.Mef.Api;

namespace Sample.Mef.GiveMeFive
{
    public class GiveFive : IGiveNumber
    {
        public int GiveInt()
        {
            return 5;
        }
    }
}

Now you should be able to build with no errors.

Add the Composition/Extensibility to Import/Export

You remember the nuget package from earlier? It gives the simple Import/Export and a few other basic functionalities. In our case, we will be using it simply for exporting the implemented interface. Let’s install it:

  • Add Nuget package “System.Composition.AttributedModel” to
    • Sample.Mef.GiveMeOne
    • Sample.Mef.GiveMeFive

Add the “Export” attribute

Now, let’s update our two previously defined classes (GiveOne, GiveFive) and add our exports. Those exports allow the Reflection, in a later stage, to discover the implementations. Some customization could also be done, but for that, rely on the MSDN documentation.

Within the two previous project mentioned, add the following Export attribute:

using System.Composition;
// Some code...

[Export(typeof(IGiveNumber))]
public class ...// some code...

Add the “Import” matching the previous “Export” within a provider

This part is only modify “Sample.Mef.Web” project. As we’re going to create a new provider, let’s make it properly by using Dependency Injection (DI). Within the provider, we will also be using System.Composition. This is will help doing the discovery of the plugin(s).

  1. Add the Nuget package reference to the project: “System.Composition“
  2. Create an IGiveNumberProvider + GiveNumberProvider
  3. Update the startup to inject the new provider

We will put IGiveNumberProvider and GiveNumberProvider within the same file, but it’s a sample and you can also do as you please.

using Sample.Mef.Api;
using System.Collections.Generic;
using System.Composition;
using System.Composition.Hosting;
using System.IO;
using System.Linq;
using System.Reflection;
using System.Runtime.Loader;

namespace Sample.Mef.Web.Providers
{
    public class GiveNumberProvider : IGiveNumberProvider
    {
        public GiveNumberProvider()
        {
            Compose();
        }

        [ImportMany]
        public IEnumerable<IGiveNumber> Services { get; private set; }

        public IEnumerable<(string Id, int Value)> GetNumberFromAllFoundServices()
        {
            return Services.Select(f => new { Id = f.GetType().ToString(), Value = f.GiveInt()})
                .AsEnumerable()
                .Select(c => (c.Id, c.Value))
                .ToList();
        }

        private void Compose()
        {
            // Catalogs does not exists in Dotnet Core, so you need to manage your own.
            var assemblies = new List<Assembly>() { typeof(Program).GetTypeInfo().Assembly };
            var pluginAssemblies = Directory.GetFiles("d:\\plugins\\", "*.dll", SearchOption.TopDirectoryOnly)
                .Select(AssemblyLoadContext.Default.LoadFromAssemblyPath)
                // Ensure that the assembly contains an implementation for the given type.
                .Where(s => s.GetTypes().Where(p=> typeof(IGiveNumber).IsAssignableFrom(p)).Any());

            assemblies.AddRange(pluginAssemblies);

            var configuration = new ContainerConfiguration()
                .WithAssemblies(assemblies);

            using (var container = configuration.CreateContainer())
            {
                Services = container.GetExports<IGiveNumber>();
            }
        }
    }

    public interface IGiveNumberProvider
    {
        IEnumerable<(string Id, int Value)> GetNumberFromAllFoundServices();
    }
}

The code should be clear enough in order for you to understand what it does. However, let’s explain quickly:

  • It’s important to note that I’ve set the Assemblies for the extensibility available in the “d:\plugins\” (Please adapt)
  • We filter using reflection on the assemblies. This will enforce keeping only those that have at least one implementation from IGiveNumber
  • In case the assembly can be loaded from the container, we take the related “Export” implementing IGiveNumber and push that within the services
  • You can now access the implementations through the “Services” (Property field)
    • I use the Tuple as a return type, some of it is new in the C# 7+. In case you’ve missed that new feature, go on the following Blogs MSDN article

Add the Dependency Injection (DI)

Now that the provider is ready, let’s add the DI within Startup.cs. The usage of Singleton is privileged since we don’t want to rediscover at each HTTP requests.

// Code ...
        // This method gets called by the runtime. Use this method to add services to the container.
        public void ConfigureServices(IServiceCollection services)
        {
            // code ...

            // Example with dependency injection for a data provider.
            services.AddSingleton<IGiveNumberProvider, GiveNumberProvider>();
        }
// Code ...

Alter the existing “api/values” endpoint in order to get the provider “values”.

using Microsoft.AspNetCore.Mvc;
using Sample.Mef.Web.Providers;

namespace Sample.Mef.Web.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class ValuesController : ControllerBase
    {
        private readonly IGiveNumberProvider _giveNumberProvider;

        public ValuesController(IGiveNumberProvider giveNumberProvider)
        {
            _giveNumberProvider = giveNumberProvider;
        }

        // GET api/values
        [HttpGet]
        public IActionResult Get()
        {
            return Ok(_giveNumberProvider.GetNumberFromAllFoundServices().ToList().Select(f=> new { f.Id, f.Value }));
        }

        // Code
}

It’s now mostly complete. I would even say that you must put your project Assemblies (DLL) in “d:\plugins” or where you’ve decided to discover them. The assemblies we’re going to look for are described next.

Time to test by moving your dll’s in the plugins folder

Since you’ve already built, or maybe not yet, your solution, it’s now the time to bring the assemblies (dll) from Sample.Mef.GiveMeOne, Sample.Mef.GiveMeFive available through the bin directory. You then move them to the “d:\plugins”, or the reconfigured folder from previous step. When you copy your bin folder, you need to copy not only the Sample.Mef.GiveMeOne, you also need to copy it’s dependencies (i.e.: Sample.Mef.Api). Since the System.Composition is included in the Web project, it is not mandatory to have a copy in related assemblies plugin folder.

The plugin folder will then contains:

  • Sample.Mef.GiveMeOne.dll
  • Sample.Mef.GiveMeFive.dll
  • Sample.Mef.Api.dll

Start your application and look at the results

Purely speaking, do a dotnet run from the Web application folder, or in Visual Studio push F5.

Go see if your API’s is answering properly before the last step

[Optional] Update your UX in order to do the call

The output is going to simply show a list from the homepage. The extensions to your project, after being loaded, will be displayed within the result of the API call.

<template>
  <div>
    <page-title title="Give the Extensions" />
    <p>He you will be finding your loaded extensions:</p>
    <ul>
      <li v-for="(data, idx) in numbers" :key="idx">{{data.id}}
          <ul>
            <li>{{data.value}}</li>
          </ul>
      </li>
    </ul>
  </div>
</template>

<script>
export default {
  data () {
    return {
      numbers: []
    }
  },

  mounted() {
    this._getNumbers()
  },

  methods: {
    async _getNumbers () {
      let response = await this.$http.get(`api/values`)
      this.numbers = response.data
    }
  }
}
</script>

Simply run the application and go on the home page. In my case, while using IIS Express, it gave me https://localhost:44307

Final result

Extra Resources

Those resource were not used that much during this blog post, but I think it can help people to also understand how it works in a different context.

  • Official: https://docs.microsoft.com/en-us/dotnet/framework/mef/
  • https://dotnetthoughts.net/using-mef-in-dotnet-core/
  • https://blog.softwarepotential.com/porting-to-net-standard-2-0-part-2-porting-mef-1-0-to-mef-2-0-on-net-core/

Thank you!

By the way, all the source code is available in GitHub (personal @Nordes repository). Enjoy 😉

Featured image original is from: https://unsplash.com/photos/l090uFWoPaI
]]>
https://blog.honosoft.com/2018/11/29/create-plugins-in-dotnet-core-or-what-we-call-managed-extensibility-framework/feed/ 0
ChartJs with SonarQube API’s – Make your own reports https://blog.honosoft.com/2018/11/22/chartjs-with-sonarqube-apis-make-your-own-reports/?utm_source=rss&utm_medium=rss&utm_campaign=chartjs-with-sonarqube-apis-make-your-own-reports https://blog.honosoft.com/2018/11/22/chartjs-with-sonarqube-apis-make-your-own-reports/#respond Thu, 22 Nov 2018 14:30:28 +0000 https://blog.honosoft.com/?p=390 Continue Reading]]> Recently asked to do reports every two weeks to all the teams about their good/bad behaviors and at the same time look if the guidelines were somewhat respected, I decided to build a dashboard using ChartJs and a connection to SonarQube via a local proxy built in Dotnet Core. That report is not built in order to hit the team or anything, it’s more in order to know our current situation and what we should improve. We can also relate the data from Jira (using the api’s) in order to know if we’re improving our bug ratio.

This article will only do a quick start using the template I’ve created recently in Dotnet Core. I won’t put too much details around the code since it’s quite straight forward.

Pre-requisites

  • Dotnet core 2.1 (at the time of writing)
  • NodeJs + NPM (>10 at the time of writing)
  • Visual Studio Code OR Visual Studio Community/Professional
  • A browser (Chrome, or anything else)

Install a quick start from a Dotnet Core template

As mentioned before, you will be using a quick start I created. I use it quite often, so there’s improvement in it happening regularly. So I suggest you to look for updates once in a while. It gives a front-end that could run without backend, and a backend in dotnet core 2.1 ready to serve API’s (including swagger).

> dotnet new -i HoNoSoFt.DotNet.Web.Spa.ProjectTemplates

Once installed, let’s create our template (vuejs-picnic-table):

> mkdir Sonar.Reports
> cd Sonar.Reports
> dotnet new vuejs-picnic-table

And to be sure it works, then let’s run at least in development mode.

> npm install
> dotnet run --environment="Development"

The installation of the NPM packages are needed most likely the first time due to the nature of the project. This could be automated, but would make the project takes a lot of time before even being able to start editing any files. (Personal choice here)

The result should look like the following screen (https://localhost:5001/) OR the demo page available here.

You might get an error the time Webpack generate the files. In that case, simply hit refresh (F5)

Once the application is running, keep it running while you edit your files (next step). You will simply have to hit F5 to refresh the changes.

Clean or add route to the template

Delete all the un-necessary things or simply add a new page. We’ll go here with the latter.

  • Create the file: ./ClientApp/pages/sonar.vue
    • <template>
       <div>
         <page-title title="Sonar" />
       </div>
      </template>
      
      <script>
      export default {
        
      }
      </script>
      
  • Add the new route for the page in ./ClientApp/router/routes.js
    • Add a import in the import section: import Sonar from 'pages/sonar'
    • Add the route after about:  { name: 'Sonar', path: '/:lang?/sonar', component: Sonar, display: 'Sonar', i18n: 'route.sonar', icon: 'icon-home', meta: { order: 6 } },
  • Add the translation/I18n for the new menu in ./ClientApp/_i18n/lang/en.js
    • In the routes group under about, add: sonar: 'Sonar'
  • Save all this and hit F5 in the browser

Congratulation! You’ve created your first page in VueJs. Now it is the time to fill the data and before that, we need to create a service provider.

Access to Sonar Qube API’s from C#

Create your provider in C#, because if you’re not on the same server as your SonarQube, you might get some issue by only using the front-end. Here, we will be creating a simple proxy without any parameters. For those of you who didn’t know, the API is available at your SonarQube URL like the following: https://mySonarQube.url.com/api/.

Generate an API key for your user

  • Go in your account (top right icon)
  • Go under security
  • Generate a new token (The name does not matters, but let’s name it: Sonar.Reports)
  • Copy and paste the key somewhere you won’t loose. Otherwise you will have to re-generate the API token.

Open your project in VS Code or Visual studio

If you’re lucky enough, you will have a Properties folder, otherwise you will have to create it and add the file launchSettings.json

{
  "$schema": "http://json.schemastore.org/launchsettings.json",
  "iisSettings": {
    "windowsAuthentication": false, 
    "anonymousAuthentication": true, 
    "iisExpress": {
      "applicationUrl": "http://localhost:21535",
      "sslPort": 44307
    }
  },
  "profiles": {
    "IIS Express": {
      "commandName": "IISExpress",
      "launchBrowser": true,
      "launchUrl": "api/values",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    },
    "Sonar.Reports": {
      "commandName": "Project",
      "launchBrowser": true,
      "launchUrl": "api/values",
      "applicationUrl": "https://localhost:5001;http://localhost:5000",
      "environmentVariables": {
        "ASPNETCORE_ENVIRONMENT": "Development"
      }
    }
  }
}

Update your ./Startup.cs and add a new Service configuration:

public void ConfigureServices(IServiceCollection services){
  // ...
  services.AddHttpClient();
  // ...
}

Create the file ./Controllers/SonarController.cs and add the following code:

using System;
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Mvc;

namespace Sonar.Reports.Controllers
{
    [Route("api/[controller]")]
    [ApiController]
    public class SonarController : ControllerBase
    {
        private HttpClient _sonarHttpClient;

        public SonarController(IHttpClientFactory httpClientFactory){
          _sonarHttpClient = httpClientFactory.CreateClient("sonar");
          _sonarHttpClient.BaseAddress = new Uri("https://sonar.yourSonarQube.com/api/");
          var basicToken = Convert.ToBase64String(Encoding.UTF8.GetBytes("f6d6fecfdcd7836612a0fef4a3f307b7725f8387:"));
          _sonarHttpClient.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Basic", basicToken);
        }

        /// <summary>
        /// GET Sonar data
        /// </summary>
        /// <returns>Returns an enumerable of value</returns>
        [ProducesResponseType((int)System.Net.HttpStatusCode.OK)]
        [HttpGet]
        public async Task<IActionResult> Get()
        {
            var result = await _sonarHttpClient.GetAsync("measures/search_history?component=yourComponentName&metrics=bugs%2Ccode_smells%2Cvulnerabilities%2Creliability_rating%2Csecurity_rating%2Csqale_rating&ps=1000");
            
            // Not perfect but it will do for the sample.
            return Content(await result.Content.ReadAsStringAsync());
        }
    }
}

Where:

  • www.yourSonarQube.com is your address of your Sonar Qube
  • The token you’ve created earlier replaces f6d6fecfdcd7836612a0fef4a3f307b7725f8387 (Don’t forget to keep the “:”, it is mandatory in a basic authentication, where it’s usually user:password in Base 64)
  • yourComponentName is replaced by your real component name in Sonar Qube (To get it, open sonar qube, go in your project you want to test, and then take the value from the URL. It’s always there.)

Now, restart your application (Ctrl+C and then re-type dotnet run, if you’re in console mode).

You can test the endpoint by going to http://localhost:5001/swagger/ and use the swagger UI in order to test it. If you have errors, you will see it in your console (or output in Visual Studio).

Add the ChartJS to your project

Go in your root folder where your project is located (csproj) and then type the following command in order to install Chart.Js:

> npm install chart.js --save

Now that you have installed this new dependency, you can add the Chart.Js within your VueJs application.

Edit your ./ClientApp/pages/sonar.vue and add a default Chart.Js available through all their demo on their site. Here, I will be using the line API

<template>
  <div>
    <page-title title="Sonar" />

    <canvas ref="chart" id="canvas"></canvas>
  </div>
</template>

<script>
// Import ChartJS
import Chart from 'chart.js'

// Colors
var chartColors = [
  'rgb(255, 99, 132)',
  'rgb(255, 159, 64)',
  'rgb(255, 205, 86)',
  'rgb(75, 192, 192)',
  'rgb(54, 162, 235)',
  'rgb(153, 102, 255)',
  'rgb(201, 203, 207)'
]

export default {
  data () {
    return {
      chart: null,
      chartConfig: {
        type: 'line',
        data: {
          labels: [],
          datasets: []
        },
        options: {
          responsive: true,
          title: {
            display: true,
            text: 'Enjoy your Chart.Js with Sonar'
          },
          tooltips: {
            mode: 'index',
            intersect: false,
          },
          hover: {
            mode: 'nearest',
            intersect: true
          },
          scales: {
            xAxes: [{
              display: true,
              type: 'time',
              scaleLabel: {
                display: true,
                labelString: 'Date'
              }
            }],
            yAxes: [{
              display: true,
              scaleLabel: {
                display: true,
                labelString: 'Value'
              }
            }]
          }
        }
      }
    }
  },

  mounted () {
    this.chart = new Chart(this.$refs.chart, this.chartConfig)
    this._loadSonarData()
  },

  methods: {
    _loadSonarData: async function () {
      let response = await this.$http.get(`./api/sonar`)
      var sonarData = response.data

      // We now have measures + data
      var measures = sonarData.measures
      let allLabels = []
      for (var m = 0; m < measures.length; m++) {
        let currentColor = chartColors[m % chartColors.length]
        var data = {
          label: measures[m].metric,
          backgroundColor: currentColor,
          borderColor: currentColor,
          tension: 0, // If you don't want bezier curves
          data: [],
          fill: false,
        }

        for (var h = 0; h < measures[m].history.length; h++) {
          let date = new Date(measures[m].history[h].date)
          if (allLabels.indexOf(date) < 0) {
            allLabels.push(date)
          }

          data.data.push(measures[m].history[h].value)
        }
        
        this.chartConfig.data.datasets.push(data)
      }

      this.chartConfig.data.labels = allLabels
      this.chart.update()
    }
  }
}
</script>

Results

If you’ve followed and didn’t make any big mistakes, the result should looks somewhat like the following

Next step?

Basically, yes, there’s a next step. Since you have been able to call the Sonar Qube Api’s, you now know that you can also configure all the requests to it. For example, you could say “I want data for the last 30 days” or in my case, for the last 3 sprints and then draw vertical lines (using a plugin) in order to show exactly where are the sprint and how it went (For example: rush at the end or not).

You could also add a dropdown at the top, or different menu or customization for other projects/stats you’d like to have.

Since you’re using the canvas, don’t forget you could also make a download or extract image using JavaScript. In a way, you could auto-generate a Powerpoint or word document, or other since you have all the data required to do so.

Conclusion

I hope you’ve enjoyed this post.

]]>
https://blog.honosoft.com/2018/11/22/chartjs-with-sonarqube-apis-make-your-own-reports/feed/ 0
.Net Core 2.1 ➕ SignalR ➕ JavaScript = ❤️ https://blog.honosoft.com/2018/07/11/net-core-2-1-plus-signalr-plus-javascript-equal-love/?utm_source=rss&utm_medium=rss&utm_campaign=net-core-2-1-plus-signalr-plus-javascript-equal-love https://blog.honosoft.com/2018/07/11/net-core-2-1-plus-signalr-plus-javascript-equal-love/#respond Wed, 11 Jul 2018 13:00:24 +0000 https://blog.honosoft.com/?p=181 Continue Reading]]> Introduction

Comme vous le savez, SignalR à été officiellement ré-écris pour .Net Core 2.1. D’ailleurs, qu’est-ce que SignalR? Bonne question, c’est une façon de communiquer avec un WebSocket, ou de façon dégradé, avec votre backend. Le serveur ici est écrit en .Net, mais si vous voulez avoir un serveur de WebSocket en JavaScript, vous avez d’autres choix (i.e.: Socket.IO + autres alternative). Cependant côté client, à mon avis, les autres librairies ne sont pas aussi bien développé que SignalR. SignalR vous permettra de vous y connecter avec quasi tous les langages connus. Afin de voir comment faire, le repository officiel GitHub de SignalR regorge d’exemples.

Comment marche les WebSocket?

Normalement, la première connection se fait avec un premier appel http(s). Ensuite, le lien se feras en WebSocket (ws ou wss). WS est l’accronyme pour WebSocket et le second S est pour Sécure, tout comme le S dans HTTPS.

But de l’article

Afin de ne pas recopier le tutoriel sur le site officiel, je vais bâtir ce qui suit:

  • Un site web basé sur le modèle WebAPI afin d’éviter à avoir Razor + NPM + etc.
  • Récupérer le JavaScript sans avoir NPM dans le projet
  • Écrire le minimum de JavaScript (SignalR)
  • Écrire le minimum de CSS (Picnic)
  • Écrire le minimum de HTML (Only pure HTML)

Avec pour résultat:

  • Un serveur qui exécutera en arrière plan un lancement d’événements aux 2 secondes sur un processus différent. (Voir BackgroundService)
  • Un hub d’événements que nous appelleront ToastHub.
  • Le front qui vous afficheras les notifications avec 3 niveaux d’alertes aléatoire

Exemple de résultat avec IE et Chrome

Création du projet

À priori, vous savez déjà comment créer le projet ;). Sinon, si vous êtes en ligne de commande, effectuez “dotnet new –help” et vous verrez les types de projets Template que vous avez sur votre pc. Normalement vous y trouverez WebAPI. Du coup dotnet new webapiet n’oubliez pas que le “nom” du projet prendra le nom de votre répertoire (excepté si vous ajoutez l’argument a dotnet new). Dans mon cas, mon projet + espace de nommage sera HoNoSoFt.SignalR.Demo. 😉

Avant de continuer, assurez vous que ça fonctionne. F5 dans Visual Studio, ou dotnet run en ligne de commande. Normalement le site devrait s’ouvrir de lui-même.

Ajout des librairies +dépendances (DI)

Si vous avez la flemme, vous pouvez tout simplement ajouter le package NuGet directement dans le fichier csproj.

<PackageReference Include="Microsoft.AspNetCore.SignalR" Version="1.0.2" />

Sinon, si vous êtes plus du type à ouvrir le package manager, veuillez tout simplement ajouter AspNetCore.SignalR (Ici le AspNetCore est très important).

Si ce n’est pas déjà fait. Effectuer un build. Ça vous permettra de vous assurer que le package NuGet est bien arrivé sur votre machine.

Ouvrons le fichier Startup.cs et allons modifier la méthode Configure(…) et ConfigureServices(…). Nous allons ajouter les fichiers par défaut, les fichiers statique, les cookies, les politiques de CORS. Est-ce tout nécessaire? Probablement pas, mais c’est pas plus mal d’avoir ça de configurer.

        public void ConfigureServices(IServiceCollection services)
        {
            // Enregistrement de nos services d'arrière plan (Injection de Dépendence)
            ////services.AddSingleton<IHostedService, ToastEventGenerator>(); 
            // Fin de nos enregistrement de HUBs
            services.Configure(options =>
            {
                options.CheckConsentNeeded = context => true;
                options.MinimumSameSitePolicy = SameSiteMode.None;
            });
            services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);

            services.AddCors(options => options.AddPolicy("CorsPolicy",
                builder =>
                    {
                        builder.AllowAnyMethod().AllowAnyHeader()
                               .WithOrigins("http://localhost:5000")
                               .AllowCredentials();
                    }));

            services.AddSignalR().AddJsonProtocol(options =>
            {
                options.PayloadSerializerSettings.ContractResolver =
                new CamelCasePropertyNamesContractResolver(); // Utile pour le JavaScript
            });

            services.AddLogging(); // Pour voir un peu plus de logs et pouvoir utiliser ILogger<...>
        }
public void Configure(IApplicationBuilder app, Microsoft.AspNetCore.Hosting.IHostingEnvironment env)
{
    if (env.IsDevelopment())
    {
        app.UseDeveloperExceptionPage();
    }
    else
    {
        app.UseHsts();
    }

    app.UseHttpsRedirection();
    // Lignes ajoutées
    app.UseDefaultFiles();
    app.UseStaticFiles();
    app.UseCookiePolicy();
    app.UseCors("CorsPolicy");
    app.UseSignalR(routes =>
    {
        routes.MapHub<Hubs.ToastHub>("/hubs/toast"); // <=== Ce sera utile pour la prochaine étape.
    });
    // Fin de l'ajout
    app.UseMvc();
}

Ok, normalement vous devriez avoir une partie encore manquante. Votre implémentation de HUB.

Ici je met une implémentation qui n’utilise pas les authorizations. Du coup, vous ne pourrez pas avoir le nom de la personne qui se connecte. Histoire de garder ça simple et que ça marche partout (même Linux). Je n’ai pas activé l’identification par NTLM.

using System;
using System.Threading.Tasks;
using Microsoft.AspNetCore.SignalR;
using Microsoft.Extensions.Logging;

namespace HoNoSoFt.SignalR.Demo.Hubs
{
    //[Authorize]
    public class ToastHub : Hub
    {
        private readonly ILogger<ToastHub> _logger;

        public ToastHub(ILogger<ToastHub> logger)
        {
            _logger = logger;
        }

        public override Task OnConnectedAsync()
        {
            // Pour le Context.user, nous avons besoin de l'attribut Authorize(Authentication).
            _logger.LogInformation("Un utilisateur vient de se joindre.");

            return Task.CompletedTask;
        }

        public override Task OnDisconnectedAsync(Exception exception)
        {
            _logger.LogInformation("Un utilisateur vient de quitter.");

            return Task.CompletedTask;
        }
    }
}
namespace HoNoSoFt.SignalR.Demo.Models
{
    public enum ToastImportance
    {
        Low,
        Medium,
        High
    }
}

 

Désormais, si vous exécuter votre projet. Ça devrait fonctionner (moyennant la correction de votre référence dans votre fichier Startup.cs).

Notez que nous n’avons toujours pas notre service en fond de tâche (BackgroundService).

Réccuppération de la librairie SignalR.js

Oui, c’est simple me direz-vous. Oui, mais non. Si vous êtes dans un projet en NodeJS oui, très simple. De même que, si votre projet vous pensez utiliser NPM/Bower ou Yarn. Dans ce cas-ci, gardons le projet sans références NPM.

Mais alors, comment réccuppérer le JavaScript et le glisser dans wwwroot/js/? C’est simple, ouvrez vous une invite commande, créez-vous un répertoire temporaire.

Exemple:

cd /
mkdir tmp/signalr
cd tmp/signalr
npm init -f 
npm install @aspnet/signalr

REM Vous pouvez ouvrir votre explorateur de fichier afin de copier/coller les fichiers qui seront dans:
REM  > node_modules\@aspnet\signalr\dist\browser\

Fichiers dans le répertoire

  • signalr.js
  • signalr.js.map
  • signalr.min.js
  • signalr.min.js.map

Les fichiers map, si vous ne le savez pas, seront utilisé lors de débuggage. Ça indique où se trouve le code dans les fichiers originaux. Dans notre cas, ce ne sera pas trop utile.

Copier ces fichiers dans votre répertoire “wwwroot/js/”.

Créer votre page html

Ici on va faire simple, nous avons du HTML pure ainsi que l’ajout des informations suivantes:

  • Picnic.css (CDN)
  • site.css (pas encore créé)
  • SignalR comme librairie

Fichier: index.html

<!DOCTYPE html>
<html lang="fr">
<head>
    <meta charset="utf-8" />
    <title>SignalR Demo (Nothing fancy!)</title>
    <link rel="stylesheet" href="https://unpkg.com/picnic">
    <link rel="stylesheet" href="./css/site.css" type="text/css" />
</head>
<body style="zoom: 1;">
    <nav>
        <a href="/" class="brand"><span>HoNoSoFt.SignalR.Demo</span></a>
    </nav>
    <main class="flex center">
        <div>
            <p>Voici une petite démo avec SignalR.</p>
            <p><strong>SignalR Hub:</strong> <i>hubs/toast</i></p>
        </div>

        <div class="third" id="toastContainer">
        </div>
    </main>

    <script src="./js/signalr.js"></script>
    <script>
        var _toastContainer = document.getElementById("toastContainer");

        function addData(sender, importance, message) {
            var span = document.createElement("span");
            span.innerText = "[" + importance + "] " + sender + " : " + message;
            span.className = "button stack " + getClassFromImportance(importance);
            _toastContainer.appendChild(span);
        }

        function getClassFromImportance(importance) {
            switch (importance) {
                case "High": return "error";
                case "Medium": return "warning";
                case "Low": return "success";
            }
        }

        // Doit s'alligner avec ce que vous avez indiqué dans le Startup.cs
        // C'est ici que notre client va se connecter au HUB SignalR backend en WebSocket (ou dégradé)
        const connection = new signalR.HubConnectionBuilder()
            .withUrl("/hubs/toast")
            .configureLogging(signalR.LogLevel.Information)
            .build();

        connection.on("SendNotification", function (sender, toastImportance, message) {
            addData(sender, toastImportance, message);
        });

        connection.start().then(function () {
            var p = document.createElement("p");
            p.innerText = "Connection au hub Toast réussi!";
            p.className = "connected";
            _toastContainer.appendChild(p);
        });
    </script>
</body>
</html>

Ajout d’un minimum de style

Comme vous l’avez vu dans la section “head”, il nous manque 1 feuille de style.

La feuille de style est wwwroot/css/site.css:

#toastContainer {
    height: 300px;
    border: 1px #ddd solid;
    overflow: scroll;
}

main {
    padding: 80px 3em 0 3em;
}

.connected {
    color: forestgreen;
    font-weight: bold;
}

Du coup, avant de continuer, vos fichiers devraient ressembler à ce qui suit:

Démarrer votre projet et regardez si au moins votre socket se connecte. (F12 et regarder la console sous chrome)

Ajout du BackgroundService pour générer des événements

Comme vous l’avez vu dans un article précédent, l’implémentation d’une classe qui hérite de BackgroundService implémente automatiquement IHostedService. Du coup, lorsque c’est enregistrer dans le Startup.cs, ça démarre automatiquement au lancement de l’application.

Créons un fichier BackgroundServices/ToastEventGenerator.cs:

using HoNoSoFt.SignalR.Demo.Hubs;
using HoNoSoFt.SignalR.Demo.Models;
using Microsoft.AspNetCore.SignalR;
using Microsoft.Extensions.Hosting;
using System;
using System.Threading;
using System.Threading.Tasks;

namespace HoNoSoFt.SignalR.Demo.BackgroundServices
{
    public class ToastEventGenerator : BackgroundService
    {
        private readonly IHubContext<ToastHub> _toastHubContext;

        /// <summary>
        /// The random, mais vous devriez utiliser la nouvelle classe secure random.
        /// </summary>
        private static Random _random = new Random((int)DateTime.Now.Ticks & 0x0000FFFF);

        public ToastEventGenerator(IHubContext<ToastHub> toastHubContext)
        {
            _toastHubContext = toastHubContext;
        }

        protected override async Task ExecuteAsync(CancellationToken stoppingToken)
        {
            while (!stoppingToken.IsCancellationRequested)
            {
                await BroadcastToastNotificationAsync((ToastImportance)_random.Next(0, 3), "Toast random")
                    .ConfigureAwait(false);

                // Devrait être random
                await Task.Delay(TimeSpan.FromSeconds(2)).ConfigureAwait(false);
            }
        }

        private Task BroadcastToastNotificationAsync(ToastImportance toastImportance, string message)
        {
            return _toastHubContext
                .Clients
                .All
                .SendCoreAsync("SendNotification", new[] { "System", toastImportance.ToString(), message });
        }
    }
}

À ce moment-ci, vous avez sans doute dans votre Startup.cs une ligne en commentaire. Celle du BackgroundService, veuillez la réactiver.

// ...
services.AddSingleton<IHostedService, ToastEventGenerator>();
// ...

Structure des fichiers (Final):

Redémarrer votre service et regardez le résultat 😀

Conclusion

Vous voyez, ce n’est pas si compliqué et ce qui est vraiment intéressant, c’est que ça fonctionne partout! Vous pouvez même utiliser un système tel que SignalR afin de faire des notifications en tant que Broker en se connectant sur un Bus de service tel que RabbitMQ, Azure Storage Queue, etc.

Code complet de la démo disponible sur GitHub.

 

]]>
https://blog.honosoft.com/2018/07/11/net-core-2-1-plus-signalr-plus-javascript-equal-love/feed/ 0